Associative Learning and Conditioning Theory: Human and Non-Human Applications

ASSOCIATIVE LEARNING AND CONDITIONING THEORY 00-Schachtman-Prelims.indd i 2/16/2011 5:45:16 PM This page intentiona...

Author: Todd R Schachtman PhD | Steve S Reilly PhD

203 downloads 1856 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

ASSOCIATIVE LEARNING AND CONDITIONING THEORY

00-Schachtman-Prelims.indd i

2/16/2011 5:45:16 PM

This page intentionally left blank

Associative Learning and Conditioning Theory Human and Non-Human Applications

Edited by

Todd R. Schachtman Steve Reilly

1

00-Schachtman-Prelims.indd iii

2/16/2011 5:45:16 PM

1

Oxford University Press, Inc., publishes works that further Oxford University’s objective of excellence in research, scholarship, and education. Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With offices in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam

Copyright © 2011 Oxford University Press Published by Oxford University Press, Inc. 198 Madison Avenue, New York, New York 10016 www.oup.com Oxford is a registered trademark of Oxford University Press All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of Oxford University Press. Library of Congress Cataloging-in-Publication Data Associative learning and conditioning theory: human and non-human applications/edited by Todd Schachtman and Steve Reilly. p. cm. Includes bibliographical references and index. ISBN 978-0-19-973596-9 (hardback) 1. Learning, Psychology of. 2. Classical conditioning. 3. Human behavior. 4. Paired-association learning. I. Schachtman, Todd R. II. Reilly, Steve. BF318.A85 2011 153.1’526—dc22 2010037600

1

3

5

7

9

8

6

4

2

Printed in the United States of America on acid-free paper

00-Schachtman-Prelims.indd iv

2/16/2011 5:45:16 PM

For Ariel and Becky (T.S.) For Elaine (S.R.)

00-Schachtman-Prelims.indd v

2/16/2011 5:45:16 PM


CONTENTS

Contributors

xi

PART I: OVERVIEW

1.

Things You Always Wanted to Know About Conditioning But Were Afraid to Ask Todd R. Schachtman and Steve Reilly

3

PART II: APPLICATIONS TO CLINICAL PATHOLOGY

2.

3.

4.

Fear Extinction and Emotional Processing Theory: A Critical Review Seth J. Gillihan and Edna B. Foa Fear Conditioning and Attention to Threat: An Integrative Approach to Understanding the Etiology of Anxiety Disorders Katherine Oehlberg and Susan Mineka Behavioral Techniques to Reduce Relapse After Exposure Therapy: Applications of Studies of Experimental Extinction Mario A. Laborda, Bridget L. McConnell, and Ralph R. Miller

27

44

79 104

5.

Learning and Anxiety: A Cognitive Perspective Peter F. Lovibond

6.

Trauma, Learned Helplessness, Its Neuroscience, and Implications for Posttraumatic Stress Disorder Vincent M. LoLordo and J. Bruce Overmier

121

Aberrant Attentional Processes in Schizophrenia as Reflected in Latent Inhibition Data Robert E. Lubow

152

7.

vii

00-Schachtman-Prelims.indd vii

2/16/2011 5:45:16 PM

viii

CONTENTS

8. Discrimination Learning Process in Autism Spectrum Disorders: A Comparator Theory Phil Reed

168

PART III: APPLICATIONS TO HEALTH AND ADDICTION

9. Conditioned Immunomodulation

191

Jennifer L. Szczytkowski and Donald T. Lysle

10. Learning, Expectancy, and Behavioral Control: Implications for Drug Abuse Muriel Vogel-Sprott and Mark T. Fillmore

213

11. Applications of Contemporary Learning Theory in the Treatment of Drug Abuse Danielle E. McCarthy, Timothy B. Baker, Haruka M. Minami, and Vivian M. Yeh

235

12. Internal Stimuli Generated by Abused Substances: Role of Pavlovian Conditioning and Its Implications for Drug Addiction Rick A. Bevins and Jennifer E. Murray

270

13. Learning to Eat: The Influence of Food Cues on What, When, and How Much We Eat Janet Polivy, C. Peter Herman, and Laura Girz

14. Conditional Analgesia, Negative Feedback, and Error Correction

290 305

Moriel Zelikowsky and Michael S. Fanselow

15. Incentives in the Modification and Cessation of Cigarette Smoking

321

Edwin B. Fisher, Leonard Green, Amanda L. Calvert, and Russell E. Glasgow

PART IV: APPLICATIONS TO COGNITION, SOCIAL INTERACTION, AND MOTIVATION

16. Social Learning and Connectionism

345

Frank Van Overwalle

17. Application of Associative Learning Paradigms to Clinically Relevant Individual Differences in Cognitive Processing Teresa A. Treat, John K. Kruschke, Richard J. Viken, and Richard M. McFall

376

18. Evaluative Conditioning: A Review of Functional Knowledge and Mental Process Theories Jan De Houwer

399

19. Instrumental and Pavlovian Conditioning Analogs of Familiar Social Processes Robert Ervin Cramer and Robert Frank Weiss

417

20. The Impact of Social Cognition on Emotional Learning: A Cognitive Neuroscience Perspective Andreas Olsson

00-Schachtman-Prelims.indd viii

454

2/16/2011 5:45:16 PM

CONTENTS

ix

21. Effects of Conditioning in Advertising

481

Todd R. Schachtman, Jennifer Walker, and Stephanie Fowler

22. Applications of Pavlovian Conditioning to Sexual Behavior and Reproduction Michael Domjan and Chana K. Akins

507

23. Hot and Bothered: Classical Conditioning of Sexual Incentives in Humans Heather Hoffmann Index

00-Schachtman-Prelims.indd ix

532 551

2/16/2011 5:45:16 PM


CONTRIBUTORS

Chana K. Akins, Ph.D. Department of Psychology University of Kentucky Lexington, KY

Michael S. Fanselow, Ph.D. Department of Psychology University of California, Los Angeles Los Angeles, CA

Timothy B. Baker, Ph.D. Center for Tobacco Research & Intervention, and Department of Medicine University of Wisconsin School of Medicine and Public Health Madison, WI

Mark T. Fillmore, Ph.D. Department of Psychology University of Kentucky Lexington, KY

Rick A. Bevins, Ph.D. Department of Psychology University of Nebraska-Lincoln Lincoln, NE Amanda L. Calvert, A.M. Department of Psychology Washington University St. Louis, MO Robert Ervin Cramer, Ph.D. Department of Psychology California State University, San Bernardino San Bernardino, CA Jan De Houwer, Ph.D. Department of Psychology Ghent University Ghent, Belgium Michael Domjan, Ph.D. Department of Psychology The University of Texas at Austin Austin, TX

Edwin B. Fisher, Ph.D. Global Director, Peers for Progress American Academy of Family Physicians Foundation Professor, Health Behavior & Health Education Gillings School of Global Public Health University of North Carolina at Chapel Hill Chapel Hill, NC Edna B. Foa, Ph.D. Center for the Treatment and Study of Anxiety University of Pennsylvania Philadelphia, PA Stephanie Fowler, M.A. Department of Psychological Sciences University of Missouri Columbia, MO Seth J. Gillihan, Ph.D. Center for the Treatment and Study of Anxiety University of Pennsylvania Philadelphia, PA xi

00-Schachtman-Prelims.indd xi

2/16/2011 5:45:16 PM

xii

CONTRIBUTORS

Laura Girz, M.A. Department of Psychology University of Toronto Toronto, ON, Canada

Robert E. Lubow, Ph.D. Department of Psychology Tel Aviv University Tel Aviv, Israel

Russell E. Glasgow, Ph.D. Institute for Health Research Kaiser Permanente Colorado Denver, CO

Donald T. Lysle, Ph.D. Department of Psychology University of North Carolina at Chapel Hill Chapel Hill, NC

Leonard Green, Ph.D. Department of Psychology Washington University St. Louis, MO C. Peter Herman, Ph.D. Department of Psychology University of Toronto Toronto, ON, Canada Heather Hoffmann, Ph.D. Department of Psychology Knox College Galesburg, IL John K. Kruschke, Ph.D. Department of Psychological and Brain Sciences Indiana University Bloomington, IN Mario A. Laborda, M.S. Department of Psychology State University of New York— Binghamton Binghamton, NY Universidad de Chile Santiago, RM, Chile Vincent M. LoLordo, Ph.D. Department of Psychology Dalhousie University Halifax, Nova Scotia, Canada Peter F. Lovibond, Ph.D. School of Psychology University of New South Wales Sydney, Australia

00-Schachtman-Prelims.indd xii

Danielle E. McCarthy, Ph.D. Department of Psychology, Rutgers The State University of New Jersey New Brunswick, NJ Bridget L. McConnell, M.S. Department of Psychology State University of New York— Binghamton Binghamton, NY Richard M. McFall, Ph.D. Department of Psychological and Brain Sciences Indiana University Bloomington, IN Ralph R. Miller, Ph.D. Department of Psychology State University of New York— Binghamton Binghamton, NY Haruka M. Minami, M.S. Department of Psychology Rutgers, The State University of New Jersey New Brunswick, NJ Susan Mineka, Ph.D. Department of Psychology Northwestern University Evanston, IL Jennifer E. Murray, Ph.D. Department of Experimental Psychology University of Cambridge Cambridge, UK

2/16/2011 5:45:16 PM

CONTRIBUTORS

Katherine Oehlberg, M.S. Department of Psychology Northwestern University Evanston, IL Andreas Olsson, Ph.D. Karolinska Institutet Department of Clinical Neuroscience Psychology Unit Stockholm, Sweden

xiii

Jennifer L. Szczytkowski, Ph.D. Department of Psychology University of North Carolina at Chapel Hill Chapel Hill, NC Teresa A. Treat, Ph.D. Psychology Department University of Iowa Iowa City, IA

J. Bruce Overmier, Ph.D. Department of Psychology University of Minnesota Minneapolis, MN

Richard J. Viken, Ph.D. Department of Psychological and Brain Sciences Indiana University Bloomington, IN

Frank Van Overwalle, Ph.D. Department of Psychology Vrije Universiteit Brussel Brussels, Belgium

Muriel Vogel-Sprott, Ph.D. Department of Psychology University of Waterloo, Waterloo, ON, Canada

Janet Polivy, Ph.D. Department of Psychology University of Toronto Toronto, ON, Canada

Jennifer Walker, M.A. Department of Psychological Sciences University of Missouri Columbia, MO

Phil Reed, D.Phil. Department of Psychology Swansea University Swansea, Wales, UK

Robert Frank Weiss, Ph.D. Department of Psychology University of Oklahoma Norman, OK

Steve Reilly, D.Phil. Department of Psychology University of Illinois at Chicago Chicago, IL

Vivian M. Yeh, M.S. Department of Psychology Rutgers, The State University of New Jersey New Brunswick, NJ

Todd R. Schachtman, Ph.D. Department of Psychological Sciences University of Missouri Columbia, MO

Moriel Zelikowsky, M.A. Department of Psychology University of California, Los Angeles Los Angeles, CA

00-Schachtman-Prelims.indd xiii

2/16/2011 5:45:16 PM


PART I

Overview

01-Schachtman-01.indd 1

2/15/2011 7:41:40 PM


CHAPTER 1 Things You Always Wanted to Know About Conditioning But Were Afraid to Ask Todd R. Schachtman and Steve Reilly

This chapter describes significant developments in the field of conditioning and associative learning over the last 40 or so years (e.g., the emergence of cognitive views, potential associative structures, and the role of contextual factors) to clarify some issues that may confuse psychologists and other researchers or practitioners. Along the way, we shall illustrate some important and, in most cases, ongoing findings that explore the processes underlying associative learning and conditioning. Our intention is to help readers better appreciate the more detailed, application-focused chapters that follow in this volume.

INTRODUCTION This chapter has two primary aims, each of which will help expose those readers unschooled in conditioning and associative learning to this still-flourishing and significant area of research. First, we will describe some of the most important and exciting developments over the last 40 years or so, including the cognitive orientation of the field, the possible associative structures underlying conditioning, and the role of contextual factors in conditioning. Second, we will clarify some specific issues that often plague psychologists and students of psychology. These issues include the following: (1) distinctions and similarities between classical (or Pavlovian) and instrumental (or operant) conditioning (e.g., when an animal [human or non-human] exhibits fear—is it an instrumental response or a classically conditioned response?); (2) the types of associations involved in the various forms of learning, for example, S-O (stimulus-outcome), R-O (response-outcome), and S-R (stimulusresponse) associations; (3) the similarities and differences among procedures such as avoidance,

punishment, and omission training; (4) differences between phenomena such as extinction, habituation, dishabituation, spontaneous recovery, and pseudoconditioning in classical conditioning. As might be expected due to the dynamic and evolving nature of the field, these two purposes are intertwined with one another in the present chapter. A few disclaimers are warranted. We will report on those issues that strike us as most interesting; and this may be, in fact, because we are more familiar with these issues. Moreover, our own theoretical predilections will be exposed. For instance, much of our chapter focuses on classical conditioning. It should be clear that the discussion in this chapter is not intended to exhaust all of the topics that are of current interest in the associative conditioning literature. The present chapter, for these and others reasons, should not be taken as a substitute for a textbook on learning and conditioning because such a text would include many more issues (e.g., Bouton, 2007; Dickinson, 1980; Domjan, 2009; Frieman, 2002; Gluck, Mercado, & Myers, 2007; Mackintosh, 1974; Pearce, 2008); and each of the

3


2/15/2011 7:41:40 PM

4

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

topics covered here requires much elaboration to fully do them justice. The present chapter aims to merely whet the appetite for the more detailed and often application-focused expositions found in the later chapters of this volume. Comments About Recent Developments in the Field of Conditioning

For many researchers, the field has become increasingly more cognitive in its orientation and conceptual development (e.g., Bolles, 1972; Dickinson, 1980; Klein & Mowrer, 1989a, 1989b; Mackintosh, 1994; Mowrer & Klein, 2001; Rescorla, 1988; Wagner, 1976). There are several findings and theoretical advances that have shaped these changes. First, the field has been largely dominated by the concepts of expectancy and prediction. The importance of expectancy was really thrust into the limelight with the discovery of the phenomenon of blocking (Kamin, 1968, 1969), and publication of the Rescorla-Wagner model (Rescorla & Wagner, 1972; Wagner & Rescorla, 1972) nearly 40 years ago. To say that after CS-US (conditioned stimulus-unconditioned stimulus) pairings, the CS allows an organism to expect the US1 or to be able to predict that the US will occur seems fairly obvious to many of us today; but, for many decades, conditioning was not often discussed within the framework of information processing and its closely related concepts of prediction and expectancy. Before discussing specific issues, we will point out two developments in the last 30 years that have greatly contributed to the changes in conditioning that we mentioned earlier. First, Rescorla’s work in the 1970s (to be discussed later) often used manipulations administered after an initial conditioning phase but prior to a final test phase. This work by Rescorla was presumably inspired in part by Rozeboom (1958), who posited that treatments administered between conditioning and testing served to help determine the contents of the learned information (for examples of such findings, see sections on “The Role of Representations in Conditioning” and “Postconditioning Representations of the Unconditioned Stimulus”). A second issue that


has been valuable is the distinction between what an organism knows and what the organism reveals that it knows by its performance. In other words, an organism does not always fully express what it actually knows. This learningperformance problem is an old one. Seventy or so years ago, Tolman’s work on latent learning showed that rats can be exposed to a maze without food present, and that they will explore the maze despite the absence of any obvious motivation to do so (Tolman & Honzik, 1930). During exploration, the rats do not reveal much evidence that they have learned about the floor plan of the maze. However, once these rats start true maze training in which food is presented at the end of the maze and the rats are now hungry, they show considerable “positive transfer” (beneficial effects of earlier experience on a current task) and reveal that they learned a lot during the earlier, nonrewarded exposure to the maze. Another example of this kind of (i.e., unexpressed) learning is that which occurs during the initial phase of a sensory preconditioning experiment as will be discussed later (see section on “Compound Conditioning”). Relatedly, there is a distinction between poor conditioned responding that occurs due to an association never being formed, and poor conditioned responding that occurs because the association is formed and intact but is poorly retrieved or expressed at the time of test. Tulving (e.g., Tulving & Pearlstone, 1966) noted this same distinction while discussing human memory research when he distinguished between an unavailable memory (one that is not present or no longer present in memory) and an inaccessible memory (one that is located in memory but unable to be retrieved at a given point in time). How might one experimentally distinguish between an acquired, intact but unexpressed association and one that was never acquired? Several decades ago, researchers such as Ralph Miller, David Riccio, and Norman Spear administered conditioning trials to rats in which the animals had an opportunity to learn about, for example, a CS which predicted a US; but the animals did not show any evidence of such learning when tested (i.e., did not show evidence of the acquired CS-US association as demonstrated by

2/15/2011 7:41:40 PM

THINGS TO KNOW ABOUT CONDITIONING

a lack of a conditioned response [CR] to the CS). Then, in a second phase, the animals were given a treatment that presumably enhanced retrieval of the already existing association. This treatment facilitated retrieval, but it did not allow for any additional learning per se. When the rats were then tested, a CR did occur, showing that the retrieval facilitation procedure (sometimes called a “reminder treatment”) allowed a previously unexpressed association to then become evident in performance (e.g., Miller, Kasprow, & Schachtman, 1986; Spear, 1978; Spear & Riccio, 1994). When animals show no evidence of learning, and then some treatment—which of itself does not allow new learning to occur— causes this learning to be expressed, one can say that the association was latent or unexpressed prior to this reminder treatment. This finding illustrates several things. First, it shows how postconditioning manipulations can be important to reveal the content of learning as Rescorla and Rozeboom posited. By “content of learning” we mean the type of association that was acquired or whether an association was acquired at all. Second, it shows that conditioning research now uses very cognitive expressions such as the retrieval of information when discussing conditioned responding. Retrieval has been discussed in the work of many distinguished learning theorists, including Bouton, Hall, Hearst, Miller, Riccio, Spear, and others. Content and Structure of Learning

An instructor at a large Midwestern university was once giving a presentation on what it was like to teach an Introductory Psychology course to hundreds of students packed into a large lecture hall. The speaker made two main points. The first one had to do with energy—using an animated voice, in-class demonstrations, and entertainment. The second point contained more shock value: He claimed, “You have to be able to lie.” He said that freshmen and sophomores are not ready to hear all the limitations of the theories that are presented, such as those of Piaget’s writings or the strengths and limitations of the various views of learned helplessness theory or many other ideas that have required


5

complicated revisions over the empirical tests of time. The lesson: When starting out, one needs to keep things simple. Similarly, when lecturing on conditioning or discussing it in a textbook, one feels a need to keep things simple. However, most researchers recognize that in any given situation where conditioning is possible, there is the potential for both classical and instrumental conditioning to occur on the same conditioning trial. One’s task might be to give classical conditioning trials to a dog in which a tone (the CS) is followed by food (the US). This, of course, is the same type of procedure used by Pavlov when he discovered classical conditioning (Pavlov, 1927). The food, professors tell their students, promotes classical conditioning to occur for the tone CS; but the food is also capable of instrumentally reinforcing a response that may happen to occur around the time that the food is delivered. Thus, when the dog makes a salivation response, it is not entirely clear whether the salivation is a classically conditioned response occurring to the tone or whether the dog happened to be salivating at the time the food was first delivered (and in the presence of the tone) such that instrumental conditioning occurred. That said, because it might be unlikely that salivation would happen to occur at the same time that the food was delivered (assuming the dog can’t smell the food coming), any salivation arising from the first conditioning trial was probably not due to operant conditioning. However, if the dog makes a salivation response to the tone on the second classical conditioning trial, then the food delivery could reward that response as instrumental learning as well as further the classical conditioning to the tone. In this way, it is easy to see that a conditioning episode does not always render itself to a pure classical conditioning analysis or a pure instrumental analysis. Indeed, the overlap between the two types of conditioning may be even more pronounced. Some associative learning theorists (e.g., Mackintosh, 1983) have noted a “symmetry” of the potential associations underlying the two types of conditioning (i.e., the two associations having some similarity in their structure). Classical conditioning is viewed as involving a

2/15/2011 7:41:40 PM

6


stimulus-stimulus (S-S) association (i.e., an association between the CS and the US), such that pairings of a tone and food result in a tone-food association. Instrumental conditioning in which a response (e.g., a rat pressing a lever) is followed by food can be said to result in a responsestimulus (R-S) association in which the lever press response is associated with a food stimulus. In the literature, this food stimulus in both types of conditioning is sometimes called an outcome. Hence, a response-outcome (R-O) association is formed in which the outcome is a food stimulus; from this perspective, classical conditioning may be termed stimulus-outcome or S-O learning (see Table 1.1, which lists some of the more common nomenclature, terminology, and potential underlying processes thought to be involved in these two forms of associative learning). The symmetry view of conditioned associations acknowledges that classical conditioning using the example provided results in a tone-food association and instrumental conditioning results in a response-food association; and so the mechanisms underlying these two associations may be similar, with the exception that one type of conditioning involves a CS that is associated with food and the other involves a response that is

associated with food. According to this view, these associations may interact. This interaction could result in the two types of associations being in competition with each other on a given trial and producing effects such as “blocking,” which will be discussed later (also see Williams, 1999). We mentioned that instrumental conditioning can occur on a trial that uses a classical conditioning procedure and classical conditioning can occur on a trial that uses an instrumental conditioning procedure. If instrumentally and classically conditioned associations can compete, then it means that learning an association between a cue and the outcome will reduce the potential for R-O learning and vice versa. There is another parallel that exists between instrumental and classical conditioning. In instrumental conditioning, a stimulus (e.g., a tone) is often present at a time when a schedule of reinforcement is in place. In the case of positive reinforcement, the subject can learn that responding leads to reward when the tone is present. If the tone is absent, then responses have no consequences. The tone, in this example, is referred to as a “discriminative stimulus” in that it signals when the schedule of reinforcement is operating. A stimulus which signals that the response will

Table 1.1 Some Features of the Two Types of Associative Learning Classical Conditioning

Instrumental Learning Early in Training

After Extensive Training

Also called:

Pavlovian conditioning or Stimulus learning or Type I learning or Respondent conditioning

Operant conditioning or Response learning or Type II learning

Habit learning

Associations are commonly known as:

CS-US or S-S or S-S∗ or S-O

R-O or R-S∗ or A-O

S-R

Potential underlying processes:

Ability to predict future events

Modification of voluntary behavior

Elicited by antecedent stimulus

A, action; CS, conditioned stimulus; O, outcome; R, response; S, stimulus; S∗, biologically significant stimulus; US, unconditioned stimulus.


2/15/2011 7:41:40 PM


lead to a certain outcome is sometimes referred to as an SD or S+. Skinner (e.g., 1938), who coined the expression “SD,” also stated that the stimulus in this case “sets the occasion” for the response to result in the reward. When a stimulus signals that the response will not lead to the outcome (extinction), then such a stimulus is referred to as an S∆ (pronounced: S delta) or “S–.” These definitions illustrate the conditional relationships that events can have: A response can lead to a reward in the presence of the SD, but a response will not lead to a reward in the absence of the SD or, alternatively, in the presence of an S∆. Parallel phenomena exist in classical conditioning. A great deal of research in the past 25–30 years has focused on the conditional relationship among stimuli in classical conditioning. These phenomena are called “occasion setting” stimuli (Holland, 1992; Schmajuk & Holland, 1998). If a tone leads to food when it is accompanied by a light but the tone does not lead to food when the light is absent, then the light is said to “set the occasion” for when the tone will be followed by food. The light is referred to as a positive “occasion setter.” To reiterate for clarity, if lighttone-food trials occur along with tone–no-food trials, then the light becomes a “positive occasion setter” for the tone–food relationship. On the other hand, if light-tone–no-food trials occurred with tone-food trials, then the light could be said to be a “negative occasion setter.” The associative processes underlying classical and instrumental conditioning may extend into other conditioning phenomena. A pairing of the CS and US results in a CS-US association, and then extinction (a postacquisition phenomenon) involves the presentation of the CS without the US. Some conditioning theorists (and this issue will be briefly discussed again later; see section on “Postconditioning Representations of the Unconditioned Stimulus”) have posited that such CS-only presentations after conditioning result in the learning of a CS-noUS association (an association reflecting that the CS is no longer being paired with the US) or, using other symbols, an S-noO associations (e.g., Bouton, 1993; Konorski, 1967; but see Rescorla, 2007). When instrumental conditioning occurs in which


7

a response produces an outcome, an R-O association is formed; and extinction involves the occurrence of response without the outcome. One can say that a response–no-outcome (R-noO) association is formed during extinction in instrumental conditioning. Hence, parallel associative processes may occur for the two types of associations (classical and instrumental). Many other important conditioning phenomena (particularly those that stem from work using classical conditioning) such as blocking, overshadowing, potentiation, spontaneous recovery, and outcome preexposure effects may produce comparable empirical effects in instrumental and classical conditioning as well as similar underlying associative mechanisms. The extent that such parallels hold true will be determined by research findings, but mention of these issues at this time helps one appreciate the potential similarities of the different types of associative learning. Rescorla and Solomon (1967) showed that one can separately train a classically conditioned response and an instrumental response for a group of subjects and then combine the two training events together to observe what happens with this combination of the two types of training. Their ideas have been termed the “twoprocess theory” of conditioning. A subject (e.g., a dog) can learn that a tone leads to food, and thus the occurrence of the tone causes the dog to expect food. The same dog can also be trained to press a lever to produce food, and this dog will press the lever frequently because it expects that the lever press will produce food. Subsequently, when placed in front of the lever and presented with the tone, the dog will have two expectancies: one from the tone (classical conditioning) and one from the lever press (instrumental conditioning). The summation of expectancies can produce even greater lever pressing because the dog has an extremely high expectancy of food. Similar interactions can be examined for all combinations of predictors of the US and even for events that predict no US (e.g., a CS that signals the absence of the US) combined with a response that produces outcome avoidance. Two-factor theory (e.g., Mowrer, 1951) is also concerned with how classical and instrumental conditioning can interact. Mowrer pointed out

2/15/2011 7:41:41 PM

8


many years ago that if a subject must make a response to avoid an aversive event (called “active avoidance” or “negative reinforcement” as we will discuss in the next section), then at least two things can be learned on such a trial. First, the subject receives classical conditioning pairings of the environmental cues (i.e., the apparatus within which the experiment is conducted or, perhaps, a small light within the apparatus that is illuminated at the beginning of each trial signaling that an aversive event will occur if the target response does not occur) with the aversive outcome. That is, a CS-O association is potentially formed. These cues (the CS) become capable of producing fear in the subject. Thus, a fear CR occurs to the cues. Second, the subject learns to make the instrumentally conditioned response (such as fleeing from the CS or the place that has been paired with shock) to prevent the aversive event from occurring. Therefore, subjects run because they have fear of the CS, which is due to classical conditioning; and they are making the response that will remove them from the aversive cues—the avoidance response (instrumental conditioning). Two-factor theory states that both classical and instrumental conditioning can occur during avoidance training. This example also reveals again how classical conditioning and instrumental conditioning can be intertwined—a CR to a cue (classical conditioning) can resemble and avoidance response (instrumental conditioning). Since classical and instrumental conditioning may overlap in terms of their associative mechanism (S-O and R-O associations, respectively), there has been interest in pitting the two types of conditioning against each other to document the roles of the two processes. The omission training procedure, which involves overlaying an instrumental contingency upon a classical contingency, has been useful in this regard (e.g., Sheffield, 1965). In omission training, a cue (CS) is presented and the subject receives a US (e.g., food) only if it does not respond to the CS. To repeat, if the subject responds to the CS, the US is omitted. Thus, a CS-US pairing occurs only when the subject makes no response to the CS. As an instrumental procedure, however, the subject is rewarded for not responding. Again, if


the subject does not respond to the cue, then the food is delivered and, consequently, a CS-US pairing is experienced. Thus, performance can be guided by both classical and instrumental conditioning in the omission procedure. As a concrete example, we can consider one of the original omission experiments done with pigeons in which the CS was a colored keylight that the birds can peck (Williams & Williams, 1969; also see Williams, 1981). When pigeons receive presentations of a keylight followed by food, they quickly learn through classical conditioning to respond by pecking that keylight (a classical conditioning phenomenon termed autoshaping or sign-tracking in which the CR to a light is a peck response; for reviews, see Hearst & Jenkins, 1974; Locurto, Terrace, & Gibbon, 1981). However, the operant (omission) contingency gives them food for not responding; and when they don’t respond they get a keylight-food pairing that makes them respond on the next trial. Both classical and operant conditioning will exert an effect on behavior in such situations: Responding is not as high as it might be because the omission contingency keeps responding low, but the CS-US pairings cause the animal to sometimes respond and, therefore, lose food presentations when they respond to the CS (the CS occurs without the US). Sixty or seventy years ago discussions of conditioning focused on S-R (stimulus-response) associations. Many readers of this chapter may have heard this expression and wondered about its contemporary status. Indeed, these are the associations that were purported to underlie conditioning when Thorndike began investigating associative learning in non-human animals at the turn of the previous century (e.g., Thorndike, 1898, 1911) and they were alleged to underlie both classical and instrumental conditioning throughout, at least, the 1950s. S-R associations involve an association between a (usually) environmental stimulus (such as a light, tone, or contextual stimulus) and the response. We say “usually” because one could have an “internal” stimulus (such as hunger pangs, moods and emotions, or states induced by the ingestion of a drug) as the stimulus of the S-R association, but we will keep things simple for this discussion.

2/15/2011 7:41:41 PM


It is important to be clear as to what this association involves and how it differs from the contemporary view of associations (S-O and R-O) described earlier. An outcome is not part of, or encoded into, the S-R association. According to this older S-R view of conditioning, when an outcome (e.g., food) follows the occurrence of response (e.g., lever press) in the presence of a stimulus (e.g., the apparatus within which the experiment is being conducted), an association is formed between the stimulus (apparatus) and the response (lever press). That is, an S-R association is formed. The outcome (food) serves to facilitate the formation of the S-R association. According to this S-R association view of conditioning, the representation of the outcome in memory is not a part of the acquired association. Once the outcome has promoted the formation of the S-R association, its function is accomplished and it no longer has a role in further processing or performance regarding the S-R association. Work by Anthony Dickinson and Robert Rescorla and their colleagues and others challenged this view experimentally (e.g., Colwill & Rescorla, 1986; Dickinson, 1985; Dickinson & Balleine, 1994; Rescorla, 1991). If the outcome is no longer important after the instrumental association is acquired, then changing the value of the outcome for the subject after conditioning and prior to testing for the learned response should have no influence on performance. Work in Dickinson’s and Rescorla’s laboratories showed that changing the value of the outcome between conditioning and testing produced a marked change in the performance of the learned response (e.g., Adams & Dickinson, 1981; Colwill & Rescorla, 1985). This makes some intuitive sense. If you train a hungry rat to press a lever to obtain food pellets, it will learn to respond at a high rate to obtain the desired food. However, if you now devalue the food pellets (e.g., by conditioning a taste aversion to them) such that the rat dislikes them, it is not surprising that the rat will not press the lever much to obtain a disliked food. Yet earlier S-R theories did not predict such a change in behavior. A similar situation exists for a dog that might salivate to a signal for food—changing how the organism values


9

the food after such training will attenuate the salivation response. The outcome seems to be part of the association. The subject has acquired an R-O association because the response activated the food representation in memory. Despite the strong focus on the S-O and R-O associations in this chapter, Dickinson and Rescorla have shown that S-R associations can be formed during conditioning in certain situations. Research in the last 30 years or so has shown that instrumental learning involves R-O associations early in training and can be said to be “goal directed” (the organism makes a response to obtain a desired goal or outcome). Moreover, this voluntary (i.e., intentional), goal-directed behavior is occurring in a specific context or in the presence of a specific stimulus (S). In certain circumstances, this stimulus, because it is invariably and repeatedly paired with the target response, may become associated with the response and an S-R association is also eventually acquired. Thus, presentation of the S triggers an activation of the response representation and the subject then responds habitually (i.e., automatically and usually without any conscious active representations), independent of the status (valued or devalued) of the outcome (Dickinson, Balleine, Watt, Gonzalez, & Boakes, 1995; but see Holland, 2004). By this analysis, habit (S-R) formation develops after, and overlays or masks, the initial goal-directed (R-O) behavior. Both associations involve the same instrumental response. As noted earlier, the S-R association, unlike the R-O association, is impervious to outcome devaluation (so clinical treatments aimed at changing behavior via some form of outcome devaluation may have limited utility if that behavior has become habitual; e.g., smoking or drinking). There is, moreover, evidence that each type of instrumental association (R-O and S-R) is subserved by an anatomically distinct brain system (Balleine, 2005; Balleine, Liljeholm, & Ostlund, 2009; Delamater, 2004; Yin & Knowlton, 2006). If these systems can be dissociated with drugs, there may be some hope that, with pharmacological intervention, extreme habits (S-R associations) can be alleviated.

2/15/2011 7:41:41 PM

10


Instrumental Contingencies

We mentioned the terms avoidance and negative reinforcement in the preceding section. These and other instrumental contingencies (see Table 1.2 for a description of the four basic procedures and their effects on behavior) can be a source of confusion, especially since textbooks are not always consistent with the nomenclature applied to the various instrumental-conditioning manipulations. As most psychologists know, positive reinforcement occurs when there is a positive relationship (or positive contingency) between the production of a response and the presentation of an appetitive (pleasurable) outcome. When the expression “positive contingency” is used, it should be viewed as comparable to the expression “positive correlation” (and conversely for the expression “negative contingency”). That is, when the subject makes the response, something pleasurable (like food) reliably occurs. This positive contingency causes the response to increase in the future. Punishment involves a positive contingency between the response and an aversive outcome (something painful or unpleasant). When the subject makes this response, an aversive event occurs and, not surprisingly, this causes the response to decrease in frequency.

There is a caveat to mention that readers will likely wish that we did not bring up (this certainly should be spared from an undergraduate lecture on this topic). We make this cumbersome point because it will help when we make related points later. With positive reinforcement in which a lever press produces a pleasurable outcome, one could imagine what it would be like for the subject that has a lot of experience with this contingency to momentarily refrain from making a response and to therefore get no pleasurable outcome. That is, with positive reinforcement: No response leads to no reward. It involves a positive contingency between no response and no reward. One can think of it as an intrinsic, necessary feature of the positive contingency involving the response and reward occurrence. In truth, for any behavioral contingency (a response leads to an outcome), there are a total of four contingencies related to it because nonresponses and no outcomes can be viewed as events: A response has a positive contingency with the outcome, a nonresponse has a positive contingency with no outcome, a response has a negative contingency with no outcome, and a nonresponse has a negative contingency with the outcome. Four contingencies exist for a simple positive reinforcement contingency! But it is best to simply focus on

Table 1.2 Basic Instrumental Contingencies of Reinforcement and Punishment and Their Usual Effect on Behavior Outcome

Positive

Appetitive

Aversive

Reward (or positive reinforcement)

Punishment (or positive punishment)

Increase in behavior

Reduction in behavior

Example: Receive a kiss after complementing partner

Example: Drive too fast and get a speeding ticket

Omission (or negative punishment)

Avoidance (or negative reinforcement)

Reduction in behavior

Increase in behavior

Example: Driving license suspended following conviction for drunk driving

Example: Open umbrella to avoid getting rained on

Contingency

Negative


2/15/2011 7:41:41 PM


the fact that positive reinforcement is a positive contingency between producing the response and the presentation of a pleasurable outcome (and let the positive contingency between nonresponses and nonrewards just rest wherever they might rest and these other contingencies can rest there, too). One can say the same thing about punishment. Nonresponses will lead to the nonoccurrence of the aversive event. This is a positive contingency between nonresponding and no outcome. Nonresponses are preferred for the organism that is experiencing a punishment contingency. But again we will let these nonresponses and nonrewards slip away without further discussion and simply say that punishment involves an association between the occurrence of a response and the presentation of an aversive outcome. Avoidance involves a negative contingency between the response and the subsequent occurrence of an aversive outcome. If the subject makes a response, then the aversive event will not occur. This is referred to as negative reinforcement since the contingency increases the frequency of the behavior (the subject makes the response to prevent the aversive event from occurring). One key note is that whenever one sees the word reinforcement (either negative reinforcement or positive reinforcement), it means that the contingency causes the frequency of the response to increase. Hence, rewarding a behavior is positive reinforcement and avoidance is negative reinforcement, but the former is a positive contingency between the behavior and an appetitive outcome while the latter is a negative contingency between a behavior and an aversive outcome.2 Omission training is a negative contingency between the occurrence of a response and the presentation of an appetitive outcome. Thus, as mentioned before, a subject will receive food if it does not make a certain response. Avoidance and omission training are both negative contingencies, but the former is between a response and an aversive outcome and the latter is between a response and an appetitive outcome. Avoidance increases the rate of responding, whereas omission decreases response rate. It can be easy to get avoidance and punishment confused since they both involve an aversive


11

outcome, and in both cases the subject is motivated to not experience the aversive event. It is good to remember that these two types of instrumental training involve different contingencies: negative and positive. That is, with avoidance the subject must make a certain response to prevent the aversive event from occurring; whereas with punishment the subject must not make a certain response to prevent the aversive event from occurring. Also, similar to the cumbersome digression expounded earlier, one can be tempted to think of an avoidance procedure as follows: A nonresponse produces an aversive event (a positive contingency between a nonresponse and an aversive event). This is true because it is the necessary converse of the following: A response prevents an aversive event—a negative contingency. But we encourage the reader to simply focus on the “occurrence of a specific response” when working through these contingencies. It is also easy to get confused about the difference between escape and avoidance. Escape does not fit nicely into the scheme outlined earlier. In escape, the organism receives the aversive event no matter what, but it can terminate the existing aversive event by making a response. Hence, with a successful escape response, a response and an aversive event occur on the same trial (which suggests that a positive contingency is occurring as in punishment, but this is not true since the response certainly did not produce the aversive event). Indeed, since the response terminates the aversive event, it makes the escape contingency share properties with an avoidance contingency—the response gets rid of the aversive event. Another realm of confusion concerns the procedure termed “passive avoidance” (which is also known as “inhibitory avoidance”), a procedure that has become extensively used in recent neuroscience research. It is a task in which the subject receives an aversive event (e.g., a footshock) if it makes a certain response (e.g., step into one side of a two-compartment shuttle box). It should be clear at this point that this is a punishment procedure—there is a positive contingency between making a response (moving into the side of the chamber) and the occurrence

2/15/2011 7:41:41 PM

12


of an aversive event. Although the names “passive avoidance” and “inhibitory avoidance” have been around for a long time, they are misnomers. They are not avoidance procedures. When scientists decided perhaps 25 years ago to favor the name “inhibitory avoidance” over “passive avoidance,” they did not correct the part of the name—avoidance—that has caused the greatest degree of confusion in the previous 70 years. While discussing instrumental work, we felt it was a good idea to illustrate some important and interesting issues with respect to schedules of reinforcement that might not always be fully appreciated. Nearly all psychologists have been exposed to the “classic” schedules of (intermittent) reinforcement: fixed ratio, fixed interval, variable ratio, and variable interval (Ferster & Skinner, 1957; see Table 1.3 for a description and some characteristics of performance on these four simple schedules of reinforcement). These are all schedules of positive reinforcement for the purposes of our exposition. Punishment contingencies can also be applied using these

schedules, but we will not consider them at this time. However, we want to describe a few of the properties of responding on these schedules, which many psychologists may not be aware of. First, an outcome is delivered on an interval schedule for making the first response on the trial after the interval times out.3 We will assume it is a fixed interval schedule for our discussion. Earlier responses (before the designated time interval elapses) are not necessary and, thus, a waste of energy. Hence, the most efficient form of responding on a fixed interval schedule is one in which only one response occurs per reinforcement. If there is a less-than-perfect positive contingency between responding and the reinforcement, it is because of the nonrewarded response “mistakes” on the part of the subject. The subject, in essence, determines the number of responses per reward based on how early in the trial he or she begins to respond (e.g., to what extent the responses are “early” and therefore not particularly functional). It is interesting that although a high response rate on an interval

Table 1.3 Characteristics of the Four Basic Types of Reinforcement Schedules Basis for Reward

Response

Fixed

Variable

Fixed ratio (FR)

Variable ratio (VR)

Reward delivered after fixed, predictable number of responses

Reward delivered after variable, unpredictable number of responses

High rate of responding and pause after reward delivered

High, steady rate of responding and little or no pause after reward delivered

Example: Working on commission or on “piecework”

Example: Gambling (e.g., slot machine)

Fixed interval (FI)

Variable interval (VI)

Reward delivered for first response after fixed, predictable time period

Reward delivered for first response after variable, unpredictable time period

Increasing rate of response and pause after reward delivered

Moderate, steady rate of responding and little or no pause after reward delivered

Example: Study patterns of students when examinations scheduled at regular intervals

Example: Trying to call someone on the phone when line is busy

Criterion

Time


2/15/2011 7:41:41 PM


schedule can be interpreted as indicative of a strong R-O or S-R association, it is highly inefficient. On ratio schedules, every response “counts for something” and the subjects are free to respond at their own rate: quickly or slowly. From a “responses per outcome perspective,” all responses are valuable, all responding is equally efficient, and all subjects end up with a comparable response/reinforcer ratio. Time is irrelevant in terms of the contingency. Thus, from a time perspective, an organism can be inefficient on a ratio schedule but all responses count; but on a fixed interval schedule an organism can be inefficient from a response-wasting or energy-wasting standpoint, but time is only wasted if the subject is slow to respond after the interval times out. Burgeoning interest in the etiology, treatment, and neural underpinnings of addiction (and related issues) has rekindled interest in the motivation value of drugs and other highly desired stimuli. Although the schedules of reinforcement discussed earlier have been used in this enterprise, comparisons of the relative value of different outcomes may sometimes be problematic. As noted previously, rates of responding do not always provide an accurate index of the value of an outcome, particularly for stimuli such as psychoactive drugs, where ceiling effects in response rate may obscure detection of the true value of a reward. In this context, the progressive ratio schedule has proven to be a highly successful method to use (e.g., Hodos, 1961). These schedules differ from those just discussed in that progressive ratio schedules can have both a response and a time requirement. Although there are many variations, the essence of the progressive ratio schedule is this: As trials progress, more responses are required to obtain each successive outcome until the subject eventually fails to obtain the outcome within a specified time limit. For example, on a progressive ratio 3, three responses are required to obtain the first outcome, six for the second, nine for the third, and so on. The number of responses produced to obtain the final outcome (the final outcome is the ratio value—it is the final one because, after that trial, the ratio value is so high that the


13

subject quits responding) is termed the “break point” and is an index of the value of the outcome (i.e., how hard the animal is willing to work to obtain the outcome). Progressive ratio schedules have been used to assess the “value” of outcomes such as gustatory stimuli (Reilly, 1999; Sclafani & Ackroff, 2003), electrical brain stimulation (Depoortere, Perrault, & Sanger, 1999; Keesey & Goldstein, 1968), and drugs of abuse (Ranaldi & Wise, 2000; Roberts, Morgan, & Liu, 2007). Another finding should be noted. We mentioned in an earlier section that research has found that changing the value of the instrumental outcome after conditioning and before testing can produce a marked change in the vigor or probability of occurrence of the instrumental response. It was noted that if you train a rat to press a lever to obtain food pellets and then you change the degree that the rat values the food pellets (i.e., pair it with a toxic drug to produce an aversion to the food pellet) such that the rat now dislikes food pellets, then the rat does not bar press much after this manipulation. Dickinson and colleagues (e.g., Adams, 1982: Dickinson, Nicholas, & Adams, 1983) found this to be more true for ratio than interval schedules (see Balleine, 2009, for a review of this literature). The Role of Representations in Conditioning

As mentioned earlier, the field of conditioning and associative learning has become very cognitive in its focus in the past several decades. Given that the word representation is integral to a cognitive approach, the event-memory model put forth by Rescorla in the 1970s (e.g., Rescorla, 1973, 1974) also encouraged, we believe, conditioning to move in a cognitive direction by promoting the idea that an event can activate the mental (and neural) representation of itself and other associated events (see, e.g., Holland & Wheeler, 2009). The event-memory model stated that when pairings of a CS and US occur for an organism, the animal learns at least three things: (1) it forms a representation of the CS in memory; (2) it forms a representation of the US in

2/15/2011 7:41:41 PM

14


memory; and (3) it forms an association between these two representations in memory, a CS-US association. Once a CS is associated with a US, then presentation of the CS will activate a representation of that CS in memory and this will, via the CS-US association, cause an activation of the US representation. A dog salivates to the sound of a tone because of a previously acquired tone– food association. That is, detection of the CS activates the US representation and the dog salivates in anticipation of, and in preparation for, the expected food. In many cases, like the present example, the CR is similar, if not identical, to the unconditioned response (UR) triggered by the US. The UR to food is salivation (a dog salivates when food is placed in its mouth) and the CR to a CS that predicts food is also salivation. However, in other cases, particularly where the US is a drug state, the CR may actually be opposite of the UR (for further discussion, see Dworkin, 1993; Eikelboom & Stewart, 1982; Siegel & Ramos, 2002). Rescorla’s event-memory research focused a great deal on extinction (e.g., Rescorla, 1973; Rescorla, 1974; Rescorla & Heth, 1975). Once a CS-US association is formed as a result of CS-US pairings, CS-alone presentations (i.e., extinction trials) will result in a loss of conditioned responding to the CS. Rescorla stated that CS-alone presentations degrade the representation of the US because the US is no longer being presented at a time that it is expected. If a sufficient number of extinction trials occurred, then not only was the US representation degraded, but the CS-US association will lose strength. Rescorla has conducted a considerable amount of research on extinction phenomena over the past few decades. His views of the underlying mechanisms of extinction in classical conditioning have changed greatly over the years (e.g., Rescorla 2001, 2004). In particular, although providing evidence that extinction does not involve unlearning of the original CS-US association (e.g., Rescorla, 1996), Rescorla may be less convinced than some contemporary theorists that extinction involves an association between the representation of the CS and the representation of the absence of the US (or no US; e.g., Rescorla, 2007).


Two CSs can also be associated if they are presented together (two or more CSs presented together are collectively known as a stimulus compound or compound CS; this stimulus compound may be either simultaneous or serial depending, respectively, on whether the two CSs occur together at the same time or if one CS terminates before the other begins). Rescorla referred to these as within-compound associations (e.g., Rescorla & Cunningham, 1978; Rescorla & Durlach, 1981). Within-compound associations have important implications for many conditioning phenomena. For instance, let’s call one CS “A” and a second CS “B.” If A and B are each associated with the same US and A and B are also associated with each other (via a withincompound association), then what degree of conditioned responding might one expect if, say, A is presented? Well, A is associated with the US and so A can activate the US representation; therefore, we expect a CR to A based on this association. However, A also has the ability to activate a representation of B, and B has the ability to activate a representation of the US, and so A can produce a CR based on this sequence of activations as well (the A representation activates the B representation, this latter representation activates the US representation, and an extra vigorous CR results; e.g., Rescorla & Durlach, 1981). Within-compound associations can influence other conditioning effects. What if we have four CSs: A, B, C, and D? Let’s say that A is associated with B such that A has the ability to activate a representation of B in memory. Assume also that C and D are associated such that C has the ability to activate D in memory. Recent work by Holland and Sherwood (2008) and Dwyer, Mackintosh, and Boakes (1998) has found that if A and C are presented together, then an association can be formed between B and D because A has the ability to activate a representation of B on the AC trial and C has the ability to activate the D representation on the AC trial. Therefore, both the B and D representations are active in the organism’s memory at the same time on the AC conditioning trial. This is a very impressive finding because it shows two events becoming associated when neither is present on the trial

2/15/2011 7:41:41 PM


that is responsible for their associative formation. Simply put, B and D become associated with each other even though they were never paired together. Along with phenomena like sensory preconditioning and higher order conditioning (to be discussed later, see section on “Compound Conditioning”), within-compound associations indicate a level of complexity and scope to classical conditioning that is not widely appreciated today. Postconditioning Presentations of the Unconditioned Stimulus

In support of his event-memory model, Rescorla (e.g., 1974) showed that one can change the representation of the US after conditioning and prior to testing. Rats were given CS-US pairings (e.g., tone paired with medium-intensity footshock) in phase 1. The CS was therefore associated with a representation of footshock which was moderate in intensity. Then in phase 2 Rescorla gave some rats a low-intensity footshock (just shocks presented in the experimental chamber— no CS was presented during this phase) in order to change the US representation from the previously formed “medium-intensity” to a new, low-intensity shock representation. These rats showed a smaller CR—which one would expect if the tone activated a low-intensity shock representation (the low-intensity shock representation replaced the older medium-intensity shock representation). Other rats received a higher intensity shock presented alone without the CS in phase 2. These rats now had a strong US representation associated with the tone (the strong shock representation replaced the medium one), so a stronger CR was expected; and, indeed, a stronger CR was observed. These results are fundamentally important because they demonstrate that, postconditioning, the representation of the USs can be changed (either deflated or inflated) in the absence of the CS. These findings reveal manipulations in classical conditioning that have yet to be fully exploited outside the learning laboratory for their clinical relevance. Another classical conditioning phenomenon has been used to document such processes. Counterconditioning is a treatment in which a


15

CS is paired with a US of one affective valence (e.g., appetitive) in phase 1; and then during phase 2 (the counterconditioning treatment) the CS is paired with a US of the opposite affective valence (e.g., aversive). Readers may recognize that counterconditioning is one of the techniques used to deter alcoholics from drinking by pairing alcohol with an aversive experience (e.g., Revusky, 2009). Counterconditioning was also used to change the US representation associated with a CS for Rescorla’s rats that had received CS-footshock pairings earlier. Counterconditioning results in a lower CR to the CS. However, it is not clear what kind of associations underlie counterconditioning, that is, what the “contents” of learning might be. If a tone is first paired with footshock and, subsequently, the tone is paired with food, then it is easy to imagine that the tone has an association with footshock as well as an association with food. If this is the case, then it is not clear that a footshock representation would truly be modified by the second US representation. Readers might ask whether these various postconditioning treatments (US representational deflation via counterconditioning and extinction) for decreasing the CR possess any similarities. As mentioned earlier, the associative structures underlying them are likely different. However, at least one similarity can be mentioned for counterconditioning and extinction: They both appear to be sensitive to context manipulations, as Bouton’s work demonstrates (e.g., Bouton, 1993, 2002). We will note one more finding before discussing context effects. This finding involves administering CS-alone extinction presentations after the CS-US conditioning pairings. Of course, extinction presentations will decrease the CR to the CS. Research conducted in the laboratories of Bouton, Miller, and Rescorla, among others, has examined the influence of the administration of USs after extinction and prior to testing the CS on performance. Similar to the ideas inherent in Rescorla’s event-memory model, extinction can be said to weaken (or deflate) the US representation. The US presentations occurring after extinction, then, can be said to increase (reestablish or reflate) the US representation.

2/15/2011 7:41:41 PM

16


This treatment is called “US reinstatement” and it does serve to increase the CR for a CS that has received extinction training. However, other interpretations of this effect exist. It has been argued that CS-US associations are learned during conditioning, a CS-noUS association is learned during extinction and the latter association causes interference with the former association, thus rendering it less retrievable. In other words, some have claimed that extinction reduces the retrievability of the CS-US association (and enhances the accessibility of a new CS-noUS association that is presumably learned during extinction); and that the US presentation during reinstatement causes rehearsal of the CS-US association, thereby making this CS-US association retrievable once again. Alternatively, it has also been pointed out that the US presentations can result in context conditioning (i.e., the formation of context-US associations) as a result of the context-US pairings at the time of the US reinstatement phase. If subjects are tested on the CS in the same context where the US reinstatement presentations occurred, then they are being tested on two events at once—the CS and the contextual cues, and both events are associated with the US, so a stronger CR is expected compared to subjects that did not receive US presentations in the context (where testing will occur) prior to the test (Bouton & Bolles, 1979). For the latter condition, only the CS has a strong association with the US. The former subjects are tested on two cues that are associated with the US. Of course, if all subjects are tested on the CS in a neutral context (a context where no other experimental treatments occurred although subjects may be previously acclimated to this context), then both groups are only tested on one event associated with the US. The Role of Contextual Cues During Conditioning

The importance of the conditioning of contextual cues has received a great deal of attention since the 1980s (see, e.g., the edited volume by Balsam & Tomie, 1985). We do not have space here to review the large amount of research findings on the role of these cues, but we will note


two things. First, since pairings between a CS and US (i.e., classical conditioning) as well as instrumental learning do not occur in a vacuum, all types of conditioning procedures have the potential to produce associations involving contextual cues. These associations may involve context-CS associations (akin to withincompound associations mentioned earlier) or context-US associations. Second, there has been a lot of attention focused on the possibility that contextual cues can influence the retrieval of CS-US associations. The laboratories of Bouton, Miller, and Schachtman (Bouton, Garcia-Gutierrez, Zilski, & Moody, 2006; Chelonis, Calton, Hart, & Schachtman, 1999; Gunther, Denniston, & Miller, 1998) have shown that extinction is context specific. That is, if CS-US pairings occur in one context (Context A) and the CS-alone presentations (extinction) occur in a second context (Context B), then performance during those extinction trials will depend on where the CS is tested. After the extinction phase, if the CS is presented in the context where extinction occurred (Context B), then poor conditioned responding will occur. If the CS is tested in a context other than the extinction context (i.e., Context A or a third, neutral context), then the extinction treatment will not be expressed there and a strong CR to the CS will be seen (e.g., Bouton, 2004). This finding has a number of clinical implications not least of which concerns the efficacy of treatments for phobias that utilize a desensitization treatment in a context other than that in which the original fear learning occurred or where the person will encounter the cue in the future (for further discussion, see Bouton, Westbrook, Corcoran, & Maren, 2006; Hermans, Craske, Mineka, & Lovibond, 2006; Redish, Jensen, Johnson, & Kurth-Nelson, 2007; Sotres-Bayon, Cain, & LeDoux, 2006). Another phenomenon that also shows great context specificity is latent inhibition (also known as the “CS preexposure effect”). Latent inhibition is the poor learning to a CS that is paired with a US after preconditioning exposure to the same CS in the absence of the US (Lubow, 1989, 2009; Lubow & Weiner, 2010). That is, in phase 1 the CS is presented several times on its

2/15/2011 7:41:41 PM


own. Then the learning that stems from CS-US pairings in phase 2 is poor because of the phase 1 CS-alone trials. We should also point out that context-CS associations can presumably be formed even when a CS is presented alone in a context, and these associations can be important (Wagner, 1976). Finally, there has also been a lot of research examining the extent to which contextual cues function just like another CS or have distinctive properties of their own (Balsam & Tomie, 1985). Compound Conditioning

Another area of research that has produced a great deal of attention since the 1970s is compound conditioning—conditioning in which more than one CS is presented on a trial. Blocking was discovered in the late 1960s, and this finding stimulated an abundance of research that (along with other findings) gave rise to the RescorlaWagner (1972) model of classical conditioning. Blocking occurs when a weak CR is elicited by a CS (the “target CS”) that had been conditioned in the presence of a second CS (the “blocking CS”) that had been previously paired with the US. That is, in phase 1, the “blocking CS” (CS A) is paired with the US (A-US trials). In phase 2, the blocking CS and target CS (CS X) are presented together and paired with the US (AX-US trials). When the target CS (X) is tested, a weak CR occurs. The target CS does not produce a CR because the information that it provides (signaling the occurrence of the US) is redundant with the information already provided by the blocking CS (A). In phase 1, the subjects learn that the US is predicted by the blocking CS, A. In phase 2, the subject already expects the US because of the presence of CS A, and so no evidence of learning to CS X occurs. CS X is redundant and learning about it is “blocked.” The Rescorla-Wagner model highlights the important idea that CSs can compete for learning and predicts a large number of conditioning phenomena (Miller, Barnet, & Grahame, 1995). This model predicts another phenomenon in which CSs compete for learning— overshadowing,4 as well as many phenomena involving conditioned inhibitors5—stimuli that predict that the US will not occur.


17

Some researchers have pointed out that CSs and instrumental responses may compete for learning. If a CS and a response both occur on a conditioning trial along with an outcome, then blocking or overshadowing of one of these events by the other event may occur (see Schachtman & Reed, 1998 for a review). If such competition occurs, then it is valuable to consider this effect because organisms are typically responding in some way at the time that a classical conditioning trial occurs, such that this behavior may compete with the CS for learning and attenuate the degree of manifest classical conditioning to the CS. Moreover, stimuli are typically present during instrumental conditioning (minimally, the cues of the experimental apparatus within which the subject is located), and these cues may compete with the response for learning and undermine the degree of expressed instrumental conditioning. Second-order conditioning and sensory preconditioning are two compound conditioning phenomena. Second-order conditioning involves pairing of a CS (A) with the US in phase 1 (A-US pairings) and then in phase 2 the target CS (X) is presented along with A but without the US (XAtrials; notice that a minus symbol is used to designate the absence of the US). A conditioned response occurs to X even though it had never been paired with the US. One early interpretation of this phenomenon by Rescorla (1980a, 1980b) is that the CR might occur to X because X is associated with A and A is associated with the US. Hence, X is capable of activating a representation of A in memory (via the A-X association acquired in phase 2) and this activation of A is capable of activating a representation of the US in memory (via the A-US association acquired in phase 1). X evokes a CR since the US representation has been activated when X is presented (via this sequence of associative linkages). Note that the association between A and X is similar to the within-compound associations mentioned earlier. Sensory preconditioning is similar in many ways to second-order conditioning. The primary difference is that the two training phases are reversed. In sensory preconditioning, then, XA– is presented in phase 1. Note that no US is

2/15/2011 7:41:41 PM

18


presented in this phase—just two CSs. In phase 2, A is paired with the US (A-US) and then, in the test phase, the occurrence of the CR to CSX is examined. A CR to X demonstrates that X can generate a CR without ever having been paired directly with the US. Instead, X was paired with A and an association was formed between them. CSA was paired with the US. Hence, a CR to X could occur if X activates a representation of A and A activates a representation of the US; and the active US representation results in a CR. However, other mechanisms may exist for why the CR to X occurs following sensory preconditioning training. These findings and related phenomena reveal interesting effects in classical conditioning and provide a glimpse at the theoretical approaches to understanding their underlying processes and structures. Distinctions Among Procedures in Which Single Conditioned Stimuli Are Presented

Researchers can have a difficult time distinguishing among the many procedures that, for the most part, involve single-stimulus presentation. Habituation, pseudoconditioning, dishabituation, latent inhibition, disinhibition, sensitization, and extinction can easily be confused. Habituation and sensitization are easy to discuss because they involve the presentation of stimuli in a situation in which no conditioning trials occur. Habituation is the progressive decrease in responding to the stimulus if it is repeatedly presented. Sensitization is the increase in responding to a stimulus if it is repeatedly presented. Thus, in terms of behavioral responding, sensitization is the opposite of habituation. However, sensitization, unlike habituation, occurs when the subject, for any of a number of reasons, is in a state of high arousal. Habituation may be characterized as learning to ignore stimuli that have proven to be of no biological consequence. By helping us not to become distracted by harmless and meaningless stimuli, habituation is one of the most pervasive phenomena in our life. Sensitization, on the other hand, involves heightened vigilance to stimuli that might be dangerous. Indeed, it has been suggested that sensitization may contribute to a number of


clinical conditions, including phobia and generalized anxiety disorder (e.g., Marks, 1987), as well as posttraumatic stress disorder (e.g., Dykman, Ackerman, & Newton, 1997). Although published many years ago, the dual-process theory of Groves and Thompson (1970; also see Thompson, 2009), which views habituation and sensitization as independent processes that function in parallel, remains one of the more prominent explanations of these two phenomena. Dishabituation is also easy to explain. This occurs when a stimulus (let’s say a tone) has already received habituation training so that the subject is no longer making a strong response to it. But some other “extraneous” stimulus occurs (say, a light) and this produces dishabituation to the tone in that a response suddenly occurs to the tone when it is presented again (especially if it is presented shortly after the light occurred). We have already discussed extinction—the decrease in CR when a CS (that has been previously conditioned using CS-US trials) is presented alone for many trials. Hence, a tone could be paired with a US and then given tone-alone extinction trials. If another “extraneous stimulus” occurs (say, a light), then the light might cause the CR to return to the CS. This increase in conditioned responding is called disinhibition. Therefore, dishabituation and disinhibition both involve an extraneous stimulus causing an increase in responding to a stimulus that had previously lost response-producing ability. Dishabituation involves a response that was not previously conditioned, whereas disinhibition involves a previously conditioned cue. Both habituation and extinction involve a decrease in responding (in the case of extinction, it is a CR that decreases; in the case of habituation, it is the waning of an unconditioned response to a stimulus). It is not clear if the mechanisms underlying extinction and habituation (as well as for disinhibition and dishabituation) are similar; however, it seems unlikely that the two phenomena are comparable. Latent inhibition, as we mentioned, involves CS-alone presentations prior to CS-US pairings, and the effect of these initial presentations of the CS is poor learning when the CS is paired with

2/15/2011 7:41:42 PM


the US. Latent inhibition shares features with a few of the other phenomena presented in this section. The procedure for latent inhibition is the opposite of extinction: CS-alone presentations precede CS-US pairings for the former, whereas CS-alone presentations follow CS-US pairings for the latter. Despite this similarity (and difference) it seems unlikely that extinction and latent inhibition possess identical mechanisms; although, as mentioned earlier, both effects involve dependence on the contextual cues present as to whether the learning that occurs during the CS-alone phase will be expressed. Both phenomena may involve retrieval processes (Miller, Kasprow, & Schachtman, 1986). The first phase of a latent inhibition procedure is similar to a habituation experiment in which the stimulus is simply presented repeatedly. Of course, a habituation experiment focuses on the response to the stimulus during this exposure phase, while latent inhibition focuses on conditioning to the stimulus in a subsequent phase; but, nonetheless, investigators have been curious as to whether similar processes may occur during the stimulus-alone phases during habituation and latent inhibition (e.g., a loss of attention). Pseudoconditioning is a procedure in which subjects receive presentations of the CS and US (e.g., a tone paired with shock) and it appears as though a CR is occurring to the CS due to conditioning (i.e., it appears as though learning has occurred), but what has really happened is that the US presentation on a previous trial has influenced the subject in such a way that they make a response (e.g., a startle response such as freezing) to the CS that looks like, but is not, the CR (e.g., a fear response such as freezing). Hence, conditioning has not occurred, but it appears as though it has occurred. Researchers sometimes use control conditions to identify (and therefore potentially rule out) the contribution that pseudoconditioning is making to manifest performance during conditioning. One reason that we have discussed the differences (and similarities) among these (primarily) single-stimulus treatments is that researchers and clinicians may become confused when administering a treatment. For instance, if someone has


19

a fear of an untethered, growling, snarling dog and seeks treatment for this fear, then the clinician may administer a form of exposure therapy for the treatment. These therapeutic interventions (exposure therapy) are often viewed as extinction treatments. They are described this way for good reason in many cases. However, was the fear response to the dog acquired through conditioning or not? If the fear was acquired through conditioning, then certainly the use of flooding or systematic sensitization should be considered extinction. But if there was no conditioning or learning of the fear (if the person has always had it), then the treatment could be viewed as habituation training. In both cases, we know that recovery of the fear can occur through spontaneous recovery (via time) or disinhibition/dishabituation (via an extraneous salient event) for the extinguished response. Both habituation and extinction can be reversed by a retention interval or an extraneous event. We also know that extinction (Bouton, Garcia-Gutierrez, Zilski, & Moody, 2006; Chelonis, Calton, Hart, & Schachtman, 1999; Gunther, Denniston, & Miller, 1998) and habituation (Marlin & Miller, 1981) are context specific; if you change contexts, the response can return. This chapter covered numerous instances of conditioning. The chapter sought to clarify some issues that may confuse psychologists and other researchers or practitioners, but it also illustrated some important findings that explore the processes and structures underlying associative learning. The following chapters examine the role of conditioning, based on contemporary research, in a number of domains, including connectionist modeling, psychoneuroimmunology, social inference and attribution, incentive learning, fears and phobias and anxiety, addictive behaviors, social learning, marketing, schizophrenia, learned helplessness, autism, analgesic responses, sexual behavior, evaluative conditioning, and consummatory behavior. These chapters were written by many of the premiere investigators in the field, and we hope they illustrate that conditioning and associative learning theory and research continues to flourish and be influential in many areas of our discipline.

2/15/2011 7:41:42 PM

20


ACKNOWLEDGMENTS Preparation of this chapter was supported in part by National Institutes of Deafness and Other Communication Disorders grant DC06456 to Steve Reilly. Thanks to Stephanie Fowler, Oskar Pineno, Jen Walker, and Jian-You Lin for their comments on this chapter.

NOTES 1. Note that the abbreviation “US” is used currently by most animal conditioning researchers rather than the more archaic “UCS” that one occasionally sees in some elementary textbooks. 2. It is a negative contingency because of the occurrence of the response means that the aversive event has not occurred and the occurrence of the aversive event means that the response did not occur. 3. A trial is defined as the events that occur prior to each reward presentation. 4. Overshadowing, like blocking, results in weaker learning to one of the two elements of a compound CS. However, overshadowing is empirically defined as the difference in conditioning to a CS that is paired with the US in the presence of a second CS relative to a group that receives conditioning to a CS paired with the US in the absence of a second CS. 5. A conditioned inhibitor is a cue that signals the absence of an otherwise expected US.

REFERENCES Adams, C. D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 34B, 77–98. Adams, C. D., & Dickinson, A. (1981). Instrumental responding following reinforcer devaluation. Quarterly Journal of Experimental Psychology, 33B, 109–121. Balleine, B. W. (2005). Neural bases of foodseeking: Affect, arousal and reward in corticostriatolimbic circuits. Physiology and Behavior, 86, 717–730. Balleine, B. W. (2009). Taste, disgust, and value: Taste aversion learning and outcome encoding in instrumental conditioning. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes


(pp. 262–280). New York, NY: Oxford University Press. Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009). The integrative function of the basal ganglia in instrumental conditioning. Behavioral Brain Research, 199, 43–52. Balsam, P. D., & Tomie, A. (Eds.). (1985). Context and learning. Hillsdale, NJ: Erlbaum. Bolles, R. C. (1972). Reinforcement, expectancy, and learning. Psychological Review, 79, 394–409. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80–99. Bouton, M. E. (2002). Context, ambiguity and unlearning: Sources of relapse after behavioral extinction. Biological Psychiatry, 52, 976–986. Bouton, M. E. (2004). Context and behavioral processes in extinction. Learning and Memory, 11, 485–494. Bouton, M. E. (2007). Learning and behavior: A contemporary synthesis. Sutherland, MA: Sinauer. Bouton, M. E., & Bolles, R. C. (1979). Role of conditioned contextual stimuli in reinstatement of extinguished fear. Journal of Experimental Psychology: Animal Behavior Processes, 5, 368–378. Bouton, M. E., Garcia-Gutierrez, A., Zilski, J., & Moody, E. W. (2006). Extinction in multiple contexts does not necessarily make extinction less vulnerable to relapse. Behavioural Research and Therapy, 44, 983–994. Bouton, M. E., Westbrook, R. F., Corcoran, K. A., & Maren, S. (2006). Contextual and temporal modulation of extinction: Behavioral and biological mechanisms. Biological Psychiatry, 60, 352–360. Chelonis, J. J., Calton, J. L., Hart, J. A., & Schachtman, T. R. (1999). Attenuation of the renewal effect by extinction in multiple contexts. Learning and Motivation, 30, 1–14. Colwill, R. M., & Rescorla, R. A. (1985). Postconditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 11, 120–132. Colwill, R. M., & Rescorla, R. A. (1986). Associative structures in instrumental learning. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol 20, pp. 55–104). Orlando, FL: Academic Press. Delamater, A. R. (2004). Experimental extinction in Pavlovian conditioning: Behavioural and

2/15/2011 7:41:42 PM


neuroscience perspectives. Quarterly Journal of Experimental Psychology, 57B, 97–132. Depoortere, R., Perrault, G., & Sanger, D. J. (1999). Intracranial self-stimulation under a progressive-ratio schedule in rats: Effects of strength of stimulation, d-amphetamine, 7-OH-DPAT and haloperidol. Psychopharmacology, 142, 221–229. Dickinson, A. (1980). Contemporary animal learning theory. Cambridge, UK: Cambridge University Press. Dickinson, A. (1985). Actions and habits: The development of behavioral autonomy. Philosophical Transactions of the Royal Society (London). B308, 67–78. Dickinson, A., & Balleine, B. (1994). Motivational control of behavior. Animal Learning and Behavior, 22, 1–18. Dickinson, A., Balleine, B., Watt, A., Gonzalez, F., & Boakes, R. A. (1995). Motivational control after extended instrumental training. Animal Learning and Behavior, 23, 197–206. Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983). The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 35B, 35–51. Domjan, M. (2009). The principles of learning and behavior (6th ed.). Belmont, CA: Wadsworth/ Thompson Learning. Dworkin, B. R. (1993). Learning and physiological regulation. Chicago, IL: University of Chicago Press. Dwyer, D. M., Mackintosh, N. J., & Boakes, R. A. (1998). Simultaneous activation of the representations of absent cues results in the formation of an excitatory association between them. Journal of Experimental Psychology: Animal Behavior Processes, 24, 163–171. Dykman, R. A., Ackerman, P. T., & Newton, J. E. (1997). Posttraumatic stress disorder: A sensitization reaction. Integrative Physiological and Behavioral Science, 32, 9–18. Eikelboom, R., & Stewart, J. (1982). Conditioning of drug-induced physiological responses. Psychological Review, 89, 507–528. Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York, NY: AppletonCentury-Crofts. Frieman, J. (2002). Learning and adaptive behavior. Belmont, CA: Wadsworth/Thompson Learning. Gluck, M. A., Mercado, E., & Myers, C. E. (2007). Learning and memory: From brain to behavior. New York, NY: Worth Publishers.


21

Groves, P. M., & Thompson, R. F. (1970). Habituation: A dual-process theory. Psychological Review, 77, 419–450. Gunther, L. M., Denniston, J. C., & Miller, R. R. (1998). Conduct exposure treatment in multiple contexts can prevent relapse. Behaviour Research and Therapy, 36, 75–91. Hearst, E., & Jenkins, H. M. (1974). Sign–tracking: The stimulus–reinforcer relation and directed– action. Austin, TX: The Psychonomic Society. Hermans, D., Craske, M. G., Mineka, S., & Lovibond, P. F. (2006). Extinction in human fear conditioning. Biological Psychiatry, 60, 361–368. Hodos, W. (1961). Progressive ratio as a measure of reward strength. Science, 134, 943–944. Holland, P. C. (1992). Occasion-setting in Pavlovian conditioning. In D. L. Medin (Ed.), The psychology of learning and motivation (Vol. 28, pp. 69–125). San Diego, CA: Academic Press. Holland, P. C. (2004). Relations between Pavlovianinstrumental transfer and reinforcer devaluation. Journal of Experimental Psychology: Animal Behavior Processes, 30, 104–117. Holland, P. C., & Sherwood, A. (2008). Formation of excitatory and inhibitory associations between absent events. Journal of Experimental Psychology: Animal Behavior Processes, 34, 324–335. Holland, P. C., & Wheeler, D. S. (2009). Representation-mediated food aversions. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 196–225). New York, NY: Oxford University Press. Kamin, L. J. (1968). “Attention–like” processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimulation (pp. 9–31). Coral Gables, FL: University of Miami Press. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279–296). New York, NY: Appleton–Century–Crofts. Keesey, R. E., & Goldstein, M. D., (1968). Use of progressive fixed-ratio procedures in the assessment of intracranial reinforcement. Journal of the Experimental Analysis of Behavior, 11, 293–301. Klein, S. B., & Mowrer, R. R. (Eds.). (1989a). Contemporary learning theories: Instrumental conditioning theory and the impact of biological constraints on learning. Hillsdale, NJ: Erlbaum.

2/15/2011 7:41:42 PM

22


Klein, S. B., & Mowrer, R. R. (Eds.). (1989b). Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory. Hillsdale, NJ: Erlbaum. Konorski, J. (1967). Integrative activity of the brain. Chicago, IL: University of Chicago press. Locurto, C. M., Terrace, H. S., & Gibbon, J. (Eds.). (1981). Autoshaping and conditioning theory. New York, NY: Academic Press. Lubow, R. E. (1989). Latent inhibition and conditioned attention theory. Cambridge, England: Cambridge University Press. Lubow, R. E. (2009). Conditioned taste aversion and latent inhibition: A review. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 37–57). New York, NY: Oxford University Press. Lubow, R. E., & Weiner. I. (Eds.). (2010) Latent inhibition: Cognition, neuroscience, and applications to schizophrenia. New York, NY: Cambridge University Press. Mackintosh, N. J. (1974). The psychology of animal learning. London, England: Academic Press. Mackintosh, N. J. (1983). Conditioning and associative learning. New York, NY: Clarendon Press. Mackintosh, N. J. (Ed.). (1994). Animal learning and cognition. London, England: Academic Press. Marks, I. M. (1987). Fears, phobias and rituals: Panic, anxiety, and their disorders. New York, NY: Oxford University Press. Marlin, N. A., & Miller, R. R. (1981). Associations to contextual stimuli as a determinant of long-term habituation. Journal of Experimental Psychology: Animal Behavior Processes, 7, 313–333. Miller, R. R., Barnet, R. C., & Grahame, J. (1995). Assessment of the Rescorla-Wagner model. Psychological Bulletin, 117, 363–386. Miller, R. R., Kasprow, W. J., & Schachtman, T. R. (1986). Retrieval variability: Sources and consequences. American Journal of Psychology, 99, 145–218. Mowrer, O. H. (1951). Two-factor learning theory: Summary and comment. Psychological Review, 58, 350–354. Mowrer, R. R., & Klein, S. B. (Eds.). (2001). Handbook of contemporary learning theories. Hillsdale, NJ: Erlbaum. Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex (G. V. Anrep, Trans.). London, England: Oxford University Press. Pearce, J. M. (2008). Animal learning and cognition (3rd ed.). New York, NY: Psychology Press.


Ranaldi, R., & Wise, R. A. (2000). Intravenous selfadministration of methamphetamine-heroin (speedball) combinations under a progressiveratio schedule of reinforcement in rats. Neuroreport, 11, 2621–2623. Redish, A., Jensen, S., Johnson, A., & Kurth-Nelson, A. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychological Review, 114, 784805. Reilly, S. (1999). Reinforcement value of gustatory stimuli determined by progressive ratio performance. Pharmacology Biochemistry and Behavior, 63, 301–311. Rescorla, R. A. (1973). Effect of US habituation following conditioning. Journal of Comparative and Physiological Psychology, 82, 137–143. Rescorla, R. A. (1974). Effect of inflation of the unconditioned stimulus value following conditioning. Journal of Comparative and Physiological Psychology, 86, 101–106. Rescorla, R. A. (1980a). Pavlovian second-order conditioning: studies in associative learning. Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1980b). Simultaneous and successive associations in sensory preconditioning. Journal of Experimental Psychology: Animal Behavior Processes, 6, 207–216. Rescorla, R. A. (1988). Pavlovian conditioning. It’s not what you think it is. American Psychologist. 43, 151–160. Rescorla, R. A. (1991). Associative relations in instrumental learning: The eighteenth Bartlett memorial lecture. Quarterly Journal of Experimental Psychology, 43B, 1–23. Rescorla, R. A. (1996). Preservation of Pavlovian associations through extinction. Quarterly Journal of Experimental Psychology, 49B, 245–258. Rescorla, R. A. (2001). Experimental extinction. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 119–154). Hillsdale, NJ: Erlbaum. Rescorla, R. A. (2004). Spontaneous recovery. Learning and Memory, 11, 501–509. Rescorla, R. A. (2007). Renewal after overexpectation. Learning and Behavior, 35, 19–26. Rescorla, R. A., & Cunningham, C. L. (1978). Within-compound flavor associations. Journal of Experimental Psychology: Animal Behavior Processes, 4, 267–275. Rescorla, R. A., & Durlach, P. J. (1981). Within-event learning in Pavlovian conditioning. In N. E. Spear & R. R. Miller (Eds.), Information processing

2/15/2011 7:41:42 PM


in animals: Memory mechanisms (pp. 81–111) Hillsdale, NJ: Erlbaum. Rescorla, R. A., & Heth, D. C. (1975). Reinstatement of fear to an extinguished conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 1, 88–96. Rescorla, R. A., & Solomon, R. L. (1967). Twoprocess learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychological Review, 74, 151–182. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton–Century–Crofts. Revusky, S. (2009). Chemical aversion treatment of alcoholism. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 445–472). New York, NY: Oxford University Press. Roberts, D. C. S., Morgan, D., & Liu, Y. (2007). How to make a rat addicted to cocaine. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 31, 1614–1624. Rozeboom, W. W. (1958). “What is learned?” An empirical enigma. Psychological Review, 65, 22–33. Schachtman, T. R., & Reed, P. (1998). Optimization: Some factors that facilitate and hinder optimal performance in animals and humans. In W. O’Donahue (Ed.), Learning and behavior therapy (pp. 301–333). Needham Heights, MA: Allyn & Bacon. Schmajuk, N. A., & Holland, P. C. (1998). Occasion setting: Associative learning and cognition in animals. Washington, DC: American Psychological Association. Sclafani, A., & Ackroff, K. (2003). Reinforcement value of sucrose measured by progressive ratio operant licking in the rat. Physiology and Behavior, 79, 663–670. Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.), Classical conditioning: A symposium (pp. 302–322). New York, NY: Appleton-Century-Crofts. Siegel, S., & Ramos, B. M. C. (2002). Applying laboratory research: Drug anticipation and the treatment of drug addiction. Experimental and Clinical Psychopharmacology, 10, 162–183. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York, NY: AppletonCentury-Crofts.


23

Sotres-Bayon, F., Cain, C. K., & LeDoux, J. E. (2006). Brain mechanisms of fear extinction: Historical perspectives on the contribution of prefrontal cortex. Biological Psychiatry, 60, 329–336. Spear, N. E. (1978). The processing of memories: Forgetting and retention. Hillsdale, NJ: Erlbaum. Spear, N. E., & Riccio, D. C. (1994). Memory: Phenomena and principles. Needham Heights, MA: Allyn & Bacon. Thompson, R. F. (2009). Habituation: A history. Neurobiology of Learning and Memory, 92, 127–134. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monograph 2 (4, Whole No. 8). Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York, NY: Macmillan. Tolman, E. C., & Honzik, C. H. (1930). Introduction and removal of reward and maze performance in rats. University of California Publications in Psychology, 4, 257–275. Tulving, E., & Pearlstone, Z. (1966). Availability versus accessibility of information in memory for words. Journal of Verbal Learning and Verbal Behavior, 5, 381–391. Wagner, A. R. (1976). Priming in STM: An information-processing mechanism for selfgenerated or retrieval-generated depression in performance. In T. J. Tighe & R. N. Leaton (Eds.), Habitation: Perspectives from child development, animal behavior, and neurophysiology (pp 95–128). Hillsdale, NJ: Erlbaum. Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in Pavlovian conditioning: Application of a theory. In R. A. Boakes & M. S. Halliday (Eds.), Inhibition and learning (pp. 301–336). New York, NY: Academic Press. Williams, B. A. (1999). Associative competition in operant conditioning: Blocking the responsereinforcer association. Psychonomic Bulletin and Review, 6, 618–623. Williams, D. R. (1981). Biconditional behavior: Conditioning without constraint. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 55–99). New York, NY: Academic Press. Williams, D. R., & Williams, H. (1969). Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, 511–520. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience. 7, 464–476.

2/15/2011 7:41:42 PM


PART II

Applications to Clinical Pathology


2/15/2011 7:42:10 PM


CHAPTER 2 Fear Extinction and Emotional Processing Theory A Critical Review Seth J. Gillihan and Edna B. Foa

The process of fear extinction in animal studies bears strong resemblance to the process of reduction of pathological anxiety in humans via exposure therapy. Thus, findings emerging from experiments of extinction can inform us about the mechanisms of exposure therapy, which may lead to modifying the manner in which therapists conduct exposure and thereby improve treatment outcomes. In this chapter we use emotional processing theory as a framework to organize the knowledge about both exposure therapy and extinction. We examine whether hypotheses and suppositions derived from the theory are consistent with knowledge emerging from extinction experiments and from treatment studies of anxiety disorders; conversely, we examine how emotional processing theory can inform the questions that extinction research needs to address. We include an examination of the neural correlates of the reduction of pathological fear, which may allow us to expand our knowledge of mechanisms of exposure therapy by adding brain processes to the existing behavioral mechanisms implicated in extinction.

INTRODUCTION We view a parallel between exposure therapy and extinction paradigms in animals and humans and therefore take the position that findings emerging from experiments of extinction can inform us about the mechanisms of exposure therapy. It is our hope that this information will lead to modifying the manner in which therapists conduct exposure and thereby improve outcomes. As clinical researchers have long noted (Baum, 1970; Dollard & Miller, 1950; Stampfl & Levis, 1967), the process of fear extinction in animal studies bears strong resemblance to the process of reduction of pathological anxiety via exposure therapy, including the amelioration of symptoms of anxiety disorders. Indeed, patients with anxiety disorders tend to show deficient fear extinction in laboratory studies (Lissek et al., 2005). Research into the phenomenon of fear

extinction therefore may have great relevance to the understanding of the learning processes that facilitate effective cognitive-behavioral treatment for anxiety disorders. In this chapter we use emotional processing theory (Foa & Kozak, 1985, 1986) as a framework to organize the knowledge about both exposure therapy and extinction. This approach will allow us to examine whether hypotheses and suppositions derived from the theory are consistent or in contradiction with knowledge emerging from extinction experiments and from treatment studies of anxiety disorders; conversely, we can examine how emotional processing theory can inform the questions that need to be asked by extinction research. Also, the review of the extinction literature may allow us to expand our knowledge of mechanisms of exposure therapy by adding brain processes to the existing behavioral mechanisms implicated in extinction.

27


2/15/2011 7:42:10 PM

28


EMOTIONAL PROCESSING THEORY Foa and Kozak’s (1985, 1986) emotional processing theory presented a heuristic model for understanding pathological anxiety and the mechanisms involved in treatment of anxiety disorders. Expanding on Lang’s (1977, 1984) bioinformational model, the starting point of emotional processing theory is the supposition that fear is represented in memory as a cognitive structure that includes information about the fear stimuli, the fear responses, and their meaning. In contrast to Lang who emphasized the role of response representations, emotional processing theory places particular emphasis on the meaning representations of the stimuli and the responses. For example, a combat soldier in Vietnam may have a fear structure that includes representations of stimuli such as persons moving through the jungle and representations of responses such as heart beating fast and muscle tension. Of particular importance, however, is the meaning of the persons moving through the jungle as “those are Viet Cong soldiers and my life is in danger”; and the meaning of heart beating fast and muscle tension as “I am afraid.” The representations of the stimuli, responses, and their meaning in the structure are related to each other such that when a stimulus and/or response in the environment matches those represented in the fear structure, the entire structure is activated. Thus, seeing an enemy soldier in the jungle will activate the representation of a moving person, the meaning associated with that representation (“I’m in danger”), and the behavioral and physiological fear responses. In addition to emphasizing the crucial role of meaning representations, Foa and Kozak (1986) outlined the distinguishing features of normal and pathological fear structures. In the example mentioned earlier, the soldier’s fear structure is normal if it is restricted to the jungle setting during wartime; in these circumstances, activation of the fear structure will lead to adaptive responses such as staying low and avoiding enemy fire. In contrast, the fear structure is pathological if it is activated by safe stimuli, for example when, upon returning to the United States


the Vietnam veteran experiences fear and “takes cover” when seeing a person walking through the woods while vacationing with his family in the Great Smoky Mountains National Park. In other words, the pathological fear structure produces overgeneralization when safe stimuli are perceived as dangerous. Pathological fear structures also comprise excessive response elements (e.g., hypervigilance). According to emotional processing theory, two conditions are required for the modification of pathological fear structures. First, the fear structure must be activated through exposure to information that is sufficiently similar to the information embedded in the fear structure. Second, information that is incompatible with the pathological elements of the fear structure must be available during exposure and incorporated into the fear structure. In the example of the combat veteran, the fear structure was activated when the veteran was exposed to persons moving through the woods; remaining in that situation and discovering that the persons do not harm him constitutes information that is incompatible with his perception that all moving persons in the woods are dangerous. The incorporation of this information into the existing pathological fear structure modifies the pathological elements in the structure. This modification, which is the essence of emotional processing, underlies the reduction in pathological fear. While it is not possible to directly observe fear structures and their modification, Foa and Kozak (1986) postulated three indicators of successful emotional processing: activation of the fear structure, as indicated by both subjective and objective measures of fear; within-session habituation,1 or the reduction of anxiety within the course of a treatment session; and betweensession habituation, that is, lower peak anxiety to fear-related stimuli during successive treatment sessions. As Foa and Kozak (1986) suggested, many sources of data can be brought to bear to test the prediction that these three indicators of emotional processing are associated with the reduction of pathological fear. The original formulations of the theory incorporated existing research, which included behavioral and

2/15/2011 7:42:10 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

psychophysiological studies conducted primarily with human subjects; an update of the theory (Foa, Huppert, & Cahill, 2006) examined the degree to which the latest research lent support to the predictions of emotional processing theory, again focusing on clinical findings from human studies. The current chapter aims to evaluate the validity of emotional processing theory in light of basic research in learning processes. Learning research offers a powerful experimentally testable model of psychopathology and of treatment. It is particularly relevant for anxiety disorder because the biology and psychology of fear have been the focus of animal studies for decades; as such, findings from the learning literature may inform theoretical models of pathological anxiety and its treatment. Since the work of Ivan Pavlov (e.g., 1927), learning research has shed light on the processes that underlie the acquisition and retention of associations between stimuli and responses. Learning research has played a key role in our understanding of anxiety disorders, including panic disorder (Bouton, Mineka, & Barlow, 2001), posttraumatic stress disorder (PTSD; Rothbaum & Davis, 2003), and simple phobia (Davey, 1992). The applicability of learning principles to the study of anxiety is particularly salient in the context of PTSD because of the clear etiological factors associated with the development of the disorder. Indeed, PTSD is the sole anxiety disorder with diagnostic criteria that require an identifiable event that preceded the onset of the disorder. Much animal research has been done to model the fear responses seen after the experience of a trauma. Fear conditioning as an animal model of PTSD was discussed at length by Foa, Zinbarg, and Rothbaum (1992). Briefly, Foa and colleagues argued that the behavioral responses seen in animals that are exposed to uncontrollable and unpredictable aversive stimuli parallel the clusters of symptoms seen in PTSD. For example, the heightened fear and arousal seen in fear-conditioned animals may be analogous to the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) criterion of persistent arousal; similarly, animals tend to show avoidance of the fear-related stimuli, just


29

as PTSD patients tend to avoid trauma-related stimuli. The authors also make the case that animal models of PTSD can be used to generate hypotheses about human trauma-related psychopathology. For instance, based on the animal literature they posit that traumatic events that lead to PTSD are perceived not only as more life threatening (e.g., Dunmore, Clark, & Ehlers, 1999; Resnick, Kilpatrick, Dansky, Saunders, & Best, 1993) but also as unpredictable or uncontrollable. Most individuals who experience a traumatic event experience fear-related symptoms that overlap with those of PTSD, including reexperiencing the event in response to reminders of it, hyperarousal, and avoidance of trauma-related stimuli (e.g., Breslau, Reboussin, Anthony, & Storr, 2005; Rothbaum, Foa, Riggs, Murdock, & Walsh, 1992). These symptoms ameliorate over time in most trauma survivors. When such reduction does not occur, chronic PTSD develops. Thus, PTSD can be viewed as a failure to extinguish conditioned fear responses. This perspective on the tenacious fear reactions in PTSD is supported by studies showing that patients with PTSD demonstrate deficient fear extinction relative to controls (Blechert, Michael, Vriends, Margraf, & Wilhelm, 2007; Lissek et al., 2005; Peri, Ben-Shakhar, Orr, & Shalev, 2000). Earlier conceptualizations viewed extinction as a passive process of undoing learned associations, and little research was devoted to understanding the processes that facilitate or inhibit extinction. More recent research revealed that fear extinction is an active process that is distinct from the process of fear acquisition. Accordingly, an extinguished fear response (conditioned stimulus [CS]) may show spontaneous recovery (e.g., Leung & Westbrook, 2008), is “reinstated” when the unconditioned stimulus (US) is presented in the absence of the CS (e.g., Rescorla & Heth, 1975), and reemerges in contexts other than the extinction setting (e.g., Alvarez, Johnson, & Grillon, 2007). These findings suggest that extinction is not simply the erasure of the acquired fear response. Instead, extinction is thought to involve the inhibition of a fear association through the formation of new stimulus–response associations. As noted by

2/15/2011 7:42:11 PM

30


Barad, Gean, and Lutz (2006), “extinction gates the expression of fear” (p. 326). This new conceptualization influenced Foa and McNally (1996) who updated emotional processing theory by suggesting that exposure therapy generates new fear structures rather than weakening the associations of the existing pathological fear structure. As discussed earlier, emotional processing theory posits three indicators that emotional processing of the fear structure has occurred (i.e., the pathological elements of the fear structure were modified): activation of the fear, and within-session and between-session habituation to fear-related stimuli. The hypotheses that are generated by the proposed indicators can be tested in fear extinction paradigms. First, the degree of fear activation during extinction training will be positively associated with a greater reduction in fear responses. Second, the degree of fear reduction within a fear extinction session will be associated with greater reduction at testing. Finally, lower peak responses in successive sessions of presentation of the previously conditioned stimulus will be associated with fear reduction at testing. We will address each of these hypotheses by examining the relevant research findings from human and animal studies from basic behavioral research to neural localization (i.e., identifying a brain area that is associated with a particular behavior or cognitive process) studies across a range of methodologies. We will focus on fear, although clearly PTSD comprises many other emotions such as anger or guilt; nevertheless, the majority of PTSD sufferers experience debilitatingly high levels of unrealistic fear. The fear is evident in the common symptoms of PTSD such as avoidance of trauma-related stimuli, stress reaction to trauma reminders, hypervigilance, and being easily startled. A Note About Terminology

For the sake of clarity we distinguish between the extinction phase, which involves presenting the CS in the absence of the US to previously fear conditioned organisms, and the test phase, which occurs at some point in time after the extinction phase.


Evaluation of Prediction 1: Fear Activation During Extinction Behavioral Studies

During fear extinction paradigms, fear is activated by definition because the organism has been conditioned to experience fear when confronted with a CS that was associated with an aversive US. The fact that fear is activated as part of the extinction paradigm results in the restriction of range (i.e., low variability) in initial fear, which in turn renders the examination of the effects of level of fear activation on subsequent fear extinction problematic given that the detection of a significant relationship between level of activation and degree of extinction requires variability in initial fear. Despite this inherent limitation, there is evidence that the level of fear activation during fear extinction learning predicts fear responding at test. Animals who were administered a chemical agent known to reduce fear activation (i.e., barbiturates) during fear extinction showed greater fear responses to later presentations of the CS during the test phase when the animals were not drugged (e.g., Barry, Etheredge, & Miller, 1965). Similarly, Bouton, Kenney, and Rosengard (1990) found that benzodiazepines commonly used in the treatment of anxiety disorders (chlordiazepoxide and diazepam) interfered with fear extinction learning. Bouton et al. paired a chamber with footshock and subsequently exposed the animals to the chamber in the absence of shock. Although rats in these experiments showed fear extinction during the extinction learning phase of the experiment irrespective of whether they received benzodiazepines, animals that had received the benzodiazepines during extinction showed significantly greater fear responses during the test phase when the animals were exposed to the chamber in an undrugged state. Given that benzodiazepines and other sedating drugs reduce fear-related behavior, these data support the premise that fear activation during the extinction of conditioned fear is associated with greater reductions in fear at test. However, these studies did not specifically address variability in the animals’ fear responses during extinction learning as a predictor of

2/15/2011 7:42:11 PM


eventual fear extinction; rather, the studies focused on the differences among groups that received different drugs. Moreover, other interpretations of these results are possible, such as state-dependent learning—that is, that optimal recall during the test phase depends on the animal’s being in the same physiological state as during the learning acquisition phase (see Overton, 1985). Nevertheless, results from these and similar studies are consistent with the hypothesis that greater activation during exposure therapy is associated with greater reductions in anxiety. More direct behavioral evidence for the role of activation in fear extinction comes from later research demonstrating that direct pharmacologic manipulation of fear responses during extinction affects fear responding during the test phase. Cain, Blouin, and Barad (2004) administered yohimbine or propranolol to mice prior to fear extinction; yohimbine blocks the α2-receptor, resulting in increased adrenergic activity and anxiety, while propranolol blocks the β-receptor, leading to decreased adrenergic activity and anxiolysis. In light of the role of the adrenergic system in anxiety disorders (Bremner, Krystal, Southwick, & Charney, 1996) and the strong evidence for the role of norepinephrine in PTSD (Southwick et al., 1999), manipulations of this neurotransmitter system have important implications for the extinction of conditioned fear. Results from Cain et al. indicated that animals treated with yohimbine showed enhanced fear extinction during the test phase; in contrast, animals treated with propranolol exhibited impaired extinction. These findings again are consistent with the hypothesis that fear activation during exposure is associated with better treatment outcomes. In addition, these results provide compelling evidence that state-dependent learning per se cannot account for the observed effects of activation on fear reduction, given that neither the propranolol- nor the yohimbine-treated mice were in the same physiological state during the test phase as during the extinction learning phase. Morris and Bouton (2007) also performed multiple experiments to test the effect of yohimbine on conditioned fear extinction, measuring


31

freezing response during extinction training under yohimbine versus saline administration. For the purpose of this discussion, the amount of freezing is taken to indicate level of fear activation during extinction training. Results indicated that a 1.0 mg/kg dose of yohimbine was associated with less fear activation during extinction, as well as less fear responding during the test phase. Given that yohimbine produces fearlike symptoms, the finding that the yohimbine group showed less fear during the test phase provides additional evidence that pharmacologically induced activation is associated with greater reductions in conditioned fear. However, because the yohimbine group did not show more fear response during the extinction learning phase than did the control group, this study does not present evidence that it was the greater initial fear activation which was responsible for the greater extinction learning. The activation hypothesis of emotional processing theory appears to be supported not only by extinction of conditioned fear but also of conditioned appetitive responses. In an elegant series of experiments, Rescorla (2000) paired various CSs with the delivery of food pellets. Animals that showed the greatest response during the extinction phase also showed the most profound degree of extinction during the test phase. These animals had been presented with compound CSs (noise plus light) during the extinction phase, which produced the greatest mean response during this phase; when presented subsequently with a single extinguished CS (noise or light) at test, the rats in this condition exhibited the lowest level of conditioned responses. These results were replicated and extended to a fear conditioning paradigm (Rescorla, 2006), further confirming the importance of activation in extinction learning. Taken together, results from extinction studies demonstrate that greater activation during extinction is associated with greater decrements in the conditioned behavior. It follows that without activation of the relevant representations, extinction is unlikely to occur; this seems true for both fear and appetitive conditioning. Given the importance of activation, optimal extinction depends on the identification of factors that promote

2/15/2011 7:42:11 PM

32


activation. On the basis of Rescorla’s findings (2000, 2006), the presentation of multiple CSs that are associated with the US are most likely to produce enhanced activation and thus greater subsequent extinction of the CS. Implications of this observation for exposure therapy will be discussed later. Comparatively little research has been conducted to test for the effects of pharmacologically manipulated activation on fear extinction in humans. Contrary to the findings for animals in which the β-blocker propranolol was associated with impaired fear extinction (Cain et al., 2004), Orr et al. (2006) found no significant effect of propranolol on fear extinction among a group of males with PTSD. Conclusive evidence for the role of pharmacological agents in augmenting activation and thereby enhancing extinction awaits future research. Of note, a process complementary to extinction also demonstrates that alteration of fear memory requires activation. Several studies have shown that learned fear can be disrupted by blocking the reconsolidation of CS-US associations. In a typical experiment in this area, animals that had learned a CS-US pairing were presented with the CS alone some time after fear training. The retrieval of the fear memory in response to the CS presentation temporarily renders the memory labile and sensitive to disruption; evidence for this phenomenon comes from studies showing that interruption of the processes that are required for reconsolidation of the retrieved memory leads to a decrease in fear responding to the CS (e.g., Lee, Milton, & Everitt, 2006; Nader, Schafe, & LeDoux, 2000). Importantly, activation of the learned fear association through presentation of the CS must precede administration of the chemical agents that interrupt reconsolidation; without activation of the fear, there is no effect on the CS-US association (Nader et al., 2000). These findings provide further evidence for the critical role of fear activation in altering learned fear associations. Neural Studies

In addition to the behavioral data summarized earlier, many studies examining neural function have demonstrated that the extinction of


conditioned fear is associated with the activation of brain structures that are involved in the representation of fear. The most widely studied brain structure in this context is the amygdala. The amygdala is an almond-shaped mass that is located in the limbic system; it plays a key role in the processing of emotional information, including emotional memory (Phelps, 2005) and making judgments about emotional facial expressions in humans (for a review see Sergerie, Chochol, & Armony, 2008). Importantly, the amygdala has been implicated in the acquisition of fear conditioning, both in animals (LeDoux, 2000) and in humans (LaBar, Gatenby, Gore, LeDoux, & Phelps, 1998). Of direct relevance to the current chapter, patients with PTSD tend to show hyperactivity of the amygdala (e.g., Rauch et al., 2000; Shin et al., 2004), indicating the chronically heightened fear and arousal experienced by these individuals. Numerous experimental paradigms have demonstrated the involvement of the amygdala in fear extinction. In animals, the blockade of N-methyl-D-aspartate (NMDA) receptors by injection of NMDA antagonists, either systemically (Baker & Azorlosa, 1996) or into the amygdala, leads to poor extinction learning (Falls, Miserendino, & Davis, 1992; SotresBayon, Bush, & LeDoux, 2007). Conversely, infusion of the NMDA agonist D-cycloserine (DCS) either systemically or directly into the amygdala enhances the extinction of conditioned fear (Walker, Ressler, Lu, & Davis, 2002). The involvement of the amygdala in fear extinction has been demonstrated through tests of other neurotransmitter receptor systems, including the cannabinoid receptor type 1 (Chhatwal, Davis, Maguschak, & Ressler, 2005) and L-type voltage-gated calcium channels (e.g., Cain, Blouin, & Barad, 2002). In addition, fear extinction is associated with depotentiation of excitatory thalamic neurons that terminate at synapses on the amygdala (Kim et al., 2007). These studies indicated that amygdala activity is recruited during the extinction of conditioned fear. However, they do not tell us definitively whether the amygdala activity represents the activation of the existing fear structure, the modification of the existing fear structure, or the

2/15/2011 7:42:11 PM


creation of an inhibitory extinction representation. A recent study by Herry et al. (2008) provides striking evidence that the basal amygdala is involved in both conditioned fear and its extinction. This research demonstrated that conditioned fear leads to the activation of “fear neurons” in response to the presentation of a CS. When the CS is presented in the absence of the US, “extinction neurons” increase their firing rate; fear neurons subsequently reduce their firing rate. Importantly, these neuronal changes preceded changes in freezing behavior. Crucially, the fear neurons continue to fire at the beginning of extinction (see Herry et al., Fig. 3c), with extinction neurons starting to fire just before fear neurons switch off. These results demonstrated that the neurons representing fear are active during the extinction of fear. Findings from the literature on fear memory reconsolidation also demonstrate the essential role of neurons in the amygdala in the modification of learned fear associations. For example, reconsolidation can be inhibited by blocking protein synthesis in the amygdala (Nader et al., 2000); blocking NMDA receptors in the amygdala (Lee et al., 2006); blocking noradrenergic transmission in the amygdala (Debiec & LeDoux, 2004); the infusion of cannabinoid CB1 receptor agonists (Lin, Mao, & Gean, 2006); and by genetic disruption of transcription in the amygdala (Mamiya et al., 2009). Again, these effects hold only when presentation of the CS precedes administration of the chemical agents, that is, after activation of the fear memory. Thus, there is ample evidence that amygdala activation underlies the activation of fear memories and subsequent modification of these memories. Human neuroimaging studies provide an essential test for the relevance of the animal findings for patients (Rauch, Shin, & Phelps, 2006). A related line of research has shown the involvement of the amygdala in human fear extinction. LaBar et al. (1998) found significant amygdala activation during the early phase of fear extinction. Later studies replicated and extended these findings. Amygdala activation was found when the association between the US and the CS was altered, both during fear acquisition and extinction (Knight, Smith, Cheng, Stein, & Helmstetter,


33

2004; Milad et al., 2007); moreover, the degree of fear extinction was strongly correlated with amount of amygdala activation (Phelps, Delgado, Nearing, & LeDoux, 2004). These data are consistent with Foa and Kozak’s (1986) emotional processing theory, which states that activation of the fear structure is necessary for its modification and the subsequent alleviation of pathological fear levels present in anxiety disorders, including PTSD. If one can view exposure therapy as analogous to extinction and the reduction in different measures of PTSD as indicators of extinction, then there are several clinical studies that also lend support to the activation hypothesis (e.g., Foa, Riggs, Massie, & Yarczower, 1995; Kozak, Foa, & Steketee, 1988; Lang, Melamed, & Hart, 1970; Pitman et al., 1996). These studies assessed level of fear activation using different measures of fear (e.g., facial fear expression, heart rate, subjective ratings), thus demonstrating the robust effect of activation on extinction. The data from the fear extinction literature in animals and humans presented earlier provide a window into the mechanisms that underlie fear reduction and suggest a line of research that will combine fear extinction paradigms and studies on exposure therapy. Evaluation of Prediction 2: Within-Session Fear Reduction

According to Foa and Kozak (1986), the second indicator of emotional processing is withinsession fear reduction. It can be argued that within-session habituation would enhance extinction in the test phase. A small number of studies have tested this prediction, with mixed results. One approach that researchers have used is to compare the effects of massed versus spaced presentation of the CS during extinction training on fear responding during extinction and later during recall of extinction—that is, during presentation of the CS without the US some time after the extinction learning. Studies have demonstrated that massed presentations produce greater within-session fear extinction; at test 1 week later, massed CS presentations were associated with lower fear responding when animals were again presented with the CS alone

2/15/2011 7:42:11 PM

34


(Cain, Blouin, & Barad, 2003). This finding is consistent with the emotional processing theory prediction. Two similar studies that examined appetitive extinction produced inconsistent results. Rescorla and Durlach (1987) obtained similar results to those of Cain et al. (2003), finding that extinction in pigeons was more durable when extinction trials were massed (intertrial interval of 10 sec) versus spaced (2 min). In contrast, Moody, Sunsay, and Bouton (2006) found no significant effect of 60- versus 240-sec intertrial intervals; although the shorter interval produced greater within-session extinction, there was no significant effect of interval at the test phase. Other studies demonstrated that the relationship between within-session fear reduction and extinction of conditioned fear during the test phase is more complicated. In a context conditioning paradigm, Li and Westbrook (2008) found that although spaced extinction trials produced less within-session fear reduction than massed extinction trials, the spaced trials resulted in greater fear extinction during the test phase. However, this study differed from that of Cain et al. (2003) in several important ways. First, Cain et al. used presentation of a discrete CS, whereas Li and Westbrook studied context conditioning, with each stint in the conditioning context representing a “session” of fear extinction learning. This distinction renders the Li and Westbrook design a test of between-session fear reduction, given that the animal was removed from the conditioned context for varying intervals between trials. Second, Cain et al.’s intertrial intervals ranged from 6 sec to 6 min, whereas those of Li and Westbrook were 4 min versus 24 hr. Therefore, the study by Li and Westbrook appears to apply to the spacing of exposure sessions rather than to the reduction of fear within a session. In summary, Foa and Kozak’s (1986) view that within-session habituation is an indicator of emotional processing has not received as strong support from studies of fear extinction as did their view of fear activation as an indicator of, as well as a necessary condition for, emotional processing. Similarly, exposure therapy studies with human beings also do not lend strong


support to the role of within-session habituation in promoting emotional processing (e.g., Kozak et al., 1988; Riley et al., 1995; van Minnen & Foa, 2006). Thus, although patients often experience within-session habituation during exposure sessions, the degree of within-session habituation is not a consistent predictor of outcome. Evaluation of Prediction 3: Between-Session Fear Reduction Different Brain Mechanisms Involved in Within- and Between-Session Habituation

The third indicator of emotional processing in Foa and Kozak’s (1986) theory is betweensession habituation of fear. Emotional processing theory proposes that although betweensession fear reduction is related to within-session fear reduction, the two involve separate processes. In extinction paradigms, between-session fear reduction can be viewed as analogous to the retention of fear extinction between the extinction phase and the subsequent test phase, as distinct from the initial acquisition of fear extinction learning, which is akin to within-session fear reduction. Foa (1979) posited that different brain mechanisms are involved in within- and betweensession habituation in exposure therapy. Building on findings by Groves and Lynch (1972), she argued that a low-level brain structure (brain stem reticular formation) is involved in within-session habituation while between-session habituation relies on forebrain structures. Foa further argued that if forebrain structures are necessary for retention of habituation (extinction) across time, then higher level cognitive processes seem to be necessary for consolidation of extinction learning. Support for the view that two distinct learning processes are involved in short- and longterm extinction comes from studies of evaluative conditioning. In these studies there is a distinction between expectancy learning and evaluative learning; in the former, the organism learns that a certain event (stimulus) predicts another event, while in the latter the organism assigns a negative or positive valence to the CS (see Chapter 18; for a review see De Houwer, Thomas, & Baeyens, 2001). For example, an animal will

2/15/2011 7:42:11 PM


learn that a red light paired with a subsequent electric shock predicts the onset of the shock (expectancy learning); assigning a negative valence to the red light itself represents evaluative learning. Interestingly, Vansteenwegen et al. (2006) note that evaluative learning is more resistant to extinction than is expectancy learning. Accordingly, a subject may stop producing an electrodermal response following extinction of a previously conditioned stimulus that predicted electric shock but still will have a negative evaluation of the conditioned stimulus (e.g., Hermans, Vansteenwegen, Crombez, Baeyesn, & Eelen, 2002; Vansteenwegen, Francken, Vervliet, De Clercq, & Eelen, 2006). It is reasonable to assume that evaluative conditioning involves higher level cognitive operations and that those operations are more resistant to extinction, given the involvement of higher order brain areas (especially the prefrontal cortex; Zysset, Huber, Ferstl, & von Cramnon, 2002) in evaluative judgments. Indeed, there is evidence that evaluative conditioning is intact even in the absence of the amygdala (Coppens et al., 2006). These observations may suggest that extinction of expectancy learning is more similar to within-session extinction and that extinction of evaluative learning is more akin to between-session extinction. The distinctions between expectancy learning and evaluative learning may have important implications for exposure therapy (see Discussion). Taken together with Groves and Lynch (1972), the results from evaluative conditioning studies reinforce the view that separate brain areas and cognitive processes underlie within- and between-session habituation. Brain Mechanisms Underlying Retention of Fear Extinction

A large body of studies has confirmed that distinct neural regions underlie the retention of fear extinction versus the acquisition of fear extinction. The vast majority of these studies have implicated the medial prefrontal cortex (mPFC) in the retention of extinguished fear in both animals and humans. In animals, many experimental paradigms have been used to examine the role of mPFC in extinction retention (for a review,


35

see Quirk, Garcia, & González-Lima, 2006). In lesion studies, the destruction of mPFC structures is associated with normal within-session extinction, but the animals do not retain the extinction learning 24 hours later at test (e.g., Lebrón, Milad, & Quirk, 2004; Quirk, Russo, Barron, & Lebron, 2000). Disruption of the targets of inhibitory mPFC projections within the amygdala also has been shown to interfere with extinction recall (Chhatwal, Stanek-Rattiner, Davis, & Ressler, 2006; Likhtik, Popa, AspergisSchoute, Fidacaro, & Paré, 2008). Additional research has provided more precise estimates of the temporal course of mPFC involvement in extinction retention. Reversible pharmacological disruption of mPFC activity (by injection of PD098059, a mitogen-activated protein kinase inhibitor) immediately after fear extinction learning also produced deficient extinction retention during the test phase (Hugues, Deschaux, & Garcia, 2004). Similar results were obtained via the infusion of an NMDA receptor antagonist immediately after extinction training (Burgos-Robles, Vidal-Gonzalez, Santini, & Quirk, 2007). Both of these studies found no significant deficit in extinction retention at test if the chemical agents were applied 2 hours after extinction training, suggesting that mPFC activity during this period is critical for extinction retention. Complementary paradigms have produced corroborating results; rather than examining the loss of function associated with destroying mPFC, Milad and Quirk (2002) measured neuronal activity in intact mPFC during fear conditioning, extinction training, and recall of extinction at test. They found that neurons in mPFC showed increased firing rates only during the 24-hour recall of extinguished fear conditioning, again supporting the involvement of this region in extinction retention. Similar studies in humans have corroborated the results from animal studies showing that the mPFC is involved in extinction retention. Kalisch et al. (2006) paired presentation of faces (CS) with electric shock during the conditioning phase and then extinguished the CR by presenting the faces without shock. The following day during the test phase, presentation of the

2/15/2011 7:42:11 PM

36


extinguished CS produced significant activation of mPFC. Milad et al. (2007) replicated these results using a slightly different paradigm, pairing electric shock with different visual contexts. In addition to finding significant mPFC activations during recall of extinction, their results showed that the degree of extinction retention at test was positively correlated with percent signal change in mPFC. Phelps et al. (2004) also found that mPFC activity increased during the recall of extinction, and they reported a positive correlation between mPFC activity during extinction recall and prior success of extinction; that is, subjects who showed greater extinction during extinction training also had greater activation of mPFC during the recall of extinction at test. In addition to the functional associations between mPFC and extinction retention, structural variability within this region was found to be correlated with recall of extinction. Milad et al. (2005) found a positive correlation between thickness of mPFC and extinction retention. Retention of extinction learning has been consistently shown to be context dependent; that is, a CS that is extinguished in one context may provoke the CR when presented in a novel context at test (e.g., Bouton & Bolles, 1979). The hippocampus appears to play a crucial role in evaluating context in recall of extinction learning. Indeed, disruption of the hippocampus following fear extinction prevented the return of conditioned fear responses in a novel context (Corcoran & Maren, 2001), suggesting that hippocampal input is necessary for context sensitivity in extinction recall. Furthermore, the hippocampus has been shown to play a similar role in the contextual modulation of human extinction recall during the test phase (Kalisch et al., 2006; Milad et al., 2007). That is, if conditioning occurs in one context and extinction in another context, then the CR ordinarily would return if the subjects are tested in a novel context (Bouton & Bolles, 1979); such return of the CR does not occur in those with hippocampal lesions. Convergence Between Experimental Laboratory and Clinical Treatment Findings

In summary, it seems that between-session habituation (i.e., extinction retention) relies on


higher cognitive processes more than does within-session habituation (extinction learning), given the higher level cortical areas that support between-session habituation. Supporting this conclusion are the findings that different brain structures are involved in each type of habituation, suggesting a distinction between extinction learning and extinction retention at test. The laboratory findings that extinction retention and extinction learning comprise different processes may explain why between-session habituation, which is akin to extinction retention, is related to treatment outcome (e.g., Kozak et al., 1988; Lang et al., 1970; Rauch et al., 2004), whereas within-session habituation generally is not. Moreover, the findings summarized earlier, suggesting the involvement of higher cognitive functioning in extinction retention, may further explain why between-session habituation is crucial for recovery from pathological anxiety. After all, to recover, patients need to remember the outcome of their previous experiences— namely, that their feared consequences were disconfirmed—and such memory relies on higher cognitive functioning.

DISCUSSION Implications of Fear Extinction Literature for Emotional Processing Theory

In this chapter our goal was to reexamine and extend emotional processing theory by considering three core tenets of the theory—activation, within-session habitation, and between-session habituation—in light of experimental extinction paradigms. Specifically, we have attempted to examine the body of information about extinction learning and retention and the related brain structures and functions in light of emotional processing theory with the goal of furthering our understanding of the mechanisms that support the reduction of pathological fear. Our review indicates that there is strong evidence from the extinction as well as from the treatment literature that initial fear activation is important in achieving extinction of conditioned fear and in reducing pathological anxiety after exposure therapy. These findings support the emotional

2/15/2011 7:42:11 PM


activation hypothesis of emotional processing theory. Behavioral, neural, and pharmacological data converge to indicate that the fear structure must be activated in order for the conditioned fear to be extinguished. These consistent findings from diverse areas will serve as a strong foundation for further research that will help us understand the basic mechanisms underlying fear extinction and exposure therapy. There have been fewer studies in the extinction literature that can shed light on the second hypothesis of emotional processing theory, that within-session fear extinction is related to outcome. This tenet of the theory has accrued less support in the clinical literature (for a review, see Craske et al., 2008), suggesting that withinsession fear reduction may not be required for fear to be extinguished or for successful exposure treatment to reduce pathological anxiety. However, the relative paucity of data argues for further examination of this hypothesis before a final verdict is rendered. There is considerable support from the animal literature on extinction learning for the supposition of emotional processing theory that between-session habitation is strongly related to outcome of exposure therapy and therefore is a valid indicator of emotional processing. Of particular interest are the findings that activity in specific neural regions, namely the mPFC, is necessary for the maintenance of extinction learning between extinction training and subsequent test sessions. These data underscore the importance of between-session fear extinction and support the third tenet of emotional processing theory. The converging evidence from the clinical literature for the importance of between-session habituation on treatment outcome (e.g., Rauch et al., 2004) reinforces the importance of fear reduction across sessions as an indicator of emotional processing. Implications of Emotional Processing Theory for Extinction Research

As we noted in the Introduction, emotional processing theory and extinction paradigms in animal research may be mutually enriching in understanding the mechanisms involved in fear


37

reduction and in the treatment of pathological anxiety. The literature reviewed in this chapter provides not only a test of the predictions put forward by emotional processing theory but also provides future directions for basic fear extinction research that may more directly inform the theory and treatment of anxiety. Few of the animal extinction studies reviewed herein were designed explicitly to test the tenets of emotional processing theory; more research in this area may further solidify the empirical basis of exposure treatment. Although the role of fear activation in extinction learning has been studied extensively and its importance is clearly supported by existing data, fewer studies have investigated the role of withinsession fear reduction on subsequent fear extinction. Similarly, although many studies have demonstrated that between-session fear reduction is a distinct process that appears to be related to extinction learning, few behavioral studies have tested the prediction that greater fear extinction across sessions is correlated with greater fear reduction at the test phase. Future studies designed specifically to bridge basic and clinical science must address issues that will help therapists modify exposure treatment in order to increase its efficacy, such as optimal level of initial fear activation and the degree of withinsession fear reduction that is necessary for successful outcome. Treatment Implications

Findings from the fear extinction literature may have important implications for the practice of exposure therapy. As Foa and Kozak (1997) noted, the initial optimism that followed the advent of the first behavioral therapies for anxiety gave way to the recognition that many patients fail to benefit from the treatments, and that a significant percentage of those that benefit fail to maintain their gains. The authors argued that the apparent “efficacy ceiling”—the leveling off of response rates to empirically tested cognitive and behavioral treatments—may be due to “an alienation from psychopathology and experimental psychology” (p. 606). They suggest that therapy researchers have not exploited findings

2/15/2011 7:42:11 PM

38


from basic research in psychopathology. In their view, greater communication between basic clinical scientists and therapy researchers has the potential to produce more powerful and more broadly applicable clinical treatments. Clinical application of the principles reviewed here in the context of emotional processing theory has the potential to raise the efficacy ceiling on the treatment of anxiety disorders. For example, the results from Cain et al. (2004), who used pharmacologic agents to enhance the extinction of conditioned fear, suggest that exposure therapy may be delivered effectively across fewer sessions if activating agents are used to enhance the inhibitory learning processes. Existing research has shown that activating agents such as D-cycloserine enhance the effects of exposure therapy for anxiety disorders (e.g., Ressler et al., 2004). Some research laboratories currently are investigating the ability of agents such as yohimbine to enhance the efficacy of exposure treatment for PTSD. If positive results are obtained, patients with PTSD may experience relief more quickly. Of particular importance is the possibility that people who might otherwise drop out before experiencing the effectiveness of the treatment might have quicker gains that encourage them to complete the treatment and reach full remittance of symptoms. Nonpharmacologic tools also may render exposure therapy more effective. For example, Powers and Emmelkamp (2008) in their metaanalysis found that virtual reality exposure (VRE) is more effective than in vivo exposure in the treatment of phobias. Virtual reality may increase the level of activation by producing a closer match with the fear structure. Effectiveness of VRE also may be driven by the ability to present multiple CSs (e.g., sounds of warfare; combat-related images) that deepen extinction (see Rescorla, 2000, 2006). In addition, VRE may help patients who are “under-engagers”; as Rothbaum (2009) points out, VR may help “patients who are reluctant to engage in recollections of feared memories or are not very good at imagining situations” (p. 210). Our current understanding of the role of within-session extinction does not provide clear guidance for the application of exposure therapy.


Nevertheless, the finding that massed extinction sessions, which result in greater within-session extinction, are more effective in reducing fear than spaced sessions, suggests the importance of fear reduction within each exposure session. Among PTSD patients treated with exposure therapy, Pitman et al. (1996) found a trend for within-session HR habituation to be correlated with outcome. However, several clinical studies (e.g., Jaycox, Foa, & Morral, 1998; Kozak et al., 1988; van Minnen & Foa, 2006) have not found significant associations between within-session habituation and outcome. Perhaps withinsession habituation is not a necessary condition for positive outcomes in exposure therapy for PTSD; alternatively, the subjective ratings of within-session distress may lack sensitivity for detecting associations between within-session experiences and outcome. Additional animal and neuroimaging studies may help resolve the question of whether within-session changes in emotional experience are important for longterm reduction in pathological fear reactions. The findings from evaluative conditioning may have implications for exposure therapy in that the negative evaluations of trauma-related stimuli such as automobiles (for motor vehicle accident survivors) or men (for rape survivors) likely will persist longer than will the fear responses to such stimuli. That is, while a patient with PTSD may no longer report distress in response to trauma-related stimuli because the expectancy has decreased in strength, the person may continue to avoid items on an exposure hierarchy in response to a persistent negative evaluation of these stimuli. An awareness of this phenomenon on the part of the therapist may be instrumental in helping the patient to overcome factors that encourage avoidance. Finally, the importance of consolidating fear extinction in order to retain the extinction learning (e.g., Hugues et al., 2004) also has important ramifications for exposure treatment. Patients should be encouraged strongly to abstain from activities, such as excessive drinking, that may interfere with memory consolidation processes, particularly in the immediate aftermath of an exposure session. Although abstaining may be more difficult in the short term, especially for

2/15/2011 7:42:11 PM


patients who rely on chemical substances to numb their fear and anxiety, in the long run the patients will maintain their extinction learning and will experience more rapid relief from their anxiety.

NOTE 1. In this chapter we generally use the term extinction in the context of fear conditioning studies because it is clear what CS-US association is being extinguished. We use the less specific term habituation in the context of fear reduction during exposure therapy because in most cases it is unknown how the pathological fear originated.

REFERENCES Alvarez, R. P., Johnson, L., & Grillon, C. (2007). Contextual-specificity of short-delay extinction in humans: Renewal of fear-potentiated startle in a virtual environment. Learning and Memory, 14, 247–253. Baker, J. D., & Azorlosa, J. L. (1996). The NMDA antagonist MK-801 blocks the extinction of Pavlovian fear conditioning. Behavioral Neuroscience, 110, 618–620. Barad, M., Gean, P-W., & Lutz, B. (2006). The role of the amygdala in the extinction of conditioned fear. Biological Psychiatry, 60, 322–328. Barry, H., Etheredge, E. E., & Miller, N. E. (1965). Counterconditioning and extinction of fear fail to transfer from amobarbital to nondrug state. Psychopharmacologia (Berlin), 8, 150–156. Baum, M. (1970). Extinction of avoidance responding through response prevention (flooding). Psychological Bulletin, 74, 276–284. Blechert, J., Michael, T., Vriends, N., Magraf, J., & Wilhelm, F. H. (2007). Fear conditioning in posttraumatic stress disorder: Evidence for delayed extinction of autonomic, experiential, and behavioural responses. Behaviour Research and Therapy, 45, 2019–2033. Bouton, M. E., & Bolles, R. C. (1979). Contextual control of the extinction of conditioned fear. Learning and Motivation, 10, 445–466. Bouton, M. E., Kenney, F. A., & Rosengard, C. (1990). State-dependent fear extinction with two benzodiazepine tranquilizers. Behavioral Neuroscience, 104, 44–55. Bouton, M. E., Mineka, S., & Barlow, D. H. (2001). A modern learning theory perspective on the


39

etiology of panic disorder. Psychological Review, 108, 4–32. Bremner, J. D., Krystal, J. H., Southwick, S. M. & Charney, D. S. (1996). Noradrenergic mechanisms in stress and anxiety: II. Clinical studies. Synapse, 23, 39–51. Breslau, N., Reboussin, B. A., Anthony, J. C., & Storr, C. L. (2005). The structure of posttraumatic stress disorder: Latent class analysis in 2 community samples. Archives of General Psychiatry, 62, 1343–1351. Burgos-Robles, A., Vidal-Gonzalez, I., Santini, E., & Quirk, G. J. (2007). Consolidation of fear extinction required NMDA-dependent bursting in the ventromedial prefrontal cortex. Neuron, 53, 871–880. Cain, C. K., Blouin, A. M., & Barad, M. (2002). L-type voltage-gated calcium channels are required for extinction, but not for acquisition or expression, of conditional fear in mice. Journal of Neuroscience, 22, 9113–9121. Cain, C. K., Blouin, A. M., & Barad, M. (2003). Temporally massed CS presentations generate more fear extinction than spaced presentations. Journal of Experimental Psychology: Animal Behavior Processes, 29, 323–333. Cain, C. K., Blouin, A. M., & Barad, M. (2004). Adrenergic transmission facilitates extinction of conditional fear in mice. Learning and Memory, 11, 179–187. Chhatwal, J. P., Davis, M., Maguschak, K. A., & Ressler, K. J. (2005). Enhancing cannabinoid neurotransmission augments the extinction of conditioned fear. Neuropsychopharmacology, 30, 516–524. Chhatwal, J. P., Stanek-Rattiner, L., Davis, M., & Ressler, K. J. (2006). Amygdala BDNF signaling is required for consolidation but not encoding of extinction. Nature Neuroscience, 9, 870–872. Coppens, E., Vansteenwegen, D., Baeyens, F., Vandenbulcke, M., Van Paesschen, W., & Eelen, P. (2006). Evaluative conditioning is intact after unilateral resection of the anterior temporal lobe in humans. Neuropsychologia, 44, 840–843. Corcoran, K. A., & Maren, S. (2001). Hippocampal inactivation disrupts contextual retrieval of fear memory after extinction. Journal of Neuroscience, 21, 1720–1726. Craske, M. G., Kircanski, K., Zelikowsky, M., Mystkowski, J., Chowdhury, N., & Baker, A. (2008). Optimizing inhibitory learning during exposure therapy. Behaviour Research and Therapy, 46, 5–27.

2/15/2011 7:42:11 PM

40


Davey, G. C. (1992). Classical conditioning and the acquisition of human fears and phobias: A review and synthesis of the literature. Advances in Behaviour Research and Therapy, 14, 29–66. Debiec, J., & LeDoux, J. E. (2004). Disruption of reconsolidation but not consolidation of auditory fear conditioning by noradrenergic blockade in the amygdala. Neuroscience, 129, 267–272. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127, 853–869. Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy. New York, NY: McGraw-Hill. Dunmore, E., Clark, D. M., & Ehlers, A. (1999). Cognitive factors involved in the onset and maintenance of posttraumatic stress disorder (PTSD) after physical or sexual assault. Behaviour Research and Therapy, 37, 809–829. Falls, W. A., Miserendino, M. J., & Davis, M. (1992). Extinction of fear-potentiated startle: Blockade by infusion of an NMDA antagonist into the amygdala. Journal of Neuroscience, 12, 854–863. Foa, E. B. (1979). Failure in treating obsessivecompulsives. Behaviour Research and Therapy, 17, 169–176. Foa, E. B., Huppert, J. D., & Cahill, S. P. (2006). Emotional processing theory: An update. In B. O. Rothbaum (Ed.), The nature and treatment of pathological anxiety (pp. 3–24). New York, NY: Guilford Press. Foa, E. B., & Kozak, M. J. (1985). Treatment of anxiety disorders: Implications for psychopathology. In A. H. Tuma & J. D. Maser (Eds.), Anxiety and the anxiety disorders (pp. 421–452). Hillsdale, N J: Erlbaum. Foa, E. B., & Kozak, M. J. (1986). Emotional processing of fear: Exposure to corrective information. Psychological Bulletin, 99, 20–35. Foa, E. B., & Kozak, M. J. (1997). Beyond the efficacy ceiling? Cognitive behavioral therapy in search of theory. Behavior Therapy, 28, 601–611. Foa, E. B., & McNally, R. J. (1996). Mechanisms of change in exposure therapy. In M. Rapee (Ed.), Current controversies in the anxiety disorders (pp. 329–343). New York, NY: The Guilford Press. Foa, E. B., Riggs, D. S., Massie, E. D., & Yarczower, M. (1995). The impact of fear activation and anger on the efficacy of exposure treatment for posttraumatic stress disorder. Behavior Therapy, 26, 487–499.


Foa, E. B., Zinbarg, R., & Rothbaum, B. O. (1992). Uncontrollability and unpredictability in posttraumatic stress disorder: An animal model. Psychological Bulletin, 112, 218–238. Groves, P. M., & Lynch, G. S. (1972). Mechanisms of habituation in the brain stem. Psychological Review, 79, 237–244. Hermans, D., Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (2002). Expectancylearning and evaluative learning in human classical conditioning: Affective priming as an indirect and unobtrusive measure of conditioned stimulus valence. Behaviour Research and Therapy, 40, 217–234. Herry, C., Ciocchi, S., Senn, V., Demmou, L., Müller, C., & Lüthi, A. (2008). Switching on and off fear by distinct neuronal circuits. Nature, 454, 600–606. Hugues, S., Deschaux, O., & Garcia, R. (2004). Postextinction infusion of a mitogen-activated protein kinase inhibitor into the medial prefrontal cortex impairs memory of the extinction of conditioned fear. Learning and Memory, 11, 540–543. Jaycox, L. H., Foa, E. B., & Morral, A. R. (1998). Influence of emotional engagement and habituation on exposure therapy for PTSD. Journal of Consulting and Clinical Psychology, 66, 185–192. Kalisch, R., Korenfeld, E., Stephan, K. E., Weiskopf, N., Seymour, B., & Dolan, R. J. (2006). Contextdependent human extinction memory is mediated by a ventromedial prefrontal and hippocampal network. Journal of Neuroscience, 26, 9503–9511. Kim, J., Lee, S., Park, K., Hong, I., Song, B., Son, G.,. . .Choi, S. (2007). Amygdala depotentiation and fear extinction. Proceedings of the National Academy of Sciences, 104, 20955–20960. Knight, D. C., Smith, C. N., Cheng, D. T., Stein, E. A., & Helmstetter, F. J. (2004). Amygdala and hippocampal activity during acquisition and extinction of human fear conditioning. Cognitive, Affective, and Behavioral Neuroscience, 4, 317–325. Kozak, M. J., Foa, E. B., & Steketee, G. (1988). Process and outcome of exposure treatment with obsessive-compulsives: Psychophysiological indicators of emotional processing. Behavior Therapy, 19, 157–169. LaBar, K. S., Gatenby, C., Gore, J. C., LeDoux, J. E., & Phelps, E. A. (1998). Human amygdala activation during conditioned fear acquisition and extinction: A mixed-trial fMRI study. Neuron, 20, 937–945.

2/15/2011 7:42:11 PM


Lang, P. J. (1977). Imagery in therapy: An information processing analysis of fear. Behavior Therapy, 8, 862–886. Lang, P. J. (1984). Cognition in emotion: Concept and action. In C. Izard, J. Kagan, & R. Zajonc (Eds.), Emotion, cognition and behavior (pp. 193–206). New York, NY: Cambridge University Press. Lang, P. J., Melamed, B. G., & Hart, J. (1970). A psychophysiological analysis of fear modification using an automated desensitization procedure. Journal of Abnormal Psychology, 76, 220–234. Lebrón, K., Milad, M. R., & Quirk, G. J. (2004). Delayed recall of fear extinction in rats with lesions of ventral medial prefrontal cortex. Learning and Memory, 11, 544–548. LeDoux, J. E. (2000). Emotion circuits in the brain. Annual Review of Neuroscience, 12, 155–184. Lee, J. L. C., Milton, A. L., & Everitt, B. J. (2006). Reconsolidation and extinction of conditioned fear: Inhibition and potentiation. Journal of Neuroscience, 26, 10051–10056. Leung, H. T., & Westbrook, R. F. (2008). Spontaneous recovery of extinguished fear responses deepens their extinction: A role for error-correction mechanisms. Journal of Experimental Psychology: Animal Behavior Processes, 34, 461–474. Li, S. H., & Westbrook, R. F. (2008). Massed extinction trials produce better short-term but worse long-term loss of context conditioned fear responses than spaced trials. Journal of Experimental Psychology: Animal Behavior Processes, 34, 336–351. Likhtik, E., Popa, D., Apergis-Schoute, J., Fidacaro, G. A., & Paré, D. (2008). Amygdala intercalated neurons are required for expression of fear extinction. Nature, 454, 642–645. Lin, H-C., Mao, S-C., & Gean, P-W. (2006). Effects of intra-amygdala infusion of CB1 receptor agnoists on the reconsolidation of fear-potentiated startle. Learning and Memory, 13, 316–321. Lissek, S., Powers, A. S., McClure, E. B., Phelps, E. A., Woldehawariat, G., Grillon, C., & Pine, D. S. (2005). Classical fear conditioning in the anxiety disorders: A meta-analysis. Behaviour Research and Therapy, 43, 1391–1424. Mamiya, N., Fukushima, H., Suzuki, A., Matsuyama, Z., Homma, S., Frankland, P. W., & Kida, S. (2009). Brain region-specific gene expression activation required for reconsolidation and extinction of contextual fear memory. Journal of Neuroscience, 29, 402–413.


41

Milad, M. R., Quinn, B. T., Pitman, R. K., Orr, S. P., Fischl, B., & Rauch, S. L. (2005). Thickness of ventromedial prefrontal cortex in humans is correlated with extinction memory. Proceedings of the National Academy of Sciences, 102, 10706–10711. Milad, M. R., & Quirk, G. J. (2002). Neurons in medial prefrontal cortex signal memory for fear extinction. Nature, 420, 70–74. Milad, M. R., Wright, C. I., Orr, S. P., Pitman, R. K., Quirk, G. J., & Rauch, S. L. (2007). Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert. Biological Psychiatry, 62, 446–454. Moody, E. W., Sunsay, C., & Bouton, M. E. (2006). Priming and trial spacing in extinction: Effects on extinction performance, spontaneous recovery, and reinstatement in appetitive conditioning. Quarterly Journal of Experimental Psychology, 59, 809–829. Morris, R. W., & Bouton, M. E. (2007). The effect of yohimbine on the extinction of conditioned fear: A role for context. Behavioral Neuroscience, 121, 501–514. Nader, K., Schafe, G. E., & LeDoux, J. E. (2000). Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature, 406, 722–726. Orr, S. P., Milad, M. R., Metzger, L. J., Lasko, N. B., Gilbertson, M. W., & Pitman, R. K. (2006). Effects of beta blockade, PTSD diagnosis, and explicit threat on the extinction and retention of an aversively conditioned response. Biological Psychiatry, 73, 262–271. Overton, D. A. (1985). Contextual stimulus effects of drugs and internal states. In P. D. Balsam & A. Tomie (Eds.), Context and Learning (pp. 357–384). Hillsdale, NJ: Lawrence Erlbaum. Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex (G. V. Anrep, Trans.). London, England: Oxford University Press. Peri, T., Ben-Shakhar, G., Orr, S. P., & Shalev, A. Y. (2000). Psychophysiologic assessment of aversive conditioning in posttraumatic stress disorder. Biological Psychiatry, 47, 512–519. Phelps, E. A. (2005). Emotion and cognition: Insights from studies of the human amygdala. Annual Review of Psychology, 57, 27–53. Phelps, E. A., Delgado, M. R., Nearing, K. I., & LeDoux, J. E. (2004). Extinction learning in humans: Role of the amygdala and vmPFC. Neuron, 43, 897–905.

2/15/2011 7:42:11 PM

42


Pitman, R. K., Orr, S. P., Altman, B., Longpre, R. E., Poiré, R. E., Macklin, M. L.,. . . Steketee, G. S. (1996). Emotional processing and outcome of imaginal flooding therapy in Vietnam veterans with chronic post-traumatic stress disorder. Comprehensive Psychiatry, 37, 409–418. Powers, M. B., & Emmelkamp, P. M. G. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. Journal of Anxiety Disorders, 22, 561–569. Quirk, G. J., Garcia, R., & González-Lima, F. (2006). Prefrontal mechanisms in extinction of conditioned fear. Biological Psychiatry, 60, 337–343. Quirk, G. J., Russo, G. K., Barron, J. L., & Lebron, K. (2000). The role of ventromedial prefrontal cortex in the recovery of extinguished fear. Journal of Neuroscience, 20, 6225–6231. Rauch, S. A. M., Foa, E. B., Furr, J. M., & Filip, J. C. (2004). Imagery vividness and perceived anxious arousal in prolonged exposure treatment for PTSD. Journal of Traumatic Stress, 17, 461–465. Rauch, S. L., Shin, L. M., & Phelps, E. A. (2006). Neurocircuitry models of posttraumatic stress disorder and extinction: Human neuroimaging research—past, present, and future. Biological Psychiatry, 60, 376–382. Rauch, S. L., Whalen, P. J., Shin, L. M., McInerney, S. C., Macklin, M. L., Lasko, N. B.,. . . Pitman, R. K. (2000). Exaggerated amygdala response to masked facial stimuli in posttraumatic stress disorder: A functional MRI study. Biological Psychiatry, 47, 769–776. Rescorla, R. A. (2000). Extinction can be enhanced by a concurrent excitor. Journal of Experimental Psychology: Animal Behavior Processes, 26, 251–260. Rescorla, R. A. (2006). Deepened extinction from compound stimulus presentation. Journal of Experimental Psychology: Animal Behavior Processes, 32, 135–144. Rescorla, R. A., & Durlach, P. J. (1987). The role of context in intertrial interval effects in autoshaping. Quarterly Journal of Experimental Psychology, 39B, 35–40. Rescorla, R. A., & Heth, C. D. (1975). Reinstatement of fear to an extinguished conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 1, 88–96. Resnick, H. S., Kilpatrick, D. G., Dansky, B. S., Saunders, B. E., & Best, C. L. (1993). Prevalence of civilian trauma and posttraumatic stress disorder in a representative national sample


of women. Journal of Consulting and Clinical Psychology, 61, 984–991. Ressler, K. J., Rothbaum, B. O., Tannenbaum, L., Anderson, P., Graap, K., Zimand, E.,. . .Davis, M. (2004). Cognitive enhancers as adjuncts to psychotherapy: Use of D-cycloserine in phobic individuals to facilitate extinction of fear. Archives of General Psychiatry, 61, 1136–1144. Riley, W. T., McCormick, M. G. F., Simon, E. M., Stack, K., Pushkin, Y., Overstreet, M. M.,. . . Magakian, C. (1995). Effects of alprazolam dose on the induction and habituation processes during behavioral panic induction treatment. Journal of Anxiety Disorders, 9, 217–227. Rothbaum, B. O. (2009). Using virtual reality to help our patients in the real world. Depression and Anxiety, 26, 209–211. Rothbaum, B. O., & Davis, M. (2003). Applying learning principles to the treatment of posttrauma reactions. Annals of the New York Academy of Sciences, 1008, 112–121. Rothbaum, B. O., Foa, E. B., Riggs, D. S., Murdock, T., & Walsh, W. (1992). A prospective examination of post-traumatic stress disorder in rape victims. Journal of Traumatic Stress, 5, 455–475. Sergerie, K., Chochol, C., & Armony, J. L. (2008). The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies. Neuroscience and Biobehavioral Reviews, 32, 811–830. Shin, L. M., Orr, S. P., Carson, M. A., Rauch, S. L., Macklin, M. L., Lasko, N. B.,. . .Pitman, R. K. (2004). Regional cerebral blood flow in the amygdala and medial prefrontal cortex during traumatic imagery in male and female Vietnam veterans with PTSD. Archives of General Psychiatry, 61, 168–176. Sotres-Bayon, F., Bush, D. E., & LeDoux, J. E. (2007). Acquisition of fear extinction requires activation of NR2B-containing NMDA receptors in the lateral amygdala. Neuropsychopharmacology, 32, 1929–1940. Southwick, S. M., Bremner, J. D., Rasmusson, A., Morgan, C. A., Arnsten, A., & Charney, D. S. (1999). Role of norepinephrine in the pathophysiology and treatment of posttraumatic stress disorder. Biological Psychiatry, 46, 1192–1204. Stampfl, T. G., & Levis, D. J. (1967). Essentials of implosive therapy: A learning-theory-based psychodynamic behavioral therapy. Journal of Abnormal Psychology, 72, 496–503.

2/15/2011 7:42:11 PM


Van Minnen, A., & Foa, E. B. (2006). The effect of imaginal exposure length on outcome of treatment for PTSD. Journal of Traumatic Stress, 19, 427–438. Vansteenwegen, D., Francken, G., Vervliet, B., De Clercq, A., & Eelen, P. (2006). Resistance to extinction in evaluative conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 32, 71–79. Walker, D. L., Ressler, K. J., Lu, K-T., & Davis, M. (2002). Facilitation of conditioned fear extinction


43

by systemic administration or intra-amygdala infusions of D-cycloserine as assessed with fearpotentiated startle in rats. Journal of Neuroscience, 22, 2343–2351. Zysset, S., Huber, O., Ferstl, E., & von Cramnon, D. Y. (2002). The anterior frontomedian cortex and evaluative judgment: An fMRI study. Neuroimage, 15, 983–991.

2/15/2011 7:42:11 PM

CHAPTER 3 Fear Conditioning and Attention to Threat An Integrative Approach to Understanding the Etiology of Anxiety Disorders Katherine Oehlberg and Susan Mineka

In this chapter, we review the emerging literatures on conditioning of fear and anxiety and attentional biases for threat. First, we identify constructs common to both phenomena and suggest reasons for an approach that integrates both processes in the study of human anxiety disorders. We also review the current literature on individual differences in fear and anxiety conditioning. Next, we provide a current account of research on attentional biases for threat, which has focused on differentiating the components of attention affected by the threat relevance of stimuli and individual differences in anxiety. Finally, we review the growing number of studies that have combined conditioning and attention paradigms to investigate the relationships between these two phenomena. We conclude by indicating how these approaches to the study of conditioning and attentional processing in human anxiety might be integrated in the future.

RELEVANCE OF CONTEMPORARY LEARNING THEORY AND ATTENTIONAL PROCESSES IN MODELS OF ANXIETY DISORDERS Early theories of associative learning formed the basis for both research on fundamental conditioning processes as well as behavioral models for the acquisition and treatment of several human anxiety disorders (see Mineka & Zinbarg, 1996, 2006, for reviews). However, subsequent research has shifted the focus to cognitive and neurobiological variables that also play a role in both specific phobias and more complex disorders, such as generalized anxiety disorder, panic disorder, and posttraumatic stress disorder (see Chapter 6; also see, e.g., Bouton, Mineka, & Barlow, 2001; Mathews & MacLeod, 2005; Mineka & Zinbarg, 1996, 2006). Although modern research on conditioning processes continues to have theoretical importance and clinical relevance, we believe

that the continued vitality of this tradition (particularly in psychopathology) must consist of integrating conditioning theories with contemporary research on human cognition and its neurobiological underpinnings, with particular attention to individual differences (Mineka & Oehlberg, 2008). In this chapter, we explore the potential that this integrative approach may have by bringing current research findings on anxiety and fear learning to bear on one important cognitive phenomenon associated with anxiety, namely, attentional biases for threat. A great deal of research has indicated that our attentional systems are highly attuned for the rapid identification of environmental threat (e.g., Esteves, Dimberg, & Öhman, 1994). Moreover, individual differences in attentional processing of threatening stimuli—typically referred to as attentional biases for threat-relevant stimuli— have been associated with both trait anxiety, a vulnerability factor for anxiety disorders, as well

44


2/15/2011 7:42:31 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

as a range of clinical anxiety disorders such as specific phobias, generalized anxiety disorder (GAD), posttraumatic stress disorder (PTSD), and social phobias (see Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van Ijzendoorn, 2007; Mathews & MacLeod, 2005, for reviews). Such biases are typically assessed using a behavioral response-time paradigm, in which a participant demonstrates selective attention toward a threatening stimulus (e.g., a threatening word, “disaster,” or an angry face) rather than a neutral stimulus (e.g., “vacuum,” or a neutral face). Several decades of research have now established that these biases appear to be multiply determined and may not be the exclusive purview of highly anxious individuals. Rather, there is evidence that biased attentional processing of threat stimuli may depend upon the intensity of the threat stimuli (Koster, Verschuere, Crombez, & Van Damme, 2005; Mogg et al., 2000; Wilson & MacLeod, 2003). Moreover, it appears that individuals low in trait anxiety tend to direct attention away from mild to moderate threat, but like highly traitanxious individuals direct attention toward highly threatening stimuli (see Bar-Haim et al., 2007; Frewen, Dozois, Joanisse, & Neufeld, 2008, for meta-analyses). A recent computational model of attentional biases for threat versus reward (Frewen et al., 2008), discussed later in this chapter, incorporates this finding as an essential feature of the anxiety-attentional bias link. Classical conditioning is widely considered to be a fundamental mechanism by which initially neutral environmental stimuli acquire the capacity to elicit fear and anxiety, and there is a long tradition of research relating classical conditioning of fear to the anxiety disorders (see Mineka, 1985; Mineka & Zinbarg, 1996, 2006 for reviews). Indeed, conditioning research has formed the basis of behavioral therapies for anxiety disorders, with successful treatment models being based upon extinction processes facilitated by gradual or massed exposures to feared stimuli. Such treatments, particularly with the specific phobias and panic disorder, represent some of the greatest successes of modern clinical psychology (Craske & Mystkowski, 2006).


45

The degree to which a stimulus is perceived as threatening, as well as the subjective experience of anxiety or fear, represent areas of construct overlap between the domains of attentional threat biases and aversive conditioning. It is plausible, for instance, that the perceived threat value of a stimulus might be a direct consequence of basic associative learning processes. Given the significance of perceived threat value in attentional biases, it makes sense to consider such processes in models of the development of attentional biases. Moreover, attention is an important moderator— if not mediator—of conditioning, and has been explicitly included in both traditional (e.g., Pearce & Hall, 1980) and current computational models of conditioning processes (Schmajuk, Lam, & Gray, 1996; Schmajuk & Larrauri, 2006). In addition, both attentional biases and learned classical associations are subject to modification, so their study can elucidate mechanisms by which diathesis-stress interactions may occur. For instance, it has been found that modification of attentional biases has little to no immediate effect on state anxiety, but that it has an effect on the amount of distress that an individual experiences when faced with a stressful event (see MacLeod, Rutherford, Campbell, Ebsworthy, & Holker, 2002; MacLeod, & Bridle, 2009; but also Amir, Beard, Burns, & Bomyea, 2009). The goal of this chapter is to review the emerging literatures in attentional biases for threat and conditioning of fear and anxiety, to identify constructs common to both phenomena (e.g., the stimulus threat value), and to suggest ways in which these approaches to the study of information processing in human anxiety might be integrated in the future. First, we distinguish between states of anxiety and fear, and briefly review recent findings on the neural systems involved in the processing of threatening stimuli and their genetic correlates. We also discuss current directions in fear and anxiety learning that have particular relevance to human anxiety disorders. Second, we discuss findings and ongoing issues in attentional bias research. Third, we critically examine the small but growing literature directly relating attention to threat and conditioning processes. Finally, we suggest some directions for future research.

2/15/2011 7:42:31 PM

46


CLASSICAL CONDITIONING IN THE ANXIETY DISORDERS Starting with Watson and Rayner (1920), Pavlovian or classical conditioning has long been implicated in the origins of fears and phobias (e.g., see also Wolpe, 1968; Mowrer, 1939). Although early conditioning models assumed that traumatic conditioning experiences were both necessary and sufficient for the development of phobic fears and other anxiety disorders, subsequent theorizing has expanded considerably upon this restrictive assumption. As discussed at length elsewhere (Mineka & Zinbarg, 1996; 2006), several important criticisms of these earlier simplistic conditioning views emerged and required careful analysis. Particularly as researchers focused on additional anxiety disorders (such as social phobia, panic disorder, and posttraumatic stress disorder), the importance of other forms of associative learning in the etiology of these disorders became evident (e.g., see Mineka, 1985; Mineka & Zinbarg, 1996). In this section we review highlights of the primary themes of contemporary research and thought on learning theory perspectives on several anxiety disorders. Distinctions Between Fear and Anxiety: Ethological, Clinical, and Neurobiological Evidence

Fear and anxiety, both common in the anxiety disorders, represent partially overlapping but also distinct emotional states. Traditionally, they were distinguished by whether there exists a clear and obvious source of danger that would be regarded as real by most people. Fear was thought to be experienced when the source of danger is obvious, and anxiety was experienced when one could not specify clearly what the danger was. In the past 20–25 years, however, many prominent researchers and theorists have proposed more fundamental distinctions between fear versus anxiety based on a strong and growing body of ethological, clinical, and neurobiological evidence (e.g., Barlow, 2002; Gray & McNaughton, 2000; Grillon, 2008). Although the theories vary in their specifics, there is general agreement on there being at least two negative threat-related


emotional states. For example, Barlow proposed that fear or panic is a basic emotion involving activation of the “fight-or-flight” response of the sympathetic nervous system. By contrast, he characterized anxiety as a relatively diffuse, future oriented state consisting of a complex blend of negative emotions and cognitions. It involves a negative affective state, worry or anxious apprehension about possible future threat or danger, and a sense of being unable to predict or control the future threat. Physiologically anxiety often creates a state of tension, chronic overarousal, and vigilance for future threat, but no activation of the fight-or-flight response (Barlow, 1988, 2002). In this section, we briefly outline several of the ways in which fear and anxiety have been distinguished. Ethological Evidence

From an evolutionary perspective, defensive systems must enable organisms to efficiently and adequately handle a variety of survival threats, including predators. A defensive fear system should motivate the animal to escape or avoid sources of imminent danger with very fast activation of defensive behaviors, but it must also allow an animal to deal with less explicit or more generalized threat cues with less effort or energy. For example, according to one prominent theory the defensive fear system in animals is organized such that different behaviors emerge depending on the animal’s psychological (and/or physical) distance from a “predator”—known as “predatory imminence” (Fanselow & Lester, 1988). That is, each type of defensive behavior has been selected through evolution to prevent movement to the next higher point on the imminence scale. If no threat exists, the animal goes about its usual daily activities. However, if predatory imminence begins to increase, even very slightly, behaviors optimal to the new situation are activated. For instance, when the predatory imminence is low but non-zero for rats, the animal’s eating patterns may change such that the animal shows heightened vigilance, and approach becomes more cautious, likely corresponding to the human affective states of anxiety and worry (Quinn & Fanselow, 2006). If a danger (like a predator) is actually detected but still at some

2/15/2011 7:42:31 PM


distance, the animal engages in freezing behavior while its heart rate slows and respiration becomes shallower; this state seems to correspond to the human emotion of fear. As the threat proximity increases even further and attack is imminent, the animal may jump, flee, or return the attack; this may correspond to human panic attacks. Thus, the specific physiological responses and overt behaviors engaged in at different levels of threat imminence differ not just quantitatively but also qualitatively. In this context, then, anxiety behaviors are prompted by less explicit or more generalized cues, and they involve physiological arousal and increased vigilance but often without organized functional behavior. By contrast, fear behaviors are hypothesized to be optimized for moderately imminent threat, and panic behaviors for immediately imminent threat. Clinical Evidence

Phenomenological evidence as well as psychometric analyses of clinical symptoms also indicate two relatively independent clusters of symptoms—fear and/or panic symptoms on the one hand, and a more general state of worry or anxious apprehension on the other hand (e.g., Barlow, 2002; Bouton et al., 2001; Brown, Chorpita, & Barlow, 1998). For example, structural equation modeling and factor analyses have uncovered two different factors when examining symptoms of panic, anxiety, and depression in clinical populations. One is exemplified by the kinds of apprehension and worry that are characteristic of anxiety and the other is exemplified by a sense of extreme fear or terror, strong autonomic arousal, and fight-or-flight action tendencies that are characteristic of panic or fear. The interrelationships between these two clusters of symptoms are surprisingly complex, and still a topic of active study. For instance, the autonomic arousal symptoms of panic seem to be inhibited in individuals with GAD, suggesting that worry functions to suppress heightened autonomic reactivity in this disorder (e.g., Borkovec, Alcaine, & Behar, 2004; Brown et al., 1998). However, there is also evidence from studies on panic-disordered individuals that panic attacks are actually potentiated when such an individual has a high level of anxiety


47

(see Bouton et al., 2001, for a review). Thus, the effect of elevated anxiety on the occurance of panic appears to vary according to perameters not yet well specified. It is not yet clear how to reconcile such apparently paradoxical findings among the anxiety disorders. Neuroanatomical Evidence

Behavioral neuroscience research in animals also supports the hypothesis of two distinct aversive motivational systems involved in conditioning, although researchers do not agree on all the details. For example, according to Davis and colleagues (e.g., Davis & Shi, 1999; Davis, Walker, & Lee, 1997; Grillon, 2008), fear is a short-term state activated by discrete Pavlovian conditioned stimuli (CSs; see Table 3.1 for a summary of conditioning terminology), whereas anxiety is a longer term state that is activated by more diffuse cues that can be either unconditioned (such as darkness for humans) or conditioned (as in contextual cues associated with threat). These states appear to be related to two distinct neural systems, namely, the amygdala and the bed nucleus of the stria terminalis (BNST), which is immediately downstream from the basolateral amygdala. Although the amygdala mediates fear responses to explicit threatening stimuli, both the amygdala and the BNST are associated with more long-lasting aversive states not clearly linked to explicit threat cues. Both conditioned and unconditioned fear and anxiety states have been modeled in research on animals, and in many cases these experimental paradigms have been extended to research in humans. One of the principal experimental models today is the fear-potentiated startle effect (Davis, 1998, 2006), which is used in both animals and humans. This effect occurs when a greater magnitude startle reflex is elicited by a loud noise occurring 3–4 seconds after an independently established light CS+ (excitatory conditioned stimulus) has been presented, compared to when the light is a novel stimulus. One model of unconditioned anxiety is the light-enhanced startle effect, which occurs when a bright light (a mild unconditioned stimulus, or US, for anxiety in rats that prefer the dark) has been turned on 5–20 minutes prior to a loud noise; this also

2/15/2011 7:42:31 PM

48 Table 3.1

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS Terminology

Construct

Definition

Current Abbreviation

Alternative Abbreviations

Unconditioned stimulus

A biologically significant stimulus or event; in this chapter, the Os of interest are drug related and include both direct effects of drug and the symptoms and signs of drug withdrawal

O

S, S∗, US

Conditioned stimulus

A previously neutral stimulus that obtains significance through association with an unconditioned stimulus

S

S, CS

Conditioned response

A response elicited by a conditioned stimulus associated with a biologically significant stimulus

R

R, CR

Pavlovian/classical conditioning

Learning in which individuals form associations between conditioned stimuli and unconditioned stimuli

S-O

S-S, S-S∗, CS-US

Instrumental/ operant conditioning

Learning in which individuals form associations between behavioral responses and outcomes

R-O

R-S

Habit learning

Learning in which stimuli come to evoke responses automatically even if the reinforcer is devalued

S-R

Incentive learning

Learning in which the value of a reinforcer is associated with certain stimuli or contexts, thus modulating the incentive value of Ss associated with the reinforcer

Occasion setter

A stimulus that signals whether a learned S-O association will hold in Pavlovian conditioning

Positive

Signals that O will follow S

S+

Negative

Signals that O will not follow S

S–

Discriminative stimulus

A stimulus that signals that a response will be reinforced in instrumental conditioning

Positive feature

A discriminative stimulus that indicates that responding will be reinforced

SD

S+

Negative feature

A discriminative stimulus that indicates that responding will not be reinforced

S∆

S–

Conditioned inhibitor

A conditioned stimulus that is associated with the absence of a biologically important stimulus. Unlike occasion setters and discriminative stimuli that are not related directly to the presence or absence of the unconditioned stimulus (O), conditioned inhibitors are directly associated with the absence of the O

S-

CS–


2/15/2011 7:42:31 PM


results in increased magnitude of the startle reflex in rats (Walker & Davis, 1997). It is referred to as an unconditioned anxiety effect, because it does not extinguish as a source of increased startle either within or across multiple test sessions (Walker & Davis, 1997). In humans, a similar increase in startle amplitude can be induced by exposure to the dark (Grillon, Pellowski, Merikangas, & Davis, 1997). What is common to the fear and anxiety paradigms is the dependent variable (an enhanced startle response), but the paradigms reflect unconditioned anxiety when the longer duration light stimulus is used, and conditioned fear when the brief presentation of a light CS+ is used in fearpotentiated startle paradigms. Lesions of the basolateral amygdala substantially reduce or abolish behavioral and autonomic responses to conditioned fear stimuli in a fear-potentiated startle task (e.g., LeDoux, 1995). However, such lesions do not reduce a rat’s preference for covered arms in a plus maze, another commonly used measure of unconditioned anxiety in rat research (Treit, Pesold, & Rotzinger, 1993). In addition, lesions of the central nucleus of the amygdala also do not reduce the lightenhanced startle reflex (Davis et al., 1997). Conversely, lesions of the BNST have no effect on the fear-potentiated startle effect but do significantly reduce light-enhanced startle reflexes (see Grillon, 2008, for a review). Anxiety can also be a conditioned response evoked by long-duration CSs. For example, Waddell, Morris, and Bouton (2006) compared the effects of BNST lesions on aversive conditioning with short (e.g., 60 sec CS) or long (e.g., 600 sec CS) signals for shock. Prior to conditioning, some of the animals had received lesions of the BNST, while others received sham lesions. The BNST lesions had no effect on conditioning of fear with the short-duration CS, but for the long-duration CS, the BNST lesion significantly reduced conditioning of anxiety. Therefore, it appears that the BNST is not only involved in the expression of unconditioned anxiety as seen with the light-induced startle effects, but it is also involved in the conditioning of anxious apprehension (as with the 600 sec CS). Finally, another model of conditioned anxiety, to be discussed


49

later, occurs when contextual cues are associated with unpredictable threat. Fear and Anxiety Conditioning Abnormalities in Anxiety Disorders

If fear and anxiety conditioning processes account in part for the development of various anxiety disorders, then individual differences in personality or past experiences may serve as diatheses and these should be detectable using classical fear or anxiety conditioning paradigms measuring the degree and speed of fear acquisition and extinction. The possibility of individual differences in acquisition (ACQ) and extinction (EXT) has been explored in both psychophysiological experiments comparing anxious and nonanxious individuals, and it has begun to be investigated in behavioral and molecular genetic studies. Ideally, a deeper understanding of these issues would greatly benefit from longitudinal prospective studies in which individual differences in conditionability of fear or anxiety could be used to predict which individuals would subsequently develop anxiety disorders following relevant learning experiences. Unfortunately, we are not aware of any such studies to date and so we will review highlights of the studies that do exist on individuals with several kinds of anxiety disorders. Conditioning Paradigms Used to Study Individual Differences

Two basic experimental designs have been employed in studies comparing conditioning in anxious and nonanxious groups (e.g., Lissek et al., 2005). In the first (simple conditioning), participants in both groups are first exposed to several CS-only trials (habituation) and are then subjected to repeated pairings of a neutral stimulus (CS) with an aversive stimulus (US). The development of a conditioned response (CR) is measured following the conditioning trials (or on nonreinforced test trials during acquisition) by measuring the CR to the CS presented alone and comparing it with responding during habituation; between-group differences are also examined. However, this simple conditioning paradigm is not ideal for measuring the amount of associative learning that has occurred because there

2/15/2011 7:42:31 PM

50


is no control for possible sensitization (nonassociative) effects. In the second type of paradigm (discriminative conditioning), two CSs are used, one paired with the aversive US (the CS+) and one explicitly unpaired with the aversive US (the CS–). By using within-subject comparisons of responding to the CS+ and CS–, one can infer whether the two groups differ in developing discriminative responding to the CS+ and CS–. Such paradigms are superior to simple conditioning paradigms because they better control for possible sensitization effects that should accrue as much to a CS– as to a CS+. Discriminative conditioning paradigms also allow for at least indirect comparisons of inhibitory as well as excitatory conditioning. There are several possible ways in which individual differences in conditioning as a function of anxiety might operate. With simple conditioning paradigms, anxious individuals, relative to nonanxious individuals, might either acquire CRs more rapidly or acquire greater magnitude CRs (to either discrete CSs paired with USs, or to contexts paired with USs). Because it is known that stronger associations extinguish more slowly (e.g., Annau & Kamin, 1961), another possibility is that individual differences might also emerge in rates of extinction. Indeed, EXT might be retarded by the presence of anxiety regardless of whether ACQ responding has been affected. With discriminative conditioning paradigms the possibilities are more complex. One theory proposes that greater differences in discriminative responding to CS+s and CS–s should emerge because of superior conditioning to the CS+ in anxious individuals (e.g., Orr et al., 2000). These differences may also be manifested in slower rates of EXT. However, before considering this issue, we first describe several different theories reviewed by Lissek et al. (2005) that were developed to provide an explanation for differences in fear conditioning in anxious versus nonanxious individuals that are also relevant to predictions about potential group differences in discriminative conditioning. One theory, advanced by Orr and colleagues (Orr et al., 2000), specifically proposed that individual differences in conditionability to fear


stimuli may help account for individual differences in anxiety. This theory predicted that anxious individuals would display stronger or more rapid acquisition of fear CRs than nonanxious individuals and therefore slower extinction. For example, GAD patients, compared with nonanxious individuals, showed similar ACQ but reduced EXT of skin conductance CRs to angry faces (Pitman & Orr, 1986). According to this theory anxious individuals, relative to nonanxious individuals, should also show a greater difference in discriminative responding to CS+s and CS–s. For example, in a study of trauma-exposed individuals, Orr et al. (2000) found that those who had developed PTSD showed greater discriminative responding during ACQ and EXT relative to those traumaexposed individuals who had not developed PTSD. Such results are consistent with the possibility that differences in diatheses for development of PTSD may be partly a function of differences in conditionability to fear cues, although a prospective design would be required to conclude this with certainty. By contrast, another theory proposed by Davis and colleagues (Davis, Falls, & Gewirtz, 2000) specifies differences in inhibitory, rather than excitatory, conditioning as most relevant to the emergence of anxiety disorders. Specifically, they proposed that it is a failure to inhibit the fear CR in the presence of safety signals (for example, a CS–, which signals the absence of the aversive US) that is one mechanism by which clinical anxiety may develop. Consistent with this proposal, some studies show that anxiety patients (relative to controls) exhibit greater fear-potentiated startle during the CS- (Grillon & Ameli, 2001; Grillon & Morgan, 1999). In other studies they showed higher magnitude electrodermal responses during the CS– (e.g., Orr et al., 2000) or greater subjective anticipatory anxiety during a CS– (e.g., Hermann, Ziegler, Birbaumer, & Flor, 2002). Thus, results using all three kinds of fear measures converge in suggesting that the anxiety patients showed smaller magnitude inhibitory CRs to a CS– than controls. However, none of these studies provide definitive direct support for this theory about differences in inhibitory learning as a function of

2/15/2011 7:42:31 PM


anxiety status, which would require direct assessment of the inhibitory power of the CS– in the two groups. Empirical Status of These Theories

Lissek et al. (2005) reviewed and performed a meta-analysis on 20 studies comparing fear conditioning in participants with (n = 453) and without (n = 455) an anxiety disorder (including panic disorder, PTSD, and GAD) in order to assess the empirical status of these theories. It was found that anxious individuals had stronger fear CRs during ACQ and EXT compared to healthy controls, although both effect sizes were small. Interestingly, these effects were found primarily in studies using a simple conditioning procedure; patients and controls showed similar rates of ACQ and EXT when discriminative conditioning procedures were used. Similar discriminative conditioning in patients and controls suggests that, in addition to stronger excitatory conditioning, anxious individuals may be less able than nonanxious individuals to suppress a fear CR in the presence of safety cues (CS–s). In fact, a number of the reviewed studies using discriminative conditioning paradigms (e.g., Grillon & Morgan, 1999; Peri, Ben-Shakhar, Orr, & Shalev, 2000) found elevated CRs in anxious individuals to both CS+ and CS– stimuli. These findings are not consistent with the Orr et al. (2000) hypothesis (or the Orr et al. results with PTSD), but they are consistent with the Davis et al. (2000) hypothesis that anxious individuals are less able to inhibit fear responding in the presence of safety cues. Again, however, such a conclusion would require an independent direct assessment of the inhibitory power of the CS– in the two groups. Researchers have recently begun addressing this question using methodology specifically designed to experimentally distinguish excitatory from inhibitory conditioning. For example, Myers and Davis (2004) adapted a discriminative conditioning paradigm called conditioned discrimination (Wagner & Rescorla, 1972), to explicitly test the inhibitory power of a CS– in rats. In this paradigm (abbreviated AX+/BX–), the excitatory A and inhibitory B stimuli are conditioned independently of one another, in


51

compound with a third stimulus X. After AX+ and BX– trials, A was excitatory, and B inhibitory. Winslow, Noble, and Davis (2008) found parallel inhibitory effects when a very similar paradigm was used in rhesus monkeys. Moreover, one study on individuals with no history of any diagnosable psychological disorder also successfully demonstrated the inhibitory effects of B using such a paradigm (Jovanovic et al., 2005). An even more recent study (Jovanovic et al., 2009) used this paradigm to examine fear inhibition in patients with fairly severe PTSD and found that they failed to show any significant fear inhibition (compared to those with mild PTSD or controls) on a compound trial. However, they were aware that a shock would not occur (i.e., they knew they were safe but they could not actually inhibit the response). It will be interesting to see whether other anxiety disorders are also characterized by failure to learn fear inhibition. In addition to the question of whether vulnerability to anxiety disorders reflects individual differences in either excitatory or inhibitory conditioning processes (or both), the question also remains as to whether individual differences in discrete-cue versus context conditioning, and fear versus anxiety conditioning, are relevant to the development of particular anxiety disorders. As discussed later, fear learning appears to be neurobiologically and behaviorally distinguishable from context conditioning (more akin to anxiety), which is the learning of associations between a US and its broader environmental context, rather than discrete cues upon which a US is contingent. If these two processes vary independently, individual differences in either or both may confer risk for specific anxiety disorders. Correlates of Individual Differences in Fear Conditioning: Personality, Neuroimaging, and Genes

If individual differences in either excitatory or inhibitory fear conditioning may act as diatheses for the development of anxiety disorders, it is sensible to ask what relationship they may have to other diatheses for anxiety. Personality factors such as neuroticism or high negative affectivity

2/15/2011 7:42:32 PM

52


have long been known to predict the development of a range of both anxiety and depressive disorders (e.g., Clark, Watson, & Mineka, 1994; Hayward, Killen, Kraemer, & Taylor, 1998, 2000). In addition, in the past 10 years there has been an explosion of findings using functional neuroimaging techniques that have uncovered individual differences in responsiveness to threat and punishment cues, and progress is being made toward associating differential neural responsivity in critical brain regions with individual differences in trait anxiety and other personality factors (see Hariri, 2009, for a review, but also Vul, Harris, Winkielman, & Pashler, 2009, for a methodological critique of this literature). Moreover, very exciting (but increasingly contested) associations have been found between specific genetic polymorphisms and personality traits, as well as clinical anxiety and mood disorders. In this section, we review research that relates individual differences in fear conditioning with personality and neural activity, as well as discussing emerging research investigating the heritability of fear conditioning and the association of fear conditioning with specific genetic polymorphisms. To prepare readers for this discussion, we begin by briefly summarizing the current research on neuroimaging and genetics associated with threat responsiveness and anxiety. We focus in particular on polymorphisms in two particular genes, the serotonin transporter (5-HTT) and catechol-O-methyltransferase (COMT), which have received considerable attention in recent years for their putative relationships with trait anxiety, fear conditioning, and attentional processing of threat. Genetic and Neuroimaging Variation Associated with Anxiety: Serotonin Transporter Polymorphisms, Anxiety, and Amygdala Activation

In recent years, there has been tremendous excitement over the discovery of a possible relationship between anxious/depressive pathology and allelic differences in a variable repeat sequence of the promoter region of the serotonin transporter gene (5-HTT), particularly in relation to stressful life events. This gene encodes for a protein that regulates the reuptake of serotonin


from the synaptic cleft, in turn affecting the synaptic concentrations of serotonin. Research has shown that having one or two short 5-HTT alleles, rather than two long alleles, is associated with as much as 50% less serotonin transporter availability, functionally increasing the synaptic concentration of serotonin (Heinz et al., 2000; Lesch et al., 1996). Initial work suggested that carriers of at least one short allele have higher levels of trait anxiety and neuroticism (Munafo, Clark, & Flint, 2004; Schinka, Busch, & Robichaux-Keene, 2004; Sen, Burmeister, & Ghosh, 2004), and increased risk for major depressive disorder after exposure to stressful life events (Caspi et al., 2003; Eley et al., 2004; Kaufman et al., 2004). With respect to the neuroanatomical correlates of these polymorphisms, healthy carriers of at least one short allele of the 5-HTT polymorphism, in comparison to those homozygous for the long allele, demonstrate greater limbic responsiveness to angry or fearful expressions (Bertolino et al., 2005; Hariri et al., 2002; Pezawas et al., 2005). Evidence also suggests an association between the presence of a short allele and stronger activation of the amygdala, together with greater coupling between the amygdala and ventromedial prefrontal cortex (vmPFC), to aversive images (Canli et al., 2005; Heinz et al., 2004). However, it is noteworthy that the analytic methodologies used in research relating neuroimaging results with self-reported mood and personality are presently under intense scrutiny (see Vul, Harris, Winkielman, & Pashler, 2009, for a pointed critique), so it is prudent to interpret such findings with caution. Enthusiasm over such findings is also currently waning in the light of several recent metaanalyses that call into question the relationship of the 5-HTT polymorphisms, particularly in interaction with life stress, with psychopathology and trait anxiety (Munafo, Durrant, Lewis, & Flint, 2009; Risch et al., 2009). However, as Monroe and Reid have argued (2008), the overwhelming majority of research on such gene– environment interactions has relied upon highly inconsistent measures of life stress across different studies, severely limiting the implications of meta-analyses on the literature to date. As this

2/15/2011 7:42:32 PM


issue receives more attention, and the considerable research on life stress conceptualization and measurement is brought to bear on behavioral genetics research (Monroe, 2008), greater clarity will hopefully be brought to the role of this promising gene candidate. On a more positive note, there appears to be greater consistency in studies of the link between serotonin transporter genotype and amygdala activation (Munafo, Brown, & Hariri, 2008), although the authors of this meta-analysis and review warn that most studies to date are lacking in statistical power, rendering estimates of effect sizes premature. Genetic and Neuroimaging Variation Associated With Anxiety: COMT Polymorphisms, Anxiety, and Prefrontal Functioning

Another promising candidate genetic variant in anxiety is the catechol-O-methyltransferase gene (COMT), which encodes for an enzyme that degrades dopamine, epinephrine, and norephinephrine in the prefrontal cortex (Gogos et al., 1998; Tunbridege, Bannerman, Sharp, & Harrison, 2004), and hippocampus (Matsumoto et al., 2003). Variations in its functional polymorphism, COMT Val158Met, influence the levels of dopamine in these areas, with the Met allele associated with a third of the enzymatic activity of Val the allele in breaking down dopamine. Functionally, this results in the Met allele producing higher levels of synaptic dopamine in these brain areas. Although many contradictory findings have been published, preliminary evidence suggests that the Val158 allele is associated with inefficiency in cognitive control, whereas the Met allele has been related to anxiety related traits and disorders (Domschke et al., 2004; Enoch, Xu, Ferro, Harris, & Goldman., 2003; Stein, Fallin, Schork, & Gelernter, 2005). Personality and Temperament in Fear/ Anxiety Conditioning

As mentioned earlier, several stable temperamental and personality factors that are partially heritable have also been identified as diatheses for various anxiety disorders. One of these is the personality trait known as neuroticism or negative affectivity (Clark et al., 1994; Watson,


53

Gamez, & Simms, 2005). In addition, trait anxiety—one of several facets of neuroticism— is also thought to be a vulnerability factor. Moreover, over the years numerous studies have shown that individuals high in trait anxiety show more rapid and stronger aversive conditioning (e.g., see Levy & Martin, 1981, for an early review; Lissek et al., 2005; Zinbarg & Mohlman, 1998). It is possible that such conditioning processes could be a mechanism through which high trait anxiety operates as a vulnerability factor for clinical anxiety disorders. In this regard it is also interesting to speculate that the partial overlap in heritability of anxiety disorders, such as specific phobias and panic disorder (e.g., Kendler et al., 1995), could be mediated by heritable differences in conditionability (see Bouton et al., 2001). Moreover, research on temperament in children indicates that young children with high levels of behavioral inhibition (a tendency to be shy, avoidant, and easily distressed by unfamiliar stimuli) in early childhood are at heightened risk for developing multiple specific phobias in childhood (Biederman et al., 1990), and social anxiety in adolescence when it is most likely to develop (e.g., Hayward et al., 1998; Schwartz, Snidman, & Kagan, 1999). Whether the effects of behavioral inhibition are mediated through differences in conditionability is not yet known, although this seems like a possibility worth investigating. Heritability and Genetic Findings in Fear Conditioning

There are several twin studies of other forms of learning, but we are only aware of one study that has specifically examined the heritability of fear conditioning. Hettema and colleagues (2003) investigated fear conditioning in 173 same-sex twin pairs (90 monozygotic and 83 dizygotic) using a standard discriminative conditioning procedure, in which pictorial stimuli (either fear-relevant [snakes and spiders] or fear-irrelevant [circles and triangles] stimuli) were paired with a mild electric shock US, measuring skin conductance (SCR) as the CR. Rates of habituation, acquisition, and extinction were all assessed in order to model the relative contributions of both associative and nonassociative processes.

2/15/2011 7:42:32 PM

54


Moderate heritability was found for all three components, with additive genetic effects accounting for 34% to 43% of the total variance. Using structural equation modeling, they found that a two-factor model best explained the observed genetic variation in twin pairs. One factor accounted for variation in habituation, acquisition, and extinction (the nonassociative factor) and one factor accounted for variation in only acquisition and extinction (the associative factor). Interestingly, the investigators also found evidence that the heritability of associative fear learning using evolutionarily relevant stimuli, such as snakes and spiders, may be greater than that of fear-irrelevant stimuli, although statistical power was not sufficient to draw this conclusion with certainty. A recent and exciting genetic study investigated the specific genetic polymorphisms 5-HTTLPR and COMT for their effects on conditioned fear acquisition and extinction (Lonsdorf et al., 2009). Forty-eight college students donated blood for DNA extraction and genotyping and then underwent a discriminative fear-conditioning paradigm using facial stimuli as CS+s and CS–s. There were nine CS+ trials, each ending with a 10 ms shock, and nine CS- trials ending with no shock. Startle probes were presented after 4–5 seconds on six of nine CS+ trials and on six of nine CS– trials; SCRs were also measured but the most important dependent variable was startle potentiation. On the following day there were 18 presentations of both the CS+ and the CS–, neither followed by a US, to assess extinction performance. As shown in Figure 3.1, results were striking for startle potentiation, but not SCRs, as the dependent measure. Carriers of one or two short alleles of the 5-HTT gene showed significantly greater fear potentiation to the CS+ than did l/l homozygous carriers during acquisition, a pattern that persisted in extinction. By contrast, the two polymorphisms of the COMT Val158Met gene did not affect startle potentiation during acquisition. However, those with the homozygous met/met COMT Val158Met polymorphism showed much greater CS+ fear potentiation during extinction than those who were val-allele carriers. The participants who showed the most pronounced startle responding in extinction


were those with the short allele(s) of the 5-HTT gene and the homozygous met allele of the COMT gene. These results suggest that polymorphisms in these two genes are differentially related to the ability to learn and maintain fear of aversive stimuli. More specifically, those with a short 5-HTT allele and two COMT met alleles are more likely to acquire fear of conditioned aversive stimuli, and less able to extinguish such learned responses. The additional finding that such results held only for the startle index of fear (and not for SCRs) is consistent with findings and ideas reviewed by Öhman and Mineka (2001) that there are two levels of fear learning: One is at a very basic emotional level and the other is at a cognitive level. Research shows that startle potentiation is a better index of a real emotional level of fear. SCRs, by contrast, are more an index of cognitive contingency learning. As Öhman and Mineka (2001) discuss, there are many examples in the literature of different parameters having selective effects on these two different indices of fear. Other Sources of Individual Differences in the Learning of Phobias and Other Anxiety Disorders Other Pathways to the Acquisition of Fear and Anxiety

So far we have only discussed traditional classical conditioning as a source of fear and anxiety. Yet not everyone developing phobias or other anxiety disorders seems to have undergone traumatic conditioning experiences. In many instances phobic fears instead may be acquired vicariously through simply watching another person (live or on a TV or movie screen) behaving fearfully with some object or situation. This is best illustrated by Mineka and Cook’s primate model of phobic fear acquisition from the 1980s (e.g., Cook & Mineka, 1991). They showed that laboratoryreared rhesus monkeys that were not initially afraid of snakes rapidly acquired an intense and long-lasting phobic-like fear of snakes after simply watching a wild-reared model monkey behaving fearfully on some trials when a toy or real snake was present and nonfearfully on other

2/15/2011 7:42:32 PM

FEAR CONDITIONING AND ATTENTION TO THREAT CS+

Startle blink magnitude (Difference from ITI; DT scores)

a

c

Conditioning: 5-HTTLPR 10

6 4

–2

6

*

–2

d 10

10

8

8

***

2

4

***

0

***

–2 s-allele carriers

10

**

long/long

val-allele carriers

Extinction: 5-HTTLPR s Carriers **

8 6

6

**

4

met/met

f Extinction: COMTval158met

Extinction: 5-HTTLPR

6

***

2

0 val-allele –2 carriers

met/met

*

4

***

0

b Startle blink magnitude (Difference from ITI; DT scores)

6

2 long/long

Conditioning: 5-HTTLPR s Carriers

8

4

s-allele carriers

e 10

*

0

Conditioning: COMTval158met

8

***

2

CS–

10

*

8

55

*

2

4 2

0

0

**

*

–2

–2 met/met

val-allele carriers

met/met

val-allele carriers

Figure 3.1 Potentiation of startle-response magnitudes as a function of genotype and stimulus. Black

bars show the difference between magnitude of the startle response elicited during the CS+ (the conditioned stimulus coupled to the unconditioned stimulus) and magnitude of the startle response elicited during the intertrial interval (ITI); white bars show the difference between magnitude of the startle response elicited during the CS− (the conditioned stimulus never coupled to the unconditioned stimulus) and magnitude of the response elicited during the ITI. Results are shown for 5-HTTLPR genotype groups (a) during conditioning and (b) during extinction, for COMTval158met genotype groups (c) during conditioning and (d) during extinction, and for COMTval158met genotype groups within 5-HTTLPR s-allele carriers (e) during conditioning and (f) during extinction. Error bars represent standard errors. Asterisks indicate significant differences, ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. (From Lonsdorf, T. B., Weike, A. I., Nikamo, P., Schalling, M., Hamm, A. O., & Öhman, A. 2009. Genetic gating of human fear learning and extinction: Possible implications for gene-environment interaction in anxiety disorder. Psychological Science, 20(2), 198–206).

trials when a neutral object was present. Vicarious acquisition of aversive conditioning has also been demonstrated in humans using psychophysiological responses (such as electrodermal responses) as the dependent variable (Green & Osborne, 1985). Several studies have also supported the influence of parental modeling on increasing children’s fears for at least 1 week


after the modeling experience (e.g., Askew & Field, 2007; Gerull & Rapee, 2002). Evidence is also accumulating of modeling of social anxiety in families of those who have developed social phobia (e.g., Bruch & Heimberg, 1994; Rapee & Melville, 1997). For example, both mothers and their socially phobic offspring reported more social avoidance in the parents

2/15/2011 7:42:33 PM

56


than seen in nonclinical control families. Moreover, parents of those who develop social phobia often seem to have discussed and thereby reinforced children’s avoidant tendencies in the context of threatening situations (e.g., Barrett, Rapee, Dadds, & Ryan, 1996). The Effects of Experiential Differences on Learning

In addition to genetic and personality differences having sometimes significant effects on conditioning, it is also well known that prior learning experience can have powerful effects on the acquisition, maintenance, and extinction of fear and anxiety (Mineka, 1985; Mineka & Zinbarg, 1996, 2006). Indeed, there are a multitude of individual experiential differences that affect the outcome of aversive learning experiences. For example, the amount of exposure an individual has had with a potential CS before encountering it paired with a US very much affects the outcome of the conditioning experience. This phenomenon in classical conditioning is known as latent inhibition and illustrates that familiar stimuli or situations result in weaker conditioning than do novel or strange objects or situations (e.g., Lubow, 1998; see Kent, 1997, for a naturalistic example). Moreover, Mineka and Cook (1986) showed that monkeys who first simply watched nonfearful monkeys behaving nonfearfully with snakes were immunized against later acquisition of a fear of snakes when they watched a fearful model behaving fearfully with snakes (which as discussed earlier ordinarily leads to strong and long-lasting conditioning). Finally, one study showed that rats initially exposed to escapable shocks later showed reduced fear conditioning both to a context and to a discrete CS relative to groups receiving no prior shocks or inescapable shocks (Baratta et al., 2007). Conversely, rats initially exposed to inescapable shocks later showed potentiated fear conditioning (see Chapter 6). Certain characteristics of a conditioning experience itself can also be important determinants of the level of fear that is conditioned. For example, the ability to escape (i.e., control) a traumatic experience dramatically reduces the magnitude of the level of fear that is conditioned


to a CS+ relative to when the same intensity of trauma is inescapable or uncontrollable (e.g., Mineka et al., 1984; Mineka & Zinbarg, 1996, 2006). Animal research has also shown that social defeat (another uncontrollable stressor) leads to exaggerated fear-conditioned responses (e.g., Williams & Scott, 1989) as well as increased submissiveness to other conspecifics (e.g., Uhrich, 1938) as in social phobia. In addition, experiences that a person has following a conditioning experience may affect the strength and maintenance of conditioned fear. For example, the inflation effect, first discovered by Rescorla (1974), suggests that someone who acquired a mild fear following a minor aversive experience with a CS might be expected to develop a more intense fear when later exposed to a much more aversive experience (even though it was not paired with the CS). These few examples of experiential factors that very much influence the onset and maintenance of fears and phobias are more complex than suggested by earlier simplistic conditioning views, although they are altogether consistent with contemporary research and theory on learning (Mineka & Zinbarg, 1996, 2006). Selective Associations in Fear Learning

Finally, some sources of important differences in who acquires phobias and other anxiety disorders reside in the nature of the objects or situations that come to be paired with aversive experiences. Indeed, human and non-human primates seem to be evolutionarily prepared to rapidly associate certain kinds of objects or situations—such as snakes, spiders, water, and enclosed spaces—with frightening or unpleasant events (e.g., Mineka & Öhman, 2002; Öhman, 1986; Öhman & Mineka, 2001). This inborn tendency to rapidly associate certain objects or situations that once posed real threats to our early ancestors probably occurred because organisms that did so would have enjoyed a selective advantage in the struggle for existence. In the social realm, the kinds of cues that are most likely to become the sources of fears are cues that signal dominance and aggression from conspecifics such as angry or threatening faces (e.g., Mineka & Öhman, 2002; Öhman & Mineka, 2001).

2/15/2011 7:42:33 PM


A large amount of experimental evidence now supports this preparedness theory of phobias. One very important series of experiments using human participants conducted by Öhman and his colleagues has found that fear is conditioned more effectively to fear-relevant stimuli (slides of snakes, spiders, or angry faces) than to fearirrelevant stimuli (such as slides of flowers, geometric objects, or happy faces) that are paired with mild electric shocks (e.g., see Öhman & Mineka, 2001, for a review). Indeed, even very brief subliminal presentations of such fearrelevant stimuli (but not fear-irrelevant stimuli) are sufficient to evoke conditioned responses (including activation of the amygdala) either when presented in acquisition or in extinction (Carlsson et al., 2004; Öhman, Carlsson, Lundqvist, & Ingvar, 2007). This subliminal activation of responses to phobic stimuli may help to account for certain aspects of the irrationality of phobias because the fear may arise from cognitive structures not under conscious control (e.g., Öhman & Mineka, 2001). Another series of experiments on observational conditioning showed that lab-reared monkeys can easily acquire fears of fear-relevant stimuli such as toy snakes or toy crocodiles but not of fear-irrelevant stimuli such as flowers or toy rabbits (Cook & Mineka, 1989, 1990). Thus, both monkeys and humans seem to selectively associate certain fear-relevant stimuli with threat or danger. Moreover, the lab-reared monkeys had no prior exposure to any of the stimuli involved (e.g., snakes or flowers) before participating in these experiments. Thus, the monkey results support the evolutionarily based preparedness hypothesis even more strongly than do the human experiments. For example, human participants (unlike the lab-reared monkeys) might show superior conditioning to snakes or spiders because of ontogenetic factors such as preexisting negative associations to snakes or spiders, rather than because of evolutionary factors. These results also demonstrate that selective associations occur not only with mild and transient conditioning as seen in the human experiments but also with intense and longlasting phobic-like fears seen in the monkey experiments (Mineka & Ohman, 2002).


57

Contextual Control of Conditioned Responses

Conditioned anxiety effects have also been observed using contextual cues associated with threat. Specifically, it has long been known that when unsignaled shocks are presented to rats in a distinctive environment, the environment itself acquires the capacity to elicit conditioned anxiety. Such contextual conditioning is important for several reasons. One reason why contextual conditioning is important stems from demonstrations that patients with PTSD and panic disorder show enhanced contextual modulation of baseline startle in experiments in which they know a fear-potentiated startle paradigm with an explicit threat cue will be included at some point (i.e., including delivery of shock) (Grillon & Ameli, 1998; Grillon & Morgan, 1999). However, anxiety-disordered individuals in these studies did not show exaggerated fear responses to explicit cues for imminent threat (as would be evidenced by greater fear-potentiated startle— that is, phasic fear to explicit cues) relative to controls. Thus, at least for people with panic disorder or PTSD, this elevated contextual modulation of startle seems to represent heightened anxiety in contexts in which something threatening may occur, but this is not accompanied by greater fear to more proximal discrete cues for threat (relative to controls). Very similar results have also been observed in children at familial risk for anxiety disorders (having a parent with an anxiety disorder) who also show enhanced contextual modulation of startle, but normal fear-potentiated startle (Grillon, Dierker, & Merikangas, 1998). Finally, individuals with high levels of neuroticism—a risk factor for most anxiety disorders—show a very similar pattern of results (Craske et al., 2009), that is, enhanced contextually modulated startle, but normal fearpotentiated startle relative to those with low or medium levels of neuroticism. Contextual conditioning is also important because of the critical role it plays in reinstatement of fear to a CS+ following the full or partial extinction that occurs when the CS is no longer followed by the US. Reinstatement is important in part because it may underlie the fluctuating

2/15/2011 7:42:33 PM

58


course of symptoms often seen in anxiety disorders (e.g., Bouton et al., 2001; Mineka & Zinbarg, 1996). Reinstatement occurs when animals or humans whose fear of a CS has been extinguished show a return of that fear (“reinstatement”) after one or a few exposures to the US alone (not paired with the CS) (Rescorla & Heth, 1975). Subsequent work showed that reinstatement only occurs if the US is presented in the same context as where testing for reinstatement is to be conducted (e.g., Bouton, 1984). According to this and other research by Bouton and colleagues, when the reinstating US is presented, the animal must associate it with the current context. The presence of that contextual danger when the CS is next presented is thought to trigger fear of the previously extinguished CS. Moreover, evidence from Bouton’s lab has also confirmed the prediction that lesions of the BNST significantly attenuate reinstatement of fear, presumably because they blocked or reduced contextual conditioning of anxiety that would ordinarily occur with the reinstating US (e.g., Waddell et al., 2006). Unpredictability and Anxiety

Unpredictable aversive events have long been known to be more stressful than predictable aversive events, and exposure to unpredictability has been hypothesized to play a role in the development of panic disorder, GAD (with more minor events), and PTSD (with more severe events) (Mineka & Zinbarg, 1996, 2006). The most widely cited hypothesis offered to explain these effects is the safety-signal hypothesis (e.g., Seligman, 1968; Seligman & Binik, 1977). The idea is that when organisms are presented with signaled (predictable) aversive events they not only know when the event will occur (during the CS+) but also when the event will not occur (when the CS is not present—a safety signal). Safety signals allow them to relax and feel safe. By contrast, organisms exposed to unpredictable aversive events have no knowledge of when the threatening events will occur, or when they can relax and feel safe, and thus are in a state of chronic anxiety in this context. Early studies in rats used both stress-induced ulceration paradigms (Weiss, 1971) and conditioned suppression paradigms (Seligman, 1968).


Recently this work has been replicated and extended using other measures of anxiety in human participants. For example, Grillon et al. (2004) modeled this situation in a human laboratory with normal participants who all experienced three conditions: one in which predictable shocks were given, one in which unpredictable shocks were given, and one in which no shocks were given. Results clearly showed that in the unpredictable shock condition participants showed both greater startle magnitude and higher subjective anxiety than in the other two conditions. Other studies by Grillon and colleagues have shown that in some situations what may be most important in determining contextual levels of anxiety is whether aversive stimuli are perceived to be predictable rather than whether they actually are predictable, with more conditioning to the context when the participants do not detect the CS-US contingency. So, for example, several studies have shown that participants in a discriminative conditioning procedure who failed to become aware of the CS-US contingency showed more contextual conditioning (i.e., potentiation of baseline startle responding in the startle context) than participants who were aware of the contingency. In the latter case they showed enhanced startle to their CS+ rather than more generalized contextual conditioning (Grillon, 2002). In one ingenious experiment by Grillon and colleagues (2006), participants were passively exposed to a virtual reality environment with three virtual rooms (one with no shock, one with predictable shocks, and one with unpredictable shocks). Participants later showed potentiated startle in the unpredictable context relative to the other two contexts, presumably because of greater contextual conditioning with the unpredictable shocks. Furthermore, when allowed to freely enter the three different rooms to find monetary rewards, there was a strong preference for the no-shock context and avoidance of the unpredictable context. Recently Grillon and colleagues extended the study of unpredictability to patients with panic disorder, PTSD, and GAD to determine whether they show elevated reactivity to unpredictable aversive events. In one study on panic disorder,

2/15/2011 7:42:33 PM


Grillon, Lissek et al. (2008) found that panic patients showed elevated contextually modulated startle when startle probes were delivered in a context where naturalistic unpredictable aversive events sometimes occurred (such as a white noise or a female scream) relative to a neutral context; startle responding in a predictable threat condition was intermediate. The healthy controls did not show this elevated startle potentiation in the unpredictable relative to the neutral condition. In another study the same investigators used nearly an identical paradigm but compared patients with PTSD, GAD and healthy controls (Grillon, Pine, Lissek, Rabin, & Bythilingam, 2009). The PTSD group showed elevated startle in the unpredictable context relative to the neutral and predictable contexts, but the other two groups showed elevated startle only in the predictable context. Thus, patients with both panic disorder and PTSD showed elevated reactivity to unpredictable aversive events relative to healthy controls and to patients with generalized anxiety. Summary

In this section we have briefly reviewed multiple sources of individual differences in associative learning processes relevant to furthering our understanding of diathesis-stress perspectives on anxiety disorders. Thus, we discussed how high trait anxiety and clinical anxiety both affect acquisition and extinction of conditioned fear using simple conditioning paradigms, whereas discriminative conditioning does not seem to be affected—possibly because anxiety is associated with poor inhibitory conditioning. We also reviewed results of several studies suggesting that fear conditioning is moderately heritable and that individuals who have different combinations of polymorphisms of the 5-HTT and COMT genes show more or less robust fear conditioning, and faster or slower fear extinction. In addition, evidence that many individual experiential differences occurring before, during, or following real or vicarious conditioning trials affect the outcome of those conditioning trials was also reviewed. Finally, some sources of individual differences reside in the nature of the objects or situations that are associated with


59

aversive consequences rather than in the individuals themselves as seen in research on selective associations. We also reviewed some of the evidence showing how individuals with certain anxiety disorders (PTSD and PD) show enhanced contextual conditioning in situations associated with uncertain threat and greater reactivity to unpredictable aversive events than do normal controls.

THREAT-RELEVANT ATTENTIONAL BIASES Although early models of anxiety disorders were primarily behavioral, many contemporary psychological models of the etiology and treatment of mood and anxiety disorders are more cognitive in nature. They posit that biased modes of processing affectively valenced material determine important characteristics of the emotional pathology (Beck, 1976; Eysenck, 1992; Williams, Watts, MacLeod, & Mathews, 1997). Within this tradition, attentional biases for emotional stimuli have been studied for over two decades and remain an exciting source of theoretical and applied questions in psychopathology. Most research on attentional biases has focused on their association with clinical anxiety and mood disorders, elevated trait anxiety, and dysphoria (Bar-Haim et al., 2007; Mogg & Bradley, 1998; Williams et al., 1997). Indeed, attentional biases for threat have been central to several etiological models of anxiety (Mathews & Mackintosh, 1998; Mogg & Bradley, 1998; Williams et al., 1997). A smaller body of research has shown the following: (1) attentional biases in individuals at risk for emotional disorders (Joormann, Talbot, & Gotlib, 2007); (2) biases may predict negative reactions or responses to stress (MacLeod & Hagan, 1992; Mogg, Bradley, & Hallowell, 1994); (3) biases may persist beyond remission of clinical disorders (e.g., Joormann & Gotlib, 2007). Particularly interesting from a learning perspective are studies suggesting that bias modification—the use of attentional training tasks to change response patterns—may reduce negative responses to stress and thereby indirectly influence mood states (e.g., Amir et al., 2009; Dandeneau et al., 2007; MacLeod et al., 2002).

2/15/2011 7:42:33 PM

60


Many of the great challenges in this research are related to developing a deeper understanding of precisely what attentional phenomena are associated with mood and anxiety pathology, the causes of these associations, their consequences for other cognitive processes, and the etiology/ maintenance of clinical disorders. MacLeod, Mathews, and colleagues, who were responsible for the first identification of the association between trait anxiety and biased attention for threat (Mathews & MacLeod, 1985), have continued to explore the relationship between biased attention to threat and stress, as well as the effects of modifying attentional biases on reactions to stress (MacLeod et al., 2002; MacLeod, Soong, Rutherford & Campbell, 2007; See et al., 2009). Drawing from classic attentional models (Posner & Petersen, 1990), Fox and others have sought to understand the specific components and time courses of emotion-relevant attentional biases (orientation/shifting, disengagement). The work of Öhman and colleagues (e.g., Öhman & Mineka, 2001) has focused on attentional processing of evolutionary fear-relevant stimuli and their relationship to the development of clinical phobias. Moreover, great strides in understanding the neural and genetic underpinnings of threat perception and attention in trait anxiety/neuroticism have been made by several research groups (e.g., Canli et al., 2005; Hariri et al., 2002). In this section, we first review the experimental paradigms typically used in attention and emotion research. Next, we summarize the most recent findings in attentional bias research, with particular focus on the concepts with potential relevance to conditioning processes. Methods of Attentional Bias Assessment

Attentional biases have traditionally been assessed using one of several experimental paradigms, including most prominently the emotional Stroop task, the dot-probe task, the exogenous spatial-cueing task, and the visual search task. All of these tasks share a common logic: Attentional biases are inferred by comparing an individual’s reaction times (RTs) on critical trials involving emotionally valenced stimuli to trials involving neutral stimuli (stimuli are variably words, photographs, or schematics of


emotional faces, objects, or scenes). That is, individual biases are inferred when mean or median RTs to trials with emotionally valenced stimuli differ significantly from those to trials with neutral stimuli. Researchers have debated and tested which of these paradigms best captures the biased attentional processing associated with psychopathology. In the following sections, we briefly describe the most frequently employed paradigms, with particular emphasis on those that have been used in conditioning paradigms. Emotional Stroop Task

The emotional Stroop task (see Williams, Mathews, & MacLeod, 1996, for a review), although widely used in early research on attentional biases, has received substantial criticism as a pure measure of selective attention. In this task, participants are required to name the color that a word is printed in; trials in which the meaning of the word is emotionally valenced versus those in which the meaning is neutral are compared. Biased attention toward emotional words is inferred by slower reaction times to the emotional versus neutral words. Several researchers (e.g., Algom, Chajut, & Lev, 2004; Baldo, Shimamura, & Prinzmetal, 1998) have indicated that the emotional Stroop task represents a “generic slowdown,” or response bias, to emotionally valenced stimuli rather than a selective attention mechanism. Despite criticisms leveled against the emotional Stroop task, it continues to be used, and several of the conditioning studies discussed in the next section employ the measure. Dot-Probe Task

In the past 10 years, the dot-probe detection task (MacLeod, Mathews, & Tata, 1986) has emerged as the most widely used paradigm for assessing selective attentional biases. In the most common form of the task, trials consist of a neutral and a valenced stimulus briefly presented (typically 15–1,250 ms) on each side of a central fixation cross displayed on a computer monitor, followed immediately by a small dot probe that replaces one of the stimuli. The probe remains on the screen until participants indicate, by pressing a keyboard button, which side of the screen the

2/15/2011 7:42:33 PM


probe appears on, and their latency to respond from the time of the dot probe onset is recorded as their reaction time. If a participant responds more quickly, on average, to probes replacing valenced than neutral stimuli, it is inferred that he or she was attending to the valenced stimulus at the time of the probe onset more frequently than to the neutral stimulus. Spatial-Cueing Task

Another task that is increasingly used is the exogenous spatial-cueing task, which was adapted by Fox and colleagues (2001) from the classic Posner cueing paradigm (Posner, 1980), in order to differentiate whether attentional biases associated with anxiety were due to facilitated orientation toward threat stimuli versus delayed disengagement from threat. Trials consist of a presentation of a single cue stimulus presented on the left or right side of a central fixation cross on the screen for a brief period of time (typically 200–600 ms), followed by a screen with only the fixation cross remaining, followed by a small target circle to which participants are required to respond with a spacebar press. Of these experimental trials, most (60%–75%) are valid (i.e., the target appears in the same location as the cue), whereas the remainder are either invalid (i.e., the target appears in the opposite location to the cue) or catch trials (no target appears after the cue). Equal numbers of trials are allotted to each cue stimulus type, and any particular stimulus has an equal probability of appearing in the left- and right-hand side boxes, and an equal probability of being followed by a valid, invalid, or catch trial. By comparing within-subject reaction times to threat versus neutral stimuli on validly cued trials, it is possible to infer that the speed of orientation is affected by the stimulus valence. By comparing reaction times to threat versus neutral stimuli on invalidly cued trials, it is possible to infer that the speed of disengagement is affected by the stimulus valence. At the longer stimulus exposure intervals (600 ms), this paradigm is used to assess the phenomenon of inhibition of return, in which all individuals are slower to reengage their attention toward a location or object that has been recently attended. Individual differences in the speed with which


61

participants respond to a target that has been validly cued by a threatening versus neutral stimulus over this longer time frame are indicative of valence-dependent inhibition of return. Visual Search Task

The visual search task has been used less frequently in attentional bias research in clinical and trait anxiety (e.g., Fox et al., 2000; Öhman, Flykt, & Esteves, 2001; Tipples, Young, Quinlan, Broks, & Ellis, 2002), but it has been advocated as a more ecologically valid attention task than those discussed earlier (Weierich, Treat, & Hollingworth, 2008). It also has the advantage of a well-established body of normative research in visual cognition. In this task, participants view an array of visual stimuli on a computer screen and indicate whether a particular “target” stimulus is present among other distracter stimuli. The speed with which the target stimulus is detected is taken to indicate the efficiency with which attention is directed toward that stimulus. In anxiety research, arrays typically consist of fear-relevant and fear-irrelevant stimuli, and individual differences are assessed in the speed with which he or she is able to detect threatening stimuli among neutral arrays versus neutral stimuli among threatening stimulus arrays. Current Theories and Open Questions in Attentional Bias Research

In general, researchers have found statistically significant within-subject differences to valenced and neutral stimuli primarily in participants with clinical anxiety or depression, or with high levels of trait anxiety or depressive symptoms. The majority of these effects have been found among individuals with anxiety disorders using threatening stimuli and shorter dot-probe or spatial-cueing task stimulus presentations (≤500 ms). Depression-related attentional biases (generally for depression-relevant stimuli, such as sad faces), by contrast, have only been consistently reported more recently, and only at exposure durations of greater than 1,000 ms (see Mogg & Bradley, 2005, for an overview). However, there are several exceptions to the failure to find biases in nonanxious, nondepressed individuals.

2/15/2011 7:42:33 PM

62


Several studies using dot-probe tasks have shown that at shorter exposure durations (100 ms), all individuals attend toward highly threatening stimuli (e.g., Cooper & Langton, 2006; Koster et al., 2005), while only anxious individuals continue to attend toward the threatening stimuli at exposure durations near 500 ms. Moreover, at least one study (Wilson & MacLeod, 2003) demonstrated that individuals both high and low in trait anxiety demonstrate a bias toward highly threatening angry faces, whereas only highly trait-anxious individuals demonstrate a bias toward moderately threatening faces. In addition, evidence has increasingly mounted for the notion that nonanxious individuals selectively attend away from mild to moderately threatening stimuli at longer exposure durations (see Bar-Haim et al., 2007; Frewen et al., 2008, for meta-analyses). Such findings point toward a rather complex interaction between individual differences in anxiety and depression, stimulus valence intensity, and stimulus exposure duration. As mentioned previously, several models of attentional biases in anxiety have been articulated by researchers (Mathews & Mackintosh, 1998; Mogg & Bradley, 1998; Williams et al., 1997). In this section, we highlight the most important theoretical issues that researchers have tackled experimentally. Time Course of Attentional Biases: Orientation, Disengagement, and Avoidance

A recent debate in attentional bias research has concerned resolving apparently conflicting findings regarding whether high trait anxiety is associated with more rapid orientation toward threat or delayed disengagement from threat. In addition, there has also been related debate about whether initial attentional engagement with threat in trait anxiety is followed by avoidance of threat at longer stimulus exposure intervals. For instance, Mogg and Bradley (1998) argued that highly trait-anxious individuals direct their attention more rapidly toward threat, but later avoid it, resulting in a failure to habituate normally to threat-relevant stimuli. By contrast, Fox and colleagues (Fox, Russo, Bowles, & Dutton, 2001; Fox, Russo, & Dutton, 2002) demonstrated


with the spatial-cueing task described earlier that highly trait-anxious individuals do not actually appear to orient more quickly toward a singly presented threatening cue, but they do tend to dwell on such stimuli longer than do low traitanxious individuals. Other researchers have found additional evidence for the delayed disengagement, or sustained attentional maintenance, hypothesis (e.g., Batty, Cave, & Pauli, 2003; Salemink, van den Hout, & Kindt, 2007). By approaching the literature from a more rigorous vision science perspective, Weierich, Treat, and Hollingworth (2008) have provided an excellent theoretical synthesis of these apparently disparate findings—that is, whether traitanxious individuals orient their attention more quickly toward threat versus whether they take longer to disengage from it, and whether these phenomena are followed by attentional threat avoidance. Their analysis is based on conceptual clarification of covert and overt attentional processes, close evaluation of the timescale at which biases are assessed, and differentiating between paradigms that require attention to a single stimulus versus competition between two stimuli (e.g., the exogenous cueing task, which uses a single stimulus, versus the dot-probe task, which presents two stimuli). In summary, they point out that these two views are compatible if it is assumed that threatening stimuli are more likely to be selected as targets of attention when in competition with other stimuli, but that the speed of covert attentional shifts toward stimuli is not influenced by threat relevance. That is, highly trait-anxious individuals may be more likely to attend to an angry face when it is presented together with a neutral face and be more likely to dwell upon it longer, but they are not faster than low trait-anxious individuals to covertly direct their attention toward the angry face. Moreover, evidence for overt and covert avoidance following disengagement from threat stimuli comes from studies assessing individual differences at longer stimulus exposure durations, particularly in eye-tracking experiments or spatial-cueing tasks showing interrupted inhibition of return among anxious individuals (Calvo & Avero, 2005; Fox et al., 2002; Pflugshaupt et al., 2005).

2/15/2011 7:42:33 PM


Trait Anxiety, Threat Responsiveness, and the Amygdala

Several comprehensive theories of attention and anxiety (Mathews & Mackintosh, 1998; Mogg & Bradley, 1998) have argued that the perceived intensity of a threat cue mediates the relationship between trait anxiety and attentional biases. Evidence for this has been mixed, but it receives considerable support from the studies described in the next section on attentional bias and conditioning. Although explicit measures of selfreported threat intensity do not appear to be related to trait anxiety (Wilson & MacLeod, 2003), neuroimaging data increasingly suggests that trait anxiety is correlated with amygdala reactivity to emotional stimuli, even when presented subliminally (e.g., using backward masking and ΣV), there is error. This resulting error serves as the reinforcing signal. Thus, the greater the analgesic feedback, the smaller the error. The model can be tested by blocking endogenous opioids with antagonists (e.g. naloxone or naltrexone). Without endogenous opioidmediated conditional analgesia, the model predicts that ∆V = αβ(λ – ΣV) becomes ∆V = αβ(λ). There is extensive evidence for this prediction. For instance, a number of findings demonstrate that opioid antagonists such as naloxone attenuate blocking if they are administered in the second phase of a blocking experiment (Fanselow & Bolles, 1979a, 1979b; Galli et al., 2009; McNally et al., 2004a) (see Fig. 14.3). In addition, Young and Fanselow (1992) showed that administration of naloxone prior to conditioning results in

311

50

25

0 Forward

Backward Forward Training condition Saline

Backward

Naloxone

Figure 14.3 Naloxone causes unblocking in a

one-trial context-blocking experiment. During Phase 1, rats received either 15 forward or 15 backward pairings of a 30-sec tone and shock. In Phase 2, the rats were given an injection of saline or naloxone and placed in a novel context. There they received a single presentation of the tone followed by shock. The left panel (Tone Fear) shows that there was more conditioning to the forwardthan backward-paired tone and naloxone did not alter the expression of this fear. The right panel (Context Fear) shows fear conditioning to the context by the single shock. In saline-treated rats, the reduced context conditioning of the forwardtrained group relative to the backward-trained group indicates blocking. Naloxone prevented this blocking effect. (Adapted from Fanselow, M. S., & Bolles, R. C. (1979b). Triggering of the endorphin analgesic reaction by a cue previously associated with shock - reversal by naloxone. Bulletin of the Psychonomic Society, 14(2), 88–90).

increased conditioning asymptotes, thereby concluding that naloxone may function to lift the limits on the US’s ability to condition in a manner analogous to increasing the intensity of the actual shock itself. More recently, we have shown that the opioid antagonist naltrexone attenuates overshadowing—another Pavlovian phenomenon thought to occur due to limitations on the US’s ability to support conditioning (Zelikowsky & Fanselow, 2010). In overshadowing, a highly salient CS reduces conditioning to a concurrently presented low-salience CS (Pavlov, 1927),

2/15/2011 7:46:27 PM

312


again suggesting that the US is limited in the amount of conditioning it can support. However, if naltrexone is administered to an animal prior to training, overshadowing of the less salient CS is significantly attenuated (Zelikowsky & Fanselow, 2010 ). The fact that naltrexone allows for the conditioning of the low-salience CSs gives further evidence that naltrexone may work to lift the limits off of the US’s ability to condition. According to the negative-feedback model, overshadowing, like blocking, occurs because conditional analgesia is elicited. The more salient CS will have a faster rate of acquisition, and hence rapidly generates a conditional analgesia that blocks conditioning to the less salient, slow to condition, CS.

DECREMENTAL ERROR CORRECTION Initial tests of the negative-feedback model specifically addressed error correction when the expectation is less than the received reinforcer (e.g., acquisition). In this case, the error term signals increments in associative strength. Another type of error correction is when the expectation is greater than the reinforcer— an error signal that leads to decrements in responding (e.g., extinction). A programmatic series of studies by McNally and colleagues (McNally, Pigg, & Weidemann, 2004b; McNally et al., 2004a) has shown that opioid antagonists also block this latter type of error correction. McNally and colleagues (2004a, 2004b) found that there are situations in which conditional analgesia exceeds the amount needed to fully cancel the reinforcing aspects of the US (i.e., when the prediction error, (λ – ΣV), is negative). In these cases, administration of an opioid antagonist blocks these “inhibitory” forms of learning such as extinction (McNally & Westbrook, 2003) and Pavlovian overexpectation (McNally et al., 2004a). In overexpectation, two CSs that have each been independently trained with a US are presented together and reinforced with the same size US such that what is expected is “double” what is actually received and hence λ – ΣV is negative. Similarly, in extinction, the CS is repeatedly presented in the


absence of the US and hence what is expected is greater than what is received. These data fit nicely with the negativefeedback model, if one assumes that there is some baseline level of activity (resting firing rate of neurons) in ascending pain pathways under unstimulated conditions. The resting firing rate would not produce changes in conditioning alone (i.e., would not support reinforcement). On the other hand, unpredicted painful events would increase firing rate and support fear acquisition. However, if activity in the descending (analgesic) arm of the circuit was greater than needed to reduce painful input, the firing rate in the pathway should drop below the resting rate (see Fig. 14.2). Such a condition would be met, for example, when a CS is presented without a US, as is the case in extinction. Instances in which the firing rate slips below baseline would promote decreases in associative strength. Consequently, opioid antagonists that prevent analgesia would hinder such decrements in associative strength. This model is physiologically plausible because morphine not only suppresses pain-induced activity of dorsal horn neurons but also suppresses the spontaneous firing rate of these neurons (Einspahr & Piercey, 1980). The application to fear conditioning is supported by the finding that naloxone—at least under some circumstances—can prevent extinction (McNally & Westbrook, 2003). The idea of a negative-feedback model of conditioning is extremely powerful in that it offers a physiological mechanism by which perception of the US changes as conditioning progresses in a manner analogous to that described so elegantly by the Rescorla-Wagner model. This model is further emboldened by the existence of anatomically independent negative-feedback loops in other forms of Pavlovian conditioning (e.g., eyeblink conditioning; Kim, Krupa, & Thompson, 1998).

ERROR CORRECTION AND ATTENTION While we have focused on error correction as conceived by US-processing models of conditioning, it should also be noted that there is a

2/15/2011 7:46:28 PM

ERROR CORRECTION PROCESSES


Tone Fear 35 Aquisition rate score

large body of literature focused on the role of CS associability in conditioning (Mackintosh, 1975b; Pearce & Hall, 1980). While US processing models suggest that the error-correction signal is a reinforcement signal, CS processing models (e.g., Mackintosh and Pearce-Hall) suggest that error-correction signals adjust “associability,” which in turn has an effect on a constant reinforcement signal. In particular, CS-processing views of conditioning lay the success or failure of conditioning on the amount of attention the CS is or is not able to garner. Most often, the more attention paid to a CS, the more it can successfully be conditioned. Although attentional theories are not necessarily in agreement regarding the factor most likely to generate an attention-grabbing CS (i.e., the CS is a good predictor of a US, a novel predictor, or simply innately salient), they agree that conditioning depends on whether attention is paid to the CS. Thus, these theories explain phenomena such as overshadowing not in terms of US limitations, but in terms of properties of the CS. One notable advantage of associability models is that they are able to explain latent inhibition. In latent inhibition, a stimulus that has been preexposed is subsequently retarded in its ability to be conditioned (Lubow, 1973, 1989). This slower rate of acquisition can be accounted for using an associability model, which focuses on the CS and its salience, where CS pre-exposure serves to reduce the salience of a CS and hence the rate of acquisition. However, because latent inhibition occurs in the absence of a reinforcer, explaining it in terms of the negativefeedback model is problematic. Indeed, Young and Fanselow (1992) failed to block latent inhibition with an opioid antagonist (see Fig. 14.4). However, there are phenomena—one-trial blocking—that cannot be explained by associability models but can be accounted for by US-processing models. In one-trial blocking (Cole & McNally, 2007; Mackintosh, 1975a) one conditioning trial with a single stimulus is followed by a conditioning trial with a compound stimulus. The stimulus introduced in the compound is blocked. Since blocking consists of earlier training experience (the pre-compound

313

30 25 20 15 15 5 0 Latent inhibition Saline

Negative-transfer Naloxone

Figure 14.4 Naloxone blocks Hall-Pearce nega-

tive transfer, but not latent inhibition. During Phase 1, rats received either one exposure to a 64-sec tone only (latent inhibition groups) or one pairing of a 64-sec tone followed by a 1-sec lowintensity footshock (negative-transfer groups) per day over 10 days. In Phase 2, rats were given an injection of saline or naloxone and received a single tone presentation followed by a high-intensity footshock per day over 2 days. The graph displays difference scores for freezing to the tone on the first versus second day of Phase 2. Low scores indicate the slow acquisition expected of latent inhibition and Hall-Pearce negative transfer. Saline groups showed latent inhibition and negative-transfer effects. However, naloxone prevented negative transfer but left latent inhibition intact. (Adapted from Young, S. L., & Fanselow, M. S. (1992). Associative regulation of Pavlovian fear conditioning: Unconditional stimulus intensity, incentive shifts, and latent inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 18(4), 400–413).

conditioning phase), a US-processing model, such as Rescorla-Wagner, can easily account for one-trial blocking. Indeed, one-trial blocking is prevented by the administration of an opioid antagonist (Cole & McNally, 2007; Fanselow & Bolles, 1979a). However, associability models, which depend on previous experience with the CS, cannot explain one-trial blocking. Thus, it is likely that changes in both US processing and changes in CS associability contribute to Pavlovian conditioning. For example, the slow rate of learning that follows after a CS has

2/15/2011 7:46:28 PM

314


been previously conditioned with a weak US (negative-transfer; Hall & Pearce, 1979) is blocked by naloxone (Young & Fanselow, 1992). These findings are summarized in Figure 14.4. Thus, by integrating CS-associability and US-processing views of conditioning, a wide breath of Pavlovian phenomena can be accounted for. Lastly, it should be noted that one-trial overshadowing cannot be accounted for by any of these models. One-trial overshadowing (Mackintosh, 1971) is a variant of the basic overshadowing effect; however the effect is achieved with a single conditioning trial of a compound CS. The occurrence of one-trial overshadowing is problematic for US-processing models such as the negativefeedback model because negative feedback is only generated after the first conditioning trial. Similarly, associability models also require prior experience to drive interaction between stimuli. This suggests that initial competition between stimuli may be driven from a purely perceptual or attentional level. Thus, in addition to US-processing and CS-associability factors, raw attentional factors may also play an important role in Pavlovian conditioning. A number of studies have provided evidence for the role of dopamine in the regulation of attentional factors in Pavlovian phenomenon. In most of these studies, administration of a dopamine (DA) agonist often attenuates the Pavlovian phenomenon of interest. For example, amphetamine (which releases DA) has been shown to disrupt blocking (Crider, Solomon, & McMahon, 1982; Ohad, Lubow, Weiner, & Feldon, 1987) as well as overshadowing (O’Tuathaigh & Moran, 2002). Further studies have narrowed this effect down to the role of the DA D1 receptor in attentional processes, as the selective D1 agonist SKF 38393 attenuates overshadowing (O’Tuathaigh & Moran, 2002; Zelikowsky & Fanselow, 2010). Importantly, the indirect dopamine (DA) agonist D-amphetamine sulphate was shown to disrupt both blocking and overshadowing within a single study (O’Tuathaigh et al., 2003). In a separate task sensitive to attentional factors, Granon et al. (2000) showed that injecting SKF 38393 directly into the medial prefrontal cortex (mPFC) enhanced attentional performance in this task, suggesting


that dopamine in the mPFC may play a role in the regulation of attentional processing in Pavlovian conditioning. An account of Pavlovian processes that attributes conditioning on the first trial to attentional factors, subsequent trials to a negative-feedback mechanism, and previous CS exposure to an associability model, would have a good chance of encompassing many of the phenomenon that occur in Pavlovian conditioning. The fact that both the opioid antagonist naltrexone and the dopamine D1 agonist SKF 38393 attenuate Pavlovian overshadowing (albeit differently), suggests that multiple mechanisms may indeed contribute to the same Pavlovian phenomena (Zelikowsky & Fanselow, 2010). These multiple mechanisms may work hand in hand in a temporal fashion and/or may even mutually compensate for each other.

RECENT ADVANCES IN ERROR CORRECTION: DOPAMINE NEURONS While error correction–calculating circuits have been described for fear and eyeblink conditioning (Fanselow, 1998; Kim et al., 1998), recent work has suggested that in positive reinforcement learning, certain groups of neurons respond as though they detect mismatches between earned and expected rewards. More specifically, a number of studies from Schultz and collaborators have suggested that firing of midbrain dopamine neurons operate according to error-correction-type rules in the regulation of reward learning (Fiorillo et al., 2003; Hollerman & Schultz, 1998; Schultz, 1997, 1998; Schultz, Dayan, & Montague, 1997; Tobler, Dickinson, & Schultz, 2003; Tobler et al., 2005; Waelti, Dickinson, & Schultz, 2001). These studies find that burst activity of midbrain dopamine neurons—that is, the “phasic” dopamine response—can be seen following food or liquid rewards. However, if a reward is already predicted by a cue (i.e., a stimulus has been well conditioned to predict a food US), this burst activity does not occur, and if an expected reward is omitted, activity in these neurons is depressed (see Schultz, 2007 for a review).

2/15/2011 7:46:28 PM


Schultz and colleagues interpret the behavior of these neurons as demonstrative of encoding the discrepancy between a predicted reward and the reward actually received. Thus, the output of these midbrain dopamine neurons seem to behave much like an error signal—with positive errors correlated to increased activity in these neurons and negative errors with depressed activity. In Rescorla-Wagner terminology, the response of these dopamine neurons is meant to represent the surprise term (λ – ΣV). Thus, Schultz and colleagues suggest that these dopamine neurons represent unexpected reinforcers and therefore act as a signal for reinforcement. This role is consistent with the long-standing view that dopamine acts as the brain’s reward system. It also implicates dopamine in the regulation of both prediction error and attention in Pavlovian conditioning. However, there are critical outstanding issues surrounding this view. First, unlike the more fully understood fear and motor learning systems, we do not know how these neurons actually calculate error. A second issue is that after conditioning, midbrain dopamine neurons will also react to a predictive CS with a phasic response. Thus, these neurons seem to both generate an expectancy type signal (RescorlaWagner’s V term) as well as an error signal (Rescorla-Wagner’s λ – V term). However, an expectancy signal drives your response based on what you have learned (V), whereas an error signal drives your learning based on how you have responded (λ – V). These are very different actions and require different computations. How are the neurons that receive these signals to discriminate between these two different meanings? A third issue, noted by Redgrave and Gurney (2006), lies in the fact that the occurrence of the phasic dopamine response has a very short latency (70–100 ms) from stimulus onset (Schultz, 1998). So short in fact, that it occurs during an animal’s “preattentive” processing phase—in other words, before the animal could actually identify a reward and/or its value. Thus, it becomes less clear what exactly these neurons contribute. In an alternative account, Redgrave and Gurney (2006) suggest that instead of signaling


315

an unpredicted reward, the phasic dopamine response signals an animal to “reselect” an action that was immediately followed by an unpredicted biologically significant event. According to this “reselection hypothesis,” the phasic dopamine response plays much more of a causal role. It allows an agent to recognize that a particular action it performed in a particular contextual backdrop preceded an unexpected biologically salient event and hence may be a probable cause. According to this hypothesis, an animal uses the phasic dopamine signal to differentiate between events for which it is responsible from events for which it is not, regardless of any immediate reward value (Redgrave & Gurney, 2006; Redgrave, Gurney, & Reynolds, 2008). This account is further supported by experiments from Winterbauer and Balleine (2007), showing that amphetamine enhances performance on a response (lever pressing) that was followed by the delivery of a simple visual stimulus. This solidifies a role for dopamine in the reselection of a response, despite the absence of any reward contingency. Taken together, these reselection studies suggest that instead of signaling reward values, dopamine neurons may signal events that should be attended to, which dovetails nicely with our earlier discussion of the role of dopamine in selective attention. Certainly stimuli that you have learned about (V) and stimuli that signal surprise (λ – V) should be attended to. Thus, the actual profile of responding of these neurons is more in line with an attentional view.

CIRCUIT SELECTION AND ERROR CORRECTION Thus far, we have described the manner by which particular circuits in the brain operate to calculate and correct for errors. We have also discussed the behavioral implications of error correction, namely that error-correction-type rules can be used to explain a wide range of Pavlovian phenomenon (e.g., overshadowing and blocking). We covered evidence consistent with a role for US limitations and negativefeedback circuits as well as the role of attention

2/15/2011 7:46:28 PM

316


and dopamine in the regulation of these phenomena. However, we would like to take the notion of error correction one step farther. We propose that the very same error-correction rules that govern “stimulus selection” may also regulate how the brain selects circuits. The case of contextual fear memory is a particular poignant example of how such “circuit selection” may be occurring in the brain. In contextual fear conditioning, an animal learns to fear an environment in which it has received an aversive US (e.g., footshock). The memory of the context is initially stored in the hippocampus for a period of time, as lesions of the hippocampus immediately following contextual fear conditioning result in a complete loss of memory (Anagnostaras, Maren, & Fanselow, 1999; Kim & Fanselow, 1992). However, it has been shown that if damage to the hippocampus is sustained prior to training, animals are able to condition fear to a context (Frankland, Cestari, Filipkowski, McDonald, & Silva, 1998; Maren, Aharonov, & Fanselow, 1997; Wiltgen, Sanders, Anagnostaras, Sage, & Fanselow, 2006). Such data suggest that hippocampal damage produces retrograde amnesia but does not necessarily produce anterograde amnesia. It appears that although an animal may “normally” use its hippocampus to learn and store a representation of a place, in the absence of the hippocampus animals can compensate and form a representation of that place. Thus, when the primary, hippocampus-based circuit is compromised, an alternate circuit may be “selected” by the brain (Fanselow, 2010). However, this alternate circuit does not learn if the hippocampus is already engaged in learning. The interesting question remains as to the source and nature of this compensation. Retrograde amnesia studies tell us that the hippocampus— and not the alternate circuit—will normally form a configural representation of a place (see Fanselow, 2000). On the other hand, anterograde studies tell us that the alternate circuit may be utilized when the hippocampus is compromised. Just as the hippocampus has been shown to be important for context learning and memory, the basolateral amygdala (BLA) has been found


to be vital for fear learning and memory (e.g., see Fanselow & LeDoux, 1999; Gale et al., 2004). Similar to the case with the hippocampus, a rodent with lesions or inactivation of the BLA may compensate and demonstrate fear learning and memory, provided that a strong regimen is used for training (Maren, 1999; Ponnusamy, Poulos, & Fanselow, 2007; Poulos et al., 2010). The same pattern holds for fear learning by subnuclei within the BLA complex (AngladaFigueroa & Quirk, 2005). Thus, fear learning and memory, like context learning and memory, seem to follow a similar pattern: A particular structure and circuit are normally used, but if they are damaged prior to—but not subsequent to—learning, then an alternate pathway may compensate. A remaining question is why the alternate circuit does not learn when the primary circuit is learning. One solution is that perhaps circuits, just like regular discrete cues, behave according to associative learning rules such as those that govern Pavlovian overshadowing. According to this idea, “salient” circuits would be selected for conditioning, while others would be overshadowed in a manner similar to discrete cues (see Fanselow, 2010). And, as is the case with an overshadowed discrete cue, an overshadowed circuit may be given the chance to learn if the limits on the amount of learning normally supported (i.e., λ) are removed or lifted. Thus, because an opioid antagonist such as naltrexone attenuates the overshadowing of a cue, by presumably lifting the limits on a US’s ability to condition (Zelikowsky & Fanselow, 2010), the same effect should be translatable to the selection of an “overshadowed” circuit. Taking the rules of associative learning and competition and applying them more broadly to circuit selection has powerful implications. Notably, it suggests that circuits, like discrete stimuli, can be learned about, despite being weaker or less salient, provided the amount of learning that is supported can be increased. This has important practical repercussions regarding patients suffering from some form of brain damage in which a primary pathway for learning and memory is compromised. It also has theoretical implications in that it suggests the depth and

2/15/2011 7:46:28 PM


breadth of error-correction is quite wide. Error correction is not simply a mechanism for enabling fine motor movements or predicting a reward; it forms the very basis and framework for how of our brains select appropriate circuits for specific types of learning.

CONCLUSIONS In this chapter, we have tried to present a picture of how error-correction processes can drive and mold the way we learn and behave. From the idea that a key component of defense is the successful inhibition of recuperation (PDR model), to more mechanistic notions of negative feedback and dopamine signaling, it seems that conditioning is driven incrementally by discrepancies between what actually happens in our environment and what we expect to happen. Whether this discrepancy is more sensitive to factors such as attention, environmental limitations on what can be learned, or what direction an error occurs (i.e., incremental vs. decremental), the basic idea remains the same: Our behavior is a result of what we expect about our environment compared to what we do not. In this chapter, we have emphasized particular mechanisms by which such errors may be calculated (e.g., analgesiamediated negative feedback or dopamine signaling and reward). Additionally, we suggest that error correction may in fact comprise a much more global mechanism. Namely, that the rules that underlie error correction are ubiquitous in the brain; they are used by specific brain circuits to perform particular functions, and they are also used by the brain overall to select circuits for more complex and integrated functions.

REFERENCES Akil, H., Mayer, D. J., & Liebeskind, J. C. (1972a). Comparison chez le rat entre l’analgesie induite par stimulation de la substance grise peri-aqueducale et l’analg&ie mor- phinique. [Comparison in the rat between analgesia induced by stimulation of periacqueducal gray matter and morphine analgesia]. Comptes Rendus Hebdomadaires des Seances de l Academie des Sciences D, 274(26), 3603–3605.


317

Akil, H., Mayer, D. J., & Liebeskind, J. C. (1972b). [Monoaminergic mechanisms involved in analgesia caused by stimulation of the brain stem in rats]. Journal of Physiology (Paris), 65, Suppl:185A. Akil, H., Mayer, D. J., & Liebeskind, J. C. (1976). Antagonism of stimulation-produced analgesia by naloxone, a narcotic antagonist. Science, 191(4230), 961–962. Anagnostaras, S. G., Maren, S., & Fanselow, M. S. (1999). Temporally graded retrograde amnesia of contextual fear after hippocampal damage in rats: within-subjects examination. Journal of Neuroscience, 19(3), 1106–1114. Anglada-Figueroa, D., & Quirk, G. J. (2005). Lesions of the basal amygdala block expression of conditioned fear but not extinction. Journal of Neuroscience, 25(42), 9680–9685. Basbaum, A. I., & Fields, H. L. (1984). Endogenous pain control systems: Brainstem spinal pathways and endorphin circuitry. Annu Rev Neurosci, 7, 309–338. Bolles, R. C. (1970). Species-specific defence reactions and avoidance learning. Psychological Review, 77, 32–48. Bolles, R. C., & Fanselow, M. S. (1980). A perceptual-defensive-recuperative model of fear and pain. Behavioral and Brain Sciences, 3(2), 291–301. Bolles, R. C., & Fanselow, M. S. (1982). Endorphins and behavior. Annual Review of Psychology, 33, 87–101. Cannon, J. T., Liebeskind, J. C., & Frenk, H. (1978). Neural and neurochemical mechanisms of pain inhibition. In R. A. Sternbach (Ed.), The psychology of pain (pp. 27–47). New York, NY: Raven Press. Chance, W. T., White, A. C., Krynock, G. M., & Rosencrans, J. A. (1978). Conditional fear-induced antinociception and decreased binding of [3H]N-leu-enkephalin to rat brain. Brain Research, 141(2), 371–374. Cole, S., & McNally, G. P. (2007). Opioid receptors mediate direct predictive fear learning: Evidence from one-trial blocking. Learning and Memory, 14(4), 229–235. Crider, A., Solomon, P. R., & McMahon, M. A. (1982). Disruption of selective attention in the rat following chronic d-amphetamine administration: Relationship to schizophrenic attention disorder. Biological Psychiatry, 17(3), 351–361. Einspahr, F. J., & Piercey, M. F. (1980). Morphine depresses dorsal horn neuron responses to

2/15/2011 7:46:28 PM

318


controlled noxious and non-noxious cutaneous stimulation. Journal of Pharmacology and Experimental Therapeutics, 213(3), 456–461. Fanselow, M. S. (1981). Naloxone and Pavlovian fear conditioning. Learning and Motivation, 12, 398–419. Fanselow, M. S. (1986). Conditioned fear-induced opiate analgesia: A competing motivational state theory of stress analgesia. Annals of the New York Academy of Science, 467, 40–54. Fanselow, M. S. (1998). Pavlovian conditioning, negative feedback, and blocking: Mechanisms that regulate association formation. Neuron, 20(4), 625–627. Fanselow, M. S. (2000). Contextual fear, gestalt memories, and the hippocampus. Behavioural Brain Research, 110(1–2), 73–81. Fanselow, M. S. (2010). From contextual fear to a dynamic view of memory systems. Trends in Cognitive Sciences, 14, 7–15. Fanselow, M. S., & Baackes, M. P. (1982). Conditioned fear-induced opiate analgesia on the formalin test: Evidence for two aversive motivational systems. Learning and Motivation, 13, 200–221. Fanselow, M. S., & Bolles, R. C. (1979a). Naloxone and shock-elicited freezing in the rat. Journal of Comparative and Physiological Psychology, 93(4), 736–744. Fanselow, M. S., & Bolles, R. C. (1979b). Triggering of the endorphin analgesic reaction by a cue previously associated with shock - reversal by naloxone. Bulletin of the Psychonomic Society, 14(2), 88–90. Fanselow, M. S., Calcagnetti, D. J., & Helmstetter, F. J. (1989). Role of mu and kappa opioid receptors in conditional fear-induced analgesia: The antagonistic actions of nor-binaltorphimine and the cyclic somatostatin octapeptide, Cys2Tyr3Orn5Pen7-amide. Journal of Pharmacology and Experimental Therapeutics, 250(3), 825–830. Fanselow, M. S., & Helmstetter, F. J. (1988). Conditional analgesia, defensive freezing, and benzodiazepines. Behavioral Neuroscience, 102(2), 233–243. Fanselow, M. S., & LeDoux, J. E. (1999). Why we think plasticity underlying Pavlovian fear conditioning occurs in the basolateral amygdala. Neuron, 23(2), 229–232. Fanselow, M. S., & Lester, L. S. (1988). A functional behavioristic approach to aversively motivated behavior: Predatory imminence as a determinant


of the topography of defensive behavior. In R. C. Bolles & M. D. Beecher (Eds.), Evolution and learning (pp. 185–212). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299(5614), 1898–1902. Frankland, P. W., Cestari, V., Filipkowski, R. K., McDonald, R. J., & Silva, A. J. (1998). The dorsal hippocampus is essential for context discrimination but not for contextual conditioning. Behavioral Neuroscience, 112(4), 863–874. Gale, G. D., Anagnostaras, S. G., Godsil, B. P., Mitchell, S., Nozawa, T., Sage, J. R., . . . Fanselow, M. S. (2004). Role of the basolateral amygdala in the storage of fear memories across the adult lifetime of rats. Journal of Neuroscience, 24(15), 3810–3815. Galli, G., Hancock, K., Hoschler, K., DeVos, J., Praus, M., Bardelli, M., . . .Stephenson, I. (2009). Fast rise of broadly cross-reactive antibodies after boosting long-lived human memory B cells primed by an MF59 adjuvanted prepandemic vaccine. Proceedings of the National Academy of Sciences USA, 106(19), 7962–7967. Granon, S., Passetti, F., Thomas, K. L., Dalley, J. W., Everitt, B. J., & Robbins, T. W. (2000). Enhanced and impaired attentional performance after infusion of D1 dopaminergic receptor agents into rat prefrontal cortex. Journal of Neuroscience, 20(3), 1208–1215. Hall, G., & Pearce, J. M. (1979). Latent inhibition of a CS during CS-US pairings. Journal of Experimental Psychology: Animal Behavior Processes, 5(1), 31–42. Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1(4), 304–309. Kamin, L. J. (1968). “Attention-like” processes in classical conditioning. In M. R. Jones (Ed.), Miami Symposium on the Prediction of Behavior: Aversive stimulation (pp. 9–33). Miami, FL: University of Miami Press. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. C. R. M. Church (Ed.), Punishment and aversive behavior (pp. 9–31). New York, NY: Appleton-Century-Crofts. Kim, J. J., & Fanselow, M. S. (1992). Modalityspecific retrograde amnesia of fear. Science, 256(5057), 675–677.

2/15/2011 7:46:28 PM


Kim, J. J., Krupa, D. J., & Thompson, R. F. (1998). Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science, 279(5350), 570–573. Kosterlitz, H. W., & Hughes, J. (1975). Some thoughts on the significance of enkephalin, the endogenous ligand. Life Sciences, 17(1), 91–96. Lewis, J. W., Cannon, J. T., & Liebeskind, J. C. (1980). Opioid and nonopioid mechanisms of stress analgesia. Science, 208(4444), 623–625. Lewis, J. W., Cannon, J. T., Stapleton, J. M., & Liebeskind, J. C. (1980). Stress activates endogenous pain-inhibitory systems: Opioid and non-opioid mechanisms. Proceedings of the Western Pharmacology Society, 23, 85–88. Lewis, J. W., Slater, S. J., Hall, J. L., Terman, G. W., & Liebeskind, J. C. (1982). Chronic stress enhances morphine analgesia in the rat. Proceedings of the Western Pharmacology Society, 25, 137–140. Lewis, J. W., Tordoff, M. G., Sherman, J. E., & Liebeskind, J. C. (1982). Adrenal medullary enkephalin-like peptides may mediate opioid stress analgesia. Science, 217(4559), 557–559. Lubow, R. E. (1973). Latent inhibition. Psychology Bulletin, 79(6), 398–407. Lubow, R. E. (1989). Latent inhibition and conditioned attention theory. Cambridge, England: Cambridge University Press. Mackintosh, N. J. (1971). An analysis of overshadowing and blocking. Quarterly Journal of Experimental Psychology, 23, 118–125. Mackintosh, N. J. (1975a). Blocking of conditioned suppression: Role of the first compound trial. Journal of Experimental Psychology: Animal Behavior Processes, 1(4), 335–345. Mackintosh, N. J. (1975b). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298. Maren, S. (1999). Neurotoxic basolateral amygdala lesions impair learning and memory but not the performance of conditional fear in rats. Journal of Neuroscience, 19(19), 8696–8703. Maren, S., Aharonov, G., & Fanselow, M. S. (1997). Neurotoxic lesions of the dorsal hippocampus and Pavlovian fear conditioning in rats. Behavioural Brain Research, 88(2), 261–274. Mayer, D. J., Wolfle, T. L., Akil, H., Carder, B., & Liebeskind, J. C. (1971). Analgesia from electrical stimulation in the brainstem of the rat. Science, 174(16), 1351–1354. McNally, G. P., Johnston, I. N., & Westbrook, R. F. (2000). A peripheral, intracerebral, or intrathecal


319

administration of an opioid receptor antagonist blocks illness-induced hyperalgesia in the rat. Behavioral Neuroscience, 114(6), 1183–1190. McNally, G. P., Pigg, M., & Weidemann, G. (2004a). Blocking, unblocking, and overexpectation of fear: A role for opioid receptors in the regulation of Pavlovian association formation. Behavioral Neuroscience, 118(1), 111–120. McNally, G. P., Pigg, M., & Weidemann, G. (2004b). Opioid receptors in the midbrain periaqueductal gray regulate extinction of pavlovian fear conditioning. Journal of Neuroscience, 24(31), 6912–6919. McNally, G. P., & Westbrook, R. F. (2003). Opioid receptors regulate the extinction of Pavlovian fear conditioning. Behavioral Neuroscience, 117(6), 1292–1301. Miller, N. E. (1948). Studies of fear as an acquirable drive .1. Fear as motivation and fear-reduction as reinforcement in the learning of new responses. Journal of Experimental Psychology, 38(1), 89–101. Miller, N. E. (1951). Learnable drives and rewards. In S. S. Stevens (Ed.). Handbook of experimental psychology, 435–472. New York: John Wiley & Sons. Mowrer, O. H. (1939). A stimulus-response analysis of anxiety and its role as a reinforcing agent. Psychological Review, 46(6), 553–565. Mowrer, O. H. (1951). Two-factor learning theory: Summary and comment. Psychological Review, 58(5), 350–354. O’Tuathaigh, C. M., & Moran, P. M. (2002). Evidence for dopamine D(1) receptor involvement in the stimulus selection task: Overshadowing in the rat. Psychopharmacology (Berl), 162(3), 225–231. O’Tuathaigh, C. M., Salum, C., Young, A. M., Pickering, A. D., Joseph, M. H., & Moran, P. M. (2003). The effect of amphetamine on Kamin blocking and overshadowing. Behavioural Pharmacology, 14(4), 315–322. Ohad, D., Lubow, R. E., Weiner, I., & Feldon, J. (1987). The effects of amphetamine on blocking. Psychobiology, 15, 137–143. Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London: Oxford University Press. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87(6), 532–552.

2/15/2011 7:46:28 PM

320


Pert, C. B., & Snyder, S. H. (1973a). Opiate receptor: Demonstration in nervous tissue. Science, 179(77), 1011–1014. Pert, C. B., & Snyder, S. H. (1973b). Properties of opiate-receptor binding in rat brain. Proceedings of the National Academy of Sciences USA, 70(8), 2243–2247. Ponnusamy, R., Poulos, A. M., & Fanselow, M. S. (2007). Amygdala-dependent and amygdalaindependent pathways for contextual fear conditioning. Neuroscience, 147(4), 919–927. Poulos, A. M., Ponnusamy, R., Dong, H. W., & Fanselow, M. S. (2010). Compensation in the neural circuitry of fear conditioning awakens learning circuits in the bed nuclei of the stria terminalis. Proc Natl Acad Sci U S A, 107(33), 14881–14886. Redgrave, P., & Gurney, K. (2006). The shortlatency dopamine signal: A role in discovering novel actions? Nature Reviews Neuroscience, 7(12), 967–975. Redgrave, P., Gurney, K., & Reynolds, J. (2008). What is reinforced by phasic dopamine signals? Brain Research Review, 58(2), 322–339. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In W. F. Prokasy (Ed.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton-Century-Crofts. Reynolds, D. V. (1969). Surgery in the rat during electrical analgesia induced by focal brain stimulation. Science, 164(878), 444–445. Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms. Current Opinions in Neurobiology, 7(2), 191–197. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1–27. Schultz, W. (2007). Behavioral dopamine signals. Trends in Neuroscience, 30(5), 203–210. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. Sherman, J. E., & Liebeskind, J. C. (1980). An endorphinergic, centrifugal substrate of pain


modulation: Recent findings, current concepts, and complexities. Research Publications Association for Research in Nervous and Mental Disease, 58, 191–204. Tobler, P. N., Dickinson, A., & Schultz, W. (2003). Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. Journal of Neuroscience, 23(32), 10402–10410. Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005). Adaptive coding of reward value by dopamine neurons. Science, 307(5715), 1642–1645. Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412(6842), 43–48. Wiertelak, E. P., Roemer, B., Maier, S. F., & Watkins, L. R. (1997). Comparison of the effects of nucleus tractus solitarius and ventral medial medulla lesions on illness-induced and subcutaneous formalin-induced hyperalgesias. Brain Research, 748(1–2), 143–150. Wieseler-Frank, J., Maier, S. F., & Watkins, L. R. (2005). Immune-to-brain communication dynamically modulates pain: Physiological and pathological consequences. Brain, Behavior, and Immunity, 19(2), 104–111. Wiltgen, B. J., Sanders, M. J., Anagnostaras, S. G., Sage, J. R., & Fanselow, M. S. (2006). Context fear learning in the absence of the hippocampus. Journal of Neuroscience, 26(20), 5484–5491. Winterbauer, N. E., & Balleine, B. W. (2007). The influence of amphetamine on sensory and conditioned reinforcement: Evidence for the re-selection hypothesis of dopamine function. Front Integr Neurosci, 1, 9. Young, S. L., & Fanselow, M. S. (1992). Associative regulation of Pavlovian fear conditioning: Unconditional stimulus intensity, incentive shifts, and latent inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 18(4), 400–413. Zelikowsky, M., & Fanselow, M. S. (2010). Opioid regulation of Pavlovian overshadowing in fear conditioning. Behav Neurosci, 124(4), 510–519.

2/15/2011 7:46:28 PM

CHAPTER 15 Incentives in the Modification and Cessation of Cigarette Smoking Edwin B. Fisher, Leonard Green, Amanda L. Calvert, and Russell E. Glasgow

We review research on the effectiveness of incentives in general health promotion, and in interventions for smoking and other drug addictions in particular. Consistent with basic principles of learning and reinforcement, and from a behavioral economic perspective, we find that (1) incentives are effective in encouraging smoking cessation and other health behaviors; (2) incentives are effective while in place but not after they are terminated; and (3) nonsmoking may be encouraged by increasing the availability of reinforcing activities that substitute for the reinforcement from nicotine. Incentives may be especially effective when smoking cessation is a priority for a specific period of time, such as during pregnancy. Incentive programs appear to influence otherwise “hard-to-reach” groups. The impact of incentives is enhanced when implemented in the context of broader programs promoting smoking cessation, and incentives applied to populations may be cost efficient by achieving modest effects for large numbers of individuals. Basic principles of reinforcement and the use of incentives understood within a behavioral economic framework should continue to inform public-health interventions and may lead to new insights into effective approaches for influencing health behavior.

INTRODUCTION The good news is that the prevalence of smoking among adults has declined from 42% in 1965 to 20.8% in 2008 (Centers for Disease Control [CDC], 2008). Of all adults in this country who have smoked, 49% have quit (CDC, 2002). The bad news is that despite these impressive strides, one-fifth of the adult population continues to smoke, and the percentage of teenagers who smoke was also 20% in 2007 (Centers for Disease Control Office on Smoking and Health, 2007). Thus, this widespread, severe public health problem will persist for at least another generation. Moreover, the prevalence of smoking remains especially high in several groups of special concern: low-income groups, minorities, those with smoking-related disease, and low-income women of childbearing age (Gilpin & Pierce, 2002).

The need for focused and vigorous approaches to reducing smoking among these groups remains especially acute. Early research on the effectiveness of smoking-cessation programs often considered the role of incentives. A review of worksite smoking programs by Orleans and Shipley (1982) included a number of case-study and anecdotal reports of successful incentive programs. They advocated controlled investigations of how incentives might enhance worksite programs. The investigation of incentives also was encouraged by a more general recognition of the role that positive reinforcement plays in smoking (Pomerleau, Collins, Shiffman, & Pomerleau, 1993), including how genetics may influence the effectiveness of nicotine metabolism as a reinforcer (Lerman et al., 1999). Additionally, the field of behavioral economics has increased our understanding of

321


2/15/2011 7:46:42 PM

322


how different incentives for drug use interact with each other and with the contexts in which they operate (e.g., Bickel & McLellan, 1996; Kagel, Battalio, & Green, 1995). These literatures raise the prospect of more rational and effective incentive programs to encourage nonsmoking and smoking cessation. Since Orleans and Shipley’s review (1982), however, research on the use and impact of incentives in smoking cessation has been modest. We conducted a series of Medline searches of journal articles published between 1966 and 2009, using cognates of reinforcer, contingency, reward, incentive, lottery, and contest. The vast majority of papers identified that also dealt with smoking were not intervention studies. Despite this limited research on incentives in smoking cessation, there are substantial literatures on incentives in general health promotion and in treating other types of drug abuse. Accordingly, we include in this chapter a brief review of incentives in other areas of health promotion along with review of the use of incentives in treating addictions to other drugs as well as to nicotine. There are two broad approaches in the use of incentives for promoting appropriate health behavior. The first approach includes incentives applied directly to the target behavior, for example, providing reinforcers for maintaining abstinence from smoking for several days. A second approach entails incentives for other behaviors that may compete with the target behavior, what in behavioral terms is referred to as “differential reinforcement of alternative behavior.” Consequently, the first part of this chapter addresses incentives applied directly to behaviors, and the second addresses incentives applied indirectly.

INCENTIVES APPLIED DIRECTLY TO BEHAVIOR Following a brief presentation of the use of incentives in health promotion in general and in the treatment of drug use other than smoking, we discuss the use of incentives in smoking cessation.


Incentives in Health Promotion

For some time, reports have indicated that incentives may be successful in a wide variety of areas of health promotion: child immunizations (Achat, McIntyre, & Burgess, 1999) and vaccinations (LeBaron, Starnes, Dini, Chambliss, & Chaney, 1998); adherence to antihypertensive medications (Feldman, Bacher, Campbell, Drover, & Chockalingman, 1998); return for reading of tuberculosis (TB) skin tests among drug users (Fitzgerald et al., 1999; Malotte, Hollingshead, & Rhodes, 1999); compliance with TB drug regimens among homeless adults (Tulksy et al., 2004); adherence to the hepatitis B vaccine regimen among injection drug users (Seal et al., 2003); promoting mammography (Janz et al., 1997); maintenance of breast self-examination (Solomon et al., 1998); reducing loss to follow-up among women with abnormal Pap tests (Marcus et al., 1998); and, of pertinence to the present chapter, reducing carbon monoxide levels (levels of carbon monoxide correlate with tobacco smoking) among individuals with chronic obstructive pulmonary disease (Crowley, MacDonald, Zerbe, & Petty, 1991). Based on results reported in individual studies and findings from other systematic reviews and meta-analyses, recent papers (Marteau, Ashcroft, & Oliver, 2009; Sutherland, Christianson, & Leatherman, 2008) have summarized the evidence for the effectiveness of incentives in health promotion. A comprehensive review of studies evaluating the role of incentives on risky behaviors, preventive care, and adherence to recommended treatment (Sutherland et al., 2008) concluded: “The findings of studies reviewed in this article suggest that financial incentives, even rather small ones, can influence health behaviors” (p. 74S). This conclusion is probably an understatement. The review documents evidence for the effects of incentives in exercise promotion, improving lipid metabolism, having a mammogram (with the odds ratio for those given incentives being 2.7; that is, the likelihood of having a mammogram was 2.7 times greater among those offered incentives

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

than among those not offered incentives) (Stone et al., 2002), follow-up after positive PAP tests, screening for colon cancer, screening and treatment for tuberculosis, prenatal and postnatal care, as well as outpatient HIV testing and attendance at classes addressing HIV prevention (see pp. 63S–73S). Providing incentives for immunizations has produced especially clear results: “In all reviews, patient-targeted incentives, used alone or in combination with other interventions, were found to be effective in increasing uptake of immunizations in both children and adults” (p. 66S). In particular, a meta-analysis that included 81 studies of incentives for immunizations reported an odds ratio of 3.4 for incentives (Stone et al., 2002). The review evaluated price reductions and reductions in copays among incentives, including, for example, price reductions that increased choices of healthy foods (p. 43S) or reductions of out-of-pocket costs that increased the likelihood of getting vaccinated (p. 66S). The use of incentives for producing weight loss has provided mixed results: “The evidence on the effect of incentives on weight loss, either in the community or at worksites, is less conclusive” than that for other health behaviors (Sutherland et al., 2008, p. 65S). As noted in reviews, methodological problems have contributed to this lack of conclusiveness, especially the use of designs in which incentives are combined with other interventions (e.g., behavioral counseling, health-promotion classes, or selfmonitoring of weight), complicating the evaluation of the effect of the incentive itself. Incentives do not preclude but rather may complement other program components. For example, either a personal trainer or monetary incentives were effective in increasing exercise in a weight-management program, and the combination was more effective than either alone (Jeffery, Wing, Thorson, & Burton, 1998). A similar pattern emerged from a review of programs designed to promote adherence to TB treatment. The odds ratios comparing program components to control conditions were 1.6 for monetary incentives, 1.2 for health education, and 2.4 for the combination of the two (Volmink & Garner, 1997).


323

Incentives for Drug-Use Problems Other Than Smoking

Various forms of treatment programs have used incentives in treating problems other than cigarette smoking. Some of these include contingency contracting, in which the patient and the treatment provider agree that specified consequences (i.e., incentives) will be contingent upon appropriate behaviors (Prendergast, Podus, Finney, Greenwell, & Roll, 2006; Silverman et al. 1996); vouchers retrievable for goods and services (Bigelow, Brooner, & Silverman, 1998; Jones, Haug, Silverman, Stitzer, & Svikis, 2001; Plebani et al., 2006; Robles et al., 2000); and the receipt of privileges, such as the freedom to take doses of methadone at home rather than at a clinic (contingent on drug-free urine tests) (Chutuape, Silverman, & Stitzer, 1999a). Additional research has demonstrated the benefits of using escalating schedules of reinforcement (e.g., Roll & Shoptaw, 2006; Silverman, Chutuape, Bigelow, & Stitzer, 1997). In escalating schedules, the criterion for earning the next reinforcer is higher than that for the previous reinforcer. Using this type of schedule to deliver reinforcers makes it more likely that individuals will make contact with the reinforcer early in treatment, a variable that has been shown to be related to treatment success (Kirby, Marlowe, Festinger, Lamb, & Platt, 1998). Moreover, because the number of reinforcers delivered decreases with time, an escalating schedule may also be cost efficient because it delivers more incentives early in the program, when they are likely to have a greater effect, and fewer later on. A set of several incentives among which individuals can choose can also serve as an extra incentive. Choice between a take-home dose of methadone or a voucher worth $25.00 contingent on drug-free urine tests resulted in more drug-free urine samples, greater latencies to drug-positive urine samples, and longer sustained abstinence than did standard care, which lacked these features (Chutuape, Silverman, & Stitzer, 1999b). As with health promotion in general, incentives do not preclude the effective use of other

2/15/2011 7:46:43 PM

324


interventions. Contingent delivery of vouchers significantly improved success when added to a drug-treatment program that already included a system of contingent privileges for such things as take-home medications and individual counseling (Iguchi, Belding, Morral, Lamb, & Husband, 1997). The literature on incentives in the treatment of drug abuse reveals an important finding: Incentives may be especially effective for those who are resistant to treatment or otherwise hard to reach. In one study, for example, individuals who had achieved fewer than 4 out of 13 weeks of cocaine abstinence in a methadone maintenance program were considered treatment resistant and eligible for a study of voucher magnitude (Silverman, Chutuape, Bigelow, & Stitzer, 1999). In counterbalanced order, the individuals received three, 9-week programs that differed in the total dollar value of the vouchers earned: $0, $382, or $3480. The $3480 total incentive amount resulted in 10 of the 22 patients (45%) achieving abstinence in ≥ 4 out of a possible 9 weeks. Only one patient in the $382 phase and none in the $0 phase achieved more than 2 weeks of cocaine abstinence. The high reward magnitude condition also resulted in a significantly higher percentage of cocaine-negative urine samples (p < 0.01). In addition to showing how incentives may be effective in reaching otherwise treatment-resistant individuals, the study also demonstrated another important finding, namely the importance of amount of reinforcement. Incentives for Smoking Cessation

Initial work by Stitzer, Bigelow, and their colleagues demonstrated that monetary incentives could reduce smoking (Stitzer & Bigelow, 1982). Importantly, incentive effects were specific to the contingencies imposed, such as with contingencies on carbon monoxide (CO) in expired air, an indicator of recent smoking. When the target CO level was 8 ppm or less, 45% of participants reduced their CO to that level, in comparison to 0% when the target CO was 16 ppm (Henningfield, Stitzer, & Griffiths, 1980; Stitzer & Bigelow, 1985). (It is to be noted that 8 ppm or


lower is generally taken as indicative of abstinence.) Having shown that smoking could be brought under the control of incentives, Stitzer, Rand, Bigelow, and Mead (1986) secured short-term smoking abstinence with payments contingent on reduction and cessation in conjunction with worksite and home CO monitoring. Extending these findings, contingent payment ($4 twice weekly) led to greater abstinence over 3 months than noncontingent payment (Rand, Stitzer, Bigelow, & Mead, 1989). These results were obtained without any provision of cessation strategies, coping skills training, or programmed social reinforcers to the participants, thus underscoring the positive impact that incentives have on behavior change. A critically important finding, one that will be emphasized in this chapter, is that incentives appear to be effective only while they remain in effect. Once the incentive is discontinued, the change in behavior typically ceases. A review of the literature conducted by the Cochrane Collaboration on smoking-cessation programs that use incentives and competitions found these programs to be effective, but only while they are in force (Cahill & Perera, 2008). Long-term maintenance of behavior change requires continuation of incentives, either through continuation of programs providing them or through generalizing program-specific reinforcers to those occurring naturally. In addition to modifying smoking among users of other drugs, incentives also have been shown to reduce smoking among schizophrenics (Roll, Higgins, Steingard, & McGinley, 1998). This finding is important because of the heightened prevalence of and difficulty of treating smoking among patients with schizophrenia as well as depression (Covey, Glassman, & Stetner, 1997; El-Khorazaty et al., 2007) and other psychological problems (Hughes, Hatsukami, Mitchell, & Dahlgren, 1986). Clinical Impact

The studies reviewed were designed to examine whether incentives would influence smoking, but they did not address the issue of whether clinically significant, sustained changes in smoking would be achieved. A study by Fortman and

2/15/2011 7:46:43 PM


Killen (1995) demonstrated the potential contributions of incentives in smoking cessation. Participants who had quit for at least 24 hours were recruited by random phone surveys for a study of self-help materials and nicotine gum. All participants were offered $100 if they were able to remain abstinent at 6 months follow-up. Remarkably, even a control group that received neither self-help materials nor nicotine gum, but did receive $100 if they remained abstinent for 6 months, achieved 20% and 16% abstinence at 6- and 12-month follow-up, respectively. Of course, the sample was a volunteer sample of those who had already quit (albeit for only a day). However, the proportion of smokers identified from the random phone surveys who qualified for and volunteered to participate in the study—14.6% (1,044 out of 7,135)—was still much higher than the percentage of smokers who volunteer for other, non-incentives-based treatment programs. This finding suggests that the $100 incentive was quite influential in promoting both enrollment as well as cessation among those who joined. More recently, Volpp and colleagues studied the effect of incentives on smoking cessation in a population of lower-income smokers in a Veterans Administration hospital (Volpp et al., 2006) and middle-income employees in a worksite-based cessation program (Volpp et al., 2009). Results from these randomized, controlled trials showed a significantly lower rate of smoking at 1 and 12 months following program completion, respectively, for the groups that were given incentives contingent on smoking cessation. Return of Deposit

A popular form of the use of incentives to change behavior has been requiring a deposit of money or valued possessions for entry into a program and then returning the deposit contingent on progress in the program. Paxton (1980) showed that return of monetary deposits increased the efficacy of a smoking-cessation program, and that, consistent with the literature on reinforcement effects, the amount of deposit returned (although not the frequency by which deposits were returned) increased the short-term impact on smoking abstinence (Paxton, 1981).


325

An obvious concern with this type of incentive implementation is whether requiring a deposit for entry into a program reduces participation. However, collecting a cash deposit in four, weekly installments rather than one initial payment increased participation rates with no lesser impact on abstinence from smoking (Paxton, 1983). In a study of the trade-off between increased efficacy and decreased participation (Jeffery, Hellerstedt, & Schmid, 1990), participants either paid $5 to enter a 6-month correspondence program or gave a $60 deposit, one-sixth of which was refunded for each month of smoking abstinence. In terms of participation, the version with the $5 entrance fee was far more popular (in terms of enrollment) than that with the $60 deposit by a ratio of about 5 to 1. However, abstinence at 6-month follow-up in the return-of-deposit condition was 20% versus 9% in the $5 condition. A similar pattern of results was found in a second group of individuals recruited for a weight-loss program in which the return of deposit was based on preset monthly weight-loss goals. Contests and Lotteries

As would be expected from research on the effects of extinction on behavior, contests and lotteries have been shown to have appreciable impact on abstinence while those contingencies are in place, but lesser impact on long-term abstinence from smoking after the contingencies have been removed (Matson, Lee, & Hopp, 1993). For example, a “Quit-to-win” contest achieved abstinence rates of 56%, 27%, and 21% at 6-week, 6-month, and 12-month follow-up evaluations (Leinweber, Macdonald, & Campbell, 1994). As with return-of-deposit programs, evaluation of lotteries and contests also must examine effects on both participation and efficacy (Matson et al., 1993). In one study, adding competitions to worksite smoking-cessation programs increased participation without reducing the cessation rate among those who participated. Thus, the competition worksites achieved cessation among an estimated 16% of all smoking employees, versus 7% in worksites that did not incorporate competitions (Klesges, Vasey, &

2/15/2011 7:46:43 PM

326


Glasgow, 1986). However, participation and cessation rates may diverge, as demonstrated in a community study that compared participation and cessation rates among those receiving smoking-cessation classes, self-help materials, or contests (Altman, Flora, Fortmann, & Farquhar, 1987). The smoking-cessation classes achieved the highest cessation rates but reached the fewest people and were the least cost efficient. The selfhelp materials had the lowest cessation rate, percentage-wise, but reached the largest numbers of people and were more cost efficient than the classes. Finally, the contests fell in between the other two approaches (i.e., the smokingcessation classes and self-help materials) in cessation rate, numbers of participants, and cost efficiency. A program for smokers with chronic obstructive pulmonary disease demonstrated the importance of the amount of incentive, the potential of combining several different types of incentives, and the utility of variation in procedures to enhance their salience (Crowley et al., 1991). When up to three public lottery tickets were contingent on CO levels below 10 ppm, there was no change in smoking behavior from baseline. When the incentive was increased to five lottery tickets combined with the use of nicotine gum, CO levels were initially reduced but recovered over time. The lottery tickets were eventually presented contingent on reduced CO levels but following random checks of CO levels (i.e., a variable ratio, VR, reinforcement schedule). This resulted in CO levels that were abruptly reduced and remained low as long as the contingencies remained in place. A VR schedule is one that produces high, constant rates of responding, in part because the organism cannot predict which response will produce reinforcement. The VR schedule may be beneficial in this type of program because participants need to maintain abstinence throughout the duration of the program in order to receive the lottery tickets when they are (unpredictably) made available.

in smoking in California and Massachusetts (CDC, 1996; Pierce et al., 1998). Because such an approach reaches entire populations, even a small impact can have substantial benefit. In California, reductions in consumption attributable to a $0.25 tax increase were greater than those attributable to an anti-smoking media campaign (Hu, Sung, & Keeler, 1995). (Such effects, of course, are dependent on the tax amount and the level of investment in media campaigns.) Uptake among youth appears especially price sensitive and therefore susceptible to the imposition of taxes. Results from surveys among adult and teen smokers in Massachusetts indicated reduced smoking following a $0.25 tax increase. Reductions were most pronounced among lowincome smokers, especially low-income teens (Biener, Aseltine, Cohen, & Anderka, 1998). Analyses of changes in smoking rates among Canadian provinces that did and did not institute reductions in cigarette taxes indicated that the tax cuts were associated with greater uptake of smoking and lower rates of quitting (Hamilton, Levinton, St-Pierre, & Grimard, 1997). The effects of taxes can be quite complex. For example, increasing taxes on cigarettes has been shown to increase use of smokeless tobacco. Presumably, the relative cost reduction for smokeless tobacco led to increased consumer choice. However, this effect was not reciprocal. Increasing taxes on smokeless tobacco did not increase consumption of cigarettes (Ohsfeldt, Boyle, & Capilouto, 1997). This nonsymmetrical result mirrors an effect found in rats: When food was restricted, rats increased the amount of water they drank (above baseline, nonrestricted levels), but when water was restricted, the rats did not increase their food consumption (relative to baseline) (Rachlin & Krasnoff, 1983). Such a finding fits within a behavioral economics framework in which issues of substitutability and complementarity are addressed (see Green & Freed, 1993, 1998). These issues will be discussed in more detail in the following sections.

Taxes

Combination of Social and Monetary Incentives

Increases in excise taxes and the use of those proceeds for smoking-prevention programs have been associated with substantial statewide reductions


In their review, Orleans and Shipley (1982) called for research into how incentives might

2/15/2011 7:46:43 PM


potentiate the influence of other components of worksite programs, such as social support among employees. The possibility of incentives enhancing other treatment components is not limited to worksites, of course. Donatelle, Prows, Champeau, and Hudson (2000) reported impressive effects from the combination of interpersonal support and voucher incentives for smoking cessation among pregnant recipients of WIC services. Every participant initially received educational materials about quitting smoking and also identified a “social supporter, preferably a female non-smoker with whom the participant had a regular, close, positive association” (p. iii67). Additionally, for those in the voucher condition, participants were eligible to earn department store vouchers worth $50 per month over 10 months, for a total of $500 in vouchers, contingent on monthly self-reports of abstinence confirmed by biological assessments (i.e., salivary cotinine and thiocyanate levels). The quit rate during the eighth month of gestation for those women who received vouchers plus interpersonal support and educational materials (voucher group) was 32%, as compared to 9% for those who received only the support and educational materials (control group). At 2 months postpartum, 21% of the voucher group had remained abstinent, whereas only 6% of the control group had done so. As reviewed by Matson and colleagues (1993), several studies have demonstrated impacts of combining support groups, competitions, and incentives in worksite smoking programs. For example, one worksite-based study (Jason, Jayaraj, Blitz, Michaels, & Klett, 1990) included the following: • Direct incentives—$10 for each of 14 meetings attended (independent of smoking status), $1 per each day that the individual was abstinent for up to 6 months following completion of the program, $30 for each period of 30 consecutive days’ abstinence, and chances in a cash lottery • Availability of support groups—participants could complete smoking-cessation procedures on their own or with a group • Availability of team competition—participants could form teams of three smokers and


327

compete for a cash prize of $300 to be given to the team with the most number of days abstinent Relative to a control worksite that received no intervention, the combination of the support group, incentives, competition, and cash prize resulted in substantially greater numbers of participants who were abstinent at the end of the program (49% versus 9%), 6-month follow-up (42% versus 13%), and at 12-month follow-up (36% versus 16%). The success of this combination of intervention tactics raises interest in further research examining sequences and combinations of such approaches. For example, a program might begin with financial incentives and then phase them out while introducing social incentives that may be more sustainable and also may be more able to link individuals to naturally occurring reinforcers among friends and families.

INFLUENCING PROBLEM BEHAVIORS INDIRECTLY THROUGH INCENTIVES FOR OTHER BEHAVIORS Behavioral economics identifies several ways in which incentives for one behavior may influence the likelihood of other behaviors. One is substitutability in which changes in the price or the availability of one good leads to opposite or compensating changes in consumption of another good. As mentioned earlier, an example would be an increase in the price of cigarettes leading to reduced consumption of cigarettes but increases in the consumption of smokeless tobacco. In this case, smokeless tobacco and cigarettes would be substitutable for each other. Another way in which consumption of two goods can be linked is complementarity, in which changes in the price or availability of one produces similar directional changes in the consumption of the other good. For example, an increase in the price of cigarettes may lead to reduced purchases of cigarettes as well as reduced purchases of coffee. Drug taking has been shown to be substitutable with other reinforcing activities. This finding is

2/15/2011 7:46:43 PM

328


in contrast to the typical image of drugs as blinding the individual to other reinforcers, thereby keeping the addict from seeking other sources of reinforcement. The substitutability of drug taking was demonstrated in an experimental, controlled economy in which participants lived and had unconstrained access to marijuana. Participants received no money other than that earned by making belts while in the experiment. In this closed economy, increases in the wage for belt making reduced marijuana smoking during the experimental period (Kagel, Battalio, & Miles, 1980; Miles et al., 1974), consistent with the view that money is substitutable for marijuana. Pleasant activities and social relationships also are substitutable for drug taking. High frequency of reinforcing activities unrelated to drug taking (Correia, Simons, Carey, & Borsari, 1998), pleasant events (Van Etten, Higgins, Budney, & Badger, 1998), and social engagement and activity (Audrain-McGovern et al., 2004; Vuchinich & Tucker, 1988) are related to lower levels of smoking, drug taking, and alcohol consumption. Similarly, a combined take-home methadone and voucher treatment showed greater cocaine abstinence as well as greater enjoyment of daily activities among methadone-maintained cocaine abusers (Rogers et al., 2008). Nondrug social reinforcers have the potential for enhancing the effectiveness of a treatment program. A review of cocaine treatment studies found that the availability of alternative, nondrug reinforcers enhanced the effects of pharmacological treatments for cocaine abuse (Higgins, 1997; LeSage, Stafford, & Glowa, 1999). The demonstration of substitutability of other reinforcers for drug taking has important implications for the promotion of abstinence. From the perspective of behavioral economics, low prices of alternative reinforcers relative to the price of drugs should increase consumption of those alternatives and decrease consumption of the drugs. For example, cigarette puffs decreased as their price increased relative to either nicotine gum or money (Johnson & Bickel, 2003). In contrast, drug taking will be encouraged if drug reinforcers are more readily available than other reinforcers, such as a rewarding job and a


supportive family and friends. For those under conditions of economic disadvantage, or when opportunities to obtain these reinforcers are blunted by prejudice, access to these non-drugrelated reinforcers may be limited. This type of situation is more likely to be found among many of the subpopulations with high rates of drug use. The effect of increasing alternative activities and goods in people’s lives as a way to reduce drug taking or to help sustain abstinence has received little attention in programs for smoking cessation or other addictions. The implications of this approach may be especially important in light of the greater prevalence of addiction problems among economically and educationally disadvantaged groups. Whatever the reasons for an individual becoming addicted, the limited access to goods and pleasurable activities associated with educational and economic disadvantage, as well as the erosion of other sources of reinforcement that follow multiple drug dependencies, may make abstinence especially difficult. Increasing other reinforcers may encourage steps toward abstinence. Community Reinforcement Approach (CRA) programs have been found to reduce alcohol consumption (Smith, Meyers, & Delaney, 1998) and opiate use (Abbott, Weller, Delaney, & Moore, 1998) through a variety of structured behavioral skills sessions focused on problem solving, abstinence, and communication skills. CRA programs often include structured drug- and alcohol-free social events. These social events may be thought of as situations in which non-drug-related social reinforcers are made accessible. The emphasis on relationships and pleasurable activities does not preclude the use of tangible reinforcers. Indeed, voucher incentives in conjunction with a CRA program were shown to produce significantly greater abstinence than the CRA alone (Higgins et al., 1994). Vouchers for work and work-related training in a therapeutic workplace for pregnant women led to 59% abstinence relative to 33% among controls (Silverman, Svikis, Robles, Stitzer, & Bigelow, 2001). Monetary reinforcement for abstinence and for attendance at a prenatal care and drug counseling program for cocaine-dependent

2/15/2011 7:46:43 PM


pregnant women led to higher rates of attendance and, strikingly, 0% as opposed to 80% adverse perinatal outcomes (Elk, Mangus, Rhoades, Andres, & Grabowski, 1998). Reinforcement of behaviors other than drug taking also can be approached through family and friends. A number of years ago, Azrin and colleagues (Hunt & Azrin, 1973) demonstrated the beneficial effects of educating spouses in providing reinforcement for behavior other than drinking and extinction for drinking and related behaviors. An intervention that trained concerned family members and significant others in reinforcing the drug user’s entering treatment and ceasing drug use, as well as reinforcing behaviors inconsistent with drug taking, increased the likelihood of the drug user’s entering treatment relative to an Alanon self-help program (Kirby, Marlowe, Festinger, Garvey, & LaMonica, 1999).

THEORETICAL ISSUES AND KEY FACTORS IN INCENTIVE PROGRAMS The results from the studies reviewed are consistent with the conclusion that, while they are in effect, incentives have reliable effects on health behaviors, including addictive behaviors and, in particular, cigarette smoking. In addition to demonstrating these effects in laboratory or experimental settings, they have been shown in a variety of clinical and prevention programs in diverse settings and with diverse populations. They also are consistent with an enormous literature on the effects of incentives/reinforcers on diverse human and animal behaviors (Rachlin, 1991). The answer to “Should we pay the patient?” (Giuffrida & Torgerson, 1997) is “Yes”— reinforcement works. That said, there still are issues to be considered in fully evaluating the effectiveness of incentives in reducing smoking. Amount and Delay of Reinforcement

Increases in the amount of reinforcement have been associated with reductions in smoking and increases in the percentage of individuals achieving a criterion of 50% of baseline CO


329

levels (Stitzer & Bigelow, 1983), increases in time since last cigarette smoked (Correia & Benson, 2006), and increases in consecutive abstinent CO readings (Roll, Higgins, & Badger, 1996). In programs for other problem behaviors, amount of reinforcement was associated with frequency of drug-free urine tests among cocaine abusers (Higgins et al, 2006; Petry et al., 2004), treatment attendance (Jones, Haug, Stitzer, & Svikis, 2000; Svikis, Lee, Haug, & Stitzer, 1997), reductions in heroin use in opioid-dependent individuals (Comer et al., 1998), and returning for reading of TB skin tests (Malotte, Rhodes, & Mais, 1998). However, as noted earlier in the discussion of contests and lotteries and as confirmed by experimental studies, delay to reinforcement reduces the effectiveness of a reinforcer (see, e.g., Lattal, 1987) and has been shown to influence the choice to smoke cigarettes (Roll, Reilly, & Johanson, 2000). Of course, the relatively immediate reinforcement of smoking, 7–9 seconds from inhaling to the time that nicotine reaches the central nervous system, has long been implicated in the strength of addiction to nicotine (Henningfield & Keenan, 1993). Greater Effectiveness of Cash Than Other Incentives

Ten dollars was more effective than the equivalent amount in grocery store coupons, bus tokens, or fast-food coupons in reinforcing return for reading of TB skin tests (Malotte et al., 1999). Similarly, cash was more effective than nonmonetary incentives in reinforcing attendance at a clinic for sexually transmitted diseases (Kamb et al., 1998) and was more effective than self-reward (i.e., a small reward that the individual administers, such as “enjoy some quiet time” or “buy yourself flowers”) in promoting breast self-examination (Solomon et al., 1998). Interestingly, the “urn” lottery, developed by Petry and colleagues (Petry, Martin, Cooney, & Kranzler, 2000) has been shown to be more effective in producing drug abstinence and more cost efficient than vouchers among opioiddependent individuals (Petry, Alessi, Marx, Austin, & Tardif, 2005). In this lottery system,

2/15/2011 7:46:43 PM

330


abstinence results in an opportunity to draw a slip of paper from a bowl, where each slip states the prize to be received. Prizes often range in value from very small to large, and the probability of drawing a prize is inversely proportional to its value. Research should compare the efficacy of monetary and other rewards when applied to smoking cessation. Effects of Context on Incentives

Tangible incentives operate within broader contexts of other program components. The importance of other program components was shown in analyses of contests within the COMMIT program, a major trial of community-based programs and activities to promote nonsmoking (The COMMIT Research Group, 1995a, 1995b). The best predictor of positive smoking outcomes for a community was the amount of money invested in the contest program (e.g., media, staff, and labor costs) that did not include the contest prizes themselves (Shipley, Hartwell, Austin, Clayton, & Stanley, 1995). This result underlines the importance of investing resources in planning and announcing programs, working with individuals, and employing appropriately trained staff, in addition to the money invested in the actual incentives, an aspect that is often overlooked in the literature. A number of other studies have evaluated the impact of the context of lotteries or contests in promoting cessation. In the Minnesota Heart Disease Prevention Program, intensive promotion of a statewide contest in Bloomington resulted in participation by about 1.06% of eligible smokers, substantially higher than the 0.2% participation in other suburbs of the Twin Cities area. With the 37% long-term abstinence rate obtained in Bloomington, this translates to a total reduction of the smoking rate in the community of about 0.39% (1.06 × 0.37). Although the longterm abstinence rate among participants in the other Twin Cities suburbs was somewhat higher (45%) than the 37% in Bloomington, the difference in participation resulted in a net reduction in the overall percentage of smokers of 0.09% in those other suburbs (0.2 × 0.45), relative to the 0.39% in Bloomington (Lando, Loken,


Howard-Pitney, & Pechacek, 1990). Despite the small percentage difference, notice that the intensively promoted program had a four-fold greater reduction in smoking, a not-inconsequential effect when considered at the population level. Combinations of contests and incentives with group smoking-cessation programs and promotional campaigns to encourage quitting have been reported to lead to 12-month abstinence rates of 36% (Jason et al., 1990) and 50% (Maheu, Gevirtz, Sallis, & Schneider, 1989). The North Karelia project in Finland (Puska, Vartiainen, Tuomilehto, Salomaa, & Nissinen, 1998) also demonstrated the importance of broader community support and promotion in enhancing the benefits of a nationwide combination of a contest and an 8-installment television program promoting smoking cessation. Relative to the city of Turku, in which community support was less intensive, rates of viewing the program, participating in the contest, attempted quits among viewers, and abstinence rates 6 months after the program all favored the region of North Karelia (Korhonen et al., 1992). Nicotine Replacement as Substitutability

In a sense, the demonstrated success of nicotine replacement (e.g., nicotine patch, nicotine gum) reflects the substitutability of one source of nicotine for another (e.g., Buchkremer, Minneker, & Block, 1991; Fiore, Smith, Jorenby, & Baker, 1994). This is joined with the finding of substitutability of cigarettes and smokeless tobacco (Ohsfeldt et al., 1997). Of course, the choice of nicotine replacement over smoking cigarettes will be influenced by their relative prices and the levels of inconvenience associated with obtaining them. A behavioral economic perspective on relatively safe sources of nicotine as a substitute for cigarettes that contain nicotine is congruent with current movements to make varied sources of nicotine readily available and competitively priced. Social Support and Interactions as Incentives Substitutable for Nicotine

A key factor in the success of smoking cessation may be the presence of supportive people around

2/15/2011 7:46:43 PM


the smokers. For example, cessation rates were higher among the 60% of participants in a Minnesota community-based contest who designated a “support person” (e.g., a friend, spouse, or other family member) than among those who did not (Pirie, Rooney, Pechacek, Lando, & Schmid, 1997). This difference was especially pronounced among those who reported that their spouses (who were not designated as the “support person”) were either smokers or nonsupportive. More generally, substantial research indicates that smoking is more likely among socially isolated individuals and that social support from friends and family is associated with greater likelihood of successful quitting (Fisher, Brownson, Heath, Luke, & Sumner, 2004). Reflecting these findings, the 2008 guidelines on smoking cessation of the Department of Health and Human Services, Treating Tobacco Use and Dependence (Fiore et al., 2008), reported that the provision of social support along with the number of contacts and the total duration of smoking-cessation interventions are all predictive of greater success. Nevertheless, there have been mixed findings regarding efforts to enhance smoking cessation with social support, such as reported in an influential review by Lichtenstein, Glasgow, and Abrams (1986). In his review of what may explain such mixed findings, Fisher (1997) emphasized a behavioral economic perspective in which social support is viewed as a reinforcer that may substitute for nicotine. From this perspective, several problems with social support interventions that were unsuccessful and several possible changes to these interventions based on behavioral economics were noted: 1. Social support was often terminated at the end of treatment. However, the ex-smokers may still need the reinforcement that social support provides as a substitute for nicotine. If support is to be an effective alternative to nicotine, then its availability needs to be sustained. 2. Some interventions focused on teaching individuals how to obtain support for quitting rather than providing social reinforcement to the quitter. Other interventions were


331

too inflexible in the ways in which the social support could be provided to the individuals. It seems reasonable, then, that these social interactions may have been less enjoyable and, thus, less reinforcing. 3. Some supportive interventions emphasized teaching participants how to obtain support rather than simply providing them with sources of support. The latter is more effective if support is to serve as an incentive that substitutes for nicotine. “Overjustification Effect” and Intrinsic Motivation

A recurrent concern about the use of incentives has centered on the possibility of “overjustification effects” in which salient, extrinsic incentives might undermine intrinsic motives for behavior. However, a critical review of the empirical literature demonstrated that such effects are, in fact, minimal in real-world settings in which manipulated incentives are not very large, in which extrinsic incentives are not administered in a manner so as to obscure the salience of other incentives for desirable behavior, or in which newly introduced reinforcers do not interfere with already existing reinforcers of established behavior (e.g., by introducing prizes for practicing the piano that impose a new requirement of keeping records that, in turn, interferes with already established reinforcers in the practice routine) (Cameron & Pierce, 1994; Fisher, 1979). In their analysis of over a quarter century of research, Cameron and Pierce (2002) found little evidence that reinforcement reduces intrinsic task interest. Reinforcement does not appear to reduce intrinsic motivation; on the contrary, Cameron, Banko, and Pierce (2001) observed that when reinforcement is linked to level of performance, intrinsic motivation increases or shows no change (see also Eisenberger & Cameron, 1996; Eisenberger, Pierce, & Cameron, 1999). The same perspective that distinguishes intrinsic and extrinsic incentives and gives rise to the hypothesized overjustification effect also leads to hypotheses that programs aimed at bolstering enduring intrinsic motivation will be

2/15/2011 7:46:43 PM

332


more successful than those that address extrinsic rewards and incentives for desired behavior. There is remarkably limited evidence to support this claim. Seattle-area smokers were offered extrinsic rewards or individually tailored feedback (i.e., a personal analysis of the participant’s progress throughout the intervention) contingent on returning a baseline questionnaire and progress reports within a self-help program (Curry, Wagner, & Grothaus, 1991). Those given the feedback were more likely to use the self-help materials, report short-term abstinence (3 months), and be abstinent (validated by measuring cotinine levels, a salivary by-product of smoking tobacco) at 12-months follow-up. These findings might be interpreted as showing that increasing intrinsic motivation is better accomplished with personalized feedback rather than with extrinsic rewards. However, consideration of the details of the procedures suggests an alternative explanation. The extrinsic reward entailed a “secret gift” (a ceramic coffee cup) along with entries into drawings for three prizes, a 1-week vacation in Hawaii, a weekend at a resort on the San Juan Islands outside of Seattle, or a weekend at a deluxe hotel in Seattle. Thus, individually tailored feedback did produce better results than did receipt of a coffee cup and chances among 607 other participants to win one of three vacation prizes. Considering the evidence regarding the importance of amount and probability of reward, the results from this study may be seen as indicating the greater impact on smoking cessation of tailored feedback than of one small prize and low odds of winning, rather than a more general advantage of intrinsic rewards. The Use of Incentives for High-Priority Behaviors During Limited Time Periods

The reviews by Marteau et al. (2009) and Sutherland et al. (2008) make clear that incentives have only a modest long-term impact on behavior after the incentives are no longer in place. As Sutherland et al. noted: “ . . . research evidence suggests that incentives can increase adoption of healthy behaviors but that positive effects may diminish over time” (p. 65S).


Although many might interpret such a conclusion as evidence for the ineffectiveness of incentives, the reader of this volume is well aware that extinction is an established phenomenon and that diminution of benefits of incentives following their termination—in the absence of other incentives to maintain the behavior—reflects the adaptability of behavior to its context, not a failure of incentives to change behavior. Therefore, the use of incentives might be highly recommended (a) for increasing key behaviors that need to occur only once or relatively few times, or (b) for increasing the likelihood of behaviors in particular settings or for particularly crucial periods. The first point is illustrated by the evidence for the use of incentives to increase immunizations mentioned earlier (Sutherland et al., 2008). A noteworthy illustration of the second point (influencing behavior in certain contexts or during critical periods) would be nonsmoking during pregnancy, when the development of the fetus and newborn are crucially affected by smoking. For example, reduction of smoking among pregnant women can reduce the risk of a low birth-weight child by 45% (Ershoff, Quinn, Mullen, & Lairson, 1990). Thus, incentive programs may be especially appropriate for promoting nonsmoking during pregnancy. Little research has explored incentives for this group, but women taking part in an incentivesbased smoking-cessation program during pregnancy and postpartum achieved a higher rate of abstinence up to 24 weeks after delivery (Higgins et al., 2004) and importantly, increased fetal weight during pregnancy (Heil et al., 2008). Incentives for nonsmoking among pregnant women could achieve substantial savings in costs such as those for caring for low birth-weight babies (Adams et al., 2002; Marks, Koplan, Hogue, & Dalmat, 1990). It may be expected that extinction of the appropriate, desired behavior will occur when incentives are withdrawn. However, such an expectation is not reason to be discouraged nor is this the only way to terminate incentivebased smoking-cessation programs. In addition to providing extrinsic incentives contingent on cessation, interventions also must plan for the

2/15/2011 7:46:43 PM


generalization of program-based reinforcers to reinforcers that occur naturally contingent on smoking cessation (e.g., money savings, social approval, better health). Generalization must occur gradually, where the most effective, most easily controlled program-based reinforcers are delivered early in the intervention in order to achieve immediate cessation. Later, then, the programmed reinforcers can be faded out, and the naturally occurring reinforcers faded in. It is a failure of many incentives-based smokingcessation programs to not plan for this needed generalization.

CONCLUSIONS The literature reviewed above provides the basis for the following conclusions for intervention programs, research, and public health policies. The Use of Incentives in Programs That Promote Smoking Cessation

Incentives do reduce drug taking, including cigarette smoking, at least while they are in effect. As with other reinforcers, amount, probability, and delay of incentives are important: Increasing amount and probability of incentives, and decreasing delay to their receipt, typically increases their effectiveness. Incentives are most promising when smoking cessation is an especially high priority for a defined period of time (e.g., among pregnant women, patients recovering from a heart attack, patients preparing for and recovering from cardiac or cancer surgery and treatment). The effects of incentive programs for cessation of substance use are not explained by other aspects of the intervention programs (e.g., educational components, interaction with intervention providers), although the benefits of incentives are often enhanced when combined with these other components. Incentives alone may have little long-term or “carry-over” effects once they are terminated. This finding is consistent with most basic and applied research on reinforcement processes. Reinforcers must be continued as long as the target behavior is desired (that is, indefinitely, for


333

substance-use cessation programs). More practically, interventions that incorporate naturally occurring reinforcers into the contingency are more likely to produce long-term behavior change. As Baer, Wolf, and Risley (1968) noted in a key paper on incentives in behavior modification over 40 years ago, “Generalization [or maintenance of behavior change] should be programmed, rather than expected or lamented” (p. 97). External reinforcers, such as incentives given in smoking-cessation programs, should be tapered toward the end of treatment and replaced with naturally occurring reinforcers before the intervention is terminated. This needs to be a planned component of the treatment program so that the appropriate behavior will be maintained and generalized, rather than a hoped-for result. A variety of goods may function as incentives, including nicotine replacement, social interaction, and feedback of progress. Incentives could be used to increase participation in a program that already achieves acceptable rates of smoking cessation, or they could be made contingent upon the use of nicotine replacement. Other reinforcing activities can be substitutable for nicotine use, such as positive social interaction, physical activity, and other activities that generally support higher levels of health. Based on research on other types of substance abuse, the use of incentives may be especially cost efficient. Cost efficiency will depend on program objectives. A program that reduces smoking during a limited period of time during which continued smoking would lead to appreciable health costs (e.g., during pregnancy or following surgery) may be quite cost efficient. Shifting from a tightly defined target group to a population perspective, modest incentives for smoking cessation deployed to large numbers of individuals can be quite cost efficient even if only a small percentage of the participants quit smoking. Recommendations for Research

The aforementioned conclusions as well as comments and recommendations from several thoughtful reviews in the field (e.g.,

2/15/2011 7:46:43 PM

334


Bigelow et al., 1998) identify priorities for research in several areas, including refining the use of incentives in smoking cessation, and integrating incentives into broader programs for health promotion and quality of life. These are presented in the following sections. Refining the Use of Incentives in Smoking Cessation

One important aspect of the use of incentives is to gain a greater understanding of the boundary conditions under which they are effective. What is the smallest amount or largest delay between the response and receipt of the incentive that can still produce satisfactory effects? If incentives are delivered intermittently, what is the most cost-efficient probability (as in a VR or escalating schedule) for delivering incentives contingent on abstinence? How effective is the use of money versus other incentive types? These parameters will be important in designing smokingcessation programs that take into account a specific population, funding opportunities, and logistical constraints. As already noted, there may be great cost efficiency in applying modest incentives to large populations of smokers, even if such interventions achieve only modest cessation rates. Dallery and colleagues have developed a very promising Web-based contingency management program for smoking cessation that could be very effective in reaching large numbers of smokers with minimal cost to both the agency and the individual (Dallery, Meredith, & Glenn, 2008; Reynolds, Dallery, Shroff, Patak, & Leraas, 2008). Large-scale studies should investigate the application to a wider population and to specific target groups. Another aspect of incentive implementation that needs to be explored further is the application of shaping procedures for the acquisition and maintenance of behavior change. Although a central concept in the psychological tradition from which incentive programs emerged, the shaping of behavior is rarely emphasized in this literature. For example, one problem with the use of incentives contingent upon abstinence is that many individuals fail to achieve the minimal level drug-negative samples (e.g., in urine


samples or CO breath levels) that are required to receive the incentive, and therefore never make contact with the reinforcement contingency. As a way to increase contact with the incentives, the use of percentile schedules of reinforcement may be especially helpful for those individuals who are more resistant to behavior change (Lamb, Morral, Kirby, Iguchi, & Galbicka, 2004). In the study of percentile schedules by Lamb and colleagues (2004), smokers were randomly assigned to conditions in which reinforcement was contingent on breath CO levels being less than the lowest 1, 3, 5, or 7 out of their previous 10 samples. That is, the several different conditions made reinforcement contingent on the current sample being less than the 10th, 30th, 50th, or 70th percentile of the previous 10 samples. Thus, the conditions differed appreciably in their stringency requirement. In the 10th percentile condition, the participant received reward only if the most recent sample was lower than 9 of the previous 10 samples. In contrast, the 70th percentile condition was quite lenient in that reinforcement was provided for any sample lower than 3 of the previous 10. It is to be noted, however, that even in this lenient condition, meeting that contingency would gradually lower the 70th percentile, moving the contingency inexorably toward zero. Results showed that CO levels were significantly lower in the 70th percentile group as the 3-month study progressed. Furthermore, for those smokers classified as “hard to treat,” the 70th percentile schedule was more effective in producing immediate CO reductions and maintaining lower CO levels than any of the other schedules. This success among hard-to-treat individuals is especially noteworthy given that the prevalence of smoking continues to decline, leaving still smoking those who are often most challenged by cessation. Fading procedures also must be explored more fully. As mentioned, incentives are effective when they are in effect. Therefore, interventions must include a procedure in which the programmed reinforcers (e.g., vouchers, lottery tickets) are faded out and replaced with reinforcers that occur naturally in an individual’s life (e.g., social approval, money savings,

2/15/2011 7:46:43 PM


better health). In order to use naturally occurring reinforcers most effectively, however, there must be a comprehensive conceptualization of the role of social influences as incentives. Several studies indicate that social support, social interaction, or feedback of progress may function as reinforcers for participation in programs as well as for not smoking. This view of social support as an incentive differs from the more traditional view in which social support has been conceptualized as an influence that enhances an individual’s skill or performance (Fisher, 1996). A fuller understanding of social influence will increase the ability to deploy it effectively. Integrating Incentives into Broader Programs for Health Promotion and Quality of Life

If incentives for nonsmoking are integrated with other program components, then nonsmoking incentives might encourage participation in other health-promotion programs (e.g., preventive care, weight loss, disease management). For example, inclusion of incentives for not smoking may increase the overall attractiveness of the smoking-cessation programs in which they are included (Klesges et al., 1986) or for broader programs such as those for cardiovascular risk reduction or rehabilitation. This is especially important given the need to reduce the socioeconomic disparities in health surrounding smoking, obesity, physical activity, and the many diseases such as diabetes that are tied to lifestyle risks. Reaching disadvantaged groups is a high priority (Glasgow, Vogt, & Boles, 1999), and including incentives in health-promotion programs may assist in pursuing it. Incentives also may provide incremental utility when added to programs with already documented benefits (e.g., smoking-cessation counseling for pregnant women). When added to otherwise successful interventions, the beneficial effect of incentives may be difficult to detect. This problem is statistical, not conceptual, in nature. It is not that the effect of incentives does not exist when other programs also are being implemented, but rather that the additional effect that is unique to the incentives may be too small to be detected statistically. A key proviso


335

for such research is that it be designed with adequate power to detect what might often be subtle or modest additive effects. Implications for Public Policy

Several aspects of the use of incentives to promote smoking cessation have implications for public policy. As already noted, these programs appear to be most appropriate for high-risk, highpriority, and hard-to-reach patients or for those for whom the importance of smoking cessation is heightened during a defined period of time. The application of incentive-based programs to these groups could be especially effective. Few studies have employed incentive programs with large populations (Morris, Flores, Olinto, & Medina, 2004), but such programs could be especially effective in reaching large groups of people (e.g., all women of childbearing age) as opposed to smaller, more selected populations for whom behavior-change interventions are most often applied. For example, incentive programs might be incorporated within primary care or general health care or financing programs, such as Medicaid and Medicare. Behavioral economic considerations of a broad range of incentives and disincentives for healthy behavior have gained increasing attention in discussion of national health care reform. Incorporating a reinforcement and behavioral economic framework in the design and implementation of health interventions offers the promise of bringing to bear the two major findings of this review: (1) for smoking cessation and other important health behaviors, incentives do work; and (2) consistent with perhaps the most reliable observations in all of psychology, for incentives to have sustained effects, the incentives themselves must be sustained.

ACKNOWLEDGMENTS This chapter is based on a review commissioned by the Robert Wood Johnson Foundation to assist in the development of Foundation-sponsored projects addressing smoking cessation. We gratefully acknowledge the assistance of Daniel Holt in the preparation of that review.

2/15/2011 7:46:43 PM

336


Preparation of the chapter was supported by the Peers for Progress program of the American Academy of Family Physicians Foundation, supported by the Eli Lilly and Company Foundation, to E. Fisher, and by National Institutes of Health grant MH055308 to L. Green.

REFERENCES Abbott, P. J., Weller, S. B., Delaney, H. D., & Moore, B. A. (1998). Community reinforcement approach in the treatment of opiate addicts. American Journal of Drug and Alcohol Abuse, 24, 17–30. Achat, H., McIntyre, P., & Burgess, M. (1999). Health care incentives in immunization. Australian and New Zealand Journal of Public Health, 23, 285–288. Adams, E. K., Miller, V. P., Ernst, C., Nishimura, B. K., Melvin, C., & Merritt, R. (2002). Neonatal health care costs related to smoking during pregnancy. Health Economics, 11, 193–206. Altman, D. G., Flora, J. A., Fortmann, S. P., & Farquhar, J. W. (1987). The cost-effectiveness of three smoking cessation programs. American Journal of Public Health, 77, 162–165. Audrain-McGovern, J., Rodriguez, D., Tercyak, K. P., Epstein, L. H., Goldman, P., & Wileyto, E. P. (2004). Applying a behavioral economic framework to understanding adolescent smoking. Psychology of Addictive Behaviors, 18, 64–73. Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97. Bickel, W. K., & McLellan, A. T. (1996). Can management by outcome invigorate substance abuse treatment? Micro-incentives for the quality of care. The American Journal on Addictions, 5, 281–291. Biener, L., Aseltine, R. H., Cohen, B., & Anderka, M. (1998). Reactions of adult and teenaged smokers to the Massachusetts Tobacco Tax. American Journal of Public Health, 88, 1389–1391. Bigelow, G. E., Brooner, R. K., & Silverman, K. (1998). Competing motivations: Drug reinforcement vs. non-drug reinforcement. Journal of Psychopharmacology, 12, 8–14. Buchkremer, G., Minneker, E., & Block, M. (1991). Smoking-cessation treatment combining transdermal nicotine substitution with behavioral therapy. Pharmacopsychiatrist, 24, 96–102.


Cahill, K., & Perera, R. (2008). Competitions and incentives for smoking cessation. Cochrane Database of Systematic Reviews, 3, Art. No. CD004307. Cameron, J., Banko, K. M., & Pierce, W. D. (2001). Pervasive negative effects of rewards on intrinsic motivation: The myth continues. The Behavior Analyst, 24, 1–44. Cameron, J., & Pierce, W. D. (1994). Reinforcement, reward, and intrinsic motivation: A meta-analysis. Review of Educational Research, 64, 363–423. Cameron, J., & Pierce, W. D. (2002). Rewards and intrinsic motivation: Resolving the controversy. Westport, CT: Bergin & Garvey. Centers for Disease Control and Prevention (CDC). (1996). Cigarette smoking before and after an excise tax increase and an antismoking campaign - Massachusetts, 1990 - 1996. Morbidity and Mortality Weekly Reports, 45, 966–970. Centers for Disease Control and Prevention (CDC). (2002). Cigarette smoking among adults— United States, 2000. Morbidity and Mortality Weekly Reports, 51, 642–645. Centers for Disease Control and Prevention (CDC). (2008). Current smoking. Retrieved from http:// www.cdc.gov/tobacco/data_statistics/tables/ index.htm Centers for Disease Control Office on Smoking and Health. (2007). Cigarette use among high school students–United States, 1991-2007. Morbidity and Mortality Weekly Reports, 57, 689–691. Chutuape, M. A., Silverman, K., & Stitzer, M. (1999a). Contingent reinforcement sustains postdetoxification abstinence from multiple drugs: A preliminary study with methadone patients. Drug and Alcohol Dependence, 54, 69–81. Chutuape, M. A., Silverman, K., & Stitzer, M. (1999b). Use of methadone take-home contingencies with persistent opiate and cocaine users. Journal of Substance Abuse Treatment, 16, 23–30. Comer, S. D., Collins, E. D., Wilson, S. T., Donovan, R., Foltin, R. W., & Fischman, M. W. (1998). Effects of an alternative reinforcer on intravenous heroin self-administration by humans. European Journal of Pharmacology, 345, 13–26. The COMMIT Research Group. (1995a). Community intervention trial for smoking cessation (COMMIT): I. Cohort results from a four-year community intervention. American Journal of Public Health, 85, 183–192. The COMMIT Research Group. (1995b). Community intervention trial for smoking

2/15/2011 7:46:43 PM


cessation (COMMIT): II. Changes in adult cigarette smoking prevalence. American Journal of Public Health, 85, 193–200. Correia, C. J., & Benson, T. A. (2006). The use of contingency management to reduce cigarette smoking among college students. Experimental and Clinical Psychopharmacology, 14, 171–179. Correia, C. J., Simons, J., Carey, K. B., & Borsari, B. E. (1998). Predicting drug use: Application of behavioral theories of choice. Addictive Behaviors, 23, 705–709. Covey, L. S., Glassman, A. H., & Stetner, F. (1997). Major depression following smoking cessation. American Journal of Psychiatry, 154, 263–265. Crowley, T. J., MacDonald, M. J., Zerbe, G. O., & Petty, T. L. (1991). Reinforcing breath carbon monoxide reductions in chronic obstructive pulmonary disease. Drug and Alcohol Dependence, 29, 47–62. Curry, S., Wagner, E., & Grothaus, L. (1991). Evaluation of intrinsic and extrinsic motivation interventions with a self-help smoking cessation program. Journal of Consulting and Clinical Psychology, 59, 318–324. Dallery, J., Meredith, S., & Glenn, I. M. (2008). A deposit contract method to deliver abstinence reinforcement for cigarette smoking. Journal of Applied Behavior Analysis, 41, 609–615. Donatelle, R. J., Prows, S. L., Champeau, D., & Hudson, D. (2000). Randomized controlled trial using social support and financial incentives for high risk pregnant smokers: Significant other supporter (SOS) program. Tobacco Control, 9, iii67–iii69. Eisenberger, R., & Cameron, J. (1996). The detrimental effects of reward: Myth or reality? American Psychologist, 51, 1153–1166. Eisenberger, R., Pierce, W. D., & Cameron, J. (1999). Effects of reward on intrinsic motivation— negative, neutral, and positive: Comment on Deci, Koestner, and Ryan (1999). Psychological Bulletin, 125, 677–691. Elk, R., Mangus, L. A., Rhoades, H., Andres, R., & Grabowski, J. (1998). Cessation of cocaine use during pregnancy: Effects of contingency management interventions on maintaining abstinence and complying with prenatal care. Addictive Behaviors, 23, 57–64. El-Khorazaty, M. N., Johnson, A. A., Kiely, M., El-Mohandes, A. A. E., Subramanian, S., Laryea, H. A., Murray, K. B., Thornberry, J. S., & Joseph, J. G. (2007). Recruitment and retention of low-income minority women in a


337

behavioral intervention to reduce smoking, depression, and intimate partner violence during pregnancy. BioMed Central Public Health, 7, 233. Ershoff, D., Quinn, V., Mullen, P., & Lairson, D. (1990). Pregnancy and medical cost outcomes of a self-help prenatal smoking cessation program in an HMO. Public Health Reports, 105, 340–347. Feldman, R., Bacher, M., Campbell, N., Drover, A., & Chockalingman, A. (1998). Adherence to pharmacologic management of hypertension. Canadian Journal of Public Health, 89, I16–I18. Fiore, M. C., Jaén, C. R., Baker, T. B., Bailey, W. C., Benowitz, N. L., Curry, S. J., . . .Wewers, M. E. (2008). Treating tobacco use and dependence: 2008 update. Clinical practice guideline. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service. Fiore, M. C., Smith, S. S., Jorenby, D. E., & Baker, T. B. (1994). The effectiveness of the nicotine patch for smoking cessation. A metaanalysis. Journal of the American Medical Association, 271, 1940–1947. Fisher, E. B. (1979). Overjustification effects in token economies: Replicability and the importance of similar, complex functional relations. Journal of Applied Behavior Analysis, 12, 407–415. Fisher, E. B. (1996). A behavioral-economic perspective on the influence of social support on cigarette smoking. In L. Green & J. H. Kagel (Eds.), Advances in behavioral economics (Vol. 3, pp. 207–236). Norwood, NJ: Ablex. Fisher, E. B., Jr. (1997). Two approaches to social support in smoking cessation: Commodity model and nondirective support. Addictive Behaviors, 22, 819–833. Fisher, E. B., Brownson, R. C., Heath, A. C., Luke, D. A., & Sumner, W. II. (2004). Cigarette smoking. In T. J. Boll, J. Raczynski, & L. Leviton (Eds.), Handbook of clinical health psychology (Vol. 2, pp. 75–120). Washington, DC: American Psychological Association. Fitzgerald, J. M., Patrick, D. M., Strathdee, S., Rekart, M., Elwood, R. K., Schecter, M. T., Montaner, J., & O’Shaughnessy, M. (1999). Use of incentives to increase compliance for TB screening in a population of intravenous drug users. Vancouver injection drug use study group. International Journal of Tuberculosis and Lung Disease, 3, 153–155. Fortman, S. P., & Killen, J. D. (1995). Nicotine gum and self-help behavioral treatment for smoking

2/15/2011 7:46:44 PM

338


relapse prevention: Results from a trial using population-based recruitment. Journal of Consulting and Clinical Psychology, 63, 460–468. Gilpin, E. A., & Pierce, J. P. (2002). Demographic differences in patterns of the incidence of smoking cessation: United States 1950-1990. Annals of Epidemiology, 12, 141–150. Giuffrida, A., & Torgerson, D. J. (1997). Should we pay the patient? Review of financial incentives to enhance patient compliance. British Medical Journal, 315, 703–707. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322–1327. Green, L., & Freed, D. E. (1993). The substitutability of reinforcers. Journal of the Experimental Analysis of Behavior, 60, 141–158. Green, L., & Freed, D. (1998). Behavioral economics. In W. T. O’Donohue (Ed.), Learning and behavior therapy (pp. 274–300). Boston, MA: Allyn & Bacon. Hamilton, V. H., Levinton, C., St-Pierre, Y., & Grimard, F. (1997). The effect of tobacco tax cuts on cigarette smoking in Canada. Canadian Medical Association Journal, 156, 187–191. Heil, S. H., Higgins, S. T., Berstein, I. M., Solomon, L. J., Rogers, R. E., Thomas, C. S., Badger, G. J., & Lynch, M. E. (2008). Effects of voucher-based incentives on abstinence from cigarette smoking and fetal growth among pregnant women. Addiction, 103, 1009–1018. Henningfield, J. E., & Keenan, R. M. (1993). Nicotine delivery kinetics and abuse liability. Journal of Consulting and Clinical Psychology, 61, 743–750. Henningfield, J. E., Stitzer, M. L., & Griffiths, R. R. (1980). Expired air carbon monoxide accumulation and elimination as a function of number of cigarettes smoked. Addictive Behaviors, 5, 265–272. Higgins, S. T. (1997). The influence of alternative reinforcers on cocaine use and abuse: A brief review. Pharmacology Biochemistry and Behavior, 57, 419–427. Higgins, S. T., Budney, A. J., Bickel, W. K., Foerg, F. E., Donham, R., & Badger, G. J. (1994). Incentives improve outcome in outpatient behavioral treatment of cocaine dependence. Archives of General Psychiatry, 51, 568–576. Higgins, S. T., Heil, S. H., Dantona, R., Donham, R., Matthews, M, & Badger, G. J. (2006). Effects of


varying the monetary value of voucher-based incentives on abstinence achieved during and following treatment among cocaine-dependent outpatients. Addiction, 102, 271–281. Higgins, S. T., Heil, S. H., Solomon, L. J., Bernstein, I. M., Plebani Lussier, J., Abel, R. L., Lynch, M. E., & Badger, G. J. (2004). A pilot study on voucher-based incentives to promote abstinence from cigarette smoking during pregnancy and postpartum. Nicotine and Tobacco Research, 6, 1015–1020. Hu, T. W., Sung, H. Y., & Keeler, T. E. (1995). Reducing cigarette consumption in California: Tobacco taxes vs. an anti-smoking media campaign. American Journal of Public Health, 85, 1218–1222. Hughes, J. R., Hatsukami, D. K., Mitchell, J. E., & Dahlgren, L. A. (1986). Prevalence of smoking among psychiatric outpatients. American Journal of Psychiatry, 143, 993–997. Hunt, G., & Azrin, N. (1973). A community reinforcement approach to alcoholism. Behaviour Research and Therapy, 11, 91–104. Iguchi, M. Y., Belding, M. A., Morral, A. R., Lamb, R. J., & Husband, S. D. (1997). Reinforcing operants other than abstinence in drug abuse treatment: an effective alternative for reducing drug abuse. Journal of Consulting and Clinical Psychology, 65, 412–428. Janz, N. K., Schottenfeld, D., Doerr, K. M., Selig, S. M., Dunn, R. L., Strawderman, M., & Levine, P. A. (1997). A two-step intervention to increase mammography among women aged 65 and older. American Journal of Public Health, 87, 1683–1686. Jason, L., Jayaraj, S., Blitz, C., Michaels, M., & Klett, L. (1990). Incentives and competition in a worksite smoking cessation intervention. American Journal of Public Health, 80, 205–206. Jeffery, R. W., Hellerstedt, W. L., & Schmid, T. L. (1990). Correspondence programs for smoking cessation and weight control: A comparison of two strategies in the Minnesota Heart Health Program. Health Psychology, 9, 585–598. Jeffery, R. W., Wing, R. R., Thorson, C., & Burton, L. R. (1998). Use of personal trainers and financial incentives to increase exercise in a behavioral weight-loss program. Journal of Consulting and Clinical Psychology, 66, 777–783. Johnson, M. W., & Bickel, W. K. (2003). The behavioral economics of cigarette smoking: The concurrent presence of a substitute and an independent reinforcer. Behavioral Pharmacology, 14, 137–144.

2/15/2011 7:46:44 PM


Jones, H. E., Haug, N., Silverman, K., Stitzer, M., & Svikis, D. (2001). The effectiveness of incentives in enhancing treatment attendance and drug abstinence in methadone-maintained pregnant women. Drug and Alcohol Dependence, 61, 297–306. Jones, H. E., Haug, N. A., Stitzer, M. L., & Svikis, D. S. (2000). Improving drug treatment outcomes for pregnant drug-dependent women using low-magnitude voucher incentives. Addictive Behavior, 25, 263–267. Kagel, J. H., Battalio, R. C., & Green, L. (1995). Economic choice theory. New York, NY: Cambridge University Press. Kagel, J. H., Battalio, R. C., & Miles, C. G. (1980). Marijuana and work performance: Results from an experiment. Journal of Human Resources, 15, 373–395. Kamb, M. L., Rhodes, F., Hoxworth, T., Rogers, J., Lentz, A., Kent, C., MacGowen, R., & Peterman, T. A. (1998). What about money? Effect of small monetary incentives on enrollment, retention, and motivation to change behaviour in an HIV/STD prevention counselling intervention. The project RESPECT study group. Sexually Transmitted Infections, 74, 253–255. Kirby, K. C., Marlowe, D. B., Festinger, D. S., Garvey, K. S., & LaMonica, V. (1999). Community reinforcement training for family and significant others of drug abusers: A unilateral intervention to increase treatment entry of drug users. Drug and Alcohol Dependence, 56, 85–96. Kirby, K. C., Marlowe, D. B., Festinger, D. S., Lamb, R. J., & Platt, R. J. (1998). Schedule of voucher delivery influences cocaine abstinence. Journal of Consulting and Clinical Psychology, 66, 761–767. Klesges, R. C., Vasey, M. W., & Glasgow, R. E. (1986). A worksite smoking modification competition: Potential for public health impact. American Journal of Public Health, 76, 198–200. Korhonen, H. J., Niemensivu, H., Piha, T., Koskela, K., Wiio, J., Johnson, C. A., & Puska, P. (1992). National TV smoking cessation program and contest in Finland. Preventive Medicine, 21, 74–87. Lamb, R. J., Morral, A. R., Kirby, K. C., Iguchi, M. Y., Galbicka, G. (2004). Shaping smoking cessation using percentile schedules. Drug and Alcohol Dependence, 76, 247–259. Lando, H. A., Loken, B., Howard-Pitney, B., & Pechacek, T. (1990). Community impact of a


339

localized smoking cessation contest. American Journal of Public Health, 80, 601–603. Lattal, K. A. (1987). Considerations in the experimental analysis of reinforcement delay. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analyses of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value (pp. 107–123). Hillsdale, NJ: Erlbaum. LeBaron, C. W., Starnes, D., Dini, E. F., Chambliss, J. W., & Chaney, M. (1998). The impact of interventions by a community-based organization on inner-city vaccination coverage. Archives of Pediatrician Adolescent Medicine, 152, 327–352. Leinweber, C. E., Macdonald, J. M., & Campbell, H. S. (1994). Community smoking cessation contests: An effective public health strategy. Canadian Journal of Public Health, 85, 95–98. Lerman, C., Caporaso, N. E., Audrain, J., Main, D., Bowman, E. D., Lockshin, B., Boyd, N. R., & Shields, P. G. (1999). Evidence suggesting the role of specific genetic factors in cigarette smoking. Health Psychology, 18, 14–20. LeSage, M. G., Stafford, D., & Glowa, J. R. (1999). Preclinical research on cocaine selfadministration: Environmental determinants and their interaction with pharmacological treatment. Neuroscience and Biobehavioral Reviews, 23, 717–741. Lichtenstein, E., Glasgow, R. E., & Abrams, D. B. (1986). Social support in smoking cessation: In search of effective interventions. Behavior Therapy, 17, 607–619. Maheu, M., Gevirtz, R., Sallis, J., & Schneider, N. (1989). Competition/cooperation in worksite smoking cessation using nicotine gum. Preventive Medicine, 18, 867–876. Malotte, C. K., Hollingshead, J. R., & Rhodes, F. (1999). Monetary versus nonmonetary incentives for TB skin test reading among drug users. American Journal of Preventive Medicine, 16, 182–188. Malotte, C. K., Rhodes, F., & Mais, K. E. (1998). Tuberculosis screening and compliance with return for skin test reading among active drug users. American Journal of Public Health, 88, 792–796. Marcus, A. C., Kaplan, C. P., Crane, L. A., Berek, J. S., Bernstein, G., Gunning, J. E., & McClatchey, M. W. (1998). Reducing loss-to-follow-up among women with abnormal Pap smears. Results from a randomized trial testing an intensive

2/15/2011 7:46:44 PM

340


follow-up protocol and economic incentives. Medical Care, 36, 397–410. Marks, J. S., Koplan, J. P., Hogue, C. J. R., & Dalmat, M. E. (1990). A cost-benefit/cost effectiveness analysis of smoking cessation for pregnant women. American Journal of Preventative Medicine, 6, 282–289. Marteau, T. M., Ashcroft, R. E., & Oliver, A. (2009). Using financial incentives to achieve healthy behaviour. British Medical Journal, 338, 983–985. Matson, D. M., Lee, J. W., & Hopp, J. W. (1993). The impact of incentives and competitions on participation and quit rates in worksite smoking cessation programs. American Journal of Health Promotion, 7, 270–280, 295. Miles, C. G., Congreve, G. R. S., Gibbins, R. J., Marshman, J., Devenyi, R., & Hicks, R. C. (1974). An experimental study of the effects of daily cannabis smoking on behavior patterns. Acta Pharmacologica et Toxocologica, 34(Suppl. 1), 1–44. Morris, S. S., Flores, R., Olinto, P., Medina, J. M. (2004). Monetary incentives in primary health care and effects on use and coverage of preventative health care interventions in rural Honduras: Cluster randomized trial. Lancet, 364, 2030–2037. Ohsfeldt, R. L., Boyle, R. G., & Capilouto, E. (1997). Effects of tobacco excise taxes on the use of smokeless tobacco products in the USA. Health Economics, 6, 525–531. Orleans, C. S., & Shipley, R. H. (1982). Worksite smoking cessation initiatives: Review and recommendations. Addictive Behaviors, 7, 1–16. Paxton, R. (1980). The effects of a deposit contract as a component in a behavioural programme for stopping smoking. Behaviour Research and Therapy, 18, 45–50. Paxton, R. (1981). Deposit contracts with smokers: Varying frequency and amount of repayments. Behaviour Research and Therapy, 19, 117–123. Paxton, R. (1983). Prolonging the effects of deposit contracts with smokers. Behaviour Research and Therapy, 21, 425–433. Petry, N. M., Alessi, S. M., Marx, J., Austin, M., & Tardif, M. (2005). Vouchers versus prizes: Contingency management treatment of substance abusers in community settings. Journal of Consulting and Clinical Psychology, 73, 1005–1014. Petry, N. M., Martin, B., Cooney, J. L., & Kranzler, H. R. (2000). Give them prizes and they will come: Variable-ratio contingency management for treatment of alcohol dependence.


Journal of Consulting and Clinical Psychology, 70, 250–257. Petry, N. M., Tedford, J., Austin, M., Nich, C., Carroll, K. M., & Rounsaville, B. J. (2004). Prize reinforcement contingency management for treating cocaine users: How low can we go, and with whom? Addiction, 99, 349–360. Pierce, J. P., Gilpin, E. A., Emery, S. L., White, M. M., Rosbrook, B., Berry, C. C., & Farkas, A. J. (1998). Has the California tobacco control program reduced smoking? Journal of the American Medical Association, 280, 893–899. Pirie, P. L., Rooney, B. L., Pechacek, T. F., Lando, H. A., & Schmid, L. A. (1997). Incorporating social support into a community-wide smoking-cessation contest. Addictive Behaviors, 22, 131–137. Plebani Lussier, J., Heil, S. H., Mongeon, J. A., Badger, G. J., & Higgins, S. T. (2006). A metaanalysis of voucher-based reinforcement therapy for substance use disorders. Addiction, 101, 192–203. Pomerleau, O. F., Collins, A. C., Shiffman, S., & Pomerleau, C. S. (1993). Why some people smoke and others do not: New perspectives [review]. Journal of Consulting and Clinical Psychology, 61, 723–731. Prendergast, M., Podus, D., Finney, J., Greenwell, L., & Roll, J. (2006). Contingency management for treatment of substance use disorders: A metaanalysis. Addiction, 101, 1546–1560. Puska, P., Vartiainen, E., Tuomilehto, J., Salomaa, V., & Nissinen, A. (1998). Changes in premature deaths in Finland: Successful long-term prevention of cardiovascular diseases. Bulletin of the World Health Organization, 76, 419–425. Rachlin, H. (1991). Introduction to modern behaviorism (Vol. 3). San Francisco, CA: Freeman. Rachlin, H., & Krasnoff, J. (1983). Eating and drinking: An economic analysis. Journal of the Experimental Analysis of Behavior, 39, 385–404. Rand, C. S., Stitzer, M. L., Bigelow, G. E., & Mead, A. M. (1989). The effects of contingent payment and frequent workplace monitoring on smoking abstinence. Addictive Behaviors, 14, 121–128. Reynolds, B., Dallery, J., Shroff, P., Patak, M., & Leraas, K. (2008). A web-based contingency management program with adolescent smokers. Journal of Applied Behavior Analysis, 41, 597–601. Robles, E., Silverman, K., Preston, K., Cone, E. J., Katz, E., Bigelow, G. E., & Stitzer, M. L. (2000).

2/15/2011 7:46:44 PM


The brief abstinence test: Voucher-based reinforcement of cocaine abstinence. Drug and Alcohol Dependence, 58, 205–212. Rogers, R. E., Higgins, S. T., Silverman, K., Thomas, C. S., Badger, G. J., Bigelow, G., & Stitzer, M. (2008). Abstinence-contingent reinforcement and engagement in non-drug-related activities among illicit drug abusers. Psychology of Addictive Behaviors, 22, 544–550. Roll, J. M., Higgins, S. T., & Badger, G. J. (1996). An experimental comparison of three different schedules of reinforcement of drug abstinence using cigarette smoking as an exemplar. Journal of Applied Behavior Analysis, 29, 495–505. Roll, J. M., Higgins, S. T., Steingard, S., & McGinley, M. (1998). Use of monetary reinforcement to reduce the cigarette smoking of persons with schizophrenia: A feasibility study. Experimental and Clinical Psychopharmacology, 6, 157–161. Roll, J. M., Reilly, M. P., & Johanson, C. E. (2000). The influence of exchange delays on cigarette versus money choice: A laboratory analog of voucher based reinforcement therapy. Experimental and Clinical Psychopharmacology, 8, 366–370. Roll, J. M., & Shoptaw, S. (2006). Contingency management: Schedule effects. Psychiatry Research, 144, 91–93. Seal, K. H., Kral, A. H., Lorvick, J., McNees, A., Gee, L., & Edlin, B. R. (2003). A randomized controlled trial of monetary incentives vs. outreach to enhance adherence to the hepatitis B vaccine series among injection drug users. Drug and Alcohol Dependence, 71, 127–131. Shipley, R. H., Hartwell, T. D., Austin, W. D., Clayton, A. C., & Stanley, L. C. (1995). Community stop-smoking contests in the COMMIT Trial: Relationship of participation to costs. Preventive Medicine, 24, 286–292. Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (1997). Reinforcement of cocaine abstinence in treatment-resistant patients: Effects of reinforcer magnitude. NIDA Monograph, 174. Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (1999). Voucher-based reinforcement of cocaine abstinence in treatmentresistant methadone patients: Effects of reinforcement magnitude. Psychopharmacology, 146, 128–138. Silverman, K., Svikis, D., Robles, E., Stitzer, M. L., & Bigelow, G. E. (2001). A reinforcement-based therapeutic workplace for the treatment of


341

drug abuse: Six-month abstinence outcomes. Experimental and Clinical Psychopharmacology, 9, 14–23. Silverman, K., Wong, C. J., Higgins, S. T., Brooner, R. K., Montoya, I. D., Contoreggi, C., Umbricht-Schneiter, A., Schuster, C. R., & Preston, K. L. (1996). Increasing opiate abstinence through voucher-based reinforcement therapy. Drug and Alcohol Dependence, 41, 157–165. Smith, J. E., Meyers, R. J., & Delaney, H. D. (1998). The community reinforcement approach with homeless alcohol-dependent individuals. Journal of Consulting and Clinical Psychology, 66, 541–548. Solomon, L. J., Flynn, B. S., Worden, J. K., Mickey, R. M., Skelly, J. M., Geller, B. M., Peluso, N. W., & Webster, J. A. (1998). Assessment of self-reward strategies for maintenance of breast self-examination. Journal of Behavioral Medicine, 21, 83–102. Stitzer, M. L., & Bigelow, G. E. (1982). Contingent reinforcement for reduced carbon monoxide levels in cigarette smokers. Addictive Behaviors, 7, 403–412. Stitzer, M. L., & Bigelow, G. E. (1983). Contingent payment for carbon monoxide reduction: Effects of pay amount. Behavior Therapy, 14, 647–656. Stitzer, M. L., & Bigelow, G. E. (1985). Contingent reinforcement for reduced breath carbon monoxide levels: Target-specific effects on cigarette smoking. Addictive Behaviors, 10, 345–349. Stitzer, M. L., Rand, C. S., Bigelow, G. E., & Mead, A. M. (1986). Contingent payment procedures for smoking reduction and cessation. Journal of Applied Behavior Analysis, 19, 197–202. Stone, E. G., Morton, S. C., Hulscher, M. E., Maglione, M. A., Roth, E. A., Grimshaw, J. M., . . .Shekelle, P. G. (2002). Interventions that increase use of adult immunization and cancer screening services: A meta-analysis. Annals of Internal Medicine, 136, 641–651. Sutherland, K., Christianson, J. B., & Leatherman, S. (2008). Impact of targeted financial incentives on personal health behavior: A review of the literature. Medical Care Research Reviews, 65(Suppl. 6), 36S–78S. Svikis, D. S., Lee, J. H., Haug, N. A., & Stitzer, M. L. (1997). Attendance incentives for outpatient treatment: Effects in methadone- and nonmethadonemaintained pregnant drug dependent women. Drug and Alcohol Dependence, 48, 33–41.

2/15/2011 7:46:44 PM

342


Tulsky, J. P., Hahn, J. A., Long, H. L., Chambers, D. B., Robertson, M. J., Chesney, M. A., & Moss, A. R. (2004). Can the poor adhere? Incentives for adherence to TB prevention in homeless adults. The International Journal of Tuberculosis and Lung Disease, 8, 83–91. Van Etten, M. L., Higgins S. T., Budney, A. J., & Badger, G. J. (1998). Comparison of the frequency and enjoyability of pleasant events in cocaine abusers vs. non-abusers using a standardized behavioral inventory. Addiction, 93, 1669–1680. Volmink, J., & Garner, P. (1997). Systematic review of randomised controlled trials of strategies to promote adherence to tuberculosis treatment. British Medical Journal, 315, 1403–1406.


Volpp, K. G., Gurmankin Levy, A., Asch, D. A., Berlin, J. A., Murphy, J. J., Gomez, A., Sox, H., Zhu, J., & Lerman, C. (2006). A randomized controlled trial of financial incentives for smoking cessation. Cancer Epidemiology, Biomarkers and Prevention, 15, 12–18. Volpp, K. G., Troxel, A. B., Pauly, M. V., Glick, H. A., Puig, A., Asch, D. A., Galvin, R., . . .AudrainMcGovern, J. (2009). A randomized, controlled trial of financial incentives for smoking cessation. The New England Journal of Medicine, 360, 699–709. Vuchinich, R. E., & Tucker, J. A. (1988). Contributions from behavioral theories of choice to an analysis of alcohol abuse. Journal of Abnormal Psychology, 97, 181–195.

2/15/2011 7:46:44 PM

PART IV

Applications to Cognition, Social Interaction, and Motivation


2/15/2011 7:46:58 PM


CHAPTER 16 Social Learning and Connectionism Frank Van Overwalle

This chapter reviews evidence to demonstrate that many judgments and biases in social cognition can be understood from a connectionist perspective. A basic feature of connectionist modelling is that many social judgments are driven by basic associative learning processes, most often by an error-minimizing algorithm as illustrated in the delta learning algorithm. Two major emergent properties falling naturally out from this learning algorithm are acquisition (sample size effects) and competition (discounting and augmentation). These properties are unique to error minimizing algorithms like delta learning. Empirical evidence is reviewed showing that causal en dispositional attributions are strongly determined by these emergent properties. In addition, a number of simulations are reviewed to illustrate that many other social judgments and biases might result from such connectionist learning processes. These simulations include person impression formation, assimilation and contrast, illusory correlations in groups, subtyping of extreme dissidents, cognitive dissonance, attitude formation through persuasive communication, and recent findings of brain imaging research on person perception. The common theme in this chapter is that a single connectionist learning mechanism—the delta algorithm—is capable of producing emerging properties that explain a rich set of empirical data in social cognition.

INTRODUCTION How might associative learning advance our understanding of social processes? Can a connectionist approach that grew out of and extended classic associative learning tell us something new about social cognition? Social cognition is a subfield of social psychology concerned with the question of how we perceive and interpret the behavior of other human beings in terms of their motives, traits, social constraints, and so on. The aim of this chapter is to demonstrate that social connectionism brings a deeper understanding to the field of social cognition. For one thing, connectionist models may provide a common framework for learning about human beings, inspired on the neural working of the brain, that the traditional approach in social psychology lacks and that explains why this field is currently replete with many unrelated, fragmentary, and

ad-hoc perspectives and theories. Moreover, given the nascent interest in social neuroscience and the growing empirical evidence on the location and timing of brain activation using novel brain-imaging techniques, there is a need to understand how these processes are shaped in the brain. Given their neurological inspiration, connectionist models may fill this gap. They explain content (what is learned and memorized) and process (how it is learned) by a single mechanism, unlike earlier approaches that often see these aspects as driven by different processes taking place at different stages. Perhaps most fundamentally, connectionism views social cognition as a constant learning and adaptation in a changing environment, and our memories, inferences, and judgments as a natural outcome of that process. This chapter is subdivided into three main sections. The first section introduces the basics

345


2/15/2011 7:46:58 PM

346


of associative or connectionist learning and illustrates how important properties in social reasoning emerge from this. The second section demonstrates how these properties are revealed in empirical research on causal and dispositional attribution that my research team has conducted in the last 10 years. The last section involves simulations of other important empirical findings in social psychology, to demonstrate the breath and value of the connectionist framework.

WHAT IS CONNECTIONIST LEARNING? In social cognition, information processing is often explained in terms of spreading activation models, in which social concepts such as persons, groups, behaviors, traits, and attitudes are represented by highly interconnected units, and social judgments are explained by the output of spreading activation between these units. In connectionist networks, these units are typically grouped together in layers, most often comprising at least an input layer and an output layer. Figure 16.1A demonstrates this architecture in the most simple model with only forward connections between the input and output layer (i.e., feedforward model), whereas Figure 16.1B demonstrates a more complex model that includes all connections between units, in both a forward and backward order (i.e., recurrent model). To provide some flesh and blood to these rather abstract models, Figure 16.1C depicts a generic example of how such a model might look like in a simulation of social phenomena. As we will see, many simulations in this chapter deal with agents (often involving a target person, and sometimes also another person, object or group, or situational constraints that limit the actions of the target) that produce some effect that can be observed in the agents’ behaviors or other characteristics (e.g., traits, preferences). The connections in this example reflect the tendency of the agents to engage in a given behavior or to possess a given trait. Common to all these models in Figure 16.1 is that the strength of the connections determines long-term memory storage and that the flow of activation along the connections reflects


processing this information. Consequently, these connectionist models are more powerful than earlier spreading activation models. These earlier models leave the development of connections and the flow of activation often unspecified. Even in more recent developments of this notion in the format of constraint satisfaction models, the strength of the connections is determined a priori by the researcher and judgment is believed to result solely from how the activation flows and settles in the network. In contrast, the feedforward en recurrent connectionist models shown in Figure 16.1 and applied throughout this chapter not only specify the flow of activation but, more crucially, also include a learning mechanism that determines how the connections between the units can change so that they provide a flexible storage for long-term memory. These two processes are done in parallel by the operation of the interconnected units themselves, so that connectionist systems have no need for a central executive. This capacity to self-learning and self-organization allows the connectionist approach to get rid of the problem of the “homunculus” in the brain that makes our mental decisions. Although these learning processes are, in principle, working to accurately understand the human environment, they sometimes lead astray into biases and shortcomings of social reasoning, some of which we review in this chapter. Learning and Adaptation

Unlike spreading activation models, most connectionist networks are able to learn over time, usually by means of a simple learning algorithm that progressively modifies the strength of the connections between the units making up the network. Learning is modeled as a process of on-line adaptation of existing knowledge to novel information provided by the environment. Specifically, the network changes the weights of the connections between units (e.g., between an agent and his or her characteristics, or between causes and their effects) and so reflects the accumulated history of co-occurrences between these stimuli. Because the weights of connections change slowly, the connections are conservative

2/15/2011 7:46:58 PM

SOCIAL LEARNING AND CONNECTIONISM

347

A. Feedforward

B. Recurrent

Output

Output

Internal input

External input

External input C. Generic Example Effects:

Agents:

Trait

Behavior

Target Person

Other/ Group

Situational Context

Figure 16.1 (A and B) A feedforward and recurrent network. The output layer is depicted at the top, and

the input layer is depicted at the bottom. External activation is fed in the network at the input layer and flows along the connections to the output layer directly (feedforward and recurrent network) or indirectly via internal connections (recurrent network only; additional connections are indicated by dotted arrows). (Reprinted with permission from Figure 3.1 of Van Overwalle, F. 2007. Social connectionism: A reader and handbook for simulations. New York, NY: Psychology Press). (C) A generic example of many simulations that are addressed in this chapter, embedded in a feedforward architecture. Most simulations use one or more of the units depicted here. The arrows (i.e., connections) in this example reflect the tendency of an agent to engage in a given behavior or to possess a given trait.

and reflect past as well as novel co-occurrences. As such, they are the repository of the network’s long-term memory, modified only in part by recent learning. The model’s memory is retrieved, not by passively reading connection weights, but by reactivating some units of interest by current inputs. As a consequence, retrieval is a reconstructive blend of input as well as a mixture of old and recent information embedded in the weights. The historic predecessors of the connectionist approach, associative learning models, also represent causal strength in memory as an association


between two stimuli and describe how the weight of this association is adjusted on-line as new information is received (for an overview, see Allan, 1993; Shanks, 1995). One of the most popular associative learning models by Rescorla and Wagner (1972) is, in fact, identical to the delta learning algorithm used in many connectionist models (McClelland & Rumelhart, 1988). This delta learning algorithm is used throughout all simulations in this chapter. Given that associative learning models were designed to explain also animal learning and conditioning, they provide the connectionist perspective with an

2/15/2011 7:46:58 PM

348


evolutionary perceptive and an extensive research base from which social researchers can draw a lot of interesting data. Thus, the phenomena presented in this chapter are naturally developed from old learning or conditioning processes that other organisms besides humans also exhibit.

more fundamental associative learning processes. We discuss two properties in more detail, because they form the basis of all empirical studies and simulations in this chapter.

EMERGENT PROPERTIES OF ACQUISITION AND COMPETITION

We have seen that, because a connectionist network gradually reduces errors, it slowly approaches the statically predicted weight between input and output, or covariation. Consider a simple case in which cause A is repeatedly presented with effect E (see Fig. 16.2A). In the network, the activation of each unit is turned on (set to +1) and activation spreads from A to E. Because initially, the A→E connection or causal strength is 0, cause A does not predict effect E at all and there is a large error. This error is gradually reduced each time A and E are presented together, so that the A→E weight increases slowly and converges toward the statistical norm of +1, at which point cause A fully explains or predicts the effect E. In contrast, when A is no longer followed by the effect (e.g., see A° from trial 9 onward), then its weight starts to decrease and converge toward 0, at which point A° does not explain or predict the effect. Thus, the network eventually learns the best weight of the connections that predict most accurately when and to what degree an output (e.g., effect E) will occur when an input (e.g., cause A) is present. This process is consistent with our intuitions and experimental evidence. Often, we do not jump immediately to conclusions; rather, we build on several experiences to shape our judgments and estimates. This property of gradual acquisition, which slowly strengthens our judgments, is also known as the sample size effect. An effect of sample size has been documented in many areas of social judgment. For instance, when receiving more supportive information, people tend to hold more extreme impressions about other persons (Anderson, 1967, 1981), make more polarized group decisions (Ebbesen & Bowers, 1974; Fiedler, 1996), endorse hypotheses more firmly (Fiedler, Walther, & Nickel, 1999), make more extreme predictions (Manis, Dovalina, Avis, & Cardoze, 1980), agree more

The delta algorithm represents learning as an adaption of the internal representation of the mental system to match the external environment. This internal representation or internal activation is determined by the activation flow in the network. Specifically, all interconnected units sent activation to each other in proportion to the weight of their connections, and the received activations are accumulated (e.g., summed) in each unit. This accumulated internal activation is compared with the external activation received from the outside environment. Differences between these two activation levels reflect errors in the system’s representation, and they are gradually reduced to reach a more accurate representation. This error reduction is determined by a learning rate that controls the speed of learning (typically around .10 in symbolic representation architectures where each unit represents a meaningful high-level concept like person, trait, etc.). Although the delta learning algorithm has no other goal than error minimization, given sufficient learning experiences, the weights between input and output converge to the probabilistic norm of contingency or covariation (see Chapman & Robbins, 1990; Van Overwalle, 1996). Thus, by reducing error gradually, the delta learning algorithm is sensitive to the accurate covariation between stimuli, such as causes and effects. Gradual learning and error minimization is crucial, because otherwise new learning would erase old memories dramatically (a phenomenon known as catastrophic interference; McCloskey & Cohen, 1989), so that covariation estimation over longer time periods would be impossible. By learning in this manner, the delta learning algorithm reveals other properties that can explain social human behavior on the basis of


Acquisition and Sample Size

2/15/2011 7:46:58 PM

SOCIAL LEARNING AND CONNECTIONISM A. Acquisition

349

Causal strength

1.0

E

A

A

0.8 0.6 0.4 0.2 A° 0.0 0

B. Competition

8 Trials

12

16

B

Causal strength

1.0

E

A

4

A

0.8 0.6 0.4

B

0.2 0.0 0

4

8 Trials

12

16

Figure 16.2 (A) Example of acquisition of cause A. (B) Example of competition between a stronger cause A and a weaker cause B. Gray denotes an active unit. Full lines denote strong connectionist weights, and broken lines denote weaker connection weights. A° denotes that cause A does not co-occur any more with the effect E (i.e., E is not activated any more during these trials). The learning rate in these examples is .20. (Reprinted with permission from Figure 3.2 of Van Overwalle, F. 2007. Social connectionism: A reader and handbook for simulations. New York, NY: Psychology Press).

with persuasive messages (Petty & Cacioppo, 1984), and make more extreme causal or dispositional judgments (Baker, Berbier, & ValléeTourangeau, 1989; Shanks, 1985, 1995; Shanks, Lopez, Darby, & Dickinson, 1996; Van Overwalle, 2003; Van Overwalle & Van Rooy, 2001a). Competition, Discounting, and Augmentation

Discounting is a general tendency where “the role of a given cause in producing a given effect is discounted if other plausible causes are also present” (Kelley, 1971, p. 8). One of the most common examples of discounting in social cognition is when internal attributions to the actor are discounted given evidence on the potent influence of external pressures. The opposite tendency is described in the augmentation principle. This principle specifies that “if for a given


effect, both a plausible inhibitory cause and a plausible facilitatory cause are present, the role of the facilitative cause in producing the effect will be judged greater” (Kelley, 1971, p. 12). For instance, a person’s success is more strongly attributed to internal capacities when the task was hard rather than easy (for an overview, see McClure, 1998). In a connectionist network, discounting and augmentation are natural consequences of the emergent property of competition. The term “competition” stems from the associative learning literature (Rescorla & Wagner, 1972). One prime example of discounting in the associative literature is blocking. Blocking predicts that when one stimulus already accurately predicts an effect through a strong A→E connection, then the development of additional connections of other stimuli with E is blocked. The reason is that prediction in the system is

2/15/2011 7:46:58 PM

350


determined by the summed activations from all causes. The accurate prediction from the strong A→E connection leaves no error in the system, so that no further learning takes place. As another example, Figure 16.2B depicts a type of overshadowing from the conditioning literature. In this example, cause A is always presented with the effect E, either alone or in compound with cause B. Specifically, while cause A always co-occurs with the effect E, cause B co-occurs only in half of the cases. Given the summed activations from both causes when the compound AB is presented, at a certain moment, the effect E is overestimated (in trial 10 the internal activation sums to 1.08), resulting in a downward adjustment of the connection weights. In the long run, the decrement is most detrimental for the weaker cause B, while the strength of A slowly increases at the expense of B (i.e., A overshadows B). Competition can be understood as though the two explanations compete for the available strength, which is limited to +1 or the maximum magnitude of the effect to explain. The opposite effect of augmentation is also a direct consequence of the competition property and is also known as superconditioning in the associative learning literature. When one of two causes is inhibitory, then the other is cause is highly facilitatory in order to compensate for the inhibitory effect. That is, when there is a negative A→E connection, given that the summed activations from both causes predict the effect, then the cause B develops a strong positive connection with E so that an accurate joint prediction of E can be reached. The notion of competition is consistent with the tendency of people to prefer a simple, single explanation. Competition is a robust finding in empirical research on human causal attribution (Hansen & Hall, 1985; Kruglanski, Schwartz, Maides, & Hamel, 1978; Shanks, 1985; Van Overwalle & Van Rooy, 2001b; Wells & Ronis, 1982) and impression formation (Gilbert & Malone, 1995; Trope & Gaunt, 2000; Van Overwalle, 2006). Besides stating that competition exists, many earlier models in social cognition are incapable of providing a processing mechanism for it (e.g., Kelley, 1971). And even though a Hebbian connectionist learning algorithm is capable of


simulating sample size differences in person and group impressions (Kashima & Kerekes, 1994; Kashima, Woolcock, & Kashima, 2000), it cannot produce competition. Because connection weights in the Hebbian algorithm depend on the accumulated number of co-occurrences of stimuli without any upper bound that limits the predicted output, it does not incorporate a competition property. Other Emergent Properties

Intriguingly, the properties we just discussed are not directly built into the model or delta learning algorithm; rather, they fall naturally out of specific learning histories and are therefore termed emergent. That is, in the model there are no a priori built-in weights or other parameters that weaken or strengthen the weights of some units, but not others. For instance, the effect of sample size depends solely on an increasing number of learning experiences. Likewise, the competition effect follows from a learning history involving strong and weak connections. This contrasts with less powerful models such as constraint satisfaction, which often require a priori built-in inhibitory connections between stimuli to produce competition between them (e.g., Read & Miller, 1993, p. 535). There are a number of other emergent properties. Information in connectionist models can be represented at a symbolic level (i.e., localist representation), where each unit represents a meaningful concept (e.g., person, cause, trait, and so on). Conversely, information can also be represented at a lower, subsymbolic level, where each unit has no symbolic meaning as such but instead represents features of it (e.g., parts of characters or letters from verbal input, perceptual features from visual input). Consequently, each symbolic concept is reflected by a pattern of activation across a large set of units representing subsymbolic features (i.e., distributed representation). This makes connectionist models even more powerful. A distributed representation solves the problem of how new concepts are represented in the system—not by new units but by redeploying existing units. This also leads to a number of other emergent properties such as the

2/15/2011 7:46:59 PM


ability to extract prototypes from a number of exemplars because these exemplars share prototypical features (prototype extraction), to recognize exemplars based on the observation of incomplete features because features are strongly interconnected so that missing information is “filled in” by activation flow (pattern completion), to generalize knowledge about features to similar exemplars because these new exemplars share similar features (generalization), and to lose stored knowledge only partially after damage because each concept is represented by a large amount of features, some of which may even be redundant (graceful degradation; for an accessible introduction, see McLeod, Plunkett, & Rolls, 1998). To keep things simple and easy understandable, however, this chapter avoids a distributed representation. We focus on a localist representation and the properties that emerge from this alone, namely, the properties of acquisition and competition. Given that earlier models in social cognition have difficulty explaining these properties, exploring these properties in social judgments can convincingly demonstrate that connectionist principles also underlie social cognition and thinking. In the next section, we turn to a series of studies that tested whether social judgments are consistent with these connectionist properties.

STUDIES EXPLORING ACQUISITION AND COMPETITION IN SOCIAL JUDGMENTS Do people obey the property of acquisition (sample size) and competition (discounting) when they make judgments of social events? Most studies with humans typically used experimental tasks that contained little social material to demonstrate an effect of sample size (Baker et al., 1989; Shanks, 1985, 1987, 1995; Shanks et al., 1996) and competition (Shanks, 1985; Van Hamme, 1994; Williams & Docking, 1995). It is thus necessary to test the robustness of these effects in the social domain. We tested these predictions in our lab using social judgments and compared the obtained results with connectionist


351

simulations of the experimental manipulations. Because these studies focused on empirical replication, we used the simplest, feedforward network for these simulations. We directed our attention to two types of judgments that dominate the social cognition literature, in particular, causal and dispositional attributions. What are these social judgments? Causal and Dispositional Attributions Inferred from Covariation Information

When trying to understand an actor’s behavior (e.g., Sally stepped on Peter’s shoes during the foxtrot), observers often seek the cause of the behavior. These causes can involve temporary and specific aspects of the actor (e.g., Sally did not pay attention); of the object—in this case, the other person (e.g., Peter made a silly remark); or of the situation (e.g., the dancing hall was too crowded). They can also refer to enduring or general dispositional traits of the actor (e.g., Sally is clumsy), of the other person (e.g., Peter is nasty), or of the situation (e.g., the foxtrot is difficult). Whereas causal attributions refer to all possible explanations, dispositional attributions refer to the subset of causes involving only enduring traits of a person. This distinction is important. Although resulting from a similar process, attributions to traits versus mere causes (i.e., that are no traits) have very different social consequences. For instance, trait attributions have strong implications on how we evaluate the other person and future interactions with him or her, whereas situational explanations have fewer consequences for the actor. Given the great variety of possible causes, however, research investigating causal attributions often limits possible answers to general categories, phrased as “something about” the person, the object, or the situation (Kelley, 1967). This is also how it was done in our studies. Other research investigates person impressions (Anderson, 1981). This typically involves general evaluative impression of people (e.g., are they likeable?) based on their behaviors or trait information. In person impression research, these behavioral or trait descriptions refer uniquely

2/15/2011 7:46:59 PM

352


and directly to the actor, so that the causal role of the actor is taken for granted. In contrast, in attribution research, observers have to infer the actor’s potential causal role from relevant covariation information. In our research presented in this chapter, we focus on consensus and distinctiveness covariation information (Kelley, 1967). In one of the most influential theories of causal attribution in social psychology, Kelley proposed that these are among the prime social sources from which people infer covariation. Consensus refers to the extent to which behaviors or outcomes of an actor generalize to other, similar actors, whereas distinctiveness refers to the extent to which outcomes given an object do not generalize to other, similar objects. High covariation of an actor is implied given low consensus (i.e., only this actor behaved in this manner; e.g., Sally stepped on Peter’s shoes whereas others did not) and this leads to strong attributions to the actor (e.g., Sally). Similarly, high covariation of an object is implied given high distinctiveness (i.e., the behavior occurred only with this object; e.g., Sally stepped on Peter’s shoes but not those of other dancers) and leads to strong attributions to the object (e.g., Peter). In contrast, low covariation is implied given the reversed patterns of high consensus or low distinctiveness. That is, high consensus (i.e., everybody behaved in this manner; e.g., Sally and many others stepped on their partners’ shoes) implies little actor causality. Low distinctiveness (the behavior generalized across stimuli; e.g., Sally stepped on Peter’s shoes and those of all her dance partners) implies low object causality. Sample Size in Causal Attribution

While many connectionist models predict an effect of sample size, surprisingly, most attribution studies and models in social cognition around the turn of the millennium ignored the question of sample size (e.g., Cheng & Novick, 1990; Försterling, 1989; Hewstone & Jaspars, 1987; Hilton & Slugoski, 1986; Kelley, 1967; Orvis, Cunningham, & Kelley, 1975; Read & Marcus-Newhall, 1993; but see Försterling, 1992). These models took a statistical approach


to causal learning and accounted only for the final end result of the process. They implied that the number of observations should not affect people’s causal estimates. Some models attempted to overcome this limitation by incorporating an updating rule that made them sensitive to sample size (Busemeyer, 1991; Hogarth & Einhorn, 1992). However, a major restriction was that the proposed rule involved only a single cause and did not take into account the influence of alternative causes (e.g., competition). Given a single cause, these models are actually mathematically identical to the delta algorithm (Wasserman, Kao, Van Hamme, Katagiri, & Young, 1996; Appendix D) and are therefore not considered here. In sum, whereas earlier prominent statistics-based models of causality predict no effect of sample size, connectionist models predict that subjects incrementally adjust social ratings in the direction of the true covariation between stimulus and effect, the more observations are made. In one of our earliest studies (Van Overwalle & Van Rooy, 2001a), we addressed the question of sample size in causal attribution. Our basic idea was simple. We repeated information on the covariation between a cause and an effect (see Fig. 16.2A), without changing the level of covariation. By simply providing the same information repeatedly or only once, this design allows analyzing the effect of sample size while controlling for covariation. In an early acquisition experiment (Van Overwalle & Van Rooy, 2001a, Experiment 1), participants (n = 97) received two different levels of covariation (0% or 100%) by manipulating consensus and distinctiveness, and this was repeated across six blocks of trials. To illustrate with the distinctiveness manipulation, the participants read information that “Jasmine deceived her friend, Corinne.” Low (0%) covariation of the target object, Corinne, was obtained when the outcome of a comparison object was identical: “Jasmine deceived her friend, Karen”; and high (100%) covariation was obtained when the outcome was absent: “Jasmine did NOT deceive her friend, Karen.” Each trial was displayed consecutively on a separate screen in a random order for each participant. After this block of

2/15/2011 7:46:59 PM



70

100%

60 50 40 30

0%

(

Attribution Rating Means, ‘ ’ Simulation)

80

20

2

4

6

8

10

12

Trial

Attribution Rating Means, ‘ ’ Simulation)

80 70

(

two trials, the participants rated the causal influence of the target. For instance, they rated how much influence “something special about Corinne” had on the outcome, using an 11-point rating scale ranging from 0 (absolutely no influence) to 100 (very strong influence), with midpoint 50 (partial influence). This was repeated for each block involving the same target object (i.e., Corinne) and a novel comparison object (i.e., a novel name instead of Karen). The manipulation of consensus was similar, and it involved the presentation of the same target actor and different comparison actors (i.e., covariation of the agent) across blocks. The data, collapsed across consensus and distinctiveness information conditions, are depicted in Figure 16.3 (top panel). Consistent with the sample size predictions, the ratings show a steady increase over trials in the 100% covariation condition and a steady decrease over trials in the 0% condition. Consistent with a delta error-correcting algorithm, the curves in the figure depict a linear trend indicating a significant increase or decrease across trials, together with a smaller quadratic trend indicating that this linear change becomes smaller toward the end (i.e., showing less learning closer to asymptote when the error reaches zero, a pattern that reflects a kind of error-minimizing process predicted by the delta algorithm). Note in this and subsequent studies that judgments start off at midrange scale values for the first ratings, which might be plausible given the little information participants have at that moment. We saw in this experiment that observers progressively adjusted their causal ratings when subsequent covariation information confirmed their initial causal judgments. A different question is whether they would also adjust their judgments when subsequent information conflicts with their initial judgments, and how far this correction would go. This is an important question, because normatively, causal ratings should return to baseline when they are contradicted by the same amount of novel data. This question was addressed in a second acquisition experiment (Van Overwalle & Van Rooy, 2001a, Experiment 3). Participants (n = 101) were given an initial block of four trials in which a given

353

20

100% 50%

60 50 40

0%

30 Control 4

5

6

7

8

Trial

Figure 16.3 Causal attribution ratings (indicated by full lines) and feedforward simulation (indicated by dotted lines) in function of covariation level and number of trials. Covariation is denoted as 100% (= Low Consensus/High Distinctiveness), 50% (= Midrange Consensus/Distinctiveness), or 0% (= High Consensus/Low Distinctiveness). (Reprinted with permission from Figures 1 and 3 of Van Overwalle, F., & Van Rooy, D. 2001a. When more observations are better than less: A connectionist account of the acquisition of causal strength. European Journal of Social Psychology, 31, 155–175).

target always covaried with the outcome (to built up sufficient causal strength), and then received information in a second block of four trials where the same amount of 100% covariation was given (confirming initial judgments) or was lowered to 50% (partly disconfirming initial judgments) or to 0% (entirely disconfirming initial judgments). The latter 0% condition was compared with a control baseline condition in which

2/15/2011 7:46:59 PM

354


subjects received 0% covariation all the time. As depicted in Figure 16.3 (bottom panel), the results confirm the connectionist predictions. The ratings show an increase over trials in the 100% condition, a marginal decrease in the 50% condition, and a substantial decrease in the disconfirming 0% condition, which closely approaches the control condition at the last trial. This latter result confirms that ratings are brought back at the baseline level, when initial impressions are immediately contradicted. The jagged-like pattern of the 50% condition is due to the fact that 50% covariation was reached on even trials only. Again, all the conditions revealed a significant linear increase or decrease of the target ratings, and also a marginal to significant quadratic trend indicating that this change became smaller at the end. To evaluate how closely a connectionist framework can approximate our data, we ran simulations using a feedforward network, with exactly the same input information and order of trials as in the experiments (Van Overwalle & Van Rooy, 2001a). The network consisted of a target unit and a comparison unit representing either agents (for consensus) or objects (for distinctiveness), and an outcome unit. When a target or comparison agent/object was present, the unit was turned on (activation = +1) and when absent, the unit was turned off (activation = 0). Similarly, when the outcome was obtained, the activation was set to +1, and when the outcome was absent, the activation was set to 0. The connections were adjusted after each trial. The target→outcome connection represents the causal influence of the target in producing the outcome. Note that in this and all other simulations in this chapter, to visually match the simulation results with the observed data, the simulated values were regressed on the observed ratings. Of importance are not the exact simulation values, but their pattern. As can be seen in Figure 16.3, these simulations closely match the observed data. This is confirmed by the high overall correlations between empirical and simulated data; r = .99 for the first and second experiment. As a way of comparison, we also ran simulations of two major statistical models of Cheng and Novick


(1990) and Försterling (1992), extended with additional parameters that take into account sample size. We did our very best to come up with extensions that were reasonable and effective. Nonetheless, the fit was generally poor for these two statistical models, because some data sets revealed zero correlations. The reason is that these sample size extensions are artificial additions to these models and are not an inherent part of it, as is the case for connectionist models. Sample Size in Dispositional Attribution

In a subsequent acquisition experiment, Van Overwalle (2003; Experiment 2) explored whether the sample size effect would also be revealed for dispositional attributions. As in the previous experiments, participants were given high or low levels of covariation, and this information was repeated for six blocks while keeping the level of covariation constant. To obtain strong and unbiased manipulations of dispositional attributions, consensus and distinctiveness were manipulated simultaneously (see Van Overwalle (1997), resulting in either 100% covariation for the target person (“Target” condition; e.g., only Jasmine and nobody else cheats all her friends), 100% for the other comparison person (“Other” condition; e.g., everybody cheats only Corinne and no other friends), or intermediate covariation levels where covariation with the actor and object are both high (“Both” condition reflecting an interaction between persons; e.g., only Jasmine cheats Corinne and nobody else) or both low (“None” condition; e.g., everybody cheats everybody else). As before, each trial was displayed consecutively on a separate screen in a random order for each participant. The acquisition of dispositional attributions was monitored by requesting dispositional trait ratings after each block of trials. For instance, the participants had to judge “to what extent is Jasmine untrustworthy” on an 11-point scale (0 = not at all untrustworthy to 10 = very much untrustworthy), and they also had to judge “to what extent is Corinne naive” on a similar 11-point scale (0 = not at all naive to 10 = very much naive). The results are depicted in Figure 16.4 (top panel) and demonstrate that dispositional

2/15/2011 7:46:59 PM

SOCIAL LEARNING AND CONNECTIONISM 9

Target Dispostion

8 Target 7 None 6 5

Both

4 3

Other 1

2

3

4

5

6

Block - Repeated 9

Target Dispostion

8

Target

Target 7

None

None 6 Both 5

Both Other

4 3

1

2

Other

3

4

5

6

Block - Reversed

Figure 16.4 Dispositional ratings of the target in function of covariation pattern. (Top) Repeated condition. (Bottom) Reversed condition. Covariation is denoted as target (100% for the target person), other (100% for the comparison person), none (0% for both target and comparison persons), or both (100% for both). (Reprinted with permission from Figure 2 of Van Overwalle, F. 2006. Acquisition of dispositional attributions: Effects of sample size and covariation. European Journal of Social Psychology, 33, 515–533).

attributions are sensitive to sample size in line with connectionist predictions. There is a steady increase when covariation is 100% for the target (see line on the top denoted “Target”), and there is a steady decrease when covariation is 100% for the other comparison person (see line on the bottom denoted “Other”). For instance, when only Jasmine cheats her friends (100% “target” covariation), participants describe her as increasingly untrustworthy, whereas if everyone cheats


355

Corinne and nobody else (100% “other” covariation), Jasmine is seen as decreasingly untrustworthy. This linear trend was significant in both conditions. In contrast, for the intermediate conditions, dispositional judgments remained relatively flat and ended at an intermediate level, also in line with connectionist predictions. There was a significant linear decrease when both factors covaried (see line denoted “Both”), but only a marginal linear trend when none of them covaried with the outcome (see line denoted “None”). I simulated these manipulations using a feedforward network and using exactly the same architecture, information input, activation settings, and order of trials as in the experiment. However, because the two information conditions were manipulated simultaneously, there were now separate units representing the actor (for consensus) and the object (for distinctiveness) and their respective comparison cases. The simulations showed a strong fit with the empirical data with a correlation of r = .91. The extended statistical models described earlier, obtained much lower correlations of r = .37 (Cheng & Novick, 1990) and r = .24 (Försterling, 1992). Van Overwalle (2003; Experiment 2) also ran another condition in which the original covariation was reversed and disconfirmed in the second half of the experiment. Normatively, this should lead to a null effect at the end. As can be seen in Figure 16.4 (bottom panel), this prediction was supported because there were no significant differences between all four conditions after the last block. Crucially, in line with an acquisition property, the ratings first increase or decrease in the first half of the experiment, and then return to their baseline in the second half. Discounting and Augmentation in Causal Attribution

Now that we have seen that causal and dispositional attributions obey the acquisition property emerging from connectionist learning, would they also reveal the competition properties of discounting and augmentation? If the number of observations for a competing explanation

2/15/2011 7:46:59 PM

356


goes up, would a target explanation be more discounted or augmented? It seems intuitively plausible that, for instance, when there is growing evidence indicating that a successful task was easy, then the person’s ability is more discounted. Conversely, the greater the evidence that the task was hard, the more the person’s ability is augmented. As indicated earlier, most learning or attribution models in psychology fail to incorporate competition, especially when combined with the property of acquisition. In contrast, the connectionist delta algorithm predicts that competition against a target cause should become stronger whenever the alternative cause becomes more facilitatory (leading to increased discounting) or more inhibitory (leading to increased augmentation). To test this connectionist prediction, Van Overwalle and Van Rooy (2001b) capitalized on the earlier sample size manipulation. They induced changes in the discounting and augmentation of a target cause by varying the number of observations (or sample size) of the alternative cause, while keeping its degree of covariation constant. For instance, consider that Theo has several tennis team players of whom five (large size) or only one (small size) won a singles tennis match earlier on. Next, Theo and his team players win a series of doubles matches. The contribution of Theo tends to be discounted given the earlier win of his team player(s), and especially so when five rather than one team player won, because in the former case one’s confidence in the tennis talents of the team players is much higher. Conversely, consider that Theo’s team players lost either five singles matches (large size) or one singles match (small size) before the successful doubles matches with Theo. In that case, Theo’s contribution for winning the doubles matches is augmented, and even more so when five rather than one team players lost their singles match, because in the former case one is much more certain about the team players’ poor tennis talents. In sum, competition effects are increased given a large rather than a small sample size of the competing cause. In a first competition experiment (Van Overwalle & Van Rooy, 2001b, Experiment 2), participants (n = 115) received information about


the outcomes of an alternative cause and the joint outcome of a target and alternative cause, much like in the example of Theo. Interestingly, this information was presented not only sequentially (i.e., one trial after the other) as in the previous acquisition experiments but also in a summary format during one screen shot. Thus, for instance, discounting of one (versus five) stimulus was described in the summary format “Annie and one (five) other salesgirl(s) attained high sales figures for perfumes,” whereas in the sequential format this information was presented trial by trial for each salesgirl separately. Both formats have ecological validity. A sequential format resembles how perceivers receive information from direct observation, whereas a summary format resembles how perceivers pick up information during social conversation. After all information was provided on a target person, participants rated how much influence “something special about [the target]” had on the outcome, using an 11-point rating scale ranging from 0 (absolutely no influence) to 100 (very strong influence), with midpoint 50 (partial influence). The results are depicted in Figure 16.5 (top panel). Consistent with a connectionist perspective, in comparison with a small sample size of the alternative cause, given a large size, the target cause is significantly more augmented or discounted. This is the case for both a sequential and a summary format (although in the sequential format, some amount of competition is already seen given a small size). Connectionist simulations using a feedforward network and using exactly the same input and order of trials as in the experiment show a strong fit with the empirical data with a correlation of r = 1.00 with either the discounting or augmentation data. The extended statistical models described earlier obtained much lower mean correlations of r = .50 (Cheng & Novick, 1990) and r = .00 (Försterling, 1992). Discounting and Augmentation in Dispositional Attribution

Van Overwalle (2006; Experiment 2) investigated whether and how discounting and augmentation of dispositional and causal attributions differ.

2/15/2011 7:46:59 PM


Target Cause

90

Summary

70 Augmentation Discounting

50

30

9

Target Cause

Sequential

Small

Large

Small

Dispositions

Large

Causes

7

Augmentation Discounting

5

3 Small

Large

Small

Large

Figure 16.5 Dispositional and causal ratings after

discounting and augmentation as a function of a small or large sample size. (Top) Causal attributions given a sequential trial-by-trial or summary format. (Reprinted and adapted with permission from Figure 3 of Van Overwalle, F., & Van Rooy, D. 2001b. How one cause discounts or augments another: A connectionist account of causal competition. Personality and Social Psychology Bulletin, 27, 1613–1626) (Bottom) Dispositional and causal attributions. (Reprinted with permission from Figure 2 of Van Overwalle, F. 2006. Discounting and augmentation of dispositional and causal attributions. Psychologica Belgica, 46, 211–234).

As in the previous experiment (see top panel of Fig. 16.5), the strength of a causal or dispositional attribution to a target actor (or object) was varied by manipulating the number of observations (i.e., sample size) of an alternative actor (or object). Participants (n = 126) viewed this material presented in a sequential format and then rated each actor on causal and dispositional ratings as described earlier. The results are depicted in Figure 16.5 (bottom panel) and


357

indicate that a greater sample size of the alternative actor (or object) resulted in greater discounting or augmentation of the target, and that this effect was alike for causal and dispositional attributions. Again, these results are consistent with a connectionist approach. Although the results thus far indicate that the competition effect is a robust phenomenon, an important limitation is that the information was relatively well structured. In all previous experiments, information on a specific target actor always appeared in one or several successive blocks of trials, while information on other target actors appeared in other successive blocks. The trials were randomized only within each block. Would these competition effects survive when information about the target is presented in a much less structured and chaotic manner, as in real life where we pick up information piece by piece about varying persons at different moments in time? Interestingly, in a follow-up experiment, Van Overwalle (2006; Experiment 3) made the encoding of this information more difficult and ecologically realistic by randomly shuffling all trial information on all stories of all target causes of the previous experiment, and presenting all this information in a single block before ratings were made. If, as suggested by a connectionist approach, dispositions and causes are developed on-line rather than by explicitly estimating frequencies and making calculations on them as statistical models suggest, then the effect of sample size and competition should also appear when it is much more difficult to extract these frequencies. The results showed that discounting and augmentation were revealed for causal attributions, but not for dispositional attributions. In contrast to the pattern revealed in Figure 16.5 (bottom left), in this latter case, the lines representing discounting and augmentation were essentially flat and overlapped each other. This is an unexpected result. One potential explanation for the lack of competition between dispositional attributions under difficult encoding conditions is the fundamental attribution error. This bias indicates that in explaining someone’s behavior, perceivers often emphasize too much an actor’s dispositions and ignore situational information

2/15/2011 7:46:59 PM

358


(e.g., competing factors). Research has shown that this occurs most often under processing constraints (Gilbert & Malone, 1995; Trope & Gaunt, 2000). Thus, the processing difficulty in this experiment may have led participants to focus more on trait-relevant information while ignoring (competing) situational information. However, this argument in itself does not explain why this attribution error occurred only for dispositional inferences and not for causal attributions. Perhaps, adjusting dispositional inferences in the light of alternative situational explanations might consume more cognitive effort than causal attributions, so that they are more vulnerable to manipulations that render the extraction of information more difficult. Is there any ground for such argument? There is. Hilton, Smith, and Kim (1995) and Van Overwalle (1997) found that because dispositions refer to stable and enduring characteristics, perceivers rely more on Kelley’s (1967) covariation evidence that reflects generalization across comparison cases such as low distinctiveness (i.e., identical behaviors across different situations). In contrast, for causal attributions, they rely more on differences such as low consensus (i.e., differences between actors’ behaviors). By relying more on the generalization of a single actor’s behavior across different situations for making dispositional attributions and less on differences between situations, attention to situationspecific information may have been limited so that competition failed.

SIMULATIONS OF SOCIAL JUDGMENTS AND BIASES Having demonstrated that connectionist networks provide a valid framework to account for various social judgments like causal and dispositional attributions, we now turn to a series of simulations in which the connectionist approach was further explored for a number of important phenomena and findings in the social cognition literature. By exploring these major findings and modeling them from a connectionist perspective, the scope and usefulness of this framework can be fully appreciated. I begin where we left off in the previous section, that is, with person


impression formation. Next I turn to biases in group judgments and then discuss attitude formation. Finally, I end with a simulation of recent social neuroimaging data. An overview of these topics and findings is listed in Table 16.1, together with the properties that drive the simulations. All the simulations are introduced in a nontechnical manner, mostly by describing or picturing the assumed properties of acquisition and competition underlying the social process, so that it only requires an intuitive understanding of how the simulations are run. Before embarking on some specific simulation, it is informative to describe a number of general characteristics of all simulations. All simulations used a recurrent network because this architecture allows reproducing more complex social effects, except for the feedforward simulations of cognitive dissonance (Van Overwalle & Jordens, 2002) that were published earlier. In each simulation, a simulated learning history replicates the exact number and order of trials in an experiment, or it makes some simplifying but reasonable assumptions on the minimal number of trials necessary to develop weak or strong connection weights to mimic how participants built up prior knowledge in their lives. Each condition is run in a separate simulation. Moreover, often the simulation of all conditions is repeated 50 or 100 times in which each “run” represents a single “participant.” This repetition introduces some realistic noise in the input data by mimicking differences between real participants in the order of trials (i.e., when randomized in the actual experiment) or in their prior knowledge (i.e., by providing slightly different initial connection weights or activation levels). The results of the simulations are often directly compared with some empirical data, by projecting the simulated values on the empirical data so that the fit in the pattern of simulated and observed data can be immediately observed and evaluated. Person Impression Formation Competition

To introduce how our simulation approach works, consider the last empirical study from the previous section (Van Overwalle, 2006). This study

2/15/2011 7:47:00 PM


359

Table 16.1 Overview of the Simulated Social Cognition Topics and the Underlying Connectionist Properties Topic

Findings

Property

Person Impression Formation Discounting and sample size

Discounting of an actor’s trait when there is more evidence on an alternative actor

Acquisition of alternative→trait link which leads to competition against target→trait link

Assimilation and contrast

Priming with. . . -a trait leads to assimilation of that trait -an exemplar leads to contrasting away from the implied trait

Acquisition: Additional trait activation is linked to the actor Competition: Exemplar→trait link competes with target→trait link

Illusory correlation

A minority group is seen as more negative despite the fact that the proportion of positive and negative behaviors is identical to a majority group. Better memory (shorter assignment latencies) for items from a minority group

Acquisition: greater sample size of desirable and undesirable traits in majority group Competition: greater sample size in majority group (or stronger majority→trait link) competes with episodic weights (or behavior→trait links)

Stereotype change

Group stereotype changes more if stereotype-inconsistent information is dispersed across many members rather than concentrated in a few

Competition: less discounting of inconsistent trait when dispersed among many members

Cognitive dissonance: Prohibition

Mild threat has more behavioral effects than severe threat.

Competition: Mild threat (or mild activation of inhibitory threat→play link) requires less compensatory augmentation of the toy→play link

Persuasion

Deliberate attitudes are determined by expectancy of effects x the value of these effects

Acquisition of values and of valued information

Covariation patterns (LLH & HLH) predicted to yield strong dispositional attributions recruit the temporo-parietal junction and medial prefrontal cortex

Acquisition: activation of brain areas mainly determined by covariation with target actor

Group Biases

Attitude Change

Brain Imaging Dispositional attribution based on covariation information

Note. None of the competition effects can be explained by information loss exemplar models (Fiedler, 1996; Smith, 1991) or connectionist models on the basis of a Hebbian learning algorithm (Kashima, Woolcock, & Kashima, 2000) without complementary assumptions.

explored discounting of dispositions and illustrated the emergent property of acquisition (e.g., to develop weak versus strong trait inferences about an alternative actor by manipulating sample size) and competition (e.g., to block the


development of similar trait inferences about a target actor). We presented the network with a learning history in which a competing actor is first engaged in a trait-implying behavior alone, and then together with the target actor (#5;

2/15/2011 7:47:00 PM

360


where # indicates the number of trials). Because the competing actor covaries with the traitimplying behavior before the target actor, this (stronger) competing cause→effect connection creates competition against the (weaker) target cause→effect connection. The crucial difference between conditions is the amount of competition exerted by the competing connection. This was manipulated by the sample size of the competing cause, or the number of times (#1 or #5) the competing actor engages in the behavior alone. As can be seen in Figure 16.6, the simulation (dotted line) demonstrates that in the large as opposed to small size condition, the perceiver attributes stronger trait attributions to the competing actor (see left portion of the figure). This leads to more competition such that fewer trait attributions are made to the target actor (see right portion). As can be seen, the simulation closely matches the empirical data. Note that for the simulation to work, it assumes that the competing information is received first to build up a strong competing connection that exerts a blocking effect on the target connection. If the competing information is received afterwards (as was the case in some conditions of the experiment),

8.5 Small Size Large Size Simulation

Trait Inference

8.0

7.5

7.0

6.5 Competing Actor

Target Actor

Figure 16.6 Discounting and sample size: Observed data from Van Overwalle (2006) are indicated by bars and recurrent simulation results (general learning rate = .13, for competing actor = .08) by dotted lines. (Reprinted with permission from Figure 9 of Van Overwalle, F., & Labiouse, C. 2004. A recurrent connectionist model of person impression formation. Personality and Social Psychology Review, 8, 28–61).


it is assumed that perceivers readjust their initial target attribution by simply reevaluating the target, that is, as though they reran that part of the simulation where information on the target actor is processed. In this manner, the strong connection of the competing actor is taken into account. Assimilation and Contrast

In a series of simulations on person attribution and impression formation, Van Overwalle and Labiouse (2004) presented many other simulations in which they modeled important findings of primacy and recency in impression formation (Hogarth & Einhorn, 1992; Kashima & Kerekes, 1994), asymmetric diagnosticity of ability- and morality-related behaviors such that estimated ability is determined more by high than low performance (e.g., sports) while estimated morality is determined more by the occurrence of low rather than high moral behaviors (e.g., lying; Skowronski & Carlston, 1987), increased recall for trait-inconsistent information (Hamilton, Katz, & Leirer, 1980), and assimilation and contrast effects in priming (Stapel, Koomen, & van der Pligt, 1997). I choose the last simulation as an illustration because it is fairly easy to understand. There is an abundance of social cognition research indicating that we often fill in unobserved characteristics of another person by temporary assumptions we make about them, and this process is termed assimilation. Thus, when primed (i.e., temporarily activated) with “violent,” we judge a nondescript or ambiguous target person as more hostile, and when primed with “friendly,” we judge that same target as less hostile. However, under some circumstances, the opposite effect may occur and leads to contrast rather than assimilation. For instance, when primed with the exemplar “Gandhi,” people may judge a target person as relatively more hostile, whereas primed with “Hitler,” they may judge the same target as relatively less hostile. Under these conditions, the exemplars Gandhi and Hitler serve as an anchor against which the target is judged, and so lead to contrast effects (Stapel, Koomen, & van der Pligt, 1997). What produces assimilation or contrast? The properties leading to each of these effects are

2/15/2011 7:47:00 PM


schematically depicted on top of Figure 16.7, and a simulation and empirical data are shown in the bottom panel (Van Overwalle & Labiouse, 2004). The network consists of units representing a target actor, the extreme exemplars like Gandhi and Hitler, and two traits (friendly and hostile). Person impressions are expressed by the target→trait connections. The network first builds up background knowledge about the extreme exemplars by linking Gandhi with the friendly trait and Hitler with the hostile trait (each #10). Because the target person is not described, no prior learning is assumed for this agent. Now comes the essential manipulation, which is schematically depicted by showing only

Trait

Trait

+

+ Impression of Ambigious Person

4.5

Target

Target

Exemplar Positive Prime Negative Prime Simulation

4.0

3.0

Person Exemplar

Trait Prime

Figure 16.7 Assimilation and contrast effects

after priming with a trait or person. (Top) A schematic illustration of the acquisition property generating assimilation and the competition property generating contrast. The primed stimulus is indicated with a plus sign. The weight of the connections is depicted in an increasing order by dotted–broken–full arrows. (Bottom) Observed data from Stapel, Koomen, and van der Pligt (1997, Experiment 3) are indicated by bars and simulation results (learning rate = .15) by dotted lines. (Reprinted and adapted with permission from of Van Overwalle, F., & Labiouse, C. 2004. A recurrent connectionist model of person impression formation. Personality and Social Psychology Review, 8, 28–61).


the essential units and connections in the top panel of Figure 16.7. As shown on the top left, when a trait concept is primed, this trait is assumed to be still active (denoted by +) when the actor is presented (#1). Through their co-occurrence and the property of acquisition, this leads to stronger positive or negative actor→trait connections in line with the prime, consistent with the empirical results (see bottom left portion of the figure). In contrast, as shown on the top right portion of the figure, when an exemplar such as Gandhi is primed (denoted by +), competition arises between this primed exemplar and the target actor in their connection to the friendly trait. The competition between the stronger Gandhi→trait connection (#10) and the weaker target→trait connection (#1) leads to discounting of the target→trait connection. This results in a contrast effect where the target is seen as less positive after comparison with a positive exemplar (Gandhi), and more positive after comparison with a negative exemplar (Hitler), which is consistent with the empirical findings (see bottom right portion of Fig. 16.7). Group Biases

3.5

2.5

361

In a series of simulations on group impression formation, Van Rooy, Van Overwalle, Vanhoomissen, Labiouse, and French (2003) modeled major biases and stereotypes in group judgments from a connectionist perspective. These biases were in the areas of group impression formation (also denoted as stereotyping) such as illusory correlation (defined later; Hamilton & Gifford, 1976), group differentiation (or the accentuation of differences between groups and the opposite tendency for differences within groups; Eiser, 1971), stereotype change under dispersed versus concentrated distribution of inconsistent information (defined later; Hewstone, Macrae, Griffiths, & Milne, 1994; Johnston & Hewstone, 1992; Weber & Crocker, 1983), and group homogeneity (or the tendency to see minority groups as less variable; see Simon & Brown, 1987). I illustrate this approach with the important phenomenon of illusory correlation.

2/15/2011 7:47:00 PM

362


Illusory Correlation

This bias occurs when perceivers erroneously see a relation between categories that are actually independent. For instance, minorities or outgroups are often stereotyped with bad characteristics, although these characteristics are sometimes present just as often in the perceiver’s ingroup. The earliest demonstration of illusory correlation in a group context comes from a study by Hamilton and Gifford (1976). Participants read about members of two groups A and B who engaged in the same ratio of desirable to undesirable behaviors, but much more behaviors were performed by members of group A than by members of group B. Although there was no objective correlation between group membership and desirability of behavior, participants showed greater liking for the majority group A than for the minority group B (for reviews see Hamilton & Sherman, 1989; Mullen & Johnson, 1990).

Illusory correlation can be explained by the acquisition property. How does it work? Consider a network with two group units (group A and B), two trait units (desirable and undesirable), and a set of units each representing a single behavior (of less importance here). Group impressions are represented by the group→trait connections, as schematically shown in the top panel of Figure 16.8. Because more behaviors are performed by members of the majority group A than by members of the minority group B, there is a larger sample size in group A (#8 and #4 for desirable and undesirable behaviors, respectively) than in group B (#4 and #2). Consequently, the group→trait connections of the majority group A are stronger at the end of learning than the corresponding connections of the minority group B. This is illustrated in the bottom panel of Figure 16.8. As a result, the relative proportion of desirable versus undesirable information is more clearly encoded in the group→trait connections for majority group A (denoted Da) than

Desirable

Undesirable

Group

0.8

Group A

Group B

Simulated Evalution

0.74 0.6

Da

0.51

0.36

0.4

Db 0.25

Desirable Undesirable 0.2

0.0

0 1 2 3 4 5 6 7 8 Statements

0 1 2 3 4

0 1 2 3 4 0 1 2 Statements

Figure 16.8 Illusory correlation. (Top) A schematic illustration of the acquisition property generating stronger desirable than undesirable trait connections for each group. (Bottom) Simulated evaluative strength in an illusory correlation design (Da,b indicates the difference between desirable and undesirable evaluation for group A and B, respectively) in which two desirable and one undesirable behavior were alternately presented to the network (learning rate = .15). (Reprinted and adapted with permission from Figure 3 of Van Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. 2003. A recurrent connectionist model of group biases. Psychological Review, 110, 536–563).


2/15/2011 7:47:00 PM



Group

Group

A

(strong)

2.7

B

(weak) Behavior

Trait

Correct Assignment RT sec)

for minority group B (denoted Db). This results in a more favorable impression overall for the majority group A. Note that the evaluations after four trials differ between groups A (.36) and B (.51) because the lateral connections between the units also differ in number between groups (i.e., there is a minor effect of the behavioral units). As can be seen in the figure, the connectionist network predicts that with little training, illusory correlation will not appear. Interestingly, it also makes the prediction that with continued training, the weights will asymptote at the same values, and illusory correlation will disappear in the model. This prediction has recently been confirmed (Murphy, Schmeer, ValléeTourangeau, Mondragon, & Hilton, 2009). Besides a decreased evaluation for minority group B, illusory correlation is often accompanied with increased source memory for undesirable group B behaviors in a task where participants have to assign each behavior to the correct group (Hamilton, Dugan, & Trollier, 1985; McConnell, Sherman, & Hamilton, 1994; Stroessner, Hamilton, & Mackie, 1992). This memory advantage might be produced by the competition property. Source memory is expressed by the behavioral→group connections. However, as illustrated in Figure 16.9 (top panel), the trait→group connections compete against these behavior→group connections. Given that the trait→group connections of group A are stronger than group B, these behavior→group connections are more discounted for group A behaviors than for group B behaviors. Based on the same logic (not illustrated), given that the trait→group connections of positive traits are stronger than negative traits (since there are typically more positive behaviors than negative behaviors), these behavior→group connections are more discounted for positive behaviors than for negative behaviors. The bottom panel of Figure 16.9 illustrates a successful simulation of this decreased memory for majority and positive behaviors (McConnell et al., 1994, Experiment 2). In this study, participants had to remember to which group each behavior belonged (i.e., group assignment), and faster responses on this task reflect stronger behavior→group links in memory. As can be

363

Behavior

Trait

Majority Group A Majority Group B Simulation

2.8 2.9 3.0 3.1 3.2

A+

A–

B+

B–

Condition

Figure 16.9 Illusory correlation and memory.

(Top) A schematic illustration of the competition property generating better memory after strong versus weak trait–group links. The weight of the connections is depicted in an increasing order by coarsely dotted–dotted–broken–full arrows. (Bottom) Observed data from McConnell, Sherman, and Hamilton (1994, Experiment 2, Table 5) are indicated by bars and simulation results (learning rate = .15) by dotted lines. (The scale is reversed so that higher values reflect better memory and, consequently, faster latencies.) (Reprinted and adapted with permission from Figure 5 of Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. 2003. A recurrent connectionist model of group biases. Psychological Review, 110, 536–563).

seen, memory for group affiliation was lowest for positive behaviors of group A (A+) and highest for negative behaviors of group B (B–). It is important to note that this illusory correlation simulation of group assignment excludes other well-known theoretical accounts of group biases such as information loss exemplar models (Fiedler, 1996; Smith, 1991), which assume that noise at the exemplar level is reduced the more exemplars are accumulated (e.g., summed) in an aggregated group judgment. It also excludes connectionist models on the basis of a Hebbian learning algorithm (Kashima et al., 2000), which simply accumulates the number of co-occurrences of

2/15/2011 7:47:00 PM


two stimuli in the weights of the connections. Because these models are based on accumulation without an upper bound rather than error minimization, which limits the amount of learning (like the delta algorithm), none of them incorporates the essence of competition and thus cannot explain the opposite relation between strong favoritism for group A and less memory for the behaviors of that group. Subtyping

The illusory correlation effect demonstrates how perceivers may make initial impressions about minority groups in society that are biased. An interesting question, then, is how can these stereotypes be abolished? Research has demonstrated that the best tactic to change group stereotypes is to distribute disconfirming information among as many group members as possible. If disconfirming information is not distributed but rather concentrated in a few members, these members are subtyped as extreme disconfirmers. Subtyping insulates the group from their extreme dissenting members, so that the disconfirming information is attributed to these extremists only and the content of the existing group stereotype is preserved (Hewstone et al., 1994; Johnston & Hewstone, 1992; Weber & Crocker, 1983). A connectionist framework can simulate subtyping through the property of competition. The network consisted of a group unit, several units representing individual members, and two trait units (consistent and inconsistent traits). Group stereotyping is represented by the group→trait connections. To built up an initial group stereotype, the network first receives preexperimental learning experiences on stereotypical (i.e., consistent) beliefs of the group (#10). Next, information is provided that is consistent (#12) and inconsistent (#12) with the stereotype. Crucially, in the concentrated condition, all inconsistent information is concentrated in two disconfirming group members, whereas in the dispersed condition, inconsistent information appears in six disconfirming members. As can be seen in Figure 16.10 (top panel), when stereotypeinconsistent information is concentrated in a few members, each of these disconfirming


Trait

Trait (few)

Group

Trains Ratings on Whole Group

364

Member

(many) Group

6

Member

Concentrated Dispersed Simulation

5

4

3

2 Consistert

Inconsistert Traits

Figure 16.10 Dispersed versus concentrated stereotype-inconsistent information. (Top) A schematic illustration of the competition property generating less group trait inconsistency after strong versus weak member–trait links given a few versus many inconsistent members. The weight of the connections is depicted in an increasing order by coarsely dotted–dotted–broken–full arrows. (Bottom) Observed data from Johnston and Hewstone (1992, Experiment 1, Table 3) are indicated by bars and simulation results (learning rate = .15) by dotted lines. (Reprinted and adapted with permission from Figure 8 of Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. 2003. A recurrent connectionist model of group biases. Psychological Review, 110, 536–563).

members engages in many inconsistent behaviors (each #6) and so develops strong member→ inconsistent trait connections that compete against and discount the group→inconsistent connections. In contrast, when the stereotypeinconsistent information is dispersed across multiple members, each individual member engages in few inconsistent behaviors (each #2) and thus develops weaker member→inconsistent connections, so that less competition arises against the group→inconsistent connection. Consequently, the group→inconsistent connection is more discounted by a few disconfirming members in the concentrated condition than by multiple members in the dispersed condition, leading to a conservation of stereotypical perceptions of the group

2/15/2011 7:47:01 PM


as a whole. The result of this simulation is shown in the bottom panel of Figure 16.10. Other theories such as the exemplar-based model by Fiedler (1996) and connectionist models using the Hebbian learning algorithm (Kashima et al., 2000) do not posses the competition property, and hence cannot make this prediction except by adding auxiliary assumptions (see e.g., Kashima et al., 2000, p. 931). Attitude Change Cognitive Dissonance

In a series of simulations, Van Overwalle and Jordens (2002) presented a connectionist implementation of cognitive dissonance, that is, a state where a person acts against his or her personal attitudes. In their network model, an attitude is represented by the connection between an attitude object and behavioral-affective outcomes. One way that dissonance can arise is when circumstantial constraints induce a mismatch between the model’s prediction based on what is known about the person and discrepant behavior or affect. A series of feedforward simulations successfully replicated several classical dissonance paradigms, including prohibition (or greater behavioral effect of mild versus severe threats; Freedman, 1965), initiation (or more liking for a group following harsher hazing; Gerard & Mathewson, 1966), forced compliance (or greater tendency to change one’s opinion in the absence of external constraints; Calder, Ross, & Insko, 1973; Collins & Hoyt, 1972; Linder, Cooper, & Jones, 1967; Sherman, 1970), free choice (or greater preference for an item chosen rather than forgone; Shultz, Léveillé, & Lepper, 1999), and misattribution (or decreased behavioral change after attributing dissonance feelings to external factors, e.g., medication; Higgins, Rhodewalt, & Zanna, 1979). As an illustration of dissonance, I now explore the effects of prohibiting a desired action (Freedman, 1965). School children were forbidden to play with an attractive toy (a robot) under either mild or severe threat of punishment, without subsequent surveillance. Actual play with the forbidden toy about 40 days later in the absence of the experimenter or any threat revealed greater


365

derogation of the toy in the mild than in the severe threat condition. The attributional explanation for these results is that mild threat alone provides insufficient justification for the counterattitudinal behavior of not playing with the attractive toy, and thus creates high dissonance that is reduced by lowering the attraction for the toy. In contrast, the high threat provides sufficient justification for not playing with the toy, and thus creates little dissonance and little attitude change. How can a connectionist framework model this process of dissonance reduction? Figure 16.11 (top panel) depicts a simulation of this effect. Three units are included in the network: toy, threat, and playing. The liking for the toy is expressed by the toy→playing connection. For the preexperimental learning phase, we assume that the most natural and most often occurring situation for the child is to play with an attractive toy, a pleasant experience (#10). In contrast, if children are severely threatened not to play with a forbidden toy, we assume that they would not play with it (#4). Figure 16.11 (acquisition curve in the bottom panel) shows the development of a facilitatory toy→play connection followed by an inhibitory threat→play connection given this learning history. At the end, these two connections have about equal but opposing strength that keeps them in balance. When severe threat is applied in the experimental condition (#1), there is no change because the network’s learning error is negligible (see middle portion of acquisition curve). However, when mild threat is simulated by activating the threat unit by only half its typical activation level (#1), this results in weaker inhibitory activation received at the play unit (–.2; see right portion of top panel in Fig. 16.11). Consequently, this leads to less compensation for the inhibitory threat→play connection or less augmentation (or a decrease) of the toy→play connection (implying that the child likes the toy less; see right portion of acquisition curve). In other terms, this weight change justifies the unexpected behavior. Contrary to an earlier constraint satisfaction approach (Shultz & Lepper, 1996), the present connectionist model changes not only temporary activation of attitude objects but also long-term connection

2/15/2011 7:47:01 PM

366


Play

Play

A

B –.2 (mild)

–.4 (severe) Threat

Toy

Toy

Threat

1.0

0.8

Prior Learning

Severe Mild Threat Threat

0.6 Connection Weight

Experimental Learning

0.4

Toy

0.2

0.0

–0.2

Threat

–0.4 0

2

4

6

8 10 12 14 Trials

01

01

Figure 16.11 Prohibition and the effect of severe

versus mild threat. (Top) A schematic illustration of the competition property generating more decrease of the toy–play link due to weaker activation received given mild versus severe threat. The weight of the connections is depicted in an increasing order by dotted–broken–full arrows. (Bottom) Changes in connection weights after each trial in the prior learning history (trials 0–14 left) and in each of the experimental conditions (trials 0–1 middle and right; learning rate = .30; feedforward network). The numbers next to the connections reflect the activation received at the play unit through these connections, not the connection weights themselves. (Reprinted and adapted with permission from Figure 2 of Van Overwalle, F., & Jordens, K. 2002. An adaptive connectionist model of cognitive dissonance. Personality and Social Psychology Review, 6, 204–231).

weights, so that it can explain why after 40 days, the dissonance manipulation was still effective. Persuasive Communication

Although the effects of dissonance are counterintuitive and highly intriguing, attitudes are more often changed through persuasive arguments, such


as campaigns and advertisement. Van Overwalle and Siebler (2005) replicated a number of important findings in this literature that are typically explained by dual-process approaches of attitudes (Chaiken, 1987; Petty & Cacioppo, 1981, 1986). A recurrent network was applied to well-known experiments involving deliberative attitude formation (defined later) as well as the use of heuristics of length, consensus, expertise, and mood (Chaiken & Maheswaran, 1994; Maheswaran & Chaiken, 1991; Petty & Cacioppo, 1984; Petty, Schumann, Richman, & Strathman, 1993). These heuristics reflect the finding that when people are not capable or motivated to attend to the message content, they are easily swept by the length of the arguments, the apparent consensus with other members of the audience, the expertise of the source, or their own moods rather than the strength of the message arguments. All these empirical phenomena were successfully reproduced in the simulation. Let us focus on the simplest case of deliberative attitude formation. Perhaps the most influential model of attitude formation that describes this sort of deliberative weighting of all salient alternatives and consequences is the theory of reasoned action by Fishbein and Ajzen (1975). According to this theory, an attitude is a function of the expectation that the behavior leads to certain consequences or outcomes (e.g., a car is fast and keeps you dry during the rain but also pollutes the air) and the person’s evaluation of these outcomes (e.g., fast and dry is good, pollution is bad). The attitude is the outcome of this weighting process, and it is computed by multiplying the expectancy and value components associated with each outcome and summing up these products. This formula of attitude formation has received considerable empirical support in many studies (see Ajzen, 1991; Ajzen & Madden, 1986; Fishbein & Ajzen, 1975). However, a limitation is that the theory remains vague about the psychological integration process underlying this formula. Can a connectionist approach mimic this integration in line with the predictions of the theory of reasoned action (Fishbein & Ajzen, 1975)? Figure 16.12 depicts a network architecture that was applied by Van Overwalle and

2/15/2011 7:47:01 PM


367

Evaluative

Fast

Dry

Neg

Pos

Prediction Simulation

Pollutes

Attitude

Car

Attitude

Cognitive

Car Bicycle Bus

Figure 16.12 Attitude formation. (Left) Network architecture with one attitude object (car) connected to three cognitive nodes (fast, dry, and pollutes) and two valence nodes. All nodes are interconnected to all other nodes, but for a clear understanding of the major mechanisms underlying attitude formation, only the most important (feedforward) connections are shown. Contrary to all previous simulations, there were two cycles to spread activation around the network so that activation at the cognitive attributes could spread further to the valence units (learning rate = .35). (Right) Predicted data from Fishbein and Ajzen (1975) are indicated by bars and simulation results by dotted lines. (Reprinted and adapted with permission from Figures 2 and 4 of Van Overwalle, F., & Siebler, F. 2005. A connectionist model of attitude formation and change. Personality and Social Psychology Review, 9, 231–274).

Siebler (2005). The expectancy variable in the Fishbein and Ajzen (1975) formula is determined by the frequency that an attitude object cooccurs with a cognitive attribute, which leads to stronger object→attribute connections. The evaluation variable is determined by the frequency of satisfaction or dissatisfaction experienced when that attribute is present, and it is stored in attribute→valence connections. In the simulation, during a prior learning phase, valences are developed (#15) and afterward, during the main learning phase, the attributes of each object (e.g., transportation vehicle) are learned. The number of trials is determined by the expectancy that the attributes are present, and a greater expectancy is reflected in a higher number of trials. Crucially, this learning does not only determine the weight of the object→attribute and attribute→valence connections through the property of acquisition, but at the same time it also shapes the direct object→valence connection, which reflects the attitude. This latter connection represents the attitude. As can be seen in


the inset of Figure 16.12, the simulated and predicted data from Fishbein and Ajzen’s (1975) formula match almost perfectly in this example. Dispositional Attributions in the Brain

We end this chapter with a simulation of recent neuroimaging findings on dispositional attributions. In a study on trait inferences using functional magnetic resonance imaging (fMRI), Harris, Todorov, and Fiske (2005) explored which dimensions of covariation information determine activity in social areas of the brain. The authors gave their participants short stories involving several combinations of Kelley’s covariation dimensions. As noted earlier, consensus reflects comparisons between the behavior of the actor and others, and distinctiveness denotes comparisons between actors’ goal objects. Consistency denotes comparisons over time. To illustrate, a strong trait-implying story reads: “John laughs at the comedian [target behavior]. Hardly anyone else laughs at the comedian [low consensus].

2/15/2011 7:47:01 PM

368


John also laughs at every other comedian [low distinctiveness]. In the past, John would always laugh at the comedian [high consistency].” While reading these stories, brain activity was measured using fMRI. As can be seen in Figure 16.13, the brain imaging data revealed that behaviors implying the strongest actor traits according to Kelley’s (1967) covariation model (denoted as LLH; see earlier example) showed the largest increase in activation at the temporo-parietal junction (TPJ) and the medial prefrontal cortex (mPFC), two brain areas typically recruited when judging and reasoning about other people. The only exception was that consensus information was underutilized, which is a typical finding in behavioral research (see also Wells & Harvey, 1977). Consequently, the HLH condition also produced very high activation in these areas. Can a connectionist simulation reproduce these brain activation results? I tested this in a simulation. The network consists of four units (e.g., actors, objects, time, and trait-implying behaviors) and four trials for each dimension, resulting in 43 or 64 pieces of behavioral information for a story

representing all possible (high versus low) combinations of consensus, distinctiveness, and consistency. I explored various alternative coding schemes to represent this information in an increasingly simpler manner, involving fewer covariation dimensions (see Table 16.2 for an example). A first simplification was to drop the consensus information (incomplete coding) because it had little effect, leaving only 16 trials. Because the information in Harris et al. (2005) was actually presented in a summary format, a graded coding schema was then developed in which the degree of activation rather than the number of trails reflects the frequency by which an actor engages in a behavior, and this information was only presented once. The best match between the simulation and the fMRI activation in Harris et al. (2005) was obtained if activation in both TPJ and mPFC areas contribute about equally, and therefore this activation was simply summed before comparing it with the simulation data. As can be seen in Figure 16.13, the simulation under a complete trial-by-trial coding scheme is almost perfect,

0.2 TPJ

Joint activity of TPJ + mPFC

0.1

mPFC

0.0 –0.1 Coding: Trial-by-trial Complete Trial-by-trial Incomplete Graded Consistency

–0.2 –0.3 –0.4 –0.5

LLH

HLH

LHH

HHH LLL Covariation

HLL

LHL

HHL

Figure 16.13 Simulation of traits following varying covariation information. Empirical data (the summed functional magnetic resonance imaging [fMRI] activity of the temporo-parietal junction [TPJ] and medial prefrontal cortex [mPFC] from Harris, Todorov, & Fiske, 2005) are indicated by bars and the simulation results (learning rate = .04) by dotted lines. The bottom axis reflects consensus, distinctiveness, and consistency, respectively, all high (H) or low (L). The inset shows the approximate location of the brain areas involved on the right hemisphere (TPJ) and the medial brain (mPFC).


2/15/2011 7:47:01 PM


Table 16.2

Different Coding Schemes Illustrated for Consensus and Consistency Coding Scheme TbT - Complete Frequency

TbT - Incomplete

Graded - Incomplete

Graded - Consistency

Actor

Behavior

Actor

Behavior

Actor

Behavior

Actor

Behavior

Low Consensus Target Actor

#16/16

1

1

1

1

16

1

4

1

Actor 2

#16/16

1

0

—

—

—

—

—

—

Actor 3

#16/16

1

0

—

—

—

—

—

—

Actor 4

#16/16

1

0

—

—

—

—

—

—

High Consensus Target Actor

#16/16

1

1

1

1

16

1

4

1

Actor 2

#16/16

1

1

—

—

—

—

—

—

Actor 3

#16/16

1

1

—

—

—

—

—

—

Actor 4

#16/16

1

1

—

—

—

—

—

—

Low Consistency Target Time

#16/4

1

1

1

1

4

1

1

1

Time 2

#16/4

1

0

1

0

12

0

3

0 (Continued)

2/15/2011 7:47:02 PM


Table 16.2

(Continued) Coding Scheme TbT - Complete Frequency

TbT - Incomplete

Graded - Incomplete

Graded - Consistency

Actor

Behavior

Actor

Behavior

Actor

Behavior

Actor

Behavior

Time 3

#16/4

1

0

1

0

—

—

—

—

Time 4

#16/4

1

0

1

0

—

—

—

—

High Consistency Target Time

#16/4

1

1

1

1

16

1

4

1

Time 2

#16/4

1

1

1

1

—

—

—

—

Time 3

#16/4

1

1

1

1

—

—

—

—

Time 4

#16/4

1

1

1

1

—

—

—

—

Note. Cells entries denoted the external activation. TbT = trial by trial; Incomplete = without consensus information; Consistency = only consistency information; = number of trials (#16 or # 4 for complete and incomplete trial-by-trial coding, respectively; #1 everywhere for graded coding). The same logic applies for distinctiveness and can be applied by changing “time” into “object.”

2/15/2011 7:47:02 PM


and it reaches a significant correlation with the fMRI data of r = .99, immediately followed by a significant correlation of the incomplete coding scheme (without consensus), r = .98. For the graded input coding scheme, the correlation is again significant for incomplete information (without consensus), r = .96, but consistencyonly information also yields an almost perfect correlation, r = .98. The strong correlations of the graded coding scheme suggest that summary information was perhaps encoded in much the same way as assumed by this coding scheme, that is, by roughly estimating the frequency of co-occurrences and adjusting judgments on the basis of these estimates. Moreover, the high correlation of the consistency-only coding suggests that focusing only on the approximate number of times the actor was paired with the target behavior, and ignoring all other covariation information, seems sufficient for this process to reach adequate results. Evidently, these are preliminary conclusions that need to be confirmed by subsequent research and future simulations on different information patterns. Graded coding of summarized social behavior has been hereto largely ignored in connectionist simulations, not in the least because it departs radically from the typical trial-by-trial input format in associative learning. Surprisingly, the simulation matches better the brain data than participants’ trait ratings, as the correlation of trait ratings with complete trial-by-trial coding was much lower, r = .71, although still significant.

CONCLUSION This chapter reviewed support for the idea that many social judgments and biases can be profitably viewed from a common connectionist framework that sees these social judgments as arising from basic associative learning processes. I provided empirical evidence to demonstrate that social judgments on causes and traits are best explained by connectionist approaches. Using connectionist simulations, I demonstrated that many other social judgments and biases might result from such basic learning processes. If there is one point that stands out and should be remembered from all this research, it is that a


371

single connectionist learning mechanism—the delta algorithm—is capable of producing a rich set of emerging properties to explain a wealth of empirical data. As far as I am aware, no other theory in social psychology is capable of doing that. Instead of developing isolated and fragmentary social psychological theories that lack psychological generality, this common framework promotes the view that social behavior is just one of many processes of a human mind, that does many other nonsocial things, such as perception and emotion, interpretation and evaluation, judgment and action. If the field of social psychology is to grow in the coming decades, it is time to leave behind our narrow pet theories, to cross the borders of our social discipline, and to look out for a broader and cumulative perspective that encompasses many fields in psychology. The next step in advancing our understanding of social learning or learning at large is not simulating the brain, but looking into the brain itself, using state-of-the-art imaging and timing techniques, and see where and when social processes and judgments are computed in the real brain. Fortunately, scientists of many disciplines are now working together in this endeavor. Collaboration is necessary because the technical obstacles are very complex, but it is also needed because the brain itself presents us with many surprises and riddles, which can only be resolved if we work together. To illustrate, on the one hand, it is clear that several parts of the brain are preferentially used for various social judgments (for reviews, see Van Overwalle, 2008; Van Overwalle & Baetens, 2009). On the other hand, it becomes increasingly clear that these social brain areas also have crucial nonsocial functions (e.g., Mitchell, 2008), perhaps because social processes may have evolved on top of more basic nonsocial computations. Understanding the evolution and interaction between social and nonsocial processes in the brain is an important question for future research, which requires psychologists from various backgrounds to provide different pieces of the puzzle. Looking only at the brain areas recruited by social tasks will not reveal to us how the brain computes these social functions (just like looking only at the steering wheel does not tell how a car works).

2/15/2011 7:47:02 PM

372


Another point that becomes increasingly clear is that brain imaging can tell us which areas are involved in psychological functioning at a higher level, whereas single-cell recording reveals to us how individual brain cells respond to stimuli at a lower level. However, to combine both levels and obtain an integrated understanding of functioning and interaction in tightly coupled brain networks, we must probably return to simulations. Given the enormous impact and insights the associative or connectionist approach has provided in the past, ultimately it may be the only way to get order and insight in the growing amount of brain data that we see coming out from research laboratories today.

REFERENCES Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179–211. Ajzen, I., & Madden, T. J. (1986). Prediction of goal-directed behavior: Attitudes, intentions, and perceived behavioral control. Journal of Experimental Social Psychology, 22, 453–474. Allan, L. G. (1993). Human contingency judgments: Rule based or associative? Psychological Bulletin, 114, 435–448. Anderson, N. H. (1967). Averaging model analysis of set size effect in impression formation. Journal of Experimental Psychology, 75, 158–165. Anderson, N. H. (1981). Foundations of information integration theory. New York, NY: Academic Press. Baker, A. G., Berbier, M. W., & Vallée-Tourangeau, F. (1989). Judgments of a 2 x 2 contingency table: Sequential processing and the learning curve. The Quarterly Journal of Experimental Psychology, 41B, 65–97. Busemeyer, J. R. (1991) Intuitive statistical estimation. In N. Anderson (Ed.) Contributions to information integration theory, Vol. 1. Cognition (pp. 189–215). Hillsdale, NJ: Erlbaum. Calder, B. J., Ross, M., & Insko, C. A. (1973). Attitude change and attitude attribution: Effects of incentive, choice, and consequences. Journal of Personality and Social Psychology, 25, 84–99. Chaiken, S. (1987). The heuristic model of persuasion. In M. P. Zanna, J. M. Olson, & C. P. Herman (Eds.), Social influence: The Ontario Symposium (Vol. 5., pp. 3–39). Hillsdale, NJ: Erlbaum.


Chaiken, S., & Maheswaran, D. (1994). Heuristic processing can bias systematic processing: Effects of source credibility, argument ambiguity, and task importance on attitude judgment. Journal of Personality and Social Psychology, 66, 460–473. Chapman, G. B., & Robbins, S. J. (1990). Cue interaction in human contingency judgment. Memory and Cognition, 18, 537–545. Cheng, P. W., & Novick, L. R. (1990). A probabilistic contrast model of causal induction. Journal of Personality and Social Psychology, 58, 545–567. Collins, B. E., & Hoyt, M. F. (1972). Personal responsibility-for-consequences: An integration and extension of the “forced compliance” literature. Journal of Experimental Social Psychology, 8, 558–593. Ebbesen, E. B., & Bowers, R. J. (1974) Proportion of risky to conservative arguments in a group discussion and choice shifts. Journal of Personality and Social Psychology, 29, 316–327. Eiser, J. R. (1971). Enhancement of contrast in the absolute judgment of attitude statements. Journal of Personality and Social Psychology, 17, 1–10. Fiedler, K. (1996). Explaining and simulating judgment biases as an aggregation phenomenon in probabilistic, multiple-cue environment. Journal of Personality and Social Psychology, 103, 193–214. Fiedler, K., Walther, E., & Nickel, S. (1999). The auto-verification of social hypotheses: Stereotyping and the power of sample size. Journal of Personality and Social Psychology, 77, 5–18. Fishbein, M., & Ajzen, I. (1975). Belief attitude, intention and behavior: An introduction to theory and research. London, England: AddisonWesley. Försterling, F. (1989). Models of covariation and attribution: How do they relate to the analogy of analysis of variance? Journal of Personality and Social Psychology, 57, 615–625. Försterling, F. (1992). The Kelley model as an analysis of variance analogy: How far can it be taken? Journal of Experimental Social Psychology, 28, 475–490. Freedman, J. L. (1965). Long-term behavioral effects of cognitive dissonance. Journal of Experimental Social Psychology, 1, 145–155. Gerard, H. B., & Mathewson, G. C. (1966). The effects of severity of initiation on liking for a

2/15/2011 7:47:02 PM


group: A replication. Journal of Experimental Social Psychology, 2, 278–287. Gilbert, D. T., & Malone, P. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21–38. Hamilton, D. L., Dugan, P. M., & Trollier, T. K. (1985). The formation of stereotypic beliefs: Further evidence for distinctiveness-based illusory correlation. Journal of Personality and Social Psychology, 48, 5–17. Hamilton, D. L., & Gifford, R. K. (1976). Illusory correlation in interpersonal perception: A cognitive basis of stereotypic judgments. Journal of Experimental Social Psychology, 12, 392–407. Hamilton, D. L., Katz, L. B., & Leirer, V. O. (1980). Cognitive representation of personality impressions: Organizational processes in first impression formation. Journal of Personality and Social Psychology, 39, 1050–1063. Hamilton, D. L., & Sherman, J. W. (1989). Illusory correlations: Implications for stereotype theory and research. In D. Bar-Tal, C. F. Graumann, A. W. Kruglanski, & W. Stroebe (Eds.), Stereotype and Prejudice: Changing conceptions (pp. 55–92). New York, NY: Springer. Hansen, R. D., & Hall, C. A. (1985). Discounting and augmenting facilitative and inhibitory forces: The winner takes all. Journal of Personality and Social Psychology, 49, 1482–1493. Harris, L. T., Todorov, A., & Fiske, S. T. (2005). Attributions on the brain: Neuro-imaging dispositional inferences, beyond theory of mind. NeuroImage, 28, 763–769. Hewstone, M., & Jaspars, J. (1987). Covariation and causal attribution: A logical model of the intuitive analysis of variance. Journal of Personality and Social Psychology, 53, 663–673. Hewstone, M., Macrae, C. N., Griffiths, R., & Milne, A. (1994). Cognitive models of stereotype change (5): Measure, developments and consequences of subtyping. Journal of Experimental Social Psychology, 30, 505–526. Higgins, E. T., Rhodewalt, F., & Zanna, M. P. (1979). Dissonance motivation: Its nature, persistence, and reinstatement. Journal of Experimental Social Psychology, 15, 16–34. Hilton, D. J., & Slugoski, B. R. (1986). Knowledgebased causal attribution: The abnormal conditions focus model. Psychological Review, 93, 75–88. Hilton, D. J., Smith, R. H., & Kim S. H. (1995). The process of causal explanation and dispositional attribution. Journal of Personality and Social Psychology, 68, 377–387.


373

Hogarth, R. M., & Einhorn, H. J. (1992) Order effects in belief updating: The belief-adjustment model. Cognitive Psychology, 24, 1–55. Johnston, L., & Hewstone, M. (1992). Cognitive models of stereotype change: (3) Subtyping and the perceived typicality of disconfirming group members. Journal of Experimental Social Psychology, 28, 360–386. Kashima, Y., & Kerekes, A. R. Z. (1994). A distributed memory model of averaging phenomena in person impression formation. Journal of Experimental Social Psychology, 30, 407–455. Kashima, Y., Woolcock, J., & Kashima, E. S. (2000). Group impression as dynamic configurations: The tensor product model of group impression formation and change. Psychological Review, 107, 914–942. Kelley, H. H. (1967). Attribution in social psychology. Nebraska Symposium on Motivation, 15, 192–238. Kelley, H. H. (1971). Attribution in social interaction. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior (pp. 1–26). Morristown, NJ: General Learning Press. Kruglanski, A. W., Schwartz, S. M., Maides, S., & Hamel, I. Z. (1978). Covariation, discounting, and augmentation: Towards a clarification of attributional principles. Journal of Personality, 76, 176–189. Linder, D. E., Cooper, J., & Jones, E. E. (1967). Decision freedom as a determinant of the role of incentive magnitude in attitude change. Journal of Personality and Social Psychology, 6, 245–254. Maheswaran, D. & Chaiken, S. (1991). Promoting systematic processing in low-motivation settings: Effect of incongruent information on processing and judgment. Journal of Personality and Social Psychology, 61, 13–25. Manis, M., Dovalina, I., Avis, N. E., & Cardoze, S. (1980). Base rates can affect individual predictions. Journal of Personality and Social Psychology, 38, 231–248. McClelland, J. L., & Rumelhart, D. E. (1988). Explorations in parallel distributed processing: A handbook of models, programs and exercises. Cambridge, MA: Bradford. McCloskey, M., & Cohen N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. The Psychology of Learning and Motivation, 24, 109–165.

2/15/2011 7:47:02 PM

374


McClure, J. (1998). Discounting of causes of behavior: Are two reasons better than one? Journal of Personality and Social Psychology, 74, 7–20. McConnell, A. R., Sherman, S. J., & Hamilton, D. L. (1994). Illusory correlation in the perception of groups: An extension of the distinctivenessbased account. Journal of Personality and Social Psychology, 67, 414–429. McLeod, P., Plunkett, K., & Rolls, E. T. (1998). Introduction to connectionist modeling of cognitive processes. Oxford, England: Oxford University Press. Mitchell, J. P. (2008) Activity in right temporoparietal junction is not selective for theory of mind. Cerebral Cortex, 18, 262–271. Mullen, B., & Johnson, C. (1990). Distinctivenessbased illusory correlations and stereotyping: A meta-analytic integration. British Journal of Social Psychology, 29, 11–28. Murphy, R., Schmeer, S., Vallée-Tourangeau, F., Mondragon, E., & Hilton, D. (2009). Making the illusory correlation effect appear and then disappear: The effects of increased learning. Quarterly Journal of Experimental Psychology, under revision. Orvis, B. R., Cunningham, J. D., & Kelley, H. H. (1975). A closer examination of causal inference: The role of consensus, distinction and consistency information. Journal of Personality and Social Psychology, 32, 605–616. Petty, R. E., & Cacioppo, J. T. (1981). Attitudes and persuasion: Central and peripheral routes to attitude change. New York, NY: Springer. Petty, R. E., & Cacioppo, J. T. (1984). The effects of involvement on responses to argument quantity and quality: Central and peripheral routes to persuasion. Journal of Personality and Social Psychology, 46, 69–81. Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 19, pp. 123–205). San Diego, CA: Academic Press. Petty, R. E., Schumann, D. W., Richman, S. A., & Strathman, A. J. (1993). Positive mood and persuasion: Different roles for affect under highand low-elaboration conditions. Journal of Personality and Social Psychology, 64, 5–20. Read, S. J., & Marcus-Newhall, A. (1993) Explanatory coherence in social explanations: A parallel distributed processing account. Journal of Personality and Social Psychology, 65, 429–447.


Read, S. J., & Miller, L. C. (1993) Rapist or “regular guy”: Explanatory coherence in the construction of mental models of others. Personality and Social Psychology Bulletin, 19, 526–541. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–98). New York, NY: AppletonCentury-Crofts. Shanks, D. R. (1985). Forward and backward blocking in human contingency judgment. Quarterly Journal of Experimental Psychology, 37b, 1–21. Shanks, D. R. (1987). Acquisition functions in contingency judgment. Learning and Motivation, 18, 147–166. Shanks, D. R. (1995). Is human learning rational? Quarterly Journal of Experimental Psychology, 48a, 257–279. Shanks, D. R., Lopez, F. J., Darby, R. J., & Dickinson, A. (1996). Distinguishing associative and probabilistic contrast theories of human contingency judgment. In D. R. Shanks, K. J. Holyoak, & D. L. Medin (Eds.), The psychology of learning and motivation (Vol. 34, pp. 265–311). New York, NY: Academic Press. Sherman, S. J. (1970). Attitudinal effects of unforeseen consequences. Journal of Personality and Social Psychology, 16, 510–520. Shultz, T., & Lepper, M. (1996). Cognitive dissonance reduction as constraint satisfaction. Psychological Review, 2, 219–240. Shultz, T. R., Léveillé, E., & Lepper, M. R. (1999). Free choice and cognitive dissonance revisited: Choosing “lesser evils” versus “greater goods.” Personality and Social Psychology Bulletin, 25, 40–48. Simon, B., & Brown, R. J. (1987). Perceived homogeneity in minority-majority contexts. Journal of Personality and Social Psychology, 53, 703–711. Skowronski, J. J., & Carlston, D. E. (1987). Social judgment and social memory: The role of cue diagnosticity in negativity, positivity and extremity biases. Journal of Personality and Social Psychology, 52, 689–699. Smith, E. R. (1991). Illusory correlation in a simulated exemplar-based memory. Journal of Experimental Social Psychology, 27, 107–123. Stapel, D. A., Koomen, W., & van der Pligt, J. (1997). Categories of category accessibility: The impact of trait concept versus exemplar priming

2/15/2011 7:47:02 PM


on person judgments. Journal of Experimental Social Psychology, 33, 47–76. Stroessner, S. J., Hamilton, D. L., & Mackie, D. M. (1992). Affect and stereotyping: The effect of induced mood on distinctiveness-based illusory correlations. Journal of Personality and Social Psychology, 62, 564–576. Trope, Y., & Gaunt, R. (2000). Processing alternative explanations of behavior: Correction or integration? Journal of Personality and Social Psychology, 79, 344–354. Van Hamme, L. J. (1994). Associative and statistical accounts of cue competition in causality judgments. Unpublished doctoral dissertation, University of Iowa, Iowa City. Van Overwalle, F. (1996). The relationship between the Rescorla-Wagner associative model and the probabilistic joint model of causality. Psychologica Belgica, 36, 171–192. Van Overwalle, F. (1997) Dispositional attributions require the joint application of the methods of difference and agreement. Personality and Social Psychology Bulletin, 23, 974–980. Van Overwalle, F. (2003) Acquisition of dispositional attributions: Effects of sample size and covariation. European Journal of Social Psychology, 33, 515–533. Van Overwalle, F. (2006) Discounting and augmentation of dispositional and causal attributions. Psychologica Belgica, 46, 211–234. Van Overwalle, F. (2007). Social connectionism: A reader and handbook for simulations. New York, NY: Psychology Press. Van Overwalle, F. (2009). Social cognition and the brain: A meta-analysis. Human Brain Mapping, 30, 829–858. Van Overwalle, F., & Baetens, K. (2009). Understanding others’ actions and intentions by mirror and mentalizing systems: A meta-analysis. Manuscript submitted for publication. Van Overwalle, F., & Jordens, K. (2002). An adaptive connectionist model of cognitive dissonance. Personality and Social Psychology Review, 6, 204–231.


375

Van Overwalle, F., & Labiouse, C. (2004). A recurrent connectionist model of person impression formation. Personality and Social Psychology Review, 8, 28–61. Van Overwalle, F., & Siebler, F. (2005). A connectionist model of attitude formation and change. Personality and Social Psychology Review, 9, 231–274. Van Overwalle, F., & Van Rooy, D. (2001a). When more observations are better than less: A connectionist account of the acquisition of causal strength. European Journal of Social Psychology, 31, 155–175. Van Overwalle, F., & Van Rooy, D. (2001b). How one cause discounts or augments another: A connectionist account of causal competition. Personality and Social Psychology Bulletin, 27, 1613–1626. Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. (2003). A recurrent connectionist model of group biases. Psychological Review, 110, 536–563. Wasserman, E. A., Kao, S-F., Van Hamme, L., Katagiri, M., & Young, M. E. (1996) Causation and association. The psychology of learning and motivation, 34, 207–264. Weber, R., & Crocker, J. (1983). Cognitive processes in the revision of stereotypic beliefs. Journal of Personality and Social Psychology, 45, 961–977. Wells, G. L., & Harvey, J. H. (1977). Do people use consensus information in making causal attributions? Journal of Personality and Social Psychology, 35, 279–293. Wells, G. L., & Ronis, D. L. (1982). Discounting and augmentation: Is there something special about the number of causes? Personality and Social Psychology Bulletin, 8, 566–572. Williams, D. A., & Docking, G. L. (1995). Associative and normative accounts of negative transfer. The Quarterly Journal of Experimental Psychology, 48A, 976–998.

2/15/2011 7:47:02 PM

CHAPTER 17 Application of Associative Learning Paradigms to Clinically Relevant Individual Differences in Cognitive Processing Teresa A. Treat, John K. Kruschke, Richard J. Viken, and Richard M. McFall

This chapter presents our efforts to examine clinically relevant individual differences in category learning with complex, socially relevant stimuli, highlighting the generalizability of cognitive scientists’ models and paradigms for the investigation of normative learning processes with simple, artificial stimuli. We document the substantial influence of individual variability in perceived dimensional salience on category learning, which extends the well-established link between normative perceived salience and category learning. It also appears that the complex, socially relevant stimuli of interest to some clinical researchers may be processed in a more holistic fashion than the artificial stimuli of primary interest to cognitive researchers, which typically are processed more separably. More integral processing may diminish the role of attentionshifting mechanisms in clinically relevant category learning, suggesting the importance of future research on the enhancement of attention shifting. More generally, the present work illustrates the utility of translating associative-learning paradigms to address applied questions about clinically and socially relevant processing of complex stimuli.

Individual differences in cognitive processing have been implicated in a range of clinical problems, such as depression, anxiety, schizophrenia, sexual aggression, and disordered eating (e.g., Beck, 1976; Kelly, 1955; McFall, 1990). Clinical scientists have been slow, however, to capitalize on the wealth of theories, measurement strategies, and analytical approaches developed by quantitative cognitive scientists, even though these models and methods seem promising for the advancement of cognitive theories of psychopathology (Treat et al., 2007). Cognitive scientists also have been slow to evaluate the generalizability of their models and methods to the more complex circumstances characteristic of “real-world” processing. The limited exploration of the integrative area of quantitative clinical-cognitive science reflects, in part,

two fundamental differences between cognitive and clinical science. First, quantitative cognitive scientists typically focus on the development and evaluation of formal mathematical models of the normative operation of component cognitive processes, such as attention, memory, and learning. Clinical scientists, in contrast, often study individual differences in abnormal processing. Second, quantitative cognitive scientists commonly study processing of simple, artificial stimuli that vary along readily identifiable dimensions, that are perceived similarly across persons, and with which persons frequently have limited experience. In contrast, clinical scientists more frequently study individual differences in processing of more complex, socially and emotionally relevant stimuli that vary along numerous dimensions, that may be perceived

376


2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

quite differently across persons, and with which persons often have prior experience. Thus, the application of cognitive science methods to clinical problems necessitates addressing the representation and assessment of individual differences, as well as far greater stimulus complexity. This chapter provides an overview of our efforts to draw on contemporary cognitive science to examine clinically relevant individual differences in category learning with complex, socially relevant stimuli. Cognitive scientists long have recognized the fundamental importance of category learning as a core cognitive process (see Ashby & Maddox, 2005; Kruschke, 2005a, for reviews). Clinical researchers tend to focus on static characterizations of cognitive processing, but examination of the way in which processing changes through time could be quite fruitful, whether the variation occurs naturally over different time scales, in response to feedback, or as a function of a theoretically relevant manipulation that is associated with exacerbation or amelioration of the clinical phenomenon. We focus in this chapter on learning as a function of feedback, because socially relevant learning processes presumably influence the development, maintenance, and modification of our interpersonal beliefs and behaviors. Moreover, the provision of trial-by-trial feedback in structured category-learning protocols ultimately may serve as a useful prevention or intervention strategy. The approach that we adopt treats participants’ perceptual organization of stimuli as a representational base for the operation of learning processes. Thus, characterization of individual differences in perceptual organization is central to our investigations of individual differences in category learning. We focus in particular on individual differences in the perceived salience of stimulus dimensions as an important determinant of individual differences in category learning with socially relevant stimuli. We document that participants show far better performance on category structures that are congruent with their underlying perceptual organization, and they struggle to learn category structures that are incongruent with the perceptual organization that they bring to the task.


377

The first section provides a detailed overview of our use of cognitive-science methods to characterize perceptual organization and categorylearning processes with complex stimuli. The second and third sections describe our use of these methods to characterize (a) individual differences in men’s perceptual organization of and learning about women’s affect (positive or negative) and physical appearance (physically exposed or not), with implications for our understanding of sexual aggression; and (b) individual differences in women’s perceptual organization of and learning about other women’s facial affect (happy or sad) and body size (heavy or light), with implications for our understanding of disordered eating.

PERCEPTUAL ORGANIZATION AND CATEGORY LEARNING Perceptual Organization

Perceptual organization refers to the representation and organization of incoming stimuli in terms of their underlying psychological attributes. We most commonly assess perceptual organization using a similarity-ratings paradigm, in which participants judge the similarity of pairs of stimuli on a scale anchored by “very different” and “very similar.” For example, participants might view numerous pairs of photographs of women who vary in terms of their affect (negative to positive) and physical exposure (covered to exposed). On a single similarityratings trial, the participant might evaluate the similarity of a physically exposed and happy woman to a physically unexposed and happy woman. Because these two photos differ only on degree of exposure, not on affect, a rating of “very different” would suggest that the participant perceives physical exposure to be more salient than affect, whereas a rating of “very similar” would suggest the opposite. Participants are told that there are no right or wrong answers and are encouraged to respond quickly with their first impression of the photo pair’s similarity. Note that this task neither specifies the stimulus attributes of interest nor directs participants to attend to particular stimulus attributes, thus

2/17/2011 10:59:41 AM



2

1 Stimulus C Affect

providing a relatively implicit assessment of participants’ perceptual organizations. A multidimensional scaling (MDS) analysis of participants’ similarity ratings provides a spatial representation of the group-level perceptual organization or “psychological space,” in which the perceived similarity between two stimuli is modeled as a decreasing function of the distance between the perceived values of two stimuli (Davison, 1992; Treat et al., 2002). Thus, two stimuli that are judged to be very similar are scaled much closer in the psychological space than two stimuli that are judged to be very dissimilar. For example, the upper panel of Figure 17.1 displays the psychological space of 24 photo stimuli that present women varying along physical exposure and affect dimensions. Stimuli A and B, which depict physically exposed women expressing negative affect, are scaled close together, reflecting the participant’s perception that they are very similar to one another. In contrast, this participant judged both stimuli to be very dissimilar to stimulus C, a physically unexposed woman expressing positive affect, who is scaled across the psychological space from stimuli A and B. The metric used to compute the distance between stimuli reflects assumptions about the extent to which the stimulus dimensions are processed in a more separable or integral manner. The Euclidean metric is used when the stimulus dimensions are processed more holistically, or integrally, such as when evaluating color patches that vary in hue and saturation (Nosofsky & Palmeri, 1996; Shepard, 1964). The Euclidean distance between two stimuli is simply the length of a straight line between them. In contrast, the city-block metric is used when the stimulus dimensions are perceived more distinctively, or separably, such as when evaluating objects that vary in size and orientation. The city-block distance between two stimuli is the sum of the distances between them along each stimulus dimension. In our experience, the better fitting metric for the more complex, ecologically valid stimulus sets of interest to clinical researchers often is Euclidean, rather than city-block, because the dimensions are difficult to isolate. The worse fit

0

–1

Stimuli A & B Category J

–2 –2

–1

Category F 0

1

2

Physical Exposure 2

Stimulus C

1 Affect

378

0 Stimuli A & B

–1 Category J –2 –2

–1

Category F 0

1

2

Physical Exposure

Figure 17.1 Multidimensional spatial representation of perceptual organizations of participants who perceive exposure to be relatively more salient than affect (top) and who perceive affect to be relatively more salient than exposure (bottom). See text for further information.

of the city-block metric suggests that the dimensions may be difficult to attend to selectively (Nosofsky & Palmeri, 1996), since the city-block metric entails independent analysis of the dimensions. In contrast, the appropriate metric for the simpler, artificial stimuli of interest to cognitive scientists typically is city block, and it is much easier to attend selectively to separably processed dimensions. As we will see, the extent to which stimulus dimensions are processed separably versus integrally has significant implications for the operation of category-learning processes with more complex stimuli. In the weighted MDS model, individual differences in participants’ similarity ratings are

2/17/2011 10:59:41 AM


modeled as participant-specific weighting of the stimulus dimensions (Carroll & Chang, 1970). Conceptually, these “salience weights” stretch and shrink the dimensions of the group psychological space. The upper panel of Figure 17.1 presents the psychological space of a participant who is influenced much more by women’s physical exposure than by their affect. The large salience weight for physical exposure serves to increase the distance between the more and less exposed women, which reflects this participant’s perception that more and less exposed women are very dissimilar. In contrast, the small salience weight for affect shrinks the distance between the women displaying positive versus negative affect, consistent with the participant’s judgment that women displaying positive and negative affect are not particularly dissimilar. The lower panel of Figure 17.1 depicts the psychological space of a participant who is influenced more by women’s affect than by physical exposure. Thus, the physically exposed and unexposed women are scaled closer together than the women exhibiting positive and negative affect. A particularly nice feature of the weighted MDS model is its simultaneous representation of both groupand participant-specific aspects of perceptual organization: Both the dimensions spanning the psychological space and the organization of the stimuli within each dimension are assumed to be shared by participants, whereas the relative salience of each dimension is allowed to vary across participants. It is important to distinguish perceived dimensional salience from selective attention to dimensions. Perceived dimensional salience is relatively static and enduring, indicating the default allocation of attention to the dimensions within the context of a particular stimulus set. Selective attention, on the other hand, refers to relatively dynamic reallocation of attention, whereby processing of dimensions might be amplified or attenuated selectively over a brief time span. For example, a person might display a perceptual organization in which affect is highly salient and exposure is relatively ignored. This relative salience would manifest itself in similarity ratings. The same person, however, might be able to reallocate attention selectively


379

if a different task demanded it (e.g., when learning an exposure-relevant category structure). Reallocation of attention might prove to be difficult for these stimuli, however, because scaling studies suggest that the dimensions are integral. We also have relied on a prototypeclassification task to provide estimates of the perceived salience of stimulus dimensions, since providing similarity ratings for all possible pair of stimuli (including prototypes) places a significant burden on participants (e.g., judging the similarity of all possible pairs of 24 stimuli takes approximately 30 minutes). In the classification task, participants first view two prototypical photo stimuli that vary along both theoretical dimensions of interest. For example, a “Type D woman” might express positive affect and be normatively heavy, whereas a “Type K woman” might express negative affect and be normatively light. Participants study the two prototypes for 10–15 seconds, then freely classify each of the remaining stimuli as an example of a Type D or a Type K woman, without corrective feedback. Because the prototypes vary along both theoretically relevant dimensions, participants can base their classifications on either or both dimensions. Here, too, the task provides a relatively implicit assessment of participants’ perceptual organizations, because it neither specifies the stimulus attributes of interest nor directs participants to attend to particular stimulus attributes. Multiple logistic-regression techniques are used to estimate individual differences in perceptual organization during the classification task. A participant’s classification judgments are regressed onto normative data for the stimulus dimensions (e.g., a separate undergraduate sample’s average judgments of the affect and body size of the women depicted in the photos). In other words, the normative data for affect and body size serve as two predictors of each participant’s dichotomous classification decisions. The slope estimates from these analyses reflect the change in the probability of classifying a stimulus into a particular category that can be predicted by the normative stimulus values for affect or body size. For instance, each participant’s slope (or utilization coefficient) for affect reflects the expected increase in the probability

2/17/2011 10:59:41 AM

380


of classifying a stimulus with the positive-affect prototype for every unit increase in the normative value of affect for the stimulus. Viken, Treat, Nosofsky, McFall, and Palmeri (2002) demonstrated a strong association between utilization coefficients (based on a prototype-classification task) and the corresponding salience weights (based on a similarity ratings task) for body size and affect, with an average correlation of 0.75. Thus, we treat the utilization estimates from the prototype-classification tasks as indicators of perceived dimensional salience. Category Learning

Category learning in the present context refers to the placement of stimuli into categories with feedback about the accuracy of classification, although experimenter instruction as to the stimulus characteristics on which to base classifications remains absent. To date, we have relied on category structures defined by a single central boundary along a single dimension. For example, a participant might view photos of women who vary along physical exposure and affect dimensions, classify the woman depicted in each photo as a member of one of two categories with arbitrary labels (e.g., “Category F” and “Category J”), and then receive feedback on the accuracy of his classifications (e.g., “Correct! She is a member of Category J.”). Note that both panels of Figure 17.1 include a category boundary indicated by a dashed line that is perpendicular to the physical exposure axis of the psychological space. In this case, the participant would be learning to classify physically exposed women as members of Category F and physically unexposed women as members of Category J. Participants are told that initially they will be guessing, because they have not been told the basis for the feedback. Participants also are informed that the basis for the feedback might change during the course of the task, and that they should attempt to learn the new category labels for the stimuli if this occurs. If a shift to an affect category structure occurred, then the participant might have to learn to classify women expressing positive affect as members of Category F and women expressing negative affect as members of Category J.


The formal process models of category learning upon which we rely assume that the underlying spatial representation of the stimuli (i.e., the perceptual organization) is the foundation for the category-learning processes (e.g., Kruschke, 1992; Kruschke & Johansen, 1999; Nosofsky, 1992). These models predict that participants will learn a category structure based on a particular stimulus feature more quickly when stimuli in different categories are perceived to be very dissimilar and stimuli in the same category are perceived to be very similar. This prediction follows purely from generalization of learning from one exemplar to another in close proximity. In particular, when a category distinction aligns with a salient dimension, the distinction should be learned rapidly, because the stretching of a salient dimension increases the distance between stimuli in different categories. In our applications with complex and socially relevant stimuli, individual differences in category learning should be related to individual differences in the perceived salience of stimulus dimensions. Note that, in this analysis, dimensional salience always is a perceived, rather than an intrinsic, property of a stimulus dimension. Consider, for example, differences in the expected rate at which an exposure category structure is learned by the participants whose perceptual organizations are depicted in Figure 17.1. The participant for whom exposure is more salient than affect (in the upper panel) should acquire the exposure category structure rapidly, because the stimuli falling into the same category are perceived to be quite similar (i.e., they are close to one another in the psychological space), whereas the stimuli falling in different categories are perceived to be very dissimilar. In contrast, the participant for whom affect is more salient than exposure (in the lower panel) should learn the exposure category structure much more slowly, because the structure is far less congruent with his perceptual organization. In both of the studies described in this chapter, we examine whether category-learning performance is congruent with participants’ perceptual organizations. We also fit Kruschke and Johansen’s (1999) connectionist model of category learning, RASHNL, which implements

2/17/2011 10:59:41 AM


three mechanisms that may underlie participants’ responses. The first mechanism sets the initial relative perceived salience of the psychological dimensions of physical exposure and facial affect. For example, participants who initially perceive affect to be more salient than exposure should be at a distinct advantage over their counterparts when learning the affect category structure, because of the greater perceived similarity of the stimuli within the same category and the greater perceived dissimilarity of the stimuli in different categories. The second mechanism is shifting of attention toward relevant dimensions and away from irrelevant dimensions. This attentional shifting allows participants to learn category structures by modifying their perceptual organization to be more consistent with the demands of the category structure. In other words, participants who initially perceive exposure to be more salient than affect could learn the affect category structure by increasing their attention to affect and decreasing their attention to exposure, thus modifying their perceptual organization to make it more similar to that of an initially affect-oriented participant. This shift in dimensional attention would minimize intracategory distances and maximize intercategory distances, and it could happen quite rapidly. The third mechanism is gradual strengthening of associations between regions of the psychological space and correct category responses. The association-learning mechanism produces incremental improvement in performance for specific exemplars and their nearby neighbors, whereas attention shifting affects the relative distribution of all exemplars simultaneously. Distinguishing these mechanisms in category learning could be beneficial for our understanding of cognitive processing of these complex, socially relevant stimuli. As discussed earlier, the initial perceived salience of the psychological dimensions (i.e., the first mechanism mentioned) is already a well-established predictor of category learning with simple, artificial stimuli. Consideration of category learning about more complex stimuli with which participants have prior experience allows us to investigate the potential influence of systematic individual


381

differences in perceived dimensional salience on category learning as well. Extensive evidence within cognitive science also supports a role for attention shifting (the second mechanism) as a primary strategy when learning category structures with artificial stimuli that are composed of separable dimensions (e.g., Kruschke, 1993, 1996). Nosofsky and Palmeri (1996) demonstrated that attention shifting played a far less central role, however, when learning structures with artificial stimuli composed of integral dimensions (e.g., color patches that vary in hue and saturation). Thus, category-learning studies have demonstrated that artificial stimuli that are best scaled by Euclidean metrics in similarity rating are best modeled with little attention shifting in category learning. Because the dimensions of the more ecologically valid stimuli of interest to clinical researchers frequently will be processed integrally, rather than separably, this may diminish the role of attention shifting in category learning with these stimuli. Moreover, a relative inability to shift attention rapidly might enhance the importance of individual differences in perceived dimensional salience for applied category learning. In this case, gradual changes in the associations between regions of the psychological space and the correct category label (the third mechanism) may become more central to the acquisition of applied category structures, particularly those that are incongruent with a person’s initial allocation of attention across dimensions. Thus, the relative importance of the three learning mechanisms instantiated in RASHNL may differ meaningfully as a function of the nature of the stimuli of primary interest to cognitive and clinical scientists.

INDIVIDUAL DIFFERENCES IN MEN’S PERCEPTIONS OF AND LEARNING ABOUT WOMEN Background

Social information-processing models of sexually coercive or aggressive behavior between acquaintances specify a critical role for the way in which men process information about women (e.g., Farris, Treat, Viken, & McFall, 2008; Johnston &

2/17/2011 10:59:41 AM

382


Ward, 1996; McFall, 1990; Segal & Stermac, 1990). More specifically, theorists suggest that men at greater risk of exhibiting sexual aggression toward acquaintances attend relatively less to information about women’s affect or sexual interest and relatively more to women’s physical sexual attributes (e.g., physical exposure, sensuality, provocativeness, and sexual attractiveness). In 2001, we conducted a study to evaluate this hypothesized link between men’s risk status and their relative attention to affect and physical exposure dimensions of full-body photos of women in newsstand magazines (Treat, McFall, Viken, & Kruschke, 2001). This study provided an opportunity to examine individual differences in participants’ performance when learning category structures that were based on either women’s affect or physical exposure. This allowed us to evaluate whether individual differences in perceptual organization facilitated or inhibited performance on the category-learning tasks, depending on the congruence of the participant’s perceptual organization with the category structure to be learned. We also fit Kruschke and Johansen’s (1999) RASHNL model to the category-learning data, so that we could evaluate the role of the three mechanisms described earlier. We hoped that attention shifting would play a central role in young men’s category learning about women’s affective and appearance-based cues, because a participant’s ability to shift attention between these cues in an artificial categorylearning task might indicate that his relative attention to such dimensions in “real-world” social environments would be malleable as well. Methods

Stimuli were 26 color slides of Caucasian women who appeared either in newsstand magazines or mass-marketing catalogs. A separate sample of undergraduate males rated these stimuli along 10-point scales for several relevant dimensions, including affect and exposure. The average ratings along the affect and exposure dimensions for each stimulus served as “normative ratings” for the stimuli. Only 14 of the 26 stimuli were used in the similarity-ratings task, given the prohibitively large number of all possible pairs


that would need to be rated if we had included all 26 stimuli in this task. All 26 stimuli were used in the category-learning tasks. The normative ratings for affect and exposure were used to classify the stimuli as having high or low values on the exposure and affect dimensions for purposes of providing feedback in the two categorylearning tasks. Seventy-one undergraduate males first completed a similarity-ratings task, in which they judged the similarity of all possible pairs of 14 photos of women on a 10-point scale ranging from 0 = very different to 9 = very similar. After finishing an implicit classification task that is not relevant to the current discussion, participants then completed two category-learning tasks. They viewed individual photos of 26 women, judged whether each woman did or did not have an unspecified characteristic, and received trialby-trial feedback on the accuracy of their classifications. The feedback was based on the woman’s normative affect in one task and on the woman’s level of physical exposure in the other task. Four blocks of trials were completed for each category structure. Finally, participants responded to the Heterosocial Perception Survey (McDonel & McFall, 1991), which indexed a participant’s perception of the justifiability of a man continuing to make sexual advances in the face of increasing resistance by a female acquaintance. Participants whose justifiability ratings declined less rapidly as the woman’s negativity increased were presumed to be at higher risk of exhibiting sexually aggressive behavior toward acquaintances. This Rape Justifiability Score was computed for all participants, with higher values indicating a greater propensity toward sexual aggression. Results

Weighted multidimensional scaling techniques were used to quantify individual differences in the relative perceived salience of the physicalexposure and facial-affect dimensions. A twodimensional configuration was imposed, with the stimulus coordinates constrained to be equal to the normative ratings for exposure and affect,

2/17/2011 10:59:41 AM



1

Proportion Correct

and salience weights for the two dimensions were estimated for each participant. Average salience weights for physical exposure and affect indicated that physical exposure was perceived to be far more salient than affect in this particular stimulus set. A single index of the relative salience of exposure versus affect was constructed from the two salience weights (MacCallum, 1977). Marked variability in relative salience scores was observed, as expected. Participants whose relative salience score fell in the upper and lower terciles were classified as exposure oriented (n = 24) and affect oriented (n = 24), respectively. As expected, exposure-oriented participants showed significantly higher Rape Justifiability Scores than affect-oriented participants. Exposure-oriented participants’ perception of the justifiability of unwanted sexual advances depended less on the degree of negative reaction from the hypothetical woman than it did for affect-oriented participants. Thus, those participants who perceived affect to be relatively more salient on the similarity-ratings tasks demonstrated greater sensitivity to the negativity of a woman’s affect in the justifiability rating task. Proportion correct was computed for each of the four blocks of both the exposure and affect category-learning tasks. Preliminary analyses indicated that the order in which the two category structures was completed did not influence the findings, so it was not included in subsequent analyses. A repeated-measures analysis with a robust estimator was applied to the data with Group (exposure oriented or affect oriented), Task (exposure or affect), and Block as factors. A significant Task effect emerged, Wald c 2 (1) = 27.39, p < 0.001, with average performance on the exposure-relevant structure (M = 0.90) far exceeding performance on the affectrelevant structure (M = 0.81). The predicted Group x Task interaction also emerged, Wald c 2 (1) = 4.01, p < 0.05; both affect-oriented and exposure-oriented participants showed better performance across blocks when learning the category structure that was more congruent with their perceptual organization. These effects are illustrated in Figure 17.2. In sum, both average perceived salience and individual differences in perceived salience predicted performance

383

0.9

0.8 AO-Affect Task EO-Affect Task AO-Exposure Task EO-Exposure Task

0.7

0.6 1

2

3

4

Block

Figure 17.2 Perceptual organization and category

structure (Task) influence performance on exposure and affect category-learning tasks in men’s learning study. AO. affect oriented; EO, exposure oriented. Bars correspond to standard error of the means.

on the exposure and affect category-learning tasks. We fit RASHNL (Kruschke & Johansen, 1999) to the proportion-correct values of the exposure- and affect-oriented groups on each block in the learning task, using a stepwise search algorithm to minimize the root-mean-squared deviation between the observed and predicted proportion-correct values. The Euclidean metric was used to compute interstimulus distances, because preliminary model fits indicated that participants perceived the stimulus dimensions integrally, rather than separably. As expected, the RASHNL model fit best with group-specific estimates of initial perceived salience, such that exposure-oriented participants had a larger exposure/affect salience ratio than affect-oriented participants. These model results are consistent with the findings described earlier; however, fitting a process model to the data also provided group-specific estimates of the perceived relative salience of the stimulus dimensions during category learning. Unexpectedly, the RASHNL model suggested that reallocating attention toward relevant dimensions and away from irrelevant dimensions did not contribute to participants’ learning. The best-fitting attention-shift rate was zero,

2/17/2011 10:59:41 AM

384


and the best-fitting association learning rates did not differ for exposure- and affect-oriented groups. In summary, fits by the RASHNL model revealed that the exposure-oriented and affectoriented groups differed only in the relative perceived salience of the dimensions. The groups learned at the same rate, and they did not learn to attend selectively to the dimensions. Conclusions

As expected, young men’s perceptions of young women’s physical-exposure and affective information showed strong relationships with men’s performance on exposure and affect category structures. Overall performance was better on the exposure than the affect category structure, consistent with the far greater average salience of exposure than affect in weighted multidimensional scaling analyses. Participants also learned a category structure much more rapidly when it was congruent with their underlying perceptual organization. That is, exposure-oriented participants performed better on the exposure category structure than affect-oriented participants, whereas affect-oriented participants performed better on the affect category structure than exposure-oriented participants. Fitting RASHNL to the category-learning data indicated that adaptive shifting of attention toward relevant stimulus dimensions and away from irrelevant stimulus dimensions did not play a role in participants’ learning, because the best-fitting value for the attention shifting parameter was zero. Rather, participants gradually learned to associate regions of the psychological space with the correct category label. The lack of attentional shifting presumably reflects the integral, rather than separable, nature of the stimulus dimensions in the current application. Holistic processing of stimulus dimensions implies that it is difficult to shift attention toward or away from specific dimensions (Nosofsky & Palmeri, 1996). It would be useful in everyday life, however, if people were able to attend selectively to these dimensions, such as when a person needs to shift attention away from a potential sexual partner’s physical exposure and toward the partner’s affective expressions of


sexual interest. Thus, it would be profitable for future research to investigate the conditions under which the distribution of attention across dimensions exhibits greater flexibility and malleability, even when the stimulus dimensions are perceived integrally. Overall, these findings suggest that the wellestablished normative link between dimensional salience and category learning with simple, artificial stimuli (e.g., Kruschke, 1992; Kruschke & Johansen, 1999) indeed generalizes to the association between individual differences in perceptual organization and learning with more complex, socially and emotionally relevant stimuli, although the processes underlying learning may differ. Numerous theories suggest that the etiology or course of clinical phenomena is influenced by features of participants’ perceptual organizations, such as attention to stimulus dimensions (e.g., Beck, 1976; Kelly, 1955; McFall, 1990). The present study provides further support for this link by demonstrating that young men who perceived women’s physical characteristics to be more salient than their affect also perceived continued sexual advances in the face of a woman’s increasing resistance to be more justified, relative to participants who perceived women’s affect to be more salient. Clinical researchers have tended to focus far more on the role that attentional processes play in psychopathology, but the present work highlights the potential utility of translating the models and methods of associative learning to clinically relevant investigations as well. Performance on category-learning paradigms with more complex stimuli may provide a window into the operation of “real-world” social learning processes that presumably underlie socially relevant attitudes and behaviors.

INDIVIDUAL DIFFERENCES IN WOMEN’S PERCEPTIONS OF WOMEN Background

Clinical researchers increasingly have focused on the role of cognitive factors, such as distorted processing of shape- and weight-related

2/17/2011 10:59:41 AM


information, in the etiology and maintenance of eating-disorder symptoms and in the development of cognitive-behavioral treatments for these symptoms (e.g., Cooper 2005; Fairburn, Cooper, & Shafran, 2003; Lee & Shafran, 2004). Vitousek and colleagues, for example, proposed that increased attention to and memory for shape-, weight-, and eating-related information influence the development and maintenance of eating-disorder symptoms (e.g., Vitousek, 1996; Vitousek & Hollon, 1990). Decreased attention to and memory for affective information also might play a significant role. Many women with eating disorders display marked deficits in interpersonal problem solving and emotion regulation, and they indicate that negative mood and social interactions are common triggers for eating-disordered behaviors (e.g., Lingswiler, Crowther, & Stephens, 2006; McFall, Eason, Edmondson, & Treat, 1999; Smyth et al., 2007). Such difficulties may reflect, in part, impoverished processing of affective information. In prior work, we have demonstrated that young women who report clinically significant disordered eating patterns (“High-Symptom women”) show altered processing of other women’s weightand affect-related information, as presented in full-body photos (Treat, Viken, Kruschke, & McFall, 2010; Viken et al., 2002). High-Symptom women, relative to Medium- and Low-Symptom women, showed greater attention to body size and less attention to affect in both similarityratings and prototype-classification tasks. HighSymptom women also showed better memory for body size and worse memory for affect, relative to the remaining participants, in a recognition-memory task. Thus, the operation of other higher order component cognitive processes, such as category learning, also merits investigation. The present work examines young women’s perceptions of full-body photos of other women. The women depicted in the photos varied in affect (negative to positive) and in body size (lighter to heavier). We assessed the perceived salience of the photo dimensions, and then we examined how perceived salience influenced learning in a multiphase category-learning task. The phases of the category-learning task were based loosely on


385

learning experiences that young women might encounter in the real world. Our society inundates young women with social environments in which thinness is glamorized and increased body size is a strong negative predictor of a variety of indicators of success. Moreover, women who struggle with disordered eating self-expose themselves to television shows and magazines that promote a thin ideal at greater rates than their peers (e.g., Botta, 2003; Stice, Schupak-Neuberg, Shaw, & Stein, 1994; Tiggemann, 2003). Thus, women’s immediate social environments provide highly influential “feedback” on the relevance of body size to happiness. Only gradually and later might affect emerge as a better indicator of happiness. In particular, if a young woman later undergoes therapy for an eating disorder, the therapy might include, implicitly or explicitly, attempts to reorient attention away from body size. In summary, a young women initially might experience real-world “training” that body size is relevant, with affect being only a later-learned cue. Finally, the young woman might experience some sort of training to reorient attention away from the initially learned cue. Participants were assigned randomly to one of four learning conditions, each of which contained multiple phases of training. Table 17.1 presents the conditions and learning phases. Because we wanted to counterbalance which dimension initially was relevant, some participants learned an initial category structure for which body size was relevant (the “Body-Size Initial” condition). Other participants learned an initial structure for which affect was relevant (the “Affect Initial” condition). The “Body-Size Initial” structure is shown in Figure 17.3. Each panel of Figure 17.3 shows the two-dimensional stimulus space, with affect plotted horizontally and body size plotted vertically. In the Initial Phase of learning in the BodySize Initial condition, photos of lighter women displaying neutral affect were to be classified into Category F, and photos of heavier women displaying neutral affect were to be classified into Category J. Only stimuli drawn from these two regions of the stimulus space were shown in the Initial Phase of the category-learning task. Participants received feedback on their

2/17/2011 10:59:41 AM

386


Table 17.1 Design of Phased-Learning Task in Women’s Learning Study Learning Phase Condition/Group

Initial

Redundant

Transfer

Shift

N

Affect Initial

Affect

Affect + Body Size

Transfer

Body Size

63

Affect Control

—

Affect + Body Size

Transfer

Body Size

59

Body-Size Initial

Body Size

Affect + Body Size

Transfer

Affect

58

Body-Size Control

—

Affect + Body Size

Transfer

Affect

62

classifications, but they were not told the basis for the feedback. Thus, body size was relevant to the category label in the Initial Phase for participants in the Body-Size Initial condition, whereas variability in affect was highly restricted and irrelevant to the category label. We expected that performance in the Initial Phase would be highly congruent with participants’ perceptual organizations (e.g., participants who perceived body size to be more salient than affect would show far better initial performance in the Body-Size Initial condition). In the second Redundant Phase of learning for those in the Body-Size Initial condition, affect became a redundant relevant cue, in that body size and affect both perfectly predicted the category label. For example, you can see in the second panel of Figure 17.3 (under “Redundant”) that participants could make correct classifications based on either the woman’s affect or body

Body Size

Body Size

J

F

J OR F Affect

Affect

?

?

? Affect

F

Shift Body Size

?

J

Affect

Transfer Body Size

Body Size

Redundant

Initial

J

J

F

F Affect

Figure 17.3 Phased-learning schemata, for Body-

Size Initial condition in women’s learning study.


size. Only stimuli drawn from the two regions of the stimulus space indicated by the letters “F” and “J” were shown in one of two versions of the Redundant Phase. Participants again received feedback on their classifications but were not told the basis for the feedback. We anticipated that the newly relevant dimension (i.e., affect) would not be learned as well, because the previously learned dimension (i.e., body size) already had proven to be a perfectly predictive cue for the category label. In other words, we anticipated that learning about the redundant relevant cue of affect would be “blocked” in the Body-Size Initial condition (e.g., Denton & Kruschke, 2006; Kamin, 1969). In contrast, participants in the Body-Size Control condition would not be expected to show blocking, because they did not experience the Initial Phase of learning. The third Transfer Phase in the categorylearning task presented photos from all four corners of the stimulus space, so we could assess utilization of the two dimensions. This Transfer Phase presented photos from the four regions of the stimulus space labelled with “?” in Figure 17.3 and recorded participants’ classifications. Critically, participants did not receive corrective feedback during the transfer trials. Suppose that a participant had experienced the Redundant phase in which she learned to classify lighter, unhappy women into Category F and heavier, happy women into Category J. If she were relying purely on body size when making these judgments, then we would expect her to place all lighter women in Category F and all heavier women in Category J, regardless of their affect. Alternatively, if she were using only affect as a basis for her judgments in the Redundant phase,

2/17/2011 10:59:41 AM


then she would place all unhappy women in Category F and all happy women in Category J, regardless of their body size. Thus, the pattern of classifications in the Transfer Phase was diagnostic of a participant’s utilization of bodysize and affect information. We anticipated that participants in the Body-Size Initial condition would transfer much more strongly to body size than to affect, because of the blocking of affect. Participants in the Body-Size Control condition, in contrast, were expected to show less transfer to body size, because affect would not have been blocked for them. The final Shift Phase in the category-learning task made relevant the dimension that was irrelevant in the Initial Phase. Thus, for participants in the Body-Size Initial condition, women displaying positive affect were to be classified into Category F, whereas women displaying negative affect were to be classified into Category J. You can see that one way to perform well on this task is to ignore the initial dimension of body size and make classification decisions based on affect. Photos of women from the four regions of the stimulus space labeled in the Shift panel in Figure 17.3 were presented, and participants received corrective feedback. This final phase was intended to encourage a shift of attention to the initially irrelevant dimension. We expected that participants in the Body-Size Initial condition would show worse performance in the Shift Phase, relative to those in the Body-Size Control condition, because participants in the former condition would be attempting to learn a category structure based on a blocked cue. Overall, we anticipated that the blocking manipulation in the Redundant Phase would result in suppressed learning about the blocked cue for participants in the experimental conditions (i.e., the Affect Initial and Body-size Initial conditions), as assessed in the Transfer and Shift phases. Notably, this prediction presupposes that the current design is analogous to the classic blocking design, in which the blocked cue does not appear at all in the initial phase. In the current design, in contrast, the blocked cue appears in the initial phase with a middling value (i.e., either neutral affect or moderate body size).


387

According to RASHNL, either attentional or associative learning mechanisms could underlie the predicted blocking effects. More specifically, among participants experiencing the Initial Phase of training, decreased transfer to the blocked cue and suppressed shift learning about the blocked cue could reflect (a) only generalization from the items learned in the Initial Phase, or (b) generalization and a shift of attention toward the initially relevant unblocked cue and away from the initially irrelevant blocked cue. The predicted main effects of blocking are qualitatively similar regardless of whether attention shifting plays a role, and one of the key benefits of quantitative modeling is parametric estimation of the magnitude of attention shifting. We hoped to demonstrate that attention shifting played a role in learning but recognized that integral processing of the stimulus dimensions might preclude this possibility. Perceptual organization also was predicted to exert a strong influence on initial learning, transfer performance, and later learning, whereby performance was superior on category structures that were congruent with the participants’ perceptual organization. Thus, we expected that the blocking manipulation would not eliminate the influence of perceptual organization on later transfer and learning. Finally, we explored the interaction between perceptual organization and the blocking manipulation. Research on blocking in human category learning with simple artificial stimuli demonstrates that it is harder to block a more salient cue (Denton & Kruschke, 2006). To the extent that the classic blocking design and the current design operate analogously, we hoped to demonstrate that the blocking effect would be weaker when the to-be-blocked dimension was more salient. This prediction also hinged on separable processing of the stimulus dimensions, which presumably would be necessary to support learning via attention-shifting mechanisms. On the other hand, it might be the case that the stimulus dimensions were processed holistically, such that learning was driven only by associative mechanisms. In this case, the current design (involving middling values on the “blocked” dimension) and classic designs are

2/17/2011 10:59:41 AM

388


not analogous. Predictions in this case could be derived accurately only from model simulations, but qualitatively we would expect that if the initially irrelevant dimension were salient, then learning of the initial categories would be relatively difficult, and transfer would be less consistent with the initially relevant dimension, compared to when the initially irrelevant dimension was not salient. The magnitude of the difference between experimental and control groups could not be predicted in advance, but it could be assayed via best-fitting parameter values in the RASHNL model. No clinical information was obtained from participants in this study, because a prohibitively large number of participants would be necessary to examine the extent to which disordered eating status moderates the hypothesized effects. We know from our earlier work that variation in women’s perceptual organization of body-size and affect information is a reliable correlate of disordered eating patterns, however. Thus, a strong connection between individual differences in perceptual organization and category-learning performance would highlight the potential utility of examining the clinical relevance of individual differences in young women’s learning about body-size and affect categories structures in future research. Methods Participants

Two hundred forty-four undergraduate females received partial course credit for their participation in the study. Mean age was 19.47 (SD = 1.82), and 83.1% of participants self-identified their ethnicity as White/Caucasian. Photo Stimuli

Stimuli were pictures of 58 paid female models recruited from the university population. Each model was photographed in a white t-shirt and black stretch pants in front of a neutral background. Body size varied naturally, and each model was instructed to display both sad and happy expressions. A sample of 60 undergraduate women provided normative ratings of the photos along affect (unhappy to happy) and body


size (slender to plump) dimensions. Mean normative ratings were used to classify each woman as light, moderate, or heavy along the body-size dimension. Women whose body size was judged to be in the lower third across all three facial expressions were classified as “light” (n = 15), and women whose body size was rated in the middle or upper third were classified as “moderate” (n = 12) or “heavy” (n = 18), respectively. Body-size classifications for the 13 remaining unique women varied across their facial expressions. The woman in each photo was classified as exhibiting “negative,” “neutral,” or “positive” affect if the mean normative rating of her affect fell in the bottom, middle, or upper third, respectively, of all mean affect ratings. Prototype-Classification Tasks

Participants performed two classification tasks, which were presented in a counterbalanced order across participants. At the beginning of the first task, participants studied two prototypical photos that varied along both affect and bodysize dimensions (e.g., a “Type D woman” received normative ratings toward the extreme “heavy” end of the body-size dimension and toward the extreme “happy” end of the facial-affect dimension, whereas a “Type K woman” received normative ratings indicating that she was viewed as “light” and “unhappy”). After participants inspected the two prototypes, they classified each of 20 remaining photo stimuli as examples of one of the two types of women. Next, participants completed the same task with two new prototypes (e.g., a happy-light, “Type V” woman, and a sad-heavy, “Type N” woman). The paucity of photo stimuli precluded the use of different stimuli in the prototype-classification and phased-learning tasks. Therefore, the stimuli used in the classification task were the same as those viewed by participants in the Redundant, Transfer, and Shift Phases of the phased-learning task. This constraint necessitated construction of four versions of the classification task, depending on whether affect and body size correlated positively or negatively in the Redundant Phase for participants in the first two and the last two groups in the phased-learning task. The prototypes

2/17/2011 10:59:41 AM


viewed in the prototype-classification tasks were the same across all four versions of the tasks, however, and the prototypes were not used in the learning task, given their markedly greater familiarity. Phased-Learning Task

In all but the Transfer Phase of the categorylearning task, participants classified individual photos as members of Category F or Category J and received accurate trial-by-trial feedback on their classifications. Participants were assigned randomly to one of four learning conditions, which are displayed in Table 17.1. A schematic depiction of the learning phases for participants in one of the four learning conditions, the Body-Size Initial condition, also is provided in Figure 17.3. Stimulus presentation order was randomized separately for each participant within each block. Eight unique stimuli were presented in each of 10 blocks in the Initial Phase of learning. Four of the stimuli received mean normative ratings in the upper third of the distribution of mean ratings for affect or body size for all 174 potential stimuli, respectively, whereas the remaining four stimuli received mean ratings in the lower third. The mean ratings for the eight stimuli along the dimension that was not the basis for the category structure fell in the middle third of the distribution of normative ratings. For example, the eight stimuli viewed in the Initial Phase of the Body-Size Initial condition received mean normative ratings in the middle third of the distribution of affect ratings, as depicted in Figure 17.3. None of the photos presented in the Initial Phase had been seen previously by participants. Participants completed eight blocks of eight trials apiece in the Redundant Phase of learning. The shift to the Redundant Phase for participants in the Affect Initial and Body-Size Initial conditions was unannounced. In the Body-Size Initial and Body-Size Control conditions, four stimuli received mean normative ratings in the upper third of the ratings distribution for body size, and four stimuli received mean ratings in the lower third of the body-size-ratings distribution. Mean affect ratings for these two groups of


389

stimuli fell in either the upper or lower third of the affect ratings distribution. As depicted in Figure 17.3, affect and body size correlated positively for half of the participants in each condition (i.e., only happy-heavy and sad-light women were presented) and negatively for the remaining participants (i.e., only happy-light and sadheavy women were presented). Analogously, in the Affect Initial and Affect Control conditions, four stimuli were judged normatively to be happy and four to be unhappy; average ratings for these stimuli along the body-size dimension fell in either the upper or lower third of the distribution of body-size ratings. Additionally, body-size and affect correlated positively for half of the participants and negatively for the other half. Structurally, the two control conditions were identical, although the stimuli used in the two conditions varied. All women presented in the Redundant Phase differed from those presented in the Initial Phase. All participants classified 16 stimuli twice and received no feedback in the Transfer Phase, as shown in the lower left corner of Figure 17.3 for the Body-Size Initial condition. According to normative-ratings distributions, four stimuli were happy-heavy, four were happy-light, four were sad-heavy, and four were sad-light. Half of the stimuli had been viewed in the Redundant Phase. In the final Shift Phase of learning, participants in the Body-Size Initial and Body-Size Control conditions completed seven blocks of an effect category structure, which is shown in the lower right corner of Figure 17.3. Four of the stimuli presented on each block were judged to be happy, and four were judged to be unhappy; the body size of two of the former stimuli and two of the latter stimuli was classified as heavy, and the remaining stimuli were classified as light. Conversely, participants in the other two conditions learned a body-size category structure, in which four stimuli were light, four stimuli were heavy, and the affect classifications for these stimuli were orthogonal to the body-size classifications. All women presented in the Shift Phase had not been seen in earlier learning phases, although they had been viewed in the prototypeclassification tasks.

2/17/2011 10:59:41 AM

390


Procedure

Participants first read and signed the consent form and then were seated in front of a computer in a subject-running booth. Participants entered their age and ethnicity and then completed the two prototype-classification tasks and the phased-learning task, as described earlier. Finally, participants were debriefed and thanked for their participation. The experiment lasted approximately 55 minutes. Data Preparation Prototype-Classification Data

Logistic-regression techniques were used to estimate individual differences in perceptual organization during the classification task. The following logistic function was fit to each participant’s classification judgments for each of the two classification tasks: P ( “T Type K” or “T Type V” | A =

1 1 + exp(−[(ak * AFF ) (b ( k*

and BS values ) ) + c])

,

where k = 1 or 2, depending on the task, and AFF and BS refer to standardized normative scale values for the stimuli along affect and body-size dimensions. The absolute value of the ratio of ak to bk indicated the relative perceived salience of affect and body size. As bk approached zero, values of this ratio became extreme and unstable. Thus, the arctangent of the ratio was taken. This value has a simple geometric interpretation in the stimulus space as the best-fitting angle of a line that separates the two category responses. The final measure of relative salience of affect and body size, that is, the perceptual organization score, was the average of these transformed ratios for the two tasks: Perceptual Organization score ⎛a ⎞ ⎛a ⎞ arctan ⎜ 1 ⎟ + arctan ⎜ 2 ⎟ ⎝ b1 ⎠ ⎝ b2 ⎠ = 2

Values for this measure ranged from 0.00 radians (or 0 degrees), when the participant relied exclusively on body size in making her


classification judgments, to 1.57 radians (or 90 degrees), when the participant utilized only affect when classifying the stimuli. Participants were classified as affect oriented if their relative salience score exceeded 1.22 radians (or 70 degrees), as body size oriented if their score was less than 0.349 radians (or 20 degrees), or as both oriented if their score lay between these two extremes. Over twice as many participants were classified as affect oriented (n = 107, 44.2%) rather than as body size oriented (n = 53, 21.9%), and one-third of the participants were classified as both oriented (n = 82, 33.9%). A chi-square test supported the independence of perceptual organization classification and learning group, χ2 (6) = 4.337, p > 0.50, indicating that the three classes of perceptual organizations were distributed uniformly across the four learning groups. Transfer Data

Analogous logistic-regression techniques were used to quantify each participant’s relative utilization of affect and body size on the transfer trials in the phased-learning task. The formula shown earlier was fit to each participant’s classifications of all 32 stimuli, the arctangent of the absolute value of the ratio of a to b was calculated, and the arctangent was transformed into degrees. The resulting values ranged between 0, which indicated perfect transfer to body size, and 90, which indicated perfect transfer to affect. Finally, these values were reflected for participants in the Body-Size Initial and Body-Size Control groups, so that values of 90 indicated perfect transfer to the initial cue and 0 indicated perfect transfer to the shift cue. Learning Data

Average percent correct was calculated for each participant for both the Initial and Shift Phases of learning. In the latter case, percent correct was calculated across only the first three blocks, because effects on the first three blocks were expected to be more revealing than when accuracies converged to ceiling levels in the later blocks. The pattern of results described later was similar when analogous analyses were conducted on average performance across all blocks, rather

2/17/2011 10:59:42 AM


than the first three blocks, but the effects were smaller in magnitude. Additionally, individual participants’ performance was examined on the last two blocks of the Redundant Phase, to determine whether their performance surpassed what would be expected on the basis of chance alone. The worst observed performance was 81.25% (i.e., correct response on 13 of 16 trials; n = 3). This would be expected less than 1.05% of the time, if the participant responded randomly, assuming a binomial distribution (n = 16 and p = 0.5). Thus, all participants’ data were retained for further analysis. Resampling Approach to Statistical Analyses

Resampling methods of statistical inference derive sampling distributions empirically, rather than theoretically, by sampling with replacement repeatedly from the observed sample and calculating the relevant test statistic on each resample (Good, 2006). The resulting distribution of test statistics serves as the empirically derived sampling distribution, and critical values can be obtained by reading off the relevant value in this distribution at the percentile of interest. In contrast, parametric statistical approaches specify the sampling distribution and relevant critical values by making stringent theoretical assumptions about the moments of the population distribution. Resampling approaches to statistical inference are preferred when the assumptions of parametric statistical approaches are violated severely. In the present case, a resampling approach was adopted for two primary reasons: one, most distributions were either bimodal or so severely skewed that they could not be transformed to normality without discretization and resulting loss of information; and two, variances tended to differ dramatically across the cells of the analyses. In each resampling-based comparison of J group means based on a total of N scores, 50,000 resamples with replacement of size N were obtained from the concatenated distribution of all J groups’ observed data, and these resampled data were distributed into J groups with


391

the appropriate sample sizes nj. To create the relevant sampling distributions to evaluate main effect and interaction questions, the relevant F statistics were calculated for each resample. Appropriate percentiles from this distribution then served as critical values against which the F statistics based on the sample data could be compared, and the p-value indicated the proportion of the bootstrapped sampling distribution that exceeded the observed test statistic. Pairwise mean comparisons then were conducted for significant omnibus tests, using analogous procedures to obtain two-tailed, bootstrapped p-values for t statistics. All computations were executed using Resampling Stats in MATLAB (Kaplan, 1999). Results

Does the congruence of perceptual organization with the category structure influence performance in the Initial Phase of learning? Performance in the Initial Phase of learning was expected to vary as a function of the congruence of participants’ perceptual organization classifications with the category structure. To increase the power of our analyses, participants in the Affect Initial and Body-Size Initial Groups were classified as exhibiting either high, medium, or low congruence with the category structure, depending on their perceptual organization classification. Thus, affect-oriented participants in the Affect Initial Group and body-size-oriented participants in the Body-Size Initial Group were classified as “high congruence,” whereas body-size-oriented participants in the Affect Initial Group and affect-oriented participants in the Body-Size Initial Group were classified as “low congruence.” The remaining both-oriented participants in both groups were classified as “medium congruence.” Figure 17.4 depicts average performance in the Initial Learning Phase as a function of the congruence of participant perceptual organization with the initial category structure. The resampling-based omnibus evaluation of whether at least two of the group means differed was significant, F = 15.29, p < 0.001. All follow-up evaluations of pair-wise differences between means were significant: high versus

2/17/2011 10:59:42 AM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS 90

1.00 0.96 0.95

0.91

0.90 0.83

0.85 0.80

Transfer (in degrees) to initial category structure

Mean Proportion Correct

392

80

82.84

78.53

Experimental Group Control Group

70.95

70 60

51.70 42.18

50 40 30 20

11.26

10 0

0.75 High

Medium

Low

Congruence of Perceptual Organizational Classification with initial Category structure

High

Medium

Low


Figure 17.5 The blocking manipulation and the Figure 17.4 The congruence of perceptual orga-

nization with the initial category structure influences performance in the initial learning phase of the women’s learning study. Bars correspond to bootstrapped standard error of the means.

congruence of perceptual organization with the initial category structure influence performance in the transfer phase of the women’s learning study. Bars correspond to bootstrapped standard error of the means.

medium, t = –3.03, p < 0.01; high versus low, t = −5.12, p < 0.001; medium versus low, t = –2.95, p < 0.01. Thus, participants performed markedly better in the Initial Phase when learning a category structure that was congruent with their perceptual organization classification and struggled markedly when learning an incongruent category structure. Do the blocking manipulation and the congruence of perceptual organization with the initial category structure influence performance in the Transfer Phase? Performance in the Transfer Phase was expected to vary as a function of both the blocking manipulation and the congruence of participants’ perceptual organization classifications with the initial category structure. As in the previous analysis, participants in the Affect Initial and Body-Size Initial Groups were classified as exhibiting either high, medium, or low congruence with the initial category structure on the basis of their perceptual organization classification. Additionally, participants in both the Affect Initial and Body-Size Initial Groups were classified as members of the experimental group for the blocking manipulation, and the remaining participants were classified as members of the control group. Figure 17.5 depicts average performance in the Transfer Phase as a function of the congruence of participant perceptual organization with the initial category structure and the blocking manipulation.

Larger values indicate greater transfer to the initial cue, with values of 90 indicating perfect transfer to the initial cue and values of 0 indicating perfect transfer to the shift cue. The main effect of the blocking manipulation was significant, F = 57.19, p < 0.001, and the difference between the experimental and control means—the overall “blocking effect”—was substantial at 25.3 degrees. As expected, the experimental group was much more likely to transfer to the initial cue, presumably because either participants shifted attention toward the initial cue and away from the blocked cue or because participants generalized from the initial-phase items. In contrast, the control group was more likely to show transfer patterns that were purely congruent with their perceptual organization. The main effect of the congruence of participants’ perceptual organization classifications with the initial category structure also was significant, F = 77.43, p < 0.001. Follow-up evaluations indicated that all pair-wise differences were significant: high versus medium, t = 5.79, p < 0.001; high versus low, t = 12.30, p < 0.001; medium versus low, t = 50.01, p < 0.001. In other words, regardless of the blocking manipulation, participants transferred more strongly to the cue that was congruent with their perceptual organization. A significant interaction between the blocking manipulation and the congruence of participants’


2/17/2011 10:59:42 AM



cue than control participants, because transfer was at a functional ceiling among control participants, thereby deflating the difference between experimental and control groups. Do the blocking manipulation and the congruence of perceptual organization with the initial category structure influence performance in the Shift Phase of learning? Performance in the Shift Learning Phase was expected to vary as a function of both the blocking manipulation and the congruence of participants’ perceptual organization classifications with the initial category structure. As in the previous analyses, participants in the Affect Initial and Body-Size Initial Groups were classified as exhibiting high, medium, or low congruence with the initial category structure, depending on their perceptual organization classification. We anticipated that low-congruence participants would perform substantially better in the shift phase than high-congruence participants, because participants with low congruence between their perceptual organization and the initially relevant cue had high congruence with the shift-phase relevant cue. Participants also were classified as members of the experimental or control groups for the blocking manipulation, as in the previous analyses. Figure 17.6 depicts average performance in the Shift Learning Phase as a function of the congruence of participant perceptual organization with the initial

1.00 Mean Proportion Correct First Three Blocks

perceptual organization classifications with the initial category structure emerged, F = 10.07, p < 0.001. Follow-up comparisons of the blocking effect for each level of the congruence factor indicated a significant blocking effect for the low-congruence group, t = 5.68, p < 0.001, and the medium-congruence group, t = 4.04, p < 0.001, and a nonsignificant trend for the high-congruence group, t = .60, p < 0.10. For example, you can see in Figure 17.5 that there was little effect of the blocking manipulation when the initial category structure was highly congruent with the perceptual organization classification; in both cases, participants transferred overwhelmingly to the dimension underlying the initial category structure. In contrast, when the initial category structure was less congruent with the perceptual organization classification, the effect of the blocking manipulation on participants’ transfer patterns was substantial (i.e., the experimental group showed much stronger transfer to the dimension underlying the initial category structure than the control group). Overall, therefore, the blocking effect was of similar strength for the medium- and low-congruence groups and significantly stronger than for the high-congruence group, which showed a nonsignificant blocking effect. Based on analogy to previous research with blocked, separably processed cues that had no middling value in the Initial Phase (Denton & Kruschke, 2006), we had anticipated that the low-congruence group would show the weakest blocking effect, because the to-be-blocked cue was highly salient, whereas the high-congruence group would have shown the strongest blocking effect, secondary to the weak perceived salience of the to-be-blocked cue. The results seem to indicate instead that the Initial-Phase items had a strong influence in the Transfer Phase, producing, in the LowCongruence group, marked generalization to the initial category structure, thereby inflating the difference between experimental and control groups. In the high-congruence condition, on the other hand, the influence of perceptual organization on initial learning was so substantial that it would have been almost impossible for experimental participants in the high-congruence group to shift more strongly to the initial

393

0.90

0.93

Experimental Group Control Group 0.82

0.81

0.80 0.70 0.70 0.60

0.59 0.57

0.50 High

Medium

Low


Figure 17.6 The blocking manipulation and the

congruence of perceptual organization with the initial category structure influence performance on the first three blocks of the shift learning phase of the women’s learning study. Bars correspond to bootstrapped standard error of the means.

2/17/2011 10:59:42 AM

394


category structure and the blocking manipulation. Analyses were conducted on average performance in the first three blocks only; performance was near ceiling for many participants in later blocks, and early learning performance was expected to be more diagnostic of the effects of interest. The main effect of the blocking manipulation was significant, F = 14.31, p < 0.001, indicating that later learning about the blocked cue was attenuated for the experimental group relative to the control group. This main effect echoes previous findings that learning about a blocked cue is retarded relative to learning about nonblocked control cues (Kruschke, 2005b; Kruschke & Blair, 2000). The main effect of the congruence of participants’ perceptual organization classifications with the initial category structure also was significant, F = 61.62, p < 0.001. Follow-up evaluations indicated that all pair-wise differences were significant: low versus medium, t = 4.14, p < 0.001, low versus high, t = 11.97, p < 0.001, medium versus high, t = 6.37, p < 0.001. In other words, regardless of the blocking manipulation, participants learned more rapidly a category structure that was congruent with their initial perceptual organization. A significant interaction between the blocking manipulation and the congruence of participants’ perceptual organization classifications with the initial category structure emerged, F = 4.50, p < 0.05. Follow-up comparisons of the blocking effect for each level of the congruence factor indicated a significant blocking effect for the low-congruence group, t = –3.66, p < 0.001, and the medium-congruence group, t = –3.85, p < 0.01, and a nonsignificant effect for the high-congruence group, t = 0.47, n.s. Inspection of Figure 17.6 facilitates understanding of this interaction. Participants in the low-congruence group by definition perceived the shift cue to be much more salient than the initial cue. These participants showed a strong blocking effect, such that those in the control group learned quickly about the shift cue (which was not blocked), whereas those in the experimental group learned far more slowly about the shift cue (which was blocked). Participants in the medium-congruence group showed a similar


pattern. But participants in the high-congruence group—who perceived the shift cue to be far less salient than the initial cue—showed no evidence of blocking. This pattern of findings was inconsistent with the analogy to previous research involving blocked cues (with no initially middling values and attentionally separable dimensions; Denton & Kruschke, 2006). This prior work suggested that it would be harder to block a more salient shift cue, such that participants in the high-congruence group would have shown the strongest blocking effect. Instead, the results are consistent with the hypothesis that the Initial-Phase items have continuing influence throughout subsequent phases and attention is not shifted easily from the enduring perceived salience of these holistically processed dimensions. Conclusions

This study evaluated three potential influences on women’s learning about other women’s affective or weight-related information: individual differences in the perceived salience of affect and body-size information, an experimental blocking manipulation, and the interaction between these two factors. We also used formal modeling techniques to evaluate whether shifts of attention toward relevant stimulus dimensions played a role in participants’ learning. As expected, congruence of individual differences in perceptual organization with the experienced category structure exerted a strong influence on performance throughout the phasedlearning task, regardless of whether participants were in an experimental or control condition for the blocking manipulation. In the Initial Learning Phase, participants who experienced a category structure that was congruent with their perceptual organization showed percent-correct scores that were 13 percentage points higher than participants who experienced an incongruent category structure. In the Transfer Phase, participants showed much stronger transfer to a cue that was congruent with their perceptual organization (82.84 degrees) than to a cue that was incongruent with their perceptual organization (51.70 degrees). And in the Shift Learning

2/17/2011 10:59:42 AM


Phase, participants learning a category structure that was congruent with their perceptual organization showed a 29-point advantage in percent correct, relative to participants learning an incongruent category structure. The greater overall perceived salience of affect than body size also influenced category learning. Average performance on the initial affect structure (M = 0.94, SD = 0.09) was significantly better than average performance on the body-size structure (M = 0.87, SD = 0.13), t(119) = 3.52, p < 0.01. Performance on the later affect structure (M = 0.80, SD = 0.20) also was superior to that on the later body-size structure (M = 0.66, SD = 0.20), t(240) = 5.72, p < 0.001. Thus, both average perceptual organization across participants (i.e., the greater perceived salience of affect than body size overall) and individual differences in perceptual organization influenced category learning, providing further evidence of the generalizability of the normative perceptual organization-learning relationship that previously has been observed with simple, artificial stimuli. The blocking manipulation produced a notable reduction in transfer to and later learning about a blocked cue. Whereas control participants on average showed moderate transfer to the dimension underlying the initial category structure (43.99 degrees), experimental participants showed much stronger transfer to the (unblocked) dimension underlying the initial structure (68.49 degrees). Experimental participants also showed attenuated shift learning of the blocked cue relative to control participants (70% versus 78% correct). The blocking effect on transfer and learning could reflect either shifting of attention away from the blocked cue or a remapping of the associations between regions of the psychological space and the category labels in the Initial Learning Phase. We fit RASHNL to the choice data from the four groups simultaneously to evaluate whether attention shifting played a role in learning. The best-fitting parameters showed an attentional shift rate of zero. This lack of attentional learning is consistent with the stimulus dimensions being integral, not separable, as was suggested by the fact that the best fitting


395

similarity-scaling metric was Euclidean, not city block. The modeling suggests conclusions here that echo those made for men’s perceptions of women in the previous section of the chapter. Women’s learning about other women’s affect and body size is influenced strongly by the observer’s perceived salience of the dimensions. It is difficult to shift attention rapidly between these dimensions, even when performance on a category-learning task would be facilitated by doing so. The congruence of perceptual organization with the initial category structure interacted with the blocking manipulation. In general, when the initial category structure was congruent with the perceptual organization that a participant brought to the task, then performance on the shift category structure was very poor indeed. When, initially, participants experienced a category structure that was incongruent with their perceptual organization, they showed an average of 83% correct. When participants only later were exposed to a category structure that was incongruent with their perceptual organization—after having experienced a category structure that either was congruent or could be perceived as congruent (the Redundant Relevant Cues phase of learning, in the case of control participants)—they showed an average of 59% correct (or 68% correct, if all blocks in the shift learning phase are considered). This disparity in performance as a function of learning history and perceptual organization congruence is quite worrisome, given the simplicity of the category structures being presented (e.g., heavy vs. light or happy vs. sad). Thus, it will be of particular interest in future work to evaluate whether women who report disordered eating patterns are particularly likely to struggle when trying to learn an affect category structure at all, and especially after experiencing a body-size category structure. If so, then the blocking paradigm might serve as a useful analog for a real-life circumstance in which the social environment’s reinforcement of young women’s preoccupation with body-size information as a royal road to happiness makes it extremely difficult to learn about alternative predictors of success.

2/17/2011 10:59:42 AM

396


CLOSING COMMENTS Category-learning processes play a central role in everyday life, and cognitive scientists have worked for decades to develop valid models and paradigms for the investigation of normative learning processes with simple, artificial stimuli. The work described in this chapter highlights the generalizability of these models and methods to the study of clinically relevant individual differences in category-learning processes with far more complex, socially and emotionally relevant information. Accounting for individual differences in category learning with complex stimuli necessitates an increased focus on individual differences in the perceived salience of stimulus dimensions. The present work documents the substantial influence of across-individual variability in perceived dimensional salience on category learning, such that participants learn category structures that are more congruent with their underlying perceptual organizations far more quickly than category structures that are less congruent. This faster learning of salience-congruent structures persists even after learning a salience-incongruent structure, most likely because the socially relevant stimuli have integral dimensions that prevent shifts of attention between dimensions. This observed link between individual differences in perceived salience and individual differences in category learning extends the well-established link between normative perceived salience and category learning. The dimensions of the complex, socially relevant stimuli of interest to some clinical researchers also may be processed in a more holistic fashion than the components of the artificial stimuli of primary interest to cognitive researchers, which typically are processed more separably. Prior research with artificial stimuli has demonstrated that attention-shifting mechanisms play a far less significant role in category learning with integral-dimension stimuli (Nosofsky & Palmeri, 1996) than separabledimension stimuli. The research described herein extends this finding to applied research with far more complex stimuli. In both reported


studies, formal fits of RASHNL (Kruschke & Johansen, 1999) to the category-learning data indicated that learning resulted not from shifting attention toward relevant stimulus dimensions and away from irrelevant dimensions (because the best-fitting attentional shift rate was zero), but rather from individual differences in the initial perceived salience of stimulus dimensions and the gradual strengthening of associations between regions of the stimulus space and category labels. The evident difficulties in dynamic reallocation of attention for integral stimuli enhance the significance of individual differences in perceptual organization in applied social learning. They also highlight the importance of future research on the conditions associated with increased dimensional attention shifting, even when the stimulus dimensions are perceived more holistically, because the ability to shift attention rapidly could be quite adaptive in social perceptual learning. More generally, the present work illustrates the utility of translating associative-learning paradigms to address applied questions about clinically and socially relevant processing of complex stimuli. These paradigms may offer useful analogs of “real-world” social learning environments that presumably contribute to the development of attitudes and beliefs of interest to clinical researchers. Formal modeling techniques then could be used to elucidate the mechanisms underlying learning, which not only might enhance clinical scientists’ understanding of the role of more dynamic aspects of processing in clinical phenomena but also might contribute to the development of novel prevention or intervention strategies that directly target deficient or maladaptive cognitive processing. Current cognitively oriented treatments rely primarily on verbally mediated techniques that emphasize the identification and modification of specific distorted thoughts and beliefs (Treat et al., 2007), but cognitive therapy might be augmented usefully by drawing to a greater degree on the plethora of associative-learning paradigms to modify problematic processing patterns or to facilitate the acquisition of important category structures.

2/17/2011 10:59:42 AM


REFERENCES Ashby, F. G., & Maddox, W. T. (2005). Human category learning. Annual Review of Psychology, 56, 149–178. Beck, A. T. (1976). Cognitive theory and the emotional disorders. New York, NY: International Universities Press. Botta, R. A. (2003). For your health? The relationship between magazine reading and adolescents’ body image and eating disturbances. Sex Roles, 48, 389–399. Carroll, J. D., & Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an N-way generalization of “EckartYoung” decomposition. Psychometrika, 35, 283–320. Cooper, M. J. (2005). Cognitive theory in anorexia nervosa and bulimia nervosa: Progress, development and future directions. Clinical Psychology Review, 25, 511–531. Davison, M. L. (1992). Multidimensional scaling. Malabar, FL: Krieger Publishing Company. Denton, S. E., & Kruschke, J. K. (2006). Attention and salience in associative blocking. Learning and Behavior, 34, 285–304. Fairburn, C. G., Cooper, Z., & Shafran, R. (2003). Cognitive behaviour therapy for eating disorders: A “transdiagnostic” theory and treatment. Behaviour Research and Therapy, 41, 509–528. Farris, C. A., Treat, T. A., Viken, R. J., & McFall, R. M. (2008). Sexual coercion and the misperception of sexual intent. Clinical Psychology Review, 28, 48–66. Good, P. I. (2006). Resampling methods: A practical guide to data analysis (3rd ed.). Boston, MA: Birkhäuser. Johnston, L., & Ward, T. (1996). Social cognition and sexual offending: A theoretical framework. Sexual Abuse: Journal of Research and Treatment, 8, 55–80. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment (pp. 279–296). New York, NY: Appleton-Century-Crofts. Kaplan, D. T. (1999). Resampling Stats in MATLAB. Arlington, VA: Resampling Stats, Inc. Kelly, G. A. (1955). The psychology of personal constructs. New York, NY: Norton. Kruschke, J. K. (1992). ALCOVE: An exemplarbased connectionist model of category learning. Psychological Review, 99, 22–44.


397

Kruschke, J. K. (1993). Human category learning: Implications for back propagation models. Connection Science, 5, 3–36. Kruschke, J. K. (1996). Dimensional relevance shifts in category learning. Connection Science, 8, 225–247. Kruschke, J. K. (2005a). Category learning. In K. Lamberts & R. L. Goldstone (Eds.), The handbook of cognition (pp. 183–201). London, England: Sage. Kruschke, J. K. (2005b). Learning involves attention. In G. Houghton (Ed.), Connectionist models in cognitive psychology (pp. 113–140). Hove, England: Psychology Press. Kruschke, J. K., & Blair, N. J. (2000). Blocking and backward blocking involve learned inattention. Psychonomic Bulletin and Review, 7, 636–645. Kruschke, J. K., & Johansen, M. K. (1999). A model of probabilistic category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1083–1119. Lee, M., & Shafran, R. (2004). Information processing biases in eating disorders. Clinical Psychology Review, 24, 215–238. Lingswiler, V. M., Crowther, J. H., & Stephens, M. A. P. (2006). Affective and cognitive antecedents to eating episodes in bulimia and binge eating. International Journal of Eating Disorders, 8, 533–539. MacCallum, R. C. (1977). Effects of conditionality on INDSCAL and ALSCAL weights. Psychometrika, 42, 297–305. McDonel, E. C., & McFall, R. M. (1991). Construct validity of two heterosocial perception skill measures for assessing rape proclivity. Violence and Victims, 6, 17–30. McFall, R. M. (1990). The enhancement of social skills: An information-processing analysis. In W. L. Marshall & D. R. Laws (Eds), Handbook of sexual assault: Issues, theories, and treatment of the offender (pp. 311–330). New York, NY: Plenum Press. McFall, R. M., Eason, B. J., Edmondson, C. B., & Treat, T. A. (1999). Social competence and eating disorders: Development and validation of the anorexia and bulimia problem inventory. Journal of Psychopathology and Behavioral Assessment, 21, 365–394. Nosofsky, R. M. (1992). Similarity scaling and cognitive process models. Annual Review of Psychology, 43, 25–53.

2/17/2011 10:59:42 AM

398


Nosofsky, R. M., & Palmeri, T. J. (1996). Learning to classify integral-dimension stimuli. Psychonomic Bulletin and Review, 3, 222–226. Segal, Z. V., & Stermac, L. E. (1990). The role of cognitions in sexual assault. In W. L. Marshall, D. R. Laws, & H. E. Barbaree (Eds.), Handbook of sexual assault: Issues, theories and treatment of the offender (pp. 161–174). New York, NY: Plenum. Shepard, R. N. (1964). Attention and the metric structure of the stimulus space. Journal of Mathematical Psychology, 1, 54–87. Smyth, J. M., Wonderlich, S. A., Heron, K. E., Sliwinski, M. J., Crosby, R. D., Mitchell, J. E., & Engel, S. G. (2007). Daily and momentary mood and stress are associated with binge eating and vomiting in bulimia nervosa patients in the natural environment. Journal of Consulting and Clinical Psychology, 75, 629–638. Stice, E., Schupak-Neuberg, E., Shaw, H. E., & Stein, R. I. (1994). Relation of media exposure to eating disorder symptomatology: An examination of mediating mechanisms. Journal of Abnormal Psychology, 103, 836–840. Tiggemann, M. (2003). Media exposure, body dissatisfaction and disordered eating: Television and magazines are not the same! European Eating Disorders Review, 11, 418–430. Treat, T. A., McFall, R. M., Viken, R. J., & Kruschke, J. K. (2001). Using cognitive science methods to assess the role of social information processing in sexually coercive behavior. Psychological Assessment, 13, 549–565. Treat, T. A., McFall, R. M., Viken, R. J., Kruschke, J. K., Nosofsky, R. M., & Wang, S. S.


(2007). Clinical-cognitive science: Applying quantitative models of cognitive processing to examination of cognitive aspects of psychopathology. In R. W. J. Neufeld (Ed.), Advances in clinical-cognitive science: Formal modeling and assessment of processes and symptoms (pp. 179–205). Washington, DC: APA Books. Treat, T. A., McFall, R. M., Viken, R. J., Nosofsky, R. M., MacKay, D. B., & Kruschke, J. K. (2002). Assessing clinically relevant perceptual organization with multidimensional scaling techniques. Psychological Assessment, 14, 239–252. Treat, T. A., Viken, R.J., Kruschke, J.K., & McFall, R. M. (2010). The role of attention, memory, and correlation-detection processes in eating disorders. Journal of Mathematical Psychology, 54, 184–195. Viken, R. J., Treat, T. A., Nosofsky, R. M., McFall, R. M., & Palmeri, T. (2002). Bulimics and controls’ differential attention to and classification of body-size and affect stimulus information. Journal of Abnormal Psychology, 111, 598–609. Vitousek, K. B. (1996). The current status of cognitive-behavioral models of anorexia nervosa and bulimia nervosa. In P. M. Salkovskis (Ed.), Frontiers of cognitive therapy (pp. 383–418). New York, NY: Guilford Press. Vitousek, K. B., & Hollon, S. D. (1990). The investigation of schematic content and processing in eating disorders. Cognitive Therapy and Research, 14, 191–214.

2/17/2011 10:59:42 AM

CHAPTER 18 Evaluative Conditioning A Review of Functional Knowledge and Mental Process Theories Jan De Houwer

Pavlovian conditioning can be defined as a change in behavior that is due to the pairing of stimuli. Evaluative conditioning is a subclass of Pavlovian conditioning effects in that it refers to the effect of stimulus pairings on liking. As is the case with other instances of Pavlovian conditioning, two questions can be asked about evaluative conditioning: (1) What elements in the environment moderate the effect of stimulus pairings on liking? (2) Which mental processes mediate the effect of stimulus pairings on liking? In this chapter, I present a brief overview of the literature pertaining to these two questions.

Applying Pavlovian conditioning to a phenomenon in daily life always boils down to the following question: Does the phenomenon qualify as an instance of Pavlovian conditioning? As is evidenced by different chapters in this book, many phenomena have been considered as instances of Pavlovian conditioning. In the present chapter, I examine whether changes in liking can also be understood from this perspective. Before I review the evidence on this topic, I first consider in more detail the meaning of the term “Pavlovian conditioning” because this determines what phenomena can be seen as instances of Pavlovian conditioning and thus how Pavlovian conditioning can be applied.

WHAT IS PAVLOVIAN CONDITIONING? In some textbooks, Pavlovian conditioning is defined in a very narrow manner as the unconscious formation of associations that results from the pairing of a conditioned stimulus (CS) and an unconditioned stimulus (US) and that

leads to changes in physiological responses (e.g., salivation) (e.g., Arnould, Price, & Zinkhan, 2004; Evans, Jamal, & Foxall, 2006). Such a definition is narrow in that it limits conditioning to changes in one particular class of responses that are due to one particular type of mental process. Other researchers impose other restrictions on the definition of Pavlovian conditioning, for instance, when arguing that “true” Pavlovian conditioning always involves biologically relevant USs (e.g., Miller & Matute, 1996). In this section, I will argue that it makes more sense to define Pavlovian conditioning in the broadest possible manner, that is, as a change in behavior that is due to the pairing of stimuli. Imposing restrictions on the nature of the changes, the nature of the responses, the nature of the stimuli, or the nature of the underlying mental processes is unnecessary and can lead to deleterious effects (see De Houwer, 2007, 2009, for an in-depth discussion). Let us consider the necessity of restrictions regarding the nature of the responses. There is no a priori reason why only changes in one type 399


2/16/2011 5:53:50 PM

400


of responses (e.g., physiological responses) should be considered as possible instances of Pavlovian conditioning. It is true that Pavlov’s (1927) seminal studies focused on changes in a physiological response (salivation) but since then many studies on Pavlovian conditioning have looked at changes in other types of behavior. For example, in studies on autoshaping, food is presented to pigeons each time a key lights up. As a result of this contingency, the pigeons start pecking the key (e.g., Brown & Jenkins, 1968). There is no logical reason why a change in salivation could count as Pavlovian conditioning but a change in key pecking could not. More generally, there is no logical reason why any type of response should be excluded on an a priori basis from the realm of Pavlovian conditioning (Eelen, 1980). There is also no a priori reason why Pavlovian conditioning should be restricted to changes in behavior that are due to a particular process like the (unconscious) formation of associations in memory. It is true that some researchers (used to) believe that conditioned changes in behavior are due to the unconscious formation of associations (e.g., Thorndike, 1911; Watson, 1913). But this was simply one possible theory about the processes that produce conditioning effects. There are also other theories of classical conditioning according to which the formation of associations does depend on awareness (e.g., Dawson & Schell, 1987) or that do not refer to the formation of associations at all. For instance, Mitchell, De Houwer, and Lovibond (2009a) have argued that Pavlovian conditioning results from the conscious formation and evaluation of propositions about CS-US relations. Why should some theories be dismissed on an a priori basis? In addition to the fact that there are no good reasons to limit Pavlovian conditioning to only a subset of effects of the pairing of stimuli, doing so can have several deleterious consequences (see De Houwer, 2007). First, it could prevent researchers from acknowledging and studying the similarities and differences between different effects of stimulus pairings. The mere fact of using different labels for different effects can draw attention away from similarities in the conditions under which the effects occur. Moreover, knowledge about the similarities and differences


between different effects of stimulus pairings can provide crucial information about the processes by which the pairing of stimuli influences behavior. Second, defining Pavlovian conditioning in terms of particular processes hampers the study of conditioning because a particular change in behavior can be considered as an instance of conditioning only when it can be established that it is produced by certain processes. This is problematic because it is often extremely difficult to determine whether a particular process is responsible for a certain change in behavior. The same problem arises with other criteria that are difficult to verify (e.g., whether stimuli are biologically relevant). Finally, a definition of Pavlovian conditioning in terms of mental processes holds the risk that doubts about the role of those mental processes lead to doubts about the existence of Pavlovian conditioning effects (Eelen, 1980). For instance, the downfall of behaviorist theories of conditioning led to a dramatic slowdown in research on conditioning effects because many researchers inferred that humans do not show “true” Pavlovian conditioning effects (i.e., effects that are due to unconscious association formation; e.g., Brewer, 1974). More generally, defining effects in terms of processes violates the scientific principle that theories (i.e., the explanans) need to be separated from the effects that they aim to explain (i.e., the explanandum). Because of these reasons, I prefer to define Pavlovian conditioning as a change in behavior that results from the pairing of stimuli. The only criterion that is left for labeling a change in behavior as an instance of Pavlovian conditioning is that the change is due to the pairing of stimuli rather than other factors such as genetic makeup (e.g., changes in behavior due to maturation), the simple repeated experience of a stimulus (e.g., habituation), or the relation between a behavior and a stimulus (e.g., operant conditioning; De Houwer, 2009). In laboratory situations, this criterion can be verified by implementing appropriate control conditions (e.g., by presenting stimuli in an unpaired manner). Although it might not always be easy to determine the environmental cause of a change in behavior, applying the minimal definition of

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

Pavlovian conditioning will always be more straightforward than applying other definitions in which other criteria are added to the minimal definition (e.g., criteria regarding the nature of the response, the nature of the US, or the type of underlying process). Some readers might not be willing to accept this minimal definition, perhaps because of historical reasons. All definitions are subject to matters of convention and everyone is thus free to add criteria to the minimal definition. If one chooses to do so, it should, however, be made explicit what the additional criteria are, why those criteria are thought to be crucial, and how it can be verified whether a certain effect of stimulus pairings meets those criteria. One should also be aware of and be willing to accept the limitations that such additional criteria impose on Pavlovian conditioning research. I prefer the minimal definition because it maximizes the scope of Pavlovian conditioning research, because it is logically coherent, and because it is the easiest to verify.

WHAT IS EVALUATIVE CONDITIONING AND HOW CAN WE STUDY IT? Adopting a minimal definition of Pavlovian conditioning allows one to fully engage in the question of whether certain changes in liking are instances of Pavlovian conditioning. The only criterion that needs to be satisfied in order to categorize a change in liking as an instance of Pavlovian conditioning is that the change is due to a relation between stimuli rather than to other environmental factors such as the simple repeated presentation of a single stimulus (as is the case in mere exposure effects; see Bornstein, 1989). A change in liking that is due to the pairing of stimuli is typically referred to as an evaluative conditioning (EC) effect (De Houwer, 2007; Levey & Martin, 1975). EC effects are thus a subset of Pavlovian conditioning effects, the only distinction being that EC effects always involve changes in liking, whereas Pavlovian conditioning effects can relate to a change in any type of observable response. The procedures that are used to study evaluative conditioning are


401

also very similar to the procedures that are used to study other forms of Pavlovian conditioning: Stimuli are paired in a certain way under certain conditions. The only systematic difference between EC procedures and other Pavlovian conditioning procedures is that all EC procedures measure on changes in liking (De Houwer, 2007). As is the case with other instances of Pavlovian conditioning (and thus all possible applications of Pavlovian conditioning), several questions can be raised regarding EC (De Houwer, 2009). A first question concerns the procedural conditions under which the pairing of stimuli results in changes in liking. These conditions refer either to the abstract nature of the relation between stimuli or to the concrete way in which the relation is implemented. The abstract nature of the relation refers to the statistical properties of the relation between stimuli (e.g., contiguity, contingency) and possible changes in those properties (e.g., extinction). Studies that examine the impact of these procedural features inform us about which aspects of the relation are crucial in establishing EC. The concrete implementation of the relation requires choices regarding the stimuli that are presented, the type of liking response that is observed, the organism that experiences the relation, the context in which the relation is present, and the way in which information about the relation is communicated. By manipulating each of these procedural elements, knowledge can be gained about those aspects of the procedure or environment that determine whether and to what extent a relation between stimuli influences the liking of those stimuli. I will refer to this knowledge as functional knowledge (De Houwer, in press). Note that functional knowledge about EC goes beyond mere empirical knowledge. First, it involves more than the description of an event in that it offers hypotheses about which elements in the procedure are assumed to exert an impact on behavior. Like all hypotheses, functional knowledge needs to be backed up by (empirical and logical) arguments. For instance, the empirical observation that the liking of a particular CS changes after it has been paired 10 times with a particular US can be explained in terms of the

2/16/2011 5:53:50 PM

402


mere co-occurrence of the CS and US or in terms of the positive statistical contingency between the CS and US (i.e., the fact that the CS and US often co-occur and never occur separately). Which procedural explanation is correct needs to be determined on the basis of additional research (e.g., by manipulating the number of CS-only and US-only trials). Second, functional knowledge goes beyond the individual event in that hypotheses about the role of procedural elements are thought to be valid in more than one situation. For instance, the hypothesis that CS-US co-occurrences are sufficient for EC is thought to be true not only for one particular CS-US pair in one specific context. Functional knowledge is therefore more than a collection of empirical findings (see De Houwer, 2009). The second question that can be raised regarding EC concerns the nature of the mental processes that are responsible for EC. Theories about the mental processes that underlie EC aim to explain functional knowledge about EC, that is, why certain aspects of the EC procedure determine the magnitude and direction of EC effects. For instance, some mental process models predict that EC will depend primarily on the number of CS-US co-occurrences, whereas other mental process models predict that the statistical contingency is important (i.e., that also the number of CS-only and US-only trials count). The merits of different mental process theories can be evaluated by examining (a) how well they can account for the available functional knowledge (i.e., the heuristic function of process theories) and (b) the extent to which they predict novel functional knowledge (i.e., the predictive function of process theories). In the remainder of this chapter, I use this framework to review the literature on EC (see De Houwer, Thomas, & Baeyens, 2001, and Field, 2005, for other reviews). The first section summarizes findings regarding the effects of the abstract nature of the relation and the concrete implementation of the relation. The second section provides an overview of mental process theories of EC. The review is not intended to be exhaustive in the sense that all relevant studies will be discussed or even cited. I will, however, attempt to present a summary of what I believe


to be the main findings and insights that are currently available in the literature on EC.

FUNCTIONAL KNOWLEDGE The Abstract Nature of the Relation The Statistical Properties of the CS-US Relation

The presence of stimuli can be related in many different ways. Therefore, the value of the claim that a change in behavior is due to a CS-US relation depends heavily on how much is known about the exact properties of the CS-US relation that determine its effect on behavior. We will consider three statistical properties of the CS-US relation: co-occurrence, contingency, and redundancy. Co-occurrence Several studies suggest that EC becomes stronger as the number of CS-US pairings increases, that is, the more often the CS and US co-occur in space and time (e.g., Baeyens, Eelen, Crombez, & Van den Bergh, 1992a; BarAnan, De Houwer, & Nosek, in press). However, after more than 10 pairings, there is no further increase (Bar-Anan et al., in press) or even a slight decrease in EC (Baeyens et al., 1992a). Although co-occurrence is important, it is not a necessary condition for EC. EC can occur even when the CS and US have never co-occurred but are related in an indirect manner through co-occurrences with a third stimulus. In a seminal study on this topic, Hammerl and Grabitz (1996) first presented pairs of neutral stimuli (e.g., A-B). Afterward, one of the stimuli of each pair was presented together with a US (e.g., B-US). During a test phase, it was found that the latter pairings changed not only the liking of the CS that was paired with the US (e.g., B) but also the liking of the other neutral stimulus (e.g., A). This effect, which is known as sensory preconditioning, has been replicated and examined in more detail by Walther (2002). Contingency Baeyens, Hermans, and Eelen (1993) examined the impact of the statistical contingency between the CS and the US by intermixing CS-US trials with trials on which only the CS or only the US occurred. Unlike what is typically found in Pavlovian conditioning

2/16/2011 5:53:50 PM


research, the magnitude of EC was not affected by the number of trials on which only the CS or only the US was presented. However, the failure of Baeyens et al. to find an impact of contingency might have been due to the low power of their statistical tests. More research on this issue is thus needed. Redundancy A relation between a CS and a US can also be described in terms of its redundancy, that is, the extent to which the CS-US relation overlaps with other CS-US relations. We know of only two studies that looked at the impact of the redundancy of the CS-US relation on EC. The relation between a CS A and the US can be described as redundant when CS A always co-occurs together with a second CS B that also co-occurs with the US. Dwyer, Jarratt, and Dick (2007) found that the change in liking of A was as large after A-US pairings (i.e., A+ trials) as after AB-US pairings (i.e., AB+ trials). That is, they failed to observe an overshadowing effect. Lipp, Neumann, and Mason (2001) on the other hand, observed a bigger change in liking of A when the A-US relation was redundant (i.e., after AB+ and B+ trials) than when the A-US relation was not redundant (i.e., after AB+ and B- trials). Again, more research on this topic is needed before clear conclusions can be drawn regarding the impact of redundancy on EC. Changes in the Statistical Properties of the CS-US Relation

The relation between a CS and US does not necessarily remain stable over context or time. The presence of a CS-US relation can be preceded or followed by the absence of the same relation. It can also vary according to the broader context in which the organism is placed. I will now discuss the impact of these various procedural elements. Presentations of the CS-Only and the US-Only Most research about changes in the CS-US relation focused on the impact of CS postexposure trials. These trials are presented after the CS-US trials and contain only the CS. Studies on Pavlovian conditioning typically show that a conditioned change in behavior can be reversed by presenting the CS on its own after


403

the CS-US trials. Surprisingly, this phenomenon, which is known as extinction, has not been found in a number of EC studies (e.g., Baeyens, Crombez, Van den Bergh, & Eelen, 1988; Baeyens, Eelen, Van den Bergh, & Crombez, 1989a; De Houwer, Baeyens, Vansteenwegen, & Eelen, 2000; Diaz, Ruiz, & Baeyens, 2005) even under conditions in which extinction of Pavlovian conditioning was observed (e.g., Vansteenwegen, Francken, Vervliet, De Clercq, & Eelen, 2006). Lipp, Oughton, and Lelievre (2003; also see Lipp & Purkis, 2006, and Blechert, Michael, Vriends, Margraf, & Wilhelm, 2007), on the other hand, did find evidence for extinction in EC when they measured CS liking during the extinction phase rather than only after the extinction phase. In a more recent study, however, Blechert, Michael, Williams, Purkis, and Wilhelm (2008) failed to find extinction even when liking of the CSs was measured during the extinction phase. In sum, the available evidence suggests that CS-only trials have little effect on conditioned changes in liking. Only two studies have examined the effects of CS-only trials that are presented before the CS-US trials (De Houwer et al., 2000; Stuart et al., 1987). In both studies, CS-only trials seemed to interfere with EC, that is, with the effect of subsequent CS-US trials. This effect of CS preexposure trials is typically referred to as latent inhibition (e.g., Lubow & Gewirtz, 1995). Finally, there is one study that looked at the effects of US-only trials. In that study, Hammerl, Bloch, and Silverthorne (1997) found that US-only trials also delayed EC, both when presented before and after the CS-US trials. Occasion Setting In the studies discussed so far, the change in the CS-US relation occurred over time. A CS-US relation can, however, also depend on the physical context that is present (e.g., a second stimulus or the color of a room). In that case, the physical context can function as an occasion setter that signals when the CS-US relation holds. Baeyens and colleagues (Baeyens, Crombez, De Houwer, & Eelen, 1996; Baeyens, Hendrickx, Crombez, & Hermans, 1998) studied occasion setting in EC by using the color of drinks as a signal for when a particular fruit

2/16/2011 5:53:50 PM

404


flavor would be followed by a bad aftertaste. For instance, a fruit flavor was followed by the bad aftertaste only in green drinks but not in blue drinks. During a test phase in which none of the drinks had the bad aftertaste, drinks with the fruit flavor that was previously paired with the bad aftertaste were liked less than drinks with other flavor, regardless of the color of the drink. Hence, the studies of Baeyens and colleagues did not provide evidence to support the conclusion that conditioned changes in liking depend on the presence of the context in which the CS-US pairings were presented. Note that Hardwick and Lipp (2000) did observe occasion setting when using modulation of the startle response as an index of learning. However, it has been argued that modulation of the startle responses does not provide a good index of evaluative conditioning because it can also be affected by factors other than the valence of the CSs (see section on “The Nature of the Evaluative Responses”). The Concrete Implementation of the CS-US Relation The Nature of the Stimuli

The Modality and Semantic Category of the CSs and USs EC effects have been observed with a wide range of visual stimuli such as photographs of human faces as CSs and USs (e.g., Baeyens et al., 1992a), paintings as CSs and USs (Levey & Martin, 1975), outdoor sculptures as CSs and USs (e.g., Hammerl & Grabitz, 1996), nonsense words as CSs and valenced words as USs (e.g., Staats & Staats, 1957), cartoon characters as CSs and valenced words and photographs as USs (e.g., Olson & Fazio, 2001), and names of fictitious products as CSs and valenced pictures as USs (e.g., Pleyers, Corneille, Luminet, & Yzerbyt, 2007; Stuart et al., 1987). EC effects have also been found with nonvisual stimuli. For instance, gustatory (taste) stimuli have been used as CSs (e.g., artificial fruit flavors) and USs (e.g., sugar or a chemical substance that produces a bad aftertaste; e.g., Baeyens, Eelen, Van den Bergh, & Crombez, 1990b; Lamote, Baeyens, Hermans, & Eelen, 2004; Zellner, Rozin, Aron, & Kulish, 1983). Olfactory (odor) stimuli have also been used successfully


as USs (e.g., Hermans, Baeyens, Lamote, Spruyt, & Eelen, 2005; Todrank, Byrnes, Wrzesniewski, & Rozin, 1995) and as CSs and USs (e.g., Stevenson, Boakes, & Wilson, 2000). Other studies demonstrated EC with somatosensory stimuli such as the touch of objects as CSs and USs (e.g., Hammerl & Grabitz, 2000) or mild electric shocks as USs (e.g., Hermans et al., 2002). Auditory stimuli such as pieces of music have also been employed successfully as USs (e.g., Bierley, McSweeney, & Vannnieuwkerk, 1985). The Valence and Identity of the US Although EC can be characterized as a general phenomenon that can involve many different kinds of stimuli, the available evidence suggests that properties of the stimuli do determine the magnitude and direction of EC. First and foremost, the valence of the US determines the direction of the change in valence of the CS. That is, a CS that is paired with a positively valenced US tends to become more positive, whereas a CS that is paired with a negatively valenced US tends to become more negative (e.g., Baeyens et al., 1992a). The importance of US valence rather than the specific identity of the US is also illustrated by the fact that EC is found not only when a CS is repeatedly paired with a single (positive or negative) US but also when it is paired with different USs that all share the same valence (e.g., Olson & Fazio, 2001; Stahl & Unkelbach, 2009). There is less certainty about the effects of the extremity of the valence of the USs. On the one hand, Baeyens et al. (1988) found equally large effects with strongly valenced USs (e.g., mutilated faces) than with mildly valenced USs (e.g., disliked photographs of intact human faces). In a more recently reported study, Jones, Fazio, and Olson (2009) selected USs on the basis of a pilot study in which the valence of each US had to be determined as quickly as possible. USs that were classified quickly (and tended to be strongly valenced) resulted in smaller EC effects than USs that were classified slowly (and tended to be mildly valenced). Although more studies are needed to examine the impact of US extremity, the available evidence gives little reason to believe that more extreme USs lead to more extreme EC. If anything, the reverse seems to be true.

2/16/2011 5:53:50 PM


There are some indications that positive USs are less effective than negative USs (e.g., Baeyens, Eelen, & Van den Bergh, 1990a). One explanation might be that positive USs tend to be less extremely valenced than negative USs. As we discussed in the previous paragraph, there is, however, little evidence to support the idea that more extreme USs produce stronger EC. A second possible explanation is that people seem to differ more in their appreciation of potentially positive stimuli than in their liking of potentially negative stimuli (Peeters & Czapinski, 1990). This would result in more variability (and thus statistically smaller effects) when positive USs are used than when negative USs are used. Finally, organisms might be genetically prepared to learn more quickly about relations that involve negative stimuli than about relations that involve positive stimuli (Peeters & Czapinski, 1990). More research is needed before a definite conclusion regarding this issue can be reached. Changes in the Valence and Identity of the US Assume that a CS is paired with a positive US, and that this results in an increase in the liking of the CS. Research showed that when the positive US is afterward made negative (e.g., by pairing it with negative stimuli), the CS will also become more negative. This finding is known as US revaluation and has been observed in a number of studies (e.g., Baeyens, Eelen, Van den Bergh, & Crombez, 1992b; Walther, Gawronski, Blank, & Langer, 2009). It should be noted, however, that Baeyens, Vanhouche, Crombez, and Eelen (1998) failed to observe a US-revaluation effect in a study where artificial fruit flavors were used as CSs and a soap-like aftertaste was used as the US. They attributed this failure to the specific nature of their US. In studies on US revaluation, the identity of the US with which a CS is paired is kept stable, whereas the valence of the US is changed. In studies on counterconditioning, both US identity and US valence are changed. For instance, after a CS has been paired with a positively valenced US, it is paired with a different US with a negative valence. Results have shown that the second pairing undoes and sometimes even


405

reverses the change in liking that was produced (e.g., Baeyens et al., 1989a). The Intrinsic Relation Between CS and US Apart from possible main effects of the nature of the CS and the nature of the US, EC can also depend on the intrinsic relation between the CS and US, that is, the interaction of the nature of the CS and the nature of the US. For instance, Baeyens et al. (1990a) found that pairing the color of a drink with a bad aftertaste did not result in a change in liking of other drinks with that color, whereas pairing the flavor of a drink with a bad aftertaste did change the liking of other drinks with that flavor. Hence, it seems that the relation between the color and the aftertaste has less effect on the liking of drinks than the relation between the flavor and the aftertaste. Likewise, Todrank et al. (1995) found that pairings of neutral photographs of human faces as CSs with odors as USs influenced the liking of the photographs only if the odors were “plausibly human” (e.g., sweat or fragrances). Such findings suggest that EC depends on the intrinsic relation between the CS and the US (see Garcia & Koelling, 1966, for similar finding in the context of other types of Pavlovian conditioning). Researchers have also looked at the impact of the perceptual similarity between the CSs and USs. Martin and Levey (1978) found that EC was stronger for CSs that were paired with perceptually similar USs than for CSs that were paired with perceptually dissimilar USs. This effect was replicated by Field and Davey (1999) but shown to be an artifact in that it remained present even when the CS-US pairs were never presented. The artifact can arise when researchers select CSs on the basis of unreliable evaluative ratings that a participant gives at the start of the experiment. It is, for instance, possible that evaluative ratings of some CSs change simply as the result of seeing other stimuli that are used in the experiment. This is most likely to happen for CSs that resemble other positive or negative stimuli in the stimulus list (see Field & Davey, 1999, for more details). Baeyens, Eelen, Van den Bergh, and Crombez (1989b) manipulated the perceptual similarity between the CS and US independently of the evaluative ratings of participants and failed

2/16/2011 5:53:50 PM

406


to find stronger EC for perceptually similar CS-US pairs. In sum, there is little evidence to support the conclusion that EC depends on the perceptual similarity between the CS and US. The Manner in Which the CS and US Are Presented Once the stimuli have been selected, they need to be presented in a certain manner. This requires decisions about the time and location at which the stimuli occur, their size, luminance, and so on. Results suggest that EC effects become stronger the more the CS and US are presented in close temporal and spatial proximity (e.g., Jones, Fazio, & Olson, 2009). Even though EC has been observed when the US always is presented before the CS (Martin & Levey, 1987; Stuart, Shimp, & Engle, 1987), effects in those situations appear to be weaker than when the presentation of the CS and US overlap or when the CS is presented briefly before the US. Finally, when using pictures as CSs and USs, Jones et al. (2009) recently found that EC effects increase in magnitude with increases in the size of the CSs. The Nature of the Evaluative Response

What sets EC apart from other forms of Pavlovian conditioning is that it involves changes in evaluative responses, that is, responses that are assumed to reflect the liking of objects. Most often, changes in direct measures of liking are examined. Such direct measures require the participant to self-assess his or her liking of the CSs and USs, for instance, by selecting a number on a Likert scale or by sorting the stimuli into separate piles for liked, neutral, or disliked pictures (e.g., Baeyens et al., 1992a; Levey & Martin, 1975). In more recent studies, indirect measures of liking have also been used, in which liking is inferred from performance during reaction time tasks (e.g., De Houwer, Hermans, & Eelen, 1998; Hermans, Vansteenwegen, Crombez, Baeyens, & Eelen, 2002; Kerkhof et al., 2009; Mitchell, Anderson, & Lovibond, 2003), from physiological responses (e.g., Vansteenwegen, Crombez, Baeyens, & Eelen, 1998), and from neurological responses (e.g., Klucken et al., 2009). One should, however, be aware of the fact that a change in a response can be labeled as EC only if it can be argued that the response is


an evaluative response, that is, if it can be argued that the response provides an index of liking. For instance, some physiological responses such as skin conductance and modulation of the startle response seem to be determined by arousal level of stimuli rather than by the evaluative properties of stimuli and therefore do not qualify as indices of liking (e.g., Vansteenwegen et al., 1998). Nevertheless, the available evidence supports the conclusion that EC has been observed in a variety of evaluative responses. Nature of the Organism That Experiences the CS-US Relation

Species Many studies on EC have been conducted with human samples (see De Houwer et al., 2001, for a review). Some studies could be described as studies on EC in non-human animals (e.g., Boakes, Albertella, & Harris, 2007; Capaldi, 1992; Delamater, Campese, LoLordo, & Sclafani, 2006). Unfortunately, the literature on human and non-human EC has developed independently, perhaps because it is not clear whether (the determinants of) evaluative responses in human and non-humans are comparable. Societal Status and Age Most studies on EC involved psychology students, but some studies involved children (e.g., Baeyens, Eelen, Crombez, & De Houwer, 2001; Field, 2006; Fulcher, Mathews, & Hammerl, 2008) and community samples (e.g., Baeyens, Wrzesniewski, De Houwer, & Eelen, 1996). However, little is known about whether EC depends on the societal status or age of the participants. Personality and Mental Disorders Surprisingly few EC studies have taken into account the personality of the participants. One exception is a study by Baeyens et al. (1992a) who measured the “evaluative style” of their participants but failed to find differences in EC depending on whether participants were “feelers” or “thinkers.” We know of one study that compared EC in patients and healthy controls. In this study, Blechert et al. (2007) observed that patients with poststraumatic stress disorder showed delayed extinction of EC compared to a healthy control group. This finding is somewhat puzzling given the fact that extinction of EC is

2/16/2011 5:53:50 PM


rarely observed in groups of healthy participants (see earlier discussion). Neurological and Genetic Properties Neurological research on EC is still in its infancy. There have been two studies on the role of the amygdaloid nuclear complex (ANC), a structure that is critically involved in Pavlovian conditioning of fear responses. Whereas Johnsrude, Owen, White, Zhao, and Bohbot (2000) found impaired EC in individuals with unilateral damage to the ANC, Coppens et al. (2006) did find intact EC in these individuals. More research is thus needed to clarify the involvement of this and other brain regions in EC. To the best of my knowledge, there are no studies on the effects of the genetic makeup of individuals or the use of chemical substances on EC. Nature of the Context in Which the Relation Is Presented

Other Tasks A relation between stimuli cannot be presented in a vacuum but always occurs in a broader context in which other regularities are present. For instance, a particular CS-US relation can be present in a context in which participants are asked to fulfill certain tasks. Several studies confirm that tasks that are present briefly before or during the presentation of the CS-US relation can influence EC. Corneille, Yzerbyt, Pleyers, and Mussweiler (2009) recently found that EC effects were stronger when, briefly before the presentation of the CS-US pairs, participants were asked to detect similarities between various kinds of pictures compared to when their task was to detect differences between pictures. Tasks that are present during the presentation of the CS-US pairs also seem to be able to influence EC. Whereas some researchers found that the presence of an attention-demanding task facilitates EC (e.g., Fulcher & Hammerl, 2001; Walther, 2002), others found that it reduces the size of EC (e.g., Field & Moore, 2005; Pleyers, Corneille, Yzerbyt, & Luminet, 2009). Finally, Baeyens, Eelen, and Van den Bergh (1990b) found that instructing participants to detect which CS goes together with which US did not influence the magnitude of EC. If anything, EC effects were smaller in the condition where participants were


407

asked to discover the CS-US pairs. In sum, although it is clear that the broader context in which CS-US pairs are presented does influence EC, it is not clear how other regularities in the environment (such as secondary tasks) affect EC. Other Effects of the CS-US Relation The context in which a CS-US relation is presented consists not only of other contingencies and their effects on the organism but also of other effects of that CS-US relation. This consideration leads to the question of whether the effects that a CS-US relation has on the liking of the CS (i.e., EC) are somehow related to other effects of that CS-US relation (e.g., effects on physiological responses or on conscious knowledge). Research on this question has focused mainly on effects of the CS-US relation on responses that can be seen as indices of awareness of the CS-US relation. The results of this research are, however, mixed. Some studies suggest that EC is independent of awareness of the CS-US relation (see De Houwer et al., 2001; Field, 2005, for reviews). For instance, Baeyens et al. (1990a; also see Dickinson & Brown, 2007, but see Wardle, Mitchell, & Lovibond, 2007) found that a contingency between a flavor and a bad aftertaste led to a change in liking of the flavor, even though participants could not indicate which flavor was paired with the bad aftertaste. When, however, there was a relation between the color of the drinks and the bad aftertaste, participants were able to indicate which color was paired with the bad aftertaste, but they did not change their liking of the drinks with that color. A similar dissociation between EC and awareness of the CS-US relation was found by Fulcher and Hammerl (2001). They found that manipulations that increased awareness of the CS-US relations (e.g., instructions to detect the contingencies, blockwise presentations of the CS-US pairs) actually decreased the magnitude of EC. In many other studies, however, a close link between awareness of the CS-US contingencies and EC has been observed. In a particularly convincing study, Pleyers et al. (2007) calculated for each participant the EC effect for CSs for which the participants could indicate the valence of the US with which it was paired and the EC effect for CSs for which

2/16/2011 5:53:50 PM

408


participants did not remember the valence of the associated US. Significant EC occurred only for the former set of CSs. Similar results were found in several recent studies (e.g., Dawson, Rissling, Schell, & Wilcox, 2007; Stahl & Unkelbach, 2009; Stahl, Unkelbach, & Corneille,2009; Wardle et al., 2007). Nature of the Way in Which the CS-US Relation Is Communicated

In most EC studies, the CS and US stimuli are physically present and are thus experienced directly by the participants. However, a CS-US relation in the world can have an impact even when the organism does not experience the CS and US stimuli directly. First, EC can result also from observing other organisms that do directly experience the CS-US relation. In studies on observational EC, Baeyens, Vansteenwegen et al. (1996) videotaped a child who drank little cups of water, some of which also contained a product that resulted in a bad, soap-like aftertaste. The model always displayed a negative facial expression after drinking cups of water with a bad aftertaste and a positive facial expression after drinking water that did not have the bad aftertaste. Other children watched the video while drinking cups of water simultaneously with the model. None of the drinks of the observers contained the bad aftertaste. Instead, each drink contained one of two neutral fruit flavors. The order of the drinks was arranged in such a way that the model always displayed a positive facial expression after the observers drank a cup of water with one flavor (e.g., apricot). When the observers drank water with the second flavor (e.g., lychee), the model always displayed a negative expression. Afterward, the observers reported that they liked the first flavor better than the second one. One explanation for this finding is that the facial expression of the model functioned as a US. From this perspective, the participants did experience the relation between the CS (flavor) and US (facial expression) directly. Another explanation is that the observers were influenced by how the model reacted to the relation between the CS (flavor) and US (bad aftertaste of the drink). A second way of presenting information about CS-US relations indirectly is by giving


verbal instructions. For instance, Gregg, Seibt, and Banaji (2006) told some participants that members of a fictitious social group called “Niffites” generally behaved in a positive manner, whereas members of a different fictitious social group called “Luupites” generally behaved in a negative manner. Simply providing this information was sufficient to create a preference for Niffites compared to Luupites, regardless of whether liking was measured using rating scales or derived from performance in a reaction time task. In a related study, De Houwer (2006) told participants the names of Niffites would be paired with positive stimuli, whereas names of Luupites would be paired with negative stimuli (or vice versa). Even though the stimuli were never actually presented, a reaction time measure of liking (i.e., the Implicit Association Test) showed that participants did like the Niffites names better than the Luupites names. Although there have been few studies that directly compare the effects of direct experiences of CS-US relations with the effects of verbal information about the same CS-US relations, the available evidence does allow for the conclusion that both ways of presenting information can lead to EC.

MENTAL PROCESS THEORIES Until now we have considered only the impact of elements of the procedure on EC. Our review shows that a lot can be learned about the determinants of EC at this level of explanation. In this section, we will try to explain this functional knowledge by describing mental processes that might underlie EC. The aim of these process theories is to explain how the pairing of stimuli can lead to changes in liking and why these changes depend on the nature of the procedure. The Conceptual-Categorization Account

According to Davey (1994), the pairing of a CS and a US can result in a change in the liking of the CS because it makes salient those features of the CS that it has in common with the US. For example, assume that an evaluatively neutral face has the features of brown eyes, long shape, full lips, and long hair. Also assume that this

2/16/2011 5:53:50 PM


neutral face is repeatedly presented together with a liked US that has the features of blue eyes, round shape, full lips, and long hair. The CS-US pairings are assumed to increase the salience of the features that the CS has in common with the US, that is, full lips and long hair. As a result, the CS is more likely to be categorized as a liked stimulus. Note that this explanation of EC does not refer to the existence of associations in memory but does attribute EC to the CS-US pairings. The model of Davey (1994) correctly predicts that EC should depend mainly on the number of co-occurrences of the CS and US because it is on these trials that the salience of the CS features can change. Once the salience of certain CS features has been increased, these changes in salience (and thus liking) might persist even when the CS or US is subsequently presented on its own. However, the model has difficulties explaining a number of other findings. First, EC has been found even when the CS and US belong to different modalities and therefore do not have features in common. Second, the model cannot explain the fact that revaluation of a particular US after the CS-US pairings influences the liking of the CS with which it was paired but not the liking of other CSs (Baeyens et al., 1992b; Walther et al., 2009). US revaluation might influence the nature of the features that a participant regards as typical for liked or disliked stimuli. This should, however, influence the liking of all CSs, not only the CS that was paired with that specific US. Third, the model does not provide an explanation for how merely instructing participants about a CS-US relation can result in EC. The Holistic Account

Martin and Levey (1978, 1994; Levey & Martin, 1975) postulated that the co-occurrence of a CS and a US automatically results in the formation of a holistic representation that encodes stimulus elements of both the CS and US, as well as the valence of the US. Once the holistic representation has been formed, the CS can activate this representation and thus the evaluation that was associated with the US. The holistic model correctly predicts that conditioned changes in liking should depend


409

mainly on CS-US co-occurrences because these trials result in the formation of the holistic representation. Subsequent CS-only trials should not alter the holistic representation and thus should also not influence the conditioned change in liking. Hence, the model is in line with those studies that failed to find extinction of EC. The model can explain the effect of US revaluation if it is assumed that the US can activate the holistic representation on the US-revaluation trials and if the new valence of the US can be integrated in the holistic representation. The holistic account also predicts that EC can occur in the absence of awareness. Some argue, however, that unconscious EC has still not been demonstrated conclusively (e.g., Dawson et al., 2007; Lovibond & Shanks, 2002). The model cannot explain that, in many cases, EC does occur only when participants are aware of the CS-US contingencies (e.g., Pleyers et al., 2007). It also fails to provide an account of how EC can occur on the basis of instructions, in the absence of any CS-US pairings. The Misattribution Account

Recently, Jones et al. (2009) proposed a misattribution theory according to which the evaluative reaction that is evoked by the US can be become associated with the CS on trials where the CS and US co-occur. In line with early behaviorist theories of conditioning (e.g., Thorndike, 1911; Watson, 1913), it is assumed that these S-R associations can be formed in the absence of awareness of the CS-US relation. Jones et al. do postulate, however, that the formation of an S-R association depends on what they call an “implicit misattribution” of the evaluative response to the CS. That is, participants need to (incorrectly) assume that the evaluation that they experience is caused by the CS rather than by the US. This misattribution can occur implicitly in that it does not depend on a conscious evaluation of the CS or US. Nevertheless, any variable that influences the likelihood that the US valence is misattributed to the CS should influence EC. Jones et al. indeed observed an impact of a number of these variables, including the size of the CS (feelings are more likely to be

2/16/2011 5:53:50 PM

410


attributed to large and thus salient CSs) and spatial proximity (feelings are more likely to be attributed to CSs that are close to a US). However, such effects could be explained also without invoking the assumption that EC depends on a misattribution of the evaluation that is evoked by the US (e.g., CS size and CS-US proximity could as such influence the formation of CS-US associations). The most striking support for the misattribution theory, however, comes from the finding that mildly valenced USs result in stronger EC effects than strongly valenced USs. Jones et al. (2009) explain this finding by assuming that the feeling evoked by strongly valenced USs is more likely to be correctly attributed to the US and thus less likely to be misattributed to the CS. However, the effect that Jones et al. observed was small (i.e., only marginally significant despite a large sample) and present only in participants who were classified as unaware. Moreover, Baeyens et al. (1988) failed to find an effect of US extremity. More research on this topic is clearly needed. The misattribution theory cannot explain the effects of US revaluation on EC because the representation that is assumed to underlie EC does not contain information about the stimulus properties of the US. Because of this, the US cannot activate the CS representation during the revaluation trials. The model can also not explain EC in the absence of CS-US co-occurrences, such as with indirect CS-US relations (i.e., sensory preconditioning) or EC as the result of instructions. The Referential Account

Baeyens et al. (1992b; Baeyens & De Houwer, 1995) postulated that there are two types of learning. The first type concerns the learning of predictive relations by which the CS becomes a signal for the upcoming presentation of the US. This type of signal learning is assumed to underlie most cases of Pavlovian conditioning, that is, most effects of the pairing of stimuli (also see Rescorla, 1988). The second type concerns the learning of referential relations by which the CS becomes a stimulus that simply refers to (i.e., makes one think of) the US without


becoming a signal for the actual presentation of the US. EC is assumed to depend on the second type of learning. Whereas Baeyens et al. (1992b) seemed to assume that the two types of learning actually depend on the formation of different types of CS-US associations in memory, De Houwer (1998; De Houwer et al., 2001) suggested that signal and referential learning depend on a single learning mechanism that produces only one type of CS-US association. According to De Houwer, signal and referential learning differ because the CS-US associations have a different effect on preparatory responses than on evaluative responses. Because referential learning is thought to be driven by the co-occurrence of stimuli, the referential model can explain why EC seems to be resistant to extinction (i.e., impervious to the effects of CS-only trials that are presented after CS-US trials). The redundancy of the CS-US relation should also not have an effect. Moreover, the model can explain the presence of US-revaluation effects because the change in liking of the US is assumed to be mediated by the activation of the US representation. Changing this representation during the US-revaluation trials should thus also affect the liking of the CS. Just like the holistic and the misattribution accounts, the referential account postulates that referential learning is independent of awareness of CS-US contingencies and should thus occur also in the absence of contingency awareness. However, the evidence on unaware EC is still inconclusive. Moreover, at least in certain cases, there does seem to be a close link between EC and contingency awareness (e.g., Pleyers et al., 2007). Finally, because association formation is assumed to be a gradual process that is driven by the actual presence of stimuli, it is difficult to see how the referential model can explain EC as the result of instructions. The Propositional Account

De Houwer (2007; De Houwer, Baeyens, & Field, 2005) put forward the suggestion that EC, like all other forms of conditioning (see De Houwer, 2009; Mitchell et al., 2009a), might result from the formation of propositions about the CS-US

2/16/2011 5:53:50 PM


relation. According to this propositional account, the liking of the CS will change only after participants have formed the conscious proposition that the CS is paired with a valenced US. Although the model does not always explain how this propositional knowledge results in a change in liking (see Mitchell, De Houwer, & Lovibond, 2009b), it does postulate that the formation of a proposition about the CS-US relation is a necessary mediating step. One possible way in which propositions can influence liking is that participants use propositional knowledge about the CS-US relation as a justification for determining how much they like the CS. For instance, the fact that a CS is paired with a negative US can be seen as a justification for disliking the CS (De Houwer et al., 2005). Because the formation of propositions is a conscious and effortful process, the propositional account predicts that EC should depend on awareness of the CS-US relation. It would thus not be able to account for convincing evidence for unaware EC. The model also predicts that other tasks that direct attention away from the CS-US pairings should hamper EC. The evidence on this issue is mixed (e.g., Fulcher & Hammerl, 2001; Pleyers et al., 2009). Although the propositional model does not make strong predictions about the statistical properties of the CS-US relation that determine EC, it is compatible with the observation that EC is driven primarily by co-occurrences of the CS and US (i.e., that contiguity rather than contingency seems to matter). Co-occurrences would be primary in those cases where EC depends not on the formation of propositions about the statistical contingency between the CS and US but on the formation of propositions about the co-occurrence of the CS and US. Because EC is assumed to depend on knowledge about the US, revaluation of the US should influence EC. Finally, because propositional knowledge can result either from experience or from instructions, the model can account for EC as the result of instructions.

CONCLUSIONS The research on EC that we reviewed in this chapter clearly shows that the pairing of stimuli


411

can lead to changes in liking of those stimuli. We have also learned that EC is a general phenomenon that occurs with many different stimuli, influences many types of evaluative responses, can be found in many different organisms and contexts, and can result both from experience, observation, and instruction. We have also learned that EC seems to be driven mostly by the co-occurrence of the CS and the US, whereas contingency and redundancy seem to be less important. Nevertheless, there are still many uncertainties about the conditions under which EC occurs and the mental processes that underlie EC. First, although EC is a general phenomenon, there have also been genuine failures to find EC (e.g., Rozin, Wrzesniewski, & Byrnes, 1998). This suggests that there are subtle but important boundary conditions that need to be fulfilled before the pairing of stimuli results in a change in liking. Second, the literature on EC is characterized by many conflicting results, including on important topics such as the relation between EC and awareness of the CS-US contingencies, the impact of US revaluation, the impact of CS postexposure trials (i.e., extinction), the impact of other tasks that direct attention toward or away from the CS-US contingencies, and the relation between EC effects that are due to experience versus instructions. Because of these conflicting results, progress regarding our understanding of the mental processes that underlie EC has been limited. Different theories make different predictions regarding the role of contingency awareness, US revaluation, extinction, attention, and instructions, but the conflicting results interfere with the selection between or refinement of these theories. As was suggested by De Houwer et al. (2005; De Houwer, 2007), it is possible that EC effects can be due to different mental processes. The conflicting results in the literature could thus be due to the fact that different processes underlie EC in the different studies. For instance, it is possible that when EC is caused by propositional processes, it will depend on awareness, US revaluation, CS postexposures, attention, and instructions. In cases where EC is independent from contingency awareness, US revaluation, CS postexposures, attention, or instructions, it might be due to more automatic processes

2/16/2011 5:53:50 PM

412


such as the formation of holistic representations. As argued by De Houwer (2007), an important task for future research on EC should thus be to uncover the variables that determine the properties of EC, that is, whether EC depends on awareness, US revaluation, CS postexposures, attention, or instructions. Such an approach can lead to new insights in the important phenomenon of evaluative conditioning.

ACKNOWLEDGMENTS The preparation of this chapter was made possible by grants BOF/GOA2006/001 and BOF09/ 01M00209 of Ghent University. I thank Helena Matute, the editors of this book, and an anonymous reviewer for comments on an earlier draft of this chapter. Correspondence should be addressed to Jan De Houwer, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium. Electronic mail can be sent to Jan.DeHouwer@ UGent.be.

REFERENCES Arnould, E., Price, L., & Zinkhan, G. (2004). Consumers (2nd ed.). New York, NY: McGraw– Hill/Irwin. Baeyens, F., Crombez, G., De Houwer, J., & Eelen, P. (1996). No evidence for modulation of evaluative flavor-flavor associations in humans. Learning and Motivation, 27, 200–241. Baeyens, F., Crombez, G., Van den Bergh, O., & Eelen, P. (1988). Once in contact, always in contact: EC is resistant to extinction. Advances in Behaviour Research and Therapy, 10, 179–199. Baeyens, F., & De Houwer, J. (1995). Evaluative conditioning is a qualitatively distinct form of classical conditioning: A reply to Davey (1994). Behaviour Research and Therapy, 33, 825–831. Baeyens, F., Eelen, P., Crombez, G., & De Houwer, J. (2001). On the role of beliefs in observational flavor conditioning. Current Psychology, 20, 183–203. Baeyens, F., Eelen, P., Crombez, G., & Van den Bergh, O. (1992a). Human EC: Acquisition trials, presentation schedule, evaluative style and contingency awareness. Behaviour Research and Therapy, 30, 133–142. Baeyens, F., Eelen, P., & Van den Bergh, O. (1990). Contingency awareness in EC: A case for unaware affective-evaluative learning. Cognition and Emotion, 4, 3–18.


Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1989a). Acquired affectiveevaluative value: Conservative but not unchangeable. Behaviour Research and Therapy, 27, 279–287. Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1989b). The influence of CS-UCS perceptual similarity/dissimilarity on human evaluative learning and signal learning. Learning and Motivation, 20, 322–333. Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1990). Flavor-flavor and colorflavor conditioning in humans. Learning and Motivation, 21, 434–455. Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1992b). The content of learning in human EC: Acquired valence is sensitive to US-revaluation. Learning and Motivation, 23, 200–224. Baeyens, F., Hendrickx, H., Crombez, G., & Hermans, D. (1998). Neither extended sequential nor simultaneous feature positive training result in modulation of evaluative flavor conditioning in humans. Appetite, 31, 185–204. Baeyens, F., Hermans, R., & Eelen, P. (1993). The role of CS-US contingency in human evaluative conditioning. Behaviour Research and Therapy, 31, 731–737. Baeyens, F., Vanhouche, W., Crombez, G., & Eelen, P. (1998). Human evaluative flavor-flavor conditioning is not sensitive to post-acquisition US-inflation. Psychologica Belgica, 38, 83–108. Baeyens, F., Vansteenwegen, D., De Houwer, J., & Crombez, G. (1996). Observational conditioning of food valence in humans. Appetite, 27, 235–250. Baeyens, F., Wrzesniewski, A., De Houwer, J., & Eelen, P. (1996). Toilet rooms, body massages, and smells: Two field studies on human evaluative odor conditioning. Current Psychology, 15, 77–96. Bar-Anan, Y., De Houwer, J., & Nosek, B. A. (in press). The role of contingency memory, intentional processes and number of pairings in evaluative conditioning. Quarterly Journal of Experimental Psychology. Bierley, C., McSweeney, F. K., & Vannieuwkerk, R. (1985). Classical conditioning of preferences for stimuli. Journal of Consumer Research, 12, 316–323. Blechert, J, Michael, T., Vriends, N., Margraf, J., & Wilhelm, F. H. (2007). Fear conditioning in posttraumatic stress disorder: Evidence for

2/16/2011 5:53:50 PM


delayed extinction of autonomic, experiential, and behavioural responses. Behavior Research and Therapy, 45, 2019–2033. Blechert, J., Michael, T., Williams, S. L., Purkis, H. M., & Wilhelm, F. H. (2008). When two paradigms meet: Does evaluative learning extinguish in differential fear conditioning? Learning and Motivation, 39, 58–70. Boakes, R. A., Albertella, L., & Harris, J. A. (2007). Expression of flavor preference depends on type of test and on recent drinking history. Journal of Experimental Psychology: Animal Behavior Processes, 33, 327–338. Bornstein, R. F. (1989). Exposure and affect: Overview and meta-analysis of research, 1968-1987. Psychological Bulletin, 106, 265–289. Brown, P.L., & Jenkins, H.M. (1968). Autoshaping of the pigeon’s key-peck. Journal of Experimental Analysis of Behavior, 5, 325–337. Capaldi, E. D. (1992). Conditioned food preferences. The Psychology of Learning and Motivation, 28, 1–33. Coppens, E., Vansteenwegen, D., Baeyens, F., Vandenbulcke, M., Van Paesschen, W., & Eelen, P. (2006). Evaluative conditioning is intact after unilateral resection of the anterior temporal lobe in humans. Neuropsychologia, 44, 840–843. Corneille, O., Yzerbyt, V. Y., Pleyers, G., & Mussweiler, T. (2009). Beyond awareness and resources: Evaluative conditioning may be sensitive to processing goals. Journal of Experimental Social Psychology, 45, 279–282. Davey, G. C. L. (1994). Defining the important questions to ask about evaluative conditioning: A reply to Martin and Levey (1994). Behaviour Research and Therapy, 32, 307–310. Dawson, M. E., Rissling, A. J., Schell, A., & Wilcox, R. (2007). Under what conditions can human affective conditioning occur without contingency awareness? Test of the evaluative conditioning paradigm. Emotion, 7, 755–766. Dawson, M. E., & Schell, A. M. (1987). Human autonomic and skeletal classical conditioning: The role of conscious cognitive factors. In G. Davey (Ed.) Cognitive processes and Pavlovian conditioning in humans (pp. 27–56). Chichester, England: Wiley. De Houwer, J. (1998). Leren is eenvoudig, doen is moeilijk: Een nieuwe visie op het onderscheid tussen evaluatieve en Pavloviaanse conditionering. [Learning is simple, performance is difficult: A new view on the distinction between evaluative


413

and Pavlovian conditioning]. Gedragstherapie, 31, 49–66. De Houwer, J. (2006). Using the implicit association test does not rule out an impact of conscious propositional knowledge on EC. Learning and Motivation, 37, 176–187. De Houwer, J. (2007). A conceptual and theoretical analysis of EC. The Spanish Journal of Psychology, 10, 230–241. De Houwer, J. (2009). Leerpsychologie [Learning Psychology]. Gent, Belgium: Academia Press. De Houwer, J. (in press). Why the cognitive approach in psychology would profit from a functional approach and vice versa. Perspectives on Psychological Science. De Houwer, J., Baeyens, F., & Field, A. P. (2005). Associative learning of likes and dislikes: Some current controversies and possible ways forward. Cognition and Emotion, 19, 161–174. De Houwer, J., Baeyens, F., Vansteenwegen, D., & Eelen, P. (2000). Evaluative conditioning in the picture-picture paradigm with random assignment of CSs to USs. Journal of Experimental Psychology: Animal Behavior Processes, 26, 237–242. De Houwer, J., Hermans, D., & Eelen, P. (1998). Affective and identity priming with episodically associated stimuli. Cognition and Emotion, 12, 145–169. De Houwer, J., Thomas, S., & Baeyens, F. (2001) Associative learning of likes and dislikes: A review of 25 years of research on human EC. Psychological Bulletin, 127, 853B869. Delamater, A. R., Campese, V., LoLordo, V. M., & Sclafani, A. (2006). Unconditioned stimulus devaluation effects in nutrient-conditioned flavor preferences. Journal of Experimental Psychology: Animal Behavior Processes, 32, 295–306. Diaz, E., Ruis, G., & Baeyens, F. (2005). Resistance to extinction of human evaluative conditioning using a between-subjects design. Cognition and Emotion, 19, 245–268. Dickinson, A., & Brown, K. J. (2007). Flavor evaluative conditioning is unaffected by contingency knowledge during training with color– flavor compounds. Learning and Behavior, 35, 36–42. Dwyer, D. M., Jarratt, F., & Dick, K. (2007). Evaluative conditioning with foods as CSs and body shapes as USs: No evidence for sex differences, extinction, or overshadowing. Cognition and Emotion, 21, 281–199.

2/16/2011 5:53:51 PM

414


Eelen, P. (1980). Klassieke conditioning: Klassiek en toch modern [Classical conditioning: Classic but nevertheless modern]. In J. R. Nuttin (Ed.), Gedrag, dynamische relatie en betekeniswereld [Behaviour, dynamic relation, and world of meaning] (pp. 321–343). Leuven, Belgium: Universitaire Pers Leuven. Evans, M., Jamal, A., & Foxall, G. R. (2006). Consumer behaviour. Chichester, England: Wiley. Field, A. P. (2006). I don’t like it because it eats sprouts: Conditioning preferences in children. Behavior Research and Therapy, 44, 439–455. Field, A. (2005). Learning to like (or dislike): Associative learning of preferences. In A. Wills (Ed.), New directions in human associative learning (pp. 221–252). Mahwah, NJ: Lawrence Erlbaum. Field, A. P., & Davey, G. C. L. (1999). Reevaluating EC: A nonassociative explanation of conditioning effects in the visual EC paradigm. Journal of Experimental Psychology: Animal Behaviour Processes, 25, 211–224. Field, A. P., & Moore, A. C. (2005). Dissociating the effects of attention and contingency awareness on evaluative conditioning effects in the visualparadigm. Cognition and Emotion, 19, 217–243. Fulcher, E. P., & Hammerl, M. (2001). When all is revealed: A dissociation between evaluative learning and contingency awareness. Consciousness and Cognition, 10, 524–549. Fulcher, E. P., Mathews, A., & Hammerl, M. (2008). Rapid acquisition of emotional information and attentional bias in anxious children. Journal of Behavior Therapy and Experimental Psychiatry, 39, 321–339. Garcia, J., & Koelling, R. A. (1966). The relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Gregg, A. P., Seibt, B., & Banaji, M. R. (2006). Easier done than undone: Asymmetry in the malleability of implicit preferences. Journal of Personality and Social Psychology, 90, 1–20. Hammerl, M., Bloch, M., & Silverthorne, C. P. (1997). Effects of US-alone presentations on human evaluative conditioning. Learning and Motivation, 28, 491–509. Hammerl, M., & Grabitz, H-J. (1996). Human evaluative conditioning without experiencing a valued event. Learning and Motivation, 27, 278–293. Hammerl, M., & Grabitz, H. -J. (2000). Affectiveevaluative learning in humans: A form of


associative learning or only an artifact? Learning and Motivation, 31, 345–363. Hardwick, S. A., & Lipp, O. V. (2000). Modulation of affective learning: An occasion for evaluative conditioning? Learning and Motivation, 31, 251–271. Hermans, D., Baeyens, F., Lamote, S., Spruyt, A., & Eelen, P. (2005). Affective priming as an indirect measure of food preferences acquired through odor conditioning. Experimental Psychology, 52, 180–186. Hermans, D., Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (2002). Expectancylearning and evaluative learning in human classical conditioning: Affective priming as an indirect and unobtrusive measure of conditioned stimulus valence. Behaviour Research and Therapy, 40, 217–234. Johnsrude, I. S., Owen, A. M., White, N. M., Zhao, W. V., & Bohbot, V. (2000). Impaired preference conditioning after anterior temporal lobe resection in humans. Journal of Neuroscience, 20, 2649–2656. Jones, C. R., Fazio, R. H., & Olson, M. A. (2009). Implicit misattribution as a mechanism underlying evaluative conditioning. Journal of Personality and Social Psychology, 96, 933–948. Kerkhof, I., Goesaert, E., Dirikx, T., Vansteenwegen, D., Baeyens, F., D’Hooge, R., & Hermans, D. (2009). Assessing valence indirectly and online. Cognition and Emotion, 23, 1615–1629. Klucken, T., Kagerer, J., Schweckendiek, J., Tabbert, K., Vaitl, D., & Stark, R. (2009). Neural, electrodermal and behavioral response patterns in contingency aware and unaware subjects during a picture-picture conditioning paradigm. Neuroscience, 158, 721–731. Lamote, S., Baeyens, F., Hermans, D., & Eelen, P. (2004). An exploration of affective priming as an indirect measure of food attitudes. Appetite, 42, 279–286. Levey, A. B., & Martin, I. (1975). Classical conditioning of human > evaluative = responses. Behaviour Research and Therapy, 4, 205–207. Lipp, O. V., Neumann, D. L., & Mason, V. (2001). Stimulus competition in affective and relational learning. Learning and Motivation, 32, 306–331. Lipp, O. V., Oughton, N., & LeLievre, J. (2003). Evaluative learning in human Pavlovian conditioning: Extinct, but still there? Learning and Motivation, 34, 219–239.

2/16/2011 5:53:51 PM


Lipp, O. V., & Purkis, H. M. (2006). The effects of assessment type on verbal ratings of conditional stimulus valence and contingency judgement: Implications for the extinction of evaluative learning. Journal of Experimental Psychology: Animal Behavior Processes, 32, 431–440. Lovibond, P. F., & Shanks, D. R. (2002). The role of awareness in Pavlovian conditioning: Empirical evidence and theoretical implications. Journal of Experimental Psychology: Animal Behavior Processes, 28, 3–26. Lubow, R. E., & Gewirtz, J. C. (1995). Latent inhibition in humans: Data, theory, and implications for schizophrenia. Psychological Bulletin, 117, 87–103. Martin, I., & Levey, A. B. (1978). Evaluative conditioning. Advances in Behaviour Research and Therapy, 1, 57–102. Martin, I., & Levey, A. B. (1987). Learning what will happen next: Conditioning, evaluation and cognitive processes. In G. Davey (Ed.), Cognitive processes and Pavlovian conditioning in humans (pp 57–82). New York, NY: John Wiley. Martin, I., & Levey, A. B. (1994). The evaluative response: Primitive but necessary. Behaviour Research and Therapy, 32, 301–305. Miller, R. R., & Matute, H. (1996). Biological significance in forward and backward blocking: Resolution of a discrepancy between animal conditioning and human causal judgment. Journal of Experimental Psychology: General, 125, 370–386. Mitchell, C. J., Anderson, N. E., & Lovibond, P. F. (2003). Measuring EC using the implicit association test. Learning and Motivation, 34, 203–217. Mitchell, C. J., De Houwer, J., & Lovibond, P. F. (2009a). The propositional nature of human associative learning. Behavioral and Brain Sciences, 32, 183–198. Mitchell, C. J., De Houwer, J., & Lovibond, P. F. (2009b). Link-based learning theory creates more problems than it solves. Behavioral and Brain Sciences, 32, 230–246. Olson, M., & Fazio, R. (2001). Implicit attitude formation through classical conditioning. Psychological Science, 5, 413–417. Pavlov, I. (1927). Conditioned reflexes. London, England: Oxford University Press. Peeters, G., & Czapinski, J. (1990). Positive-negative asymmetry in evaluations: The distinction between affective and informational negativity effects. European Review of Social Psychology, 1, 33–60.


415

Pleyers, G., Corneille, O., Luminet, O., & Yzerbyt, V. (2007). Aware and (dis)liking: Itembased analyses reveal that valence acquisition via EC emerges only when there is contingency awareness. Journal of Experimental Psychology: Learning, Memory and Cognition, 33, 130–144. Pleyers, G., Corneille, O., Yzerbyt, V., & Luminet, O. (2009). Evaluative conditioning incurs attentional costs. Journal of Experimental Psychology: Animal Behavior Processes, 35, 279–285. Rescorla, R. A. (1988) Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43, 151–160. Rozin, P., Wrzesniewski, A., & Byrnes, D. (1998). The elusiveness of EC. Learning and Motivation, 29, 397–415. Staats, A. W., & Staats, C. K. (1957). Meaning established by classical conditioning. Journal of Experimental Psychology, 54, 74–80. Stahl, C., & Unkelbach, C. (2009). Evaluative learning with single versus multiple USs: The role of contingency awareness. Journal of Experimental Psychology: Animal Behavior Processes, 35, 256–291. Stahl, C., Unkelbach, C., & Corneille, O. (2009). On the respective contributions of awareness for US valence and US identity in valence acquisition through evaluative conditioning. Journal of Personality and Social Psychology, 97, 404–420. Stevenson, R. J., Boakes, R. A., & Wilson, J. P. (2000). Resistance to extinction of conditioned odour perceptions: Evaluative conditioning is not unique. Journal of Experimental Psychology: Learning, Memory and Cognition, 26, 423–440. Stuart, E. W., Shimp, T. A., & Engle, R. W. (1987). Classical conditioning of consumer attitudes: Four experiments in an advertising context. Journal of Consumer Research, 14, 334–351. Todrank, J., Byrnes, D., Wrzesniewski, A., & Rozin, P. (1995). Odors can change preferences for people in photographs: A cross-modal evaluative conditioning study with olfactory USs and visual CSs. Learning and Motivation, 26, 116–140. Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York, NY: MacMillan. Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (1998). Extinction in fear conditioning: Effects on startle modulation and evaluative self-reports. Psychophysiology, 35, 729–736. Vansteenwegen, D., Francken, G., Vervliet, B., De Clercq, A., & Eelen, P. (2006). Resistance to extinction in evaluative conditioning. Journal of

2/16/2011 5:53:51 PM

416


Experimental Psychology: Animal Behavior Processes, 32, 71–79. Walther, E. (2002). Guilty by mere association: EC and the spreading attitude effect. Journal of Personality and Social Psychology, 82, 919–934. Walther, E., Gawronski, B., Blank, H., & Langer, T. (2009). Changing likes and dislikes through the back door: The US-revaluation effect. Cognition and Emotion, 23, 889–917. Wardle, S. G., Mitchell, C. J., & Lovibond, P. F. (2007) Flavor evaluative conditioning and


contingency awareness. Learning and Behavior, 35, 233–241. Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177. Zellner, D. A., Rozin, P., Aron, M., & Kulish, C. (1983). Conditioned enhancement of human’s liking for flavor by pairing with sweetness. Learning and Motivation, 14, 338–350.

2/16/2011 5:53:51 PM

CHAPTER 19 Instrumental and Pavlovian Conditioning Analogs of Familiar Social Processes Robert Ervin Cramer and Robert Frank Weiss

Participation in conversation is reinforced by the opportunity to speak in reply. People will learn an instrumental response, the sole reinforcement for which is the deliverance of another human being from suffering. Increasing or decreasing N-opponents in a competitive situation facilitates learning an instrumental response. Attitudinal agreements are less reinforcing from a person who, as a result of the agreements, is increasingly more attractive. And a supervisor will rate a new worker’s causal agency for high productivity lower if a consistently productive worker also is present. These fascinating relationships predicted and discovered in our speaking in reply, altruism, competition, interpersonal attraction, and causal relationship detection research, respectively, illustrate the power of learning theory for illuminating social process. A great body of research with roots in the work of Thorndike and Pavlov, and in the Hull-Miller-Spence tradition, informed and guided the social psychological experiments described in this chapter.

With human social behavior, psychology provides the principles of learning and of innate drives, cues, rewards, and responses. The other social sciences, such as sociology and social anthropology, describe the conditions of learning— or in other words, the location of the rewards, punishments, and other conditions of the social maze. One must know both the psychological principles and the social conditions in order to predict human behavior. —Neal Miller (1959)

INTRODUCTION Do learning principles apply only to rats and neurotics? Many psychologists continue to think so, despite the impressive gains of recent years in the learning-theoretical study of human behavior. Indeed, it is not so long since some rat runners, intimidated by the propagandists of (central processing unit) “cognitive revolution” could be heard to say, “I don’t care if it only applies to

animals and abnormal behavior, it’s what I do and I like it.” We ourselves ought not to accept a dismissive attitude toward learning-based clinical research, not only because of its practical importance but because what applies to abnormal behavior largely applies ipso facto to normal behavior. Together with others in this volume, we seek to convey a more expansive view of the reach of learning principles. Our own work extends conditioning principles to the realm of complex social behavior, and we will here show in detail how instrumental and Pavlovian principles govern social process in altruism, competition, interpersonal communication, social attraction, and the role of human agency in causal relationship detection. For each of these five social domains we have constructed a theory, employing a method common in physics by which a relatively wellknown body of knowledge (here Pavlovian or instrumental conditioning) is used by analogy as 417


2/16/2011 6:50:45 PM

418


a model to predict one of our less-well-developed social domains. Analogy runs deep: For each theory we devise an artificial social structure (experimental apparatus and procedure) to the blueprint of instrumental or Pavlovian conditioning. To discover principles governing social cues we use the laws of Pavlovian conditioning (with particular emphasis on compound cues) jointly with Rescorla-Wagner theory as models to predict two of the five social domains. To similarly illuminate social motivation (social drives) and social reinforcement, we use discrete-trials instrumental conditioning as a model for three other domains. We now offer a brief example, just enough to give an impression of what is to come. Suppose that engaging in competition engenders an aversive drive: Then one good way to develop this supposition is by an instrumental escape conditioning model. Theoretical analogies can be developed easily if we diagram the structure of a reinforced trial, here (see Table 19.1) using the language of Miller and Dollard’s classic Social Learning and Imitation (1941). Because we are interested in the drive and reinforcement aspects of competition (if such things be), we have made the cue and response analogs as unproblematic as possible. Then, if engaging in competition is like electric shock, inducing aversive drive, the offset of competition should be reinforcing. Number of competition trials is the analog of number of (reinforced) escape conditioning trials. The time from the presentation of the (scoring-method “shift”) signal light until the participant makes the button-push provides an analog of escape response latency and its reciprocal, speed. Because the relationship between number of escape conditioning trials and speed is known to be a negatively accelerated increasing function in the conditioning model, it therefore follows that the same relationship is predicted

to hold between the analogous variables in competition. That is the relationship that our experiments show. A look at Table 19.1 invites us to examine the time between the button-push and the offset of competition as an analog of delay of reinforcement. Theory predicts that asymptotic speed of button-push should be a negatively accelerated decreasing function of delay of competition offset: That gradient was revealed by experiment. There is good reason to take N-opponents as an analog of drive intensity when participants compete as individuals (no coalitions or persecution of isolates, etc.), and speed was faster when competing against three opponents rather than one. In escape conditioning, shock can be reduced as reinforcement rather than terminated completely, and speed is then a negatively accelerated increasing function of the amount of shock reduction. Similarly, speed of the button-push is a negatively accelerated increasing function of the amount of reduction in N-opponents. To find out how far our analogies hold true, and in hope of discovering social psychological facts that are not pieces to different jigsaw puzzles, we shall use Pavlovian compound-cue and instrumental conditioning models to provide more demanding analyses of five social domains than those offered in this brief illustration.

THEORETICAL METHOD Modeling

Using a general approach termed “extension of liberalized S-R theory” by Neal Miller (1959), we draw close analogies between familiar instrumental and Pavlovian conditioning variables and variables assumed to be important in the social process under investigation. Rules of

Table 19.1 Structure of a (Reinforced) Instrumental Escape Conditioning Trial and a Competition Trial: Analogous Sequences Drive

Cue

Response

Reinforcement

Competition-Onset

Signal Lights

Button-Push

Competition-Offset

Shock-Onset

Door Up

Hurdle Jump

Shock-Offset


2/16/2011 6:50:45 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

Correspondence or a Dictionary of Analogies relate the independent and dependent variables in the conditioning model to the corresponding (analogous) independent and dependent variables in the social process to be predicted and explained. Testable implications are feasible when analogies among conditioning and social process independent and dependent variables are specified, and when the conditioning model the social variables are analogous to is specified (i.e., instrumental reward conditioning, instrumental escape conditioning, Pavlovian conditioning). Upon this construction, the functional relations holding among the variables in the conditioning model must, theoretically, hold among the corresponding social process variables (Hesse, 1966, 1974, 1980; Masterman, 1980; Oppenheimer, 1956). By systematically using instrumental and Pavlovian models and procedures for investigating social processes, it is possible to use the known principles of conditioning to determine whether analogous principles operate in the social process under investigation.

419

Analogy and Reduction

Our experiments do not duplicate natural social situations in all their richness and complexity. Rather, artificial social situations that rigorously conformed to discrete-trials instrumental and Pavlovian conditioning paradigms were created in which the operation of our social analogs of conditioning could be observed clearly under special, “pure” circumstances. Nevertheless, social processes are not assumed to represent a “higher level” of phenomena, which can only be fully explained by reducing them to a “lower level” of phenomena such as instrumental and Pavlovian conditioning. Rather, the theoretical method involved the use of a relatively well-understood model for investigating that, which was presently less well understood. Our programmatic assumptions, in fact, are commonplaces of scientific method. That is, well-constructed models and analogies helped to stimulate and guide research, and to integrate broad ranges of knowledge through an underlying set of common principles. Boundary Conditions

Models and Translations

The systematic use of conditioning models to investigate social processes does not represent the “mere translation” of one behavioral science language into another. In fact, a “mere translation” requires that a hypothesis describing a relationship among the social process variables being investigated already exists in the research literature, and the relationship was merely translated into statements involving conditioning principles. Rather, it has been our experience that conditioning models quite often generate intriguing hypotheses about relationships not found in the social psychological literature. When Cramer, Lutz, Bartell, Dragna, and Helzer (1989), for example, reported social analogs of partial reinforcement, delay of reinforcement, and intermittent shock in interpersonal communication involving a female participant, a masculine male, and an androgynous male, these reinforcing and motivational relationships did not exist in the scientific literature on sex roles.


As in all theoretical constructions, the specification of analogies between conditioning variables and variables in the social process under investigation was intended to apply within a limited range of conditions (Logan, 1959). Because our social psychological research is informed by instrumental and Pavlovian conditioning models, many of the most essential boundary conditions of our research including the use of discrete-trials procedures, conditioning of a single response, controlling and manipulating temporal parameters, varying stimulus intensities, control of competing responses, and the specification of conditioning dependent variables result from analogies with conditioning principles and procedures. Although the recognition of familiar boundary conditions may be unnecessary for learning researchers, the specification of even an illustrative list is not a trivial matter to social psychologists who assume that human thoughts, feelings, and actions result from cognitive capacities not granted to rats.

2/16/2011 6:50:45 PM

420


INSTRUMENTAL CONDITIONING ANALOGS Instrumental conditioning principles and procedures have, in our laboratories, been extended to social processes, including altruism (e.g., Weiss, Buchanan, Altstatt, & Lombardo, 1971), competition (e.g., Steigleder, Weiss, Cramer, & Feinberg, 1978), interpersonal attraction (e.g., Lombardo, Libkuman, & Weiss, 1972), interpersonal communication (e.g., Weiss, Lombardo, Warren, & Kelley, 1971), nonconformity (Seybert & Weiss, 1974), sex roles (Cramer et al., 1989), and social facilitation (Weiss & Miller, 1971). The experimental paradigm corresponded to discrete-trials instrumental conditioning involving drive, cue, response, and reinforcement (Miller & Dollard, 1941). For example, in the Cramer et al. sex role experiments a communication trial began with the onset of a putative noxious social stimulus (drive): communication with a masculine male. The participant learned, upon presentation of a cue, to make an instrumental response (IR, switch-press) that was followed by the contingent opportunity to listen to an androgynous male (reinforcement). By exploiting analogs of instrumental conditioning principles and procedures it has been possible to demonstrate that a number of familiar social processes, in fact, exhibit known characteristics of learning-theoretical aversive stimuli (e.g., shock) and negative reinforcers, and therefore are analogous to known aversive drives and negative reinforcement. Rules of Correspondence (Instrumental Escape Conditioning)

General rules of correspondence relating instrumental escape conditioning variables to social process variables are provided in Table 19.2. These general rules, numbered here for later reference, are illustrative rather than exhaustive, but they are described in sufficient detail to illuminate our interpersonal communication, altruism, and competition research. Interpersonal Communication: Speaking in Reply

For people who enjoy a spirited discussion with its normal “give and take,” it is reasonable to


assume that an opportunity to put in their “two cents” when another person disagrees with their opinions is positively reinforcing. In such a case people find a spirited discussion a pleasant pastime. This insight, together with the (intuitive) resemblance of the reply to a goal response (RG), was a key element in the stock of pretheoretical ideas that gave rise to the appetitive first version of the theory. If disagreements are noxious, then the opportunity to reply could be, arguably, negatively reinforcing, and the reply might even intuitively be regarded as a coping-response that inhibits aversive drive (e.g., McAllister, McAllister, & Benton, 1983; Miller & Dollard, 1941, p. 60; Mowrer & Viek, 1948). Enjoyable spirited discussion in the lab led us to entertain both instrumental reward and escape models, and experiments resolved the question. General Method

On each interpersonal communication trial, participants first listened to another person disagree with their opinion on a preselected topic of general interest, and then, upon presentation of a cue, threw a switch, the reinforcement for which was the opportunity to speak in reply to the other person. To mask the conditioning contingencies, the experiment was described as an opinion change study and opinion change was duly measured. Nonverbal communication effects were controlled by having the “other person,” participant, and experimenter sit in separate rooms, with the instructions and all comments taking place via an intercom system. Although the “other person” was, in fact, simulated by tape recordings, the participant could hear the experimenter giving the “other person” instructions and an occasional request to speak louder. An invariant cycle of events masked the experiment’s discrete-trials nature. A communication cycle began with participants listening to another person disagreeing with their opinion for approximately 20 s. When the “other person” finished commenting, the participant received the cue “throw switch if you wish to comment” and threw the “comment” switch (IR). The IR was measured using a latency timer. On the reinforced trials the participant received a “talk”

2/16/2011 6:50:45 PM


421

Table 19.2 Rules of Correspondence Relating Instrumental Escape Conditioning Variables and Social Process Variables Rule

Escape Conditioning (Model)

Social Process (Analog)

I-1

N Reinforced trials

N trials instrumental response analog (IR) eliminates/reduces noxious social stimulus

I-2

N Nonreinforced trials

N trials IR does not eliminate/reduce noxious social stimulus

I-3

Partial reinforcement

IR sometimes does and sometimes does not eliminate/reduce noxious social stimulus

I-4

Extinction

N I-2 trials following series of I-1 trials

I-5

Omission of noxious stimulus (drive)

Omission of noxious social stimulus

I-6

Intermittent shock

Mixture of trials with and without noxious social stimulus

I-7

Delay of reinforcement

Time interval between IR and the reinforcing elimination/reduction of noxious social stimulus

I-8

Magnitude of reinforcement

Quantity of reduction of noxious social stimulus

I-9

Magnitude of reinforcement

Quality of reduction of noxious social stimulus

I-10

Correlated reinforcement

Elimination/reduction of noxious social stimulus contingent on slow IR speed

I-11

Intensity of noxious stimulus (drive)

Intensity of noxious social stimulus

I-12

Speed of instrumental response

Speed of instrumental response analog

signal and spoke in reply. Because interrupting was likely to be punishing (Mandler & Watson, 1966), the participants were allowed to exceed a recommended 20 s speaking time. On the nonreinforced trials the “talk” signal was not illuminated and the participant waited 20 s. Speaking in reply (reinforcement) was contingent on performing the IR, with response speed being the dependent variable. At the end of a conversation cycle, participants performed the masking task by registering their opinion change. Reinforcing Effects of Speaking in Reply Delay of Reinforcement Analog

In instrumental conditioning, response speed is faster with short delays of reinforcement than with long delays of reinforcement (Fowler &


Trapold, 1962; see also McAllister & McAllister, 1992; Tarpy & Koster, 1970; Tarpy & Sawabini, 1974). If, after listening to another person disagree, the opportunity to speak in reply functions as reinforcement for the switch-throwing response, then the time interval between the response and the opportunity to reply should have functional properties of delay of reinforcement (Rules of Correspondence I-1, I-7, and I-12). Weiss, Lombardo et al. (1971) reported that response speeds increased as a function of the number of reinforced trials, that immediate reinforcement produced faster responding than delayed reinforcement, and that a multiplicative relationship existed between the delay of reinforcement and the number of conditioning trials. A six-group parametric study revealed a perfectly monotonic analog of a delay of reinforcement

2/16/2011 6:50:45 PM

422


gradient (Fig. 19.1; Weiss, Boyer, Colwick, & Moran, 1971). In a 2x2 study of delay shifts, speed was once again faster when the opportunity to speak in reply was immediate rather than delayed. Participants shifted from short-tolong delay matched speeds with the constant long-delay controls, while participants shifted from long-to-short delay matched speeds with constant short-delay controls (Weiss, Steigleder, Cramer, & Feinberg, 1977). Correlated Reinforcement Analog

In Logan’s correlated reinforcement procedure (discrete-trials DRL), reinforcement is made contingent upon the subject responding slower than an established cutoff value (Logan, 1960, 1961; escape conditioning, Bower, 1960). Response speed increases over conditioning trials until the cutoff is exceeded and reinforcement stops, following which speed decreases until it stabilizes just below the value of the cutoff. Figure 19.2 shows interpersonal communication effects that have an almost eerie similarity to correlated reinforcement effects in instrumental

conditioning (Rule I-10). A yoked control group receiving the same sequence of reinforcement as the correlated reinforcement group continued to improve after its experimental counterpart began to match the cutoff (Weiss, Boyer et al., 1971). We have replicated correlated reinforcement effects (Weiss, Cluts, Williams, & Miller, 1977). Observing correlated reinforcement effects in interpersonal communication, or in any other social process, is not a trivial matter because it shows that the effects were not due to the simple development of motor skill as a function of practice. Both the correlated reinforcement group and the yoked controls received the same amount of practice, the same number of reinforcements and nonreinforcements, and even the same sequence of reinforcements and nonreinforcements. The same design logic obtains in investigations of correlated delay of reinforcement. If the participant responds faster than the cutoff value, then the replyreinforcement is delayed (rather than omitted)

100

90

90 Speed (100/Latency)

85 80 Speed (100/Latency)

NON-CORRELATED

75 70

80

70 - - - - - - - - - - - - - - - - - - - - - - - CUTOFF - 60 CORRELATED

65 50 60 40

55 50

1 2 0

3

6

9 15 Delay (s)

21

2 3

3 4

4 5

5 6

6 7

7 8

8 9 10 11 9 10 11 12

Rolling Blocks of Two Trials

Figure 19.2 Correlated reinforcement analog: Figure 19.1 Delay

of reinforcement analog: Response speed as a function of delay of the reinforcing opportunity to speak in reply. (Redrawn from Weiss, Boyer, Colwick, & Moran, 1971).


Acquisition curves of response speed under correlated (experimental group) and non-correlated (yoked control group) opportunity to speak in reply. (Redrawn from Weiss, Boyer et al., 1971).

2/16/2011 6:50:45 PM


and participants learn to match the cutoff value just as do Logan’s rats (Logan, 1960; Weiss, Beck, & Stich, 1972). The yoked controls acquire the response under variable delay. The acquisition, delay of reinforcement, and correlated reinforcement effects found in our interpersonal communication experiments reported thus far are compatible with either an instrumental reward conditioning or an instrumental escape conditioning model. By developing additional analogs of instrumental conditioning principles and procedures, we were able to answer the question, “Is speaking in reply analogous to positive reinforcement or negative reinforcement?” Partial Reinforcement Analog

In instrumental reward conditioning, asymptotic response speed is faster under partial than it is under continuous reinforcement (e.g., Amsel, 1992; Capaldi, 1978; Goodrich, 1959). But in escape conditioning this pattern is reversed: partial slower than continuous (e.g., Bower, 1960; Gray, 1982; Woods, Markman, Lynch, & Stokely, 1972). In both paradigms, resistance to extinction is generally superior following acquisition with (say, 50%) partial reinforcement. If the opportunity to speak in reply occurs on only half the trials, it should then have the functional properties of partial reinforcement (Rule I-3). Weiss, Lombardo et al. (1971) obtained extinction effects typical of instrumental conditioning with gradually decreasing response speeds over extinction trials (Rule I-4) and greater resistance to extinction for the partial group. They discovered that acquisition speeds were analogous to those of escape conditioning: partial slower than continuous. These acquisition results first led us to think that we were dealing with an analog of negative reinforcement, so we conducted an experiment with extended acquisition trials that replicated the slower speed of demonstrably asymptotic partials. Disagreement-Induced Drive Drive Intensity Analog

In escape conditioning, response speed is an increasing function of the intensity of the


423

noxious stimulus (drive) when the drive is terminated completely after each escape (Franchina, 1969b; Trapold & Fowler, 1960). However, in their classic experiment, Campbell and Kraeling (1953) showed that when the amount of reinforcing drive reduction is held constant, speed is a decreasing function of the intensity of the noxious stimulus (see also Campbell, 1968; McAllister & McAllister, 1992; McAllister et al., 1983; Myers, 1969). An analog of Campbell and Kraeling effects was obtained by varying the formidability with which disagreements were set forth. While the formidable disagreements were superior to our standard disagreements in their logic, clarity, and factual support (drive intensity; Rule I-11; also Byrne, 1971), they did not generate differences in either the duration or quality of the participant’s reply (constant drive reduction). If the reply is viewed as a coping response that inhibits aversive drive (e.g., McAllister et al., 1983; Miller & Dollard, 1941, p. 60; Mowrer & Viek, 1948), then standard disagreements were all that our participants could cope with. As in conditioning, escape speed was actually faster for low drive (standard disagreement) than for high drive (formidable disagreement; Weiss, Lombardo et al., 1971). Energization Analog

Because attitudinal disagreements function as noxious stimuli, they should produce effects analogous to energization by irrelevant drive (Lombardo, Libkuman et al., 1972). To test this possibility, participants were given an opportunity to make the initial comment on topics of low and high interest. Participants, depending on treatment group, were then either disagreed with on high-interest topics and agreed with on low-interest topics (high drive analog; Rule I-11) or agreed with on high-interest topics and disagreed with on low-interest topics (low drive analog). Following the drive arousing interpersonal communication phase of the experiment, the participants immediately performed a pairedassociates task (Spence, Farber, & McFann, 1956). Verbal and nonverbal paired-associates tasks have been used because of their success in experiments studying the energization properties of aversive drives: manifest anxiety (Spence &

2/16/2011 6:50:45 PM

424


Spence, 1966), frustration (Schmeck, 1970), conflict (Castaneda, 1965), time stress (Castaneda & Lipsitt, 1959), cognitive dissonance (Waterman, 1969), audience observation (Cottrell, 1968), and unsolvable-uncontrollable intellectual tasks (Feinberg, Miller, Weiss, Steigleder, & Lombardo, 1982, Exp. 6). As predicted from a modest expansion of our theory and Hull-Miller-Spence learning theory for aversive drives (Dollard & Miller, 1950; Hull, 1943; Logan, 1959; Spence, 1956), the energizing effects of disagreement-induced drive interacted with the competitive and noncompetitive paired-associates lists. In terms of the number of errors made and trials to criterion, the performance of the high-drive group compared to the low-drive group was enhanced on the noncompetitive list and impaired on the competitive list. In an unpublished study in Weiss’ laboratory, we conditioned a learned drive by pairing a conditioned stimulus (CS) analog with the unconditioned stimulus (US) of disagreement. The CS analog acquired the ability to energize both correct and erroneous associations in the verbal paired-associates lists, with effects on errors similar to those reported earlier. Because all known learned drives are based on aversive primary drives (US) such as pain, frustration, and nausea (Brown & Farber, 1968, indispensable for its review of numerous unpublished attempts to condition appetitive drive; Mineka, 1975; Mongeluzi, Rosellini, Caldarone, Stock, & Abrahamsen, 1996, and its escape companion, Van Sommers, 1963), these results imply that disagreement-induced drive is aversive. Intermittent Shock Analog

In intermittent shock experiments, the shock is omitted on some trials, with the subjects continuing to respond with fear-motivated escape behavior on the nonshock trials. An analog of intermittent shock was achieved by having the “other person” state a disagreeing opinion on shock trials only, offering no opinion on nonshock trials (Rule I-1, I-5 and I-6). Results too were analogous to conditioning: Participants disagreed with at the outset of only half the trials still performed the IR, but more slowly than did participants disagreed with at the outset of all


trials (Weiss, Miller, Steigleder, & Denton, 1977; Weiss, Williams, & Miller, 1972; escape conditioning, Franchina, 1966, 1969a). The manipulation permits us to specify the locus of the drive in the disagreeing opinion. There is no fully corresponding manipulation in instrumental reward conditioning, not least because of the fear-motivated escape on nonshock trials, and the fact that learned drives are aversive (e.g., Brown & Farber, 1968; Mineka, 1975). Implications Logic of Modeling

In physics, theoretical analogy quite often turns out to be logically or empirically imperfect. Such disanalogy characterizes even remarkably successful and long-lived theories. Thus, (a) radio and light waves are theoretically analogous to the once better known (b) sound and water waves, except for the fact that sound and water waves require a medium in which to propagate, while radio and light waves travel unabashed through a vacuum. This was no quibble to theoretical physicists, who invented several ingenious varieties of “ether” (a couple of them very ethereal indeed) in which radio and light waves might propagate with propriety. The logic of modeling leads to extensive testing of the escape conditioning model of speaking in reply because of the problem of disanalogy. Indeed, despite some effort, we have never been able to devise a satisfactory analog of magnitude of reinforcement. The logic of modeling also leads us to conduct a program of experiments because analogy is not identity, and we cannot conclude from an acquisition curve that all delay of reinforcement effects are guaranteed in advance of experiment. Functional Significance of Speaking in Reply

A definitive difference between the monologue of mass communications and the dialogue of personal conversation is the opportunity to speak in reply. This fundamental dichotomy is reflected in laboratory research. In the laboratory monologue of persuasive communication or impression formation messages are typically directed by a communicator to a research participant

2/16/2011 6:50:45 PM


who has no overt opportunity to reply; in the laboratory dialogue of experimentation on group discussion or interviews, however, the participant has that opportunity. The study of the reply already enjoys a position of cardinal importance in dialogue research as a dependent variable. We thought it likely that if the opportunity to speak in reply were a fundamental aspect of interpersonal communication, then it might be illuminating and enjoyable to turn the problem on its head by investigating the effects of replying on the person who makes the reply. With the problem thus inverted, theory and experiment reveal a convincingly coherent and demandingly detailed blueprint of the reinforcing function of speaking in reply. Speaking in reply can now be seen to be a definitive difference between the monologue of mass communication and the dialogue of personal conversation that possesses an equally fundamental functional significance. Altruism

Much human behavior exhibits altruistic features, most dramatically in emergencies, warfare, and social movements, where group loyalties often take precedence over individual needs. An abundance of experiments on altruism delineates the circumstances under which people will help others who are in need: The focus is on the antecedents of altruistic behavior rather than on its consequences for the altruist (e.g., Staub, 1978, 1979). Developmental and some social psychologists have asked how socially constructive or altruistic behavior can be learned, generalized, and, perhaps, internalized through extrinsic reinforcement, thus viewing altruism as a response (e.g., Grusec, 1991; Grusec & Redler, 1980; Staub, Bar-Tal, Karylowski, & Reykowski, 1984). We thought it interesting to invert the question, standing the problem on its head, the better to view altruism as a reinforcer and examine the reinforcing consequences of altruism. This perspective, we hoped, might also give us a chance of further illuminating the extraordinary strength and prevalence of altruism, some of it in places that social psychologists did not think to look, including warfare and revolution.


425

General Method

To study the predicted action of altruistic reinforcement, we devised an artificial social structure (experimental apparatus and procedure) to the blueprint of instrumental conditioning. The participants were to learn, upon presentation of a cue (“report” signal), to make an instrumental response (button-push), the reinforcement for which was the deliverance of another human being from suffering (simulated shock and also bright light). The cue and IR were concealed in the masking task: Participants observed and evaluated a trained confederate who ostensibly received continuous painful electric shock while performing an aviation-related tracking task. A participant sat in a darkened room in front of an array of visual signals, knobs, and buttons deployed so as to focus the participant’s attention on a window into an adjacent lighted booth there to see another person (the confederate) wearing a “shock bracelet.” Inside the booth were “controls” for tracking, a preprogrammed radar-like screen, a blower fan, houselights sufficient for visibility, and a flood lamp placed 0.30 m from the confederate’s face. To enhance the verisimilitude of the shock manipulation, the confederate exhibited agonized expressions, “nervous” genuine sweating, and occasional verbal expressions of pain and “reflexive” kicking of the wall. Were it not for the booth’s wall, the participant and confederate could easily have touched one another. When the shock-on signal was illuminated, the participant observed the confederate’s tracking, and, upon presentation of a series of evaluation signals, set dials to evaluate the confederate’s performance (masking only). Upon completion, participants then received the “report” signal (cue) and pressed, in sequence, six “report buttons.” A latency timer automatically measured the time from the cue to the first buttonpush (IR). Altruistic reinforcement immediately followed the sixth button-push: shock-offset ([a] “shock-on” signal offsets and [b] shock-off signal onsets), (c) the confederate breathes a sigh of relief with the receipt of a 10 s break from the shock, (d) the sweat-inducing flood lamp offsets,

2/16/2011 6:50:45 PM

426


and (e) the efficient blower fan onsets, visibly drying the “nervous” sweat. Because the houselights remained on in a darkened room, the reinforcing events were clearly visible. Altruistic Reinforcement Analogs of Acquisition, Partial and Delay of Reinforcement

People will learn an instrumental response, the reward for which is the deliverance of another human being from suffering. Moreover, the functional relationships found using altruistic reinforcement were analogous to those governing the effects of conventional reinforcement in instrumental conditioning. Speeds of responses delivering the confederate from suffering gradually increased over the course of our analog of trials (Rule I-1 and I-12); steadily diverged from no-reinforcement controls over trials (Rule I-2); were faster when deliverance from suffering was immediate rather than delayed (Rule I-7) and continuous rather than partial (Rule I-3; Weiss, Boyer, Lombardo, & Stich, 1973; Weiss, Buchanan et al., 1971). There is a decreasing negatively accelerated delay gradient. The delay gradient is steep, probably indicating that the threshold value for effective reinforcement (e.g., Campbell, 1955; Tarpy, 1969) is quite high (Weiss, Cecil, & Frank, 1973). Magnitude of Reinforcement Analog

If altruism is reinforcing, then the degree to which the participant is able to alleviate the other person’s suffering should have the functional properties of magnitude of reinforcement. In escape conditioning, for any given level of drive, response speed is an increasing function of magnitude of reinforcement (Rule I-8; e.g., Bower, Fowler, & Trapold, 1959; Campbell & Kraeling, 1953; McAllister, McAllister, Brooks, & Goldman, 1972; Woods & Holland, 1966). Weiss, Boyer et al. (1973) manipulated magnitude of altruistic reinforcement using three levels of shock and flood lamp stress reduction. The high and zero magnitudes of altruistic reinforcement were complete reduction of shock and flood lamp stress (reinforcement) and no reduction (no reinforcement). The medium


magnitude consisted of reducing the shock the confederate experienced but not terminating it altogether, and dimming the flood lamp but not turning it off. Pilot studies made clear that manipulating a medium magnitude of altruistic reinforcement would require extraordinary care because, to participants incomplete reduction in shock, flood lamp effects and confederate’s suffering tend to resemble zero magnitude. Two boundary conditions were revealed: (1) there is a high threshold value for effective reinforcement, so medium reinforcement must be substantial (e.g., Campbell, 1955; Tarpy, 1969); (2) the reduction in suffering must be made very clearly discriminable. To meet the boundary conditions, participants first witnessed a “shock calibration procedure” wherein the confederate’s predetermined responses established a reduced shock level defined as “uncomfortable but not painful.” During medium reinforcement the confederate acted in accordance with experiencing a reduced shock level that, albeit uncomfortable, was not painful, and a dimmer flood lamp permitted a slower and sometime incomplete drying of the confederate’s sweat. Second, the standard shock-on and shock-off signals were, for the first time, buttressed by auditory feedback. Rigorous attention to boundary conditions enabled us to carve nature at the joint. Figure 19.3 shows striking correspondences between magnitude of reinforcement in instrumental escape conditioning and magnitude of altruistic reinforcement (Weiss, Boyer et al., 1973). Altruistic Drive Intermittent Shock Analog

In intermittent shock experiments, the shock is omitted on some trials, with the subject continuing to respond with fear-motivated escape behavior on the shock-free trials. An analog of such a manipulation in our altruism experiments involved omitting shock and flood lamp stress to the confederate on some trials (Rule I-6; Weiss, Boyer et al., 1973). In escape conditioning, response speed is a decreasing function of the percentage of trials on which shock is omitted (Franchina, 1966, 1969a). By analogy, the speed

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES HIGH 130

Speed (100/Latency)

120 110 100

MEDIUM

confederate’s shock- and flood lamp-induced suffering. The present results are clearly aversive, without carrying any necessary implication that all altruistic behavior is aversively motivated: The experimental situation itself fairly reeks of pain and stress. Implications Coign of Vantage

90 80

ZERO

70 60 50 1 2 3 4 5 6 7 8 9 10 11 12 13 2 3 4 5 6 7 8 9 10 11 12 13 14 Rolling Blocks of Two Trials

Figure 19.3 Magnitude of reinforcement analog:

Acquisition curves of response speed under different magnitudes of altruistic reinforcement. High = complete reduction of shock and flood lamp stress; Medium = reduction of shock but not terminating it, and dimming of flood lamp but not turning it off; Zero = no reduction of shock and flood lamp stress. (Redrawn from Weiss, Boyer, Lombardo, & Stich, 1973).

of altruistically reinforced responses should be a decreasing function of the percentage of trials on which “shock” and flood lamp stress is omitted. Intermittent shock effects in altruistic learning were analogous to those found in escape conditioning, with response speeds being faster when shock was “administered” to the confederate on all of the trials than it was when shock was given on only one-third of the trials. Because the 33% shock group received altruistic reinforcement on one-third of the trials, participants in this group, as expected, responded faster than no-reinforcement controls. The rate of learning under 33% “shock” suggests that responding on shock-free trials was motivated by secondary altruistic drive (learned on the shock trials) and reinforced by reduction of the secondary altruistic drive. Our analog of intermittent shock pinpoints the source of altruistic drive in the


427

Since the early days of social and developmental psychology, much interest has been shown in the question of how socially constructive or altruistic behavior can be learned and maintained through extrinsic reinforcement. In sharp contrast, our research demonstrates that instrumental behavior can be learned and maintained through the reinforcing function of altruism. This inversion-in-fact stems, in part, from the initial inversion-in-theory when first we proposed to stand the question on its head and examine altruism as a reinforcer rather than as a response. If this theoretical curiosity is likely to hold some charm for learning psychologists, then so too may some curious aspects of experimental apparatus and procedure. For clarity’s sake one of the two experimenters running a participant has always been called the confederate, but it can now be seen that we have an experimenter (the confederate) in the box (booth) and the subject outside the box! The subject is motivated by “shock” administered to an experimenter and reinforced by “offset of shock” to that experimenter. This is, as they say, “no accident, comrade.” It was part of the fun in designing apparatus and procedure to fit the theory. Failures of Altruism

Precisely because altruism is ubiquitous in emergencies, warfare, and social movements, these events afford some of the most vivid and haunting instances of its failure. Just as defection is of major concern in the social psychology of social movements (e.g., Weiss, 1963), so “leaving the scene” is one of the principal problems addressed in the experimental study of altruism (e.g., Piliavin, Dovidio, Gaertner, & Clark, 1981). Emergency intervention and leaving the scene

2/16/2011 6:50:46 PM

428


are typically studied in situations in which people choose between those two responses, thus violating the boundary conditions of conditioning and discouraging any analog of forced trials in a T-maze. Moreover, experiments on leaving the scene focus on a single, crucial, episode rather than on a series of such episodes or “trials.” At a more molecular level, it is difficult to see how one might control an analog of (say) delay of reinforcement on each trial/episode. But if we stand the problem on its head so that the scene leaves the participant, then the problem simplifies nicely as instrumental conditioning. In the leaving-the-scene condition, the buttonpush caused the temporary disappearance of the still-suffering confederate behind an opaque and soundproof shutter, thereby permitting a series of discrete trials with delay, magnitude, and schedule of reinforcement controlled. Leaving the scene proved to be weakly, almost trivially, reinforcing, reaching asymptote on the second block of trials, ending up just a little above the controls and far below the altruistic reinforcement group (Stich, Weiss, Cramer, & Feinberg, 1987). Replication

The delay effect has been replicated and the fundamental effect of acquisition under altruistic reinforcement very well so. Altruistic drive and reinforcement effects apply to males and females in equal measure. This is a typical finding in our research (Weiss, Weiss, Wenninger, & Balling, 1981). There are doubtless gender differences in socialization, but we study underlying processes (drive, cue, reinforcement) while conduct is both more targeted by parents and sometimes more malleable (e.g., than primary drives). The discovery of altruistic rewards reveals a profound similarity between altruistic and conventional, nonaltruistic drives and reinforcers. And it had never previously been demonstrated that the roots of altruism are so deep that people not only help others but find doing so rewarding.

major social sciences according these processes considerable interest. If, like the sociologist Georg Simmel (1955), we view competition as a form of social conflict, then there is a consequent increase in the breadth and vigor of interest. Competition and cooperation have typically been viewed by psychologists as mere dependent variables, as behaviors, and even in the hands of some developmentalists, as responses to be learned. For the third and last time in this set of five theories, we will invert the question to examine the consequences of competition for the competitor: Engaging in competition engenders aversive drive. Turning the question on its head yielded a unique perspective in the case of functions of speaking in reply and of altruistic rewards. Other psychologists have attributed motivational properties to competition (e.g., Church, 1962; Cronbach, 1963; Shaw, 1958). This research began with a complimentary pair of instrumental reward and escape conditioning models, nicely separated by a clear boundary condition, but only the escape model survived and prospered. This report is written in light of what we learned by affording full opportunity for the logic of modeling and ideas from sociology to realize their full potential. General Method

Participants played a competitive game of Labyrinth with one or more opponents, and then, upon presentation of a cue performed the IR, the reinforcement for which was the termination (or reduction) of competition. The experiments were described as studies of the effects of different scoring methods on competitive behavior. This masking provided the procedural flexibility necessary for testing theoretically exciting predictions regarding competitive drive and reinforcement by manipulating (a) the N-opponents the participant initially competed against, and (b) the N-opponents the participant continued to compete against after performing the IR.

Competition

Constant N-Opponents

Competition and cooperation have captivated social theorists for decades, with each of the

The cue and IR were concealed in the masking task: One of two competitors, the data-generating


2/16/2011 6:50:46 PM


participant, made the IR that provided a measure of response speed by ostensibly resetting the machine performing “real-time scoring and analysis.” Because of the machine’s “limited capacity,” point tallying, and therefore competition, could not occur during the reset period (reinforcement). Each competitor played the game in a separate, partitioned section of the laboratory in front of a control panel that included an array of visual signals, the manipulandum, and headphones. Verbal feedback, and especially visual feedback consisting of control panel lights and different colored lights (e.g., white, red, blue) mounted beneath milk glass and the translucent game board, made the critical segments of a competition trial easily discernable. Without needing to take their eyes off the game, participants could effortlessly distinguish between the (a) aversive competition-on period (shock-box), (b) cue, and (c) reinforcing competition-free period (goal-box). Each trial began with a 45 s competition-on period, following which the participant received the “reset/end-tallying” signal (cue) and performed the “reset/end-tallying” IR. A latency timer automatically measured the time from the cue to the IR which (a) lit the “machine reset-no scoring” lights on the control panel and below the game board indicating that scoring was no longer occurring, and (b) initiated a reinforcing 20 s competition-free, no-scoring period. During the competition-free, no-scoring period, competitors were instructed to continue practice-playing the game: Competition offset was not confounded with task offset. Following reinforcement the next competition trial began. Varying N-Opponents

As in the constant N-opponents experiments, the experimenter kept the participant informed of the number of opponents also playing the game during the initial competition-on period (e.g., 45 s). The number of people sharing the waiting room, the apparatus, instructions, and procedure, and the physical arrangement of the laboratory, were carefully crafted to create, for the participant, the “reality” of a competitive situation with multiple opponents. For example,


429

the data-generating participant was led past several partitioned sections of the laboratory each outfitted with the game, headphones, and a control panel. The masking task provided a plausible rationale for reducing the N-opponents the participants continued to compete against following the IR. To test different scoring methods, a signaled shift from one method to another would be necessary during the cycle of operation. The data-generating participant, upon presentation of the “shift” signal (cue), pushed the “removebutton” (IR), thereby providing a measure of response speed and reducing the N-opponents. Following the IR, verbal feedback indicated that scoring methods had shifted, and if competition continued, how many opponents the participant would continue to compete against. If all of the opponents were removed, participants experienced a reinforcing 20 s competition-free, practice-play period. If fewer than all of the opponents were removed, a 20 s competition-on period followed with a reduced N-opponents. All of the participants had a programmed 13 s rest period separating the competition trials. Again, verbal feedback specified N-opponents before and after pushing the remove-button. The verbal feedback assisted control panel lights and colored lights mounted beneath milk glass and the translucent game board in making each critical segment of a competition trial easily discernable. Without needing to take their eyes off the game, the (a) aversive competition-on period (shock-box), (b) cue, (c) reinforcing reduction in the N-opponents in the goal-box, and (d) intertrial interval were readily apparent to the participants as (say) blue lights softly illuminated the maze from below. Constant N-Opponents Delay of Reinforcement Analog

If the offset of competition is analogous to reinforcement, then delaying that offset is analogous to delay of reinforcement (Rule I-7). Figure 19.4 shows an elegant experimental analog of a delay of reinforcement gradient (Steigleder et al., 1978), negatively accelerated in shape just like rat-generated curves (Rule I-12; Fowler &

2/16/2011 6:50:46 PM

430


analog in the duration of the competition-free period (Rule I-8). After the IR, the participants were free from competition either for 10, 20, or 30 s before the next cycle began (Steigleder et al., 1978). The competitive and competition-free periods were very thoroughly demarcated, in a manner charmingly similar to Franchina’s hurdle-box. Response speeds were a monotonic increasing function of the duration of the competition-free period, as the escape model predicts. The longer shock-free period permits more relaxation and more conditioning of approach to the cues of the competition-free period. The effects of duration of the competition-free period replicate nicely.

Speed (100/Latency)

80

70

60

50

40

0

1

3

5

Delay (s)

Figure 19.4 Delay

of reinforcement analog: Response speed as a function of delay of the reinforcing offset of competition. (Redrawn from Steigleder, Weiss, Cramer, & Feinberg, 1978).

Trapold, 1962; see also McAllister & McAllister, 1992; Tarpy & Koster, 1970). Partial Reinforcement Analog

If the termination of competition functions as reinforcement, then the termination of competition on only half the trials should have the functional properties of partial reinforcement (Rule I-3). Partial reinforcement facilitates response speed in reward conditioning (e.g., Amsel, 1992; Capaldi, 1978; Goodrich, 1959) but impairs it in escape conditioning (e.g., Bower, 1960; Gray, 1982; Woods et al., 1972). Experimental results fit the escape pattern, indicating that engaging in competition engenders an aversive drive (Steigleder et al., 1978). Analog of Duration of the Shock-Free Period

A curious variable, well researched in avoidance conditioning under the guidance of relaxation theory (Denny, 1971, 1991) but also studied in escape conditioning as part of Franchina’s valuable effort to isolate the escape component of avoidance (Franchina, Kash, Reeder, & Sheets, 1978; Franchina & Schindele, 1975), the duration of the shock-free period finds its social


Intermittent Shock Analog

In intermittent shock experiments, the shock is omitted on some trials, with the subject continuing to respond with fear-motivated escape behavior on the nonshock trials. We had our participants compete either on 33%, 66%, or 100% of the trials (Rule I-6) and found that their speed was a monotonic increasing function of the percentage of competitive trials (Steigleder, Weiss, Balling, Wenninger, & Lombardo, 1980). The acquisition curves paralleled the intermittent shock data of Franchina (1966) in almost uncanny detail and provide further evidence for the aversiveness of competitive drive. Competence as a Drive Variable

Fear of situational stimuli is a decreasing function of mastery of a two-way avoidance task (McAllister & McAllister, 1991; McAllister et al., 1983). Similarly, audience-induced drive is a decreasing function of competence in humans (Cottrell, Rittle & Wack, 1967; Cramer, McMaster, Bartell, & Dragna, 1988). If competition engenders an aversive drive, it then seems likely that competent participants would find competition less aversive. The analogy is reasonable and interesting, if not strictly entailed by the escape conditioning model. The experiment proceeded in two phases: competence induction followed by escape conditioning. Competence induction was not confounded with the experience of success or failure (Luginbuhl, 1972). Both competent and

2/16/2011 6:50:46 PM


incompetent participants gradually acquired the IR over the course of trials. The two curves gradually diverge, with the competents reaching the lower asymptote and the incompetents the higher, in a classic Drive x Trials interaction. We have replicated the drive effects of competence (Steigleder et al., 1978). Learned Drive

If engaging in competition engenders aversive drive, then stimuli appropriately associated with this aversive drive (US) should themselves acquire the capacity to elicit a learned drive. This holds, whether the drive induced by competing is primary or secondary, since there is ample evidence for second-order conditioning of drives (e.g., Anderson, Johnson, & Kempton, 1969; Marlin, 1983; McAllister & McAllister, 1964; Rescorla, 1980). We would have preferred a nonsocial CS such as a light or tone, but since this CS must appear in both the competitive phase and the testing phase the requirements of plausibility led to using a person as the CS and, in consequence, our design was fully controlled for audience effects. Drive conditioned to a stimulus associated with competition energized performance of associations on the Spence, Farber, and McFann paired-associates lists (Spence et al., 1956; Steigleder et al., 1980). Because all known learned drives are based on aversive primary drives (US) such as pain, frustration, and nausea (e.g., Brown & Farber, 1968; Mineka, 1975), these results imply that the drive induced by competition is aversive. Aversive or Appetitive

Let us briefly gather the evidence now, the better to next focus on the strange phenomena and intriguing implications encountered when we vary the N-opponents. We will not need to rehearse the conditioning references already given at the individual variables and at “the functions of speaking in reply.” Asymptotic partial reinforcement curves followed the escape (continuous faster than partial) rather than reward (partial faster than continuous) pattern. Excellent intermittent shock (intermittent competition) results were obtained, there being no satisfying


431

analog of this in reward conditioning especially because of the implied learned drive. We did not settle for an implied learned drive but conducted a separate experiment in which we conditioned an analog of a learned drive. All known learned drives are based on aversive primary drives (or on aversive learned drives via second-order conditioning). Those experiments provide the definitive evidence, but we have in addition the experiments on competence and the analog of the duration of the shock-free period. Competent participants had slower speeds but continued to escape from competition. We actually considered the possibility that a competence/incompetence manipulation might be a boundary condition separating a reward conditioning model from the escape conditioning model, but this potentially beautiful development did not come to pass. The predicted effect of duration of the competition-free period does not directly imply the aversiveness of competition-induced drive, but, given the evidence already provided, this analog has something of an amplifying effect, with our participants relaxing in the “shock-free goal-box.” In reaching a clear-headed conclusion, it is well to consider not only the experimental results but also (to some extent) the atmosphere of the experimental situation which, for the altruism research, led us to limit our conclusions because the experimental situation “fairly reeks of pain and stress.” We obviously do not have such a problem here, where the experimental task is the game Labyrinth, which is manufactured commercially and played recreationally and where participants do not experience failure. The drive induced by competition is aversive. Varying N-Opponents

The variable “number of people” is frequently manipulated when social processes such as social judgment (Knowles, 1983), collective action and social movements (Macy, 1990; Marwell & Oliver, 1993), diffusion of social impact (Latané & Bourgeois, 1996; Seta & Seta, 1996), family influence (Zajonc, 1976; Zajonc & Mullally, 1997), cognitive dissonance (Festinger, 1957), and social facilitation (Crisson, Seta, & Seta,

2/16/2011 6:50:46 PM


1995; Weiss & Miller, 1971; Zajonc, 1965) are investigated. Social facilitation research suggests that number of people can function as a drive variable (Geen & Gange, 1977; Martens & Landers, 1972; Weiss & Miller, 1971). We take N-opponents as an analog of drive-intensity when participants compete as individuals, without alliances or power blocs. N-opponents, like number of trials, is a discrete variable that can mimic a continuous variable when N is reasonably large. In conditioning research, drive is usually manipulated as a continuous variable (in volts, milliamps, hours-of-deprivation, etc.) but is also certainly manipulated (e.g., shock + noise) as an inherently discrete variable (e.g., Campbell, 1968; McAllister & McAllister, 1992). Drive Intensity Analog

In escape conditioning, speed is an increasing function of drive when drive is terminated after each escape (e.g., Nation, Wrather, & Mellgren, 1974: Trapold & Fowler, 1960). Analogously, when participants compete as individuals and the IR terminates competition, the theory predicts that speed is an increasing function of the N-opponents (Rule I-11). There were two levels of drive: 1 versus 3 opponents. A more inclusive description of the two groups compares 1 opponent, reduced by 1 as reinforcement, = 0 opponents remaining in the goal-box (1 – 1 = 0) with 3 opponents, reduced by 3 as reinforcement, = 0 opponents remaining in the goal-box (3 – 3 = 0). High drive was indeed faster than low (Steigleder et al., 1980). At the inception of the second series of competition studies, we thought it essential to replicate the drive effect and to test out a second complete two-competitor apparatus system. Drive intensity effects replicate nicely as shown in Figure 19.5. Drive effects also replicate cleanly across the two apparatus systems, despite the fact that the competitors were close together in one system and widely separated for the other. (All data from the second series of competition experiments are as yet unpublished studies from Weiss’ laboratory.) Drive Downshifts

There were three drive conditions: a constant high drive, 3-opponent control (3 – 3 = 0), a constant


80 3–3=0

Speed (100/Latency)

432

70

60 1–1=0

50

40

1

2

3

4

5

6

7

Trials

Figure 19.5 Shock-intensity

(Drive) analog: Acquisition curves of response speed as a function of N-opponents. Conditions: 1 – 1 = 0 (1 opponent, reduced by 1 as reinforcement, = 0 opponents remaining in the goal box) and 3 – 3 = 0 (3 opponents, reduced by 3 as reinforcement, = 0 opponents remaining in the goal box).

low drive, 1-opponent (1 – 1 = 0) control and a high-to-low drive downshift (3 – 3 = 0)/(1 – 1 = 0) condition. The high-drive controls were faster than the low-drive controls. The speeds of the downshift participants rapidly downshifted, meeting the speed of the low-drive controls within two trials, continuing down to show a negative contrast effect as in the escape conditioning model (Nation et al., 1974; Woods & Schutz, 1965). Campbell and Kraeling Analog

Campbell and Kraeling (1953) showed that when the amount of reinforcing shock reduction is held constant, speed is a decreasing function of shock intensity. Equivalent results have been shown with different drive manipulations (e.g., McAllister & McAllister, 1992; Myers, 1969). McAllister and McAllister do not directly address this question, but their virtuoso experimental manipulations produced this effect in several different ways and their data may be easily

2/16/2011 6:50:46 PM


regraphed to show it. McAllister and McAllister (personal communication) agree with this analysis. So now we need another analogy that has been present in the equation labeling all along. Magnitude of reinforcement is an increasing function of the reduction in the N-opponents. The most perspicuous way to present this Campbell and Kraeling experiment is via Figure 19.6. Reinforcement is held constant at reduction of two opponents: The remove-button removes two opponents. Drive is once more varied by N-opponents, 2, 3, or 4. The figure shows a clear Campbell and Kraeling effect with speed as a decreasing function of N-opponents (drive). Notice the crucial number after the equals sign,

90 2–2=0

Speed (100/Latency)

80

3–2=1

70

433

the 2-opponent group leaves no opponents waiting for the participant in the goal-box after the participant pushes the remove-button but the 4-opponent participants must face two opponents remaining in the goal-box after they make their instrumental response! If we recall the continuity between competition and social conflict in the sociology of Georg Simmel (1955), then this is not a happy outcome. Even in his brief treatment of games Simmel uses such words as fight, victory, and kampf (which word in the very different sociology of Marx is commonly rendered as “struggle”). Acquisition Trials

The five acquisition curves in Figures 19.5 and 19.6 show gradual acquisition over the course of trials. Three of the curves show full reinforcement with no opponents remaining in the goal-box, while two others do have opponents remaining in the goal-box after the IR has been made. There are some 27 more acquisition curves not depicted here, but these should be sufficient to convey the pattern and some variations. Magnitude of Reinforcement Analog

60 4–2=2 50

40

30

1 2 3

2 3 4

3 4 5

4 5 6

5 6 7

6 7 8

7 8 9

8 9 10

Rolling Blocks of Three Trials

Figure 19.6 Campbell

and Kraeling analog: “Paradoxical” drive effects when increasing initial N-opponents and holding the reinforcing reduction in N-opponents constant. Conditions: 2 – 2 = 0 (2 opponents, reduced by 2 as reinforcement, = 0 opponents remaining in the goal box), 3 – 2 = 1 (3 opponents, reduced by 2 as reinforcement, = 1 opponent remaining in the goal box), and 4 – 2 = 2 (4 opponents, reduced by 2 as reinforcement, = 2 opponents remaining in the goal box).


Response speed is a negatively accelerated increasing function of magnitude of reinforcement, the amount of reinforcing reduction of aversive drive (e.g., Bower et al., 1959; McAllister et al., 1972; Woods & Holland, 1966). Since decreasing the N-opponents decreases drive intensity, the speed of an instrumental response that reduces the N-opponents is a negatively accelerated increasing function of the reinforcing reduction in the N-opponents (Rule I-8). After learning how to handle the variable, we essayed a 5-group parametric. Participants competed with an initial number of five opponents. Pushing the remove-button reduced the N-opponents by 1, 2, 3, 4, or 5 opponents. A description of just three cells will elucidate the experiment from a different perspective: “high” (5 – 5 = 0), 5 opponents, reduced by 5 as reinforcement, = 0 opponents remaining in the goalbox; “medium” (5 – 3 = 2), 5 opponents, reduced by 3 as reinforcement, = 2 opponents lurking in the goal-box; “low” (5 – 1 = 4), 5 opponents, reduced by 1 as reinforcement, = 4 opponents

2/16/2011 6:50:46 PM

434


Speed (100/Latency)

85 80 75 70 65 60 1

2 3 4 5 Reduction in N-Opponents

Figure 19.7 Magnitude of reinforcement analog:

Asymptotic speed as a function of reduction (R) in N-opponents. Speed = M(1 – 10 -iR) + d, where M is the asymptotic speed of the magnitude of reinforcement gradient, i is the negatively accelerated rate of change, d is the intercept, and R is the reduction in N-opponents, as fitted to individualparticipant mean speeds.

waiting in the goal-box. It was with a sense of wonder that we first saw the asymptotic magnitude of reinforcement results graphed in Figure 19.7. The asymptotic response speed of competing human beings is a negatively accelerated increasing function of the reinforcing reduction in the N-opponents. Number of People and Models

This analogical theory does not merely translate the language of competition into the language of instrumental conditioning. Rather, the systematic use of an escape conditioning model generated predictions about competitive behavior likely to be overlooked by social psychologists. As in a variety of research areas in social psychology, it would stand to reason just to manipulate the variable “number of people.” But using escape conditioning as our model, we were able to ascertain that the variable “number of people” will have three different effects, depending upon where in the cycle of operation it is manipulated. Decreasing the variable by reducing the N-opponents (reinforcement) contingent


upon the escape response will improve performance. “Paradoxically,” decreasing the initial N-opponents (drive intensity), whether between-groups or even within-subjects (from one trial to the next as in drive-downshifts), will impair performance provided that there is complete offset of competition as reinforcement. In sharp contrast, increasing the initial N-opponents (between groups) while keeping reinforcement constant will actually impair performance in a Campbell and Kraeling effect. The theory tells us where in the cycle of operation to manipulate “number of people” and to anticipate multiplicative effects in ways that did not “stand to reason” to social psychologists lacking this theoretical tool. Implications Galileo, Harvey, and Hobbes: A Nondigression

Hobbes is surprisingly comfortable reading for scientists like us, because he had an intimate knowledge of the best science of his day and used it in his own work. Hobbes acquired this knowledge not only from reading but also from his close friend Harvey, who discovered the circulation of the blood and who had been a student of Galileo at the University of Padua. All this is nicely brought out, and deepened, in a useful little book in the philosophy of science by Randall (1961). Hobbes tried to apply the theoretical method of Galileo, as it was taught to Harvey by Galileo and several other professors at the University of Padua, to society. Galileo analyzed the motions of the heavenly bodies into purely theoretical “elements” (intervening variables) in a purely theoretical space, and then, via appropriate composition rules recombined the theoretical elements so as to predict the actual motions of actual bodies in actual space. Hobbes in turn analyzed society into purely theoretical equal individuals (e.g., stripped of civilized restraint) in a theoretical space, which he called the State of Nature. “The fact that the State of Nature is a logical and not an historical hypothesis is generally understood” (Macpherson, 1962, p. 21). Hobbes then recombines these purely theoretical individuals via appropriate, but not

2/16/2011 6:50:46 PM


quantitative, composition rules to explain society empirically and normatively. Hobbes conceives humans as self-maintaining engines (we would say robots, automata) in motion, regardless of whether they are in or out of the State of Nature. The importance of motion is an insight carried over from Galileo (“nevertheless it moves”) and Harvey (the blood circulates) and Hobbes identifies it with life itself. We too carry over such scientific insights, as when Hull captured the hard-won learning-performance distinction in the form of Drive x Habit, and when the Rescorla-Wagner equation appears in one form or another, not only in learning and social learning-analog theories, but even in cognitive theories. The War of All Against All

There are, of course, no peace officers or any other protections peculiar to organized society in the State of Nature “wherein we suppose contention between men by nature equal, and able to destroy one another” (Hobbes, 1640/1984, chap. 14, No. 12, p. 73), but “warre of every man against every man” (Hobbes, 1651/1950, chap. 13, p. 105). Although we are testing our own theory, not Hobbes’s, it can now be seen that in our experiments in which N-opponents is varied, we have a laboratory analog of a war of all against all. An outstanding feature of the laboratory war is the elimination of opponents, also an outstanding feature of Hobbes’s State of Nature. “It may seem strange to some man, that has not well weighed these things; that Nature should thus disassociate, and render men apt to invade, and destroy one an other” (Hobbes, 1651/1950, chap. 13, p. 104). “From this equality of ability, ariseth equality of hope in the attaining of our Ends. And therefore if any two men desire the same thing, which neverthelesse they cannot both enjoy, they become enemies; and in the way to their End,… endeavor to destroy, or subdue one an other” (Hobbes, 1651/1950, chap. 13, p. 102). Hobbes certainly does not propose that competition induces aversive drive, but he places great emphasis upon it, even in his later works. Competition can be seen or inferred in some of the preceding quotes, but it is certainly available elsewhere. “Competition of


435

Riches, Honour, and Command or other power, enclineth to Contention, Enmity and Warre; Because the way of one Competitor, to the attaining of his desire, is to kill, subdue, supplant, or repel the other” (Hobbes, 1651/1950, chap. 11, p. 80). “But the most frequent reason why men desire to hurt each other, ariseth hence, that many men at the same time have an Appetite to the same thing; which yet very often they can neither enjoy in common, nor yet divide it; whence it followes that the strongest must have it, and who is strongest must be decided by the Sword” (Hobbes, 1647/1983, chap. 1, No. 6, p. 46). Hobbes does not assume that men are innately competitive, but competition follows ineluctably from the social environment of the State of Nature (Gauthier, 1969, pp. 14-20, 208). We have found the juxtaposition of Hobbes to our competition theory to be provocative as well as intriguing, and we intend, in the future, to offer a learning-theoretical exit from the war of all against all in place of the social contract.

PAVLOVIAN CONDITIONING ANALOGS Analogs of Pavlovian conditioning principles and procedures have been extended to the attribution of liking (Cramer, Helzer, & Mone, 1986), persuasive communication (Weiss, 1968), interpersonal attraction (Cramer, Weiss, Steigleder, & Balling, 1985), and causal relationship detection (Cramer, Weiss, William, Reid, Nieri et al., 2002). Only the last two social processes will be discussed here because the use of compound social cues and Rescorla-Wagner theory yields research that is socially rich and surprising. Rules of Correspondence (Pavlovian Conditioning)

General rules of correspondence relating Pavlovian conditioning variables to social process variables are provided in Table 19.3. In our interpersonal attraction research, attitudinal agreements were assumed to elicit approach responses (person-directed actions), whereas in our causal relationship detection research

2/16/2011 6:50:46 PM

436


Table 19.3 Rules of Correspondence Relating Pavlovian Conditioning Variables and Social Process Variables Rule

Pavlovian Conditioning (Model)

Social Process (Analog)

Fundamental Concepts P-1

Conditioned stimulus (CS; A, B, or X)

Discriminable social stimulus (a person)

P-2

Unconditioned stimulus (US)

Social stimulus reliably eliciting a response (e.g., attitudinal agreement in attraction)

P-3

Unconditioned response (UR)

Response elicited by US analog (e.g., approach in attraction)

P-4

Conditioned response (CR)

Conditioned form of UR analog

Variables P-5

N Forward trials (CS-US or A+)

N CS analog-US analog trials

P-6

N Nonreinforced/Extinction trials (A−)

N CS analog without US analog trials following series of P-5 trials

P-7

N US alone trials

N US analog without CS analog trials

P-8

N Backward trials (US-CS)

N US analog-CS analog trials

P-9

Compound CS (AX, BX)

Compound stimulus containing two or more social stimuli (two or more people)

P-10

N Forward compound CS trials (AX-US or AX+)

N Compound CS analog-US analog trials

P-11

N Backward compound CS trials (US-AX)

N US analog-compound CS analog trials

P-12

N Nonreinforced compound CS trials (BX−)

N Compound CS analog without US analog trials

P-13

CS intensity

CS analog intensity

P-14

US intensity

US analog power to elicit response (e.g., agreement strength)

P-15

Positive CS/US contingency

p(US analog/CS analog) > p(US analog/no CS analog)

P-16

Zero CS/US contingency

p(US analog/CS analog) = p(US analog/no CS analog)

P-17

CR speed

CR analog speed

P-18

CR strength/amplitude

CR analog strength/amplitude

an event or outcome such as production information (US analog) was assumed to elicit causality-seeking actions (UR analog). The power of agreements to elicit approach responses in attraction and the power of production information (e.g., level of production) to elicit


causality-seeking actions are social analogs of US intensity. Again, the general rules, numbered here for later reference, do not exhaust the possibilities, but they are described in sufficient detail to support our empirical work on interpersonal attraction and causal relationship detection.

2/16/2011 6:50:46 PM


Interpersonal Attraction

In research guided by the neo-Hullian theory of Rescorla and Wagner (1972; Wagner & Rescorla, 1972) and by close analogs of Pavlovian compound-cue conditioning principles and procedures, we tested novel predictions regarding context effects in interpersonal attraction when a participant encountered more than one agreeable person (Cramer et al., 1985). Attitudinal agreements are reinforcing social stimuli (Byrne, 1971; Lombardo, Libkuman et al., 1972; Lombardo, Weiss, & Buchanan, 1972) that elicit person-directed actions such as approach responses or “striving for” behaviors (Domjan, Lyons, North, & Bruell, 1986; Ganesan & Pearce, 1988; Hearst & Jenkins, 1974; O’Connell & Rashotte, 1982; Staats, 1975). General Method

A social stimulus (a person or persons/CS analog) was repeatedly paired with social reinforcement (attitudinal agreement/US analog), and the participant’s attraction (approach response speed/CR analog) to the social stimulus measured across trials. From the participant’s perspective, the experiments, masked as studies of opinion change, involved a continuous cycle of opinion, discussion, feedback, and opinion change. After a group of people listened to and then briefly discussed the participant’s opinion on a controversial topic, either one spokesperson or two spokespersons reported that a majority of the group members agreed with the opinion. The registration of opinion change immediately followed. Ostensibly, the “other people” in the study were in another room, and all communication would take place via an intercom. The “other people” were, in fact, simulated by tape recordings. For clarity of exposition we now use the theoretical labels A and X to describe the apparatus and procedure. The instructions and signals used colors (e.g., blue and orange) to refer to Person A and Person X in fully counterbalanced designs. A+ Attraction Conditioning

Conceptually, each single-stimulus (A+) cycle consisted of (1) a forward conditioning trial


437

pairing a CS analog (Person A) with a US analog (agreement with the participant’s opinion), and (2) a nonreinforced CSA test trial. At the beginning of an A+ cycle, the experimenter gave the participant the topic to be discussed, followed by illumination of the “press switch to open intercom to Person A” (nonreinforced CSA test trial) signal. The attraction dependent variable was the speed of a response opening a line of communication to Person A by throwing a toggle switch toward a light that stood for Person A. The instructions to the participants, the apparatus configuration, and the procedure maximized from the first the occurrence of the to-be-learned approach response and minimized the occurrence of competing responses. After the participants gave their opinion, and following a brief group discussion, the “reporting” signal was illuminated while Person A reported that a majority of group members agreed with the opinion (CS-US acquisition trial). Participants then performed the masking task by registering their opinion change, and the apparatus automatically reset to begin a new cycle. AX+ Attraction Conditioning

Each compound-stimulus (AX+) cycle included (1) a forward conditioning trial pairing a compound CS analog (Person A and Person X) jointly with a US analog (agreement with the participant’s opinion), (2) a nonreinforced CSX test trial, and (3) a nonreinforced CSA test trial. The AX+ cycle differed from the A+ cycle in two ways. First, approach response speeds were measured on two nonreinforced test trials, CSX, “press switch to open intercom to Person X,” followed by CSA. Throwing a switch toward lights that stood for Person A or Person X opened a line of communication to Person A or to Person X. Again, the instructions, the apparatus, and the procedure maximized from the first the occurrence of the to-be-learned approach response and minimized the occurrence of competing responses. Hence, the participants threw a single switch opening lines of communication to Person A and to Person X. Second, Person A and Person X were identified as spokespersons for a group of people listening

2/16/2011 6:50:46 PM

438


to and briefly discussing the participant’s opinion. A “reporting” signal was illuminated while Person A and Person X in compound jointly reported that a majority of group members agreed with the opinion expressed. Having two spokespersons (AX), or as on the A+ cycles a single spokesperson (A), report that a majority of group members agreed with the participant served to control an analog of US intensity by having the feedback be a “single agreement” on each trial. Because agreement comes from the group, the number of people agreeing is not confounded with the number of social stimuli. Participants then performed the masking task by registering their opinion change, and the apparatus automatically reset to begin a new cycle. Acquisition and Blocking Analogs

Among the most frequently investigated stimulus selection or context problems in conditioning is the blocking effect. Reinforcing a novel target stimulus in the context of a stimulus paired separately with the US attenuates or blocks response acquisition to the target stimulus (e.g., Barnet, Grahame, & Miller, 1993; Kamin, 1968, 1969; Kremer, 1978; Wagner, 1969). Acquisition and blocking effects in interpersonal attraction were investigated using an analog of an interspersedtrials procedure (Wagner, 1969) to mix A+ and AX+ cycles of operation in the blocking group; an acquisition group received only AX+ cycles (Rules of Correspondence P-5 and P-10). Approach response speed to Person A (Rule P-17) was an increasing function of the number of A+ trials, with the reinforcing effects of agreement being less effective as the attractiveness of Person A increased. Acquisition effects to Person X were observed in the blocking and acquisition groups, but despite Person X’s consistent objective relationship with agreement in both groups, speed to Person X was attenuated in the blocking group (Cramer et al., 1985). Three additional experiments employing analogs of a discretephases conditioning procedure (Kamin, 1968, 1969) and modest variations in method replicated the acquisition and blocking effects (Cramer et al., 1985; Weiss, McDonald, Little, & Shull, 1991).


Inhibition Analogs

Retardation of attraction acquisition and summation of inhibition of attraction were investigated in unpublished research in Weiss’ laboratory using an analog of a familiar procedure for producing conditioned inhibition, backward conditioning (Pavlov, 1927; Siegel & Domjan, 1974; Williams & Overmier, 1988; Zbroznya, 1958). These social analogs of retardation and summation (Pavlov, 1927; Rescorla, 1969) were investigated using a common procedure where testing is omitted in the initial phases of conditioning and is reserved for the final phase of conditioning. Our standard cycle of operation was modified so that participants received a series of testless backward compound conditioning trials. During the training phase, spokespersons A and X in compound jointly delivered the group’s agreement with the participant’s previously expressed opinion, with the reporting signal indicating the two persons speaking lit only after the spokespersons delivered the group’s agreement (Rule P-11). Unlike our standard cycle of operation, participants did not open a line of communication to the spokespersons during the training phase. In a retardation of attraction acquisition study, two groups of participants received either zero or two backward compound conditioning trials in an initial training phase. The training phase was followed by six standard AX+ cycles of operation, with the participant’s approach response speeds to Person A and Person X measured separately (Rule P-10 and P-17). A third group of participants received six backward compound conditioning trials followed by six forward compound trials, with speed measured to the compound social stimulus containing persons A and X. Figure 19.8 shows increasing approach speeds across the AX+ trials. The combined speeds to the single inhibitors were slower for the group receiving two backward trials compared to zero backward trials in the initial phase of the experiment. Moreover, the effect of inhibition of attraction on the retardation of attraction acquisition was most pronounced when, following six backward compound trials, approach speeds to a double inhibitor social stimulus were measured.

2/16/2011 6:50:46 PM


439

Zero Backward 90

90

Single Inhibitor 80

70

Two Backward

60 50 Six Backward Double Inhibitor

40

70 60 Double Inhibitor

50 40 30

30 20

Speed (100/Latency)

Speed (100/Latency)

80

20 1 2

2 3

3 4

4 5

5 6


Figure 19.8 Inhibition of attraction: Retardation

of the acquisition of approach response speed to persons A and X as a function of number of backward conditioning trials to those people in the previous phase. In the Zero Backward and Two Backward conditions, speed was measured to Person A and Person X separately; in the Six Backward/Double Inhibitor condition, speed was measured to persons A and X in compound.

1 2

2 3

3 4

4 5

5 6


Figure 19.9 Summation of two inhibitors of

attraction: Retardation test shows two inhibitors to be more potent than one. In the Single Inhibitor condition, approach response speed was measured to Person A and Person X separately; in the Double Inhibitor condition, speed was measured to persons A and X in compound.

Super-Conditioning Analog

Figure 19.9 shows an analog of summation of inhibition of attraction. Following six backward compound conditioning trials participants received six single inhibitor or double inhibitor nonreinforced test trials (Rule P-6 and P-12). In phase 2 approach speeds were measured either to a single inhibitor social stimulus, Person A and Person X separately, or to a double inhibitor stimulus, persons A and X in compound. The combined speeds in the single inhibitor test condition were faster than in the double inhibitor test condition. Despite the consistent objective relationship of Person A and Person X to social reinforcement, participants were more attracted to the spokespersons when they were tested as single inhibitors than when they were tested as a double inhibitor, twoperson group.


Blocking effects in attraction occur when a target social stimulus, Person X, is paired with agreement in the context of another person participants find attractive, Person A. By extending Pavlovian conditioning principles further, we observed, in unpublished research in Weiss’ laboratory, an effect in opposition to blocking, attraction super-conditioning. In animal learning, when a stimulus compound containing an inhibitory stimulus and a novel stimulus are reinforced, increased responding or superconditioning to the target stimulus was observed (e.g., Navarro, Hallam, Matzel, & Miller, 1989). By analogy, the use of backward conditioning in the first phase of the experiment (Rule P-8) results in Person A becoming a conditioned inhibitor of attraction, which then results in super-conditioning of attraction to Person X in a second, compound stimulus conditioning phase

2/16/2011 6:50:46 PM

440


60 FIVE

Speed (100/Latency)

50 TWO

40

30

ZERO

20

10

0

1 2

2 3

3 4

4 5


Figure 19.10 Super-conditioning of attraction:

Acquisition of approach response speed to Person X as a function of number of backward conditioning trials to Person A.

(Rule P-10). Figure 19.10 shows approach speeds to Person X were an increasing function of the number of first phase, backward conditioning trials (0, 2, or 5) to Person A. The stronger the conditioned inhibition of attraction to Person A, the stronger the participant’s attraction to Person X when persons A and X jointly agreed. Implications Losses in Attraction Strength

Attitudinal agreements, like conventional reinforcers, do not always result in greater attraction. In fact, as an agreeable person’s attractiveness increases, an agreement become less and less effective as reinforcement, and provided agreement magnitude remains constant, attraction, like conventional conditioned responses, eventually reaches asymptote. Moreover, agreements may actually result in “losses in attraction strength.” It is now well supported in conditioning research and theory, that when highly conditioned stimuli are reinforced in compound, the response strength elicited by each stimulus


is reduced (overexpectation effect; Ganesan & Pearce, 1988; Kremer, 1978; Lattal & Nakajima, 1998; Levitan, 1975; Rescorla, 1999; Rescorla & Wagner, 1972). Losses in response strength occur because the combined strength of the compounded stimuli exceeds the asymptote of conditioning supportable by the magnitude of reinforcement (Rescorla & Wagner, 1972). By analogy, if Person A and Person X are equally attractive and their combined attractiveness exceeds the asymptote supportable by the magnitude of agreement (Rule P-14), having persons A and X, in compound, jointly agree with the participant decreases by an equal amount their individual attraction strengths. If, on the other hand, Person A is liked more than Person X, and their combined attractiveness exceeds the asymptote supported by the magnitude of agreement, attraction to each person will be reduced, but with the attraction strength of Person A remaining higher than that of Person X. Consistently agreeable people, despite their objective relationship to social reinforcement, may not only fail to elicit attraction, but may actually contribute to a reduction in their attractiveness and the attractiveness of others. Social and Nonsocial Stimulus Interactions

The contexts in which attraction often develops are not composed only of individuals, but include nonsocial stimuli such as physical objects and conditions, setting, and ambience. Should any of these nonsocial stimuli be attractive, they will, in theory, block the acquisition of attraction among people who agree with one another. One strategy for avoiding attraction blocking is to reduce the salience of any competing nonsocial stimuli (Rule P-13). Reinforcing someone, hoping to engender attraction, therefore, is likely to be more effective in an environment with affectively neutral nonsocial stimuli than in one where the nonsocial stimuli are attractive. Dare we speculate that the development of interpersonal relationships will be facilitated in the sterile “stainless steel” environments often envisioned by futurists and science fiction writers? The familiar phrase “no one is more fanatical than a convert” succinctly summarizes a social analog of super-conditioning when a defector

2/16/2011 6:50:47 PM


from one social movement joins another (Weiss, 1963). Because few conversions are complete when the defection takes place, the new social movement provides the defector with numerous social rewards in the context of other social and nonsocial stimuli likely to continue inhibiting the defector’s attraction to the new movement. In theory, the defector’s experience, compared to that of a “raw recruit,” results in fanatical or super-conditioned attraction and devotion to the new movement. Attraction in Context

Attraction is no mere matter of repeated reinforcement. Our observations of blocking, retardation, summation, and super-conditioning analogs amply demonstrate the fundamental role social context plays in the acquisition of attraction. Further amplification of this vital role is provided in the novel predictions of losses in attraction strength, and of blocking and superconditioning effects when social and nonsocial stimuli interact. The blocking and super-conditioning predictions are particularly instructive because they suggest that the effect of a social stimulus (person) becomes subject to a radical transformation by the action of nonsocial stimuli. Moreover, the role of nonsocial, situational stimuli becomes more interesting when they occur in compound with social stimuli. Our illustrative examples of attraction in context, either reported or predicted here, are outside the boundary conditions of simple contiguity models of attraction like the reinforcement-affect theory (e.g., Byrne, 1971; Clore & Byrne, 1974). Social psychologists should take note that these limitations do not constitute a “disproof” of Byrne’s law of attraction, one of the most robust results in social psychology. Human Agency and Causal Relationship Detection

Because experiments in selective learning normally begin with response tendencies of equal strength, associative accounts of causal relationship detection among nonsocial events and outcomes begin without a bias among the research participants for one putative cause over another


441

(e.g., Dickinson & Shanks, 1985; Dickinson, Shanks, & Evenden, 1984; Shanks, 1991; Shanks & Dickinson, 1987). What social psychologists have long recognized is that when event-outcome relationships include the possibility of human agency, causal relationship detection is commonly affected by the correspondence bias (Gilbert & Malone, 1995; Jones, 1979; Jones & Harris, 1967), the general tendency to view people as particularly powerful “agents” or “at cause” for the outcomes they experience. In research guided once again by the theory of Rescorla and Wagner (1972; Wagner & Rescorla, 1972) and by close analogs of Pavlovian compound-cue conditioning principles and procedures, we investigated the implications for an associative account of causal relationship detection involving social stimuli when a possibility exists that a participant’s trial-by-trial sensitivity to the procedures is compromised, and a social stimulus evokes the target response prior to training (Cramer et al., 2002). General Method

Participants played the role of a production supervisor evaluating the effectiveness of parttime workers in a fictional company. A computerized evaluation system presented graphic material representing a worker (event) and the company’s month-end level of production (outcome) across several reporting periods, and enabled participants to rate a worker’s causal relationship to production. For clarity of exposition, the theoretical labels A and X are used to describe the procedure. In the experiments, however, the instructions and computer labels used proper names when referring to the workers. A+ Conditioning

From the participant’s perspective, the experiment proceeded as a continuous cycle of evaluation. Conceptually, however, a single-worker evaluation cycle (A+) consisted of a forward conditioning trial pairing a CS analog (Worker A) and a US analog (company month-end level of production). In a social analog of delay conditioning, Worker A was displayed alone for 5 s before the worker was paired with graphic

2/16/2011 6:50:47 PM


production information for an additional 10 s. Following the conditioning component of the cycle, participants rated the strength of the causal relationship (CR) of the worker to production using a 100-point scale anchored by 0 = Totally Ineffective and 100 = Totally Effective. AX+ Conditioning

A compound stimulus or multiple worker evaluation cycle (AX+) consisted of a forward conditioning trial on which two workers, Worker A and Worker X, were jointly paired with the company’s month-end production level. The compound CS analog was displayed for 5 s before the compound CS analog was paired with the US analog for an additional 10 s. Following the compound conditioning component of the cycle, participants rated the CR strength of a target worker to production, either Worker A or Worker X. Methodological Study

Our expectation that social stimuli are viewed as more “at cause” than nonsocial stimuli was confirmed by participants asked to rank order three plausible causes for company productivity. In a brief survey, a worker was judged as more effective than either a production quota or quality control standards in causing company productivity. These results, and other related findings confirming the agency of social stimuli (i.e., agency of a student and an athlete for examination and sport team performance, respectively), were consistent with the possibility that an associative account of causal relationship detection involving social stimuli may be constrained by the correspondence bias. CS/US Contingency Analogs

Despite possessing a priori agency, the acquisition of CR strength of a worker to production was determined by repeatedly pairing a worker with production information, and the contingency relationship between the worker and production. Figure 19.11 shows that the CR strength of a worker to production gradually increased across trials when a positive contingency (Rescorla, 1967) existed between the worker and production information (Rules


85 CR Strength of Worker to Production

442

GROUP + 80 75 70 65

GROUP 0

60 55 50

1

2

3 4 5 6 7 Acquisition Trials

8

9

Figure 19.11 Human

agency conforms to Pavlovian CS/US contingency. CR = causal relationship; Group + = positive contingency; Group 0 = zero contingency. (Redrawn from Cramer, Weiss, William, Reid, Nieri et al., 2002).

of Correspondence P-15 and P-18). No reliable increase in CR strength was observed when a zero contingency existed between the worker and production information (Rule P-16). US Intensity Analog

Because ample evidence indicates that response strength increases as US intensity increases (e.g., Hoehler & Leonard, 1981; O’Connell & Rashotte, 1982; Prokasy, Grant, & Myers, 1958; Spence, 1956), we anticipated that workers paired with different production levels (Rule P-14; effect intensity = low, medium, and high) would not be uniformly evaluated as “effective” or “at cause.” Participants, as expected, were sensitive to the level of production paired with a worker, with the CR strength of the worker to production being an increasing function of US analog intensity. Blocking Analog

Blocking effects have been reported in studies of causal and contingency judgments involving nonsocial stimuli (e.g., Aitken, Larkin, & Dickinson, 2000; Kruschke & Blair, 2000; Shanks, 1985;

2/16/2011 6:50:47 PM


Van Hamme, Kao, & Wasserman, 1993). And Van Overwalle and Van Rooy (1998, 2001) demonstrated that the delta rule of connectionist theory (McClelland & Rumelhart, 1988) reliably accounts for blocking effects using social stimuli. Blocking effects in causal relationship detection involving social stimuli were observed by manipulating, for the first time, two close analogs of Pavlovian conditioning (Cramer et al., 2002). An analog of an interspersed-trials procedure (Wagner, 1969) mixed A+ and AX+ trials in a blocking group; an acquisition group received only AX+ trials (Rule P-5 and P-10). Figure 19.12 shows that the CR strength of Worker X to production measured on reinforced X+ test trials was lower in the blocking group than in the acquisition control group. Blocking effects replicated using an analog of discrete-phases conditioning (Fig. 19.12; Kamin, 1968, 1969). These results indicated that causal relationship learning involving social stimuli was no mere matter of repetition, or even reinforced repetition.

443

Rather, CR blocking occurred despite the participants’ a priori beliefs that workers are likely to be responsible or “at cause” for production, and despite Worker X’s consistent objective relationship to production. Super-Conditioning Analog

Aitken et al. (2000) observed super-conditioning in causality detection, manipulating an analog of the conditional procedure for creating a conditioned inhibitor (Pavlov, 1927; Rescorla, 1979) in the context of a food-allergic reaction prediction task. Super-conditioning of a causal relationship involving social stimuli was obtained in unpublished research in Cramer’s laboratory using close analogs of the conditional procedure. Participants in the experimental group received, in turn, five A+, AB–, and BX+ conditioning trials followed by five reinforced X+ test trials (Rule P-5, P-10, and P-12); the controls received only 10 reinforced X+ trials. Figure 19.13 shows two effects of particular interest to causal relationship researchers: a social analog of inhibition and of super-conditioning. Based on our laboratory

85

80 CR Strength of Worker to Production

CR Strength of Worker X to Production

85

75

70

65

60 Blocking (Interspersedtrials)

Blocking (Discretephases)

75

65

55

Acquisition

Experimental Groups

45 A

B X (Experimental)

X (Control)

Workers

Figure 19.12 Blocking of a causal relationship (CR)

in two kinds of experienced situations (discrete phases and interspersed trials) despite worker agency.


Figure 19.13 Super-conditioning of causal rela-

tionship (CR) strength of Worker X to production via an analog of an A+/AB−/BX+ paradigm.

2/16/2011 6:50:47 PM

444


experience in measuring CRs and on the Group 0 findings reported in Figure 19.11, the subnormal CR strength of Worker B (Fig. 19.13) is indicative of an inhibition analog. Worker B’s CR strength is well below that of the zero contingency group on the initial trial and on the last two trials. Consequently, Figure 19.13 also shows that pairing workers B and X with production information on a series of reinforced compound trials in the BX+ phase resulted in super-conditioning of CR strength of Worker X to production. Implications Super-Conditioning and the Augmenting Effect

We join the connectionists Van Overwalle and Van Rooy (1998, 2001) in recognizing that when the events preceding an outcome include both excitatory/facilitatory and inhibitory stimuli, learning and social psychologists alike predict super-conditioning or augmenting, respectively. The explanation for augmenting preferred by social researchers, however, relies on the attributer’s causal schemata and logical inferences (e.g., Kelley, 1972a, 1972b) rather than on the operation of mechanical learning principles. Moreover, social researchers typically study augmenting in described situations (e.g., Hansen & Hall, 1985) rather than in experienced situations. Interestingly, a more complete understanding of causal agency is possible by studying experienced situations, where dynamic variables like the frequency of event-outcome pairings and the order of events and outcomes play a powerful role. For example, when a forward conditioning procedure is used to pair Worker A with production information, subsequent reinforcement of workers A and X in compound results in blocking of CR strength to Worker X. In sharp contrast, if production information is paired with Worker A using an analog of backward conditioning (Rule P-8), Worker A, in theory, becomes a conditioned inhibitor (e.g., Pavlov, 1927; Williams & Overmier, 1988), and we predict that subsequent reinforcement of workers A and X in compound results in augmenting or super-conditioning of CR strength to


Worker X. Merely manipulating the order in which a worker is paired with production information provides a demonstration of that variable’s fundamental role in determining the strength of a supervisor’s evaluation of a worker’s causal agency. Multiplicity of Discounting

Worker A and Worker X are individually paired with production information until their individual CR strengths reach asymptote. What happens to their individual CR strengths if the two workers are subsequently paired in compound with the same analog of US intensity? Because the sum of the individual CR strengths is greater than the strength supportable by the US analog (Rule P-14), the CR strengths of Worker A and Worker X are predicted to decrease despite the workers’ continuing objective relation to production (e.g., Ganesan & Pearce, 1988; Kremer, 1978; Lattal & Nakajima, 1998; Levitan, 1975; Rescorla, 1999; Rescorla & Wagner, 1972). The anticipated losses in CR strength in conditioning are analogous to discounting in social psychology where, according to attribution theory, the mere presence of multiple plausible causes for an effect leads one “logically” to discount the agency of each cause (e.g., Jones & Davis, 1965; Kelley, 1972a, 1972b). The logic of modeling does not provide a challenge to the traditional explanations of discounting by merely translating the language of attribution into the language of conditioning. Rather, the logic of modeling provides for the specification of a multiplicity of discounting rendered theoretically determinate by Pavlovian principles. What we mean by a multiplicity of discounting can be exemplified by different types of unequal discounting, each with its own underlying causal mechanism. Consider an experiment in which the CR strengths of Worker A and Worker X are unequal (A > X) prior to being paired in compound with production information, and in which their combined CR strength is greater than the US analog can support. Pairing workers A and X in compound with production information is predicted to generate equal trialby-trial losses in the CR strength of each worker,

2/16/2011 6:50:47 PM


with the final result being Worker A still stronger than Worker X (unequal discounting). Now turn to a different experiment in which the CS intensity of Worker A is greater than that of Worker X and in which their individual CR strengths are at asymptote before compound conditioning, the losses in CR strength will be related to their intensities (e.g., brightness or vividness of clothing color and pattern; Rule P-13). Pairing Worker A and Worker X in compound with production information is predicted to generate trial-by-trial losses in the CR strength of each worker, but with the more salient worker, A, losing more CR strength than the less salient worker, X (unequal discounting). Our term inverse overshadowing nicely captures the predicted effect in discounting and in conditioning (see Kamin & Gaioni, 1974; Kremer, 1978). When the underlying causal mechanism involves differences in the workers’ associative strengths, continued reinforcement of the workers in compound generates equal trial-bytrial losses in the CR strength of each worker and eventuates in a stronger Worker A than Worker X (unequal discounting). In sharp contrast, when the causal mechanism involves differences in the workers’ intensity, continued reinforcement generates unequal trial-by-trial losses in the CR strength of each worker, with both the trial-by-trial and final loss in CR strength being greater for the “more intense” worker than the “less intense” worker (unequal discounting). These predictions of unequal discounting, which are guided by underlying causal mechanisms that determine where in the learning process unequal discounting is to be anticipated, provide powerful testimony indeed for the logic of modeling in social research. Because associative theory does not confuse the phenomenon of discounting with its underlying causal mechanisms, an Aristotelian error that Lewin (1935) long ago warned against, it effortlessly accounts for the richness and interest of a multiplicity of discounting.

SUMMARY The social psychological experiments reported in this chapter were informed and guided by


445

research with roots in the work of Thorndike and Pavlov, and in the Hull-Miller-Spence tradition. Close analogs of instrumental escape and Pavlovian compound-cue conditioning were devised and manipulated to predict and observe a compelling number of correspondences between the learning models and five domains of social behavior. For convenience, Table 19.4 summarizes the correspondences (with particular emphasis on their replication) observed between variables in instrumental conditioning and in interpersonal communication, altruism, and competition, and between variables in Pavlovian conditioning and in attraction and causal relationship detection. The logic of modeling led to experimentally delineated analogical portraits of various social processes that are well fleshed out and can be exquisite in their completeness. The beauty of the experimental delineation of each social process also comes, in no small part, from their origin as theoretical predictions: The experimental facts do fit together to make theoretically coherent portraits. By continuing to construct artificial social structures to the blueprints of learningtheoretical models, the understanding of social behavior is enriched in ways both distinctive and fascinating.

THE EXCELLENCE OF CONDITIONING RESEARCH As scientists, we are accustomed to test the truth of a hypothesis by its consequences in experiment and to evaluate the goodness of the knowledge thus obtained by the degree to which these results are determinate, by p-values, by elegance of curves, by replicability, and so on. Replicability imposes an additional test of consequences, but there is a powerful further test of consequences about which we seldom speak: It is the ability of knowledge to serve as a tool in further inquiry (see Dewey, 1938). Such a tool may be used narrowly, as when Judson Brown’s ingenious startle-reflex technique is used once again as an index of conditioned fear. Broadly, a wellorganized body of knowledge acquires the ability to be used by analogy to predict a different domain. The special excellence of inquiry in

2/16/2011 6:50:47 PM

446


Table 19.4 N-Experiments in Which Social Analogs of Instrumental and Pavlovian Conditioning Were Obtained Instrumental Analogs

Social Domains Communication

Altruism

16

6

12

Delay of reinforcement

3

2

1

Delay of reinforcement shifts

1

Partial reinforcement

2

1

1

1

1

Acquisition

Magnitude of reinforcement Duration of shock-free period

2

Correlated reinforcement

2

Correlated delay of reinforcement

1

Extinction

1

Extinction (partial reinforcement)

1

Drive intensity Campbell and Kraeling effects

Competition

3 1

1

Drive downshifts

1

Drive energization

2

1

Learned drive

1

1

Competence (drive variable) Intermittent shock Pavlovian Analogs Acquisition

2 2

1

1

Attraction

Human Agency

7

5

CS/US contingency

1

US intensity

1

Blocking (interspersed-trials)

1

1

Blocking (discrete-phases)

3

1

Forward conditioning

6

5

Backward conditioning

3

Acquisition of inhibition

3

Retardation test of inhibition

1

Summation of two inhibitors

1

Super-conditioning via backward conditioning

1

Super-conditioning via conditional procedure


1

2/16/2011 6:50:47 PM


conditioning generates knowledge of such versatility that we could use it as a model to predict five different domains of social behavior. Nor was only one kind of conditioning able to serve as a model. In vivid contrast to the vast body of social psychology, we have been able to predict curve shapes. How was this possible? Because the conditioning models include the analogs of those curve shapes, and they often led us to continuous variables that varied on physical continua and on ratio scales or to approximations thereof. The scope widens to include “a war of all against all,” complete with the elimination of opponents, and still the conditioning model predicts. When we make a particularly satisfying discovery, we thank the rat runners, those excellent scientists who make our discoveries possible.

REFERENCES Aitken, M. R. F., Larkin, M. J. W., & Dickinson, A. (2000). Super-learning of causal judgments. Quarterly Journal of Experimental Psychology, 53B, 59–81. Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge, England: Cambridge University Press. Anderson, D. C., Johnson, D., & Kempton, H. (1969). Second-order fear conditioning as revealed through augmentation of a startle response: Part II. Psychonomic Science, 16, 7–9. Barnet, R. C., Grahame, N. J., & Miller, R. R. (1993). Temporal encoding as a determinant of blocking. Journal of Experimental Psychology: Animal Behavior Processes, 19, 327–341. Bower, G. H. (1960). Partial and correlated reward in escape conditioning. Journal of Experimental Psychology, 59, 126–130. Bower, G. H., Fowler, H., & Trapold, M. A. (1959). Escape learning as a function of amount of shock reduction. Journal of Experimental Psychology, 58, 482–484. Brown, J. S., & Farber, I. E. (1968). Secondary motivational systems. In P. R. Farnsworth (Ed.), Annual review of psychology (Vol. 19, pp. 99–134). Palo Alto, CA: Annual Reviews, Inc. Byrne, D. (1971). The attraction paradigm. New York, NY: Academic Press. Campbell, B. A. (1955). The fractional reduction in noxious stimulation required to produce “just noticeable” learning. Journal of Comparative and Physiological Psychology, 48, 141–148.


447

Campbell, B. A. (1968). Interaction of aversive stimuli: Summation or inhibition? Journal of Experimental Psychology, 78, 181–190. Campbell, B. A., & Kraeling, D. (1953). Response strength as a function of drive level and amount of drive reduction. Journal of Experimental Psychology, 45, 97–101. Capaldi, E. J. (1978). Effects of schedule and delay of reinforcement on acquisition speed. Animal Learning & Behavior, 6, 330–334. Castaneda, A. (1965). The paired-associates method in the study of conflict. In L. P. Lipsitt & C. C. Spiker (Eds.), Advances in child development (Vol. 2, pp. 1–18). New York, NY: Academic Press. Castaneda, A., & Lipsitt, L. P. (1959). Relation of stress and differential position habits to performance in motor learning. Journal of Experimental Psychology, 57, 25–30. Church, R. M. (1962). The effects of competition on reaction time and palmar skin conductance. Journal of Abnormal and Social Psychology, 65, 32–40. Clore, G. L., & Byrne, D. (1974). A reinforcementaffect model of attraction. In T. L. Huston (Ed.), Foundations of interpersonal attraction (pp. 143– 170). New York, NY: Academic Press. Cottrell, N. B. (1968). Performance in the presence of other human beings: Mere presence, audience and affiliation effects. In E. C. Simmel, R. A. Hoppe, & G. A. Milton (Eds.), Social facilitation and imitative behavior (pp. 91–110). Boston, MA: Allyn & Bacon. Cottrell, N. B., Rittle, R. H., & Wack, D. L. (1967). The presence of an audience and list type (competitional or noncompetitional) as joint determinants of performance in paired-associates learning. Journal of Personality, 35, 425–434. Cramer, R. E., Helzer, K., & Mone, R. (1986). Variations in the conditions of reinforcement and the attribution of liking. Journal of General Psychology, 113, 341–349. Cramer, R. E., Lutz, D. J., Bartell, P. A., Dragna, M., & Helzer, K. (1989). Motivating and reinforcing functions of the male sex role: Social analogues of partial reinforcement, delay of reinforcement, and intermittent shock. Sex Roles, 20, 551–573. Cramer, R. E., McMaster, M. R., Bartell, P. A., & Dragna, M. (1988). Subject competence and minimization of the bystander effect. Journal of Applied Social Psychology, 18, 1133–1148. Cramer, R. E., Weiss, R. F., Steigleder, M. K., & Balling, S. S. (1985). Attraction in context: Acquisition and blocking of person-directed

2/16/2011 6:50:47 PM

448


action. Journal of Personality and Social Psychology, 49, 1221–1230. Cramer, R. E., Weiss, R. F., William, R., Reid, S., Nieri, L., & Manning-Ryan, B. (2002). Human agency and associative learning: Pavlovian principles govern social process in causal relationship detection. Quarterly Journal of Experimental Psychology, 55B, 241–266. Crisson, J. E., Seta, J. J., & Seta, C. E. (1995). The influence of expectations on task performance in audience and solitary settings. Basic and Applied Social Psychology, 17, 357–370. Cronbach, L. J. (1963). Educational psychology. New York, NY: Harcourt, Brace & World. Denny, M. R. (1971). Relaxation theory and experiments. In F. R. Brush (Ed.), Aversive conditioning and learning (pp. 235–295). New York, NY: Academic Press. Denny, M. R. (1991). Relaxation/relief: The effect of removing, postponing, or terminating aversive stimuli. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 199–230). Hillsdale, NJ: Erlbaum. Dewey, J. (1938). Logic: The theory of inquiry. New York, NY: Holt. Dickinson, A., & Shanks, D. R. (1985). Animal conditioning and human causality judgment. In L. G. Nilsson & T. Archer (Eds.), Perspectives on learning and memory (pp. 167–191). Hillsdale, NJ: Erlbaum. Dickinson, A., Shanks, D. R., & Evenden, J. L. (1984). Judgment of act-outcome contingency: The role of selective attribution. Quarterly Journal of Experimental Psychology, 36A, 29–50. Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy. New York, NY: McGraw-Hill. Domjan, M., Lyons, R., North, N. C., & Bruell, J. (1986). Sexual Pavlovian conditioned approach behavior in male Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 100, 413–421. Feinberg, R. A., Miller, F. G., Weiss, R. F., Steigleder, M. K., & Lombardo, J. P. (1982). Motivational aspects of learned helplessness. Journal of General Psychology, 106, 273–311. Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row, Peterson. Fowler, H., & Trapold, M. A. (1962). Escape performance as a function of delay of reinforcement. Journal of Experimental Psychology, 63, 464–467. Franchina, J. J. (1966). Effects of shock schedules on the acquisition and extinction of escape behavior. Psychonomic Science, 4, 277–278.


Franchina, J. J. (1969a). Intertrial intervals and shock schedules in escape conditioning. Journal of Comparative and Physiological Psychology, 67, 504–509. Franchina, J. J. (1969b). Escape behavior and shock intensity: Within-subject versus between-groups comparisons. Journal of Comparative and Physiological Psychology, 69, 241–245. Franchina, J. J., Kash, J. S., Reeder, J. R., & Sheets, C. T. (1978). Effects of exteroceptive feedback and safe-box confinement durations on escape behavior in rats. Animal Learning & Behavior, 6, 423–428. Franchina, J. J., & Schindele, T. E. (1975). Nonshock confinement duration and shock- and safebox similarity during escape training in rats. Animal Learning & Behavior, 3, 297–300. Ganesan, R., & Pearce, J. M. (1988). Interactions between conditioned stimuli for food and water in the rat. Quarterly Journal of Experimental Psychology, 40B, 229–242. Gauthier, D. P. (1969). The logic of Leviathan. Oxford, England: Clarendon Press. Geen, R. G., & Gange, J. J. (1977). Drive theory of social facilitation: Twelve years of theory and research. Psychological Bulletin, 84, 1267–1288. Gilbert, D. T., & Malone, P. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21–38. Goodrich, K. P. (1959). Performance in different segments of an instrumental response chain as a function of reinforcement schedule. Journal of Experimental Psychology, 57, 57–63. Gray, J. A. (1982). The neuropsychology of anxiety: An inquiry into the functions of the septohippocampal system. London, England: Oxford University Press. Grusec, J. E. (1991). The socialization of altruism. In M. S. Clark (Ed.), Review of personality and social psychology (Vol. 12, pp. 9–33). Newbury Park, CA: Sage. Grusec, J. E., & Redler, E. (1980). Attribution, reinforcement, and altruism: A developmental analysis. Developmental Psychology, 16, 525–534. Hansen, R. D., & Hall, C. A. (1985). Discounting and augmenting facilitative and inhibitory forces: The winner takes almost all. Journal of Personality and Social Psychology, 49, 1482–1493. Hearst, E., & Jenkins, H. M. (1974). Sign-tracking: The stimulus-reinforcer relation and directedaction. Austin, TX: The Psychonomic Society.

2/16/2011 6:50:47 PM


Hesse, M. (1966). Models and analogies in science. Notre Dame, IN: University of Notre Dame Press. Hesse, M. (1974). The structure of scientific inference. Berkeley, CA: University of California Press. Hesse, M. (1980). Revolutions and reconstructions in the philosophy of science. Bloomington, IN: Indiana University Press. Hobbes, T. (1640/1984). The elements of law: Natural and politic (F. Tönnies, Ed., 2nd ed.). London, England: Cass. Hobbes, T. (1647/1983). De Cive. Oxford, England: Clarendon Press. Hobbes, T. (1651/1950). Leviathan. New York, NY: Dutton. Hoehler, F. K., & Leonard, D. W. (1981). Motivational vs. associative role of the US in classical conditioning of the rabbit’s nictitating membrane response. Animal Learning & Behavior, 9, 239–244. Hull, C. L. (1943). Principles of behavior. New York, NY: Appleton-Century-Crofts. Jones, E. E. (1979). The rocky road from acts to dispositions. American Psychologist, 34, 107–117. Jones, E. E., & Davis, K. E. (1965). From acts to dispositions: The attribution process in person perception. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 7, pp. 219– 266). New York, NY: Academic Press. Jones, E. E., & Harris, V. A. (1967). The attribution of attitudes. Journal of Experimental Social Psychology, 3, 1–24. Kamin, L. J. (1968). “Attention-like” processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior, 1967: Aversive stimulation (pp. 9–31). Coral Gables, FL: University of Miami Press. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279–296). New York, NY: Appleton-Century-Crofts. Kamin, L. J., & Gaioni, S. J. (1974). Compound conditioned emotional response conditioning with differentially salient elements in rats. Journal of Comparative and Physiological Psychology, 87, 591–597. Kelley, H. H. (1972a). Attribution in social interaction. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior (pp. 1–26). Morristown, NJ: General Learning Press.


449

Kelley, H. H. (1972b). Causal schemata and the attribution process. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior (pp. 151–174). Morristown, NJ: General Learning Press. Knowles, S. S. (1983). Social physics and the effects of others: Tests of the effects of audience size and distance on social judgments and behavior. Journal of Personality and Social Psychology, 45, 1263–1279. Kremer, E. F. (1978). The Rescorla-Wagner model: Losses in associative strength in compound conditioned stimuli. Journal of Experimental Psychology: Animal Behavior Processes, 4, 22–36. Kruschke, J. K., & Blair, N. J. (2000). Blocking and backward blocking involve learned inattention. Psychonomic Bulletin & Review, 7, 636–645. Latané, B., & Bourgeois, M. J. (1996). Experimental evidence of dynamic social impact: The emergence of subcultures in electronic groups. Journal of Communication, 46, 35–47. Lattal, K. M., & Nakajima, S. (1998). Overexpectation in appetitive Pavlovian and instrumental conditioning. Animal Learning & Behavior, 26, 351–360. Levitan, L. (1975). Tests of the Rescorla-Wagner model of Pavlovian conditioning. Bulletin of the Psychonomic Society, 6, 265–268. Lewin, K. (1935). A dynamic theory of personality. New York, NY: McGraw-Hill. Logan, F. A. (1959). The Hull-Spence approach. In S. Koch (Ed.), Psychology: A study of a science (Vol. 2, pp. 293–358). New York, NY: McGraw-Hill. Logan, F. A. (1960). Incentive. New Haven, CT: Yale University Press. Logan, F. A. (1961). Discrete-trials DRL. Journal of the Experimental Analysis of Behavior, 4, 277–279. Lombardo, J. P., Libkuman, T. M., & Weiss, R. F. (1972). The energizing effects of disagreementinduced drive. Journal of Experimental Research in Personality, 6, 133–141. Lombardo, J. P., Weiss, R. F., & Buchanan, W. (1972). Reinforcing and attracting functions of yielding. Journal of Personality and Social Psychology, 21, 359–368. Luginbuhl, J. E. (1972). Role of choice and outcome on feelings of success and estimates of ability. Journal of Personality and Social Psychology, 22, 121–127. Macpherson, C. B. (1962). The political philosophy of possessive individualism: Hobbes to Locke. Oxford, England: Clarendon Press.

2/16/2011 6:50:47 PM

450


Macy, M. W. (1990). Learning theory and the logic of critical mass. American Sociological Review, 55, 809–826. Mandler, G., & Watson, D. L. (1966). Anxiety and the interruption of behavior. In C. D. Spielberger (Ed.), Anxiety and behavior (pp. 263–288). New York, NY: Academic Press. Marlin, N. A. (1983). Second-order conditioning using contextual stimuli as S1. Learning & Behavior, 11, 290–294. Martens, R., & Landers, D. M. (1972). Evaluation potential as a determinant of coaction effects. Journal of Experimental Social Psychology, 8, 347–359. Marwell, G., & Oliver, P. (1993). The critical mass in collective action. Cambridge, England: Cambridge University Press. Masterman, M. (1980). Braithwaite and Kuhn: Analogy-clusters within and without hypothetico-deductive systems in science. In D. H. Mellor (Ed.), Science, belief, and behavior (pp. 61–86). Cambridge, England: Cambridge University Press. McAllister, D. E., & McAllister, W. R. (1964). Second-order conditioning of fear. Psychonomic Science, 1, 383–384. McAllister, D. E., & McAllister, W. R. (1991). Fear theory and aversively motivated behavior: Some controversial issues. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 135–163). Hillsdale, NJ: Erlbaum. McAllister, D. E., McAllister, W. R., Brooks, C. I., & Goldman, J. A. (1972). Magnitude and shift of reward in instrumental aversive learning in rats. Journal of Comparative and Physiological Psychology, 80, 490–501. McAllister, W. R., & McAllister, D. E. (1992). Fear determines the effectiveness of a feedback stimulus in aversively motivated instrumental learning. Learning and Motivation, 23, 99–115. McAllister, W. R., McAllister, D. E., & Benton, M. M. (1983). Measurement of fear of the conditioned stimulus and of situational cues at several stages of two-way avoidance learning. Learning and Motivation, 14, 92–106. McClelland, J. M., & Rumelhart, D. E. (1988). Explorations in parallel distributed processing: A handbook of models, programs and exercises. Cambridge, MA: Bradford. Miller, N. E. (1959). Liberalization of basic S-R concepts: Extensions to conflict behavior, motivation and social learning. In S. Koch (Ed.),


Psychology: A study of a science (Vol. 2, pp. 196–292). New York, NY: McGraw-Hill. Miller, N. E., & Dollard, J. (1941). Social learning and imitation. New Haven, CT: Yale University Press. Mineka, S. (1975). Some new perspectives on conditioned hunger. Journal of Experimental Psychology: Animal Behavior Processes, 104, 134–148. Mongeluzi, D. L., Rosellini, R. A., Caldarone, B. J., Stock, H. S., & Abrahamsen, G. C. (1996). Pavlovian aversive context conditioning using carbon dioxide as the unconditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 22, 244–257. Mowrer, O. H., & Viek, P. (1948). An experimental analogue of fear from a sense of helplessness. Journal of Abnormal and Social Psychology, 43, 193–200. Myers, A. K. (1969). Effects of continuous loud noise during instrumental shock-escape conditioning. Journal of Comparative and Physiological Psychology, 68, 617–622. Nation, J. R., Wrather, D. M., & Mellgren, R. L. (1974). Contrast effects in escape conditioning of rats. Journal of Comparative and Physiological Psychology, 86, 69–73. Navarro, J. I., Hallam, S. C., Matzel, L. D., & Miller, R. R. (1989). Superconditioning and overshadowing. Learning and Motivation, 20, 130–152. O’Connell, J. M., & Rashotte, M. E. (1982). Reinforcement magnitude effects in first- and second-order conditioning of directed action. Learning and Motivation, 13, 1–25. Oppenheimer, R. (1956). Analogy in science. American Psychologist, 11, 127–135. Pavlov, I. (1927). Conditioned reflexes. Oxford, England: Oxford University Press. Piliavin, J. A., Dovidio, J. F., Gaertner, S. L., & Clark, R. D., III. (1981). Emergency intervention. New York, NY: Academic Press. Prokasy, W. F., Jr., Grant, D. A., & Myers, N. A. (1958). Eyelid conditioning as a function of unconditioned stimulus intensity and intertrial interval. Journal of Experimental Psychology, 55, 242–246. Randall, J. H. (1961). The school of Padua and the emergence of modern science. Padova, Italy: Editrice Antenore. Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychological Review, 74, 71–80.

2/16/2011 6:50:47 PM


Rescorla, R. A. (1969). Conditioned inhibition of fear resulting from negative CS-US contingencies. Journal of Comparative and Physiological Psychology, 67, 504–509. Rescorla, R. A. (1979). Conditioned inhibition and extinction. In A. Dickinson & R. A. Boakes (Eds.), Mechanisms of learning and motivation: A memorial volume to Jerzy Konorski (pp. 83–110). Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1980). Pavlovian second-order conditioning. Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1999). Summation and overexpectation with qualitatively different outcomes. Animal Learning & Behavior, 27, 50–62. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: AppletonCentury-Crofts. Schmeck, R. R. (1970). Error-produced frustration as a factor influencing the probability of occurrence of further errors. Journal of Experimental Psychology, 86, 153–156. Seta, C. E., & Seta, J. J. (1996). When more is less: An averaging/summation analysis of social anxiety. Journal of Research in Personality, 30, 496–509. Seybert, J. A., & Weiss, R. F. (1974). The negative reinforcing functions of nonconformity. Memory and Cognition, 2, 791–795. Shanks, D. R. (1985). Forward and backward blocking in human contingency judgment. Quarterly Journal of Experimental Psychology, 37B, 1–21. Shanks, D. R. (1991). On similarities between causal judgments in experienced and described situations. Psychological Science, 2, 341–350. Shanks, D. R., & Dickinson, A. (1987). Associative accounts of causality judgments. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 21, pp. 229–261). San Diego, CA: Academic Press. Shaw, M. E. (1958). Some motivational factors in cooperation and competition. Journal of Personality, 26, 155–169. Siegel, S., & Domjan, M. (1974). The inhibitory effect of backward conditioning as a function of the number of pairings. Bulletin of the Psychonomic Society, 4, 122–124. Simmel, G. (1955). Conflict and the web of group affiliations. Glencoe, IL: The Free Press.


451

Spence, J. T., & Spence, K. W. (1966). The motivational components of manifest anxiety: Drive and drive stimuli. In C. D. Spielberger (Ed.), Anxiety and behavior (pp. 291–326). New York, NY: Academic Press. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press. Spence, K. W., Farber, I. E., & McFann, H. H. (1956). The relation of anxiety (drive) level to performance in competitional and noncompetitional paired-associates learning. Journal of Experimental Psychology, 52, 296–305. Staats, A. W. (1975). Social behaviorism. Homewood, IL: Dorsey Press. Staub, E. (1978). Positive social behavior and morality: Social and personal influences (Vol. 1). New York, NY: Academic Press. Staub, E. (1979). Positive social behavior and morality: Socialization and development (Vol. 2). New York, NY: Academic Press. Staub, E., Bar-Tal, D., Karylowski, J., & Reykowski, J. (1984). Development and maintenance of prosocial behavior. New York, NY: Plenum Press. Steigleder, M. K., Weiss, R. F., Balling, S. S., Wenninger, V. L., & Lombardo, J. P. (1980). Drivelike motivational properties of competitive behavior. Journal of Personality and Social Psychology, 38, 93–104. Steigleder, M. K., Weiss, R. F., Cramer, R. E., & Feinberg, R. A. (1978). Motivating and reinforcing functions of competitive behavior. Journal of Personality and Social Psychology, 36, 1291–1301. Stich, M. H., Weiss, R. F., Cramer, R. E., & Feinberg, R. A. (1987). Reinforcing functions of altruism and leaving the scene. The Journal of Psychology, 121, 459–473. Tarpy, R. M. (1969). Reinforcement difference limen (RDL) for delay in shock escape. Journal of Experimental Psychology, 79, 116–121. Tarpy, R. M., & Koster, E. D. (1970). Stimulus facilitation of delayed-reward learning in the rat. Journal of Comparative and Physiological Psychology, 71, 147–151. Tarpy, R. M., & Sawabini, F. L. (1974). Reinforcement delay: A selective review of the last decade. Psychological Bulletin, 81, 984–997. Trapold, M. A., & Fowler, H. (1960). Instrumental escape performance as a function of the intensity of noxious stimulation. Journal of Experimental Psychology, 60, 323–326. Van Hamme, L. J., Kao, S. F., & Wasserman, E. A. (1993). Judging interevent relations: From cause

2/16/2011 6:50:47 PM

452


to effect and from effect to cause. Memory & Cognition, 21, 802–808. Van Overwalle, F., & Van Rooy, D. (1998). A connectionist approach to causal attribution. In S. J. Read & L. C. Miller (Eds.), Connectionist models of social reasoning and social behavior (pp. 143–171). Mahwah, NJ: Erlbaum. Van Overwalle, F., & Van Rooy, D. (2001). How one cause discounts or augments another: A connectionist account of causal competition. Personality and Social Psychology Bulletin, 27, 1613–1626. Van Sommers, P. (1963). Carbon dioxide escape and avoidance behavior in the brown rat. Journal of Comparative and Physiological Psychology, 56, 584–589. Wagner, A. R. (1969). Stimulus selection and a “modified continuity theory.” In G. H. Bower & J. T. Spence (Eds.), Psychology of learning and motivation (Vol. 3, pp. 1–41). New York, NY: Academic Press. Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in Pavlovian conditioning: Application of a theory. In R. A. Boakes & M. S. Halliday (Eds.), Inhibition and learning (pp. 301–336). New York, NY: Academic Press. Waterman, C. K. (1969). The facilitating and interfering effects of cognitive dissonance on simple and complex paired associates learning tasks. Journal of Experimental Social Psychology, 5, 31–42. Weiss, R. F. (1963). Defection from social movements and subsequent recruitment to new movements. Sociometry, 26, 1–20. Weiss, R. F. (1968). An extension of Hullian learning theory to persuasive communication. In A. G. Greenwald, T. C. Brock, & T. M. Ostrom (Eds.), Psychological foundations of attitudes (pp. 109–145). New York, NY: Academic Press. Weiss, R. F., Beck, C. M., & Stich, M. H. (1972). Correlated delay of reinforcement in the instrumental conditioning of conversational behavior. Psychonomic Science, 28, 211–212. Weiss, R. F., Boyer, J. L., Colwick, J. T., & Moran, D. J. (1971). A delay of reinforcement gradient and correlated reinforcement in the instrumental conditioning of conversational behavior. Journal of Experimental Psychology, 90, 33–38. Weiss, R. F., Boyer, J. L., Lombardo, J. P., & Stich, M. G. (1973). Altruistic drive and altruistic reinforcement. Journal of Personality and Social Psychology, 25, 390–400.


Weiss, R. F., Buchanan, W., Altstatt, L., & Lombardo, J. P. (1971). Altruism is rewarding. Science, 171, 1262–1263. Weiss, R. F., Cecil, J. S., & Frank, M. J. (1973). Steep delay of reinforcement gradient in escape conditioning with altruistic reinforcement. Bulletin of the Psychonomic Society, 2, 372–374. Weiss, R. F., Cluts, M. J., Williams, M. J., & Miller, F. G. (1977). Communication and conditioning: Correlated reinforcement. Bulletin of the Psychonomic Society, 10, 37–38. Weiss, R. F., Lombardo, J. P., Warren, D. R., & Kelley, K. A. (1971). Reinforcing effects of speaking in reply. Journal of Personality and Social Psychology, 20, 186–199. Weiss, R. F., McDonald, N. B., Little, C., & Shull, R. D. (1991, November). Pavlovian principles in human parallel distributed processing. Paper presented at the meeting of the Psychonomic Society, San Francisco, CA. Weiss, R. F., & Miller, F. G. (1971). The drive theory of social facilitation. Psychological Review, 78, 44–57. Weiss, R. F., Miller, F. G., Steigleder, M. K., & Denton, D. A. (1977). Drive effects on instrumental response speed induced by intermittent disagreement in conversation. Bulletin of the Psychonomic Society, 9, 5–7. Weiss, R. F., Steigleder, M. K., Cramer, R. E., & Feinberg, R. A. (1977). Delay of reinforcement and delay shifts in dyadic communication. Bulletin of the Psychonomic Society, 9, 193–196. Weiss, R. F., Weiss, J. J., Wenninger, V. L., & Balling, S. S. (1981). Sex differences in social influence: Social learning. Bulletin of the Psychonomic Society, 18, 233–236. Weiss, R. F., Williams, M. J., & Miller, C. R. (1972). Disagreement-induced drive in conversation: A social analog of intermittent shock in escape conditioning. Psychonomic Science, 29, 342–344. Williams, D. A., & Overmier, J. B. (1988). Backward inhibitory conditioning with signaled and unsignaled unconditioned stimuli: Distribution of trials across days and intertrial interval. Journal of Experimental Psychology: Animal Behavior Processes, 14, 26–35. Woods, P. J., & Holland, C. H. (1966). Instrumental escape conditioning in a water tank: Effects of constant reinforcement at different levels of drive stimulus intensity. Journal of Comparative and Physiological Psychology, 62, 403–408. Woods, P. J., Markman, B. S., Lynch, W. C., & Stokely, S. N. (1972). Partial reinforcement

2/16/2011 6:50:47 PM


effects in instrumental escape conditioning. Learning and Motivation, 3, 279–292. Woods, P. J., & Schutz, L. J. (1965). Performance in instrumental escape conditioning following a shift in drive stimulus intensity. Proceedings of the 73rd Annual Conference of the American Psychological Association, 1965, 1, 23–24. Zajonc, R. B. (1965). Social facilitation. Science, 149, 269–274. Zajonc, R. B. (1976). Family configuration and intelligence. Science, 192, 227–236.


453

Zajonc, R. B., & Mullally, P. R. (1997). Birth order: Reconciling conflicting effects. American Psychologist, 52, 685–699. Zbroznya, A. W. (1958). On the conditioned reflex of the cessation of the act of eating. I. Establishment of the conditioned cessation reflex. Acta Biologiae Experimentalis, 18, 137–162.

2/16/2011 6:50:47 PM

CHAPTER 20 The Impact of Social Cognition on Emotional Learning A Cognitive Neuroscience Perspective Andreas Olsson

In this chapter, I discuss human emotional learning in social situations, bridging the research literatures on emotional learning and social cognition. I begin by relating basic research on social-emotional learning to everyday learning outside the laboratory and to clinical applications. Then, I discuss the mechanisms underlying classical fear conditioning, which has served as a model for our current understanding of the formation of emotional associations. Next, I review relevant findings from research on the neural bases of social cognition, the study of the perception and understanding of other individuals. These lines of research will show that the neural systems involved in emotional learning and social cognition are partly overlapping, highlighting important commonalities. This will lead to a discussion about how social cognition can affect two specific forms of learning; observational fear learning in humans and other animals, and instructed fear learning in our species. Next, I will review recent work on the impact of social cognition on the learning to fear others. I will end by discussing a recently proposed neural model of social fear learning that may help us to better understand how social interactions can shape the acquisition, expression, and modification of emotional learning from and about others.

INTRODUCTION Imagine as a child you are observing your father arguing with the neighbor that just moved in next door. Pretend that the squabble escalates. Suddenly, the neighbour makes a swift move towards your parent, and you clearly see his fearful face and posture. You are freezing at the sight of your threatened relative facing the raging neighbor. Fortunately, the argument does not end in casualties. Nevertheless, the episode has a strong and long-lasting impression on you. A fear memory has been created. Although most neighborly interactions might be of the friendly and collaborative kind, such as praising the garden next door and borrowing baking powder, our close social environment is, and has been across our evolution as social

primates, a source of potentially lethal threat. Contemporary theorizing poses that threatening situations involving conspecifics have provided evolutionary pressures shaping the behavior and thus the brain of all social animals, enabling them to adaptively navigate their social and physical environment (Byrne & Whiten, 1988; Dunbar, 2003). Like many other social animals, humans are equipped with a fast and frugal ability to detect and react to conspecifics that are signaling fear, such as that in your father’s face, and aggression, such as that displayed by your neighbor. In addition to this automatic or “reflexive” ability, humans might be uniquely equipped to “read” the minds of others by means of attributing mental states, such as fear and the intention to harm, in a more controlled or “reflective” manner (Gilbert, 1998). Research on

454


2/15/2011 7:53:52 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

reflexive and reflective evaluations of social cues, such as fearful and angry faces, suggests that these processes are supported by interacting, but partially dissociable, neural networks in the brain (Adolphs, 2009; Amodio & Frith, 2006; Lieberman, 2007; Olsson & Ochsner, 2008). Interestingly, these two ways of processing social information appear to be paralleled by emotional learning mechanisms operating on output from reflexive and reflective information processing, respectively, suggesting common bases for social cognition and emotional learning. Indeed, as we will see, this is becoming increasingly clear as research on the neural bases of social cognition finds itself investigating neural networks partially overlapping with those previously known from the literature on emotional learning. In this chapter, I am going to discuss human emotional learning in social situations, bridging the literatures on social cognition and emotional learning. As we saw in the earlier example, other individuals can be both the communicating source of a threatening event (your father) and the threat itself (the neighbor). In other words, we may learn both from and about others, reflecting two key functional characteristics of individuals around us. Here, the emphasis will be on research relevant to emotional learning from others. In particular, I will discuss the learning of fear, which is the kind of emotional learning currently most extensively investigated on both the behavioral and neural level. In addition, fear learning has several important implications for the development and maintenance of psychological disorders associated with fear and anxiety. I begin by highlighting the interrelatedness of knowledge from three perspectives on fear learning: the basic experimental; the adaptive, everyday learning outside the lab; and finally, its translation into clinical applications. Then, to better understand emotional learning from others, I review relevant findings in two separate domains of research—classical conditioning and social cognition—providing insights into the basic mechanisms underlying the formation of emotionally colored associations, and perceiving and understanding other individuals, respectively. To this aim, I first summarize what is known about the mechanisms underlying classical


455

conditioning, which has served as a model for our current understanding of the formation of emotional associations. Then, I review relevant findings in the research on the mechanisms of social cognition, which concerns itself with questions related to the perception and understanding of other individuals—the ones who we are learning from and about. As this discussion will clarify, the neural systems involved in emotional learning and social cognition share many functional regions, suggesting important commonalities. This will naturally lead the discussion into specific questions of social emotional learning that brings together the literatures on emotional learning and social cognition. In particular, I will review behavioral and neurobiological studies on two forms of socially mediated fear learning: observational fear learning in humans and other animals, and instructed fear learning in our species. I will end by discussing a neural model of social fear learning that may help us to understand how social interaction can shape the acquisition, expression, and eventually modification, of emotional learning from and about others.

FROM CONDITIONING TO THE CLINIC Returning to the example in the beginning of the chapter, it becomes clear that watching someone else expressing strong fear can be a powerful learning episode shaping the observer’s future behavior. In the example, the child may acquire several bits of emotionally relevant information about the neighbor but also about the parent. First, even without experiencing the aversive qualities of the neighbor firsthand, the child is likely to develop negative evaluations, possibly also a fear, of the neighbor and the place where the episode took place. This fear might then lead to avoidance behavior, which could be adaptive. The neighbor might indeed be dangerous. However, if the child’s initial perceptions were mistaken and the neighbor in fact is friendly minded, avoidance would not be adaptive. In addition, if the fear is generalized to other, current and future, neighbors, it can cause problems. Even worse, if the fear is generalized to other grown-up males, looking like the neighbor,

2/15/2011 7:53:52 PM

456


or even creating anxiety in the presence of unknown individuals in general, it might be the beginning of a self-perpetuating vicious circle of dysfunctional anxiety and avoidance behaviors. Fears can also be acquired through verbal communication, such as hearing stories told by parents or peers. In fact, clinical research suggests that a sizable proportion of anxiety disorders, such as social anxiety and phobias, are related to fearful vicarious experiences or verbally communicated threat information (Askew & Field, 2008; Mineka & Zinbarg, 2006; Rachman, 1968). These claims are mainly based on retrospective self-reports about the origin of the fear and phobias. Self-reports are notoriously problematic because they are often impossible to verify, and research shows that fear learning can occur without the involvement of conscious awareness (Esteves, Parra, Dimberg, & Ohman, 1994). Also, the focus on etiology provides little insight into the proximate mechanisms of fear learning. Instead, much of our current understanding of the basic mechanisms believed to underlie maladaptive responses to traumatic and stressful events relies on research on classical conditioning. However, this research also has important limitations, such as its restricted ecological validity raising doubts about the applicability of conditioning research to clinical situations. Whereas most of our learned emotional responses acquired outside the research laboratory might be transmitted from fellow humans, often being about other individuals, classical conditioning in the lab is asocial, requiring firsthand experience of an aversive event, and often being about nonsocial stimuli, such as tones and colored squares. In other words, fear conditioning protocols are trading ecological validity to increase control, which reduces the generalizability of the results. However, new experimental research is trying to address these shortcomings of both clinical research and the traditional conditioning model. This research is greatly aided by the development of neuroimaging methods to study the mechanisms of the working brain, and improved techniques to present more ecological stimuli. Another important consideration related to the translation of basic research of emotional learning into more applied settings is the fact


that individuals vary greatly in their responses to emotional stimuli. In particular, they differ in both their propensity to acquire emotional responses (e.g., conditionability), and in how they are reacting to social stressors with farreaching consequences for their well-being and the likelihood of developing related psychological disorders, such as anxiety disorders and depression. There is currently a growing understanding about how variations in genetic makeup (e.g., Canli & Lesch, 2007 Hettema, Annas, Neale, Kendler, & Fredrikson, 2003; Lonsdorf et al., 2009; Munafo, Brown, & Hariri, 2008) and personality (e.g., Lissek et al., 2008; Mineka & Zinbarg, 2006; Olsson, Carmona, Bolger, Downey, & Ochsner, 2007) are related to differences in emotional learning and social reactivity, and importantly, how such interactions may contribute to disorders. Furthermore, an individual’s personal learning history is in itself an important factor contributing to making the individual more or less vulnerable to respond maladaptively to future stressors and trauma implicated in the origins of anxiety disorders (Mineka & Zinbarg, 2006 See also chapter 3, this volume). To enhance the understanding of the underlying mechanisms mediating the impact of genetic vulnerabilities on dysfunctional responses to stressful events, there is a growing interest in research on neural responses to emotionally and socially evocative stimuli in individuals with specific genetic and personality markers (Bishop, 2008; Canli & Lesch, 2007; Munafo et al., 2008). On the whole, the consideration of the factors discussed here; the ecological validity of experimental manipulations and individual differences in learning, will likely lead to a deeper, as well as a more applicable, understanding of the etiology and maintenance of common psychological disorders.

DIFFERENT PROCEDURES, SAME UNDERLYING PROCESSES? One fundamental question related to social fear learning is whether the processes critical to classical conditioning are the same, or similar, to those supporting the acquisition of fear and anxiety from and about others. In other words, are

2/15/2011 7:53:52 PM


these different learning procedures drawing on the same underlying learning processes? If the associative processes that are at the core of classical conditioning are also involved in forming social fear learning, then, much information can be gained about learned emotions outside the lab—be they normal or dysfunctional—by studying classical conditioning. For example, well-explored conditioning phenomena, such as latent inhibition, unconditioned stimulus (US) devaluation, and extinction (i.e., phenomena that can reduce the size of the conditioned response [CR]; see Chapter 1, this volume), would then be expected to constrain observational and instructed fear just as they do with conditioned fear. The extinction of learned fear responses is a particularly good illustration, because it is governed by principles that have been used to successfully guide behavioral therapies in treating phobias and other anxiety disorders for decades. In addition, recent efforts to develop extinction protocols both in combination with (Davis, Barad, Otto, & Southwick, 2006) and without (Monfils, Cowansage, Klann, & LeDoux, 2009) pharmacological treatment raise hopes for improved outcomes. Indeed, research over the last 40 years has shown that fear learning from others by observation display several commonalities with fear learning through direct personal experience, classical conditioning, providing a validation of the conditioning model (Askew & Field, 2008; Bandura, 1977; Bandura & Menlove, 1968; Hygge & Ohman, 1978; Mineka, Davidson, Cook, & Keir, 1984; Mineka & Zinbarg, 2006; Olsson & Phelps, 2007; also see Green & Osborne, 1985). However, some earlier work (Berber, 1962; see also Lanzetta & Englis, 1989) and a more recently emerging line of research on the neural aspects of observational fear learning (Hooker, Germine, Knight, & D’Esposito, 2006; Olsson, Nearing, & Phelps, 2007) suggests that in order to understand socially mediated fear learning, the conditioning model needs to be complemented by recent developments in social cognition and social cognitive neuroscience (Olsson & Ochsner, 2008; Olsson & Phelps, 2007). Also research on learning through verbal communication suggests that this form of


457

learning draws on similar mechanisms as learning by observation and conditioning (Phelps et al., 2001). However, as will be discussed in greater detail in later sections, this form of symbolically mediated learning also displays important differences. The relevance of the conditioning model is also pertinent to learning about others. In the former example, the child gains emotionally relevant information about the individuals present in the situation, about the impulsive nature and possibly malicious intentions of the neighbor and the parent’s anxious disposition. These pieces of information might confirm or update existing expectancies based on previous experiences of the parent or provide new diagnostic information about the neighbor. Recent research suggests that the principles guiding emotional learning about other individuals are echoing known principles of classical conditioning, such as dependence on prediction errors (Delgado, Li, Schiller, & Phelps, 2008). Just as in the case of learning from others, research has only begun to understand how emotional learning about others is affected by social cognition in normal, healthy individuals (Navarrete et al., 2009; Olsson, Ebert, Banaji, & Phelps, 2005; Singer et al., 2006; Todorov, Said, Engell & Oosterhof, 2008), and individuals showing deviations in social cognitive performance, such as an increased (Kross, Egner, Ochsner, Hirsch, & Downey, 2007; Lissek et al., 2008; Mineka & Zinbarg, 2006) or decreased (Blair, Colledge, Murray, & Mitchell, 2001) anxiety in social settings.

CLASSICAL CONDITIONING IN THE BRAIN Most of our knowledge about the neurobiological mechanisms of emotional learning comes from research on classical conditioning. In a traditional fear-conditioning procedure, a neutral conditioned stimulus (CS) is paired with a naturally aversive stimulus (US), leading to a conditioned fear response to the CS. The extensive use of this procedure since Pavlov (1927) has established classical conditioning as a model of fear learning (Phelps & LeDoux, 2005). Consistencies

2/15/2011 7:53:52 PM

458


in the physiological expression of conditioned fear elicited by the basic protocol indicate that mechanisms of emotional learning are analogous across species. The Role of the Amygdala in Classical Conditioning

Research on the neurobiology of fear conditioning has focused on the amygdala, a nut-shaped structure in the bilateral medial temporal lobes and a key structure in the brain’s fear circuitry (Figs. 20.1a, 20.1b, 20.2a). Although the mygdala processes a wide range of emotionally relevant information, much of its anatomy and functional role in fear conditioning has been conserved throughout evolution (LeDoux, 2000). The amygdala is composed of several subnuclei, some of which serves specific functions in fear conditioning. In short, sensory information is believed to arrive in the lateral nucleus from centers in the thalamus and sensory cortices (Amaral, 1986; LeDoux, Farb, & Ruggiero, 1990). The lateral nucleus also receives nocioceptive information specific for the US, providing a biological basis for the convergence of CS-US information (Fig. 20.2a). Supporting this conclusion, research suggests that this is the locus of synaptic plasticity shaping associations between representations of the CS and US (Blair, Schafe, Bauer, Rodrigues, & LeDoux, 2001; Quirk, Armony, & LeDoux, 1997; Romanski, Clugnet, Bordi, & LeDoux, 1993). The lateral nucleus then relays information to the central nucleus and basal nucleus that mediates the output to other regions that regulate the expression of fear and anxiety (LeDoux & Gorman, 2001). For example, projections to the hypothalamus are important for mediation of autonomic responses (Price & Amaral, 1981), which in humans can be measured through the skin conductance response (Davis & Whalen, 2001). Other areas of projection, such as the ventral tegmental area (Simon, Le Moal, & Calas, 1979) and the central gray (Hopkins & Holstege, 1978), serve roles in regulation of behavioral expressions of fear. Another fear-related behavior, avoidance, is mediated by input to the basal ganglia from the basal nucleus (Everitt & Robbins, 1992). The striatum, which is the input region of


Figure 20.1 Amygdala activity during condi-

tioned and socially learned fear. (a) The outlined box contains the region of the medial temporal lobe that includes the bilateral amygdala. (b–d) Amygdala activation to the CS is seen bilaterally after fear conditioning (b) and observational fear learning (c), and unilaterally (d) in the left amygdala after instructed fear. Reprinted from Olsson, A. & Phelps, E. A. (2007). Social Learning of Fear. Nature Neuroscience, 10, 1095–1102.

the basal ganglia, is known for signaling prediction errors during reward learning (Schultz, Dayan, & Montague 1997), but it has also been increasingly implicated in aversive learning (Delgado et al., 2008). As will be discussed later, the interconnectivity both within and between the amygdala and other subcortical regions, such as the striatum, appears to have important implications for social-emotional learning.

2/15/2011 7:53:52 PM

a Conditioned fear

Cortically distributed (AI, ACC, hipp.) representation of the CS US CS-US/pairing

CE B

LA

Visual cortex CS

Primary (SI) and secondary (SII) somatosensory cortex

US

Visual thalamus Somatosensory thalamus CS

US

Autonomic output

b Observational fear Cortically distributed (AI, ACC, hipp.) representation of the CS-US/pairing CS

MPFC: mentalizing

ACC, AI Empathetic emotion

CE B

Visual cortex

Visual cortex

CS

Visual thalamus

Visual thalamus

Social US

CS

Autonomic output

c Instructed fear Left hemisphere: language representation

Threat

Left hemisphere:Cortically distributed (AI, ACC, hipp.) representation of the CS-US/pairing CS Threat

CE B

LA

Visual cortex CS

Visual thalamus CS

Autonomic output

Figure 20.2 A Neural Model of Social Fear Learning. The arrows describe the flow of information between different functional brain regions. Although the arrows point only in one direction, the connectivity might be bidirectional. (a) Fear conditioning occurs by associating the visual representation of the CS with the somatosensory representation of the aversive US. The lateral nucleus (LA), in which sensory representations (Continued) 459


2/15/2011 7:53:52 PM

460


Dissociating Amygdala and Hippocampal Function in Classical Conditioning

As hinted at earlier, the role of the amygdala in classical fear conditioning is best understood together with other functional regions within a greater circuitry of fear learning. This neural network involves sensory input and motor output systems, as well as regions that contribute to explicit and conscious aspects of learning and expression of fear. For example, the hippocampus, a medial temporal lobe structure adjacent to the amygdala, is critical for coding contextual information about the fear learning situation, such as relationships between different features and the timing of events (Eichenbaum & Cohen, 2002). This indicates that whereas the amygdala is responsible for developing associations between somatosensory states and representations of individual stimuli (cue learning), the hippocampus appears to encode relations between the various cues that comprise the learning context (contextual learning). Patients with bilateral and unilateral amygdala damage can verbally report the CS-US contingency, but they lack the normally associated autonomic response (LaBar, LeDoux, Spencer, & Phelps, 1995), suggesting that the amygdala is necessary only for implicit, nonverbal processes underlying acquisition and expression of learned fear. In contrast, the hippocampus is essential for consolidation and retention of explicit or declarative memory of the CS-US contingency (Bechara et al., 1995) and the environmental contexts that regulate conditioned fear responses (LaBar & Phelps, 2005). These distinctions will prove to be of

importance in the discussion about different kinds of social-emotional learning. The Frontal Cortex in Classical Conditioning

The anterior insula (AI) and the anterior cingulate cortex (ACC) are two cortical regions consistently implied in aversive conditioning studies (Büchel, Morris, Dolan, & Friston, 1998). These two regions both receive ascending viscerosensory inputs (Decety & Jackson, 2004; Gallese, Keysers, & Rizzolatti, 2004; Iacoboni & Dapretto, 2006; Morrison et al., 2004; Singer et al., 2004; Zaki, Ochsner, Hanelin, Wager, & Mackey, 2007). The AI has a role in the awareness of an external threat (e.g., the US and the conditioned CS), and it is believed to support affective experience in part through interoceptive awareness of somatosensory inputs (Craig, 2009; Critchley, Wiens, Rotshtein, Ohman, & Dolan, 2004). The ACC is thought to code affective attributes of pain, such as the perceived unpleasantness (as opposed to sensory-discriminative properties, such as location and intensity) (Eisenberger, Lieberman, & Williams, 2003; Hutchison, Davis, Lozano, Tasker, & Dostrovsky, 1999) and motivate appropriate behavior, such as avoidance, through projections to motor and autonomic centers (Critchley et al., 2004). Across species, the prefrontal cortex (PFC) is a major player in the regulation of conditioned and other affective responses through its impact on the activation in subcortical regions, such as the amygdala (Ochsner & Gross, 2005; Robbins, 2005). In particular, the ventral (infralimbic)

Figure 20.2 (Continued)

of the CS and US converge, is believed to be the site of learning. The amygdala also receives input from the hippocampal memory system (hipp.), anterior insula (AI) and anterior cingulate cortex (ACC) containing secondary representations of the CS and US, information about the learning context and the internal state of the organism. (b) In observational fear learning, the visual representation of the CS is modified by its association with a representation of the distressed other, serving as the US. As in fear conditioning, it is hypothesized that representations of the CS and the US converge in the LA. The strength of the US may be modified by MPFC input related to the interpretation of the other’s mental state, as well as cortical representations of empathic pain through the ACC and AI. (c) Instructed fear learning occurs by modifying the processing of the visual representation of the CS through its association with an abstract representation of threat. Instead of being coded in the amygdala, the CS–’threat’ US contingency is likely to be represented in a cortically distributed network, critically depending on the hippocampal memory system. Reprinted from Olsson, A. & Phelps, E. A. (2007). Social Learning of Fear. Nature Neuroscience, 10, 1095–1102.


2/15/2011 7:53:53 PM


region of the medial prefrontal cortex (mPFC) is necessary for the retention of extinction of conditioned fear responses in rats (Quirk, Garcia, & Gonzalez-Lima, 2006), and the human homolog of this region is involved in extinction in humans (Phelps, Delgado, Nearing, & LeDoux, 2004). Strategic regulation of affective expressions by purposeful interpretation (appraisal) of the emotional meaning of a given situation involves more dorsal and lateral regions of the PFC. For example, the dorsolateral prefrontal cortex (dlPFC) has been assigned a key role in the up- and down-regulation of affective responses to images through appraisal strategies by their impact on amygdala activity (Ochsner & Gross, 2008; Ochsner, Ray, et al., 2004). Consistent with these findings, a recent study reported that subjects using appraisal strategies recruited the dlPFC to down-regulate their conditioned fear responses and the accompanying amygdala activation to the CS (Delgado et al., 2008). Interestingly, this study also showed activation in the mPFC that overlapped with regions previously implicated in studies of extinction of conditioned fear in humans, arguing that there are both similarities (mPFC) and differences (dlPFC) during passive (extinction) and active (reflective) regulation of learned emotional responses. In this section, I have discussed several functional brain regions that are involved in classical conditioning. Some of these (e.g., the amygdala) support the reflexive and implicit aspects of emotional learning. Other regions, such as the hippocampus and the PFC, support explicit and more controlled forms of emotional learning. As we will see in the next section, many of the regions highlighted here are also involved in the processing of social cognitions, hinting toward common functional bases.

SOCIAL COGNITION IN THE BRAIN Recent advances in brain imaging and work with brain-lesioned patients have dramatically increased our understanding about the social brain. For example, as will be discussed in greater detail later, the amygdala has been implicated in


461

the initially fast and frugal processing of social cues (e.g., your parent’s fearful face) signaling the presence of a potential threat (e.g., the neighbor), thus allowing for a quick adaptive response. In contrast, the mPFC has been implicated in the reflective attribution of mental states (e.g., fearfulness to your parent and harmful intentions to your neighbor). The dual involvement of these and other regions in both nonsocial emotional learning and social cognition might be explained by the fact that the social cues triggers learning and/or that some more basic neural computations of motivational relevance contribute to both kinds of psychological functions. The Role of the Amygdala in Social Cognition

The amygdala has long been assigned a key role in social functioning. Since the seminal findings by Kluver and Bucy (1939), reporting severe impairments in a variety of social behaviors in monkeys following bilateral temporal lobectomy, this structure has become one of the most studied regions in a neural network now known to support the perception and evaluation of social stimuli. Although the amygdala has an ancient evolutionary past, its interconnectedness to neocortex has increased substantially in primates. The basolateral complex in the primate amygdala has strong reciprocal connections to visual cortex, in particular to the inferotemporal region that responds to face identity and to facial expression (Kanwisher & Yovel, 2006; Rolls, Tovée, Purcell, Stewart, & Azzopardi, 1994). Moreover, the basolateral complex has direct connections to the ventral part of the mPFC and indirectly with more dorsal regions of the mPFC (Barton, Aggleton, & Grenyer, 2003; Young, Scannell, Burns, & Blakemore, 1994). Interestingly, these observations corroborate the proposal that the primate amygdala may be particularly prone to form associations between more complex socioemotional stimuli, especially when they are visually represented. The chief role of the amygdala in social cognition appears to be the orchestration of quick responses to potentially dangerous stimuli and events in the social environment in order to

2/15/2011 7:53:53 PM

462


disambiguate its meaning (Adolphs, 2009; Adolphs et al., 2005; Whalen, 2007). Indeed, no aspect of our environment might be more ambiguous than its social domain. Through learning about our fellow humans, they may be become predictable to a certain degree, but never certain. For example, the swift alternation of facial expressions, signaling a change from benevolent to adversary intentions in a conspecific illustrates the rapidly unfolding of events that may have fatal implications for the individual if not responded to adaptively. In response to such environmental challenges, a neural system for rapid detection of potentially harmful cues in the environment, centered on the amygdala, has evolved (Adolphs, 2009; de Gelder, Snyder, Greve, Gerard, & Hadjikhani, 2004; LeDoux, 2000 Ohman & Mineka, 2001; Whalen et al., 1998). Consistent with the assumption that its role in social cognition is to rapidly respond to potentially dangerous and ambiguous stimuli, amygdala activation to faces and body postures can influences early visual and attentional processing (Anderson & Phelps, 2001; Morris, Ohman & Dolan, 1998a; Phelps, Ling, & Carrasco, 2006; Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004) and action representations (de Gelder et al., 2004). In addition to the earlier discussed role of the amygdala in the implicit responding to presentations of nonsocial conditioned stimuli, research shows that social stimuli may be especially suited to access this brain system, even in the absence of conscious awareness of their presence (see Chapter 18, this volume). For example, a series of studies by Ohman and colleagues has demonstrated conditioned responses as measured by physiological arousal to subliminally presented (and reportedly not seen) images of angry faces that were previously paired with a shock (Ohman & Mineka, 2001). Imaging studies employing a similar technique to subliminally present facial stimuli have highlighted the role of the amygdala (Morris, Ohman & Dolan, 1998b; Whalen et al., 1998). Many of the studies on the role of the amygdala in social cognition have suggested that this neural structure is particularly sensitive to facial stimuli expressing fear (Adolphs, 2009;


Whalen et al., 1998), consistent with the assumption that fearful faces are particularly ambiguous relative to the location of the threat. However, it is now clear that the amygdala responds to a much wider range of stimuli and situations. For example, it has been shown to be interested in other facial expressions (Adolphs, 2009), untrustworthy versus trustworthy looking faces (Winston, Strange, O’Doherty, & Dolan, 2002) and to neutral faces belonging to a racial group other than the observer’s (Cunningham et al., 2004; Harris & Fiske, 2006; Hart et al., 2000; Lieberman, Hariri, Jarcho, Eisenberger, & Bookheimer, 2005; Phelps et al., 2000), and it is modulated by contextual information about social cues (Kim et al., 2003; Ochsner, Ray, et al., 2004). Based on a lesion study, Adolphs (2009) argues forcefully that the amygdala contributes to the generation of actions that can disambiguate the situation, such as guiding the direction of attention toward cues that are specifically diagnostic for the emotional significance of a situation. Finally, some studies argue for a more elaborate role of the amygdala during development to enable the attributions of mental states, such as intentions, desires, and emotions to other individuals (Baron-Cohen et al., 2000). Taken together, existing research on the role of the amygdala in social cognition suggests that it may serve several functions of exploring salient cues in the inherently interactive social environment, which demands rapidly concerted responding of different social cognitive, emotional processes and motor actions. The fact that many of these social cues and events may be of critical importance for the individual to remember makes the amygdala a likely hub interconnecting neural circuitries supporting emotional learning and social cognition. The Role of the Frontal Cortex in Social Cognition

In addition to the amygdala, several other brain regions are implicated in a distributed functional network supporting social cognition. Here, the focus of the discussion will be on cortical regions that have already been reviewed in relation to

2/15/2011 7:53:53 PM


emotional learning, in particular the anterior insula (AI), anterior cingulate cortex (ACC), and medial prefrontal cortex (mPFC). In addition, there are many other, more posteriorly located, cortical regions that have been consistently associated with social cognition and that are likely to serve various functions in socialemotional learning through their direct or indirect connectivity with the amygdala and other regions important for emotional learning, but less is known about their functions in such learning tasks. To this group of regions that will not be discussed here in any detail belongs the superior temporal sulcus (STS), which is known to be involved in the integration of information about body movements and higher levels of processing (Castelli, Frappe, Frith, & Frith, 2000), and the right temporal-parietal junction (rTPJ), which is implicated in mental state attributions about true or false beliefs to others (Saxe, Moran, Scholtz, & Gabrieli, 2006). Social cognition, including the understanding of emotional experiences of other individuals through the attribution of mental states (e.g., fear), is likely to be aided by the brain’s system for dual representations of one’s own and others’ experiences. This overlap between one’s own and others’ mental states enables an internal simulation in terms of firsthand experiential understanding of the others’ emotions and intentions, which aids the prediction of their actions. Evidence for such a system initially emerged from imaging studies showing that certain motor regions respond during both the execution and observation of specific movements (Gallese, Keysers, & Rizzolatti, 2004). It was argued that if motor regions code the intentions behind one’s own action, then if activated when observing another individual engaging in the same action, they might support an understanding of that individual’s intention through simulation of his or her experiences (Blakemore & Decety, 2001; Gallese, Keysers, & Rizzolatti, 2004). This “dual representation” logic has guided a host of studies on the direct experience and observation of pain or emotion that also show activation of overlapping neural systems, including most prominently the AI and the ACC.


463

The engagement of these regions is thought to facilitate the automatic sharing of, and hence direct experiential understanding of, others’ affective states, thus providing a substrate for empathy (e.g., with your fearful parent) (Decety & Jackson, 2004; Singer et al., 2004). In turn, these processes are likely to affect the way we learn in the situation. In addition to the automatic sharing of experiences through simulation, such as that described in the dual-representation literature, more reflective attributions of mental states may be required to understand another individual’s emotional state. These controlled attributions allow us to purposefully take other peoples’ perspectives and make judgments about their emotions or diagnostic elements of their emotional dispositions, thereby changing empathic responding (Batson, Thompson, & Chen, 2002) and recruitment of the anterior insula and the ACC (Lamm, Batson, & Decety, 2007). These reflective processes have been shown to depend on a network of regions of the dorsal mPFC, including Brodmann area 10 (BA 10), and the right temporo-parietal junction (rTPJ; Mitchell, Macrae, & Banaji, 2006; Ochsner, Ray, et al., 2004; Saxe et al., 2006). Interestingly, some of the same regions involved in reflecting upon another individual’s emotional state are also involved in reflecting upon our own emotions (Ochsner, Ray, et al., 2004; Saxe et al., 2006), consistent with the conjecture that we sometimes treat ourselves as an “other” when making self-judgments (Ochsner, Ray, et al., 2004). Of course, the reverse might also be true. In other words, we might use information about our own mental states and traits when we reflect upon the mental states and traits of others, especially if these others seem to be similar to ourselves. Further supporting this idea, judgments about known or similar, as compared with less familiar or dissimilar, others draw on medial frontal regions overlapping with those used for self-referential processing (Mitchell et al., 2006; Ochsner, Ray, et al., 2004). Strikingly, recent research shows that taking others’ perspective can increase the self-other overlap by shifting the recruitment of the prefrontal activation more ventrally (Ames, Jenkins, Banaji, & Mitchell, 2008).

2/15/2011 7:53:53 PM

464


Throughout the last two sections, I have discussed a range of brain regions belonging to the widely distributed neural network supporting both rapid and reflexive, and more reflective social cognitions. These regions are either directly overlapping with, or closely connected with, the neural network responsible for emotional learning that was reviewed earlier. Indeed, processes computed through the interaction between these two networks will be important for the understanding of social-emotional learning and its neural bases. This will be discussed next.

LEARNING FROM OTHERS As I have been arguing above, the individual neural regions involved in classical conditioning and social cognition can only be fully understood by recognizing their interactions with other functional regions in the greater neural circuitries to which they belong. Analogous to this, the full understanding of an individual’s learning experiences in a natural environment requires the understanding of the intricate social-interactive environment in which it occurs. Similar to classical conditioning, the transmission of fear signals through social channels is well documented in a range of species (Hauser, 1996). The ability to detect and respond appropriately to signs of fear and pain in a conspecific probably has conferred significant selective advantage during evolution. However, these social cues not only alert the receiver about potential imminent dangers, as I have discussed earlier, but they also assign a threat value to cues or the context that are associated with the threat display. For example, a conspecific’s (e.g., your parent’s) fear expression may serve as a US, eliciting immediate reflexive aversive response in the observer that becomes associated with the paired stimuli (e.g., the neighbor and the location where the episode unfolds). However, observational learning may also be subserved by social inference, in which the conspecific’s fear expression is a CS that was previously associated with a directly experienced aversive event (US) and may act as a secondary punisher in future learning. To continue with the earlier example, you may have come to associate your parent’s


fear face with the pain you felt when you cut yourself with a knife at an earlier time. Now, you are inferring the negative value of your neighbor through his association with your parent’s expression. Although these alternative views (i.e., emotional faces eliciting an unconditioned or conditioned response in the observer) are not mutually exclusive, existing research supports the former view. Our ultrasocial environment provides ample opportunities to watch and learn from others’ emotional responses to both social and nonsocial stimuli (Bandura, 1977; Rachman, 1977). In addition to this ability, which we share with other animals, our unique linguistic ability enables us to acquire information about emotional qualities through verbal communication. In 1968, Rachman proposed that observation and instruction, along with classical conditioning, constitute three main pathways to the development of fears (Rachman, 1968). Indeed, subsequent research has shown that they all can produce strong and persistent fear learning in humans and that they all can be related to the development of psychological disorders (Askew & Field, 2008; Mineka & Zinbarg, 2006; Olsson & Phelps, 2007). Next, I will first review research on observational fear learning in humans and non-human animals, followed by the literature on instructed fear learning. Observational Fear Learning in Non-Human Animals

Observational fear learning has been studied in many species, including birds (Curio, 1988), mice (Kavaliers, Choleris, & Colwell, 2001), cats (John, Chesler, Bartlett, & Victor 1968), cows (Munksgaard, DePassille, Rushen, Herskin, & Kristensen, 2001), and primates (Berber, 1962; Hygge & Ohman, 1978; Mineka & Cook, 1993; Mineka, Davidson, Cook, & Keir, 1984; Olsson & Phelps, 2004; Olsson et al., 2007; Vaughan & Lanzetta, 1980). In a study on mice (Kavaliers et al., 2001), model mice were attacked by biting flies while observer mice watched. When exposed 24 hours later to biting flies, whose biting parts had been removed, the model and observer mice expressed conditioned analgesia

2/15/2011 7:53:53 PM


and avoidance responses to similar degrees, implying that individual and social fear learning were equally effective. The strength of the model’s fear response during its own learning episode was not correlated with expressed fear learning in the observer at a later test. However, such a relationship was found during observational fear learning in rhesus monkey (Mineka & Cook, 1993), indicating that there may be a greater reliance on the model’s emotional expressions during the learning process in primates. Indeed, this conjecture is supported by the rich and flexible musculature of the primate face, especially in humans, allowing us to produce a greater variety of emotional expressions, superior to that of other species (Ekman, 1982). The cortical areas dedicated to face processing are also relatively enlarged in primates (Rolls, 1999), implying an augmented reliance on facially transmitted emotional information. In monkeys (Mineka & Cook, 1993; Mineka et al., 1984) and humans (Gerull & Rapee, 2002; Olsson & Phelps, 2004; Olsson & Phelps, 2007; Vaughan & Lanzetta, 1980) facial fear expression appears to serve as a reliable US. For example, in an important series of studies by Mineka and colleagues, cage-reared monkeys were shown either live presentations or movies of model monkeys reacting fearfully to snakes (toy or real) or to non–fear-relevant objects (Mineka & Cook, 1993; Mineka et al., 1984). When fear-relevant objects were used, the relationships between the strength of a learning model’s expressed distress, the observer’s immediate emotional response to the model’s distress, and the learned fear in the observer as measured at a later time were similar to the known relationships between the US, unconditioned response (UR), and CR in classical fear conditioning (Mineka & Cook, 1993; Mineka et al. 1984). This single social encounter with a fearful fellow monkey produced a strong and robust fear response that was measured several months after the encounter (Mineka & Cook, 1993). These findings strongly indicate that observational fear learning draws on the same underlying processes as fear conditioning. Still, evidence for what neural processes are supporting observational fear learning in non-human animals is lacking.


465

Observational Fear Learning in Humans

In a study inspired by the experimental work in monkeys by Mineka and colleagues (e.g., 1984), Gerull and Rapee (2002) showed children rubber snakes and rubber spiders together with their mothers’ facial expressions of either fear or happiness. Following the observation of the fear faces as compared to the happy face, the children showed behavioral expressions of fear and stimulus avoidance both at a direct test and at a later time, corroborating the findings of Mineka and colleagues in humans. Indeed, children with subclinical animal phobias or extreme fears toward certain situations, such as darkness, often report having observed parents fearful in the same or similar situations (Bandura & Menlove, 1968; Mineka & Zinbarg, 2006). Interestingly, a more recent follow-up study showed that exposure to positive maternal modeling, prior to fearful modeling as in the Gerull and Rapee study (2002), prevented the acquisition of fear during negative modeling (Egliston & Rapee, 2007; see also a similar “immunization” effect in monkeys, Mineka & Cook, 1986). Exposure to the stimulus alone did not have the same effect, suggesting that latent inhibition was not effective in this observational paradigm. In another line of studies, Field and colleagues have been using self-reported fear beliefs as an index of fear learning in children who were exposed to the combination of fearful faces and novel stimuli (Askew & Field, 2008; Field, Argyris, & Knowles, 2001). Although retrospective reports of emotional states and preferences are notoriously problematic (Johansson et al., 2005; Mineka & Zinbarg, 2006), these studies support the conclusion that observational fear learning in children is both strong and persistent (in some cases observed 3 months post observation) and appear to share various characteristics with classical conditioning. Also adults learn to fear through observing others. Indeed, over the last half a century, behavioral and psychophysiology research has produced consistent evidence of strong and persistent fear learning through social observation in adults (Bandura, 1977; Hygge & Ohman, 1978; Olsson & Phelps, 2004; Vaughan &

2/15/2011 7:53:54 PM

466


Lanzetta, 1980, but see Green & Osborne, 1985). A study directly comparing observational and instructed fear learning with classical fear conditioning supported this conclusion. Olsson and Phelps (2004) demonstrated fear learning of equal magnitude following the three ways of learning when the CSs were presented subliminally (seen). Strikingly, when CSs were presented subliminally (and reportedly not seen), learned fear responses were only displayed in the conditioning and observational learning groups. This supported the idea that similar learning processes and representations underlie observational and conditioned fear. These claims have since been substantiated by imaging research. Olsson and colleagues (2007) asked subjects to watch a movie of another person expressing distress when receiving electric shocks paired with a CS. Later, subjects expected to receive shocks along with the same stimulus that was paired with the model’s distress in the movie they just had watched. Importantly, no shocks were administered to the subjects during the test stage to ensure that their representation of the US-CS pairing was based solely on indirect, vicarious experiences. The results showed that, similar to previous fear-conditioning studies, the bilateral amygdala was involved during both the learning (observation) and the subsequent expression (test) of learned fear, strongly supporting the assumption that similar associative mechanisms and their underlying neural processes support both conditioned and observational fear learning (Figs. 20.1a–c). Similar conclusions can be drawn from an experiment by Hooker and colleagues (2006). In this study, subjects watched images of emotional faces (fearful and happy) alone or together with arbitrary figures (learning objects). These results showed that the amygdala was more responsive to faces when their emotional expressions were associated with a learning object than presentations of the emotional expressions or objects alone, pointing toward the specific role of facial expressions as a source of emotional learning. These lines of evidence and the rich connectivity between the amygdala and visual and ventral parts of the mPFC as discussed earlier,


indicates that, at least in primates, representation of fear learning through observation and classical conditioning may be rather similar within the amygdala. However, despite the many features shared between conditioned and observational fear, nonsocial and social forms of learning must differ in several fundamental ways, implying involvement of partially dissociable neural networks outside the amygdala. For example, a conspecific’s expression of distress may be naturally aversive and serve as a US eliciting an immediate unconditioned response in the observer that becomes associated with a CS. However, this response is also mediated by the observer’s perception of the model, which can be influenced by various social cognitive processes, such as emotional perspective taking and mental attributions. Indeed, in one of the first studies of observational fear learning, Berber (1962) showed that another person’s arm movement in response to a shock acted as an US, but only when the observer believed that it was caused by a shock, not when the model’s arm moved without a shock or when a shock was delivered without arm movements. These findings support the conclusion that perceptual properties of the learning model interact with the observer’s understanding of the model’s (the demonstrator’s) mental states to instigate an unconditioned response. Similarly, information about another person’s spider phobia can induce an aversive response to a spider that is presented to the allegedly phobic model, even without any physical cues of distress (Hygge & Ohman, 1978), and the affective response in an observer can be modified by social context (Lanzetta & Englis, 1989; Singer et al., 2006). A recent psychophysiology study supports the idea that mental state attributions (e.g., the experience of pain) to the model during observation can causally impact the observer’s subsequent responses to the CS (Olsson, Ochsner, & Phelps, 2007). Mental state attributions may, in turn, be dependent on social factors, such as familiarity, relatedness, social status, and interpersonal learning history. Indeed, mice observing a familiar, but not an unfamiliar, mouse experiencing pain displayed enhanced sensitization to pain on a later test (Langford et al., 2006).

2/15/2011 7:53:54 PM


The intrinsic aversiveness of observing a conspecific in pain is evidenced by the willingness of monkeys to starve themselves if a shock is administered to a fellow monkey every time the observer attempts to eat (Masserman, Wechkin, & Terris, 1964); but again, this altruistic behavior is influenced by familiarity and past experience of the conspecific (Hauser et al., 2003; Masserman, Wechkin, & Terris, 1964). Relating back to the example at the inception of this chapter, the child might have had a different learning experience if it instead watched a fearful other, such as your neighbor, especially if the threat was your raging father. For sure, in such an alternative scenario, the child would have learned something quite differently both from and about the parent. Taken together, available research highlights both similarities and differences between conditioned and observational fear learning. Although some studies on observational learning in rats have failed to replicate various phenomena that are predicted from the principles guiding classical conditioning, including blocking, overshadowing, and latent inhibition (Bennet, Galef & Durlach, 1993), such phenomena have been replicated in primate observational learning (Bandura, 1968; Lanzetta & Orr, 1980; Mineka & Cook, 1993). It is possible that social-emotional learning shows greater interspecies variability than does classical conditioning. In addition, the greater reliance on well-developed systems for perceiving and signaling emotions through facial expressions in primates might make observational fear learning more similar to conditioning in primates as compared to other social species. Two Interacting Pathways in Observational Fear Learning

The studies reviewed so far suggest that fear learning through observation is supported by two interacting pathways mediating classical conditioning and social cognition, respectively. The role of conditioning is highlighted by work on observational fear learning in primates (Mineka & Cook, 1993; Olsson & Phelps, 2004; Olsson et al., 2007), showing that a conspecific’s


467

expression of distress can be intrinsically aversive, which suggests that somatosensory representations may be reflexively triggered by mere observation of another individual’s emotional display without necessarily being accompanied by higher order social cognition. The same conclusion is supported by the demonstration of unconscious behavioral mimicry (Dimberg, Thunberg, & Elmehed, 2000) and physiological responses to subliminally presented faces (Ohman & Mineka, 2001; Whalen et al., 1998). Finally, dual-representation models of emotion perception and empathy in humans as described earlier (Dimberg, Thunberg, & Elmehed, 2000; Gallese, Keysers, & Rizzolatti, 2004; Preston & de Waal, 2002) provide additional support of the conditioning account. The importance of social cognition is underscored by the fact that factors related to the social context, shared representations of emotional states, and reflections about others’ mental states may moderate the ensuing learning. Indeed, as described earlier, affective responses to emotional faces and their recruitment of the amygdala depend on the context provided (Kim et al., 2004) and on cognitive appraisals by means of prefrontal brain systems (Lamm et al., 2007; Ochsner & Gross, 2005). Moreover, basic emotional responses to another’s distress are affected by interpersonal learning history and the goals of the observer. For example, an observer’s affective response to another’s distress depends on whether the other person is expected to cooperate or compete in a future interactive game (Lanzetta & Englis, 1989), and imaging work shows that brain regions, such as the AI and ACC, are modulated accordingly (Singer et al., 2006). These changes in affective response to another’s distress mediated by social cognition are also likely to influence the ensuing learning. Providing preliminary support of this conclusion, an imaging study on observational fear learning (Olsson et al., 2007, Figs. 20.3a, b) found activation in the AI and ACC both during observation of another person receiving shocks paired with a CS and in the later test stage when the person being imaged expected to receive shocks paired with the same stimulus, indicating that regions linked to empathy may be involved

2/15/2011 7:53:54 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS Conditioned response

468

3.0

b

3.0

2 r = 0.61

1

1.0 0.0 –2.0 0.0

2.0 1.0 0.0 –1.0

r = 0.67

2.0

0.0 1.0 Betas

2.0



a

2.0 4.0 Betas

6.0

3.0 r = 0.49 2.0 1.0 0.0 –2.0

0.0 2.0 Betas

4.0

Figure 20.3 Brain activation during observational fear learning predicts the level of expressed fear at a

subsequent time. Functional activation maps displaying brain activation during an observational fear learning task (Olsson, Nearing, & Phelps, 2007); (c) a coronal view of activation in the right AI when observing a learning model’s pain response to a shock. The adjacent graph shows that the magnitude of this activation predicts the strength of the conditioned response (indexed by the skin conductance response) at a later time to a cue associated with the learning model’s pain. (d) A saggital view of activation in the (1) mPFC and (2) ACC during the observation a learning model’s pain response to a shock. As in (c), adjacent graphs display the positive relationship between magnitude of activation during observation and subsequent conditioned response. Modified from Olsson, A & Ochsner, K. N. (2008). The Relationship Between Emotion and Social Cognition. Trends in Cognitive Science, 12, 65–71.

in observational fear learning. Important to this context, activation in both these regions (AI and ACC) during observation predicted learning as expressed in the subsequent test stage. In addition, another region of interest, the rostral mPFC, was only activated during the observation stage. Responses in this region marginally predicted the magnitude of subsequent learning, further indicating that social cognition is involved in observational learning of fear. The conjecture that mental state attributions can be causally related to the learning response following observation was strengthened by a recent study demonstrating that experimental manipulation of the observers’ empathic appraisals through increasing or decreasing empathy with a distressed learning model (demonstrator) affected later learning responses (Olsson, Brodbeck, Bolger, & Ochsner, 2008). To sum up, in accordance with research on non-human animals, the findings of observational fear learning in humans demonstrate, on the one hand, reflexive processes independent of conscious awareness and strategic regulation of affective responses and, on the other, reflective


processes dependent on social-cognitive manipulations. It is also clear that these two different levels of learning interact. For example, the amygdala, supporting reflexive responses, interacts with frontal cortical mechanisms of shared affective representations and hippocampal representations about context and relevant social information about the learning model (such as social status and familiarity). In addition, these regions receive social-cognitive information from prefrontal cortices, such as the mPFC. Interestingly, although in rodents the ventral mPFC has a role in some social behaviors (Schneider & Koch, 2005), the primate mPFC is likely to be more important in social perception and learning, as shown by deficits in social behavior after prefrontal lesions in both monkeys (Myers, Swett, & Miller, 1973) and humans (Beer, Heerey, Keltner, Scabini, & Knight, 2003). It is worth noting that the more anterior-rostral region of the mPFC is both quantitatively and qualitatively more developed in humans than other primates (Ongür, Ferry, & Price, 2003), implying the intriguing possibility of a unique neural substrate for the support of more complex

2/15/2011 7:53:54 PM


mental representations that might be involved in human observational learning. In fact, a metaanalysis of imaging studies reports that this region (specifically Brodmann’s area 10) is especially sensitive to experimental manipulations involving both social and emotional tasks (Gilbert et al., 2006). Instructed Fear Learning in Humans

Humans possess the unique ability to obtain emotional information through language. Whereas fear learning through observation involves visual representation of emotional properties of a stimulus, language is only arbitrarily related to, and thus detached from, its referent in the world. Language forces the receiver to rely on past experiences and internally generated imagery to form an emotional memory. In this process, brain regions involved in linguistic processing are likely to play an important role. In addition, imagery and self-projection into the future are thought to rely on neural systems similar to those involved in perception (Kosslyn & Thompson, 2003) and the construction of episodic memory (Tulving, 2002). Indeed, similar to the recollection of the past, self-projection into the future is impaired after hippocampal lesions (Hassabis, Kumaran, Vann, & Maguire, 2007). In addition, regions of the mPFC implicated in the simulation of future events (Buckner & Carroll, 2007; Schacter & Addis, 2007) overlap with those involved in thinking about others’ minds. These findings suggest that hippocampal and frontal regions involved in explicit and reflective processes are especially important during indirect, social forms of learning. Both clinical accounts that retrospectively target the etiology of phobic fears (King, Gullone, & Ollendick, 1998) and experimental studies on children involving fear provoked through storytelling (Askew & Field, 2008; Field, Argyris, & Knowles, 2001) reveal that verbal instructions can be a strong stimulus for fear learning. Indeed, adults instructed to expect a shock paired with a specific CS and later exposed to the same CS show learned responses similar to those seen after classical fear conditioning (Grillon, Ameli,


469

Merikangas, Woods, & Davis, 1991; Hugdahl & Ohman, 1977; Olsson & Phelps, 2004; Phelps et al., 2001). These lines of research suggest that, at least partially, instructed fear learning draws on the same neural network as conditioned fear. To examine the neural mechanisms underlying expression of fears acquired through verbal instruction, Phelps and colleagues (Phelps et al., 2001) told subjects that they might receive a shock when shown a particular CS (“threat” stimulus), but not another CS (“safe” stimulus). Supporting extension of the fear-conditioning model to instructed fear, there was robust activation of the left amygdala, which correlated with the physiological expression of fear learning (Fig. 20.1d). Activation of the left insular cortex also correlated with expression of learning. As discussed earlier, the insular cortex is a critical component for conveying a cortical representation of pain to the amygdala (Shi & Davis, 1999) and for subjective awareness of physiological states (Critchley et al., 2004). The verbally mediated learning is likely to have resulted in an abstract cortical representation of the potentially painful shock, which may have been communicated to the amygdala through projections from the insular cortex (Fig. 20.2c). The left lateralization of the activation is consistent with the common view that the left hemisphere is more involved in language processing (Gazzaniga & LeDoux, 1978). However, brain imaging results cannot rule out involvement of the right amygdala, or indicate a critical role for the left amygdala in expression of fears learned through verbal instruction. Further support that the left amygdala mediates physiological expression of instructed fear learning was demonstrated in subjects with unilateral amygdala damage after a similar learning protocol. Those with damage to the left, but not right, amygdala showed an impaired expression of instructed fear (Funayama, Grillon, Davis, & Phelps, 2001). Taken together, these results suggest that there might be both common and unique mechanisms involved in instructed and conditioned fear learning. In an attempt to directly compare fears acquired through conditioning, observation, and verbal instruction, Olsson and Phelps (2004)

2/15/2011 7:53:54 PM

470


manipulated the learning procedure, keeping other factors constant. Conditioned stimuli acquired their threat value through being paired with either (1) a shock (conditioning group); (2) an observed fear expression in a learning model (observational group); or (3) the experimenter’s verbal instructions (instructed group). Fear responses to the CS were of comparable magnitude after the three kinds of learning. In addition, replicating previous findings (Ohman & Mineka, 2001), a subliminally presented (unperceived) CS triggered a response in the fear-conditioning group. The observational, but not the verbally instructed, group also showed a learning response to subliminal presentations of the CS, further supporting the common basis for conditioned and observational fear learning, and indicating that a different mechanism may support learning through language. These results support the notion that there are partially dissociable systems involved in different modes of social-emotional learning. Classical conditioning and observational learning, which humans share with many other species, might be supported by an evolutionarily old system that predates the emergence of language. In contrast, learning based on language is unique to humans and is likely to be, at least initially, dependent on representations in cortical areas that also support conscious processes. Indeed, these findings indicate that such cortically represented fear associations might depend on conscious awareness, which is in accordance with the observation that the manipulation of conscious awareness can be used to distinguish subdivisions of conditioning (e.g., context and trace conditioning, but not cue and delayed conditioning, are dependent on awareness).

LEARNING ABOUT OTHERS In this chapter, I have focused on the mechanisms underlying learning from others through observation and verbal communication. However, as announced at the outset, we are also frequently learning about others. Indeed, a good share of our waking hours are spent learning emotionally important information about others, either indirectly, for example, through


gossiping, or directly through interaction (Dunbar, 2003). In the fearful episode at the outset of this chapter, the child acquired emotionally significant information about both the parent (e.g., anxious disposition) and the neighbor (e.g., aggressive disposition). How does this kind of learning differ from learning about, for instance, a colored square or a tone? In light of the discussion about how social cognition affects our learning from someone else, it is likely that such cognitions also affect the way we develop likes or dislikes of our fellow humans. The two types of social cognitive processes (reflexive and reflective) that support learning from others also provide crucial diagnostic information about others’ stable social-emotional dispositions (e.g., aggressive personality) as well as their current intentions (e.g., the intention to harm). Consider a social interaction that unfolds over time. During the initial moments of contact, stimulus-driven systems might assess the affective value of social targets in a reflexive manner. For example, the rapid evaluation of either potentially threatening and untrustworthy or attractive faces activates either the amygdala and AI regions implicated in aversive learning, as described earlier (Adolphs, 2009; Cunningham et al., 2003; Todorov, Said, & Engell, 2008; Winston et al., 2002) or striatal regions implicated in reward and reinforcement learning (O’Doherty et al., 2004). Other social categories, such as gender, age, and racial belonging are also rapidly coded (Greenwald et al., 2002). This initial processing is followed by more reflective processing of the social target in terms of mental state attributions. These and other ways of categorizing other individuals based on social categories may also have an impact on how we learn about them. Interestingly, a line of research that has until recently not been connected to the study on social cognition, has shown that all CS are not created equal in the sense that certain stimuli are more easy to aversively condition than others. For example, studies on so-called prepared fear conditioning have found that conditioned responses to certain natural categories of fear-relevant stimuli (e.g., snakes and angry faces) as compared to fear-irrelevant stimuli (e.g., birds and happy faces) resist extinction

2/15/2011 7:53:54 PM


(Cook, Hodes, & Lang, 1986; Ohman & Mineka, 2001; see also Chapter 18, this volume). These observations, combined with the superior fear conditioning observed in non-human animals to certain types of ecologically relevant stimuli, has led researchers to posit that these particular stimuli may be prepared by evolution to engage in aversive associations (Ohman & Mineka, 2001; Seligman, 1971). These lines of work led Olsson and colleagues (Olsson et al., 2005) to ask whether similarly biased learning processes might support the acquisition of certain social group biases frequently reported in research on social cognition (Greenwald et al., 2002). Using classical conditioning, they showed that conditioned fear responses to an unknown, neutral looking male belonging to a racial outgroup versus ingroup was more resistant to extinction (Olsson et al., 2005). These results held true for both African American and Caucasian subjects who displayed similar results. A follow-up study showed that this conditioning bias is found only when male outgroup faces are used as CS (Navarrete et al., 2009). Of course, the affective value of a social target is also determined both by their dispositions and by situational variables (Gilbert, 1998). Turning back to the example at the outset of this chapter, the identical behavior of your neighbor might be taken as aggressive or playful depending on your assessment of his intent. In turn, such interpretations might affect the way the episode is learned from and later remembered. Supporting this prediction, a recent study using male faces demonstrated a similarly persistent conditioned fear response to facial images belonging to individuals believed to intentionally as compared to unintentionally inflict harm to the subject through delivering a mild electric shock (Olsson et al., 2008). This and other suggestive findings (Gray & Wegner, 2008; Young, Cushman, Hauser, & Saxe, 2007) indicate that attributions about intent can also impact how we develop fears of others. In many of the studies discussed thus far, the subjects were learning to fear others through the pairing with the direct presentations of an electro-tactile US (shock). However, when we learn about others, we often do this through the consequences of their behavior during an interaction.


471

With repeated interaction with other individuals, imaging studies suggest that the striatum might encode which actions produce desired social outcomes, consistent with its general role in reinforcement learning (Delgado, 2007; O’Doherty et al., 2004). In such interactive games, activity in the striatum accompanies the development of both cooperation and trust (Delgado, 2007; Delgado, Frank, & Phelps, 2005; King-Casas et al., 2005; Sanfey, Loewenstein, McClure, & Cohen, 2006), but it is also involved in punishing a previously unfair partner (Quervain, Fischbacher, Treyer, Schellhammer, Schnyder, Buck, 2004) or learning that they are in pain (Singer et al., 2006). More recent work has begun to test learning models combining knowledge of reinforcement learning and mental state attributions to describe the development of strategic interactions (Hampton, Bossaerts, & O’Doherty, 2008). In summary, the amygdala is likely to be involved in the rapid, reflexive processing of stimuli-based categories, such as sex and race cues, followed by the involvement of ventral regions of the frontal cortex during mental state attributions (e.g., my neighbor is angry), and the striatal activation to regulate moment-to-moment behavior beyond the first impression (e.g., I should avoid him). In contrast, more dorsal regions of the mPFC might support explicit reflections about the consequences of actions, as evidenced by its activation during strategic games (Quervain, Fischbacher, Treyer, Schellhammer, Schnyder, Buck, 2004; Gallagher, Jack, Roepstorff & Frith, 2002).

A NEURAL MODEL OF SOCIAL FEAR LEARNING Social-emotional learning offers the opportunity to study transmission of biologically relevant information between individuals. Indeed, some have argued that social learning at large may lie at the core of the forces that create and maintain culture (Plotkin & Odling-Smee, 1981; Whiten, Horner, & de Waal, 2005), which might then have an impact on biological evolution (Danchin, Giraldeau, Valone, & Wagner, 2004; Plotkin & Odling-Smee, 1981). A model of

2/15/2011 7:53:54 PM

472


social-emotional learning based on existing knowledge of the neurobiology of classical conditioning and social cognition might serve as an important bridge between biological principles of learning and cultural evolution. In addition, such a model might also provide an ecologically improved account to support the understanding of how clinical fears and anxiety develop and are maintained. Next, I will discuss a suggestive model of social-emotional learning originally proposed by Olsson and Phelps (2007). This framework outlines the relationship between neural mechanisms underlying fear conditioning and two forms of social learning—observational and instructed fear—and how these might be modulated by both reflexive and reflective processes. The model is centered on the amygdala, which is critical to physiological expression of learned fear, regardless of how learning is acquired. As outlined earlier, in classical fear conditioning (Fig. 20.2a), information about the CS is communicated to the lateral nucleus of the amygdala by way of the sensory cortices and thalamus; this information converges with US input from the somatosensory cortex and thalamus. Through synaptic plasticity in the lateral nucleus, the CS-US association is formed. In addition, a distributed cortical representation of the CS-US contingency is acquired through the hippocampal memory system and may be expressed in regions associated with pain processing, such as the ACC and insular cortex. In the presence of the CS, learned fear is expressed through projections from the lateral nucleus to the central nucleus, which in turn mediates autonomic expression. This might be the way the child’s parent in the example acquires a fear of the neighbor—through direct, firsthand aversive experiences. Other means of expression may depend on other pathways (LeDoux & Gorman, 2001). In addition, projections from the cortical representation of the CS-US contingency to the amygdala may contribute to autonomic expression of fear learning when there is subjective awareness of the CS-US contingency. The model further proposes that the mechanisms underlying learning through social observation (Fig. 20.2b) may be similar, with a few exceptions.


For example, the US in observational fear learning may be the perceived fear expression of a conspecific, such as that in the attacked parent. A representation of the fearful face is conveyed to the lateral nucleus through the sensory cortices and possibly the sensory thalamus. Importantly, the representation of the strength of the US in the lateral nucleus may be modified by cortical representation of empathic pain through input from the ACC and insular cortex and the perception and interpretation of the learning model’s mental state during the observed painful experience as supported by the mPFC. The model suggests that, similar to classical fear conditioning, the lateral nucleus is a site of plasticity underlying memory for the CS-US association, in addition to a distributed cortical representation of the CS-US association acquired through the hippocampal memory system. The output mechanism for observational fear learning does not differ from that for fear conditioning. Finally, the model poses that fears that are acquired through verbal communication (Fig. 20.2c) rely on a somewhat different representation, because of its symbolic nature. It is unlikely that abstract representations of verbal threat are represented in subcortical structures, such as the amygdala. Although sensory information about the CS is conveyed to the lateral nucleus, the association between the CS and the verbal threat is likely to be represented solely in a distributed cortical network. Furthermore, this cortical representation is leftlateralized, reflecting the verbal nature of the threat. In accordance with the model, memory for this cortical association depends on the hippocampal complex for acquisition, and plasticity in the amygdala is not necessary. Still, autonomic expressions of instructed fears are modulated through communication of the cortical representation of the CS-US association and the potential for pain to the amygdala, perhaps by way of the insular cortex. As with other means of fear learning, the central nucleus is believed to mediate autonomic expression of instructed fear. Following the same logic as in the model outlined here, the representation of a social CS, such as the appearance of your neighbor in the initial

2/15/2011 7:53:54 PM


example, is likely to be affected by social cognitive processes. It is, for example, possible that brain regions involved in the rapid categorization of your neighbor in terms of sex, race, and trustworthiness assign an initial valence to him as a social CS. Then, these initial evaluations are likely to be modified trough further learning experiences by the influence of regions supporting reflective social cognition, such as the attribution of intent, and in the case of social interaction, brain regions supporting reinforcement learning. As highlighted by Olsson and Phelps (2007), the presented model remains speculative and should be viewed in light of some important caveats. First, the striatum is not highlighted in the model. Human brain-imaging studies on both conditioned (Büchel, Morris, Dolan, & Friston, 1998; LaBar, Gatenby, Gore, LeDoux, & Phelps, 1998) and social (Olsson et al., 2007; Phelps et al., 2001) fear learning report activation of the striatum. As we have seen, this region is also shown to be involved in learning through social interaction (Hampton et al., 2008; O’Doherty et al., 2004). Second, the model highlights unidirectional projections between brain regions, but most of the regions we have discussed have bidirectional connections with the amygdala. Third, this framework outlines how fear learning is initially expressed after social and nonsocial means of acquisition. Once a CS is experienced and a fear reaction occurs, further learning may result, which could change the nature of the representation further. For example, in instructed fear, co-occurrence of the CS and autonomic arousal may cause the CS to act as a secondary reinforcer, which projects its emotional salience to the lateral nucleus to facilitate an amygdala-dependent representation of the CS-threat association that was not present after initial verbal instruction. In this way, representation of verbally communicated fears may change over time and be experienced to be more similar to conditioned fears. Despite these caveats, the proposed framework represents a neural model that may help us to better understand the complexity and subtlety of human fear learning in a social and cultural environment.


473

CONCLUDING REMARKS In this chapter, I have attempted to bring together relevant lines of work on the behavioral and neurobiological level from two domains of research: emotional (fear) learning and social cognition with the aim to better understand emotional learning in social situations in which we learn from and about others. As we have seen, brain regions in the two greater networks supporting emotional learning and social cognitions are partially overlapping, suggesting important commonalities. Investigations directly targeting social-emotional learning have mostly validated classical conditioning as a model for emotional learning through social means. However, in light of the existing literature, I have argued for an extension of this model to account for the socialcognitive aspects of social-emotional learning. To this aim, the model by Olsson and Phelps (2007) provides a first, tentative step that needs further validation through continued work. Future research on social-emotional learning will also hopefully provide important knowledge about the underlying socio-emotional impairments that are hallmarks of many psychological disorders, such as phobias and anxiety disorders, characterized by dysfunctional assignment of emotional value to certain stimuli and situations. A next important step will be to improve our understanding of how dysfunctional learning can be changed. In other words, how can we help individuals who have developed maladaptive responses to their social environment due to adverse emotional experiences, be they caused by war, abusive parents, or raging neighbors? On a wider scale, a better understanding of the neural mechanisms supporting social-emotional learning is essential to integrate our current knowledge about the biological foundations of learning with an understanding of cultural change and evolution at large.

REFERENCES Adolphs, R. (2009). The social brain: Neural basis of social knowledge. Annual Review of Psychology, 60, 693–716. Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005).

2/15/2011 7:53:54 PM

474


A mechanism for impaired fear recognition after amygdala damage. Nature, 433, 68–72. Amaral, D. G. (1986). Amygdalohippocampal and amygdalocortical projections in the primate brain. Advances in Experimenta Medicine and Biology, 203, 3–17. Ames, D. L., Jenkins, A. C., Banaji, M. R., & Mitchell, J. P. (2008). Taking another person’s perspective increases self-referential neural processing. Psychological Science, 19, 642–644. Amodio, D. M., & Frith, C. D. (2006). Meeting of minds: The medial frontal cortex and social cognition. Nature Reviews Neuroscience, 7, 268–277. Anderson, A.K., & Phelps, E.A. (2001). Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature, 411, 305–309. Askew, C., & Field A. P. (2008). The vicarious learning pathway to fear 40 years on. Clinical Psychology Review, 28, 1249–1265. Bandura, A. (1977). Social learning theory. New York, NY: General Learning Press. Bandura, A., & Menlove, F. L. (1968), Factors determining vicarious extinction of avoidance behavior through symbolic modeling. Journal of Personality and Social Psychology, 8, 99–108. Baron-Cohen, S., Ring, H. A., Bullmore, E. T., Wheelwright, S., Ashwin, C., & Williams, S. C. (2000). The amygdala theory of autism. Neuroscience and Biobehavioral Reviews, 24, 355–364. Barton, R. A., Aggleton, J. P., & Grenyer, R. (2003). Evolutionary coherence of the mammalian amygdala. Proceedings of the Royal Society of London B: Biological Sciences, 270, 539–543. Batson, C. D., Thompson, E. R., & Chen, H. (2002). Moral hypocrisy: Addressing some alternatives. Journal of Personality and Social Psychology, 83, 330–339. Bechara, A., Tranel, D., Damasio, H., Adolphs, R., Rockland, C., & Damasio, A. R. (1995). Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans. Science, 269, 1115–1118. Beer, J. S., Heerey, E. A., Keltner, D., Scabini, D., & Knight, R. T. (2003). The regulatory function of self-conscious emotion: Insights from patients with orbitofrontal damage. Journal of Personality and Social Psychology, 85, 594–604. Bennet, G., Galef, B. G., & Durlach, P. J. (1993). Absence of blocking, overshadowing, latent inhibition in social enhancement of food


preferences. Animal Learning and Behavior, 21, 214–220. Berber, S. M. (1962). Conditioning through vicarious instigation. Psychology Review, 69, 450–466. Bishop, S. J. (2008). Neural mechanisms underlying selective attention to threat. Annals of the New York Academy of Science, 1129, 141–152. Blair, H. T., Schafe, G. E., Bauer, E. P., Rodrigues, S. M., & LeDoux, J. E. (2001). Synaptic plasticity in the lateral amygdala: A cellular hypothesis of fear conditioning. Learning and Memory, 8, 229–242. Blair, R. J., Colledge, E., Murray, L., & Mitchell, D. G. (2001). A selective impairment in the processing of sad and fearful expressions in children with psychopathic tendencies. Journal of Abnormal Child Psychology, 29, 491–498. Büchel, C., Morris, J., Dolan, R. J., & Friston, K. J. (1998). Brain systems mediating aversive conditioning: An event-related fMRI study. Neuron, 20, 947–957. Buckner, R. L., & Carroll, D. C. (2007). Selfprojection and the brain. Trends in Cognitive Sciences, 11, 49–57. Byrne, R. W., & Whiten, A. (1988). Machiavellian intelligence. Oxford, England: Oxford University Press. Canli, T., & Lesch, K. P. (2007). Long story short: The serotonin transporter in emotion regulation and social cognition. Nature Neuroscience, 10, 1103–1109. Castelli, E., Happe, F., Frith, U., & Frith, C. (2000). Movement and mind: A functional imaging study of perception and interpretation of complex intentional movement. NeuroImage, 12, 314–325. Cook, E. W., Hodes, R. L., & Lang, P. J. (1986). Preparedness and phobia: Effects of stimulus content on human visceral conditioning. Journal of Abnormal Psychology, 95, 195–207. Craig, A. D. (2009). How do you feel now? The anterior insula and human awareness. Nature Reviews Neuroscience, 10, 59–70. Critchley, H. D., Wiens, S., Rotshtein, P., Ohman, A., & Dolan, R. J. (2004). Neural systems supporting interoceptive awareness. Nature Neuroscience, 7, 189–195. Cunningham, W. A., Johnson, M. K., Gatenby, J. C., Gore, J. C., & Banaji, M. R. (2003). Neural components of social evaluation. Journal of Personality and Social Psychology, 85, 639–649.

2/15/2011 7:53:54 PM


Cunningham, W. A., Johnson, M. K., Raye, C. L., Gatenby, J., Gore, J. C., & Banaji, M. R. (2004). Separable neural components in the processing of Black and White faces. Psychological Science, 15, 806–813. Curio, E. (1988). Cultural transmission of enemy recognition by birds. In T. R Zentall & B. G. Galef, Jr. (Eds), Social learning: Psychological and biological perspectives (pp. 75–97). Hillsdale, NJ: Erlbaum. Danchin, E., Giraldeau, L. A., Valone, T. J., & Wagner, R. H. (2004). Public information: From nosy neighbors to cultural evolution. Science, 305, 487–491. Davis, M., Barad, M., Otto, M., & Southwick, S. (2006). Combining pharmacotherapy with cognitive behavioral therapy: Traditional and new approaches. Journal of Traumatic Stress, 19, 571–581. Davis, M., & Whalen, P. J. (2001). The amygdala: Vigilance and emotion. Molecular Psychiatry, 6, 13–34. Decety, J., & Jackson, P. L. (2004). The functional architecture of human empathy. Behavioral and Cognitive Neuroscience Reviews, 3, 71–100. de Gelder, B., Snyder, J., Greve, D., Gerard, G., & Hadjikhani, N. (2004). Fear fosters flight: A mechanism for fear contagion when perceiving emotion expressed by a whole body. Proceedings of the National Academy of Sciences USA, 101, 16701–16706. de Quervain, D. J., Fischbacher, U., Treyer, V., Schellhammer, M., Schnyder, U., Buck, A., et al. (2004). The neural basis of altruistic punishment. Science, 305, 1254–1258. Delgado, M. R., Frank, R. H., & Phelps, E. A. (2005). Perceptions of moral character modulate the neural systems of reward during the trust game. Nature Neuroscience, 8, 1611–1618. Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008). The role of the striatum in aversive learning and aversive prediction errors. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 363, 3787–3800. Dimberg, U., Thunberg, M., & Elmehed, K. (2000). Unconscious facial reactions to emotional facial expressions. Psychological Science, 11, 86–89. Dunbar, R. (2003). Psychology. Evolution of the social brain. Science, 302,1160–1161. Egliston KA & Rapee RM. (2007). Inhibition of fear acquisition in toddlers following positive modelling by their mothers. Behaviour research and therapy, 45, 1871–182.


475

Ekman, P. (1982). Emotion in the human face. New York, NY: Cambridge University Press. Eichenbaum, H., & Cohen, N. J. (2002). From conditioning to conscious recollection: Memory systems of the brain. Oxford, England: Oxford University Press. Eisenberger, N. I., Lieberman, M. D., & Williams, K. D. (2003). Does rejection hurt? An FMRI study of social exclusion. Science, 302, 290–292. Esteves, F., Parra C., Dimberg U., & Ohman A. (1994). Nonconscious associative learning: Pavlovian conditioning of skin conductance responses to masked fear-relevant facial stimuli. Psychophysiology, 31, 375–385. Everitt, B. J., & Robbins, T. W. (1992). Amygdalaventral striatal interactions and reward-related processes. In J. P. Aggleton (Ed.), The amygdala: Neurobiological aspects of emotion, memory, and mental dysfunction (pp. 401–429). New York, NY: Wiley-Liss. Field, A. P., Argyris, N. G., & Knowles, K. A. (2001). Who’s afraid of the big bad wolf: A prospective paradigm to test Rachman’s indirect pathways in children. Behavioral Research Therapy, 39, 1259–1276. Funayama, E.S, Grillon, C., Davis, M., & Phelps, E.A. (2001). A double dissociation in the affective modulation of startle in humans: Effects of unilateral temporal lobectomy. Journal of Cognitive Neuroscience, 13, 721–729. Galef, B. G., & Laland, K. N. (2005). Social learning in animals: Empirical studies and theoretical models. BioScience, 55, 489–499. Gallese, V., Keysers, C., & Rizzolatti, G. (2004). A unifying view of the basis of social cognition. Trends in Cognitive Sciences, 8, 396–403. Gallagher, H. L., Jack, A. I., Roepstorff, A., & Frith, C. D. (2002). Imaging the intentional stance in a competitive game. Neuroimage, 16, 814–821. Gazzaniga, M. S., & LeDoux, J. E. (1978). The integrated mind. New York, NY: Plenum Press. Gerull, F. C., & Rapee, R. M. (2002). Mother knows best: Effects of maternal modeling on the acquisition of fear and avoidance behavior in toddlers. Behaviour Research and Therapy, 40, 279–287. Gilbert, D. T. (1998). Ordinary personology (4 ed.). New York: McGraw Hill. Gilbert, S. J., Spengler, S., Simons, J. S., Steele, J. D., Lawrie, S. M., Frith, C. D., & Burgess, P. W. (2006). Functional specialization within rostral

2/15/2011 7:53:54 PM

476


prefrontal cortex (area 10): A meta-analysis. Journal of Cognitive Neuroscience, 18, 932–948. Gray, K., & Wegner, D. M. (2008). The sting of intentional pain. Psychological Science, 19, 1260–1262. Green, G., & Osborne, J. G. (1985). Does vicarious instigation provide support for observational learning theories? A critical review. Psychological Bulletin, 97, 3–77. Greenwald, A. G., Banaji, M. R., Rudman, L. A., Farnham, S. D., Nosek, B. A., & Mellott, D. S. (2002). A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept. Psychaology Review 109. 3–25. Grillon, C., Ameli, R., Merikangas, K., Woods, S. W., & Davis, M. (1991). Fear-potentiated startle: Effects of anticipatory anxiety on the acoustic blink reflex. Psychophysiology, 28, 588–595. Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2008). Neural correlates of mentalizing-related computations during strategic interactions in humans. Proceedings of the National Academy of Sciences USA, 105, 6741–6746. Harris, L. T., & Fiske, S. T. (2006). Dehumanizing the lowest of the low: Neuroimaging responses to extreme out-groups. Psychological Science, 17, 847–853. Harris, L. T., Todorov, A., & Fiske, S. T. (2005). Attributions on the brain: Neuro-imaging dispositional inferences, beyond theory of mind. NeuroImage, 28, 763–769. Hassabis, D., Kumaran, D., Vann, S. D., & Maguire, E. A. (2007). Patients with hippocampal amnesia cannot imagine new experiences. Proceedings of the National Academy of Sciences USA, 104, 1726–1735. Hauser, M. D. (1996). The evolution of communication. Cambridge, MA: MIT Press. Hauser, M. D., Chen M. K., Chen, F., & Chuang, E. (2003). Give unto others: Genetically unrelated cotton-top tamarin monkeys preferentially give food to those who altruistically give food back. Proceedings of the Royal Society of London B: Biological Sciences, 270, 2363–2370. Hettema, J. M., Annas, P., Neale, M. C., Kendler, K. S., & Fredrikson, M. (2003). A twin study of the genetics of fear conditioning. Archives of General Psychiatry, 60, 702–708. Heyes, C. M. (1993). Imitation without perspective-taking. Behavioral and Brain Sciences, 16, 524–525. Hooker, C. I., Germine, L. T., Knight, R. T., & D’Esposito, M. (2006). Amygdala response to


facial expressions reflects emotional learning. Journal of Neuroscience, 26, 8915–8922. Hopkins, D. A., & Holstege, G. G. (1978). Amygdaloid projections to the mesencephalon, pons and medulla oblongata in the cat. Experimental Brain Research 32, 529–547. Hutchison, W. D., Davis, K. D., Lozano, A. M., Tasker, R. R., & Dostrovsky, J. O. (1999). Painrelated neurons in the human cingulated cortex. Nature Neuroscience, 2, 403–405. Hygge, S., & Ohman, A. (1978). Modeling processes in the acquisition of fears: Vicarious electrodermal conditioning to fear-relevant stimuli. Journal of Personality and Social Psychology, 36, 271–279. Iacoboni, M. Dapretto, M. (2006). The mirror neuron system and the consequences of its dysfunction. Nature Review Neuroscience, 7, 942–951. John, E. R., Chesler, P., Bartlett, F., & Victor, I. (1968). Observation learning in cats. Science, 59, 1489–1491. Johansson, P., Hall, L., Sikstrom, S., & Olsson, A. (2005). Failure to detect mismatches between intention and outcome in a simple decision task. Science, 310, 116–119. Kanwisher, N., & Yovel, G. (2006). The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society London B: Biological Sciences, 361, 2109–2128. Kavaliers, M., Choleris, E., & Colwell, D. D. (2001). Learning from others to cope with biting flies: Social learning of fear-induced conditioned analgesia and active avoidance. Behavioral Neuroscience, 115, 661–674. Keller, M., Perrin, G., Meurisse, M., Ferreira, G., & Levy, F. (2004). Cortical and medial amygdala are both involved in the formation of olfactory offspring memory in sheep. European Journal of Neuroscience, 20, 3433–3441. Kim, H., Somerville, L. H., Johnstone, T., Polis, S., Alexander, A. L., Shin, L. M., & Whalen, P. J. (2004). Contextual modulation of amygdala responsivity to surprised faces. Journal of Cognitive Neuroscience, 16, 1730–1745. King, N. J., Gullone, E., & Ollendick, T. H. (1998). Etiology of childhood phobias: Current status of Rachman’s three pathways theory. Behaviour Research and Therapy 36, 297–309. King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., & Montague, P. R. (2005). Getting to know you: Reputation and

2/15/2011 7:53:54 PM


trust in a two-person economic exchange. Science, 308, 78–83. Klüver, H., & Bucy, P. (1939). Preliminary analysis of functioning of the temporal lobes in monkeys. Archives of Neurological Psychiatry, 42, 979–1000. Kosslyn, S. M., & Thompson, W. L. (2003). When is early visual cortex activated during visual mental imagery? Psychological Bulletin, 129, 723–746. Kross, E., Egner, T., Ochsner, K., Hirsch, J., & Downey, G. (2007). Neural dynamics of rejection sensitivity. Journal of Cognitive Neuroscience, 19, 945–956. LaBar, K. S., Gatenby, J. C., Gore, J. C., LeDoux, J. E., & Phelps, E. A. (1998). Human amygdala activation during conditioned fear acquisition and extinction: A mixed-trial fMRI study. Neuron, 20, 937–945. LaBar, K. S., LeDoux, J. E., Spencer, D. D., & Phelps, E. A. (1995). Impaired fear conditioning following unilateral temporal lobectomy in humans. Journal of Neuroscience, 15, 6846–6855. LaBar, K. S., & Phelps, E. A. (2005). Reinstatement of conditioned fear in humans is context dependent and impaired in amnesia. Behavioral Neuroscience, 119, 677–686. Lamm, C., Batson, C. D., & Decety, J. (2007). The neural substrate of human empathy: Effects of perspective-taking and cognitive appraisal. Journal of Cogntive Neuroscience, 19, 42–58. Langford, D. J., Crager, S. E., Shehzad, Z., Smith, S. B., Sotocinal, S. G., Levenstadt, J. S., Chanda, M. L., Levitin, D. J., & Mogil J. S. (2006). Social modulation of pain as evidence for empathy in mice. Science, 312, 1967–1970. Lanzetta, J. T., & Englis, B. G. (1989). Expectations of cooperation and competition and their effects on observers’ vicarious emotional responses. Journal of Personality and Social Psychology 56, 543–554. Lanzetta, J. T., & Orr, S. P. (1980). Influence of facial expressions on the classical conditioning of fear. Journal of Personality and Social Psychology 39, 1081–1087. LeDoux, J. E. (2000). Emotion circuits in the brain. Annual Review of Neuroscience, 23, 155–184. LeDoux, J. E., & Gorman, J. M. (2001). A call to action: Overcoming anxiety through active coping. American Journal of Psychiatry, 158, 1953–1955. LeDoux, J. E., Farb, C., & Ruggiero, D. A. (1990). Topographic organization of neurons in the


477

acoustic thalamus that project to the amygdala. Journal of Neuroscience, 10, 1043–1054. Lieberman, M. D. (2007). Social cognitive neuroscience: A review of core processes. Annual Review of Psychology, 58, 259–289. Lissek, S., Levenson, J., Biggs, A.L., Johnson, L.L., Ameli, R., Pine, D.S., & Grillon, C. (2008). Elevated fear conditioning to socially relevant unconditioned stimuli in social anxiety disorder. American Journal of Psychiatry, 165, 124–132. Lonsdorf, T. B., Weike, A. I., Nikamo, P., Schalling, M., Hamm, A. O., & Ohman, A. (2009). Genetic gating of human fear learning and extinction: Possible implications for geneenvironment interaction in anxiety disorder. Psychological Science, 20, 198–206. Masserman, J. H., Wechkin, S., & Terris, W. (1964). “Altruistic” behavior in rhesus monkeys. American Journal of Psychiatry, 121, 584–585. Mineka, S., & Cook, M. (1986). Immunization against the observational conditioning of snake fear in rhesus monkeys. Journal of Abnormal Psychology, 95, 307–318. Mineka, S., & Cook, M. (1993). Mechanisms involved in the observational conditioning of fear. Jounral of Experimental Psychology: General, 122, 23–38. Mineka, S., Davidson, M., Cook, M., & Keir, R. (1984). Observational conditioning of snake fear in rhesus monkey. Journal of Abnormal Psychology, 93, 355–372. Mineka, S., & Zinbarg, R. (2006). A contemporary learning theory perspective on the etiology of anxiety disorders: It’s not what you thought it was. American Psychologist, 61, 10–26. Mitchell, J.P., Heatherton, T.F., & Macrae, C.N. (2002). Distinct neural systems subserve person and object knowledge. Proceedings of the National Academy of Sciences USA, 99, 15238–15243. Mitchell, J. P., Macrae, C. N., & Banaji, M. R. (2006). Dissociable medial prefrontal contributions to judgments of similar and dissimilar others. Neuron, 50, 655–663. Monfils, M. H., Cowansage, K. K., Klann, E., & LeDoux, J. E. (2009). Extinction-reconsolidation boundaries: Key to persistent attenuation of fear memories. Science, 324, 951–955. Morris, J. S., Ohman, A., & Dolan, R. J. (1998). Conscious and unconscious emotional learning in the amygdala. Nature, 393, 467–470. Morrison, I., Lloyd, D., di Pellegrino, G., & Roberts, N. (2004). Vicarious responses to pain

2/15/2011 7:53:55 PM

478


in anterior cingulate cortex: is empathy a multisensory issue? Cogn Affect Behav Neurosci, 4, 270–278. Munafo, M. R., Brown, S. M., & Hariri A. R. (2008). Serotonin transporter (5-HTTLPR) genotype and amygdala activation: A meta-analysis. Biological Psychiatry, 63, 852–857. Munksgaard, L., DePassille, A. M., Rushen, J., Herskin, M. S., & Kristensen, A. M. (2001). Dairy cows’ fear of people: Social learning, milk yield and behaviour at milking. Applied Animal Behaviour Science, 73, 15–26. Myers, R. E., Swett, C., & Miller, M. (1973). Loss of social group affinity following prefrontal lesions in free-ranging macaques. Brain Research, 64, 257–269. Navarrete, C. D., Olsson, A., Ho, A. K., Mendes, W. B., Thomsen, L., & Sidanius, J. (2009). Fear extinction to an out-group face: The role of target gender. Psychological Science, 20,155–158. Ochsner, K. N., & Gross, J. J. (2005). The cognitive control of emotion. Trends in Cognitive Sciences, 9, 242–249. Ochsner, K. N., Knierim, K., Ludlow, D. H., Hanelin, J., Ramachandran, T., & Glover, G., Mackey, S. C. (2004). Reflecting upon feelings: An fMRI study of neural systems supporting the attribution of emotion to self and other. Journal of Cogntive Neuroscience, 16, 1746–1772. Ochsner, K. N., Ray, R. D., Cooper, J. C., Robertson, E. R., Chopra, S., Gabrieli, J. D. E., & Gross, J. J. (2004). For better or for worse: Neural systems supporting the cognitive downand up-regulation of negative emotion. NeuroImage, 23, 483–499. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452–454. Ohman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychology Review, 108, 483–522. Ohman, A., & Soares, J. (1993). On the automatic nature of phobic fear: Conditioned electrodermal responses to masked fear-relevant stimuli. Journal of Abnormal Psychology, 102, 121–132. Olsson, A., Brodbeck, C., Bolger, N., & Ochsner, K. N. (2008, April 12–15). Attributions about others’ intentions affect emotional learning


from their actions. Abstract presented at the Annual Meeting of Cognitive Neuroscience Society, San Francisco, CA. Olsson, A., Carmona, S., Katherine R., Downey, G., & Ochsner, K. N. (2007, May 5–8) Perceiving Threat and Learning to Fear: Linking Rejection Sensitivity to Biases in Social Perception and Learning. Abstract presented at the Annual Meeting of Cognitive Neuroscience Society, New York City, New York. Olsson, A., Ebert, J. P., Banaji, M. R., & Phelps, E. A. (2005). The role of social groups in the persistence of learned fear. Science, 309, 785–787. Olsson, A., Nearing, K. I., & Phelps, E. A. (2007). Learning fears by observing others: The neural systems of social fear transmission. Social Cognitive and Affective Neuroscience, 2, 3–11. Olsson, A., & Ochsner, K. N. (2008). The relationship between emotion and social cognition. Trends in Cognitive Sciences, 12, 65–71. Olsson, A., Ochsner, K. N., & Phelps, E. A. (2007, May 5–8). The role of the neural circuitry underlying empathy and mental attribution in observational fear learning. Abstract presented at the Annual Meeting of Cognitive Neuroscience Society, New York City, New York. Olsson, A., & Phelps, E.A. (2004). Learned fear of “unseen” faces after Pavlovian, observational, and instructed fear. Psychological Science, 15, 822–828. Olsson, A., & Phelps, E. A. (2007). Social learning of fear. Nature Neuroscience, 10, 1095–1102. Ongür, D., Ferry, A. T., & Price, J. L. (2003). Architectonic subdivision of the human orbital and medial prefrontal cortex. Journal of Comparitve Neurology, 460, 425–449. Pavlov, I. P. (1927). Conditioned reflexes. Oxford, England: Oxford University Press. Phelps, E. A., Delgado, M. R., Nearing, K. I., & LeDoux, J. E. (2004). Extinction learning in humans: role of the amygdala and vmPFC. Neuron, 43, 897–905. Phelps, E. A., & LeDoux, J. E. (2005). Contributions of the amygdala to emotion processing: From animal models to human behavior. Neuron, 48, 175–187. Phelps, E.A., Ling, S., Carrasco, M. (2006). Emotion Facilitates Perception and Potentiates the Perceptual Benefits of Attention. Psychological Science,17, 292–299. Phelps, E. A., O’Connor, K. J., Cunningham, W. A., Funayma, E. S., Gatenby, J. C., Gore, J. C., &

2/15/2011 7:53:55 PM


Banaji, M. R. (2000). Performance on indirect measures of race evaluation predicts amygdala activity. Journal of Cogntive Neuroscience, 12, 1–10. Phelps, E. A., O’Connor, K. J., Gatenby, J. C., Gore, J. C., Grillon, C., & Davis, M. (2001). Activation of the left amygdala to a cognitive representation of fear. Nature Neuroscience 4, 437–441. Plotkin, H. C., & Odling-Smee, F. J. (1981). A multiple level model of evolution and its implications for sociobiology. Behavioral and Brain Sciences, 4, 225–268. Preston, S. D., & de Waal, F. B. (2002). Empathy: Its ultimate and proximate bases. Behavioral and Brain Sciences, 25, 1–20. Price, J. L., & Amaral, D. G. (1981). An autoradiographic study of the projections of the central nucleus of the monkey amygdala. Journal of Neuroscience, 1, 1242–1259. Quirk, G. J., Armony, J. L., & LeDoux, J. E. (1997). Fear conditioning enhances different temporal components of tone-evoked spike trains in auditory cortex and lateral amygdala. Neuron, 19, 613–624. Quirk, G. J., Garcia, R., & Gonzalez-Lima, F. (2006). Prefrontal mechanisms in extinction of conditioned fear. Biological Psychiatry, 60, 337–343. Rachman, S. (1968). Phobias: Their nature and control. Springfield, IL: Charles C. Thomas. Rachman, S. (1977). The conditioning theory of fear acquisition: A critical examination. Behaviour Research and Therapy, 19, 439–447. Robbins, T. W. (2005). Chemistry of the mind: Neurochemical modulation of prefrontal cortical function. Journal of Comparative Neurology, 493, 140–146. Rolls, E. T. (1999). The brain and emotion. New York, NY: Oxford University Press. Rolls, E. T., Tovée, M. J., Purcell, D. G., Stewart, A. L., & Azzopardi, P. (1994). The responses of neurons in the temporal cortex of primates and face identification and detection. Experimental Brain Research, 101, 473–484. Romanski, L. M., Clugnet, M. C., Bordi, F., & LeDoux, J. E. (1993). Somatosensory and auditory convergence in the lateral nucleus of the amygdala. Behavioral Neuroscience, 107, 444–450. Sanfey, A. G., Loewenstein, G., McClure, S. M., & Cohen, J. D. (2006). Neuroeconomics: Crosscurrents in research on decision-making. Trends in Cognitive Sciences, 10, 108–116.


479

Saxe, R., Moran J. M., Scholz, J. K., & Gabrieli, J. D. E. (2006). Overlapping and non-overlapping brain regions for theory of mind and self reflection in individual subjects. Social Cognition and Affective Neuroscience, 1, 229–234. Schacter, D. L., & Addis, D. R. (2007). The cognitive neuroscience of constructive memory: Remembering the past and imagining the future. Philosophical Transactions of the Royal Society of London, 362, 773–786. Schneider, M., & Koch, M. (2005). Deficient social and play behavior in juvenile and adult rats after neonatal cortical lesion: Effects of chronic pubertal cannabinoid treatment. Neuropsychopharmacology, 30, 944–957. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. Seligman, M. E. P. (1971). Phobias and preparedness. Behavior Therapy, 2, 307–321. Shi, C., & Davis, M. (1999). Pain pathways involved in fear conditioning measured with fearpotentiated startle: Lesion studies. Journal of Neuroscience, 19, 420–430. Simon, H., Le Moal, M., & Calas, A. (1979). Efferents and afferents of the ventral tegmentalA10 region studied after local injection of [3H] leucine and horseradish peroxidase. Brain Research, 178, 17–40. Singer, T., Seymour, B., O’Doherty, J., Kaube, H., Dolan, R. J., & Frith, C. D. (2004). Empathy for pain involves the affective but not sensory components of pain. Science, 303, 1157–1162. Singer, T., Seymour, B., O’Doherty, J., Stephan, K. E., Dolan, R. J., & Frith, C. D. (2006). Empathic neural responses are modulated by the perceived fairness of others. Nature, 439, 466–469. Todorov, A., Said, C. P., Engell, A. D., & Oosterhof, N. N. (2008). Understanding evaluation of faces on social dimensions. Trends in Cognitive Sciences, 12, 455–460. Tulving, E. (2002). Episodic memory: From mind to brain. Annual Review of Psychology, 53, 1–25. Vaughan, K. B., & Lanzetta, J. T. (1980). Vicarious instigation and conditioning of facial expressive and autonomic responses to a model’s expressive display of pain. Journal of Personality and Social Psychology, 38, 909–923. Vuilleumier, P., Richardson, M. P., Armony, J. L., Driver, J., & Dolan, R. J. (2004). Nature Neuroscience, 7, 1271–1278. Whalen, P. J. (2007). The uncertainty of it all. Trends in Cognitive Sciences, 11, 499–500.

2/15/2011 7:53:55 PM

480


Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. (1998). Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. Journal of Neuroscience, 18, 411–418. White, D. J., & Galef, B. J. (1998). Social influence on avoidance of dangerous stimuli by rats. Animal Learning and Behavior, 26, 433–438. Whiten, A., Horner, V., & de Waal, F. B. (2005). Conformity to cultural norms of tool use in chimpanzees. Nature, 437, 737–740. Winston, J. S., Strange, B. A, O’Doherty, J., & Dolan, R. J. (2002). Automatic and intentional brain responses during evaluation of trustworthiness of faces. Nature Neuroscience, 5, 277–283.


Young, L., Cushman, F., Hauser, M., & Saxe, R. (2007). The neural basis of the interaction between theory of mind and moral judgment. Proceedings of the National Academy of Sciences USA, 104, 8235–8240. Young, M. P., Scannell, J. W., Burns, G. A., & Blakemore, C. (1994). Analysis of connectivity: Neural systems in the cerebral cortex. Nature Reviews Neuroscience, 5, 227–250. Zaki, J., Ochsner, K. N., Hanelin, H., Wager, T. D., & Mackey, S. (2007). Different circuits for different pain: Patterns of functional connectivity reveal distinct networks for processing pain in self and others. Social Neuroscience, 2, 276–291.

2/15/2011 7:53:55 PM

CHAPTER 21 Effects of Conditioning in Advertising Todd R. Schachtman, Jennifer Walker, and Stephanie Fowler

A large number of advertisements pair the presentation of a product or brand name with another stimulus that possesses affective value. The pairing of these two stimuli can result in a change in behavior (e.g., attitude, purchasing probability, attention to the product in the marketplace) toward the product or brand name. These pairings resemble the procedure of classical conditioning. This chapter discusses some of the research that has been done in the area of conditioning and advertising as well as some of the recent developments in conditioning theory and research that may assist in advertising research and its application. The chapter will address such topics as useful parameters for producing conditioning, the roles of affect and cognition, and the role of awareness; and many potentially relevant conditioning phenomena are discussed that might be of relevance to advertising.

INTRODUCTION Advertisements often pair two events together: the product or brand name with a pleasurable stimulus. The pairing of these two stimuli (sometimes called a “trial” when occurring in an experimental situation) results in a change in behavior (e.g., attitude, purchasing probability, attention to the product in the marketplace) toward the product or brand name. This pairing clearly resembles the procedure of classical conditioning (Pavlov, 1927). During classical conditioning, a neutral stimulus is paired with an event that typically has some affective value for the animal (typically something with biological significance, such as food for a hungry animal or a painful event). The neutral stimulus is referred to as the conditioned stimulus (CS), and the event that already has affective value for the organism is the unconditioned stimulus (US). The pairing of these two events results in a change in the organism’s response to the CS.

Using Pavlov’s well-known experiments with dogs as an example (given that Pavlov developed this procedure), a tone might serve as the CS and food can serve as the US. The dogs would, of course, salivate to the food if food is placed in the dog’s mouth. This salivation is an unconditioned response and does not involve any learning. Before the tone and food were paired together, the dog had no tendency to salivate to the tone; but after the two events were paired together, the dog began to salivate to the tone. This latter response is the CR, and it is the measure of conditioning. If the organism makes a CR after such pairings (and if various control conditions rule out other possibilities), then it is assumed that the organism has formed an association between the tone and food. Returning to the case of paired events during advertisements, if the individual changes his or her behavior (attitude change, interest in purchasing the item) in the presence of the product or brand (the CS) as a function of pairings of

481


2/16/2011 5:55:21 PM

482


this CS with an affective stimulus (the US), then this behavior can be said to be a conditioned response; and the pairing can reflect the development of an association between the product or brand (i.e., the CS) and the US. Note that even if the ad is an “informational” one rather than aiming at conditioning per se (pairing the ad with some pleasurable or attractive stimulus), it is very hard to avoid conditioning during exposure to the ad in that the individual(s) in the ad delivering the information is likely doing so with a pleasant voice; and various cues may be present that can promote conditioning. Gresham and Shimp (1985, p. 11) purported that classical conditioning “is the most widely discussed mechanism of [attitudes towards an ad] on consumers’ brand attitudes.” Interestingly, they go on to discuss the direction of causality of the influence between the ad and the brand; they mention that the attitude toward the brand can influence the attitude toward the ad, and the attitude toward the ad can influence the attitude toward the brand. The former may be more important for mature brands, and the latter may be important for new brands (Gresham & Shimp, 1985). Classical conditioning is a procedure in which two events (stimuli) are presented in the manner just described (e.g., Janiszewaki & Warlop, 1993: Kim, Lim, & Bhargava, 1998); and if a CR occurs, then one can state that classical conditioning as an effect has occurred. As a procedure and effect, classical conditioning is silent with respect to the underlying mechanisms (associative processes, cognitive processes, reflexive processes, etc.) that might be responsible for the change in the CR (see also Janiszewski & Warlop, 1993). Some recent work has used the expressions “classical conditioning” or “Pavlovian conditioning” to refer to a particular theoretical process (i.e., expectancy learning, see later section on “Evaluative Conditioning”); but we (and most researchers in the field of conditioning) feel it is best to refer to classical or Pavlovian conditioning as an effect. Hence, all advertising that involves the pairing of events and results in a change in a learned response can be said to be instances of classical conditioning. However, one of many mechanisms may be responsible for this effect.


The principles of operant conditioning have also been applied to advertising situations. Winters and Wallace (1970) discuss operant conditioning methodology, and how measures such as choice and giving the participant control over exposure to the ad can provide valuable assessment devices. Reed, McCarthy, Latif, and DeJongh (2002) show an interesting way to examine cues experimentally in the marketplace as a means of testing innovative conditioning phenomena in an assimilated natural environment. Given the role that classical conditioning plays in advertising, it is surprising how few review articles are available that specifically examine advertising with a focus on conditioning per se. The perusal of dozens of subject indices of various marketing and advertising textbooks and edited volumes produced few entries for “conditioning,” and even fewer chapters specifically devoted to that topic. McSweeney and Bierley (1984) and van Osselaer (2008) provided a valuable review of classical conditioning effects that could be of use to marketing researchers. Another chapter by Allen and Shimp (1990) provided a worthwhile overview of research on conditioning in advertising and some important methodological considerations (see also Cohen, 1990). Of course, it is quite possible, as Allen and Shimp stated about 20 years ago, that the relationship between conditioning and marketing is still in its early stages of development with respect to research efforts: “Classical conditioning research is in the introductory stage of a potentially gainful life cycle in consumer behavior” (p. 29). Moreover, the mechanisms of conditioning are still being investigated by learning and conditioning researchers, and its role in advertising is still being pursued. Indeed, Kim, Allen, and Kardes (1996, p. 318) noted that a “major reason why advertising researchers have failed to embrace knowledge products of the Pavlovian tradition is that no consensus has emerged about how or why conditioning procedures yield their effects on brand attitude.” The present chapter hopes to follow in the suit of these earlier reviews in providing a discussion of the following: (1) some of the research that has been done in the area of conditioning

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

and advertising; (2) some of the recent developments in conditioning theory and research that may assist in advertising research and its application; (3) some interesting issues that are relevant to the confluence of the fields of conditioning and advertising. Since we come to this chapter as conditioning researchers rather than marketing researchers, our focus may strike the reader as flavored in that way; and it will fall a bit short of providing an exhaustive scope of findings and issues in the field of marketing. Nonetheless, we expect that some of the points made will be useful. Of course, before we begin with this review, we acknowledge that one can question the value of advertising per se. The effectiveness of advertising, and, therefore, of pairings of a product or brand with attractive stimuli, for a company has been challenged by Ehrenberg (1974, p. 32), who stated that advertising is not particularly effective although cutting it can lose sales for a company. He said that advertising is really used to “reinforce feelings of satisfaction for brands already being used.” D’Souza and Rao (1995, p. 32) similarly claimed that “advertising may be working to simply maintain the status quo [in sales].” If so, then maintaining the status quo for a product in high use may require advertising to keep this high use position. In this way, such a project may make use of the processes underlying conditioning that potentially occur during advertising.

ISSUES CONCERNING THE BEHAVIORIST-COGNITIVE DEBATE AND THE CURRENT STATUS OF CONDITIONING THEORY The putative debate between behaviorism and cognition has been discussed by many marketing researchers (e.g., Allen & Janiszewski, 1989; Allen & Shimp, 1990). The debate arose even within the field of psychology because the initial decades of conditioning research began at a time when psychology was dominated by behaviorists. Many behaviorists steer clear of hypothetical constructs such as expectancies, memories, and associations. Later, in the 1970s, the field of human learning and memory and that of animal conditioning began a “cognitive revolution”


483

during which, among other things, researchers became interested in how acquired knowledge was organized in memory. Within the area of conditioning theory, the dramatic change toward cognitive theorizing occurred due to at least three factors. First, research in the late 1960s and early 1970s on compound conditioning (when two CSs are present on a conditioning trial) and a phenomenon known as “conditioned inhibition” (Pavlov, 1927; Rescorla, 1969) gave large emphasis to the concept of expectancy (and the interaction between CSs during the formation of such expectancies) in classical conditioning. Second, Rescorla’s work in the 1970s focused extensively on the content of associative learning—what are the representations of the events that are associated (i.e., “what is associated with what?” with respect to the events represented in memory). Finally, there was renewed focus on processes that influence conditioning besides that of acquisition (e.g., retrieval, rehearsal, motivation, and the reactivation of memories; see Lewis, 1979; Miller, Kasprow, & Schachtman, 1986; Spear, 1978). As discussed in more detail in the first chapter to the present volume, all three of these factors or “directions” that the field took in the 1970s are very cognitive in nature. Many researchers, indeed, a sizable number of psychologists, do not recognize that the field of conditioning and learning has gone through very substantive changes, such as those just described, in the past 35 years. While classical conditioning remains an experimental procedure with a behavioral outcome, the theoretical discourse on the mechanisms of conditioning has taken on a very cognitive focus since the 1970s. The importance of parsimony, Morgan’s Canon, and Occam’s razor certainly remains as a scientific tool; but some research findings indicate that animal conditioning effects are best (or only) explained in terms of cognitive processes, such as the activation of representations of events and their interaction with associative mechanisms—as well as nonacquisition types of information processing mentioned earlier (rehearsal, reactivation, etc.). In other words, many conditioning phenomena are only explained by evoking cognitive processes.

2/16/2011 5:55:21 PM

484


During the viewing of an advertisement, there are a multitude of types of information processing that may take place. The person can experience a change in the emotional value of the CS (the brand or product) without any cognitive effort or awareness (i.e., implicitly). Alternatively, one can experience a change in the emotional value of the CS (or beliefs about it), and this change can come about with awareness on the part of the individual. The individual can also form an expectancy of the US when the CS occurs (e.g., Baeyens, Crombez, Van Den Bergh, & Eelen, 1988), and this learning may occur with or without awareness. The person can also be aware (or not aware) of the contingency between the CS and the US. All of these processes and/or others can give rise to a conditioned response (e.g., attitude toward a product, change in likelihood of purchasing a product). One valuable goal of conditioning research and advertising research is to design studies to examine the nature of the processes underlying the acquisition of information and the elicitation of the CR when exposed to a CS used in advertisements.

SOME OF THE EARLIER WORK EXAMINING CONDITIONING DURING ADVERTISING Allen and Shimp (1990) and Cohen (1990) summarized many of the research findings through to the date of their writing, and so interested readers can turn to those resources. However, some of those findings will be summarized briefly here; and we mention a few valuable points about them. The present discussion will be a far cry from any kind of exhaustive presentation of the research in this area; but, rather, we will provide a description of a few studies and findings to “set the table” a little before we provide information about specific topics. A description of these early studies will highlight, albeit briefly, examples of procedures as well as theoretical issues involved in such research. As Allen and Shimp (1990) point out in their review, Staats and Staats (1957, 1958) conducted a very early study and found that awareness is not necessary for learning an association. Since we will briefly discuss the issue of awareness and


conditioning later in this chapter, we will keep our comments about this topic even more brief for the time being. The study by Gorn (1982) was innovative in some respects in that it was a relatively early paper; and yet it discussed many of the issues that are critical to research examining the effects of conditioning in advertising. Specifically, the authors were interested in whether object preferences could be classically conditioned. Gorn (reported in the Allen and Shimp chapter as the first experimental study on conditioning and marketing) had participants rate different kinds of music, and he used the most appealing music as the appetitive US (i.e., pleasurable) and the least attractive music as the aversive US (and only participants who rated this musical piece as attractive were included for the pairing of the CS with the appetitive US, and only those participants who rated the piece as unattractive were included for the pairing of the CS with the aversive US). Seventy-nine percent of subjects given a pairing of the colored pen with the attractive music chose this pen over a nonexposed pen when given a choice, and only 30% of the participants chose the pen paired with unattractive music if they had received a pairing of this pen with the aversive music (obviously a percentage of 50% would reflect indifference to the pens when given a choice). Since a single pairing was used, it shows that significant conditioning can occur with one conditioning trial. Second, the CS and US presentations were simultaneous, participants heard one (of two) musical clips while viewing a slide image of one of the pens, thereby showing that such an arrangement of CS and US can produce appreciable conditioning. Finally, “mere exposure” (see section on “Mere Exposure”) cannot explain the subjects choosing the nonexposed pen over the one paired with the aversive music. The mere exposure effect refers to the increase in attraction to a stimulus simply because the individual has encountered it in the past. It is possible that mere exposure made the pen paired with attractive music more attractive (rather than the increase in attractiveness being due to the pairing with attractive music); however, the pairing of a pen with aversive music produced an aversion to this pen despite any

2/16/2011 5:55:21 PM


possible (and perhaps unlikely) mere exposure effect working in the opposite direction (making it more attractive). Regarding the Gorn study, there has been a discussion in the advertising literature about the role of demand characteristics in this research (see Darley & Lim, 1993; Shimp, Hyatt, & Snyder, 1993). (By the way, we also wish to point out that Kahle, Beatty, and Kennedy [1987] stated that proponents of conditioning theory discount or trivialize the issues of awareness and demand characteristics; but, to us, this claim seems unsubstantiated and contentious.) Overall, producing classical conditioning in the laboratory is not easily obtained given that some reports have found poor conditioning or the results have been mixed or subject to alternative interpretation (Allen & Madden, 1985; Bierley, McSweeney, & Vannieuwkerk, 1985; Gorn, 1982), whereas other studies have been more promising in showing conditioning (Shimp, Stuart, & Engle, 1987; see Cohen, 1990; Allen & Shimp, 1990 for reviews).

CONDITIONING PARAMETERS AND PROCEDURAL ISSUES IN ADVERTISING RESEARCH This section will discuss some of the procedural variables that have been (or are suggested) to be useful in conditioning research. Some variables (e.g., US or CS preexposure prior to conditioning) can be both “a conditioning phenomenon” and a “procedural variable”; we will reserve our discussion of these until a subsequent section on conditioning phenomena. We realize that the distinction between “What is a procedural variable?” and “What is a conditioning effect?” is arbitrary for some effects. For instance, trace conditioning refers to conditioning effect and a manipulation in the interstimulus interval between the CS and the US; but such instances will be placed in one section or the other, and we hope that these sections still provide some usefulness. Arrangement of the Conditioned Stimulus and Unconditioned Stimulus in Time

Stuart et al. (1987) compared backward conditioning with delayed conditioning using a brand


485

as the CS and an attractive stimulus as the US. Backward conditioning is a classical conditioning procedure in which the US onset precedes the CS onset. Forward conditioning (often called delayed conditioning) is classical conditioning in which the CS precedes the onset of the US and the CS offset does not occur prior to US onset (since the latter would be “trace conditioning”). In one of the early experiments in that report, the authors used a forward conditioning procedure in which the CS not only preceded the onset of the US but also overlapped the US, and they found that forward conditioning was superior to backward conditioning. Many or most forward conditioning procedures do not have any overlap between the CS and the US (i.e., the CS onset precedes US onset but CS offset occurs at the same time as US onset). When the CS and US have simultaneous onsets and offsets (complete overlap), then this is referred to as “simultaneous conditioning.” Since Stuart et al.’s initial experiment (Exp. 1) used a forward procedure that contained this element of a simultaneous arrangement, they conducted another experiment in which a forward conditioning procedure was used, but the CS and US did not overlap. Forward conditioning continued to produce a better CR than backward conditioning. Macklin (1996) used school-aged children to compare forward and simultaneous conditioning in an advertising situation and found that the former produced better conditioning. As mentioned previously, Gorn (1982) obtained good conditioning with a simultaneous arrangement of the CS and US. Other conditioning arrangements have been used as well. Baker (1999) used trials (the product was paired with pleasant photographs) in which the CS was presented alone, followed by a presentation of the US alone, followed by the CS and US together; and conditioning resulted. Baker, Honea, and Russell (2004) examined the effectiveness of placing the brand name at the beginning of the ad or at the end; and, like Stuart et al. (1987), they found that conditioning effects were stronger when brands were placed at the beginning of the ad. Interestingly, they also included a group that received the brand

2/16/2011 5:55:21 PM

486


name at the beginning and the end of the ad, and found that this group exhibited conditioning that was as poor as the “end-only” condition. That is, this condition received the brand name at the beginning of the ad like the group that showed good conditioning; but the placement of the brand at the end caused poorer conditioning than the “beginning-only” group, suggesting that the placement at the end offset the positive effects of placement at the beginning. Baker et al. point out quite correctly that this was probably due to the fact that the total exposure time to the brand name during the ad was 5 seconds for all conditions such that the “beginning” and “end” group receive the brand name for 2.5 seconds on two occasions, and the other two conditions (end only and beginning only) received the brand exposure once for 5 seconds. Therefore, 2.5 seconds may not be long enough to be effective for brand name exposure. But other possibilities exist. The possibility exists that presenting the brand at the end produces some cognitive interference with the forward conditioning trial that just occurred at the start of the trial. Future research will likely enjoy teasing apart these alternatives as well as testing other possible explanations for this interesting effect. Allen and Shimp (1990) argued that simple contiguity is not responsible for conditioning (p. 30), but this point requires elaboration. Contiguity refers to the degree to which two events occur together in time or space (and only temporal contiguity is discussed here). Simple contiguity is neither necessary nor sufficient for conditioning to occur. As mentioned, Blair and Shimp (1992) found second-order conditioning during an advertisement experiment (see section on “Second-Order Conditioning”) in which the target event (brand) was never paired with the US (an unpleasant, boring textbook experience) and, yet, conditioning occurred showing that contiguity is not necessary. The fact that particular values for various parameters (CS novelty, US novelty, and others) are needed or are important in order to obtain conditioning—despite contiguity between the CS and the US—reveals that contiguity is not sufficient for conditioning.


Relationship of the Stimulus Properties of the Conditioned Stimulus and Unconditioned Stimulus to Each Other

Thorndike’s concept of “belongingness” and, more recently and to much more acclaim, John Garcia’s discovery that some CSs condition more effectively if paired with certain USs (but not others); and some USs support conditioning more effectively if paired with certain CSs but not others (Garcia & Koelling, 1966) were important findings for the field of conditioning (Freeman & Riley, 2009). Belongingness may have an important effect on conditioning in advertising (Allen & Shimp, 1990; McSweeney & Bierley, 1984; see also Kellaris, Cox, & Cox, 1993, as discussed later). Kim et al. (1998) found that if a CS and US have little or no preexperimental conceptually based relationship with each other, classical conditioning can still occur as long as the subject does not hold any beliefs about the stimulus that might preclude conditioning. Conditioning researchers have found that second-order conditioning effects (discussed in section on “Second-Order Conditioning”) are also sensitive to the modality of the two cues used in experiments with animal subjects (e.g., Nairne & Rescorla, 1981; Rescorla & Gillan, 1980). Kellaris, Cox, and Cox (1993) found that recall and recognition of brand name as well as the “point of the message” in the ad increases if attention-getting music is used; and recall and recognition are especially enhanced if there is a “congruency” between the meaning communicated nonverbally by the music and that verbally communicated by the ad; marketing researchers may wish to note the degree to which the thematic qualities of the background information matches that of the verbal message. Partial Reinforcement

Bierley, McSweeney, and Vannieuwkerk (1985), in a study relevant to advertising since music is often used as an unconditioned stimulus in ads, found conditioning of colors paired with attractive music. Partial reinforcement did not produce any conditioning of a color paired with the music. It can also be noted that the groups were

2/16/2011 5:55:21 PM


not equated for number of reinforcers, but rather, received the same number of trials as the continuous reinforcement condition because there are two ways to produce a partial reinforcement condition: equating reinforcers or trial number. Researchers need to decide which to manipulation or, rarely, to include various groups that manipulate both. Properties of the Conditioned Stimulus, Other Cues, and the Role of Attention

The intensity or salience of the CS can also have a considerable influence on conditioning during advertising (see, for example, Cohen, 1990, for a discussion). A more noticeable presentation of the brand name will likely help produce stronger conditioning. Additionally, the salience of cues may be determined, in part, by the characteristics of the individual. Gorn (1982) noted that those interested in purchasing a product may find the product information (brand name, etc.) more salient than those not interested in a future purchase. To investigate the validity of this possibility, he examined participants who were in either a decision- or nondecisionmaking context with respect to their relative sensitivity to background cues (i.e., music in this case) and product information. Gorn discovered that non-decision-making participants (what might be considered “less involved” subjects) were most influenced by the music, whereas decision makers (more involved) were more influenced by the information provided. Hence, the impact of cues can be influenced by the person’s motivation. The duration of the CS can influence conditioning (Gibbon & Balsam, 1981; Miller & Schachtman, 1985; Miller & Matzel, 1988; a factor also noted by Allen and Shimp, 1990, p. 30). Conditioning theorists have discovered that extended CS durations can reduce the degree of conditioned responding. Of course, the stimulus must minimally be exposed long enough for the individual to detect or process it. The modalities of the cues presented in the ad can also play a vital role. Stammerjohan, Wood, Chang, and Thorson (2005) examined whether using multiple modalities for ads (visual


487

and auditory rather than just visual or auditory) might influence processing of the ad. As noted by Stammerjohan et al., research on encoding variability (Tulving & Thomson, 1971) suggests that presenting information in more than one context or modality will improve memory and impact the degree of attitude change (and see Stemmerjohan et al. for a discussion of supporting findings and see also Cacioppo & Petty, 1985). Although the authors did not provide unequivocal support for these ideas, this important issue warrants more research. Vakratsas and Ambler (1999), when discussing persuasive hierarchy models, point out in their review that “varied ads” improve ad recall (Rao & Burnkrant, 1991; Zielske & Henry, 1980). Stammerjohan et al. mention that multiple modality input during ads also could include the subject-generated elaborations that occur during or following an ad, such that if the ad provides only auditory information (e.g., a radio ad) but the individual elaborates by imagining the product visually, then multiple-modality processing can be said to be occurring. Research can therefore determine whether elaboration-produced cues are comparable to having more than one modality present in the ad itself. Stammerjohan et al. cite Kahneman (1973) as having mentioned that a large amount of attention is given to items that are both complex and familiar and those that are both simple and novel, but this is not true for simple-familiar items nor for complex-novel stimuli. They also mention the “positivity effect” (that positive stimuli are processed more than negative stimuli). Hence, negative advertising should be less effective than positive advertising (see Cohen, 1990). However, Cohen points out that unpleasantness in an ad can sometimes generate attention and interest in a product such that the product may be expected to resolve this unpleasantness. Baker (1999) notes that high familiarly can result in less information being processed; hence, high brand familiarity means that providing product information can be less valuable since individuals will not process this information as well in an ad about a highly familiar product. Individuals already have opinions about a familiar product and do not always

2/16/2011 5:55:21 PM

488


process new information about it (see also section on “The Role of Prior Belief ”). In such situations, it is important to provide stimulating material to prevent boredom during an ad with a familiar product. Janiszewski and Warlop (1993) noted that attention to a stimulus often involves an orienting response to the stimulus (Hall & Channell, 1985; Hall & Schachtman, 1987; Sokolov, 1963). They reasoned that this orienting response can be very important to advertisers since one hopes for conditioning to the brand name (and attention is important for conditioning), but one also aims for a strong response to the brand later—at the time of purchase. Consistent with the findings of Hall and Channell (1985), Janiszewski and Warlop point out that if a brand (CS) (even a familiar one) is presented in a novel context, then the orienting response will be high in this context even if orienting had waned in the conditioning context (i.e., where the brand-US pairings occurred). Janiszewski and Warlop (1993) found that attention is increased to a CS as a function of conditioning (pairings of the brand with an attractive US). These researchers also said that if the brand is paired with the US in one context, but then the brand is seen in a store (new context) for the first time, then orienting might be strong and this strong reaction may increase the chance of purchase. Janiszewski and Warlop (1993) found that if a brand is conditioned by pairing the brand with a US, then this brand will “pop out” perceptually when presented subsequently on a screen with other items (see Johnston & Hawley, 1994, for more information on such effects with various stimuli) revealing that conditioning, perhaps not surprisingly, can add to the attention-getting properties of a brand name. However, we can also imagine how novelty will promote attention to a product. The maturity (versus novelty) of a product can have a large influence on conditionability. It is also worth mentioning that conditioning researchers have posited that this orienting response is an index of associability (e.g., Swann & Pearce, 1988). Associability refers to the potential of a CS to enter into an association with a CS. Oxoby and Finnigan (2007) found that attention


to one feature of a product (e.g., cost or brand quality information) caused poor attention to other—subsequently presented—information about the product. Gresham and Shimp (1985) suggested that mature (familiar) brands will be more influenced by the impact of the attitude of the brand on the attitude toward the ad, whereas newer products may experience the attitude toward the ad influencing the attitude toward the product. Research by Alpert and Kamins (1995) revealed that novel brands possess attention-getting properties that facilitate processes on some measures (attitude and purchase intention) but not others (recall or actual purchase behavior). Hence, attention to a CS can have an influence on conditioning, and conditioning can impact attention to a CS (Mackintosh, 1975; Pearce & Hall, 1980). Number of Trials

Kroeber-Riel (1984) stated that numerous trials are needed for conditioning, but this is not true. Kim, Lim, and Bhargava (1998) obtained conditioning with a single trial (see also Ehrenberg, 1974). Stuart et al. (1987) also obtained asymptotic conditioning with a single trial. Kim et al. examined the effect of the number of trials and found that affective conditioning requires fewer repetitions than cognitive belief acquisition. Vakratsas and Ambler (1999) mention that there may be an optimal number of trials to produce favorable advertising effects. A minimum number of trials is needed to get an effect (the “wear-in effect,” see, e.g., Blair, 1987) and the effect of advertising decreases after a certain number of exposures to an ad. There is an inverted-U shape to the effectiveness of advertising as a function of the number of conditioning trials. One valuable way to offset this inverted-U function—that is, to continue to get effects with additional trials—is to vary the ad somewhat so that individuals get exposed to a slightly different variation (Rao & Burnkrant, 1991; Zielske & Henry, 1980). As mentioned, conditioning can involve many processes besides the acquisition of an

2/16/2011 5:55:21 PM


association (e.g., the retrieval, retention, elaboration, rehearsal of information). So even if good learning is apparent after one or two conditioning trials, additional, beneficial processing might occur if additional trials are given. Batra and Ray (1983) note that Krugman (1972) stated that there are only three truly effective trial types during an advertisement: an initial exposure that produces recognition, a second exposure that involves the subject’s processing, and the third and all subsequent exposures, which simply serve as reminders of what the viewer has already seen and thus maintain such processing. We note again that these later trials may produce rehearsal/retrieval-practice that can influence behavior on certain measures. One other point about the number of trials used can be made: Allen and Janiszewski (1989) found that more pairings resulted in more demand characteristics and so this concern should be addressed. Allen and Janiszewski provide extensive discussion of demand characteristics in their report (see also Kahle et al., 1987). The influence of the number of trials is also discussed in section on “Cognition and Affect.” Intertrial Interval

The intertrial interval can have a large effect on conditioning. Similar to the “spacing effect” in human learning (Crowder, 1976), classical conditioning is greater if a longer period of time occurs between trials during the experimental session. Such effects may also occur in an advertising situation (as mentioned by Allen & Janiszewski, 1989). Some conditioning theories have posited that such effects are due to the relative durations of the CS and the contextual cues that are exposed between trials (Gibbon & Balsam, 1981; Miller & Matzel, 1988; Miller & Schachtman, 1985). Different Behavioral Measures

Allen and Shimp (1990) discussed advertising research in which the experimental manipulation changes the participant’s attitude toward a brand or product, and then tests this change by giving the participant a preference between that item and a control item. They stated that


489

preference is a “demanding measure” and may not be the best dependent variable. They note that Macklin (1998) used a “buy back measure” rather than a preference measure with children as subjects, and found that the former was a much more sensitive assessment. Another interesting assessment tool was used, as noted earlier, by Janiszewski and Warlop (1993), who, when using an eye-tracking device, found that conditioned brand names will receive a large amount of attention in a display. Rothschild and Hyun (1990) used electroencephalography (EEG) as a measure. Clearly many assessment tools are available to marketing researchers (see Cohen, 1990). Control Conditions for Conditioning

Many different kinds of control conditions can be used in an experiment to ensure that the conditioning is due to the contiguous pairing of the CS and US and the positive contingency between these events. McSweeney and Bierley (1984) discuss this issue in some detail and so readers may wish to refer to this resource. Some researchers have used random CS and US presentations (Bierley et al., 1985; Janiszewski & Warlop, 1993). Some researchers have used a procedure for the control condition in which the CS and US are presented randomly with respect to each other, except with the constraint that the two events not be paired together by chance (Grossman & Till, 1998; Priluck & Till, 2004; Stuart et al., 1987). One possible problem a group for which the CS and US never occur together (akin to “explicitly unpaired” CS and US presentations) is that explicitly unpairing the events produces a negative contingency such that one event comes to signal that the other event will not occur—a phenomenon known as conditioned inhibition (Rescorla, 1969). Rescorla proposed in the late 1960s that a reasonable control condition that will produce neither a positive contingency between the two the events, nor an unpaired arrangement which might produce conditioned inhibition, is to present the events randomly. Yet random presentations can produce their own problem as they can produce “learned irrelevance” between the CS and US (see Matzel,

2/16/2011 5:55:21 PM

490


Schachtman, & Miller, 1988; but see Bonardi & Ong, 2003) and, as Grossman and Till (1998), Priluck and Till (2004), and Stuart et al. (1987) likely realize (since they avoided this problem), the chance pairings produce problems of their own. However, little such conditioning from chance pairings in a random condition seems to occur when this treatment is compared with a group that did not receive a US during conditioning (Stuart et al., 1987). Janiszewski and Warlop (1993) gave conditioning in which the trial consisted of three temporal phases: the CS period, the US period, and a posttrial period for the experimental condition. The control received these three phases in a random order on the trials—certainly a reasonable control condition (although perhaps too conservative in that some conditioning could occur in the control group since associations could be formed with a long interstimulus interval). McSweeney and Bierley (1984) mentioned that presenting the CS constantly and then presenting USs intermittently during the CS exposure does not produce a CR in animal research (Brown & Jenkins, 1968). This latter procedure is not too different from some of the procedures of advertising research in which several USs occur during a CS presentation in which CRs are produced. There is a need to examine various parameters (e.g., CS duration) to find out why this difference in conditioning may exist. There are many different control conditions available, each with its advantages and disadvantages. Retention of Conditioning

Obviously, it is important for advertisers to know how long the effects of advertising might last, because the interval between viewing an ad (i.e., conditioning) and product purchase might be lengthy. Grossman and Till (1998), using six conditioning trials, found that attitudes toward experimentally conditioned brands lasted at least 3 weeks (the longest interval tested). They also discussed research by Mitchell (1993), who found that attitudes toward a brand decreased over a 2-week delay but intention to purchase did not decrease. Moore and


Hutchinson (1985) found that a 7-day retention interval caused attitudes toward the brand to be reduced, but brand awareness (e.g., choosing the product category for a brand) increased. Kellaris et al. (1993) found that recall and recognition of brand name as well as the “point-ofthe-message” in the advertisement increases if attention-getting music is used. Gardiner, Mitchell, and Russo (1985) found that low involvement (defined as viewing product information for its entertainment value; see section on “High Versus Low Involvement”) resulted in poorer memory (on four different measures) for product information, but resulted in a greater positive evaluation of the brand (when evaluating 22 attributes of the product, this condition had more positive judgments in 19 of them). Cohen (1990) pointed out that peripheral aspects of the message (voice quality of the ad, affective responses to the message) will play a significant role in how much or whether elaboration occurs. Memory researchers know well that elaboration can improve retention with various types of information. Vakratsas and Ambler (1999) noted that “varied ads” improve ad recall (Rao & Burnkrant, 1991; Zielske & Henry, 1980). Burke and Srull (1988, p.65) examined interference and memory in advertising and found proactive and retroactive interference effects for ad information. They reported that “recall interference occurred when subjects rated the target ads on interest value but not when the advertised brands were evaluated for purchase”; and they went on to suggest that the greatest interference seems to occur for “consumers who are not in the market for a product, or who do not have the ability and/or motivation to process ads in a manner that will enhance information retrievability.” More research on retention and the effects of interference is needed. Use of Music in Advertising

As mentioned, Kellaris et al. (1993) found an increase in recall and recognition of the brand name and the point of the message in the ad if attention-getting music is used. Bruner (1990) provided an extensive review of music in

2/16/2011 5:55:21 PM


advertising, in which he suggested the use of music when products induce low cognitive involvement (e.g., jewelry, beer). Bruner discusses various kinds of music that can be used for ads. He points out the drawback of using familiar songs (overexposure to a song can render it less effective, as mentioned earlier) and suggests that advertisers use a song that has already been written but is not familiar or have one written for the purpose of the ad since it will obviously be novel. Hung (2001), in an advertising study, found that music can produce an image in the consumer’s mind (e.g., “successful,” “imaginative”) as well as an emotion (e.g., “calm,” “boring,” “annoying”). Hung found that music (classical versus hard rock) influenced (1) the estimated price of objects for sale at the advertised shopping mall; and (2) the perception that the store was darker or lighter with respect to its lighting. Some music also produced much greater within-group variability (i.e., the selection of classical music used produced much more consistent responses across subjects than the rock music). As mentioned earlier, Gorn (1982) experimentally conditioned products (pens) using liked and disliked music. Pitt and Abratt (1988) also used music in a classical conditioning experiment. Music has been used in many additional studies; unfortunately, an exhaustive review cannot be provided here, but this medium obviously impacts advertising in many ways.

Order of Ad Exposure and Experience with a Product

Vakratsas and Ambler (1999), when discussing low-involvement hierarchy models, point out that advertising is more effective when it precedes usage experience (see also the section on “The Role of Prior Belief” for the influence of past experience on future processing). In their review, they mention that Smith (1993) showed that if an individual experiences an ad prior to a negative experience with the product, then the ad can reduce the negative impact; but the ad has no effect if the person has had a positive


491

experience. Ads that occur prior to experience have a much greater effect.

SOME CONDITIONING PHENOMENA AND THEORETICAL ISSUES INVOLVING ADVERTISING This section explores a variety of conditioning effects that can be useful for marketing researchers. Researchers may benefit from an appreciation of the underlying mechanisms of the effects discussed herein. Alternatively, researchers may wish to explore these effects in an advertising experimental framework as many marketing researchers have done with a few of these effects. Mere Exposure

The mere exposure effect refers to the increase in attractiveness of a stimulus simply because it has been previously exposed to the individual. Batra and Ray (1983) stated that, for a person with high involvement, the affective changes that occur as a function of mere exposure are not the same kind of affective changes that can occur during CS-US pairings for a person with high involvement. In the latter case, cognitive processes such as awareness and comprehension will occur; and such stages are necessary for affective attitude change (although a definition of what is and what is not attitude change seems needed). As mentioned, Gorn ruled out mere exposure as the cause of a conditioned preference during a treatment of group in which a product was paired with an aversive US. Some theories of advertising state that simple exposure to an advertisement will increase liking due to familiarity such that this can happen independently of awareness or attention to the attributes of the product (Vakratsas & Ambler, 1999). Unconditioned Stimulus Preexposure

US preexposure involves the effects of administering US presentations prior to the pairings of the CS and US. Many marketing researchers (e.g., Bruner, 1990) have noted that earlier exposure to the US (i.e., an attractive US such as pleasurable music) can attenuate the effects of conditioning using that

2/16/2011 5:55:21 PM

492


US. The conditioning literature shows that US preexposure can reduce the effectiveness of the US in supporting conditioning (Gordon & Weaver, 1988; Randich & Ross, 1984; Tomie, Murphy, Fath, & Jackson, 1980). Allen and Shimp (1990), when discussing this effect, note that Bierley et al. (1985) required 28 pairings of a color with attractive music that was highly familiar (Star Wars theme) to the subjects in order to obtain conditioning. Allen and Shimp also discussed the tradeoff of using familiar celebrities and popular music (which has obviously been exposed to the individuals in the past), which has some advantages for conditioning versus the hindering effect such exposure might have. A few issues can be noted about these effects. First, these kinds of tradeoffs are not new to conditioning researchers. Many fear-conditioning researchers (using primarily animals, that is, non-humans) will give one or two exposures to the CS (in contrast to the US preexposure being discussed) prior to conditioning in order to remove the unwanted effects (e.g., startle) that a novel CS can have on the initial conditioning trials. Giving such an exposure or two prior to the conditioning trial will greatly decrease such unwanted responses on the conditioning trial, but event preexposure of this sort also often involves some price to pay. Although this effect involves an issue regarding prior CS exposures rather than US preexposures, it provides another example of a researcher dealing with the tradeoff among various factors. Conditioning researchers often weigh these various tradeoffs when using procedures in which the stimuli are novel or familiar. Conditioning theorists claim that the poor conditioning that results from US preexposure occurs by one of two processes. First, this effect may be the result of habituation to the US. Habituation is the loss of responding to a stimulus that has been presented repeatedly. This repeated presentation can cause the individual to stop responding to the US; a poor response to a stimulus can also be indicative of poor ability to support conditioning to a CS. A second process is that the US preexposures cause an association between the contextual cues (e.g., the environment that the individual is in) and the US. This association “blocks” the learning of


an association between the CS (e.g., the brand/ product) and the US (e.g., Gordon & Weaver, 1988). Blocking (and overshadowing, a very related conditioning effect) will be reviewed next. The multifaceted effects of US preexposure make the decision to use a familiar US versus a novel one a challenging decision. Latent Inhibition or the Conditioned Stimulus Preexposure Effect

Latent inhibition (Lubow & Moore, 1959) is the poor conditioning that occurs to a CS (a product or brand) if this stimulus is presented many times by itself (i.e., without the US) prior to the conditioning trials (i.e., being paired with the US). This poor learning is compared to a conditioning that did not receive the CS-alone exposures (and this latter group shows normal, strong conditioning). Stuart et al. (1987, Exp. 2) examined latent inhibition in an advertising experiment by exposing participants to a particular brand name on either 8 or 20 occasions prior to pairing it with an attractive US on the conditioning trials. Participants in these conditions showed poorer conditioning to the brand name than those in a control condition who received conditioning without any CS (i.e., brand name) preexposure. It is valuable for marketing professionals to know that brand exposure prior to conditioning can hinder conditioning during advertisements. Not surprisingly, advertising researchers have noted that ads are much more effective for new products with names that have not received much or any exposure prior to the ads (e.g., Baker, 1999; Gresham & Shimp, 1985; Vakratsas & Ambler, 1999). Baker (1999) noted that that high familiarly can reduce the amount of information processed when later exposures occur; in other words, high brand familiarity can mean that providing product information will be less valuable since individuals will not use this information in an ad about a highly familiar product. Overshadowing and Blocking

Blocking and overshadowing involve competition among CSs for processing (or competition between the context cues and a CS) as described

2/16/2011 5:55:21 PM


in the preceding section. McSweeney and Bierley (1984) and van Osselaer (2008) also discuss these phenomena in their reviews. Overshadowing refers to the poor learning that occurs to a CS if it is conditioned (i.e., paired with the US) in the presence of a second CS relative to a condition that receives this CS paired with the US in the absence of a second CS. Hence, if we use a symbol for the “target” CS (e.g., the brand or product that one is interested in assessing for the degree of CR); “CSX”; and we use the symbol “CSA” to refer to the second CS, and we use the symbol “+” to refer to the US, then overshadowing refers to the poor conditioning that occurs to CSX when it is paired with the US when CSA is also present (hence, CSXCSA+ trials). This group’s performance is compared to a control group that simply receives CSX+ trials. Simply put, CSX is learned about much more if it is paired with the US alone rather than in the presence of a second CS (CSA). When an overshadowing effect is obtained, researchers will often say that CSA overshadowed learning to CSX; often CSA is a very salient stimulus and CSX is relatively less salient, explaining why CSA obtains learning at the expense of CSX, that is, CSA benefits from the competition between the cues. For instance, a product name may be presented during an ad along with another salient item and these two stimuli will compete for becoming associated with the US. Salience of the CSs will influence the degree of overshadowing. As mentioned earlier, researchers have noted that some cues will be more salient depending on certain factors such as the amount of involvement (Gorn, 1982). Those interested in purchasing a product may find the product information more salient than those not interested in purchasing the product, and the latter individuals may find the music, if one can think of the music as a cue that may compete with the product for processing, in the ad more salient. Conditioning theorists have known about overshadowing for over 100 years (since the work of Pavlov), but a more recently discovered phenomenon is “potentiation.” Potentiation is the opposite of overshadowing, but it uses the same procedure. Potentiation refers to the increased conditioning to a CS (e.g., a brand or


493

product) due to its pairing with a US in the presence of a second CS. This group’s performance is compared to that of a control group in which the CS was paired with the US in the absence of a second CS to show the influence of the second CS in the other (experimental) group. (The treatment conditions are the same as in an overshadowing experiment.) One major interpretation of potentiation is a mechanism like that of second-order conditioning (described later). It is valuable for marketing researchers to know that having a second stimulus (perhaps a different product) present during an advertisement could potentially promote the conditioning to the target product, although competition for conditioning between the stimuli (overshadowing) may be the more likely outcome in most conditioning situations (i.e., overshadowing is likely more common that potentiation). Blocking (Kamin, 1969) refers to the poor conditioning that occurs to a target CS (a brand) when it is paired with the US in the presence of a second CS, when that second CS was previously paired with the US. That is, if CSA is paired with the US in an initial phase of the experiment (CSA+ trials) and then CSA and CSX are both paired with the US in a second phase of the experiment (CSACSX+), then CSX is poorly learned about. CSA is said to “block” learning about CSX. CSA has an advantage in the competition for learning since it already predicts the US because of its initial training (CSA-US pairings). The performance by CSX for this group is compared to a condition that received the same treatment to the group described except no CSA+ trials occurred in the initial phase; this control group only receives the CSACSX+ trials. The control condition will show a stronger CR to CSX because CSA was not pretrained. Both CSs will still compete for learning on these trials for the control condition, but CSA will not benefit from the great advantage of having been previously paired with the US in the initial phase. The control group produces a greater CR to CSX than the blocking condition. Blocking can be an important phenomenon for marketing professionals interested in producing an effective ad. If the ad involves pairing a product (e.g., sunglasses) with an attractive US

2/16/2011 5:55:21 PM

494


(people having fun on the beach), and this ad contains other cues that already predict fun at the beach, then those latter cues may block the product from being learned about. One troubling issue concerns how to discern which stimuli serve as the US in such a circumstance and what cues are stimuli that might compete with the target CS (the product). For instance, if the ad contains people playing sand volleyball, is this cue part of the US—people having fun at the beach? Or is it a competing CS—a cue that already predicts fun at the beach which will then prevent learning about the product? The answer is not clear to us, but marketing researchers may appreciate knowing about the different possible outcomes and the conditioning processes believed to underlie these effects. In support of finding competition between cues in an advertising situation, Van Osselaer and Alba (2000) found that learning about one characteristic of a brand (e.g., that the brand is high quality) will result in poor subsequent learning about more reliable information (see Oxoby & Finnigan, 2007 for a discussion). Oxoby and Finnigan (2007) point out that such an advantage of first-learned information means that companies should be careful about the initial messages that are delivered to consumers since subsequent information may not be adequately processed. They also address how “brand extentions” may not receive adequate processing for similar reasons. Indeed, Oxoby and Finnigan found blocking not just for subsequently exposed attributes about that same product but also for related products. Janiszewski and van Osselaer (2000) examined interactions among brand names and found that such interactions between two different brand names can occur during advertising. That is, when a product has two brand names, a regular brand and a “subbrand,” associated with it (e.g., a certain brand of ice cream with Hershey’s chocolate mixed in). These researchers mention that such interactions are consistent with connectionist, least-mean-squares models (e.g., the Rescorla-Wagner model, Rescorla & Wagner, 1972) of conditioning rather than models in which associations are formed independently of each other.


Van Osselaer and Janiszewski (2001, as reviewed by Oxoby & Finnigan, 2007) examined the learning processes that underlie consumers’ processing according to the human associative memory (HAM) model (Anderson & Bower, 1973) and adaptive network (AN) models. The former model involves associations being acquired independently, whereas AN models allow for associations to compete (they are not necessarily learned independently). Van Osselaer and Janiszewski conclude that HAM models describe performance when the participants do not have a specific processing goal, whereas AN models describe performance when the subject does have a processing goal. Given their import in conditioning theory, the effects of competition may receive more empirical attention in the future. Second-Order Conditioning

Second-order conditioning refers to the conditioning that occurs to a target CS because that CS was paired with another (nontarget) CS, and this latter CS had been previously paired with the US. That is, the target CS (CSA) is paired with the US in the initial phase of the experiment (CSA+ trials). Then, CSX is paired with CSA. Note that CSX is never paired with the US, yet CSX produces a CR. Second-order conditioning is held as evidence that contiguity between the CS (CSX) and the US is not necessary to produce conditioning. A similar procedure, sensory preconditioning, is essentially the same as secondorder conditioning except that the two phases of conditioning are reversed: CSA and CSX are paired together first (CSACSX trials) and then CSA is paired with the US (CSA+). As with second-order conditioning, a CR occurs to CSX as a result of this procedure. Second-order conditioning can be said to occur if one product is paired with an attractive US (and the product now becomes attractive). A second product is then paired with the first product and the second product is now attractive because of its association with the first product. Blair and Shimp (1992) obtained secondorder conditioning during an advertisement experiment in which music (music served as the

2/16/2011 5:55:21 PM


CSA, which was perceived as very neutral to the participants at the start of the experiment) was paired with an aversive US (an unpleasant, boring experience while reading a selection from a certain textbook). Then CSX (a fictitious sports brand) was paired with the music (CSA). Firstorder conditioning to the music was found, and second-order conditioning to the brand was also obtained. That is, when subjects were tested on their liking of the music (CSA) after it had been paired with the aversive event (the US), they found the music aversive. The subjects also found the brand (CSX) aversive after it had been paired with the music (CSA) a second-order conditioning effect. Note that the brand itself was never paired with the US (the aversive event). One major explanation for second-order conditioning (and sensory preconditioning) is that the subject forms an association between the two CSs. CSA becomes associated directly with the US. Cognitive conditioning theorists will claim that, if an association between CSA and the US is formed, the presentation of CSA will activate a representation of CSA, in the individual’s memory network. The activation of this representation (CSA) will cause, via the association between CSA and the US, the US representation in memory to become activated. Since CSA and CSX are associated, when CSX is presented after all phases of conditioning are completed, it will be able to activate a representation of CSA due to the association between the CSs. The activation of CSA will cause activation of the US. This indirect activation of the US when CSX is exposed (i.e., via the CSA representation) is the reason for the CR to CSX (see Pearce, 2008 for a discussion). The implications of second-order conditioning is that marketing professionals may find that associations are formed between stimuli during ads and changing the value of one of these stimuli may influence the value of the other stimulus. Effects of Contextual Cues

Many marketing researchers mention the impact of contextual factors (e.g., Allen & Shimp, 1990; Cohen, 1990), but what is meant by that term


495

varies greatly. We will view contextual cues the way they are often viewed in conditioning experiments: environmental cues, including mood and circadian cues. Allen and Shimp (1990) pointed out that contextual cues can be critical as determinants of the CR; and, given its import in many conditioning theories and phenomena (e.g., Balsam & Tomie, 1985), more experimental work needs to be done on contextual factors and advertising. Janiszewski and Warlop (1993) noted, as mentioned earlier, that changes in the attentional response or orienting response to a brand or product may be context dependent such that, even if the response has dissipated for a certain product or brand, presentation of the item in a new context (at the point of purchase) may increase that orienting or attentional response. Stammerjohan et al. (2005), as noted earlier, mentioned that presenting information in more than one context or modality would improve memory and attitude change. One interesting context effect in the conditioning literature should be noted before we move to the next section of this review. “Comparator theories” of classical conditioning (Gibbon & Balsam, 1981; Miller & Matzel, 1988) argue that the reason that long intertrial intervals and short CS durations enhance conditioning is due to a comparator process in which the durations of CS exposure and context exposure are compared. Conditioned responding is greater to the extent that the CS duration is short and the context duration (the intertrial interval) is long; and it is poorer to the extent that the CS duration is long and the contextual period is short (Miller & Schachtman, 1985). So these theories would predict that conditioning to an ad will be greater if the session period (the “period” in which the brand name is paired with the US, which might be the entire ad itself) is lengthened. Shorter CS (brand name) exposure will also help conditioning. In sum, if the entire commercial happens to serve as a “context” for the CS (the presentation of the brand name) then longer commercials might facilitate conditioning. For instance, let’s assume that the ad shows a person using a lawn mower of a particular brand (CS) and the commercial shows that

2/16/2011 5:55:22 PM

496


using the mower leads to some happy outcome (the US); that is, the person on the mower is happy and his or her neighbors wave (they like him or her, in part, because of the lawn mower). If this “trial” is contained within a commercial (context) and the trial is longer rather than shorter, then conditioning to the brand may be enhanced. Conversely, if the ad simply gives the CS-US pairing in 10 seconds and that is the extent of the commercial, then conditioning may be poor (meaning many trials might be needed to get conditioning), according to comparator theory. Hence, it could turn out to be more lucrative to have a longer, albeit more effective, commercial than a shorter commercial that requires many exposures. As another issue pertaining to the role of context in advertising, Gordon (2001) mentions “need states” as a context for the consumption or purchase of products (e.g., coffee). Clearly, researchers and marketing specialists need to take contextual cues into consideration when assessing the efficacy of advertisements. Cognition and Affect

Vakratsas and Ambler (1999) as well as many other researchers (e.g., Stout & Rust, 1993) have discussed the complex relationship between cognitive processes, including beliefs and affective processes, during advertising. They refer to “affect” and “cognition” as “intermediate effects” in modeling the processing that occurs during the viewing of an ad. The issue of “which comes first: emotion or cognition?” is as old as the famous Cannon-Bard/James-Lang debate of a century ago and other early debates in the literature exist (e.g., Lazarus, 1981; Zajonc, 1980). Cohen (1990) assumes that affective traces must be interpreted by the cognitive system before they can become manifest in behavior. Vakratsas and Ambler’s fine review article on models of advertising show that there are many different ways of conceptualizing the relationship between affect and cognition (see also Cohen, 1990). Among the many potential processes by which conditioning might change behavior following advertising, “affect transfer” and a “change in beliefs” are two such mechanisms.


Kim et al. (1998) investigated the role of forming beliefs as well as acquiring affective properties of a product during advertising. They reported that beliefs are not the entire story to successful conditioning in that affective properties can also be transferred from the US to the CS. The two effects are not mutually exclusive during an advertising experience (see also Kim et al., 1996). Lutz (1985) claimed that affect transfer is more likely when low involvement occurs. Kim et al. (1998) used a single conditioning trial and obtained conditioning; they concluded that only affect (not belief) could have been responsible for the conditioning effect they observed because, they claimed, the US they used did not provide any belief-related information. However, one could say that the participants generated belief information on their own through elaboration. Nonetheless, Kim et al. concluded that multiple conditioning trials produce belief information in addition to the previously produced affect, and they stated that both affect and belief can occur during advertising. When they used multiple pairings, they found that the size of the effects of these two processes (affective and belief formation) were found to be statistically indistinguishable from each other. They concluded that the learning of affective properties was stronger than the forming of cognitive beliefs with a single trial but both processes are equally influential with multiple trials. Kim et al. discusses an article by Pechmann and Stewart (1988) in which affective conditioning occurred with fewer trials than cognitive-based ads. Allen and Madden (1985) argue against affect transfer as a mechanism of classical conditioning effects during advertising effects. Kroeber-Riel (1984) believed that classical conditioning occurs without cognition, while stating that limiting cognitive processes during classical conditioning is important. He may have been assuming that conditioning would be worse if such cognitive processes (such as awareness) occurred, which we know to not be true because cognition/awareness can enhance conditioning (see next section). Kroeber-Riel also said that cognitive responses are always accompanied with emotional responses.

2/16/2011 5:55:22 PM


Many researchers agree that noncognitive processing of advertisements (suggesting less involvement as will be discussed later) lead to more positive evaluation of the ad than more in-depth cognitive processing. Vakratsas and Ambler (1999), when discussing advertising, also pointed out that, according to some models, the order in which cognitions, affective responses, and memories for past experiences occur during ad viewing depends on the level of involvement. They also mention that “affect is relatively more important in low involvement and nonelaborative situations” and that “cognitive and affective beliefs may occur independently in these circumstances” (see also Janiszewski, 1988). They point out how hard it is to dissociate cognition and affect; for instance, asking about feelings will give rise to cognitive processes. This topic of modeling of the processes that occur during ad viewing is so complex that this terse sketch of the issue hardly does it justice; but viewers are encouraged to turn to the Vakratsas and Ambler (1999) review for much more detail about the subject. Evaluative Conditioning (as a Type of Referential Learning) Versus Classical Conditioning (Expectancy Learning) and the Role of Awareness During Advertisements

It is useful to distinguish between the implicit and explicit processes that can underlie classical conditioning, and the degree that conscious (or nonconscious) and effortful (or noneffortful) processing is involved (e.g., Baeyens, Crombez, Van Bergh, & Esten, 1988; Dawson, Beers, & Kelly, 1982; Gordon, 2001). Many possibilities exist. A learning experience can involve awareness or lack of awareness and may be effortful or noneffortful. These types of processes can be applied to the experience of association formation, as well as to the time of performance when this learned information is used for behavior. A stimulus can acquire an association with another event automatically (i.e., without any cognitive effort or awareness of the processes involved) or explicitly (with awareness). This stimulus can evoke the conditioned response


497

after such learning through an automatic, implicit process or does so explicitly (i.e., the subject is consciously aware that this stimulus predicts an outcome stimulus and this awareness promotes the CR). Allen and Shimp (1990) correctly point out that classical conditioning is often described as a theoretical explanation of the changes in behavior that can occur as a result of advertising (rather than as a procedure). Classical conditioning is the mechanism that is usually associated with the processing of ads with low-involvement products. This theoretical explanation usually makes assumptions about the processes that underlie such learning; that is, they assume the conditioning is a noncognitive process and it might be assumed that the process is implicit and automatic. However, these authors (p. 22) also point out that awareness during classical conditioning is possible and can even be expected during such conditioning. Kahle et al. (1987) claim that conditioning theory requires that classical conditioning in adults occurs without awareness. Although, as Kahle et al. point out, conditioning was discovered and developed by early researchers with such a view in mind, it seems bold and erroneous to make such a claim in the 1980s. Kahle et al. also mention that awareness during conditioning “implies that participants grasp the nature of the hypotheses of the study” [italics added]. Although awareness can give rise to knowledge of the hypothesis, this implication is a bold assumption. Other researchers (e.g., Brewer, 1974) have claimed that awareness is necessary for conditioning. Allen and Janiszewski (1989) found that contingency awareness existed when conditioning occurred in an advertising situation, suggesting that conditioning may not occur without awareness. The nonaware participants did show a small conditioning effect in their initial experiment, and so their data are not unequivocal with respect to the role of awareness on conditioning. Priluck and Till (2004) found that awareness of the contingency in an ad increased the degree of conditioning. Bierley et al. (1985) found conditioning of colors paired with attractive music; and awareness increased conditioning

2/16/2011 5:55:22 PM

498


but conditioning did not require it. Shimp, Stuart, and Engle (1991) found that awareness increased conditioning as well. Baeyens and colleagues, albeit not using an advertising situation, have found that evaluative conditioning in humans, a type of conditioning that seems quite related to advertising effects since outcomes are used to modify the attitude toward a stimulus, is not dependent on contingency awareness (see Chapter 18, this volume). Baeyens et al. (1988) provided evidence that evaluative conditioning is retained for at least a 2-month period and is resistant to extinction. Extinction refers to the typical loss in conditioned responding when the CS is now presented without the CS following the original conditioning (pairings of the CS and the US). Baeyens and De Houwer and colleagues have discussed the difference between signal learning and evaluative conditioning (the latter is also assumed to be a form of referential learning). Expectancy learning involves a cue that predicts or signals the presence or absence of an outcome, and an expectancy regarding this outcome is produced. This theoretically defined form of learning corresponds to Pavlovian conditioning or classical conditioning. In referential learning, the CS makes one think (consciously or unconsciously) of the outcome without activating an expectancy of the US (see De Houwer et al., 2001). It may be valuable to map various advertising effects onto these processes; that is, is the processing that occurs during ad exposure more like expectancy learning or referential learning? The current and published work on contingency awareness during ad viewing may begin to answer these questions. It is easy to imagine a larger wave of assent for a referential learning view of ad processing than a signal learning view. Nonetheless, I can imagine many young adults claiming quite consciously that they expect to have a good time when they are drinking Bud Lite. Bud Lite signals a good time. These conclusions could be the results of experience with Bud Lite or the result of viewing ads, or both. But referential learning also seems quite common in low-involvement advertising effects, and many would claim that explicit awareness of the contingency is not needed for such effects. Baeyens


et al. (1988) also dissociated expectancy learning from referential learning in that signal learning may be less resistant to extinction. Reflecting on Brewer’s conclusion that explicit knowledge about the relationship between a CS and a US can influence the CR, we agree with the conclusion of McSweeney and Bierley (1984) that just because manipulation of awareness can influence conditioning does not necessarily mean that awareness is necessary. Awareness of the contingency appears to enhance conditioning, but it is likely not necessary for conditioning to occur. It is our suggestion that the referential-signal learning distinction is an extremely important one (see also the Introduction to Reilly & Schachtman, 2009), but that implicit and explicit processes can apply to both of them (depending on the circumstances). One very promising approach that mirrors much of the research in animal conditioning and human cognition is work that attempts to empirically dissociate different processes that might be producing an effect. Such research makes predictions that one outcome of the experiment will occur if one theoretical process is at work while another outcome will occur if the alternative process is influencing the subjects. This approach was pursued by Janiszewski and Warlop (1993).

PERSONALITY AND INDIVIDUAL DIFFERENCE VARIABLES This section will review a few characteristics of the individual that can affect the influence of conditioning during advertising. Many advertising researchers mention in their reports that they acknowledge individual differences among people with respect to such effects; and, rather than approaching an advertisement with a tabula rasa, people are exposed to a trial while possessing a history of experience as well as personality differences and acquired biases and heuristics. High Versus Low Involvement

Many marketing researchers have examined the role of involvement during advertising. Involvement is a concept initiated by Krugman

2/16/2011 5:55:22 PM


(1967), but there has been little agreement on the definition of involvement (see, for example, Burnkrant & Sawyer, 1983). Involvement can be said to refer to the connections a person has with the stimulus prior to the target experience of interest. High involvement, of course, suggests a stronger and qualitatively different set of connections with the stimulus, which often means that such a person might be interested in purchasing the item. The definition of involvement provided by Celsi and Olsen (1988) is the degree of perceived personal relevance. Cohen (1990) describes high involvement as perceiving the product as having a large number of perceived benefits. Some individuals may have low involvement for a particular product, brand, stimulus, or US, whereas others may have high involvement. Involvement can have a large impact on conditioning. Batra and Ray (1983) discovered that different processes may result from exposure to advertisements with low versus high involvement. Specifically, low involvement results in simple awareness (a low level of cognitive process), which may result in action, but little affective attitude change will result from the process; whereas high involvement can results in awareness, comprehension, action, and then affective attitude change. Lutz (1985) stated that affect transfer is higher when low involvement occurs. Gardiner et al. (1985) provide an extensive discussion of involvement. As mentioned earlier, Gardiner et al. found that memory differences occur for low- versus high-involvement processing. Grossman (1996) found more conditioning for highly involved participants. Priluck and Till (2004) noted that highly involved subjects will use belief information, whereas those low in involvement may only be subject to affective transfer. This point appears to conflict with the conclusions of Batra and Ray, who suggest that beliefs are needed for affective transfer to occur. Celsi and Olsen (1988) found that participants devote more attention to stimuli (brands) if they have high involvement (i.e., described as situational factors of the person’s environment that contribute to personal relevance). A high amount of a second type of involvement, “intrinsic involvement,” also resulted in a large amount


499

of time attending to the stimulus, but it did not produce an independent source of attention (based on other measures they looked at, see Celsi & Olsen, 1988). Note that Celci and Olsen found that greater involvement produced more time attending to the information, whereas Gardner et al. (1985) found that the noninvolved group spent a longer time looking at each ad. It should be pointed out that Gardner et al. had the noninvolvement condition engaged in a mundane but potentially demanding task in which they had to look for grammatical, word-sound, and conceptual features, which can explain why this group looked at the ad such a long time. Gardner et al. (1985) distinguish between stages of processing based on low and high involvement such that low involvement while viewing advertisements involves basic, minimal comprehension in which the basic meaning of the elements in an ad are recognized (see also Vakratsas & Ambler, 1999), whereas high involvement produces elaboration such that internally generated information occurs. They discussed the issue of distraction during ad presentation, noting it can produce more favorable brand attitudes because distraction requires attention that disrupts elaborative processes (such as the production of counterarguments). If you are not interested in purchasing the product, then you may not engage in elaboration, but purchase-oriented individuals will elaborate and make inferences, and associate these inferences with the product (and counterarguments may arise). Gorn (1982, Experiment 2) examined subjects that were in a non-decision-making context (e.g., what can be considered eliciting low involvement) and those in a decision-making context with respect to their relative sensitivity to background cues (i.e., music) and to product information. Gorn found that non-decisionmaking participants were most influenced by the music, whereas decision makers (more involved) were most influenced by the information provided. Some additional points about involvement will be made in the following text, but we will not have printed space to be able to elaborate on these issues at this time. Bruner (1990) discusses

2/16/2011 5:55:22 PM

500


a fact likely well known to marketing researchers— some items elicit more cognitive involvement than others for most people. For instance, items with high cognitive involvement include appliances and vehicles, whereas low cognitive involvement occurs for other products such as jewelry and beer (and Vakratsas & Ambler, 1999 noted that frequently purchased packaged goods often induce low involvement). Complex ads require inferences that use cognitive elaborative analysis and high-involvement occurs (see Vakratsas & Ambler, 1999 for a discussion). Alternatively, it has been said that simple conditioning effects stem from less complex ads, such ads will be processed without elaborative cognitive evaluation, and lower involvement will occur (Petty, Cacioppo, & Schumann, 1983). Vakratsas and Ambler stated that with low involvement, “advertising merely serves to reinforce behavior rather than causing it” and that “the ‘weak theory’ of advertising (Jones, 1990) … is similar to operant, or instrumental, conditioning … .” This article mentions that, according to one model (the IIRM model), ads influence low-involvement situations by increasing awareness via lower order beliefs and introducing uncertainty, such that experience with the product will resolve the uncertainty and allow the expectations to be confirmed or not. Higher order beliefs occur with high-involvement products or after many purchases of a product (but note that frequent purchases may produce lowinvolvement interactions with the product even if higher order beliefs exist). Need for Cognition

Priluck and Till (2004) examined the role of “need for cognition” on advertising. Need for cognition (Cacioppo & Petty, 1982) is a personality construct in which individuals enjoy thinking about events and engaging in difficult cognitive processes. Priluck and Till found that individuals with high need for cognition showed the greatest conditioning during advertisements, and they recalled the information better later. They stated that these participants were more likely, due to their extra thought processing, to be aware of the pairing of the events. Priluck and Trill also suggested ways that one can inspire


other individuals to engage in such extra processing: (1) pay them for paying attention; (2) tell them that they will be questioned about the material later; or (3) make the information relevant to their lives so they are motivated to process the information. Perhaps related to high need for cognition, Cohen (1990) discussed the dimension of “cognitive complexity” and research findings by Zinkhan and Martin (1983) that those individuals that are high in cognitive complexity preferred more complex ads (and those low in cognitive complexity prefer simple ads) and so the adage that “simple ads are always better” may not always hold true (see Cohen, 1990). The Role of Prior Belief

Many marketing researchers (e.g., Kim et al., 1998; Stammerjohan et al., 2005) note that an individual possesses prior beliefs about a product and this can greatly influence the individual’s current assessment of the product when viewing an ad. Vakratsas and Ambler (1999, p.27) remind us that: “… the consumer’s mind is not a blank sheet awaiting advertising but rather already contains conscious and unconscious memories of product purchasing and usage. Thus, behavior feeds back to experience …” Stammerjohan et al. (2005) point out that individuals that already have opinions about a familiar product do not always process new information very effectively. When analyzing the relationship between two types of information—the current “situational data” in the present advertisement and the data from past experience—we cannot help but allude to Alloy and Tabachnik’s (1984) article on the relationship between these types of information. Like the conclusions of many marketing researchers, this article points out how influential previously acquired knowledge can be for the processing of current information; they discuss many of the learning phenomena mentioned in this review, including latent inhibition, blocking, and the US preexposure effect. However, we wish to point two opposing processes that highlight this interaction between past and current information. On the one hand, there are a lot of data showing that individuals

2/16/2011 5:55:22 PM


(relatively speaking) disregard current data because of knowledge that has already been acquired. On the other hand, effects such as the “hindsight bias” (see Hawkins & Hastie, 1990) show that subjects exposed to contemporaneous events (e.g., the outcome of an election) will bias their past processing greatly in favor of making that event fit into their existing schema. Hence, if Candidate A wins the election, people will distort their own history to convince themselves (erroneously) that they predicted Candidate A would win “all along.” Future research should isolate which variables determine when current data are weighed so heavily that they result in the transformation of existing data in memory (e.g., hindsight bias) and when current data are more or less neglected because of the strong influence of previously acquired knowledge (e.g., blocking, latent inhibition).

CONCLUSION As most marketing researchers know, certain factors have been found to have a large impact on the effectiveness of advertising, including (1) the degree of involvement; (2) the temporal placement of the brand name during the ad; (3) the use of music; (4) the relationship between product information and affective qualities; and (5) the extent that the processing during an ad is implicit (occurs without awareness) or explicit (occurs with awareness), to name just a few. Given that many (or most ads) involve a classical conditioning procedure, it is not surprising that these processes are also critical in the field of conditioning theory per se and they also illustrate how conditioning theory can shed light on the processing of ads. As a final note, we wish to point out three additional issues in conditioning research that might offer interest for marketing researchers. These three conditioning phenomena are, in our opinion, among the more recent and fascinating areas of such work. First, marketing researchers should be aware that conditioning can occur for cues that are not present on a particular conditioning trial. For example, if two CSs are associated (CSA and CSB) and two other events are associated (CSC and the US), if CSA and CSC are


501

presented, then CSB and the US can become associated since CSA will activate the representation of CSB and CSC will activate a representation of the US. Hence, the representations of CSB and the US will be contiguously active in memory. Research has found many instances in which CSs that are not present on a trial are changed in their associative strength (Dickinson & Burke, 1996; Holland & Wheeler, 2009; Van Hamme & Wasserman, 1994). Such outcomes, sometimes called “representation-mediated conditioning” are an important and exciting area of conditioning research and may apply to a brand that is not even present during an ad. Secondly, many instances in which an individual shows no evidence of having acquired information are cases in which the information has been acquired but is not retrieved or performed due to lack of motivation or poor retrievability of the information (Lewis, 1979; Miller et al., 1986; Tolman & Honzik, 1930; Warrington & Weiskrantz, 1968). Many of the phenomena discussed in this chapter (blocking, latent inhibition, overshadowing) in which a relatively poor CR is observed have been shown to be due to a retrieval problem rather than a lack of acquisition of the association (Miller et al., 1986). Hence, marketing researchers may value knowing that if few trials are used (or a brief stimulus presentation) and that causes the individual to show little evidence of a change in attitude, acquisition of such a change in attitude or affect might have occurred although it is not expressed. Certain tests can be used to show that processing did occur in the past even though it is not presently manifest in behavior. The effects reported in this chapter illustrate the ways in which conditioning and marketing can benefit from the interdisciplinary confluence of findings and theories. As mentioned in this chapter, D’Souza and Rao (1995) describe two different models of information processing: an accumulation model and a replacement model (see Stewart, 1989). The accumulation model claims that new information is acquired alongside of the old information, and it is the relative strength of the new and old responses that determine which will be expressed. A view of performance deficits that focuses on retrievability will

2/16/2011 5:55:22 PM

502


likely claim that new information is acquired along with old information, but the less seemingly adaptive information will be poorly retrieved and other, more adaptive information will be better retrieved (Miller et al., 1986). Indeed, newly acquired information can compete for retrievability with the earlier learned information. Hence, associations compete for retrievability and the relative retrievability of the cues/brands will determine which are expressed in behavior. Finally, as noted earlier, brand competition effects during an ad have begun to be explored by advertising researchers (Janiszewski & van Osselaer, 2000; van Osselaer & Alba, 2000; van Osselaer & Janiszewski, 2001). Moreover, van Osselaer and Janiszewski (2001) also examined a conditioning phenomenon often referred to in the conditioning literature as “retrospective revaluation” or, as van Osselaer and Janiszewski called it, “backlooking learning.” This phenomenon involves competition between features or brands (CSA and CSB) such that one brand or product (CSA) is dominant in getting control over the participant’s behavior (e.g., purchasing power or attention or memory) as a result of the ad. Then, subsequently, this brand or feature has its status changed such that it loses value (the feature or product becomes less credible or less interesting). When this happens, even without any additional presentations of the other brand (CSB), this alternative brand or product is increased in its status. That is, CSB now is improved in its ability to influence the person’s performance, even though it had not been presented between the viewing of the ad and the final assessment of this behavioral control (i.e., only CSA was manipulated). The competition between cues can be influenced by the later change in status of one cue, and this will influence the other cue. This finding, retrospective revaluation, has been very impactful in the human contingency judgment literature as well as in animal conditioning; and van Osselaer and Janiszewski (2001) were the first to examine its potential in an advertising situation (see also Chapters 1 and 8, this volume, for additional discussion or examples of this effect).


There are many ways that the fields of conditioning and marketing potentiate each others’ findings and theories. Many additional conditioning phenomena have been and will continue to be applied to an advertising setting (e.g., Till & Priluck, 2000 and its application to brand extensions) for the mutual benefit of both fields.

REFERENCES Allen, C. T., & Janiszewski, C. A. (1989). Assessing the role of contingency awareness and attitudinal conditioning with implications for advertising research. Journal of Marketing Research, 26, 30–43. Allen, C. T., & Madden, T. J. (1985). A closer look at classical conditioning. Journal of Consumer Research, 12, 301–315. Allen, C. T., & Shimp, T. A. (1990). On using classical conditioning methods for researching the impact of ad-evoked feelings. In S. J. Agras, J. A. Edell, & T. M. Dubitsky (Eds.), Emotion in advertising: Theoretical and practical explorations. Westport, CT: Quorum Books. Alloy, L., & Tabachnik, N. (1984). Assessment of covariation by humans and animals: The joint influence of prior expectations and current situational information. Psychological Review, 91, 112–149. Alpert, F. H., & Kamins, M. A. (1995). An empirical investigation of consumer memory, attitude and perceptions toward pioneer and follower brands. Journal of Marketing, 59, 34–45. Anderson, N., & Bower, G. H. (1973). Human associative memory. Oxford, England: Winston & Sons. Baeyens, F., Crombez, G., Van Den Bergh, O., & Eelen, P. (1988). Once in contact always in contact: Evaluative conditioning is resistant to extinction. Advances in Behavior Research and Therapy, 10, 179–199. Baker, W. E. (1999). When can affective conditioning and mere exposure directly influence brand choice? Journal of Advertising, 28, 31–46. Baker, W. E., Honea, H., & Russell, C. A. (2004). Do not wait to reveal the brand name: The effect of brand-name placement on television advertising effectiveness. Journal of Advertising, 33, 77–85. Baeyens, F., Crombez, G., Van den Bergh, O., & Eelen, P. (1988). Once in contact always in

2/16/2011 5:55:22 PM


contact: Evaluative conditioning is resistant to extinction. Advances in Behavior Research and Therapy, 10, 179–199. Balsam, P. D., & Tomie. A. (1985). Context and learning. Hillsdale, NJ: Erlbaum. Batra, R., & Ray, M. L. (1983). Advertising situations: The implications of differential involvement and accompanying affective responses. In R. J. Harris (Ed.), Information processing research in advertising (pp. 127–151). Hillsdale, NJ: Erlbaum. Bierley, C., McSweeney, F. K., & Vannieuwkerk, R. (1985). Classical conditioning of preferences for stimuli. Journal of Consumer Research, 12, 316–323. Blair, M. (1987). An empirical investigation of advertising wearin and wearout. Journal of Advertising Research, 27, 45–50. Blair, M. E., & Shimp, T. A. (1992). Consequences of an unpleasant experience with music: A second-order negative conditioning perspective. Journal of Advertising, 21, 35–43. Bonardi, C., & Ong, S. Y. (2003). Learned irrelevance: A contemporary overview. Quarterly Journal of Experimental Psychology, 56B, 80–89. Brewer, W. F. (1974). There is no convincing evidence for operant or classical conditioning in adult humans. In W. B. Weimer, & D. S. Palermo (Eds.), Cognition and the symbolic processes. Hillsdale, NJ: Lawrence Erlbaum. Brown, P. L., & Jenkins, H. M. (1968). Autoshaping of the pigeon’s key-peck. Journal of the Experimental Analysis of Behavior, 11, 1–8. Bruner, G. C., II. (1990, October). Music, mood and marketing. Journal of Marketing, 54(4), 94–104. Burke, R. R., & Srull, T. K. (1988). Competitive interference and consumer memory for advertising. Journal of Consumer Research, 15, 55–68. Burnkrant, R. E., & Sawyer, A. G. (1983). Effects of involvement and message content on information-processing intensity. In R. J. Harris (Ed.), Information processing research in advertising (pp. 43–64). Hillsdale, NJ: Erlbaum. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116–131. Cacioppo, J. T., & Petty, R. E. (1985). Central and peripheral routes to persuasion: The role of message repetition. In A. A. Mitchell & L. F. Alwitt (Eds.), Psychological processes and advertising effects (pp. 91–112). Hillsdale, NJ: Erlbaum.


503

Celsi, R. L., & Olson, J. C. (1988). The role of involvement in attention and comprehension processes. Journal of Consumer Research, 15, 210–224. Cohen, J. (1990). Attitude, affect, and consumer behavior. In B. M. Moore & A. M. Isen (Eds.), Affect and social behavior (pp. 152–206). New York, NY: Cambridge University Press. Crowder, R. G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum. Darley, W. K., & Lim, J-S. (1993). Assessing demand artifacts in consumer research: An alternative perspective. Journal of Consumer Research, 20, 489–495. Dawson, M. E., Beers, J. R., Schell, A. M. & Kelly, A. (1982). Allocation of cognitive processing capacity during human autonomic classical conditioning. Journal of Experimental Psychology: General, 111, 273–295. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127, 853–869. Dickinson, A., & Burke, J. (1996). Withincompound associations mediate the retrospective revaluation of causality judgments. Quarterly Journal of Experimental Psychology, 49B, 60–80. D’Souza, G., & Rao, R. C. (1995). Can repeating an advertisement more frequently than the competition affect brand preference in a mature market? Journal of Marketing, 59, 32–42. Ehrenberg, A. S. C. (1974). Repetitive advertising and the consumer. Journal of Advertising Research, 14, 25–34. Freeman, K. B., & Riley, A. L. (2009). The origins of conditioned taste aversion learning: A historical analysis. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 9–36). New York, NY: Oxford University Press. Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Gardiner, M. P., Mitchell, A. A., & Russo, J. E. (1985). Low involvement strategies for processing advertisements. Journal of Advertising, 14, 4–12, 56. Gibbon, J., & Balsam, P. D. (1981). Spreading association in time. In C. M. LoCurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York, NY: Academic Press.

2/16/2011 5:55:22 PM

504


Gordon, W. (2001). The darkroom of the mind–What does neuropsychology now tell us about brands? Journal of Consumer Behavior, 1, 280–292. Gordon, W. C., & Weaver, M. S. (1988). The role of contextual stimuli in the blocking paradigm. Psychobiology, 16, 59–66. Gorn, G. J. (1982). The effects of music in advertising on choice behavior: A classical conditioning approach. Journal of Marketing, 46, 94–101. Gresham, L. G., & Shimp, T. A. (1985). Attitude toward the advertisement and brand attitudes: A classical conditioning perspective. Journal of Advertising, 14, 10–17. Grossman, R. P. (1996). Neo-classical conditioning and the elaboration likelihood model: A new perspective on the role of involvement. Dissertation Abstracts International, 56, 4859. Grossman, R. P., & Till, B. D. (1998). The persistence of classically conditioned brand attitudes. Journal of Advertising, 27, 23–31. Hall, G., & Channell, S. (1985). Differential effects of contextual change on latent inhibition and on the habituation of the orienting response. Journal of Experimental Psychology: Animal Behavior Processes, 11, 470–481. Hall, G., & Schachtman, T. R. (1987). Differential effects of a retention interval on latent inhibition and habituation of an orienting response. Animal Learning and Behavior, 15, 76–82. Hawkins, S. A., & Hastie, R. (1990). Hindsight: Biased judgments of past events after the outcomes are known. Psychological Bulletin, 107, 311–327. Holland, P. C., & Wheeler, D. S. (2009). Representationmediated food aversions. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Neural and behavioral processes (pp.196–225). New York, NY: Oxford University Press. Hung, K. (2001). Framing meaning perceptions with music: The case of teaser ads. Journal of Advertising, 30(3), 39–49. Janiszewski, C. (1988). Implications of hemispheric processing theory for consumer research. Dissertation Abstracts International, 48, 2107. Janiszewski, C., & van Osselaer, S. M. J. (2000). A connectionist model of brand-quality associations. Journal of Marketing Research, 37, 331–350. Janiszewski, C., & Warlop, L. (1993). The influence of classical conditioning procedures on subsequent attention to the conditioned brand. Journal of Consumer Research, 20, 171–189. Johnston, W. A., & Hawley, K. J. (1994). Perceptual inhibition of expected inputs: The key that opens closed minds. Psychonomic Bulletin and Review, 1, 56–72.


Jones, J. P. (1990). Advertising: Strong force or weak force? Two views an ocean apart. International Journal of Advertising, 9, 233–246. Kahle, L. R., Beatty, S. E., & Kennedy, P. (1987). Comment on classically music on ad processing: A contingency explanation. Journal of Marketing, 57(4), 114–125. Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice-Hall. Kamin, L. J. (1969). Selective association and conditioning. In N. J. Mackintosh & W. K. Honig (Eds.), Fundamental issues in associative learning (pp. 42–64). Halifax, Nova Scotia: Dalhousie University Press. Kellaris, J. J., Cox, A. D., & Cox, D. (1993). The effect of background music on ad processing: A contingency explanation. Journal of Marketing, 57, 114–125. Kim, J., Allen, C. T., & Kardes, F. R. (1996). An investigation of the meditational mechanisms underlying attitudinal conditioning. Journal of Marketing Research, 33, 318–328. Kim, J., Lim, J. S., & Bhargara, M. (1998). The role of affect in attitude formation: A classical conditioning approach. Journal of the Academy of Marketing Science, 26, 143–152. Kroeber-Riel, W. (1984). Emotional product differentiation by classical conditioning (with consequences for the “low involvement hierarchy”). Advances in Consumer Research, 11, 538–543. Krugman, H. E. (1967). The measurement of advertising involvement. Public Opinion Quarterly, 30, 583–596. Krugman, H. E. (1972). Why three exposures may be enough. Journal of Advertising Research, 12, 11–14. Lazarus, R. S. (1981). A cognitivist’s reply to Zajonc on emotion and cognition. American Psychologist, 36, 222–223. Lewis, D. J. (1979). Psychobiology of active and inactive memory. Psychological Bulletin, 86, 1054–1083. Lubow, R. E., & Moore, A. U. (1959). Latent inhibition: The effect of nonreinforced preexposure to the conditioned stimulus. Journal of Comparative and Physiological Psychology, 52, 415–419. Lutz, R.J. (1985). Affective and cognitive antecedents of attitude toward the ad: A conceptual framework. In L. F. Alwitt & A. A. Mitchell (Eds.), Psychological processes and advertising effects: Theory, Research and Application. Hillsdale, NJ: Erlbaum. Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.

2/16/2011 5:55:22 PM


Macklin, M. C. (1996). Preschoolers’ learning of brand names from visual cues. Journal of Consumer Research, 23, 251–261. Matzel, L. D., Schachtman, T. R., & Miller, R. R. (1988). Learned irrelevance exceeds the sum of CS-preexposure and US-preexposure deficits. Journal of Experimental Psychology: Animal Behavior Processes, 14, 311–319. McSweeney, F. K., & Bierley, C. (1984). Recent developments in classical conditioning. Journal of Consumer Research, 11, 619–631. Miller, R. R., Kasprow, W. J., & Schachtman, T. R. (1986). Retrieval variability: Psychobiological sources and consequences. American Journal of Psychology, 99, 145–218. Miller, R. R., & Matzel, L. D. (1988). The comparator hypothesis: A response rule for the expression of associations. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 22, pp. 51–92). San Diego, CA: Academic Press. Miller, R. R., & Schachtman, T. R. (1985). Conditioning context as an associative baseline: Implications for response generation and the nature of conditioned inhibition. In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 51–88). Hillsdale, NJ: Erlbaum. Mitchell, A. A. (1993). Attitude toward the advertising effects over time and in attitude change situations. In A. A. Mitchell (Ed.), Advertising exposure, memory and choice. Hillsdale, NJ: Erlbaum. Moore, D. L., & Hutchinson, J. W. (1985). The influence of affective reactions to advertising: Direct and indirect mechanisms of attitude change. In L. F. Alwitt & A. A. Mitchell (Eds.), Psychological processes and advertising effects: Theory, research and application (pp. 65–87). Hillsdale, NJ: Erlbaum. Nairne, J. S., & Rescorla, R. A. (1981). Secondorder conditioning with diffuse auditory reinforcers in the pigeon. Learning and Motivation, 12, 65–91. Oxoby, R. J., & Finnigan, H. (2007). Developing heuristic-based quality judgments: Blocking in consumer choice. Psychology and Marketing, 24, 295–313. Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London, England: Oxford University Press. Pearce, J. M. (2008). Animal learning and cognition: An introduction (3rd ed.). New York, NY: Psychology Press.


505

Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not unconditioned stimuli. Psychological Review, 87, 532–552. Pechmann, C., & Stewart, D. W. (1988). Advertising repetition: A critical review of wearin and wearout. Current Issues and Research in Advertising, 11, 285–329. Petty, R. E., Cacioppo, J. T., & Schumann, D. (1983). Central and peripheral routes to advertising effectiveness: The moderating role of involvement. Journal of Consumer Research, 10, 135–146. Pitt, L. F., & Abratt, R. (1988). Music in advertisement for unmentional products–A classical conditioning experiment. International Journal of Advertising, 7, 130–137. Priluck, R. & Till, B. D. (2004). The role of contingency awareness, involvement, and need for cognition in attitude formation. Journal of the Academy of Marketing Science, 32, 329–344. Randich, A., & Ross, R. T. (1984). Mechanisms of blocking by context. Learning and Motivation, 15, 106–117. Rao, U. H., & Burnkrant, R. E. (1991). Effects of repeating varied ad executions on brand name memory. Journal of Marketing Research, 28, 406–426. Reed, P., McCarthy, J., Latif, N., & DeJongh, J. (2002). The role of stimuli in a virtual shopping environment: A test of predictions derived from conditioning models of marketing firms. Journal of Economic Psychology, 23, 449–467. Reilly, S., & Schachtman, T. R. (Eds.). (2009). Conditioned taste aversion: Neural and behavioral processes. New York, NY: Oxford University Press. Rescorla, R. A. (1969). Pavlovian conditioned inhibition. Psychological Bulletin, 72, 77–94. Rescorla, R. A., & Gillan, D. J. (1980). An analysis of the facilitative effect of similarity on second-order conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 6, 339–351. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: AppletonCentury-Crofts. Rothschild, M. L., & Hyun, Y. J. (1990). Predicting memory for components of TV commercials from EEG. Journal of Consumer Research, 16, 472–478.

2/16/2011 5:55:22 PM

506


Shimp, T. A. (1991). The role of subject awareness in classical conditioning: A cause of opposing ontologies and conflicting evidence. Advances in Consumer Research, 18, 158–163. Shimp, T. A., Hyatt, E. M., & Snyder, D. J. (1993). A critique of Darley and Lim’s “Alternative Perspectives.” Journal of Consumer Research, 20, 496–501. Shimp, T. A., Stuart, E. W., & Engle, R. W. (1991). A program of classical conditioning experiments testing variations in the conditioned stimulus and context. Journal of Consumer Research, 18, 1–12. Smith, R. E. (1993). Integrating information from advertising and trial: Processes and effects on consumer response to product information. Journal of Marketing Research, 30, 204–219. Sokolov, Y. N. (1963). Higher nervous functions: The orienting reflex. Annual Review of Physiology, 25, 545–580. Spear, N. E. (1978). The processing of memories: Forgetting and retention. Hillsdale, NJ: Erlbaum. Staats, A. W., & Staats, C. K. (1958). Attitudes established through classical conditioning. Journal of Abnormal and Social Psychology, 57, 37–40. Staats, C. K., & Staats, A. W. (1957). Meaing established through classical conditioning. Journal of Experimental Psychology, 54, 74–80. Stewart, D. W. (1989). Measures, methods, and models in advertising research. Journal of Advertising Research, 29, 54–60. Stammerjohan, C., Wood, C. M., Chang, Y., & Thorson, E. (2005). An empirical investigation of the interaction between publicity, advertising, and previous brand attitudes and knowledge. Journal of Advertising, 34, 55–67. Stout, P. A., & Rust, R. T. (1993). Emotional feelings and evaluative dimensions of advertising: Are they related? Journal of Advertising, 22, 61–71. Stuart, E. W., Shimp, T. A., & Randall, W. (1987). Classical conditioning of consumer attitudes: Four experiments in an advertising context. Journal of Consumer Research, 13, 334–339. Swan, J. A., & Pearce, J. M. (1988). The orienting response as an index of associability in rats. Journal of Experimental Psychology: Animal Behavior Processes, 14, 292–301. Till, B. D., & Priluck, R. L., (2000). Stimulus generalization in classical conditioning: An initial investigation and extension. Psychology and Marketing, 17, 55–72.


Tolman, E. C., & Honzik, C. H. (1930). Introduction and removal of reward, and maze performance in rats. University of California Publications in Psychology, 4, 257–275. Tomie, A., Murphy, A. L., Fath, S., & Jackson, R. L. (1980). Retardation of autoshaping following pretraining with unpredictable food: Effects of changing the context between pretraining and testing. Learning and Motivation, 11, 117–134. Tulving, E., & Thomson, D. M. (1971). Retrieval processes in recognition memory: Effects of associative context. Journal of Experimental Psychology, 87, 116–124. Vakratsas, D., & Ambler, T. (1999). How advertising works: What do we really know? Journal of Marketing, 63, 26–43. Van Hamme, L. J., & Wasserman, E. A. (1994). Cue competition in causality judgments: The role of nonpresentation of compound stimulus elements. Learning and Motivation, 25, 127–151. Van Osselaer, S. M. J. (2008). Associative learning and consumer decisions. In C. P. Haugtvedt, P. M. Herr, & K. R. Kardes (Eds.), Handbook of consumer psychology (pp. 699–729). New York, NY: Erlbaum. Van Osselaer, S. M. J., & Alba, J. W. (2000). Consumer learning and brand equity. Journal of Consumer Research, 27, 1–16. Van Osselaer, S. M. J., & Janiszewski, C. (2001). Two ways of learning brand associations. Journal of Consumer Research, 38, 202–223. Winters, L. C., & Wallace, W. H. (1970). On operant conditioning techniques. Journal of Advertising Research, 10, 39–45. Warrington, E. K., & Weizkrantz, L. (1968). A new method of testing long-term retention with special reference to amnesic patients. Neuropsychologia, 6, 283–291. Winters, L. C., & Wallace, W. H. (1970). On operant conditioning techniques. Journal of Advertising Research, 10, 39–45. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inference. American Psychologist, 35, 151–175. Zielske, H., & Henry, W. (1980). Remembering and forgetting television advertisements. Journal of Advertising Research, 20, 7–13. Zinkhan, G.M., & Martin, C.R., Jr. (1983). Message characteristics and audience characteristics: Predictors of advertising response. Advances in Consumer Research, 10, 27–31.

2/16/2011 5:55:22 PM

CHAPTER 22 Applications of Pavlovian Conditioning to Sexual Behavior and Reproduction Michael Domjan and Chana K. Akins

Pavlovian conditioning procedures can be readily adapted to the sexual behavior system. In reviewing studies of sexual conditioning, we focus on how conditioning modifies sexual behavior and how this contributes to reproductive success. We first consider how principles of learning that have been developed in conventional laboratory settings generalize to the sexual behavior system. We then discuss how studies of sexual conditioning have contributed to the understanding of sexual functioning. Finally, we examine how the results of sexual conditioning studies with laboratory animals (mostly domesticated quail and rats) might be used in the design and interpretation of human studies of sexual conditioning. Our review focuses on studies with male participants because much more research has been conducted with males than with females. However, we have included studies of female sexual conditioning in both human and non-human animals to the extent that they were available and relevant.

INTRODUCTION The basic Pavlovian or classical conditioning paradigm is highly familiar. It is commonly described as pairing an initially “neutral” or ineffective stimulus (the conditioned stimulus [CS]) with a biologically significant event (the unconditioned stimulus [US]). After a sufficient number of pairings, the CS is no longer behaviorally neutral and comes to elicit a conditioned response. Pavlov presented food as the unconditioned stimulus to the dogs in his experiments and measured salivation as the conditioned response. Since salivation was initially elicited as a reflex response to food, Pavlov’s procedure was characterized as the conditioning of a reflex. Food continues to be used as the unconditioned stimulus in many contemporary studies of Pavlovian conditioning but more often with rats and pigeons as experimental participants than dogs. Other common conditioning

preparations include fear conditioning, in which a brief shock to the feet of laboratory rats serves as the US, and eyeblink conditioning, in which irritation of the skin near the eye serves as the US. The objective of many of these experiments is to elucidate general mechanisms of learning rather than examine how Pavlovian conditioning modifies the operations of the feeding or defensive behavior systems or how Pavlovian conditioning contributes to the adaptive functioning of the organism. Our goal in this chapter is to review research on the applications of Pavlovian conditioning to sexual behavior. Our focus is not on the conditioning of a reflex. Rather, we are interested in how conditioning modifies the sexual behavior system and how this contributes to reproductive success. Like feeding, sexual behavior and reproduction are critical to the evolutionary survival and success of a species. In fact, one might argue that sexual behavior is so important that it cannot be left to the uncertainties of conditioning 507


2/16/2011 6:52:29 PM

508


and learning, which typically requires multiple training trials. Perhaps because of this perspective, most prior research on sexual behavior has focused on its neuroendocrine mechanisms and the emergence of those mechanisms in ontogenetic development (e.g., Ball & Balthazart, 2004; McCarthy & Bell, 2008). However, research in the last 25 years has shown that sexual behavior is no less susceptive to learning than feeding or defensive behavior. Furthermore, sexual conditioning can lead not only to anticipatory conditioned responses (analogous to anticipatory salivation) but also increases in the efficiency of the unconditioned response (copulation) and increases in the number of offspring that are produced by a copulation episode. We will concentrate on studies of sexual conditioning involving the Japanese or domesticated quail, Coturnix japonica, because this species has been studied most extensively in this type of research and because we are most familiar with sexual conditioning in this species. However, we will mention findings with other species, including Homo sapiens, when appropriate. Coturnix japonica was domesticated in the 12th century in Japan and is a common species in poultry science research. Domesticated quail have been popular in studies of sexual conditioning because of their small size and ready adaptation to laboratory housing. They are seasonal breeders in the wild, but the breeding season can be extended in the laboratory by maintaining the birds on a long photoperiod (16 hr light and 8 hr darkness daily). Under these conditions, females lay an egg nearly every day, and male-female pairs readily copulate when brought together. The male typically initiates copulation by grabbing the back of the female’s neck, mounting on top of the female with both feet, and then making a series of cloacal thrusts, juxtaposing its cloaca against that of the female for sperm transfer. Unlike unconditioned salivation to food or an eyeblink response to irritation of the eye, copulation between a male and a female is followed by a substantial refractory period. This limits the frequency with which copulation can be used as a US. In most of the research described in this chapter, conditioning trials were spaced at least 24 hours apart.


SEXUAL CONDITIONING AND THE GENERALITY OF THE PRINCIPLES OF LEARNING The first study of the conditioning of sexual behavior in domesticated quail was conducted by Howard Farris in 1964, as a Ph.D. dissertation project at Michigan State University, part of which was published (Farris, 1967). Conditioning trials consisted of presenting a soft buzzer as the CS for 10 seconds to male quail, followed by access to a female (the US). Three males received the CS paired with the US and two males received the CS and US unpaired. Conditioning was evident in components of male courtship behavior being elicited by the CS. These included increased body tonus, increased stiffening of the legs, toe walking, vocalization, and feather puffing. Farris’s original experiment generated considerable interest but no empirical replications for about 20 years. The next study of sexual conditioning in domesticated quail reported several experiments in which a light served as the CS preceding access to a female quail (Domjan, Lyons, North, & Bruell, 1986). In addition to recording the courtship and vocalization responses that were described by Farris, Domjan et al. measured approach to the CS, hoping to observe the development of sign tracking as the conditioned response (Hearst & Jenkins, 1974). Eight males received paired presentations of the CS and sexual reinforcement and eight males received an explicitly unpaired control procedure. The floor area of the experimental chamber measured 121 cm x 91 cm, with the CS positioned in the middle of one wall. The subjects were considered to have approached the CS if they were in a small area (40 cm x 30 cm) in front of the CS light. As predicted, sexual conditioning resulted in rapid acquisition of approach to the CS or sign tracking (see Fig. 22.1). By the sixth trial, acquisition was close to asymptote for the paired subjects, who spent significantly more time near the CS than the unpaired control group. Interestingly, in this experiment the action patterns that Farris identified as conditioned courtship responses did not occur in response to the CS or in anticipation of copulation. Rather, these responses

2/16/2011 6:52:29 PM

SEXUAL BEHAVIOR AND REPRODUCTION 10

Seconds Near CS Light

CS-US Paired 8 6 4 CS/US Unpaired 2 0 0

5

10

15

20

25

Trials

Figure 22.1 Acquisition of sexual conditioned approach behavior to the conditioned stimulus (CS) in male quail. The solid line represents data for subjects that received paired presentations of the CS with sexual reinforcement. The dotted line represents data for unpaired control subjects.

occurred predominantly after copulation with the female. Because approach to the CS or sign tracking was the predominant conditioned response in the study by Domjan et al. (1986), numerous subsequent experiments employed this response measure. However, the interpretation of those studies requires resolving two major questions. First, was the CS–approach response controlled by a Pavlovian CS-US relationship, or was it controlled by instrumental reinforcement? Second, were the subjects approaching the CS or approaching the door where the female was to be released? Since the light that served as the CS was close to the door from which the female was released, approaching the female door (goal tracking) would have been measured as CS approach. The omission control procedure was developed to determine whether responses that are acquired in a Pavlovian conditioning experiment reflect a Pavlovian CS-US relation or instrumental reinforcement of the conditioned response (Sheffield, 1965). In the omission control procedure, the CS is followed by the US if the subject does not make the CR. However, if the CR occurs on a particular trial, the US is omitted on that trial. Thus, the omission contingency insures


509

that the CR is not reinforced by presentation of the US. Crawford and Domjan (1993) showed that sexually conditioned CS approach behavior is acquired by male domesticated quail even if an omission control procedure is implemented, thus ruling out instrumental conditioning as the mechanism of the learning. Let us next turn to whether sexually conditioned quail are approaching the CS (sign tracking) or approaching the location where the female US is released (goal tracking). Goal tracking is a common measure of appetitive Pavlovian conditioning. Rats conditioned with food as the US, for example, will often nose the food cup during the CS in anticipation of the US (e.g., Boakes, 1977; Nelson, 2009). To help decide whether a conditioned approach response reflects sign tracking rather than goal tracking, the recommended strategy is to position the CS some distance from the goal object (Hearst & Jenkins, 1974). Figure 22.2 shows the floor plan of the apparatus in one such experiment (Burns & Domjan, 1996, Experiment 2). The CS was a small wood block lowered from the ceiling. For independent groups, the CS was presented in the front of the experimental chamber (where the female was released) or in the middle or the back (91 cm from the door to the female compartment). In addition, after 15 conditioning trials, each group was tested with the CS in each of the three locations. The subjects

+

122 cm

Figure 22.2 Floor plan for measuring sign track-

ing and goal tracking. The conditioned stimulus (CS), represented by star, was presented at different distances from the door that provided access to the female for independent groups of males.

2/16/2011 6:52:30 PM

510


approached the CS regardless of the group to which they were assigned. Furthermore, the quail continued to approach the CS even if it was moved from the training location to one of the other locations during test trials. Experiments in which the CS is located some distance from the location of the US have been called “long-box” experiments, because the original study of this phenomenon employed a Skinner box for pigeons in which the food cup was located about 90 cm from a key light that served as the CS. To test the limits of sexually conditioned CS approach behavior, Burns and Domjan (2000) substantially increased the length of the experimental chamber they previously used, with the CS now located 233 cm from the location of the female. Such a spatial separation between CS and US is unprecedented in the conditioning literature. However, it did not prevent sexual approach conditioning. Even with such a long distance between the CS and the US, the conditioned behavior that developed was approach to the CS rather than approach to the goal location. Using CS approach as a measure of sexual conditioning, studies have shown that virtually any phenomenon that has been found in more familiar appetitive and fear-conditioning situations can be replicated in sexual conditioning. So far we have described acquisition, omission control, and “long-box” sign tracking. Other effects that have been documented include retention of conditioned behavior (Domjan et al., 1986), extinction (Domjan et al., 1986; Krause, Cusato, & Domjan, 2003), CS-US interval effects (Akins, Domjan, & Gutierrez, 1994), trace conditioning (Akins & Domjan, 1996), simple and conditional discrimination learning (e.g., Domjan, Akins, & Vandergriff, 1992), conditioned inhibition (Crawford & Domjan, 1996), context conditioning (Domjan, Greene, & North, 1989; Hilliard, Nguyen, & Domjan, 1997), US devaluation effects (Holloway & Domjan, 1993a), blocking (Köksal, Domjan, & Weisman, 1994), second-order conditioning (Crawford & Domjan, 1995), and observational conditioning (Köksal & Domjan, 1998). This impressive range of learning effects clearly proves that Pavlovian conditioning can be


extended to sexual behavior. If the goal of the application is simply to demonstrate the generality of learning, one could declare victory and consider the job finished. However, the extension of learning to a new situation should also provide novel insights into the nature of the target system. We turn to that issue next.

WHAT SEXUAL CONDITIONING TELLS US ABOUT SEXUAL BEHAVIOR Unconditioned Stimulus Factors in Sexual Conditioning

So far we have described sexual conditioning as involving the presentation of a CS paired with a US that consists of the opportunity for a male to copulate with a female. Copulation is a highly effective reinforcer for sexual learning. However, if sexual conditioning only occurred in situations in which copulation took place, it would be of limited applicability because copulation is a relatively rare event in the lives of many animals. In an early experiment involving a straight alley runway, Sheffield, Wulff, and Backer (1951) found that male rats did not have to copulate to ejaculation to learn to run down the alley for access to a female. Mounting and intromission were sufficient to reinforce the instrumental running response. In a subsequent experiment, Zamble, Hadad, Mitchell, and Cutmore (1985) provided evidence of sexual Pavlovian conditioning in male rats using a procedure in which exposure to a female behind a barrier was the US. Noncopulatory exposure to a female also served as the US in studies of sexual conditioning with blue gourami fish (Hollis, Cadieux, & Colbert, 1989; Hollis, Pharr, Dumas, Britton, & Field, 1997). Visual exposure to sexual scenes often serves as the US in human studies of sexual conditioning (e.g., Hoffmann, Jannsen, & Turner, 2004; Lalumière & Quinsey, 1998; Langevin & Martin, 1975). Although the preceding evidence clearly indicates that copulation is not necessary for sexual conditioning, copulation may be more effective in producing learning than exposure to a potential sexual partner on the other side of a barrier.

2/16/2011 6:52:30 PM

SEXUAL BEHAVIOR AND REPRODUCTION

This question was addressed by Holloway and Domjan (1993b), who compared sexual approach conditioning in two groups of male quail. For one group the 30-sec CS was followed by the release of a female with whom the male could copulate. For a second group, the female was presented for an equal period of time, but behind a wire-mesh screen that allowed the transmission of female visual, olfactory, and auditory cues but limited physical contact. A third group received the CS followed by no US. Acquisition of CS approach behavior is presented in Figure 22.3 for each group of subjects. Clearly the strongest level of responding occurred when the subjects were permitted to copulate with the female. However, significant CS approach also developed with exposure to female cues in the absence of copulation. Furthermore, a subsequent experiment confirmed that this CS approach was an associative effect, because it did not develop if the CS was presented unpaired with exposure to a female behind the wire screen. The fact that the visual and other features of a female quail can be reinforcing for males is also evident if the male is permitted to watch a female through a small window, as illustrated in Figure 22.4. Male quail spend 75%–80% of their daylight hours near a window through which they can observe a female, provided that they

previously received opportunities to copulate with a female (Domjan & Hall, 1986). This social proximity behavior persists without decline for at least 2 weeks with continual window exposure in the absence of physical access to a female. It appears that copulation with a female provides sexual reinforcement, which then increases the reinforcing value of visual and other female cues that are encountered when a male sees a female through a window. If those female features are not paired with copulation at least once, the female cues are much less reinforcing and social proximity and looking behavior are not maintained (Domjan, 1998).

Exposure Only CS Only

20 Seconds

Figure 22.4 A male quail looking through a narrow vertical window at a female quail, after visual exposure to the female was paired with copulation.

Conditioned Stimulus Factors in Sexual Conditioning

Copulation 30

511

10

0 0

2

4

6

8

10

Trials

Figure 22.3 Acquisition of sexual conditioned approach behavior to a conditioned stimulus (CS) that was paired with copulation, exposure to a female behind a wire screen, or presented alone.


Early studies of Pavlovian sexual conditioning followed the Pavlovian tradition of using a CS that is convenient and initially “neutral” or unrelated to the US. Domjan et al. (1986) used a light as the CS in their first studies with domesticated quail. Hollis et al. (1989) also used a light CS in their study of sexual conditioning in the blue gourami. Arbitrary olfactory cues were used in studies of sexual conditioning with rodents (Graham & Desjardins, 1980; Kippin & Pfaus, 2001; Zamble et al., 1985). These experiments were successful in demonstrating the applicability of Pavlovian procedures to sexual behavior. However, given the arbitrary nature of the conditioned stimuli employed, it is not entirely clear

2/16/2011 6:52:30 PM


Conditioning of Contextual Cues with Sexual Reinforcement

Sexual behavior in the wild does not occur in unpredictable and unfamiliar places. Rather, sexual behavior most likely occurs in specific locations or territories that are defended by the male or occupied by the female. Contextual cues present during a sexual encounter may be additionally limited by the time of day, time of the month, or the time of year. For example, sexual behavior may occur only in areas with particular plants and food sources that are characteristic of the breeding season. Thus, contextual cues are one set of stimuli that may serve as CSs in sexual conditioning outside the laboratory. Research has shown that contextual cues are readily associated with sexual reinforcement. In addition, contextual sexual conditioning has two important behavioral consequences: spatial preference and potentiation of responding to female cues. The spatial preference outcome of sexual context conditioning was demonstrated in an experiment by Akins (1998). In this experiment, male quail showed an initial preference for a distinctive context where they had been housed for 2 weeks. They then received brief exposure (5 min) to an alternate nonpreferred distinctive context (CS) followed by copulation with a receptive female quail (US) in that context. One such context conditioning trial was conducted each day. An unpaired group of males received similar stimulus exposures except that they received the US in their colony cage 2 hr before placement into the CS context. Preference tests,


which involved giving free access to the initially preferred context and the CS context, were given after the first five conditioning trials and again after five more conditioning trials. Figure 22.5 (top panel) shows that male quail shifted their preference to the context that had been paired with the sexual US after five CS-US pairings and this shift was maintained after five additional pairings. Quail in the paired group also demonstrated increased locomotor activity during the 5 min prior to the introduction of the female quail, thus demonstrating anticipatory responding to the introduction of the female (Fig. 22.5, bottom panel). Experiments conducted with rodents have demonstrated similar effects of contextual cues in sexual conditioning.

120 % Time in Trained Context

how the sexual learning that was observed in these laboratory experiments would occur in the natural environment of the animals. One would have to argue that some arbitrary stimulus that an animal happened to experience before a sexual encounter would become sexually conditioned to elicit the conditioned response. But, for that scenario to be viable, the arbitrary CS would have to appear with several sexual encounters and not without them. That is, there would have to be a naturally occurring positive contingency between the CS and the US. What cues could have that kind of relationship outside the laboratory?

Group-Paired Group-Unpaired

100 80 60 40 20 0 1

2 Preference Tests

3

200 Line Cross Frequency

512

150

100

50

0 0

1

2

3

4

5

6

7

8

9 10

Conditioning Trials

Figure 22.5 Mean (± SE) percentage of time that

Group Paired and Group Unpaired spent in the trained context across preference tests (top) and their line cross-frequency as a function of conditioning trials (bottom).

2/16/2011 6:52:31 PM


For example, male rats increase their movements from one level to another in a bilevel chamber in anticipation of the introduction of a sexually receptive female rat (Mendelson & Pfaus, 1989; Pfaus, Mendelson, & Phillips, 1990). Male rats also develop a conditioned place preference for a context that has become associated with copulation with a receptive female (Agmo & Berenfeld, 1990; Hughes, Everitt, & Herbert, 1990; Mehrara & Baum, 1990; Paredes & Alonso, 1997). The importance of contextual cues during a sexual encounter is also evident in studies with females. For example, female rodents show a place preference for a distinct context in which copulation with a male took place (Oldenburger, Everitt, & de Jonge, 1992). Furthermore, female rodents demonstrate a more robust place preference for a context in which they are allowed to pace the rate of copulation by controlling the speed and frequency of male copulatory responses (Jenkins & Becker, 2003; Paredes & Alonzo, 1997; Paredes & Vazquez, 1999). The spatial preference that results from context conditioning should increase preference for these areas during the breeding season. Thus, contextual conditioning can determine the location where sexual activity is likely to take place. Further facilitating that process is the fact that sexually conditioned contextual cues enhance the effectiveness of female cues in generating male approach and copulatory behavior. The first evidence of this effect was obtained by Domjan et al. (1989), who tested male quail with

Frequency

Grab

513

a model that had some features of a female quail but not enough to elicit much behavior without learning. The model consisted of a taxidermically prepared head and about 5 cm of neck feathers of a female quail mounted on a vertical dowel in front of a gray foam block that the males could use to make mount and cloacal thrusting responses. The model had so few of the cues of a live female that it did not stimulate copulatory behavior as an unconditioned response. To determine whether context conditioning would enhance the effectiveness of limited female cues, Domjan et al. (1989) permitted one group of male quail to copulate with a female on 15 trials in a distinctive experimental chamber. A second group received an equal number of copulation trials and was equally familiar with the experimental chambers but for them the copulatory experiences occurred in their home cages. All of the subjects were then tested with the female head+neck model in the experimental chamber, and the frequency of copulatory responses directed toward the model was recorded. Note that this was the first time any of the subjects encountered the taxidermic head+neck. The results are presented in Figure 22.6. Subjects for whom the experimental chamber was paired with sexual reinforcement made significantly more grab, mount, and cloacal contact responses directed at the head+neck model than subjects that received copulatory

Mount

10

10

8

8

6

6

4

4

2

2

0

0 Paired

Cloacal Contact 10 4 2 0

Unpaired

Figure 22.6 Mean (± SE) frequency of grabs, mounts, and cloacal contact responses directed toward a model with female head and neck features, for subjects who had the test context paired or unpaired with sexual reinforcement.


2/16/2011 6:52:32 PM

514


experiences in the home cage. These findings show that context conditioning increased the effectiveness of the limited female cues in eliciting copulatory responses on the part of the male subjects. Similar evidence for the enhancement of copulatory responses by contextual cues has been found in rodents. Sachs and Garinello (1978), for example, found that the penile reflex of male rats was enhanced when males were placed in a test cage where they had previously copulated. Domjan et al. (1989) conducted 15 conditioning trials. However, subsequent research has shown that a single context conditioning trial is sufficient to produce enhanced reactivity to female cues in male subjects (Hilliard et al., 1997). This makes context conditioning a powerful mediator of sexual behavior. Males prefer areas where they previously encountered females and are more likely to respond to limited female stimuli in a sexually conditioned context. Outside the laboratory, some of the contextual cues experienced during copulation, such as the male’s territory or seasonal cues, may be familiar to the copulation partners. The extent to which such familiarity produces a latent inhibition effect and attenuates learning by one or both participants has not been examined so far. However, the fact that sexual conditioning of contextual cues is robust enough to occur in one trial suggests that latent inhibition may not have a major impact on this type of learning. Conditioning of Body Adornments

Contextual cues provide one way in which Pavlovian mechanisms could modify sexual behavior in the wild. Another category of stimuli that could be conditioned under natural mating circumstances are unique features of a sexual partner. All copulatory encounters are preceded by telereceptive cues of the sexual partner that are initially encountered at a distance. As the sexual partners come closer together, these telereceptive cues are followed by more proximal visual, olfactory, tactile, and other cues. If a potential sexual partner is identified by a unique body feature (height, pattern of coloration, or a distinctive posture or call), that feature may become associated with sexual


reinforcement and thereby elicit future sexual responses. The possibility that a unique body feature may become sexually conditioned was first examined in male domesticated quail by adding artificial orange feathers to a female quail (Domjan, O’Vary, & Greene, 1988). On conditioning trials, the adorned female was presented for 30 seconds behind a wire screen as the CS. A normal unadorned female was then introduced to provide the US (copulatory reinforcement). A control group received the CS and US females in an unpaired fashion. Initially, the males did not approach the adorned female, but after four to five pairings with copulation they came to spend nearly all of the 30-second CS period near her. After the conditioning phase, the wire screen restraining the adorned female was removed to see whether the males would copulate with her. Significantly more grab, mount, and cloacal contact responses occurred with the adorned female in the conditioned group than in the unpaired control group. Thus, the CS-US pairings produced not only CS approach responses but also conditioned copulatory responses. In research with laboratory rats, adornment of female rats is typically accomplished by adding a CS odor to the female. In one study, for example, Kippin and colleagues (Kippin, Talianakis, Schattmann, Bartholomew, & Pfaus, 1998) gave male rats access to female rats that were either scented with an almond odor (CS) or not scented. Copulatory preferences were tested subsequently by allowing males to copulate with either a scented or an unscented female. Males that were previously conditioned with scented females displayed a copulatory preference for the scented female, indicating that the unique features of the sexual partner facilitated copulatory behavior through Pavlovian conditioning. Conditioning of Natural Female Features

Experimental studies with body adornments demonstrate that unique features of a sexual partner can become conditioned, but the manipulations in the aforementioned experiments were admittedly exaggerations of the type of body features that might occur in nature. Females quail

2/16/2011 6:52:32 PM



Head CS

No Head CS

Figure 22.7 Terrycloth objects used as condi-

tioned stimuli in studies of sexual conditioning. The object on the left includes taxidermically prepared features of a female quail, making that a more ecologically relevant conditioned stimulus (CS) object.

a signal for the opportunity to copulate with a female quail. In one experiment, Köksal et al. (1994) tested two groups of male quail. For one group, the female head model was paired with the opportunity to copulate with a live female. For a control group, exposure to the CS and copulation occurred in an unpaired fashion. Every third trial was a nonreinforced test trial with just the CS. Males that received the paired CS-US procedure quickly came to approach and remain near the CS object. Learning was evident after the first two conditioning trials and reached asymptote by the fourth trial (see Fig. 22.8).

30

Seconds Near CS

do not sport bright orange feathers in the wild nor do female rodents typically smell of almonds. This raises the question of whether body features that are within the range of normal variation for a species can serve as a basis for Pavlovian sexual conditioning. As we noted earlier, all sexual encounters begin with exposure to cues from the male and female that are encountered at a distance. These cues serve to identify a potential sexual partner. When the male and female are far apart, these cues are likely to be partially occluded by other objects and will be smaller, of lower intensity, and less distinctive than they are when the male and female are close to each other. Perhaps one of the things that brings sexual partners together is having learned to respond to incomplete and low-intensity distal cues of a sexual partner as signals for a more intimate social interaction. The aforementioned scenario is highly likely for Japanese quail. Japanese quail are grounddwelling birds that live in grassy areas in the wild (Schwartz & Schwartz, 1949). When a male first encounters a female, she is likely to be mostly hidden by the grass. Only as the two birds come together will they fully see each other. Thus, in nature, intimate social interactions are likely to be preceded by partial cues of the potential sexual partner. These partial cues might come to serve as a signal or CS for the ensuing intimate social interaction. This sequence of events may be modeled in the laboratory by presenting partial female cues as a CS preceding unhindered access to a female quail, which would serve as the US. The first studies of sexual conditioning with partial female cues as the CS were performed by Köksal et al. (1994). The left panel of Figure 22.7 shows a drawing of a CS object similar to that used by Köksal et al. (1994). The object was made of terrycloth stuffed with soft polyester fiber and had a vertical and a horizontal section to permit the male to grab and mount the object. The vertical section of the CS object included a taxidermically prepared head of a female quail and limited neck feathers. As we noted earlier, such an object lacks sufficient female cues to elicit sexual behavior unconditionally. However, it can rapidly become conditioned if it is used as

515

Paired Unpaired

20

10

0 1

2

T1

3

4

T2

5

6

T3

Figure 22.8 Acquisition of conditioned approach

behavior to a conditioned stimulus (CS) object that includes cues of a female’s head in subjects that had this CS paired or unpaired with sexual reinforcement.

2/16/2011 6:52:33 PM


Significantly less approach behavior was evident in the unpaired control group. The acquisition of the conditioned approach response occurred much more quickly in the experiment by Köksal et al. (1994) than in the earlier study in which a light served as a CS (see Fig. 22.1). Thus, including partial female features in a CS object facilitated the acquisition of sexual approach responding. Other experiments have shown that sexual learning is much more robust in many ways if the CS object includes partial cues of a female quail (see Domjan, Cusato, & Krause, 2004, for a review). In these investigations, the CS object was either the terrycloth model that included a taxidermic female head or a terrycloth model that was the same size and shape that did not include the partial female cues (see Fig. 22.7). Akins (2000) examined the effects of the CS-US interval using independent groups of male quail that received either the head CS object or the no-head CS object paired or unpaired with copulatory opportunity. For one set of groups, the CS-US interval was 1 minute. For another set of groups, the CS-US interval was 20 minutes. (With both intervals, the CS remained visible until the presentation of the US.) The results of the experiment are summarized in Figure 22.9. When the CS object did not include female features, only subjects conditioned with the 1-minute CS-US interval showed acquisition of approach behavior. In contrast, when the CS object included a taxidemic female head, substantial conditioned approach behavior developed with both the 1-minute and the 20-minute CS-US interval, although responding developed more slowly with the 20-minute interval. These results show that including female features in a CS object makes the ensuing learning of the approach response more resistant to increases in the CS-US interval. 1 Other experiments have shown that sexual conditioning with a CS object that includes partial female cues is resistant to the blocking effect (Köksal et al., 1994). That is, if such a CS object is presented at the same time as a second CS that was previously well conditioned, the presence of the previously conditioned cue does not disrupt the conditioning of the head CS. A CS object


100

75 % Time in Zone 0

516

50

25

0 0

5

10

15

Trails

HN-Short Unpaired HN-Long Paired

T-Short Paired T-Short Unpaired T-Long Paired

HN-Long Unpaired

T-Long Unpaired

HN-Short Paired

Figure 22.9 Conditioned approach behavior to a

conditioned stimulus (CS) object that includes cues of a female head and neck (HN) or is made entirely of terrycloth (T), when this CS is paired or unpaired with sexual reinforcement. The CS was presented for either 1 min (Short) or 20 min (Long) during the conditioning trials. The data were obtained during the first minute of each trial.

that includes partial female cues is also less likely to undergo extinction when the CS is repeatedly presented without the US (Krause et al., 2003), and such a CS object also supports stronger second-order conditioning (Domjan et al., 2004). Range of Sexually Conditioned Responses

So far we have emphasized approaching the CS as the primary behavioral manifestation of sexual conditioning. Although this is a commonly observed conditioned response, it is not the only type of behavior that develops with sexual conditioning. The nature of the conditioned response depends on the nature of the CS as well as the CS-US interval. CS approach behavior is most likely to develop if the CS is not diffuse so that the subject has a clear location to approach and if the CS-US interval is relatively short (1 minute or less). If a long CS-US interval is used (e.g., 20 minutes), the predominant conditioned response that emerges is an increase in

2/16/2011 6:52:33 PM


nondirected locomotor behavior (Akins, 2000; Akins et al., 1994). Conditioned approach to a CS with a 20-minute CS-US interval develops as a conditioned response only if the CS includes partial female cues (Akins, 2000). Investigators have also been interested in conditioning components of copulatory behavior. The cloacal gland of adult male quail contains a foamy substance that facilitates sperm transport and fertilization of eggs in the female’s oviduct. Males engage in rhythmic contractions of the cloacal sphincter muscle when they are in the presence of a female. These cloacal sphincter contractions facilitate production of cloacal foam in the visual presence of a female. Holloway, Balthazart, and Cornil (2005) demonstrated that cloacal sphincter contractions can also occur as a Pavlovian conditioned response to a CS that has been paired with visual access to a female. Thus, sexual conditioning serves to elicit cloacal contractions in anticipation of copulation. In a related study, Domjan, Blesbois, and Williams (1998) demonstrated that sexual conditioning also increases the quantity of sperm that is released into the cloaca in anticipation of copulation. As we described earlier in the section on conditioning of body adornments, a sexually conditioned stimulus can also come to elicit components of copulatory behavior (grab, mount, and cloacal contact responses). In those experiments CS body adornments were attached to a female quail. Therefore, the copulation that occurred was with a live female that had been altered by the adornments. The power of Pavlovian conditioning in modifying sexual behavior is more dramatically illustrated by cases in which conditioned copulatory behavior is directed toward an artificial inanimate object. Conditioned copulation with an artificial object is of interest because such behavior may provide an animal model of sexual fetishism. For sexual conditioning to generate copulatory responses directed toward the CS, the CS has to have a shape and texture that support grabs, mount, and cloacal thrusts. In the study of body adornments by Domjan et al. (1988), one group of subjects was conditioned with a CS that was a small stuffed toy in the shape of a dog.


517

Conditioning resulted in approach behavior, but this object evidently did not have the required configuration to elicit copulatory responses. Köksal and his associates (Köksal et al., 2004) were more successful in conditioning copulatory responses to a CS using a terrycloth object similar to that shown in the right panel of Figure 22.7. However, even after 30 conditioning trials, the CS elicited grab, mount, and cloacal thrust responses in only about 50% of the subjects (see also Çetinkaya & Domjan, 2006). Interestingly, these subjects showed much more resistance to extinction when the CS was subsequently presented without sexual reinforcement. More consistent conditioned copulatory responses occur if the CS object has partial cues of a female quail, such as a taxidermically prepared head and a bit of neck feathers (see Fig. 22.7, left panel). As we noted earlier, such a CS object does not elicit male sexual behavior unconditionally. However, if the CS is paired with sexual reinforcement, it quickly comes to elicit conditioned approach responses and also elicits grabs, mount, and cloacal contact responses (Cusato & Domjan, 1998). In general, the more of a female’s features that are included in a CS object, the more quickly copulatory responses come to be directed toward the CS. In addition, once the copulatory responses develop, they persist, even if the female features are subsequently gradually covered with terrycloth (Domjan, Huber-McDonald, & Holloway, 1992). These experiments leave no doubt that sexual conditioning can result in copulatory responses being elicited by an inanimate CS object. However, such conditioned behavior critically depends on the specific configuration of the CS object. Effects of Psychostimulants on Conditioned Sexual Behavior

A fairly recent approach to the study of sexual conditioning has been to investigate the effects of psychostimulants. This approach is of great interest because psychostimulant use has been linked to increased sexual desire (Volkaw et al., 2007) and high-risk sexual behaviors (e.g., Zule, Costenbader, Meyer, & Wechsberg, 2007)

2/16/2011 6:52:33 PM

518


in humans. Therefore, studies of the effects of psychostimulants on sexual conditioning may have special clinical relevance. Research in both quail and rodents provides evidence for the modification of conditioned sexual behavior by prior psychostimulant exposure. In one experiment, Levens and Akins (2004) utilized the sexual conditioning paradigm to determine the effects of a history of cocaine treatment on subsequent conditioned sexual approach and copulatory responses. Male quail received administration of cocaine (10 mg/kg ip) or saline once a day for 6 days. After a 10-day withdrawal period, conditioning trials were given that consisted of presentation of a CS followed by copulation with a female quail (US). Unpaired control groups (preexposed to either chronic cocaine or saline) received the same treatment except that the US was given 3 hr prior to the CS. Figure 22.10 shows that male quail that received paired CS-US trials showed greater approach to the CS across trials compared to unpaired groups. Of most importance, subjects with a history of chronic cocaine administration showed considerably more CS approach than any of the other groups.

The cocaine paired group also had shorter latencies to copulate with the female partner (see Fig. 22.11), made more cloacal contact responses, and was more efficient at copulating than any of the other groups (see Fig. 22.12). In a related study, Fiorino and Phillips (1999) administered chronic preexposure of amphetamine 3 weeks before sexual conditioning in male rats. During sexual conditioning, the male rats showed increased frequency of level changing behavior in bilevel chambers in anticipation of female presentation and also showed shorter latencies to mount and intromit. These studies demonstrate that a history of psychostimulant administration may enhance sexual conditioning, even after a period of withdrawal. It should be noted that in the study by Levens and Akins, withdrawal was probably complete within an hour after each cocaine administration (though no metabolic data are available in quail). Conducting the sexual conditioning trials after 10 days of withdrawal insured that motor and other physiological effects resulting from acute drug administration would not influence the results. The 10-day withdrawal period was also employed because the purpose of the study

350

25 20 15 10

250 200 150 100 50

5

0

0 2

3

4

5

6

7

8

9

10

Trial

Figure 22.10 Mean time (s) spent in the condi-

tioned stimulus (CS) zone (±SE) during the 30 s presentation of the CS across sexual conditioning trials.


Paired Cocaine Paired Saline

300

Paired Cocaine Paired Saline Unpaired Cocaine Unpaired Saline

Latency to Copulate (S)

Time in Goal Zone (s)

30

1

2

3

4

5 6 Trial

7

8

9

10

Figure 22.11 Mean latency (s) to copulate (±SE)

during a 5-min test for paired subjects that received a conditioned stimulus (CS) followed by copulatory opportunity across sexual conditioning trials.

2/16/2011 6:52:33 PM


Copulation Efficiency Ratio

25

20

that predicted copulation with a female. Another group of males were sexually deprived prior to testing. Male quail displayed significantly less approach to the CS when sexually satiated than when sexually deprived.

Paired Cocaine Paired Saline Unpaired Cocaine Unpaired Saline

15

10

Conditioned Modifications of Interactions with a Sexual Partner

5

0 1

2

3 4 Female Trials

5

Figure 22.12 Mean copulatory efficiency ratio (±SE) calculated as number of cloacal contact movements divided by the number of grabs plus mounts plus cloacal contact movements. This number is multiplied by 100. Perfect efficiency is 33.

was to investigate relatively long-term changes in the brain due to prior cocaine administration rather than acute drug effects on sexual conditioning. In contrast to the literature that demonstrates how psychostimulants enhance sexual motivation and consummation, a recent experiment conducted with Japanese quail indicated that chronic preexposure to methamphetamine impaired sexual motivation as indicated by a slower running time down a runway toward a female quail compared with saline controls (Bolin & Akins, 2009). Similarly, some nondrug manipulations have been shown to reduce sexual motivation. Sexual motivation of male quail conditioned to approach an arbitrary stimulus in a Pavlovian sexual conditioning paradigm was reduced by exposing them to a short photoperiod (Holloway & Domjan, 1993a). Responding was restored when males were returned to a long photoperiod and when exogenous testosterone was administered. In another experiment, Hilliard and Domjan (1995) sexually satiated male quail by allowing them to copulate repeatedly with receptive female quail prior to testing for Pavlovian conditioned approach to a visual CS


519

The behavioral consequence of Pavlovian conditioning is commonly described as the development of a new response to the initially ineffective CS. In fact, the conditioned response is typically defined as a learned response to the CS. In contrast to this view, Domjan (2005) has argued that from a functional perspective the most important thing that is learned in Pavlovian conditioning is not a new response to the CS but a more effective manner of responding to the US. After all, the US is the biologically significant event that is a challenge for the organism. How the organism deals with the US is critical to its survival and biological success. Conditioned responses made in the absence of the US are merely false starts that have no functional significance. In the sexual behavior system, the US is a sexual partner. If sexual conditioning is of functional significance, it should improve how the organism interacts with its sexual partner. Numerous studies have shown that this is indeed the case. In an early study of sexual conditioning (Graham & Desjardins, 1980), male rats presented with a CS that reliably predicted access to a sexually receptive female rat showed increases in serum levels of testosterone and luteinizing hormone after CS presentation. In another experiment, Zamble and colleagues (Zamble et al., 1985) found a decrease in latency of conditioned rats to ejaculate when permitted to copulate with a female rat after exposure to the CS. Subsequent more detailed studies with rodents have provided more evidence for the functional significance of sexual conditioning by demonstrating enhanced conditioned partner preferences (e.g., Coria-Avila, Ouimet, Pacheco, Manzo, & Pfaus, 2005; Coria-Avila et al., 2006).

2/16/2011 6:52:34 PM

520


An especially dramatic example of conditioned modifications in how a male interacts with a female was provided by a study with blue gourami fish (Hollis et al., 1997). After 18 conditioning trials in which the presentation of a light for 10 seconds was followed by visual access to a female, the CS was presented for 10 seconds and the barrier separating the male from the female was removed, allowing the two fish to interact for the next several days. Presentation of the CS for 10 seconds prior to this extended social interaction had profound effects. In comparison to an unpaired control group, conditioned males showed less aggression toward the female, more nest-building behavior, more clasping behavior, and shorter latencies to spawn. As a result of these enhanced sexual responses, conditioned males also ended up with about 40 times more offspring than males in the control group. It is remarkable that all of these behavioral and physiological effects occurred because of a 10-second sexually conditioned light presented before the extended social interaction. A sexually conditioned stimulus also improves the ways in which male quail interact with a sexually receptive female. As is the case with rats and gouramis, a sexually conditioned stimulus decreases the latency of male quail to initiate copulation with a female (Domjan et al., 1986). Furthermore, this decrease in copulatory latency provides conditioned males with an advantage when two males compete for access to the same female (Gutiérrez & Domjan, 1996). In such sexual competition, the male who receives a Pavlovian CS before the mating opportunity is the one who copulates with the female first. Sexual conditioning also increases the efficiency of copulatory behavior. In Japanese quail, the copulatory response sequence begins with the male grabbing the back of the female’s neck. The female then squats, allowing the male to mount and make cloacal contact responses. If the female is not sexually receptive, she will attempt to run away or throw the male off when he tries to mount. A persistent male will then reinitiate the grab and mount responses. The efficiency of the sexual encounter can be quantified by assessing how many grab and


mount responses a male bird has to make before he can achieve cloacal contact with the female. Copulatory efficiency is related to female genetic characteristics and is positively correlated with female squatting behavior (Domjan, Mahometa, & Mills, 2003). Interestingly, the efficiency of copulatory interactions increases as a function of Pavlovian conditioning trials (Mahometa & Domjan, 2005). However, this effect requires that both the male and the female receive the Pavlovian CS that signals the forthcoming mating opportunity. If only the male or only the female receives the signal, copulatory behavior is no more efficient than if the CS is omitted for both subjects (Mahometa & Domjan, 2005). This last finding is a bit puzzling because Gutiérrez and Domjan (1997) found that sexually conditioned female quail are more apt to squat in the presence of a male than females in an unpaired control group. Evidently, if a male is not also conditioned, he is unable to take full advantage of the increased receptivity of a sexually conditioned female. Sexual Conditioning and Fertilization Success

As described in the preceding section, exposure to a sexually conditioned stimulus changes how a male and female interact. In quail, the latency to copulate decreases and the efficiency of copulation increases. In the blue gourami, presentation of a sexually conditioned CS prior to the sexual interaction decreases aggression and the latency to spawn and increases nest-building and clasping behavior. These behavioral changes appear to be improvements in the quality of the sexual interaction. Biologically, the ultimate measure of the quality of a sexual interaction is fertilization success or the number of offspring that are produced. Therefore, one might predict that presentation of a sexually conditioned stimulus will increase the number of offspring that result from the sexual interaction. Hollis et al. (1997) were the first to measure the reproductive consequences of sexual Pavlovian conditioning in their study of the blue gourami. They found that the sexual interactions

2/16/2011 6:52:34 PM


of gourami that received paired presentations of a light and a potential sexual partner resulted in more than 1,000 offspring, whereas the number of offspring produced by the unpaired control group was about 25. Thus, the shorter latencies to spawn, the decreased aggression, and the increased nest-building and clasping behavior that were evident in the Pavlovian conditioned group served to increase the effectiveness of their sexual interactions by the measure that is ultimately the critical outcome sexual activity— namely, reproductive success. In Japanese quail, Domjan et al. (2003) found that shorter latencies to copulate, increased durations of female squatting, and increased copulatory efficiency were all positively correlated with the proportion of fertilized eggs the female laid in the next 10 days. Since these behavioral changes can be elicited by a sexually conditioned stimulus, rates of fertilization also should be increased by a sexual CS. That direct link between Pavlovian conditioning and fertilization success was demonstrated by Adkins-Regan and MacKillop (2003), who first conditioned male and female subjects by pairing exposure to distinctive contextual cues with a potential sexual partner. The quail were then permitted to copulate with a novel (and experimentally naïve) sexual partner either in the sexually conditioned context or an equally familiar control context. The eggs subsequently laid by the females were assessed for successful fertilization. Exposure to a sexually conditioned context increased the rate of fertilization whether the contextual cues had been conditioned for the male or the female subjects. Increased fertilization rates were also found by Mahometa and Domjan (2005) in a study that employed a light rather than contextual cues as the sexually conditioned CS. However, Mahometa and Domjan (2005) found increased fertilization rates only if both the male and the female received the Pavlovian CS prior to the test copulation. Presenting the conditioned light CS to just the male or just the female did not have a significant effect. The requirement of signaling both the male and the female in the study by Mahometa and Domjan (2005) may have been related to the fact that increased copulatory efficiency occurred


521

only when the Pavlovian signal was presented to both of the participants. Pavlovian conditioning can also increase fertilization success in sperm competition. Sperm competition occurs when two males inseminate the same female. Various mechanisms have evolved (e.g., dislodging a sperm plug) that give one of the males the upper hand in fathering the offspring under these conditions. Pavlovian conditioning is one of those mechanisms. This was demonstrated by Matthews, Domjan, Ramsey, and Crews (2007), who used genetic fingerprinting to determine paternity in Japanese quail after females copulated with two males in succession. During the initial conditioning phase of the study by Matthews et al. (2007), both male and female quail received exposures to two distinctively different experimental chambers. For female subjects, exposure to both contexts was paired with copulation. In contrast, for the males one context was paired with sexual reinforcement and the other was unpaired or not reinforced. On the critical sperm competition day, each female copulated with two males in succession, one in each of the experimental contexts. One of the males was in its paired context and the other male was in its unpaired context. Thus, the test copulation was signaled for one of the males but not the other. Order of exposure to the signaled and unsignaled male was counterbalanced across females. Eggs were collected from each female for 10 days starting 2 days after the sperm competition copulations. The eggs were incubated for 5 days and subjected to genetic analysis to determine paternity. Of the 78 eggs laid by the females, 39 were fertilized. Genetic analysis indicated that among the eggs that were fertilized, 28 (72%) had been fertilized by the signaled male and 11 (28%) were fertilized by the unsignaled male. Whether the female copulated first with the signaled male or the unsignaled male did not matter. These results show that sexual conditioning provides a significant paternity advantage in a situation where a female copulates with more than one male and sperm competition ensues. In a subsequent previously unpublished study, Matthews and Domjan examined the role of Pavlovian conditioning in a sperm allocation

2/16/2011 6:52:34 PM

522


situation. Here each male was allowed to copulate with two females in succession, and the question was how many of the eggs produced by each female turned out to be fertilized. The two copulations were separated by 15 minutes. In a control group, both of the females were presented to the male subjects in the absence of a sexually conditioned CS. Under these conditions, 24% of the eggs produced by the first female were fertilized but only 6% of the eggs laid by the second female were fertilized (see Table 22.1). This reflects a common sperm depletion effect. Having copulated with the first female, the males had less sperm available to inseminate the second female. What if access to the second female were signaled by a Pavlovian CS? This was evaluated in another group of male quail. Signaling the second female did not change the proportion of fertilized eggs that were laid by the first female with whom the male copulated (23% of the eggs in this group were fertilized as compared to 24% in the control group). However, signaling the second female significantly increased the proportion of fertilized eggs that were laid by the second female (27% in this group were fertilized as contrasted to 6% in the control group; see Table 22.1). Matthews and Domjan also examined the effects of signaling the first female but not the second. Under those conditions, 40% of the eggs laid by the first female were fertilized, an increase of 16% from the control group. As expected, having fertilized so many of the eggs of the first female, the males managed to fertilize only 5% of the eggs of the second female. These experiments show that Pavlovian conditioning also influences sperm allocation, with a greater

effect on fertilization rates of the second female the male encounters. The aforementioned studies are important because they demonstrate the adaptive significance of sexual conditioning in particular and of Pavlovian mechanisms in general. The modifications of behavior and physiology that result from Pavlovian signaling facilitate reproduction in disparate species (blue gourami and quail) and in a variety of situations that include not only one-to-one male/female interactions but also sperm competition and sperm allocation paradigms. The mechanisms of these effects remain to be worked out. In Japanese quail, increased fertility related to the sexual conditioning of the male participants is probably mediated by conditioned cloacal sphincter contractions (Holloway et al., 2005) that help produce cloacal foam which facilitates sperm transport. An additional mechanism is the increased sperm release that occurs in male quail exposed to a sexually conditioned stimulus (Domjan et al., 1998). How sexual conditioning of females facilitates fertilization is a bit less clear, but it might be related to the fact that conditioned females spend more time squatting in the presence of a male (Gutiérrez and Domjan, 1997) and this increases the copulatory efficiency of the male (Domjan et al., 2003).

LESSONS FROM SEXUAL CONDITIONING FOR THE STUDY OF LEARNING Ideally a project of application is not just a oneway street, exporting knowledge from one area to another but an enterprise that brings new

Table 22.1 Pavlovian Signaling When One Male Copulates with Two Females in Succession Percent Fertilized Eggs Female Signaled

1 Female

2 nd Female

Neither

24%

6%

1 st Female

40%

5%

23%

27%

nd

2 Female


st

2/16/2011 6:52:34 PM


insights and perspectives to the source of the application. We next turn to considering the implications of research on sexual conditioning for the study of basic learning processes. Attending to Various Manifestations of Conditioned Behavior

In embarking on an application of Pavlovian conditioning, there are numerous decisions to make. One that has stood out in our experience is deciding what aspect of behavior to measure to obtain evidence of learning. As we described, sexual conditioning can result in changes in a variety of different aspects of appetitive and consummatory sexual behavior and can also alter how sexual partners interact in what is perhaps best described as modifications of unconditioned or instinctive behavior. Success in studies of sexual conditioning would have been severely limited had investigators adopted rigid preconceptions of what a sexually conditioned response should be. However, adopting a broader conception of conditioned behavior yielded benefits beyond providing more sensitive behavioral indices of learning. By measuring a broader range of behaviors, the experiments revealed learning processes that otherwise would not have come to light. Here we describe two of those effects, one involving studies of the CS-US interval and the other involving studies of the C/T ratio in conditioning. Effects of the CS-US Interval

One of the first unexpected findings that emerged from using multiple response measures occurred during studies of the CS-US interval in sexual conditioning. Akins et al. (1994) conditioned independent groups of male quail with CS-US intervals of 0.5, 2.5, 5, 10, 15, and 20 minutes using a gray foam block as the CS and copulation with a female as the US. As expected, less conditioned approach behavior developed in groups that were conditioned with longer CS-US intervals. In fact, no conditioned approach occurred with the 20-minute CS-US interval, suggesting that these subjects did not learn. However, the investigators noticed that quail conditioned with the longer CS-US intervals


523

seemed “restless” and paced from one side of the experimental chamber to the other. Subsequent follow-up experiments indicated that this increased locomotor behavior was a result of the sexual conditioning procedure, since unpaired control groups did not show the effect (Akins et al., 1994, Experiment 2; Akins, 2000). The findings mentioned earlier clearly indicate that the decreases in conditioned behavior that are commonly observed with longer CS-US intervals should not be automatically interpreted as reflecting decrements in learning. Rather, longer CS-US intervals may support learning that is manifest in different behavioral changes. This interpretation is squarely in line with behavior systems theory, which assumes that learning occurs within the context of the behavior system that is activated by a conditioning procedure (Timberlake, 2001). Components of a behavior system are organized in terms of their temporal and spatial proximity to the primary reinforcer. In the feeding system, for example, general search behaviors predominate when the subject is hungry but food has not yet been found and is unavailable. Focal search responses predominate when a potential food source has been located or food is about to become available. Finally, consummatory responses occur when the food is actually encountered. Behavior systems theory assumes that the CS-US interval determines which behavioral component comes to be elicited by the CS. Longer CS-US intervals come to elicit general search responses and short CS-US intervals elicit focal search or consummatory responses (Silva & Timberlake, 1997, 1998). The pacing behavior that developed with a 20-min CS-US interval in the experiments by Akins et al. (1994) and Akins (2000) was probably reflective of “general search” for a sexual partner, whereas the CS approach behavior that is evident with shorter CS-US intervals is reflective of “focal search” behavior. Effects of the C/T Ratio on Conditioning

The aforementioned studies illustrate the importance of the CS-US interval in learning. However, some investigators have argued that the critical factor for learning is not the duration of the CS

2/16/2011 6:52:34 PM


or the CS-US interval but the ratio between the CS duration or trial time (T) and how long the subject spends in the experimental context (C) overall (e.g., Gallistel & Gibbon, 2000). According to this idea, learning will be evident only if the duration of the conditioning trial (T) is substantially shorter than the time that subjects spend in the experimental context (C). This relation can be quantified by dividing time spent in the experimental context (C) by the duration of the conditioning trial (T) to form the C/T ratio. Learning is predicted to occur if the C/T ratio exceeds a critical or threshold value. Burns and Domjan (2001) examined the effects of the C/T ratio in studies of sexual conditioning using a “long box” in which the CS (a wood block) was presented 112 cm from the door that provided access to a female quail. Male quail received one conditioning trial per day for 15 days and were returned to their home cages after each trial. The CS was always presented for 30 seconds, immediately followed by access to the female. To vary the C/T ratio, independent groups of male quail were permitted to remain in the experimental chamber for different lengths of time before the CS was presented. These manipulations created C/T ratios of 1.5, 4.5, 45, and 180. Based on previous theories such as scalar expectancy theory (Gibbon & Balsam, 1981) and rate estimation theory (Gallistel & Gibbon, 2000), learning was expected when the C/T ratio was 45 or 180 but not when it was 1.5 or 4.5. To measure learning, Burns and Domjan (2001) recorded not only approach to the CS area (sign tracking) but also approach to the door from which the female was to be released (goal tracking) on the opposite side of the experimental chamber. The results of these two response measures are summarized in Figure 22.13. Data on sign tracking were as predicted by previous claims that longer C/T ratios are necessary for learning. Sign tracking did not occur with a C/T ratio of 1.5 but was prominent at C/T ratios of 45 and 180. In contrast, the opposite results were obtained when goal tracking served as the response measure. Goal tracking only occurred at the low C/T ratios of 1.5 and 4.5. Additional evidence indicated that the goal tracking that occurred with the low C/T ratios was an associate


60 Sign Tracking Goal Tracking

50 Percentage of Time

524

40 30 20 10 0 1.5

4.5

45

180

C/T Ratio

Figure 22.13 Sign tracking and goal tracking as a function of the C/T ratio.

effect but reflected context conditioning rather than conditioning of the target CS. (For related evidence and discussion, see Domjan, 2003.) These findings provide additional evidence that multiple responses have to be measured in learning experiments to fully appreciate what is learned as a result of a particular training procedure. Conditioning Effects and the Nature of the Conditioned Stimulus

We previously discussed the major impact that the nature of the CS has on the sexual conditioned response. Learning investigators have frequently taken advantage of the fact that different conditioned stimuli generate different conditioned responses. For example, a light conditioned with food comes to elicit rearing behavior in rats, whereas a tone comes to elicit a head-jerk response (Holland, 1977). Using these contrasting conditioned responses, one can determine whether a tone or a light is the controlling stimulus when the two are presented in the same situation. The CS effects that have been discovered in sexual conditioning go beyond providing such methodological advantages. As we noted, use of a CS that includes partial cues of a female not only leads to more rapid learning and a wider range of conditioned responses but also leads to novel learning effects, namely resistance to blocking, resistance to increases in the CS-US interval, and resistance

2/16/2011 6:52:34 PM


to extinction. These findings are reminiscent of examples of adaptive specializations in learning that were prominently discussed some years ago (e.g., Rozin & Schull, 1988). Studies of sexual conditioning bring a new perspective to those theoretical issues. Earlier “adaptive specializations” in learning were discovered by chance and were identified primarily by their inconsistency with prevailing views of learning at the time (Domjan & Galef, 1983). The present findings suggest that natural precursors of a sexual interaction have a privileged status in becoming associated with sexual reinforcement. This suggests a method of discovery of learning specializations. Namely, facilitated learning is predicted to occur whenever the CS used is a precursor of the US in the natural habitat of the organism. This prediction is based on the evidence we reviewed as well as on the basis of the assumption that evolution has shaped how animals learn about cues in their natural habitat rather than how they learn about “arbitrary” cues. Thus, one would expect more robust learning about natural CSs than arbitrary cues. In the present experiments, the primary natural precursor of a copulatory interaction was provided by including partial cues of a female quail in the CS. Including such female cues did not turn the CS into an unconditioned stimulus but allowed the CS to become associated with sexual reinforcement more readily. This suggests that a continuum of sexual conditioning effects may be obtained by varying the extent to which the CS includes components of the US. Increasing those components should facilitate learning and decreasing them should make the CS more like an arbitrary cue. This continuum is reminiscent of the concept of evolutionary preparedness for learning originally proposed by Seligman (1970). However, the current conception lacks the circularity of the original formulation. (For a more detailed discussion of these issues, see Domjan, 2008.)

IMPLICATIONS FOR THE STUDY OF SEXUAL CONDITIONING IN PEOPLE The insights into sexual conditioning that have been provided by research with quail and other non-human species have important implications


525

for the study of sexual learning in Homo sapiens. In contrast to the numerous investigations of sexual conditioning in non-human animals, empirical evidence of sexual conditioning in humans is scant (see Akins, 2004, for a review). In addition, studies in which sexual conditioning may be evident sometimes lack appropriate control groups, thereby making the results difficult to interpret. Hoffmann and colleagues (Hoffmann et al., 2004; see Chapter 23, this volume for further discussion) demonstrated increased genital arousal in men and women to a sexually relevant picture that was presented subliminally and paired with an erotic film. Both and colleagues (Both et al., 2008) found similar results in women when they paired an erotic picture (presented subliminally) with genital vibrotactile stimulation. Thus, there appears to be increasing evidence for sexual conditioning in humans. However, one might argue that sexual conditioning studies with humans could benefit from greater efforts to take advantage of factors that have been found to facilitate sexual conditioning in animal experiments. The animal research suggests that human studies of sexual conditioning should not focus on a single response measure, such as penile tumescence or vaginal lubrication, to detect conditioned responses to a CS. While a few human studies have observed sexual conditioning using measures other than genital responses (Both et al., 2008; Letourneau & Donohue, 1997; see Chapter 23, this volume), the majority of human studies suggest that genital responses may not show strong conditioning effects. Genital responses are close to consummatory responses at the end of the sexual behavior sequence. Research with non-human species has shown that appetitive components of the sexual behavior sequence (e.g., CS approach or increased locomotion in a bilevel chamber) are most easily conditioned. Research with quail suggests that conditioning of genital components of sexual behavior would require special conditioned stimuli, namely CSs that are part of the causal sequence of events that lead to copulation outside the laboratory. Furthermore, conditioning may be more successful if the CSs included some of stimulus elements of a sexual unconditioned stimulus. Indeed, successful

2/16/2011 6:52:34 PM

526


demonstrations of sexual conditioning in humans have utilized erotic pictures and arousing videotapes of heterosexual sexual interactions as the CS (Both et al., 2008; Kantorowitz, 1978; Lalumière & Quinsey, 1998). In contrast to these recommendations, most studies of human sexual conditioning have typically employed entirely arbitrary CSs (e.g., pictures of squares and triangles) that have no inherent relation to normal sexual activity. The animal research also indicates that copulation is much more effective as a US than visual exposure to a potential sexual partner. Yet visual exposure to sexual pictures or film clips is typically employed in human research (e.g., Hoffmann et al., 2004; Langevin & Martin, 1975; Rachman, 1966) with the exception of a few (e.g., Kantorowitz, 1978; see also Chapter 23, this volume). Furthermore, the individual pictures cannot be characterized as potential sexual partners because no actual sexual activity takes place in the human studies. Another major issue is the duration of the intertrial interval. Pavlovian conditioning in conventional laboratory preparations (e.g., eyeblink conditioning or appetitive conditioning with food) involves multiple conditioning trials in each session, with the trials spaced no more than a minute or two apart. The use of such massed trials has been incorporated in human studies of sexual conditioning (e.g., Letourneau & O’Donohue, 1997; Plaud & Martini, 1999). This contrasts with studies of sexual conditioning in non-human subjects, which typically provide no more than one conditioning trial per day. The rationale for long intertrial intervals is that sexual behavior has a substantial refractory period and is therefore only reinforcing if a long intertrial interval is used. Success in human research on sexual learning is likely to involve the same fundamental considerations that have facilitated the study of such learning in non-human species. In particular, investigators have to be cognizant of the behavioral and temporal organization of human sexual behavior under natural conditions and then take this information into account in


making decisions about the temporal organization of the conditioning procedure, the CS and US that are used, and the responses that are expected to change as a result of the conditioning procedure.

SUMMARY AND CONCLUSION Research on sexual conditioning has clearly shown that the sexual behavior system is readily susceptible to learning effects. Given the range of learning effects that have been documented, it is safe to say that there is probably no conditioning effect that occurs in conventional learning preparations that cannot be also found in the sexual behavior system. Thus, the research has amply demonstrated that the generality of learning extends to sexual behavior. Research on sexual conditioning has also provided new insights into the nature of sexual behavior. It has told us that the stimulus control of various components of sexual behavior is not limited to stimuli that elicit the behavior unconditionally. Rather, the stimulus control of various components of sexual behavior can be extended to a wide range of new stimuli that include visual and olfactory cues, as well as various three-dimensional objects. There appear to be few limitations on the types of stimuli that can come to control appetitive components of sexual behavior (e.g., approach responses). However, consummatory sexual responses (grabs, mount, and cloacal contact responses in quail) are most easily conditioned if the CS object includes at least limited species typical features or stimulus elements that are part of the courtship → copulation sequence in the natural environment of the species. Studies have also demonstrated that conditioning can modify not only behavioral aspects of sex but also the preparation of cloacal foam, the quantity of sperm that are released, and the number and paternity of the offspring that are produced. Thus, the impact of Pavlovian learning on sexual behavior is broad and of substantial evolutionary significance. Studies of sexual conditioning have also provided important lessons about how to investigate learning and how learning operates in

2/16/2011 6:52:34 PM


situations that more closely resemble the natural environment than common laboratory situations. In particular, studies of sexual conditioning illustrate the importance of taking multiple measures of behavior in a learning experiment rather than focusing on the detection and measurement of a single index of conditioning. They also illustrate that rather than just examining conditioned responses elicited by the CS, it is important to consider how Pavlovian conditioning may modify the organism’s interactions with the unconditioned stimulus. These learned modifications on the unconditioned response may be of greater adaptive significance than the development of a new response to the CS. Finally, studies of sexual conditioning have demonstrated that the nature of the CS contributes to much more than just the topography of the conditioned response. The nature of the CS may also modify the learning effects that are observed, especially if the CS is part of the natural causal chain that leads to the US in the wild and includes species typical features. Studies of sexual conditioning with human participants has been far more limited than with laboratory animals. Future studies with human participants may benefit from greater attention to the factors that have been identified as important in the animal experiments. These factors include the use of more naturalistic conditioned and unconditioned stimuli, the measurement of various components of appetitive sexual behavior rather than just focusing on genital responses, and the use of more widely spaced conditioning trials. Overall, the application of conditioning to sexual behavior has provided a wealth of information relevant to learning theory, biological fitness, and reproductive behavior. More recently, neurochemical and neurobiological aspects of sexual conditioning have been also explored (e.g., Coria-Avila & Pfaus, 2007; CoriaAvila et al., 2008; Taziaux, Kahn, Moore, Balthazart, & Holloway, 2008). The application of learning to sexual behavior also continues to be used to investigate the causal relationship between sexual arousal/motivation and other clinically related behaviors such as substance abuse (e.g., Troisi & Akins, 2004).


527

NOTE 1. The CS was present during the entire duration of the CS-US interval in this experiment so that all paired subjects would receive a delayed conditioning procedure. If the CS duration had been kept constant at 1 minute, subjects conditioned with a 20-minute CS-US interval would have experienced a trace conditioning procedure with a long trace interval. For a study of trace conditioning in the sexual behavior system, see Akins and Domjan (1996).

REFERENCES Adkins-Regan, E., & MacKillop, E. A. (2003). Japanese quail (Coturnix japonica) inseminations are more likely to fertilize eggs in a context predicting mating opportunities. Proceedings of the Royal Society of London, Series B, 270, 1685–1689. Agmo, A., & Berenfeld, R. (1990). Reinforcing properties of ejaculation in the male rat: The role of opioids and dopamine. Behavioral Neuroscience, 107, 177–182. Akins, C. K. (1998). Context excitation and modulation of conditioned sexual behavior. Animal Learning and Behavior, 26, 416–426. Akins, C. K. (2000). Effects of species-specific cues and the CS-US interval on the topography of the sexually conditioned response. Learning and Motivation, 31, 211–235. Akins, C. K. (2004). The role of Pavlovian conditioning in sexual behavior: A comparative analysis of human and non-human animals. International Journal of Comparative Psychology, 17, 241–262. Akins, C. K., & Domjan, M. (1996). The topography of sexually conditioned behaviour: Effects of a trace interval. Quarterly Journal of Experimental Psychology, 49B, 346–356. Akins, C. K., Domjan, M., & Gutiérrez, G. (1994). Topography of sexually conditioned behavior in male Japanese quail (Coturnix japonica) depends on the CS-US interval. Journal of Experimental Psychology: Animal Behavior Processes, 20, 199–209. Ball, G. F., & Balthazart, J. (2004). Hormonal regulation of brain circuits mediating male sexual behavior in birds. Physiology and Behavior, 83(2), 329–346. Boakes, R. A. (1977). Performance on learning to associate a stimulus with positive reinforcement.

2/16/2011 6:52:34 PM

528


In H. Davis & H. M. B. Hurwitz (Eds.), OperantPavlovian interactions (pp. 67–101). Hillsdale, NJ: Erlbaum. Bolin, B. L., & Akins, C. K. (2009). Methamphetamine impairs sexual motivation but not sexual performance in male Japanese quail. Experimental and Clinical Psychopharmacology, 17, 10–20. Both, S., Spiering, M., Laan, E., Belcome, S., van den Heuvel, B., & Everaerd, W. (2008). Unconscious classical conditioning of sexual arousal: Evidence for the conditioning of female genital arousal to subliminally presented sexual stimuli. Journal of Sexual Mediciine, 5, 100–109. Burns, M., & Domjan, M. (1996). Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica). Journal of Experimental Psychology: Animal Behavior Processes, 22, 297–306. Burns, M., & Domjan, M. (2000). Sign tracking in domesticated quail with one trial a day: Generality across CS and US parameters. Animal Learning and Behavior, 28, 109–119. Burns, M., & Domjan, M. (2001). Topography of spatially directed conditioned responding: Effects of context and trial duration. Journal of Experimental Psychology: Animal Behavior Processes, 27, 269–278. Çetinkaya, H., & Domjan, M. (2006). Sexual fetishism in a quail (Coturnix japonica) model system: Test of reproductive success. Journal of Comparative Psychology, 120, 427–432. Coria-Avila, G. A., Gavrila, A. M., Boulard, B., Charron, N., Stanley, G, & Pfaus, J. G. (2008). Neurochemical basis of conditioned partner preference in the female rat: II. Disruption by flupenthixol. Behavioral Neuroscience, 122(2), 396–406. Coria-Avila, G. A., Jones, S. L., Solomon, C. E., Gavrila, A. M., Jordan, G. J., & Pfaus, J. G. (2006). Conditioned partner preference in female rats for strain of male. Physiology and Behavior, 88, 529–537. Coria-Avila, G. A., Ouimet, A. J., Pacheco, P., Manzo, J., & Pfaus, J. G. (2005). Olfactory conditioned partner preference in the female rat. Behavioral Neuroscience, 119(3), 716–725. Coria-Avila, G. A., & Pfaus, J. G. (2007). Neuronal activation by stimuli that predict sexual reward in female rats. Neuroscience, 148(3), 623–632. Crawford, L. L., & Domjan, M. (1993). Sexual approach conditioning: Omission contingency tests. Animal Learning and Behavior, 21, 42–50.


Crawford, L. L., & Domjan, M. (1995). Secondorder sexual conditioning in male Japanese quail (Coturnix japonica). Animal Learning and Behavior, 23, 327–334. Crawford, L. L., & Domjan, M. (1996). Conditioned inhibition of social approach in male Japanese quail (Coturnix japonica) using visual exposure to a female. Behavioural Processes, 36, 163–169. Cusato, B., & Domjan, M. (1998). Special efficacy of sexual conditioned stimuli that include species typical cues: Tests with a CS preexposure design. Learning and Motivation, 29, 152–167. Domjan, M. (1998). Going wild in the laboratory: Learning about species typical cues. In D. L. Medin (Ed.), The psychology of learning and motivation (pp. 155–186). San Diego, CA: Academic Press. Domjan, M. (2003). Stepping outside the box in considering the C/T ratio. Behavioural Processes, 62, 103–114. Domjan, M. (2005). Pavlovian conditioning: A functional perspective. Annual Review of Psychology, 56, 179–206. Domjan, M. (2008). Adaptive specializations and generality of the laws of classical and instrumental conditioning. In J. Byrne (Series Ed.) & R. Menzel (Vol. Ed.), Learning and memory: A comprehensive reference: Vol. 1. Learning and behavior theory (pp. 327–340) Oxford, England: Elsevier. Domjan, M., Akins, C., & Vandergriff, D. H. (1992). Increased responding to female stimuli as a result of sexual experience: Tests of mechanisms of learning. Quarterly Journal of Experimental Psychology, 45B, 139–157. Domjan, M., Blesbois, E., & Williams, J. (1998). The adaptive significance of sexual conditioning: Pavlovian control of sperm release. Psychological Science, 9, 411–415. Domjan, M., Cusato, B., & Krause, M. (2004). Learning with arbitrary versus ecological conditioned stimuli: Evidence from sexual conditioning. Psychonomic Bulletin and Review, 11, 232–246. Domjan, M., & Galef, B. G., Jr. (1983). Biological constraints on instrumental and classical conditioning: Retrospect and prospect. Animal Learning and Behavior, 11, 151–161. Domjan, M., Greene, P., & North, N. C. (1989). Contextual conditioning and the control of copulatory behavior by species-specific sign stimuli in male Japanese quail. Journal of Experimental Psychology: Animal Behavior Processes, 15, 147–153.

2/16/2011 6:52:34 PM


Domjan, M., & Hall, S. (1986). Determinants of social proximity in Japanese quail (Coturnix coturnix japonica): Male behavior. Journal of Comparative Psychology, 100, 59–67. Domjan, M., Huber-McDonald, M., & Holloway, K. S. (1992). Conditioning copulatory behavior to an artificial object: Efficacy of stimulus fading. Animal Learning and Behavior, 20, 350–362. Domjan, M., Lyons, R., North, N. C., & Bruell, J. (1986). Sexual Pavlovian conditioned approach behavior in male Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 100, 413–421. Domjan, M., Mahometa, M., & Mills, A. D. (2003). Relative contributions of the male and female to sexual behavior and reproductive success in the Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 117, 391–399. Domjan, M., O’Vary, D., & Greene, P. (1988). Conditioning of appetitive and consummatory sexual behavior in male Japanese quail. Journal of the Experimental Analysis of Behavior, 50, 505–519. Farris, H. E. (1967). Classical conditioning of courting behavior in the Japanese quail, Coturnix coturnix japonica. Journal of the Experimental Analysis of Behavior, 10, 213–217. Fiorino, D. F., & Phillips, A. G. (1999). Faciliation of sexual behavior in male rats following d-amphetamine-induced behavioral sensitization. Psychopharmacology (Berlin), 142(2), 200–208. Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107, 289–344. Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York, NY: Academic Press. Graham, J. M., & Desjardins, C. (1980). Pavlovian conditioning: Induction of luteinizing hormone and testosterone secretion in anticipation of sexual activity. Science, 210, 1039–1041. Gutiérrez, G., & Domjan, M. (1996). Learning and male-male sexual competition in Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 110, 170–175. Gutiérrez, G., & Domjan, M. (1997). Differences in the sexual conditioned behavior of male and female Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 110, 135–142.


529

Hearst, E., & Jenkins, H. M. (1974). Sign tracking: The stimulus-reinforcer relation and directed action. Austin, TX: Psychonomic Society. Hilliard, S., & Domjan, M. (1995). Effects of sexual conditioning of devaluing the US through satiation. Quarterly Journal of Experimental Psychology, 48B, 84–92. Hilliard, S., Nguyen, M., & Domjan, M. (1997). One-trial appetitive conditioning in the sexual behavior system. Psychonomic Bulletin and Review, 4, 237–241. Hoffmann, H., Janssen, E., & Turner, S. L. (2004). Classical conditioning in the effects of sexual arousal of women and men: Effects of varying awareness and biological relevance of the conditioned stimulus. Archives of Sexual Behavior, 33(1), 43–53. Holland, P. C. (1977). Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. Journal of Experimental Psychology: Animal Behavior Processes, 3, 77–104. Hollis, K. L., Cadieux, E. L., & Colbert, M. M. (1989). The biological function of Pavlovian conditioning: A mechanism for mating success in the blue gourami (Trichogaster trichopterus). Journal of Comparative Psychology, 103, 115–121. Hollis, K. L., Pharr, V. L., Dumas, M. J., Britton, G. B., & Field, J. (1997). Pavlovian conditioning provides paternity advantage for territorial male blue gouramis. (Trichogaster trichopterus). Journal of Comparative Psychology, 111, 219–225. Holloway, K. S., Balthazart, J., & Cornil, C. A. (2005). Androgen mediation of conditioned rhythmic cloacal sphincter movements in Japanese Quail (Coturnix japonica). Journal of Comparative Psychology, 119(1), 49–57. Holloway, K. S., & Domjan, M. (1993a). Sexual approach conditioning: Tests of unconditioned stimulus devaluation using hormone manipulations. Journal of Experimental Psychology: Animal Behavior Processes, 19, 47–55. Holloway, K. S., & Domjan, M. (1993b). Sexual approach conditioning: Unconditioned stimulus factors. Journal of Experimental Psychology: Animal Behavior Processes, 19, 38–46. Hughes, A. M., Everitt, B. J., & Herbert, J. H. (1990). Comparative effects of preoptic area infusions of opioid peptides, lesions, and castration on sexual behavior in male rats: Studies of instrumental behavior, conditioned place preference, and partner preference. Psychopharmacology, 102, 243–256.

2/16/2011 6:52:34 PM

530


Jenkins, W. J., & Becker, J. B. (2003). Female rats develop conditioned place preference for sex at their preferred interval. Hormones and Behavior, 43, 503–507. Kantorowitz, D. A. (1978). Personality and conditioning of tumescence and detumescence. Behaviour Research and Therapy, 16, 117–123. Kippin, T. E., Talianakis, S., Schattmann, L., Bartholomew, S., & Pfaus, J. G. (1998). Olfactory conditioning of sexual behavior in the male rat (Rattus norvegicus). Journal of Comparative Psychology, 112, 389–399. Kippin, T. E., & Pfaus, J. G. (2001). Nature of the conditioned response mediating olfactory conditioned ejaculatory preference in the male rat. Behavioral Brain Research, 122, 11–24. Köksal, F., & Domjan, M. (1998). Observational conditioning of sexual behavior in the domesticated quail. Animal Learning and Behavior, 26, 427–432. Köksal, F., Domjan, M., Kurt, A., Sertel, Ö., Sabiha, Ö., Bowers, R., & Kumru, G. (2004). An animal model of fetishism. Behavioural Research and Therapy, 42, 1421–1434. Köksal, F., Domjan, M., & Weisman, G. (1994). Blocking of the sexual conditioning of differentially effective conditioned stimulus objects. Animal Learning and Behavior, 22, 103–111. Krause, M. A., Cusato, B., & Domjan, M. (2003). Extinction of conditioned sexual responses in male Japanese quail (Coturnix japonica): Role of species typical cues. Journal of Comparative Psychology, 117, 76–86. Lalumière, M. L., & Quinsey, V. L. (1998). Pavlovian conditioning of sexual interests in human males. Archives of Sexual Behavior, 27, 241–252. Langevin, R., & Martin, M. (1975). Can erotic response be classically conditioned? Behavior Therapy, 6, 350–355. Letourneau, E. J., & O’Donohue, W. (1997). Pavlovian conditioning of female sexual arousal. Archives of Sexual Behavior, 26, 63–78. Levens, N., & Akins, C. K. (2004). Chronic cocaine pretreatment facilitates Pavlovian sexual conditioning in male Japanese quail. Pharmacology, Biochemistry, and Behavior, 79(3), 451–457. Mehrara, B. J., & Baum, M. J. (1990). Naloxone disrupts the expression but not the acquisition by male rats of a conditioned place preference response for an oestrous female. Psychopharmacology, 101, 118–125. Mahometa, M. J., & Domjan, M. (2005). Classical conditioning increases reproductive success in


Japanese quail, Coturnix japonica. Animal Behaviour, 69, 983–989. Matthews, R. N., Domjan, M., Ramsey, M., & Crews, D. (2007). Learning effects on sperm competition and reproductive fitness. Psychological Science, 18, 758–762. McCarthy, M. M., & Bell, G. F. (2008). The neuroendocrine control of sex-specific behavior in vertebrates: Lessons from mammals and birds. Current Topics in Developmental Biology, 83, 213–248. Mendelson, S. D., & Pfaus, J. G. (1989). Level searching: A new assay of sexual motivation in the male rat. Physiology and Behavior, 45, 337–341. Nelson, J. B. (2009). Contextual control of firstand second-learned excitation and inhibition in equally ambiguous stimuli. Learning and Behavior, 37, 95–106. Oldenburger, W. P., Everitt, B. J., & de Jonge, F. H. (1992). Conditioned place preference produced by sexual interaction in female rats. Hormones and Behavior, 26, 214–228. Paredes, R. G., & Alonso, A. (1997). Sexual behavior regulated (paced) by the female induces conditioned place preference. Behavioral Neuroscience, 111, 123–128. Paredes, R. G., & Vazquez, B. (1999). What do females like about sex? Paced mating. Behavioural Brain Research, 105, 117–127. Pfaus, J. G., Mendelson, S. D., & Phillips, A. G. (1990). A correlational and factor analysis of anticipatory and consummatory measures of sexual behavior in the male rat. Psychoneuroendocrinology, 15, 329–340. Plaud, J. J., & Martini, J. R. (1999). The respondent conditioning of male sexual arousal. Behavior Modification, 23, 254–269. Rachman, S. (1966). Sexual fetishism: An experimental analogue. Psychological Record, 16, 293–296. Rozin, P., & Schull, J. (1988). The adaptiveevolutionary point of view in experimental psychology. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. Duncan Luce (Eds.), Stevens’ handbook of experimental psychology (2nd ed., Vol. 1, pp. 503–546). New York, NY: Wiley. Sachs, B. D., & Garinello, L. D. (1978). Interaction between penile reflexes and copulation in male rats. Journal of Comparative Physiological Psychology, 92, 759–678. Seligman, R. E. P. (1970). On the generality of the laws of learning. Psychological Review, 77, 406–418.

2/16/2011 6:52:34 PM


Sheffield, F. D., Wulff, J. J., & Backer, R. (1951). Reward value of copulation without sex drive reduction. Journal of Comparative Physiological Psychology, 44, 3–8. Schwartz, C. W., & Schwartz, E. R. (1949). A reconnaissance of the game birds in Hawaii. Hilo, Hawaii: Hawaii Board of Commissioners of Agriculture and Forestry. Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.), Classical conditioning (pp. 302–322). New York, NY: AppletonCentury-Crofts. Silva, K. M., & Timberlake, W. (1997). A behavior systems view of conditioned states during long and short CS-US intervals. Learning and Motivation, 28, 465–490. Silva, K. M., & Timberlake, W. (1998). A behavior systems view of responding to probe stimuli during an interfood clock. Animal Learning and Behavior, 26, 313–325. Taziaux, M., Kahn, A., Moore, J., Balthazart, J., & Holloway, K. S. (2008). Enhanced neural activation in brain regions mediating sexual responses following exposure to a CS that predicts copulation. Neuroscience, 151(3), 644–658.


531

Timberlake, W. (2001). Motivational modes in behavior systems. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 155–209). Mahwah, NJ: Erlbaum. Troisi, J. R., & Akins, C. K. (2004). The discriminative stimulus effects of cocaine in a Pavlovian sexual approach paradigm in male Japanese quail. Experimental and Clinical Psychopharmacology, 12(4), 237–242. Volkow, N. D, Wang, G. J., Fowler, J. S., Telang, F., Jayne, M., & Wong, C. (2007). Stimulantinduced enhanced sexual desire as a potential contributing factor in HIV transmission. American Journal of Psychiatry, 164(1), 157–160. Zamble, E., Hadad, G. M., Mitchell, J. B., & Cutmore, T. R. H. (1985). Pavlovian conditioning of sexual arousal: First- and second-order effects. Journal of Experimental Psychology: Animal Behavior Processes, 11, 598–610. Zule, W. A., Costenbader, E. C., Meyer, W. J., Jr., & Wechsberg, W. M. (2007). Methamphetamine use and risky sexual behaviors during heterosexual encounters. Sexually Transmitted Diseases, 34(9), 689–694.

2/16/2011 6:52:34 PM

CHAPTER 23 Hot and Bothered Classical Conditioning of Sexual Incentives in Humans Heather Hoffmann

Cross-cultural and individual variation in erotic taste indicate that what we find sexually attractive depends on experience. Partner and other environmental cues can acquire sexually arousing (or inhibiting) properties through a variety of different types of learning processes, including imprinting, mere exposure, social learning, verbal relational learning, and operant conditioning. Most laboratory research on sexual learning, however, has employed classical conditioning procedures. Numerous studies demonstrate the impact of such conditioning on a wide range of sexual behaviors in non-humans, yet relatively few studies have shown such effects in humans. The present chapter reviews the experimental research on classical conditioning of sexual arousal in humans, highlighting newer studies that use women participants and more diverse paradigms. Individual differences in conditionability and a distinction between signal versus evaluative learning are also considered. Such research has the potential to contribute to the literature on (human) learning theory as well as to enhance learning-based therapies used to alter problematic sexual responding, for example, in the case of sexual risk taking and/or sexual compulsivity.

The idea that learning plays an important role in sexual responding is not new (e.g., Binet, 1888; Craig, 1918), and it is now commonly assumed that conditioning processes affect the development of normative as well as atypical (e.g., fetishism) sexual arousal patterns (e.g., Ågmo, 1999; Gaither, Rosenkranz, & Plaud, 1998; Hardy, 1964; McConaghy, 1987; Pfaus, Kippin, & Centeno, 2001; Roche & Barnes, 1998; Woodson, 2002). Indeed, numerous experimental studies have demonstrated the impact that learning has on a wide range of sexual behaviors across a variety of species (see Akins, 2004; Chapter 2, this volume; Domjan & Holloway, 1998; Pfaus, Kippin, & Centeno, 2001, for reviews). However, there is still relatively little empirical evidence of sexual conditioning from studies using human and/or female subjects. Furthermore, the animal

and human sexual conditioning literatures are not clearly integrated (Akins, 2004), and precisely how conditioning processes affect what we find erotic remains unclear. Only a narrow range of stimuli can be regarded as primary or “inherent” sexual incentives (i.e., sexually attractive cues). Stimuli typically acquired sexually arousing properties through experience. While a few studies have shown that partner and other environmental stimuli can become sexually arousing through imprinting (e.g., Bateson, 1978; Kendrick, Hinton, Atkins, Haupt, & Skinner, 1998), mere exposure (e.g., Dewsbury, 1981; Lisk & Baron, 1982), observational learning (e.g., Köksal & Domjan, 1998; White, 2004), and verbal learning (Gavin, Roche, & Ruiz, 2008; Roche & Barnes, 1998), acquired preferences are more

532


2/16/2011 6:05:41 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

commonly attributed to classical (Pavlovian) conditioning and/or operant (instrumental) conditioning (e.g., Akins, 2004; Laws & Marshall, 1991; McConaghy, 1987; Pfaus et al., 2001; Woodson, 2002). Classical conditioning consists of learning about the relationship between an initially innocuous cue (the conditioned stimulus [CS]) and a biologically significant one (the unconditioned stimulus [US]). This type of conditioning, in contrast to operant conditioning, appears most directly related to how cues can acquire arousing properties and therefore is the focus of the present chapter. Nonetheless, operant procedures have been shown to influence human sexual arousal (Rosen, Shapiro, & Schwartz, 1975), and both classical and instrumental processes/procedures most likely interact in the development of sexual preferences (e.g., Junginger, 1997; McGuire, Carlisle, & Young, 1965; also see Chapter 1, this volume).

THE NATURE OF SEXUAL CONDITIONING Pavlovian procedures have been employed in most animal and human studies aimed at examining the role of learning in sexual arousal. Although viewed as a reflexive process with limited applicability to higher level human behavior, we now recognize that classical conditioning can serve a number of important functions. For example, it can prepare organisms for interaction with biologically significant cues or events (signal or expectancy learning), and it can alter the preference for stimuli associated with such cues or events (evaluative conditioning). The outcome of classical conditioning is typically regarded as signal learning in which the CS comes to predict the occurrence of the US and hence prepares the organism for it (e.g., Rescorla, 1988). Extensive laboratory data using animals support this conceptualization, and recent research deriving from a functional approach illustrates how classical conditioning leads to adaptive behaviors in the naturalistic environment (Domjan; 2005, Hollis, 1997; Timberlake, 2001). More specifically, there is evidence that sexual conditioning involves S-S rather than merely S-R relations (e.g., Hilliard &


533

Domjan, 1995; Hilliard, Domjan, Nguyen, & Cusato, 1998; Holloway & Domjan, 1993) and that classical conditioning increases reproductive fitness (Adkins-Regan & MacKillop, 2003; Hollis, Pharr, Dumas, Britton, & Field, 1997; Mahometa & Domjan, 2005; Matthews, Domjan, Ramsey, & Crews, 2007). Evaluative conditioning (EC) is a more recently recognized form of classical conditioning that involves a (associative) transfer of affective value, or valence, as a result of exposure to CS-US pairings (De Houwer, Thomas, & Baeyens, 2001; Chapter 18, this volume). In contrast to signal learning, EC has been almost exclusively researched in humans (largely because the subjective experience of liking or disliking is difficult to measure in animals). The prototypical EC paradigm involves pairing of a neutral CS (e.g., a neutrally rated face) with pictures of liked or disliked stimuli, and it results in a change in liking for the CS. Evaluative conditioning appears to be robust in some instances (yet fails to appear in others; see De Houwer, Baeyens, & Field, 2005; Rozin, Wrzesniewski & Barnes, 1998) and has been demonstrated in a variety of paradigms employing diverse cues, including CSs from a range of sensory modalities and more biologically relevant USs as well as with direct and indirect measures (De Houwer et al., 2001). EC appears to be involved in the development of likes and dislikes (e.g., DeHouwer et al., 2005) as well as attitudes (e.g., Baccus, Baldwin, & Packer, 2004; Dijksterhuis, 2004; Gawronski & Bodenhausen, 2006; Karpinski & Hilton, 2001; Livingston & Drwecki, 2007), and recent research suggests it may also occur during sexual conditioning (Both et al., 2008; Hoffmann, 2007; Hoffmann & Janssen, 2006). Signal and evaluative conditioning can be dissociated within the same paradigm (e.g., Hermans, Vansteenwegen, Crombez, Baeyens, & Eelen, 2002), suggesting they are distinct, although not necessarily independent processes. Furthermore, EC appears less sensitive to CS-US contingency and modulation (e.g., Baeyens, Crombez, De Houwer, & Eelen, 1996; Baeyens, Hendrickx, Crombez, & Hermans, 1998; Olson & Fazio, 2001; but see Hardwick & Lipp, 2000; Lipp & Purkis, 2005) and more resistant to

2/16/2011 6:05:41 PM

534


extinction (e.g., Baeyens, Crombez, Van den Bergh, & Eelen, 1988; Hermans, Crombez, Vansteenwegen, Baeyens, & Eelen, 2002; but see Lipp & Purkis, 2005, 2006). In addition, there is evidence that EC can be acquired without awareness of the CS-US contingency (e.g., De Houwer, Hendrickx, & Baeyens, 1997; Dickinson & Brown, 2007; Walther & Nagengast, 2006) and that its expression may be more automatic (Neumann, Forster, & Strack, 2003; Öhman & Mineka, 2001; Yin & Knowlton, 2006). It has been proposed that classical conditioning may incorporate at least two different types of processes that may employ distinct algorithms for association (e.g., Baeyens, Vansteenwegen, Hermans, & Eelens, 2001; DeHouwer et al., 2001) and/or expression of learning (Field, 2005). Baeyens et al. (2001) proposed that signal learning is governed by an expectancy system that requires more cognitive resources to process or translate complex information resulting in anticipation of an object or event. On the other hand, EC is mediated by a more “primitive” referential system that employs more rudimentary learning or performance rules resulting in changes in affective value that can influence the direction of behavior (approach/avoid) and modulate (facilitate/suppress) responses generated by the expectancy system. It is possible that sexual conditioning involves both signal and evaluative learning. It is also possible that the nature of the association(s) acquired in a sexual setting may depend on factors such as type of cue, learning context, or individual personality. It was with these ideas that I began to explore what was known about the specific conditions under which classical conditioning affects (human) sexual responding and to plan and execute studies to better understand the nature of classical associations acquired in sexual situations.

SEXUAL CONDITIONING EXPERIMENTS: PAST AND PRESENT Non-Humans

The majority of studies demonstrating a role for classical conditioning in sexual behavior have


been conducted in animals (see Akins, 2004; Chapter 22, this volume; Domjan & Holloway, 1998; Pfaus et al., 2001, for review). For example, Domjan and colleagues have found conditioned approach, conditioned courtship, and conditioned copulatory behaviors in male quail in the presence of CSs (e.g., colored lights, orange feathers, bird models, and contextual cues) that were previously paired with either visual exposure to a female or the opportunity to copulate with a female (for review, see Chapter 22, this volume; Domjan & Holloway, 1998). In male rats, Zamble, Hadad, Mitchell, and Cutmore (1985) found conditioned decreases in ejaculatory latency in the presence of a CS (plastic tub) that had previously been paired with exposure to a female, and Pfaus and colleagues found conditioned ejaculatory preference for females scented with an odor that was previously associated with the opportunity to copulate (Kippin, Talianakas, & Pfaus, 1997; Kippin, Talinakis, Schattmann, Bartholomew, & Pfaus, 1998). Sexual conditioning in males has also been evidenced using physiological and other nonbehavioral measures. For example, cues paired with access to a receptive female can increase serum lutenizing hormone (LH) and testosterone levels in male rats (Graham & Desjardins, 1980), sperm volume and concentration in quail (Domjan, Blesbois, & Williams, 1998), the number of offspring in quail (Adkins-Regan & MacKillop, 2003; Mahometa & Domjan, 2005; Matthews, Domjan, Ramsey, & Crews, 2007), and the number of offspring in blue gourami fish (Hollis, Pharr, Dumas, Britton, & Field, 1997). Although there is less research conducted using female non-humans, there are a few studies showing that they also can be sexually conditioned. Gutiérrez and Domjan (1997) found increased squatting behavior, an index of sexual receptivity, in female quail following the paired presentation of a particular compartment (CS) and copulatory opportunity. Coria-Avila, Ouimet, Pachero, Manzo, and Pfaus (2005) and Coria-Avila, Jones, Solomon, Garvrila, Jordan, and Pfaus (2006) showed a conditioned partner preference in female rats (increased solicitations, higher magnitude lordosis) for males bearing cues paired with the ability to pace copulation,

2/16/2011 6:05:41 PM


which is rewarding for female rats (Paredes & Alonso, 1997). Furthermore, female hamsters show conditioned placed preference for an environment paired with sexual interaction (Meisel & Joppa, 1994) and an environment paired with vaginal stimulation (Kohlert & Olexa, 2005). Humans

O’Donohue and Plaud (1994) and Akins (2004) provide the most recent reviews of the conditioning of human sexual arousal. Most studies have tested male subjects using visual stimuli and have measured learning via changes in genital responding. Specifically, CSs have been nonsexual or sexual images, most often photographs, presented in slides or on a computer screen. Several studies, particularly the more recent ones, have employed a differential conditioning design. including a CS+ and a CS– (i.e., a cue presented during acquisition but not explicitly un/paired with the US nor explicitly un/paired with the CS+). The most commonly used USs have been sexually explicit photographs or films. Genital responding (i.e., increased blood flow) has been monitored by use of a penile plethysmograph (e.g., electromechanical strain

535

gauge’ Barlow, Becker, Leitenberg, & Agras, 1970; see Fig. 23.1) or vaginal photoplethysmograph (Hoon, Wincze, & Hoon, 1976; Sintchak & Geer 1975; see Fig. 23.2). Participants place these monitors on themselves in private with instruction from researchers. The strain gauge, placed approximately 1 cm from the base of the penis, detects changes in penile circumference. The photoplethysmograph resembles an acrylic tampon and its depth and position can be controlled by a perspex plate placement device (Laan, Everaerd, & Evers, 1995). This vaginal device monitors light reflection off vaginal walls, with greater back-scattered light representing increased blood flow. It yields two analyzable signals, yet the AC-coupled signal known as vaginal pulse amplitude (VPA), which appears to reflect phasic changes in pressure in vaginal blood vessels, is the most commonly used measure. Although there is still some uncertainty about what changes in VPA represent (e.g., Levin, 1998), they appear to be specific to sexual as opposed to other types of stimuli (Laan et al., 1995). Due to their ease of use, both of these plethysmographs are standard in sexual psychophysiological research (Geer & Janssen, 2000) However, care should be taken in generalizing from human sexual psychophysiological

Figure 23.1 Penile plethysmograph. Courtesy Nikki Prause.


2/16/2011 6:05:41 PM

536


Figure 23.2 Vaginal photoplethysmograph. Courtesy Jorge Ponseti.

studies as a volunteer bias has been documented (e.g., participants have more as well as more varied sexual experience, higher rates of masturbation, lower sexual guilt and inhibition, higher sensation-seeking tendencies, and are lower in conformity; Bogaert, 1996; Morokoff, 1986; Plaud, Gaither, Hegstad, Rowan, & Devitt, 1999; Strassberg & Lowe, 1995). Although numerous case studies indicate that behavior or response modification techniques can alter patterns of sexual arousal/ behavior (e.g., see Akins, 2004; Gaither et al., 1998 for review), relatively few experiments have shown that Pavlovian procedures can be used to condition arousal in a nonclinical sample. Rachman’s (1966) study was among the first attempts. After 24–65 pairings of a slide of a color photograph depicting a pair of women’s knee-high black boots immediately followed by one of six slides of attractive nude women, the three male participants showed enhanced genital responding to the slide of the boots during extinction testing. However, methodological concerns (e.g., that lack of a control group) in this study as well as several other experiments conducted in the 1970s (e.g., Kantorowitz, 1978; Langevin & Martin, 1975; Rachman & Hodgson, 1968) precluded a convincing demonstration of classical conditioning of human sexual arousal.


More carefully controlled experiments carried out in the late 1990s revealed unequivocal evidence of sexual conditioning, at least in men. Lalumière and Quinsey (1998), using a CS-only control in addition to an experimental group, found enhanced genital responding to slides of partially nude females after they had been paired with sexually explicit videotapes of heterosexual interactions. Plaud and Martini (1999), using a backward and a random control in addition to an experimental group, found conditioned increases in penile circumference to a penny jar after it was paired with slides of nude or partially nude females. Recent Research

More recent sexual conditioning research includes women and more diverse CSs (e.g., olfactory cues), USs (e.g., vibrogenital stimulation), and measures of conditioned responding (e.g., subjective arousal, general affective ratings, skin conductance). Our laboratory has also shown conditioning of sexual arousal in men outside of the laboratory (i.e., in their residence). The first published study to examine the role of classical conditioning in women’s sexual response was conducted by Letourneau and O’Donohue (1997), but they failed to show

2/16/2011 6:05:41 PM


conditioned genital or subjective responses to a CS (an amber light) that had been repeatedly paired with an erotic film. The first clear demonstration of sexual conditioning in women was found in our laboratory (Hoffmann, Janssen, & Turner, 2004). Like Letourneau and O’Donohue, we also used a visual stimulus as the CS and erotic film clips as USs, yet our US and intertrial interval durations were shorter (paralleling those used in studies finding conditioned genital arousal in men) and our USs were more effective than theirs in inducing genital arousal. However, we did not directly test whether these variables were critical for effective conditioning. In fact, Both, Spiering, Laan, Belcome, Heuvel, and Everaerd (2008) and Both, Laan, Spiering, Nilsson, Oomens, and Everaerd (2008) have shown conditioned sexual arousal in women who reported that the US was moderately as opposed to highly sexually arousing (these latter studies are discussed more fully later in this chapter). Our original study was aimed at more than simply showing that women’s sexual arousal could be influenced by conditioning. We also tested gender differences by comparing males and females, as well as examined selective associations (e.g., whether particular CSs were more readily associated with sexual USs) and the role of conscious awareness in human sexual conditioning. We used a differential conditioning (CS+/CS–) design, an explicitly unpaired control group, and the same procedures to examine the conditioning of genital arousal in both women and men—using erotic film clips (of heterosexual interactions) that had been rated as arousing by both men and women (Janssen, Carpenter, & Graham, 2003) as USs. Domjan and Hollis (1988) proposed that males might show conditioned sexual arousal more readily, and to a wider range of cues, than females. It has also been suggested that women’s sexual arousal may not be as readily conditionable as men’s (Bancroft, 1989; Kinsey, Pomeroy, Martin, & Gebhard, 1953). However, Baumeister’s (2000) proposal that women are more erotically plastic (e.g., flexible in what they find sexually attractive) suggests that they could be more sensitive to conditioning than men.


537

We also explored the notion of biological preparedness (Garcia & Koelling, 1966; Seligman, 1970) since it appears that sexual incentives are not randomly distributed across the range of cues found in sexual situations. Stimuli associated with potential partners (e.g., feathers or more complete models of partners in birds) appear more effective than arbitrary ones in sexual conditioning in non-humans (Akins, 2000; Cusato & Domjan, 1998), and selective associations have been shown in human fear conditioning (e.g., Öhman, Esteves, & Soares, 1995). Unpublished studies examining cue-toconsequence specificity in sexual conditioning in men (Clement, 1989; De Gagne, 1988) yielded equivocal results. We proposed that a sexually relevant CS (photograph of an abdomen of the opposite sex) would be more effective as a conditionable cue than a sexually irrelevant CS (photograph of a gun). Finally we also manipulated the awareness of CS presentation because we assumed subjects would be less likely to alter their expression of arousal if they did not realize they were being conditioned. Specifically, we presented CSs either “subliminally” (i.e., for 30 msec followed immediately and hence backward masked by the film US) or “consciously” (i.e., for 10 sec). Many argue that humans do not learn classical associations unless they are aware of the CS-US contingency (Lovibond & Shanks, 2002); however, examples of Pavlovian learning without awareness in humans exists (e.g., Bechara et al., 1995; Clark & Squire, 1998; Esteves, Parra, Dimberg, & Öhman, 1994; Öhman et al., 1995; Öhman & Mineka, 2001; Öhman & Soares, 1994; Öhman & Soares, 1998; Soares & Öhman, 1993), including cases of such learning when evaluative conditioning paradigms are used (e.g., De Houwer et al, 1997; Dickinson & Brown, 2007; Walther & Nagengast, 2006; but see, e.g., Dawson, Rissling, Schell, & Wilcox, 2007; Pleyers, Corneille, Luminet, & Yzerbyt, 2007). Our results showed that when stimuli were presented outside the subjects’ awareness, both women and men showed conditioned genital arousal to the abdomen CS but not to the gun CS. These results are similar to those found by Öhman, Esteves, and Soares (1995) using a

2/16/2011 6:05:43 PM

538


fear-conditioning paradigm; that is, they found conditioned increases in skin conductance to fear-relevant but not fear-irrelevant stimuli that were presented outside the subject awareness and paired with a mild shock to the fingers. Even though our manipulation and measurement of awareness were rather crude, our results suggest a prepared link between sexually relevant stimuli and genital responses and support an independent role for automatic processing in sexual responding consistent with some models of sexual arousal (Janssen, Everaerd, Spiering, & Janssen, 2000). When consciously perceived CSs were used, however, men again showed conditioned increases in penile tumescence to the abdomen but not the gun CS, whereas women showed the opposite effect (i.e., conditioning to the gun but not the abdomen stimulus). The latter result was unexpected. Perhaps the gunarousal associations in women may have been facilitated by increased attention (Beylin & Shors, 1998; Shors & Matzel, 1997) or excitation transfer (Hoon, Wincze, & Hoon, 1977; Meston & Gorzalka, 1995; 1996; Meston & Heiman, 1998) as women (but not men) showed increased skin conductance responses to the gun (but not the abdomen) CS and a few women (anecdotally) reported that the picture of the gun made them slightly anxious. Few studies have concurrently examined sexual conditioning in men and women (also see Klucken et al., 2009) Across a range of studies detailed later, our work has shown that men are more readily and consistently conditionable; however, since our main measure of learning is physiological and we use different genital measures of arousal, it prevents a direct gender comparison. It could be that VPA is not as sensitive to conditioning effects as changes in penile circumference. A subsequent unpublished study conducted in our laboratory found that if the subjects were explicitly told that they were being conditioned they were more likely to show learning. In particular, men showed conditioned genital arousal to a cartoon sketch of a mason jar after it was paired with erotic films clips but only when they were aware of being conditioned, although there was a nonsignificant trend for learning in


unaware subjects. The effects appeared weaker in women, with only aware subjects showing a nonsignificant trend for learning. Perhaps awareness of the contingency may facilitate conditioning to sexually irrelevant CSs, but learning about sexually relevant stimuli may occur without such knowledge. Both, Spiering, Laan, Belcome, Heuvel, and Everaerd (2008) have provided stronger evidence of unconscious learning about (relevant) sexual stimuli in women. They paired an erotic photograph as the CS+ with clitoral vibrostimulation over 24 trials. A CS– (a different erotic photo) was also presented during acquisition. Both cues were displayed for 30 ms followed immediately by a 100 ms masking stimulus. A forced-choice recognition test verified lack of awareness of CS-US contingency. During training as well as the first nine extinction trials, women showed higher VPA to the CS+ versus the CS–. This differential conditioning was impressive considering the similarity between CS+ and CS– stimuli (i.e., both depicted heterosexual intercourse, differing only by actor and position). Both, Laan, Spiering, Nilsson, Oomens, and Everaerd (2008) showed another demonstration of appetitive sexual conditioning in women as well as providing the first demonstration of attenuation of sexual response by aversive conditioning in women. For appetitive conditioning they used a consciously displayed black and white drawing of a neutral male face as the CS+ paired with vibrogenital stimulation (and another face as the CS– presented during 10 conditioning trials. They observed increased VPA to the CS+ relative to the CS–. Furthermore, they tested for evaluative conditioning, finding a marginally significant preference for the CS+ over the CS– face. However, it is unclear if this represented an increase in preference for the CS+ or a decrease in preference for the CS–, or both. Unpublished results from our laboratory suggest that the affective value of both CSs can change (in opposite directions) during sexual conditioning (see later discussion). Nonetheless, Both et al. found genital and the evaluative CRs to be positively correlated. For aversive conditioning an erotic photograph was paired with a 50 ms wrist shock during aversive conditioning.

2/16/2011 6:05:43 PM


Decreased VPA to CS+ relative to the CS– during acquisition and extinction testing as well as lower affective preference for CS+ versus CS– were found. However even though subjects liked the CS+ picture less than the CS– picture, they found them equally sexually arousing (We also have not found changes in subjective arousal to the CS after conditioning). Evidence from Both and her colleagues as well as from our laboratory corroborate the existence of sexual conditioning in women. Yet, as with men, the learning has not been robust. Hence, we investigated whether changing the CS (to an olfactory cue) or the conditioning context (to the “real world” as opposed to the laboratory) could increase the strength of learning. Furthermore, we have begun to explore individual differences in sexual conditionability. Olfactory Conditioned Stimuli

Olfactory cues play a large role in sexual arousal in other mammals, and odors have been used as effective CSs in the conditioning of sexual arousal in a variety of non-human species (Domjan & Holloway, 1998; Pfaus et al., 2001). While noxious odor stimuli have been used as USs to decrease sexual arousal in clinical settings (e.g., Colson, 1972; Earls & Castonguay, 1989; Junginger, 1997), until recently olfactory stimuli have not been used as CSs in the conditioning of human sexual arousal. Studies indicate that smell plays a significant role in human sexual attraction. Herz and Cahill (1997) found that odor is an important guide for mate selection in women and men, and that women valued odor more so than men. Herz and Inzlicht (2002) showed that, for women, body odor was more important than looks (the reverse was true for men); and that, for women, smell was more valuable than all but one social factor (i.e., pleasantness). Because people report that smell is important in attraction and since the majority of fetishes are related to olfactory or tactile stimuli (Money, 1988), we reasoned that humans, and in particular women, may be likely to associate sexual arousal with odor cues. Hoffmann and Janssen (2006) used discrete odor cues as CSs and 30-sec erotic film clips as USs. Subjects received 28 pairings of either


539

lemon or strawberry odor delivered via an olfactometer with a different film clip over a 2-day period. A control group received explicitly unpaired presentations of the CS and US. Genital and subjective arousal to the CS, as well as preference for the odor that served as the CS, were assessed before and after training. Subsequently, Hoffmann (2007) used a similar procedure except that there were fewer conditioning trials (22), a randomized control as opposed to explicitly unpaired control group was used, and most important, the US was more primary (vibrotacile stimulation of the genitals; 30 sec per trial). Results for each of these studies were nearly identical. In both of these studies men showed appreciable learning. Although genital conditioned responses (CRs) were not robust, there was a significant difference in conditioned responding between the experimental and control groups and the CRs were comparable in strength to those obtained with visual CSs (e.g., Hoffmann et al., 2004; Lalumière & Quinsey, 1998). There was some suggestion that women also showed learning, but it was much less convincing. Neither men nor women reported a change in subjective arousal to the CS after conditioning; however, there was a nonsignificant trend for men to prefer the odor CS more after conditioning with the film clip US, whereas a trend for both men and women to show an increased preference for the odor CS after training occurred with the vibrogenital US. Hence, it appears that evaluative in addition to signal or predictive learning may occur in certain sexual conditioning situations. That is, in addition to showing increased genital responding to the CS+, participants showed an increased preference for the odor CS. Field Conditioning

Human sexual conditioning effects are less robust compared with those obtained in other animals. This may be because of the artificiality of the laboratory environment and/or because of the choice of US (watching erotic film clips as opposed to participating in sexual activity). Although a field study is not as controlled as a laboratory experiment, it offers the potential of a

2/16/2011 6:05:43 PM

540


more appropriate context for sexual arousal and a more effective US. Furthermore, affective learning may be stronger in real-world settings (Baeyens, Wrzesniewski, De Houwer, & Eelen, 1996; Öhman & Mineka, 2001; Rozin, Wrzesniewski, & Byrnes, 1998). We predicted that enhanced sexual conditioning would occur outside the laboratory. Heterosexual couples were instructed to include a novel scent during sexual interaction (CS+) and another novel scent during nonsexual interaction (CS–). Control couples used both scents during nonsexual interaction. Conducted over a 2-week period, both experimental and control couples had three sexual interactions (oral sex and/or intercourse) during this time. In addition, experimental couples had three, while the controls had six, nonsexual interactions (e.g., studying together, watching a movie, playing video games) that also involved the presence of a novel (CS–) odor. Genital responding to and affective preference for the odors were assessed in the laboratory before and after the experience in the men. We only tested men for several reasons. Based on our previous studies men were more likely to show conditioning, women are less likely to agree to use the genital monitors, and we needed the assistance of one of the partners to “conduct the conditioning”; hence, men knew they were in a study on odors and sexual arousal but did not know they were being conditioned. We found evidence of conditioning using both the genital and affective measures. We observed significantly increased genital responding to the CS+ in the experimental group relative to the control group; however, CRs were not much stronger than those obtained in the laboratory conditioning. One reason for the relatively weak conditioning (the conditioning in the laboratory, as mentioned earlier, was also weak) may have been that the males, although instructed to contact the experimenter as soon as they finished the study, did not return to the laboratory until several (as many as 11) days after completion. However there was no correlation between the length of the “retention interval” and the strength of the CR, but our sample was small (n = 7 in each group). Affective learning, that is, the increased preference for the CS+


odor and decreased preference for the CS– odor, was evident and stronger than in laboratory studies. However, we used more neutrally rated odors as CS in the field study, potentially increasing the sensitivity of affective preference test. It was interesting to see learning about the CS– odor, and in fact the decreased preference for the CS– was greater than the increased preference for the CS+, although we did not explicitly test whether the CS– became a conditioned inhibitor. We have seen a trend for this effect in other studies, and the demonstration of conditioned inhibition in a sexual situation has potential for enhancing learning-based therapies to be used to alter problematic sexual responding. Individual Differences

Individual differences in classical conditionability exist in humans (e.g., Martin, 1997). Certain types of people appear more susceptible to expectancy conditioning (e.g., Cook, Hodes, & Lang, 1986; Hamm & Vaitl, 1996; Hodes, Cook, & Lang, 1985; Kvale, Psychol, & Hugdahl, 1994; Mineka & Zinbarg, 2006) as well as evaluative conditioning (Baeyens et al., 1996; Yeomans, Mobini, Elliman, Walker, & Stevenson, 2006). A few studies report individual differences in sexual conditionability. In an early study Kantorowitz (1978) showed a significant correlation between extraversion and conditioning of preorgasmic sexual arousal and a significant correlation between introversion and postorgasmic sexual arousal, indicating that certain personality types are more susceptible to sexual conditioning. Our studies have found that conditionability did not appear to be related to the amount of experience subjects had with erotic film; however, there was not much variation among our participants on this amount of experience. We also collected survey data from participants in some of our studies and correlated it with the strength of the CR. Specifically, we measured introversion/extraversion and Sexual Sensation Seeking (SSS; Kalichman et al. 1994), and we administered the Sexual Experience Scales (SES) subscale for Psychosexual Stimulation (Frenken, 1981) and the Sexual Inhibition

2/16/2011 6:05:43 PM


Scales/Sexual Excitation Scales (SIS/SES) (Janssen, Vorst, Finn, & Bancroft, 2002). The SES measures the extent that someone seeks sexual stimuli of an auditory-visual or imaginary kind and the SISSES measures sexual excitation (SE) or the propensity for sexual arousal and two forms of sexual inhibition (SIS-1 and SIS-2). SIS-1 is inhibition related to performance failure and SIS-2 is inhibition related to other negative consequences. For men we have not found a clear relationship between the strength of the CR and our survey measures, yet our male participants appeared to be a somewhat homogeneous group. For women we found that the strength of the CR tended to be inversely related (near significant correlation coefficients) to their scores on both inhibition scales (SIS-1 and SIS-2), so women who are low in sexual inhibition may be more likely to show conditioned sexual arousal. Both et al. (2008) found that women higher in sexual functioning showed stronger genital CRs and those higher in sexual arousability showed stronger genital along with affective CRs. Perhaps women who are more comfortable with sexuality are more conditionable. Exploring individual differences in sexual conditionablity may yield insights into sexual problems such as sexual dysfunction and the development of paraphilias, for example, fetishism. We recently have begun to consider the role of conditioning and conditionablity in sexual risk taking and sexual compulsion. Application to Sexual Risk Taking and Sexual Compulsion

The term sexual compulsivity refers to a preoccupation with and/or engagement in a range of sexual behaviors (e.g., masturbation, partnered sex, use of erotic/pornographic imagery) that are experienced as being excessive or out of control and that may have a negative impact on physical, emotional, and behavioral functioning of the individual. It has been estimated, although not studied in nationally representative samples, to occur in up to 10% of the population. It appears to be more common in men, and even more common in men who have sex with men (MSM; Black, Gates, Sanders, & Taylor, 2000; Black,


541

Kehrberg, Flumerfelt, & Schlosser, 1997; Parsons et al., 2008). While a number of hypotheses exist on the origins of sexual compulsivity, little empirical research has examined its etiology. Sexual compulsivity and other forms of “out-ofcontrol” behaviors tend to be subjectively experienced as involving uncontrollable internal desires or “urges”; however, most models of sexual response and sexual motivation emphasize the role of external cues as triggers for sexual arousal and associated behavior (Barlow, 1986; Hardy, 1964; Pfaus, 1999; Toates, 2009; Whalen, 1966). Hence, sexual behavior is not seen as primarily “pushed” from within but rather starts with and depends on the effectiveness of sexual cues and incentives. Parsons, Kelly, Bimbi, Muench, and Morgenstern (2007) interviewed MSM who reported experiencing compulsive sexual behavior about the triggers for such behavior. They reported two main types: (1) event triggers such as relationship turmoil and other catastrophes and (2) contextual triggers such as particular locations, past partners, substance use, and erotic material. The acknowledgment that external cues can precipitate or activate sexual compulsivity suggests that learning and in particular conditioning may play a role in the etiology and maintenance of out-of-control sexual behavior (Goodman, 1998; Parsons et al., 2007; Putnam, 2000). Thus, a way to conceptualize compulsive sexual behavior that leads to testable hypotheses on etiology involves the idea that the opposing tendencies of appetitive (sexual) motivation and restraint are not balanced to yield appropriate (safe) sexual choices in certain individuals and/ or under certain conditions (Orford, 2001). Specifically, people who engage in compulsive sexual behaviors may be, or may have learned to be, more responsive to sexual stimuli (e.g., show stronger excitatory responses) and/or they may be less able to control their response to such cues (e.g., show weaker regulatory or inhibitory responses). I am particularly interested in whether sexually compulsive individuals could be more responsive to sexual cues because they are more (sexually) conditionable. Support for the idea that classical conditioning is involved in compulsive behavior can be

2/16/2011 6:05:43 PM

542


found in the drug addiction literature. Incentive sensitization theory is an experimentally validated neurobiological theory of addiction (Berridge & Robinson, 2003; Robinson & Berridge, 1993, 2000, 2003) implicating classical conditioning as the main mechanism by which stimuli paired with abused drugs acquire incentive properties. This theory proposes that drugs of abuse can produce long-lasting changes in the mesolimbic dopamine (i.e., brain reward) system. Once a drug (US) is predicted by environmental cues (CSs), increased dopamine in the reward circuit does not occur to the drug itself but rather to its predictive cues (Schultz, Dayan, & Montague, 1997). More specifically, drug-induced changes in the reward system contribute to sensitization or a heightened responsiveness to the drug itself as well as to the stimuli associated with drug use. Drugpaired cues are said to become “motivational magnets” for inducing drug-seeking behavior. The phenomenon of sign tracking also links classical conditioning to compulsive behavior. Sign tracking involves continued approach to a CS that previously predicted an appetitive (e.g., food or water) US, even when such a response delays or eliminates presentation of the appetitive US (e.g., Brown & Jenkins, 1968; Costa & Broakes, 2007; Davey & Cleland, 1982; Jenkins & Moore, 1973; Peterson, Ackil, Frommer, & Hearst, 1972; Williams & Williams, 1969). Signtracking behavior is seen as being maladaptive, and it has been incorporated in models of addiction (Luenberger, 1979; Newlin, 2002; Tomie, Grimes, & Pohorecky, 2008). Recent evidence from animal studies shows that sign tracking occurs with ethanol- (Cunningham & Patel, 2007) and cocaine- (Uslaner, Acerbo, Jones, & Robinson, 2006) related cues. Furthermore, Flagel, Watson, Robinson, and Akil (2007) found individual differences in the propensity for sign tracking and proposed that such differences may be due to differences in mesolimbic reward circuitry. Although sexual behavior does not involve “exogenous” alteration of the mesolimbic dopamine system (as is the case with drug use), sexual compulsivity is often associated with substance abuse and has been conceptualized as an


addiction itself. Furthermore, there is good evidence that the mesolimbic dopamine system is involved with sexual reward (Balfour, Yu, & Coolen, 2004; Bradley & Meisel, 2001; Fiorino, Coury & Phillips, 1997; Hull, Du, Lorrian, & Matuszewich, 1995; Kohlert & Meisel, 1999; Mermelstein & Becker, 1995; Parades & Ågmo, 2004; Pfaus, Damsma, Wenkstern, & Fibigar, 1995), and cross-sensitization between drug and sexual reward has been demonstrated (Levens & Akins, 2004; Mitchell & Stewart, 1990; Nocjar & Panksepp, 2002). Sexual compulsion could result from stronger arousal and/or reward system responding to sexual cues and/or from being aroused/motivated by a wider range of cues, either of which could derive from being more sexually conditionable. It is also possible that in sexually compulsive individuals conditioning processes lead to changes in sexual cue valence (evaluative conditioning), possibly leading to stronger and more enduring conditioned sexual responding. A few studies provide evidence for a link between sexual conditioning and compulsive behavior in the form of sexual sign tracking. Burns and Domjan (1996, 2000) found that male Japanese quail would approach and remain close to a wooden block CS that predicted copulatory access to a female, even when the block was located a distance from the goal door in which the female would appear. In addition, Kimura, Fukui, and Inaki (1990) also showed that men and women came to fixate on and press a button, whose lighting preceded the showing of an erotic film clip, despite the fact that such behavior was unrelated to presentation of the erotic US. We are currently examining the role of sexual conditionability in sexual risk taking and sexual compulsion.

CONCLUSION Although it was and perhaps still is commonly assumed that classical conditioning plays a role in what we find sexually arousing, particularly in cases of deviant arousal patterns such as fetishes, a strict learning interpretation of the development of sexual preferences has fallen out of favor. Yet with modern developments in learning theory it seems appropriate to renew the

2/16/2011 6:05:43 PM


investigation of contributions and limitations of conditioning processes in explaining how cues acquire erotic meaning. Further research into human sexual conditioning and the relative contributions of expectancy versus evaluative conditioning may contribute to the literatures on human learning theory and in particular evaluative conditioning. Moreover, such research may help us to better understanding the impact that erotic stimuli have on sexual arousal and subsequent behavior, potentially allowing us to alter such responses to improve sexual functioning. Such information could have direct application to managing sexual risk taking, sexual compulsion, and paraphilic (e.g., fetishistic) behavior as well as in attempting to restore sexual functioning after rape or other sexual trauma.

REFERENCES Adkins-Regan, E., & MacKillop, E. A. (2003). Japanese quail (Coturnix japonica) inseminations are more likely to fertilize eggs in a context predicting mating opportunities. Proceedings of the Royal Society of London B: Biological Sciences, 270, 1685–1689. Ågmo, A. (1999). Sexual motivation—an inquiry into events determining the occurrence of sexual behavior. Behavioural Brain Research, 105, 129–150. Akins, C. K. (2000). Effects of species specific cues and the CS-US interval on the topography of the sexually conditioned response. Learning and Motivation, 31, 211–235. Akins, C. K. (2004). The role of Pavlovian conditioning in sexual behavior: A comparative analysis of human and nonhuman animals. International Journal of Comparative Psychology, 17, 241–262. Baccus, J. R., Baldwin, M. W., & Packer, D. J. (2004). Increasing implicit self-esteem through classical conditioning. Psychological Science, 15, 498–502. Baeyens, F., Crombez, G., De Houwer, J., & Eelen, P. (1996). No evidence for modulation of evaluative flavor-flavor associations in humans. Learning and Motivation, 27, 200–241. Baeyens, F., Crombez, G., Van den Bergh, O., & Eelen, P. (1988). Once in contact always in contact: Evaluative conditioning is resistant to


543

extinction. Advances in Behaviour Research and Therapy, 10, 179–199. Baeyens, F., Hendrickx, H., Crombez, G., & Hermans, D. (1998). Neither extended sequential nor simultaneous feature positive training result in modulation of evaluative flavor-flavor conditioning in humans. Appetite, 31, 185–204. Baeyens, F., Vansteenwegen, D., Hermans, D., & Eelen, P. (2001). Chilled white wine, when all of a sudden the doorbell rings: Mere reference and evaluation versus expectancy and preparation in human Pavlovian learning. In F. Columbus (Ed.), Advances in psychology research (Vol. 4, pp. 241–277). Hauppauge, NY: Nova Science Publishers. Baeyens, F., Wrzesniewski, A., De Houwer, J., & Eelen, P. (1996). Toilet rooms, body massages, and smells: Two field studies on human evaluative odor conditioning. Current Psychology, 15, 77–96. Balfour, M. E., Yu, L., & Coolen, L. M. (2004). Sexual behavior and sex-associated environmental cues activate the mesolimbic system in male rats. Neuropsychopharmacology, 29, 718–730. Bancroft, J. (1989). Human sexuality and its problems (2nd ed.). Edinburgh, Scotland: Churchill Livingstone. Barlow, D. H. (1986). Causes of sexual dysfunction: The role of anxiety and cognitive interference. Journal of Consulting and Clinical Psychology, 54, 140–148. Barlow, D. H., Becker, R., Leitenberg, H., & Agras, W. S. (1970). A mechanical strain gauge for recording penile circumference change. Journal of Applied Behavioral Analysis, 3, 73–76. Bateson, P. P. G. (1978). Sexual imprinting and optimal outbreeding. Nature, 273, 659–660. Baumeister, R. (2000). Gender differences in erotic plasticity: The female sex drive as socially flexible and responsive. Psychological Bulletin, 126, 347–374. Bechara, A., Tranel, D., Damasio, H., Adolphs, R., Rockland, C., & Damasio, A. R. (1995). Double dissociation of conditioning and declarative knowledge relative to the amygdale and hippocampus in humans. Science, 269, 1115–1118. Berridge, K. C., & Robinson, T. E. (2003). Passing reward. Trends in Neurosciences, 26, 507–513. Beylin, A. V., & Shors, T. J. (1998). Stress enhances excitatory trace eyeblink conditioning and opposes acquisition of inhibitory conditioning. Behavioral Neuroscience, 112, 1327–1338.

2/16/2011 6:05:43 PM

544


Binet, A. (1888). Etudes de psychologie experimentale. Paris, France: Octave Doin. Black, D., Gates, G., Sanders, S., & Taylor, L. (2000). Demographics of the gay and lesbian population in the United States: Evidence from available systematic data sources. Demography, 37, 139–154. Black, D. W., Kehrberg, L. L. D., Flumerfelt, D. L., & Schlosser, S. S. (1997). Characteristics of 36 subjects reporting compulsive sexual behavior. American Journal of Psychiatry, 154, 243–249. Bogaert, A. F. (1996). Volunteer bias in human sexuality research: Evidence for both sexuality and personality differences in males. Archives of Sexual Behavior, 25, 125–140. Both, S., Laan, E., Spiering, M., Nilsson, T., Oomens, S., & Everaerd, W. (2008). Appetitive and aversive classical conditioning of female sexual response. Journal of Sexual Medicine, 5, 1386–1401. Both, S., Spiering, M., Laan, E., Belcome, S., Van den Heuvel, B., & Everaerd, W. (2008). Unconscious classical conditioning of sexual arousal: Evidence for the conditioning of female genital arousal to subliminally presented sexual stimuli. Journal of Sexual Medicine, 5, 100–109. Bradley, K. C., & Meisel, R. L. (2001). Sexual behavior induction of c-Fos in the nucleus accumbens and amphetamine-stimulated locomotor activity are sensitized by previous sexual experience in female Syrian hamsters. Journal of Neuroscience, 21, 2123–2130. Brown, P. L., & Jenkins, H. M. (1968). Auto-shaping the pigeon’s key peck. Journal of the Experimental Analysis of Behavior, 11, 1–8. Burns, M., & Domjan, M. (1996). Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica). Journal of Experimental Psychology: Animal Behavior Processes, 22, 297–306. Burns, M., & Domjan, M. (2000). Sign tracking in domesticated quail with one trial a day: Generality across CS and US parameters. Animal Learning and Behavior, 28, 109–119. Clark, R. E., & Squire, L. R. (1998). Classical conditioning and brain systems: The role of awareneness. Science, 280, 77–81. Clement, D. (1989). Le conditionnement classique et la réponse sexuelle masculine. Unpublished master’s thesis, Univeriste de Montreal, Montreal, Montreal, Quebec.


Colson, C. E. (1972). Olfactory aversion therapy for homosexual behavior. Journal of Behavior Therapy and Experimental Psychiatry, 3, 185–187. Cook, E. W., Hodes, R. L., & Lang, P. J. (1986). Preparedness and phobia: Effects of stimulus content on human visceral conditioning. Journal of Abnormal Psychology, 95, 195–207. Coria-Avila, G. A., Jones, S. L., Solomon, C. E., Gavrila, A. M., Jordan, G. J., & Pfaus, J. G. (2006). Conditioned partner preference in female rats for strain of male. Physiology and Behavior, 88, 529–537. Coria-Avila, G. A., Ouimet, A. J., Pachero, P., Manzo, J., & Pfaus, J. G. (2005). Olfactory conditioned partner preference in the female rat. Behavioral Neuroscience, 119, 716–725. Costa, D. S. J., & Broakes, R. A. (2007). Maintenance of responding when reinforcement becomes delayed. Learning and Behavior, 35, 95–105. Craig, W. (1918). Appetites and aversions as constituents of instincts. Biological Bulletin, 34, 91–107. Cunningham, C. L., & Patel, P. (2007). Rapid induction of Pavlovian approach to an ethanolpaired visual cue in mice. Psychopharmacology, 192, 231–241. Cusato, B., & Domjan, M. (1998). Special efficacy of sexual conditioned stimuli that include species typical cues: Tests with a conditioned stimuli preexposure design. Learning and Motivation, 29, 152–167. Davey, G. C. L., & Cleland, G. G. (1982). Topography of signal-centered behavior in the rat: Effects of deprivation state and reinforcer type. Journal of the Experimental Analysis of Behavior, 38, 291–304. Dawson, M. E., Rissling, A. J., Schell, A. M., & Wilcox, R. (2007). Under what conditions can human affective conditioning occur without contingency awareness? Test of the evaluative conditioning paradigm. Emotion, 7, 755–766. De Gagne, R. (1988). Le conditionnement classique et la réponse sexualle humaine. Unpublished manuscript, Universite de Montreal, Montreal, Montreal, Quebec. De Houwer, J., Baeyens, F., & Field, A. P. (2005). Associative learning of likes and dislikes: Some current controversies and possible ways forward. Cognition and Emotion, 19, 161–174. De Houwer, J., Hendrickx, H., & Baeyens, F. (1997). Evaluative conditioning with “subliminally”

2/16/2011 6:05:43 PM


presented stimuli. Consciousness and Cognition, 6, 87–107. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127, 853–869. Dewsbury, D. A. (1981). Effects of novelty of copulatory behavior: The Coolidge effect and related phenomena. Psychological Bulletin, 89, 464–482. Dickinson, A., & Brown, K. J. (2007). Flavorevaluative conditioning is unaffected by contingency knowledge during training with color-flavor compounds. Learning and Behavior, 35, 36–42. Dijksterhuis, A. (2004). I like myself but I don’t know why: Enhancing implicit self-esteem by subliminal evaluative conditioning. Journal of Personality and Social Psychology, 86, 345–355. Domjan, M. (2005). Pavlovian conditioning: A functional perspective. Annual Review of Psychology, 56, 179–206. Domjan, M., Blesbois, E., & Williams, J. (1998). The adaptive significance of sexual conditioning: Pavlovian control of sperm release. Psychological Science, 9, 411–415. Domjan, M., & Hollis, K. L. (1988). Reproductive behavior: A potential model system for adaptive specializations in learning. In R. C. Bolles & M. D. Beecher (Eds.) Evolution and learning (pp. 213–232). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Domjan, M., & Holloway, K. S. (1998). Sexual learning. In G. Greenberg & M. M. Haraway (Eds.), Comparative psychology: A handbook (pp. 602–613). New York, NY: Garland. Earls, C. M., & Castonguay, L. G. (1989). The evaluation of olfactory aversion for a bisexual pedophile with a single-case multiple baseline design. Behavior Therapy, 20, 137–146. Esteves, F., Parra, C., Dimberg, U., & Öhman, A. (1994). Nonconscious associative learning: Pavlovian conditioning of skin conductance responses to masked fear-relevance facial stimuli. Psychophysiology, 31, 375–385. Field, A. P. (2005). Learning to like (or dislike): Associative learning of preferences. In A. J. Wills (Ed.), New directions in human associative learning (pp. 221–252). Mahwah, NJ: Lawrence Erlbaum Associates Publishers. Fiorino, D. F., Coury, A., & Phillips, A. G. (1997). Dynamic changes in nucleus accumbens


545

dopamine efflux during the Coolidge effect in male rats. Journal of Neuroscience, 17, 4849–4855. Flagel, S. B., Watson, S. J., Robinson, T. E., & Akil, H. (2007). Individual differences in the propensity to approach signals vs. goals promote different adaptations in the dopamine system of rats. Psychopharmacology, 191, 599–607. Frenken, J. (1981). Sexual experience scale manual. Zeitst, Netherlands: Swets & Zeitlinger BV. Gaither, G. A., Rosenkranz, R. R., & Plaud, J. J. (1998). Sexual disorders. In J. J. Plaud & G. H. Eifert (Eds.) From behavior theory to behavior therapy (pp. 152–177). Needham Heights, MA: Allyn & Bacon. Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Gavin, A., Roche, B., & Ruiz, M. R. (2008). Competing contingencies over derived relational responding: A behavioral model of the implicit association test. Psychological Record, 58, 427–441. Gawronski, B., & Bodenhausen, G. V. (2006). Associative and propositional processes in evaluation: Conceptual, empirical, and metatheoretical issues. Psychological Bulletin, 132, 745–750. Geer, J. H., & Janssen, E. (2000). The sexual response system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed., pp. 315–341). New York, NY: Cambridge University Press. Graham, J. M., & Desjardins, C. (1980). Classical conditioning: Induction of luteinizing hormone and testosterone secretion in anticipation of sexual activity. Science, 210, 1039–1041. Gutiérrez, G., & Domjan, M. (1997). Differences in the sexual conditioned behavior of male and female Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 111, 135–142. Goodman, A. (1998). Sexual addiction: An integrated approach. Madison, CT: International Universities Press, Inc. Hamm, A. O., & Vaitl, D. (1996). Affective learning: Awareness and aversion. Psychophysiology, 33, 698–710. Hardy, K. R. (1964). An appetitional theory of sexual motivation. Psychological Review, 71, 1–18. Hardwick, S. A., & Lipp, O. V. (2000). Modulation of affective learning: An occasion for evaluative

2/16/2011 6:05:43 PM

546


conditioning? Learning and Motivation, 31, 251–271. Hermans, D., Crombez, G., Vansteenwegen, D., Baeyens, F., & Eelen, P. (2002). Expectancylearning and evaluative learning in human classical conditioning: Differential effects of extinction. In S. P. Shohov (Ed.), Advances in psychology research (Vol. 12, pp. 17–40). Hauppauge, NY: Nova Science Publishers. Hermans, D., Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (2002). Expectancy learning and evaluative learning in human classical conditioning: Affective priming as an indirect and unobtrusive measure of conditioned stimulus valence. Behaviour Research and Therapy, 40, 217–234. Herz, R. S., & Cahill, E. D. (1997). Differential use of sensory information in sexual behavior as a function of gender. Human Nature, 8, 275–286. Herz, R. S., & Inzlicht, M. (2002). Sex differences in response to physical and social factors involved in human mate selection: The importance of smell for women. Evolution and Human Behavior, 23, 359–364. Hilliard, S., & Domjan, M. (1995). Effects of sexual conditioning of devaluing the US through satiation. The Quarterly Journal of Experimental Psychology B: Comparative and Physiological Psychology, 48, 84–92. Hilliard, S., Domjan, M., Nguyen, M., & Cusato, B. (1998). Dissociation of conditioned appetitive and consummatory sexual behavior: Satiation and extinction tests. Animal Learning and Behavior, 26, 20–33. Hodes, R. L., Cook, E. W., & Lang, P. J. (1985). Individual differences in autonomic response: Conditioned association or conditioned fear? Psychophysiology, 22, 545–560. Hoffmann, H. (2007, August). Classical conditioning of sexual arousal in men and women with an olfactory CS and vibrotactile US: Evidence for signal and affective learning. Paper presented at the meeting of the International Academy of Sex Research, Vancouver, Canada. Hoffmann, H., & Janssen, E. (2006, July). Classical conditioning of sexual arousal to an olfactory cue in women and men: Who learns and what is learned? Paper presented at the meeting of the International Academy of Sex Research, Amsterdam, Netherlands. Hoffmann, H., Janssen, E., & Turner, S. (2004). Classical conditioning of sexual arousal in


women and men: Effects of varying awareness and biological relevance of the conditioned stimulus. Archives of Sexual Behavior, 33, 43–53. Hollis, K. L. (1997). Contemporary research on Pavlovian conditioning: A “new” functional analysis. American Psychologist, 52, 956–965. Hollis, K. L., Pharr, V. L., Dumas, M. J., Britton, G. B., & Field, J. (1997). Classical conditioning provides paternity advantage for territorial male blue gouramis (Trichogaster trichopterus). Journal of Comparative Psychology, 111, 219–225. Holloway, K. S., & Domjan, M. (1993). Sexual approach conditioning: Tests of unconditioned stimulus devaluation using hormone manipulations. Journal of Experimental Psychology: Animal Behavior Processes, 19, 47–55. Hoon, P. W., Wincze, J. P., & Hoon, E. F. (1976). Physiological assessment of sexual arousal in women. Psychophysiology, 13, 196–204. Hoon, P. W., Wincze, J. P., & Hoon, E. F. (1977). A test of reciprocal inhibition: Are anxiety and sexual arousal in women mutually inhibitory? Journal of Abnormal Psychology, 86, 65–67. Hull, E. M., Du, J., Lorrian, D. S., & Matuszewich, L. (1995). Extracellular dopamine in the medial preoptic area: Implications for sexual motivation and hormonal control of copulation. Journal of Neuroscience, 15, 7465–7471. Janssen, E., Carpenter, D., & Graham, C. A. (2003). Selecting films for sex research: Gender differences in erotic film preference. Archives of Sexual Behavior, 32, 243–251. Janssen, E., Everaerd, W., Spiering, M., & Janssen, J. (2000). Automatic processes and the appraisal of sexual stimuli: Towards an information processing model of sexual arousal. Journal of Sex Research, 37, 8–23. Janssen, E., Vorst, H., Finn, P., & Bancroft, J. (2002). The Sexual Inhibition (SIS) and Sexual Excitation (SES) Scales: I. Measuring sexual inhibition and excitation proneness in men. Journal of Sex Research, 39, 114–126. Jenkins, H. M., & Moore, B. R. (1973). The form of the auto-shaped response with food or water reinforcers. Journal of the Experimental Analysis of Behavior, 20, 163–181. Junginger, J. (1997). Fetishism: Assessment and treatment. In D. R. Laws & W. O’Donohue (Eds.), Sexual deviance: Theory, assessment and treatment (pp. 92–110). New York, NY: Guilford Press.

2/16/2011 6:05:43 PM


Kalichman, S. C., Johnson, J. R., Adair, V., Rompa, D., Multhauf, K., & Kelly, J. A. (1994). Sexual sensation seeking: Scale development and predicting AIDS-risk behavior among homosexually active men. Journal of Personality Assessment, 62, 385–397. Kantorowitz, D. A. (1978). Personality and conditioning of tumescence and detumescence. Behaviour Research and Therapy, 16, 117–123. Karpinski, A., & Hilton, J. L. (2001). Attitudes and the implicit association test. Journal of Personality and Social Psychology, 81, 774–788. Kendrick, K. M., Hinton, M. R., Atkins, K., Haupt, M. A., & Skinner, J. D. (1998). Mothers determine sexual preference. Nature, 395, 229–230. Kimura, H., Fukui, I., & Inaki, K. (1990). Autoshaping of a button-push response and eye movement in human subjects. Japanese Journal of Psychology, 61, 351–355. Kinsey, A. C., Pomeroy, W. B., Martin, C. E., & Gebhard, P. H. (1953). Sexual behavior in the human female. Oxford, England: Saunders. Kippin, T. E., Talianakis, S., & Pfaus, J. G. (1997). The role of ejaculation in the development of conditioned sexual behaviors in the male rat. Social Behavioral Neuroendocrinology Abstracts, 1, 38. Kippin, T. E., Talianakis, S., Schattmann, L., Bartholomew, S., & Pfaus, J. G. (1998). Olfactory conditioning of sexual behavior in the male rat (Rattus norvegicus). Journal of Comparative Psychology, 112, 389–399. Köksal, F., & Domjan, M. (1998). Observational conditioning of sexual behavior in the domesticated quail. Animal Learning and Behavior, 26, 427–432. Kohlert, J. G., & Meisel, R. L. (1999). Sexual experience sensitizes mating-related nucleus accumbens dopamine response of female Syrian hamsters. Behavioural Brain Research, 99, 45–52. Kohlert, J. G., & Olexa, N. (2005). The role of vaginal stimulation for the acquisition of conditioned place preference in female Syrian hamsters. Physiology and Behavior, 84, 135–139. Kvale, G., Psychol, C., & Hugdahl, K. (1994). Cardiovascular conditioning and anticipatory nausea and vomiting in cancer patients. Behavioral Medicine, 20, 78–85. Laan, E., Everaerd, W., & Evers, A. (1995). Assessment of female sexual arousal: Response


547

specificityandconstructvalidity.Psychophysiology, 32, 476–485. Lalumière, M. L., & Quinsey, V. L. (1998). Pavlovian conditioning of sexual interests in human males. Archives of Sexual Behavior, 27, 241–252. Langevin, R., & Martin, M. (1975). Can erotic response be classically conditioned? Behavioral Therapy, 6, 350–355. Laws, D. R., & Marshall, W. L. (1991). Masturbatory reconditioning with sexual deviates: An evaluative review. Advances in Behaviour Research and Therapy, 13, 13–25. Letourneau, E. J., & O’Donohue, W. (1997). Classical conditioning of female sexual arousal. Archives of Sexual Behavior, 26, 63–78. Levens, N., & Akins, C. K. (2004). Chronic cocaine pretreatment facilitates Pavlovian sexual conditioning in male Japanese quail. Pharmacology, Biochemistry and Behavior, 79, 451–457. Levin, R. J. (1998). Assessing human female sexual arousal by vaginal photoplethysmography: A critical examination. Sexologies, 6, 26–31. Lipp, O. V., & Purkis, H. M. (2005). No support for dual process accounts of human affective learning in simple Pavlovian conditioning. Cognition and Emotion, 19, 269–282. Lipp, O. V., & Purkis, H. M. (2006). The effects of assessment type on verbal ratings of conditional stimulus valence and contingency judgments: Implications for the extinction of evaluative learning. Journal of Experimental Psychology: Animal Behavior Processes, 32, 431–440. Lisk, R. D., & Baron, G. (1982). Female regulation of mating location and acceptance of new mating partners following mating to sexual satiety: The Coolidge effect demonstrated in the female golden hamster. Behavioral and Neural Biology, 46, 416–421. Livingston, R. W., & Drwecki, B. B. (2007). Why are some individuals not racially biased? Susceptibility to affective conditioning predicts nonprejudice towards Blacks. Psychological Science, 18, 816–823. Lovibond, P. F., & Shanks, D. R. (2002). The role of awareness in Pavlovian conditioning: Empirical evidence and theoretical implications. Journal of Experimental Psychology: Animal Behavior Processes, 28, 3–26. Luenberger, D. G. (1979). Introduction to dynamic systems: Theory, models, and applications. New York, NY: Wiley. Mahometa, M. J., & Domjan, M. (2005). Classical conditioning increases reproductive success in

2/16/2011 6:05:43 PM

548


Japanese quail (Coturnix japonica). Animal Behavior, 69, 983–989. Martin, I. (1997). Classical conditioning and the role of personality. In H. Nyborg (Ed.), The scientific study of human nature: Tribute to Hans J. Eysenck at eighty (pp. 339–363). Amsterdam, Netherlands: Pergamon/Elsevier Sciences, Inc. Matthews, R. N., Domjan, M., Ramsey, M., & Crews, D. (2007). Learning effects on sperm competition and reproductive fitness. Psychological Science, 18, 758–762. McConaghy, N. (1987). A learning approach. In J. Geer & W. O’Donohue (Eds.), Theories of human sexuality (pp. 287–334). New York, NY: Plenum Press. McGuire, R. J., Carlisle, J. M., & Young, B. G. (1965). Sexual deviations as conditioned behaviour: A hypothesis. Behavioral Research and Therapy, 3, 185–190. Meisel, R. L., & Joppa, M.A. (1994). Conditioned place preference in female hamsters following aggressive or sexual encounters. Physiology and Behavior, 56, 1115–1118. Meston, C. M., & Gorzalka, B. B. (1995). The effect of sympathetic activation on physiological and subjective sexual arousal in women. Behaviour Research and Therapy, 33, 651–664. Meston, C. M., & Gorzalka, B. B. (1996). The effects of immediate, delayed, and residual sympathetic activation on sexual arousal in women. Behaviour Research and Therapy, 34, 143–148. Meston, C. M., & Heiman, J. R. (1998). Ephedrineactivated physiological arousal in women. Archives of General Psychiatry, 55, 652–656. Mermelstein, P. G., & Becker, J. B. (1995). Increased extracellular dopamine in the nucleus accumbens and striatum of the female rat during paced copulatory behavior. Behavioral Neuroscience, 109, 354–365. Mineka, S., & Zinbarg, R. (2006). A contemporary learning theory perspective on the etiology of anxiety disorders: It’s not what you thought it was. American Psychologist, 61, 10–26. Mitchell, J. B., & Stewart, J. (1990). Facilitation of sexual behaviors in the male rat in the presence of stimuli previously paired with systemic injections of morphine. Pharmacology, Biochemistry and Behavior, 35, 367–372. Money, J. (1988). Gay, straight and in-between: The sexology of erotic orientation. New York, NY: Oxford University Press.


Morokoff, P. J. (1986). Volunteer bias in the psychophysiological study of female sexuality. Journal of Sex Research, 22, 35–51. Newlin, D. B. (2002). The self-perceived survival ability and reproductive fitness (SPFit) theory of substance use disorders. Addiction, 97, 427–445. Neuman, R., Forster, J., & Strack, F. (2003). Motor compatibility: The bidirectional link between behavior and evaluation. In J. Musch & K. Klaver (Eds.), The psychology of evaluation: Affective processes in cognition and emotion. Mahwah, NJ: Lawrence Erlbaum Associates. Nocjar, C., & Panksepp, J. (2002). Chronic intermittent amphetamine pretreatment enhances future appetitive behavior for drug- and natural-reward: Interaction with environmental variables. Behavioural Brain Research, 128, 189–203. O’Donohue, W., & Plaud, J. J. (1994). The conditioning of human sexual arousal. Archives of Sexual Behavior, 23, 321–344. Öhman, A., Esteves, F., & Soares, J. J. F. (1995). Preparedness and preattentive associative learning: Electrodermal conditioning to masked stimuli. Journal of Psychophysiology, 9, 99–108. Öhman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychological Review, 108, 483–522. Öhman, A., & Soares, J. J. F. (1994). “Unconscious anxiety”: Phobic responses to masked stimuli. Journal of Abnormal Psychology, 103, 231–240. Öhman, A., & Soares, J. J. F. (1998). Emotional conditioning to masked stimuli: Expectancies for aversive outcomes following nonrecognized fear-relevant stimuli. Journal of Experimental Psychology: General, 127, 69–82. Olson, M. A., & Fazio, R. H. (2001). Implicit attitude formation through classical conditioning. Psychological Science, 12, 413–417. Orford, J. (2001). Addiction as excessive appetite. Addiction, 96, 15–31. Paredes, R. G., & Ågmo, A. (2004). Has dopamine a physiological role in the control of sexual behavior? A critical review of the evidence. Progress in Neurobiology, 73, 179–226. Paredes, R. G., & Alonso, A. (1997). Sexual behavior regulated (paced) by the female induces conditioned place preference. Behavioral Neuroscience, 111, 123–128. Parsons, J. T., Kelly, B. C., Bimbi, D. S., DiMaria, L., Wainberg, M. L., & Morgenstern, J. (2008).

2/16/2011 6:05:43 PM


Explanations for the origins of sexual compulsivity among gay and bisexual men. Archives of Sexual Behavior, 37, 817–826. Parsons, J. T., Kelly, B. C., Bimbi, D. S., Muench, F., & Morgenstern, J. (2007). Accounting for the social triggers of sexual compulsivity. Journal of Addictive Diseases, 26, 5–16. Peterson, G. B., Ackil, J. E., Frommer, G. P., & Hearst, E. S. (1972). Conditioned approach and contact behavior toward signals for food and brain-stimulation reinforcement. Science, 177, 1009–1011. Pfaus, J. G. (1999). Neurobiology of sexual behavior. Current Opinion in Neurobiology, 9, 751–758. Pfaus, J. G., Damsma, G., Wenkstern, D., & Fibigar, H. C. (1995). Sexual activity increases dopamine transmission in the nucleus accumbens and striatum of female rats. Brain Research, 693, 21–30. Pfaus, J. G., Kippin, T. E., & Centeno, S. (2001). Conditioning and sexual behavior: A review. Hormones and Behavior, 40, 291–321. Plaud, J. J., Gaither, G. A., Hegstad, H. J., Rowan, L., & Devitt, M. K. (1999). Volunteer bias in human psychophysiological sexual arousal research: To whom do our research results apply? Journal of Sex Research, 36, 171–179. Plaud, J. J., & Martini, J. R. (1999). The respondent conditioning of male sexual arousal. Behavior Modification, 23, 254–268. Pleyers, G., Corneille, O., Luminet, O., & Yzerbyt, V. (2007). Aware and (dis)liking: Item-based analyses reveal that valence acquisition via evaluative conditioning emerges only when there is contingency awareness. Journal of Experimental Psychology: Learning, Memory and Cognition, 33, 130–144. Putnam, D. E. (2000). Initiation and maintenance of online sexual compulsivity: Implications for assessment and treatment. CyberPsychology and Behavior, 3, 553–563. Rachman, S. (1966). Sexual fetishism: An experimental analogue. Psychological Record, 16, 293–296. Rachman, S., & Hodgson, R. J. (1968). Experimentallyinduced “Sexual Fetishism” replication and development. Psychological Record, 18, 25–27. Rescorla, R. A. (1988). Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43, 151–160. Robinson, T. E., & Berridge, K. C. (1993). The neural basis of drug craving: An incentivesensitization theory of addiction. Brain Research Reviews, 18, 247–291.


549

Robinson, T. E., & Berridge, K. C. (2000). The psychology and neurobiology of addiction: An incentive-sensitization view. Addiction, 95, S91–S117. Robinson, T. E., & Berridge, K. C. (2003). Addiction. Annual Review of Psychology, 54, 25–53. Roche, B., & Barnes, D. (1998). The experimental analysis of human sexual arousal: Some recent developments. The Behavior Analyst, 21, 37–52. Rosen, R. C., Shapiro, D., & Schwartz, G. (1975). Voluntary control of penile tumescence. Psychosomatic Medicine, 37, 479–483. Rozin, P., Wrzesniewski, A., & Byrnes, D. (1998). The elusiveness of evaluative conditioning. Learning and Motivation, 29, 397–415. Schultz, W., Dayan, P., Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. Seligman, M. E. P. (1970). On the generality of the laws of learning. Psychological Review, 77, 406–418. Shors, T. J., & Matzel, L. D. (1997). Long-term potentiation: What’s learning got to do with it? Brain and Behavioral Sciences, 20, 597–655. Sintchak, G., & Geer, J. H. (1975). A vaginal plethysmograph system. Psychophysiology, 12, 113–115. Soares, J. J. F., & Öhman, A. (1993). Backward masking and skin conductance responses after conditioning to nonfeared but fear-relevant stimuli in fearful subjects. Psychophysiology, 30, 460–466. Strassberg, D. S., & Lowe, K. (1995). Volunteer bias in sexuality research. Archives of Sexual Behavior, 24, 369–382. Timberlake, W. (2001). Motivational modes in behavior systems. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 155–209). Mahwah, NJ: Erlbaum. Tomie, A., Grimes, K. L., & Pohorecky, L. A. (2008). Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse. Brain Research Reviews, 58, 121–135. Uslaner, J. M., Acerbo, M. J., Jones, S. A., & Robinson, T. E. (2006). The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine. Behavioral Brain Research, 169, 320–324. Walther, E., & Nagengast, B. (2006). Evaluative conditioning and the awareness issue: Assessing contingency awareness with the four-picture recognition test. Journal of Experimental Psychology: Animal Behavior Processes, 32, 454–459.

2/16/2011 6:05:43 PM

550


Whalen, R. E. (1966). Sexual motivation. Psychological Review, 73, 151–163. White, D. J. (2004). Influences of social learning on mate-choice decisions. Learning and Behavior, 32, 105–113. Williams, D. R., & Williams, H. (1969). Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, 511–520. Woodson, J. C. (2002). Including “learned sexuality” in the organization of sexual behavior. Neuroscience and Biobehavioral Reviews, 26, 69–80.


Yeomans, M. R., Mobini, S., Elliman, T. D., Walker, H. C., & Stevenson, R. J. (2006). Hedonic and sensory characteristics of odors conditioned by pairing with tastants in humans. Journal of Experimental Psychology: Animal Behavior Processes, 32, 215–228. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience, 7, 464–476. Zamble, E., Hadad, G. M., Mitchell, G. B., & Cutmore, T. R. (1985). Pavlovian conditioning of sexual arousal: First- and second-order effects. Journal Experimental Psychology: Animal Behavior Processes, 11, 598–610.

2/16/2011 6:05:43 PM

INDEX

Note: Page numbers followed by “f” or “t” indicate figures and tables, respectively. abstinence alcohol, 219–20 bupropion and, 254 cocaine, 324 through drug replacement therapy, 251 positive attention and, 256 related expectancies, 230 ACC. See anterior cingulate cortex acceptance, 257 acquisition, 49–50, 349f advertising and, 501 altruistic reinforcement and, 426 analogs, 423 attraction conditioning, 438 Campbell and Kraeling analog and, 433 conditional analgesia and, 312 context, 94 CS processing and, 313 emergent properties of, 348–51 with natural female features, 515f reacquisition in contrast to, 246 renewal and, 81 sample size and, 348–49 with sexual conditioning, 508, 509f social judgments and, 351– 58 verbal instruction and, 105 acquisition-extinction interval, 95–96 ACTH. See adrenocorticotropic hormone activation amygdala, 52–53 of behavioral control, 222–25 external, 348 factors for, 31–32 fear, 30–33 of fear structure, 37 inappropriate expectancies and, 226–27 internal, 348 psychomotor, 236 spreading, 346 adaptation, connectionist learning and, 346–48

adaptive specializations, 525 addiction associative bases for, 235 associative learning and, 285 cocaine, 274 context in, 249–50 cost of, 235 cue exposure and, 255–56 drug effects in contrast to, 222 drugs and, 13 habit learning and, 240 integrated treatment for, 261–62 interdrug associations and, 280–81 interoceptive conditioning and, 278–85 learning theory and, 261 reinforcement schedules and, 243 sexual compulsivity and, 541–42 S-R associations and, 240 substance abuse treatment and, 250–61 through substitution, 283 ADHD. See attention deficit/hyperactivity disorder adrenergic binding sites, 192 adrenergic system, 31 adrenocorticotropic hormone (ACTH) HPA axis and, 191, 193f inescapable shocks and, 129 learned helplessness and, 125 advertising, 481–83 acquisition and, 501 affect in, 496–97 attention in, 487–88 behavioral measures, 489 blocking in, 492–94 brand placement within, 485–86 cognition and, 496–97 conditioning during, 484–85 conditioning parameters, 485–91 contextual cues in, 495–96 contingency awareness during, 497–98 control conditions, 489–90

551

24-Schachtman-Index.indd 551

2/16/2011 9:31:21 AM

552 advertising (Cont’d) evaluative conditioning in contrast to classical conditioning, 497–98 exposure in, 489 food preferences and, 291 individual difference and, 498–501 intertrial interval in, 489 involvement in, 498–500 latent inhibition in, 492 music in, 490–91 need for cognition in, 500 overshadowing in, 492–94 partial reinforcement in, 486–87 personality variables and, 498–501 positivity effect in, 487 prior belief and, 500–501 second-order conditioning in, 494–95 US preexposure in, 491–92 affect in advertising, 496–97 body size and, 388 eating disorders and, 385 men’s perceptions of women and, 382–84 perceived salience of, in women, 394 transfer, 496 aggression, 454 disinhibiting effects of alcohol and, 227–28 sexual, 376, 382–84 stop-signal model of behavioral control and, 223 AI. See anterior insula alcohol antidepressants and, 217 approach-avoidance behavior and, 227 binge drinking, 229 comorbidity with smoking, 281 compensatory reactions and, 224 conflicting expectancies and, 227–28 cue-exposure therapy for, 281–82 disinhibiting effects of, 227–28 effects, 214 evaluative conditioning and, 90 extinction and recovery and, 84 impairment, 216–18 inhibitory control and, 224–25 as learned behavior, 235 mediating expectancies with, 221 motor skills and, 218–19, 219f Pavlovian features of, 271 placebo in contrast to, 215–16 related expectancies, 224–25 response-appropriate expectancies and, 225–27 resurgence and, 247–48 S*d-Rd expectancies and, 216–18 social drinking and, 216–17 stop-signal model of behavioral control and, 223 tolerance, 213, 218–21 use and abuse, 229 allergic reactions, 199


INDEX altruism, 417, 425–28 delay effect in, 428 failures of, 427–28 replication of, 428 Alzheimer’s disease, 200 AM404, 261 ambiguity, extinction learning and, 82, 249 amnesia circuit selection and, 316 infantile, 112 amphetamine, 227 blocking and, 314 dopamine neurons and, 315 between drug conditioning and, 272–73 LI and, 154 overshadowing and, 314 amygdala, 32–33 activation, 52–53 activity, 458f in classical conditioning, 458 fear and anxiety behavior and, 47 observational fear learning and, 466 in social cognition, 461–62 social fear learning and, 472 startle reflex and, 49 threat responsiveness and, 63 trait anxiety and, 63 analgesia, 308–12 animals aversive stimuli and, 200 evaluative conditioning and, 406 expectancy in, 214 interference effect and, 122–23 learned fear in, 464–65 PTSD in, 29 R-O associations in, 9 sexual conditioning, 534–35 single-system propositional model and, 107 S-R associations in, 8–9 SSDRs and, 307 anterior cingulate cortex (ACC), 460, 463, 467 anterior insula (AI), 460, 463, 467 anticipatory behavior, 107 antidepressants, 217 antigenic challenge, 192 antipain mechanisms, 306 antisocial personality, 223 anxiety. See also pathological anxiety; trait anxiety attenuated LI and, 159 brain lesions and, 49 CBT for, 113–14 classical conditioning and, 45 clinical, 59 clinical evidence for, 47 conditioning, 53 conditioning paradigms, 49–50 as CR, 49 exposure therapy and, 84, 113

2/16/2011 9:31:21 AM

INDEX fear in contrast to, 46 generalized fear and, 456 genetic variation and, 52–53 individual differences in cognitive processing and, 376 inescapable shocks and, 129 irrational, 109–10 learned, 113 manifest, 423–24 neuroanatomical evidence in, 47–49 neuroimaging variation and, 52–53 observational learning and, 114 other pathway for acquisition of, 54–56 panic attacks and, 47 pathology, 60 physiology of, 46 reactions, 110 routes to, 110–13 single-system propositional model and, 107–16 treatment implications, 113–15 unpredictability and, 58–59 anxiety disorders attentional processes in models of, 44–45 conditioning abnormalities in, 49 differences in learning, 54–59 emotional processing theory and, 28 etiology of, 80 individual differences in fear conditioning and, 52 single-system propositional model and, 107–16 trait anxiety in contrast to, 69 apomorphine, 194 appetitive outcomes aversive in contrast to, 10t, 431 avoidance behavior in contrast to, 243 drug states and, 279 extinction and, 245 within-session fear reduction and, 34 applied behavior analytics, 171–72 approach-avoidance behavior, 227 approach behavior, 508–9 to CS, with copulation, 511f aroma, 295–96 ASD. See autism spectrum disorder Asperger’s syndrome, 168 assimilation, 360–61, 361f associability in advertising, 488 associative learning and, 159 CS, 313–14 associations algorithms, 534 in fear learning, 56–57 forming, 400 interdrug, 280–81 intra-administration, 273–74 learned, 238f types of, 3 within-compound, 14–15 associative learning


553 addiction and, 285 anxiety reactions and, 110 associability and, 159 circuit selection and, 316–17 cognitive processes in, 105–6 dual-system model of, 105–6 expectancy and, 104 nonreinforcement and, 281 single-system propositional model of, 106–7 social processes and, 345–46 in women’s learning study, 387 associative structures, 3, 15. See also specific associative structures attention in advertising, 487–88 dimensional, 381 error correction and, 312–14 negative, 256 positive, 256 reallocating, 383–84 selective, 379 attentional biases assessment, 60–61 automaticity of, 66–68 conditioning in, 68 CS+ in contrast to CS-, 64–65 depression-related, 61–62 fear conditioning and, 64–69 index, 66f theories, 61–64 threat-relevant, 44–45, 59–64 time course of, 62 trait anxiety and, 59, 68–69 attentional deficits, 152, 169 attentional shifting, 65–66 artificial stimuli and, 396 blocking and, 395 category learning and, 381 hypothesis, 130 in women, 394 attention deficit/hyperactivity disorder (ADHD), 223 attitude change, 359t, 365–67 formation, 367f attraction conditioning, 437–41 in context, 441 interpersonal, 437–41 losses in strength of, 440 social, 417 superconditioning, 439–40 augmentation, 349–50 attribution ratings after, 357f in causal attribution, 355–56 of causal relationship detection, 444 in dispositional attribution, 356–58 of human agency, 444

2/16/2011 9:31:21 AM

554 autism spectrum disorder (ASD), 168–69 central coherence theory and, 170–71 cognitive approaches to, 169–72 comparator theory of, 178–82 conceptualization of, 171 diagnosing, 168 discrimination learning and, 172–78 discriminations and, 169 executive dysfunction and, 170 extinction treatment for, 183–84 high-functioning in contrast to low-functioning, 183–84 implications for interventions, 182–84 importance of discrimination learning for, 173–74 language development and, 176–77 learning theory and, 171–72 MTS procedure and, 175–76 overselective responding and, 177–78 sensitive comparator and, 181–82 theory of mind and, 170 within-stimulus learning and, 177 automatic behavior, 107 automatic processing, 160 autoshaping, 274–75, 509f. See also sign tracking C/T ratio and, 524f sexual compulsivity and, 542 sexual conditioning and, 509–10 aversive conditioning energization properties of, 423–24 response learning and, 244 smoking and, 252–53 aversive drive, 428, 431 aversive outcomes, 10t, 11, 431 avoidance, 3, 10 appetitive outcomes in contrast to, 243 attentional bias and, 62 aversive conditioning and, 244 counseling and, 257 escape in contrast to, 11 extinction and, 245 generalized fear and, 456 instrumental, 107, 109 interference effect and, 122 negative contingencies and, 11 passive, 11–12 PTSD and, 139 punishment in contrast to, 11 punishment of, 245 situational, 110 of trauma-related stimuli, 29 two-factor theory and, 8 awareness, 497–98

basal ganglia, 458 basolateral amygdala (BLA) circuit selection and, 316 dorsal raphe nucleus and, 133


INDEX drug abuse and, 203–4 fear conditioning and, 47 bed nucleus of the stria terminalis (BNST) fear and anxiety behavior and, 47–49 fear reinstatement and, 58 learned helplessness and, 135 behavial measures, 489 behavioral control, 221–28, 222f behavioral economics, 321–22, 327–28 behavioral mimicry, unconscious, 467 behavioral therapy, 113–14 behaviorism, 483–84 behaviorist-cognitive debate, 483–84 benzodiazepine, 30, 133 benzodiazepine chlorodiazepoxide (CDP), 133 benzodiazepine inverse agonists, 133–34 between-session habituation, 28 cognitive processes in, 36 fear reduction and, 34–36 implications of, 37 biases group, 359t, 361–65 hindsight, 501 modification, 59 simulations of social, 358–70 unit, 298 binge drinking, 229 biological preparedness, 537 BLA. See basolateral amygdala blocking, 6–7 in advertising, 492–94 amphetamine and, 314 attentional shifting and, 395 attraction conditioning, 438 causal relationship detection and, 442–43, 443f compound conditioning and, 17 discounting and, 349–50 human agency and, 442–43 negative feedback and, 310–12 one-trial, 313 perceptual organization and, 387, 392–94, 392f, 393f R-O associations, with drug antagonists, 253–54 BNST. See bed nucleus of the stria terminalis body adornments, 514 body size, 385, 388 boundary conditions, 419 brain. See also neuroanatomical evidence dispositional attributions in, 367–71 imaging, 359t instrumental learning and, 9 mechanisms underlying retention of fear extinction, 35–36 brain lesions anxiety and, 49 conditioned immunomodulation and, 198 fear and, 132 brand names competition effects and, 502

2/16/2011 9:31:21 AM

INDEX contexts for, 488 as CS, 482–83 familiarity with, 487–88 placement of, 485–86, 501 bupropion, 254, 284

caffeine, 217 calculating circuits, 314 Campbell and Kraeling analog, 432–34, 433f cannabanoid reuptake inhibitors, 261 carbon monoxide (CO), 324 catechol-O-methyltransferase gene (COMT), 52 heritability, 54 polymorphisms, 53, 59 threat attention and, 63 category learning, 377 dimensional salience and, 384 of men, about women, 382–83 in men’s learning study, 383f perceptual organizing and, 377–81 role of, 396 category structure, 392–93, 392f causal attributions, 351 discounting and augmentation in, 355–56 inferred from covariation information, 351–52 ratings, 353f ratings, after discounting and augmentation, 357f sample size in, 352–54 causal relationship detection, 417, 441–45 CBT. See cognitive behavioral therapy CDP. See benzodiazepine chlorodiazepoxide central coherence theory, 170–71 central nervous system (CNS), 222 cephalic-phase responses, to food cues, 293 cessation signals, 131. See also smoking cessation childhood disintegrative disorder, 168 children food neophobia and, 290 food preferences of, 290–91 overcoming satiety in, 293 portion size and, 297 responsiveness to food cues of, 292 chlorpromazine, 154 circuit selection, 315–17 classical conditioning, 3. See also Pavlovian conditioning advertising and, 481–83, 497–98 anxiety and, 45 in anxiety disorders, 46–59 fear and, 45 features of, 6t frontal cortex in, 460–61 neuroanatomical evidence, 457–61 occasion setters and, 7 schizophrenia and, 153 sexual behavior and, 533 S-S association and, 5–6 clinical science, 376–77


555 CNS. See central nervous system CO. See carbon monoxide cocaine, 202–3 abstinence, 324 addictive liability of, 274 drug-related expectancies, 229 sexual conditioning and, 518 sign tracking and, 275 Cochrane Collaboration, 324 cognition, 500 cognitive behavioral therapy (CBT), 113–14 cognitive control behavior and, 222–23 counseling and, 256–57 cognitive disorganization, 152 cognitive dissonance, 365–67 energization and, 424 N-opponents and, 430–31 cognitive enhancers, pharmaceutical, 85 cognitive facilitation, 238 cognitive load, 105 cognitive processes in advertising, 496–97 alcohol and, 44 in anxiety disorders, 44 behaviorism in contrast to, 483–84 between-session habituation in contrast to withinsession habituation, 36 drug effects and, 222 individual differences in, 376–77 in learning, 105–6 need for, in advertising, 500 cognitive restructuring, for preventing relapse, 84 cognitive revolution, 417, 483 cognitive science, 376–77 cognitive therapy, 113 coign of vantage, 427 commitment, to counseling, 257 communication interpersonal, 417, 420–21 persuasive, 366–67 Community Reinforcement Approach (CRA) programs, 328 community support, 330 comorbidity, between alcohol and smoking, 281 comparator theory of autism spectrum disorder, 178–82 intertrial intervals and, 495 compensatory reactions, 224 competence, 430 competition, 349f, 417, 428–31 brand names and, 502 connectionist learning and, 356 dispositional attribution and, 357–58 emergent properties of, 348–51 Hobbes and, 435 N-opponents in, 429–30 person impression formation and, 358–60

2/16/2011 9:31:21 AM

556 competition (Cont’d) response, 130 social judgments and, 351–58 subtyping and, 364–65 complex social behavior, 417–18 COMT. See catechol-O-methyltransferase gene COMT Val158 Met polymorphism, 53, 63–64 conceptual-categorization account, of evaluative conditioning, 408–9 conditioned drug tolerance, 213, 218–21, 273, 283 conditioned fear backward CS and, 131 escape behavior and, 126 fear in contrast to, 115 learned helplessness and, 130–31 conditioned immunomodulation, 191–92 aversive stimuli and, 199–201 drug abuse and, 201–5 historical perspectives, 193–95 immunostimulatory agents and, 198–99 with immunosuppressive agents, 195–98 conditioned responses (CR) anxiety as, 49 conscious propositional knowledge and, 116 contextual control of, 57–58 contingency awareness in contrast to, 106 impairments to, 105 recovery of extinguished, 81, 83–84 varying, 15 verbally expressed knowledge in contrast to, 104 conditioned stimuli (CS) arrangement of, in time, 485–86 associability, 313–14 backward, 131 body adornments as, 514 brand names as, 482–83 competition between, 17 conditioning effects of, 524–25 distinctions among procedures with, 18–19 drug state as, 281 factors, in sexual conditioning, 509f, 511–12 for fear activation, 31–32 fear response to, 309 gustatory, 197, 199 intrinsic relation of, with US, 405–6 modality and semantic category of, 404 natural female features as, 514–16 olfactory, 197, 539 only, in evaluative conditioning, 403 in Pavlovian conditioning, 399 preexposure effect, 492 presentation of, in evaluative conditioning, 406 processing, 313 properties of, in advertising, 487–88 relationship to US of, 486 salience of, with overshadowing, 493 as second excitor, 90–92 second-order conditioning and, 494–95


INDEX sexually relevant, 537 within-compound associations and, 14–15 conditioned stimulus-no-outcome unconditioned stimulus (CS-noUS) associations, 7, 16 conditioned stimulus-unconditioned stimulus (CS-US) associations, 4–5, 400 awareness of, 411 causal relationship detection, 442 communication of, 408 conditioning and, 16 contextual cues and, 16 evaluative conditioning and, 402 event-memory model and, 13–14 experimental extinction in, 80–81 human agency, 442 implemented in evaluative conditioning, 404–8 interval, 523 misattributions in, 409–10 omission training and, 8 other effects of, 407–8 statistical properties of, 402–3 conditioned suppression paradigms, 58 conditioning. See also classical conditioning; evaluative conditioning; fear conditioning; instrumental conditioning; instrumental escape conditioning; interoceptive conditioning; Pavlovian conditioning; sexual conditioning abnormalities, anxiety disorders and, 49 during advertising, 484–85 anxiety, 53 in attentional biases, 68 attraction, 437–38 causal relationship detection, 441–42 compound, 17–18 context, 57–58, 512–14 contextual cues during, 16–17 CS-US associations and, 16 developments in, 4–5 discriminative, 50 between drug, 272–74 escape, 426–27 evaluative, 35 excitatory in contrast to inhibitory, 51 expectancy, 35 field, 539–40 higher order, 15 human agency, 441–42 number of trials for, 488–89 observational, 57 overlap between types of, 5–6 paradigms, 49–51 representation-mediated, 501 representations in, 13–15 research, 445–47 retention, in advertising, 490 second-order, 15, 17, 180, 494–95 simple, 49–50 symmetry of, 5–6

2/16/2011 9:31:21 AM

INDEX theory, 483–84 trace, 252, 485 conditioning-extinction theories, of discrimination learning, 172 conflict, 424 response, 259 connectionist learning, 346–48 causal attribution and, 354 cognitive dissonance and, 365–66 dispositional attribution and, 355 persuasive communication and, 366–67 simulations, 358–70 conscious belief, 104 consensus, 352 coding schemes, 369t–370t manipulating, 353–54 neuroimaging of, 368–71 consistency, 368–71, 369t–370t constraint satisfaction models, 346 contests, 322, 325–26 context acquisition, 94 in addiction, 249–50 for attraction, 441 for brand names, 488 conditioned drug tolerance and, 273 conditioning, 57–58, 512–14 of CR, 57–58 enhancing, 96 for evaluative conditioning, 407–8 of extinction, 94 extinction in contrast to acquisition, 94 extinction retention and, 36 incentives and, 330 manipulations, 15 multiple, in extinction, 86–88, 248–49 multiple, in massive extinction, 88–89 reducing, 253, 260 renewal and, 247 temporal, 83, 96 test, 96 unpredictability and, 58–59 contextual cues in advertising, 495–96 during conditioning, 16–17 sexual conditioning, 512–14, 512f US preexposure and, 492 contiguity, 486 contingencies in CS-US associations, 402–3 deposit, 325 escape, 131 instrumental, 10–13, 10t management, 254–55 masking, 420 negative, 10–11 positive, 10–11 punishment, 12


557 smoking cessation and, 322 stimulus-outcome, 106 contingency awareness, 67 in advertising, 497–98 CR in contrast to, 106 contrast, 360–61, 361f contrast effects, to stimuli, 243–44 controllability, 128f, 130 controlled processing, 160 coping, 257 anxiety and, 108 appraisal, 108 copulation, 510, 511f with artificial objects, 517 efficiency, 520 correlation, illusory, 362–64, 362f, 363f correlation, positive, 10 corticosterones, 129–30, 192 corticotrophin-releasing hormone (CRH), 134–35, 191, 193f cortisol, 193f Coturnix japonica, 508 Coturnix quail, 508 counseling, 256–58 counterconditioning, 15 aversive, 260 aversive conditioning and, 253 methadone and, 252 response conflict and, 259 covariation, 348, 351–52 CR. See conditioned responses craving, 256–57 CRH. See corticotrophin-releasing hormone CS. See conditioned stimuli CS+. See positive conditioned stimulus CS-. See negative conditioned stimulus CS-0. See nominal target stimulus CsA. See cyclosporine-A CS-noUS associations. See conditioned stimulus-nooutcome unconditioned stimulus associations CS-US associations. See conditioned stimulusunconditioned stimulus associations C/T ratio, 523–24, 524f cued go/no-go task, 225 cue-exposure therapy, 281–82 cues. See also food cues drug, 201, 203, 215, 241 exposure, 255–56 extinction, 89–90, 90f reminder, 128–29, 140–41 retraining of extinguished, 81 retrieval, 89–90, 249 salience of, 182 second CS, accompanying a CS during extinction, 90–92 social, 454–55 CY. See cyclophosphamide cyclophilins, 197

2/16/2011 9:31:22 AM

558 cyclophosphamide (CY), 193, 195–97 cyclosporine-A (CsA), 197 cytokines, 192, 197, 202

data preparation, 390–91 DCS. See D-cycloserine D-cycloserine (DCS), 32 in exposure therapy, 38 pharmacoptherapy, 260–61 for preventing relapse, 85 defensive behavior, 307–9 delta learning algorithm, 347–48 delusions, 159 depression attentional bias and, 61–62 hopelessness, 139 individual differences in cognitive processing and, 376 individual differences in fear conditioning and, 52 learned helplessness and, 138–39 desensitizing treatment, 16 dexamethasone, 125 DGT. See discriminated goal tracking DHHS. See Department of Health and Human Services Diagnostic and Statistical Manual of Mental Disorder, fourth edition (DSM-IV), 29, 139, 152, 168 diazepam, 135, 279 differential responding, 178 direct experience, 114 disagreement-induced drive, 423–24 discounting, 349–50 attribution ratings after, 357f in causal attribution, 355–56 in dispositional attribution, 356–58 multiplicity of, 444–45 discriminated goal tracking (DGT), 275–78, 277f, 281 discriminated taste aversion (DTA), 275 discrimination, 6–7, 169 autism spectrum disorder and, 172–78 conditional, 175–77 conditioning-extinction theories of, 172 drug, 278 enhanced, 177 simple, 174–75 stimulus prompts and, 173–74 two-card, 181f within-compound, 180 disengagement attentional bias and, 62 in nonanxious individuals, 65–66 threat, 61 dishabituation, 3, 18 disinhibited behavior, 222 disinhibition alcohol abuse and, 229 drug-related expectancies and, 228 of restrained eating, 299–300 single-stimulus presentation and, 18–19


INDEX disordered eating, 376, 384–85 dispositional attribution, 351 connectionist learning and, 355 discounting and augmentation in, 356–58 inferred from covariation information, 351–52 neuroimaging of, 367–71 ratings, 355f, 357f sample size in, 354–55 dissociations to avoid renewal, 247 between CS-US associations and evaluative conditioning, 407–8 dual-system model of learning and, 106 dissonance. See cognitive dissonance distinctiveness, 352–54 distractibility in healthy individuals, 156 schizophrenia and, 154 distraction, 84, 105 distress tolerance, 257 disturbance, locus of, 110 disuse, theory of, 248 dlPFC. See dorsolateral prefrontal cortex DMCM, 133–34 dopamine attention and, 314 in BLA, 204 bupropion and, 254 cocaine addiction and, 274 drug-induced neural plasticity and, 235 immunostimulatory agents and, 199 neurons, 314–15 schizophrenia and, 162 dopamine-receptor antagonists, 154 dorsal periaqueductal gray (DPAG), 132–33, 138f dorsal raphe nucleus (DRN), 132–34, 137f, 142, 200 dorsolateral prefrontal cortex (dlPFC), 461 dot-probe task, 60–63, 66 Down syndrome, 174 DPAG. See dorsal periaqueductal gray drive intensity, 423, 432 DRN. See dorsal raphe nucleus drug antagonists, 253–54, 259, 311 drug replacement therapy, 250–52. See also substitution drugs, 13. See also addiction; alcohol; smoking abuse, effects of, 229 acute effects of, 221–28 altering stimulant effects of, 283–84 AM404, 261 conditioned immunomodulation and, 201–5 conditioned tolerance, 213, 218–21, 273, 283 conditioned tolerance and, 273 cues, 201, 203, 215, 241 deprivation, 240 discrimination, 278 disinhibited behavior and, 222 extinction and, 245–50 identifying triggers for, 257

2/16/2011 9:31:22 AM

INDEX incentive learning and, 236 incentive salience and, 239 incentives for, 323–24 integrated treatment for, 261–62 intransigence of use of, 251 as learned behavior, 235 long-term memory and, 238 metabolites of, 272 model situation, 215f motivation, 241 negative attention and, 256 neural plasticity induced by, 235 polydrug abuse, 280–81 psychoactive, 13 reinstatement and, 246 related expectancy, 214–21 self-administration of, 244, 259 smoking in contrast to, 243 S-R associations and, 9 substance abuse treatment and, 250–61 substitutability of, 327–28 drug states appetitive outcomes and, 279 as CS, 281 between drug conditioning with, 272–74 factors impacting, 272 as Pavlovian stimuli, 271–78 substitutions for, 282–83 transfer of occasion setting, 283f DSM-IV. See Diagnostic and Statistical Manual of Mental Disorder, fourth edition dual-system model of associative learning, 105–6 dysphoria, 59, 292

eating, 290. See also food; restrained eating disinhibited, 299–300 memory and, 295 motivations for, 293–94 quantity of, 295–300 restrained, 292 of snacks, 295 social facilitation of, 297 timing of, 292–95 eating disorders cognitive factors in, 384–85 individual differences in cognitive processing and, 376 symptomatic differences in, 385 ego threats, 300 emergent properties, 350–51 emotional learning, 454–55 procedures in contrast to processes, 456–57 research, 455–56 emotional processing indicators, 28 theory, 27–37, 31 within-session fear reduction and, 34 emotional reactions, innate in contrast to learned, 116


559 emotional regulation, 385 encoding, 169 endorphins, 125, 307, 308–9 energization, 423–24 energy, 5 environmental factors for anxiety, 111 in evaluative conditioning, 407 induction in contrast to test, 127–28 in learned helplessness, 127–28 epinephrine, 193f error correction, 305–6 attention and, 312–14 calculating circuits, 314 circuit selection and, 315–17 decremental, 312 dopamine neurons and, 314–15 perceptual-defensive-recuperative model of, 306–8 errors minimization, 348 negative, 306 signals, 315 escape. See also instrumental escape conditioning aversive conditioning and, 244 avoidance in contrast to, 11 conditioning, 426–27 contingencies, 131 dorsal raphe nucleus and, 132–33 learned helplessness and, 126–27 learning, 131 PTSD and, 141 ethanol. See alcohol evaluative conditioning, 35, 67, 401–2 advertising and, 497–98 alcohol and, 90 communication of CS-US association in, 408 conceptual-categorization account of, 408–11 context for, 407–8 CS-only in, 403 CS-US associations implemented in, 404–8 CS-US CO-occurrence in, 402–3 holistic account of, 409 intrinsic relation between CS and US in, 405–6 mechanisms of, 411–12 mental process theories, 408–11 misattribution account of, 409–10 occasion setting in, 403–4 organisms experiencing, 406–7 presentation of CS and US in, 406 propositional learning account of, 410–11 referential account of, 410 sexual conditioning and, 533–34 stimuli in, 404–6 US-only in, 403 valence of US in, 404–5 evaluative learning, 534 event-memory model, 13–14 excitatory properties, acquired, 278–80

2/16/2011 9:31:22 AM

560 excitors, second, 90–92 executive function autism spectrum disorder and, 170 PTSD and, 143 schizophrenia and, 152 expectancies, 4, 104 abstinence-related, 230 acute drug effects and, 221–28 alcohol and, 224–25, 229 conditioning, 35 conflicting, 227–28 drug-related, 214–21, 228–29 inappropriate, 226 inhibitory control and, 224–25 learned, 228, 230 learned anxiety and, 113 mediating, 221 outcome differences and, 220 perceptual-defensive-recuperative model and, 307 persuasive communication and, 367 placebos and, 215–16 Rd-S*, 218–21, 224 response-appropriate, 225–27, 226f R-O associations and, 239 S*d-Rd, 216–18 signals, 315 single-system propositional model and, 107–9 S-O associations and, 239 sources of, 214 S-S*d, 216 stimulus-outcome contingencies and, 106 experience-based interventions, 114. See also behavioral therapy experiential differences, 56 experimental design, in anxiety, 49–50 experimental neurosis, 79 exposure in advertising, 489 cues, 255–56 food preferences and, 291 men’s perceptions of women and, 382–84 mere, 484, 491 exposure therapy anxiety and, 84, 113 between-session fear reduction and, 34–35 beyond habituation, 84 changing environments for, 84 D-cycloserine in, 38 experimental extinction as model of, 80–81 extinction paradigms in contrast to, 27 fear reactions and, 115 fear stimuli during, 84 increasing sessions of, 84 massed, 84 nonpharmacologic tools for, 38 over time, 115 relapse models after, 81–84 safety behaviors and, 108


INDEX within-session fear reduction and, 34 extinction, 3, 49–50, 195, 245–50. See also fear extinction; massive extinction ambiguity and, 82, 249 attentional bias and, 66f aversive conditioning in contrast to, 253 avoidance in contrast to appetitive outcomes, 245 as clinical tool, 183–84 to combat overselectivity, 183–84 context of, 94 CS-US associations and, 7, 95 cue exposure and, 255–56 cues, 89–90, 90f deepening, 96 event-memory model and, 14 experimental, 80–81, 84 exposure therapy in contrast to, 27 fear activation during, 30–33 habituation in contrast to, 39n1 implications of emotional processing theory in, 37 interval after acquisition, 95 of learned fear, 457 learning, 82 moderate in contrast to massive, 89f in multiple contexts, 86–88, 248–49 neurons, 33 optimizing, 248–50 overselectivity and, 179 pharmacoptherapeutic, 260–61 protection from, 109 punishment and, 245 recovery and, 83–84 reducing recovery after, 84–96 retention, 35–36 retrieval cues from, 89–90 R-O associations, 260 salient retrieval cues for, 249 with second excitor, 90–92 sessions, spaced, 93–94 single-stimulus presentation and, 18–19 S-O associations and, 260 spaced training in, 92–93, 93f S-R associations and, 258 US and, 15–16, 94–95, 95f verbal instruction and, 105

facial expressions amygdala and, 32, 64 in animals, 465 fear and, 462 fading procedures, 334–35 families, anxiety acquisition in, 55–56 fear. See also conditioned fear; social fear learning activation, during extinction, 30–33 anxiety in contrast to, 46 between-session reduction of, 34–36 brain lesions and, 132

2/16/2011 9:31:22 AM

INDEX classical conditioning and, 45 clinical evidence for, 47 conditioned fear in contrast to, 115 conditioning abnormalities, 49 CS-induced, 309 disinhibited eating and, 300 emotional learning and, 455 etiology of, 80 facial expressions and, 462 freeze behavior as measure of, 126 generalized, 455–56 inhibition of, 29–30 latent, 112 measures of, 28 memory, 32 neural studies of, 32–33 neuroanatomical evidence in, 47–49 neurons, 33 other pathway for acquisition of, 54–56 PDR and, 307–8 processing pathways, 66 PTSD in contrast to, 29 reaction mechanisms, 115–16 reduction, 30 reinstatement of, 57–58 selective associations in learning, 56–57 signals, transmission of, 464 stimuli, 84 structures, 28, 30, 37 threat imminence and, 47 within-session reduction of, 33–34 yohimbine-induced, 31 fear conditioning, 507 analgesia and, 309 attentional biases and, 64–69 circuit selection and, 316 empirical status of, 51 with exteroceptive stimuli, 274 genetics and, 53–54 individual differences in, 51–52 learning about others and, 471 negative feedback in, 309–12, 310f nonanxious individuals and, 65–66 personality and temperament in, 53 fear extinction, 27, 29–30 brain mechanisms underlying retention of, 35–36 consolidation of, 38–39 efficacy ceiling in, 37 fear structure activation in, 37 implications of, 36–39 fear learning, 456–57. See also social fear learning in animals, 464–65 instructed, 469–70 interacting pathways in, 467–69 observational, 464–69, 469f pain and, 466–67 feedback, 377. See also negative feedback eating disorders and, 385


561 feedforward networks, 347f fertility, sexually conditioned, 520–22, 522t fever, 192 FGF-2. See fibroblast growth factor 2 fibroblast growth factor 2 (FGF-2), 143 field conditioning, 539–40 5-HT neurons, 132–34, 136–37 5-HTT. See serotonin transporter gene flashbacks, 129 flight behaviors, 306 flight-or-flight response, 46 fMRI. See functional magnetic resonance imaging food. See also eating memories of, 299 neophobia, 290 palatable in contrast to unpalatable, 296 portion size of, 297–98 preferences, 290–92 presentation of, 294 restricted access to, 291–92 as US, 507 variety of, 298–99 words, 291 food cues, 290 aroma, 295–96 auditory, 296–97 cephalic-phase responses, 293 food variety as, 298–99 mealtime as, 294–95 memories as, 299 normative, 297–98 palatability, 295–96 portion size as, 298 responsiveness to, 291–92 restrained eating and, 292 sensory, 295–97 social, 292 timing of eating and, 292–95 verbal, 292 visual, 292, 296 forced-choice recognition test, 66 forgetting, extinction in contrast to, 245 formalin, 309 freezing behavior, 126f, 127f fear and, 46–47 fear extinction and, 31 as measure of fear, 126 perceptual-defensive-recuperative model and, 306 frontal cortex, 460–64 functional knowledge, 401–8 functional magnetic resonance imaging (fMRI), 64, 368, 368f

GABA neurons, 133–34, 142 GAD. See generalized anxiety disorder Galileo, 434–35 gambler’s fallacy effect, 106

2/16/2011 9:31:22 AM

562 generalization, 351 ABA renewal and, 81 decrement, 91 stimulus learning and, 241 generalized anxiety disorder (GAD), 45, 58 genetics anxiety and, 52–53 emotional learning and, 456 evaluative conditioning and, 407 fear conditioning and, 51–52, 53–54 as route to anxiety, 111 threat attention and, 63–64 genital responding, 535 glucocorticoids, 193f glutamate, 235 glutamatergic output, 136, 142 goal tracking, 276, 509f, 524f. See also discriminated goal tracking group biases, 359t, 361–65 group differentiation, 361 group homogeneity, 361 group impression formation, 361

habitual behavior, 9 habituation, 3. See also between-session habituation; within-session habituation exposure therapy beyond, 84 extinction in contrast to, 39n1 protection from, 116 single-stimulus presentation and, 18 US preexposure and, 492 Hall-Pearce negative transfer, 313f hallucinations, 159 halperidol, 154 HAM model. See human associative memory model health promotion, 322–23, 335 Hebbian algorithm, 350 helplessness. See learned helplessness heritability, fear conditioning and, 53–54 heroin, 201–4, 282. See also opioids Heterosocial Perception Survey, 382 higher order conditioning, 15, 494–95 high-priority behaviors, 332–33 hippocampus circuit selection and, 316 COMT and, 53 dissociating, 460 extinction retention and, 36 learned helplessness and, 142–43 PTSD and, 142–43 histamine, 199 Hobbes, Thomas, 434–35 holistic account, of evaluative conditioning, 409 homophones, 170–71 hopelessness, 139 HPA axis. See hypothalamic-pituitary-adrenal axis


INDEX human agency, 417, 441–45, 442f human associative memory (HAM) model, 494 hyperarousal, 29, 139 hypervigilance, 142 hypothalamic-pituitary-adrenal (HPA) axis, 191–92, 193f hypothalamus, 192, 200

ICD-10. See International Classification of Diseases, 10th Edition IL-1. See interleukin-1 illusory correlation, 362–64, 362f, 363f IL region. See infralimbic region immune system, 191–92. See also conditioned immunomodulation immunization effect interference effect and, 124 mPFCv and, 136 training, 132f immunizations, 322–23 immunostimulatory agents, 198–99 immunosuppression, 196 immunosuppressive agents, 195–98 Implicit Association Task, 67 implicit misattribution, 409 impulsivity, 223, 229 inattention, 154–55 incentive learning, 239 during drug replacement treatment, 251–52 drugs and, 236 incentives carry-over effects of, 333 clinical impact of, 324–25 community support as, 330 context and, 330 directly applied to behavior, 322–27 for drug-use problems, 323–24 factors for, 329–33 with fading procedures, 334–35 future research for, 333–34 in health promotion, 322–23, 335 for high-priority behaviors, 332–33 indirect influence of, 327–29 lasting effect of, 332 monetary, 323, 326–28 monetary in contrast to nonmonetary, 329–30 public policy, 335 refining, for smoking cessation, 334–35 salience, 239 sensitization, 238, 240–41 sexual, 532–33 smoking cessation and, 322, 324–25 social, 326–27 individual difference variables, 498–501 induction, 127–28 inescapable shocks (IS), 124–26, 129, 133

2/16/2011 9:31:22 AM

INDEX infantile amnesia, 112 inflation effect, 56 information processing, 501–2 information representations, 350–51 infralimbic (IL) region, 135 inhibition. See also latent inhibition attraction conditioning, 438–39, 439f of behavioral control, 53, 222–25 conditioned, 259, 284–85, 483 control, 224–25 inappropriate expectancies and, 226–27 of return, 61 instrumental conditioning, 3 advertising and, 482–83 analogs, 420–35, 446t complex social behavior and, 417–18 features of, 6t Pavlovian-instrumental transfer and, 240–41 response learning and, 242–43 R-S association and, 6 instrumental escape conditioning, 418, 418t boundary conditions, 419 modeling, 418–19 reduction, 419 rules of correspondence and, 420, 421t translations, 419 instrumental learning, 105 brain system and, 9 goal-directed, 9 interdrug associations, 280–81 interference effect generality of, 123–24 learned helplessness hypothesis and, 122–23 response competition and, 130 interleukin-1 (IL-1), 192 intermittent shock, 424, 426–27, 430 internal activation, 348 International Classification of Diseases, 10th Edition (ICD-10), 168 interoceptive conditioning acquired excitatory properties in, 278–80 drug addiction and, 278–85 interdrug associations and, 280–81 interoceptive stimuli, 270–71 feature-positive in contrast to feature negative, 271f Pavlovian conditioning with, 285 interpersonal communication, 417, 420–21 intertrial interval, 489, 495 sexual conditioning and, 526 interval schedules, 12, 12t intrinsic motivation, 331–32 introversion/extraversion measurements, 540–41 inverse overshadowing, 445 involvement, in advertising, 498–500 irrational anxiety, 109–10


563 keyhole limpet hemocyanin (KLH), 198, 200 KLH. See keyhole limpet hemocyanin knowledge associative, 106 conscious contingency, 105 conscious propositional, 116 expression of, 4 functional, 401–8 propositional, 411 verbally expressed, 104

language, 104 in anxiety interventions, 114 anxiety reactions and, 110 autism spectrum disorder and, 171 development, autism spectrum disorder and, 176–77 instructed fear learning and, 469 latencies, in response learning, 243 latent inhibition (LI), 16–17, 195 abnormal, 159 accounting for, 159–62 in advertising, 492 attenuation, 159, 161–62 basic procedure, 154–55 CS processing and, 313 dependent variables in research of, 157t–158t experiential differences and, 56 experiments, 153, 155t in healthy individuals, 156–59 masking tasks and, 155–56 naloxone and, 313f pharmacological effects on, 154 potentiated, 161–62 pre-exposure and, 241–42 rationale for exploring, 154 schizophrenia and, 154–59 single-stimulus presentation and, 18–19 theories of, 159–60 lateral nucleus, 458 learned helplessness, 108, 121–22, 134–35 alternative perspectives on, 129–31 duration of, 124–27 escape behavior and, 126–27 escape contingencies, 131 fear and, 130–31 hypothesis, 122–23 neural bases of, 141–43 neural mechanisms of, 131–38 prefrontal cortex and, 135–38 psychopathological applications of, 138–43 PTSD and, 121, 139–40 reminder cues and, 128–29 stress and, 130 learned irrelevance, 123

2/16/2011 9:31:22 AM

564 learning. See also associative learning; category learning; conditioning; connectionist learning; discrimination learning; fear learning; incentive learning; instrumental learning; social fear learning; social learning about others, 470–71 attentional, 387 autism spectrum disorder and, 169 automaticity of, 66–68 between/within-stimulus, 173 content of, 5 discrimination-reversal, 229 effects of experiential differences on, 56 evaluative, 534 extinction, 82 gradual, 348 habit, 239–40, 251 hierarchy, 174, 175t incentive, 236 latent, 4 measurements, 524 observational, 114 from others, 464–70 performance in contrast to, 258 propositional, 410–11 as reasoning, 106 referential, 410 response, 240, 242–45 reward, 458 second, 82–83 sexual conditioning and, 522–25 signal, 410 social-emotional, 472–73 S-R associations and, 111 stimulus, 236–42, 255 structure of, 5–9 types of, 236–45 unconscious, 106 within-stimulus, 177 learning acquisition phase, 31 learning data, 390–91 learning disabilities, 173–74 learning theory addiction and, 261 autism spectrum disorder and, 171–72 aversive conditioning and, 253 contemporary, 44–45 interference effect and, 123 learned helplessness and, 152 LH. See lutenizing hormone LI. See latent inhibition limbic area, 63 lipopolysaccharide (LPS), 192 lithium chloride, 194 localist representations, 350–51 locus coeruleus (LC), 133, 200 logistic-regression techniques, 379, 390 lotteries, 322, 325–26


INDEX LPS. See lipopolysaccharide lupus erythmatosus, 197 lutenizing hormone (LH), 535 lymphocytes, 197, 200

maladaptive beliefs, 110 masking tasks, 155–56 altruism and, 425 automatic processing of CS-0 and, 160 competition and, 429 load, 160–61 massive extinction, 85 moderate extinction in contrast to, 89f in multiple contexts, 88–89 sessions, 34, 38 smoking and, 248 spaced trials, 93 match-to-sample (MTS) procedure, 175–76, 176f MDS model. See multidimensional spatial model mecamylamine, 253–54 medial prefrontal cortex (mPFC) in classical conditioning, 461 dispositional attributions and, 368 extinction retention and, 35–36 PTSD and, 141–42 SKF 38393 and, 314 in social cognition, 461, 463 memory consolidation, 38–39 declarative, 240 eating and, 295 emotional, 32 episodic, 107, 469 fear, 32 fear as cognitive structure in, 28 of food, 299 illusory correlation and, 363f inaccessible, 4 long-term, 108, 238 unavailable, 4 verbal, 142–43 men perceptions of women, 381–84 sexual aggression of, 382–84 sexual arousal in, 537 mental attribution, 466 mental process theories, of evaluative conditioning, 408–11 mental state attributions, 466–67 mere exposure, 484, 491 Met allele, 53 methadone, 250–52. See also opioids as incentive, 323–24 sign tracking with, 275 methamphetamine altering stimulant effects of, 283–84 discriminated goal tracking and, 276

2/16/2011 9:31:22 AM

INDEX Pavlovian features of, 271 mind, theory of, 170 mindfulness, 257 Minnesota Heart Disease Prevention Program, 330 misattribution account, of evaluative conditioning, 409–10 modeling, logic of, 424 monetary incentives, 323, 326–30 mood disorders, 59 morphine, 125–26. See also opioids conditioned drug tolerance and, 283 conditioned immunomodulation and, 201–4 DTA and, 275 learned helplessness and, 136 motor learning systems, 315 motor skills, 218–20, 219f mPFC. See medial prefrontal cortex mPFCv. See ventral medial prefrontal cortex MTS procedure. See match-to-sample procedure multidimensional spatial (MDS) model, 378–79, 378f muscimol, 136–37 music, 490–91, 494–95, 501

nadolol, 205 naloxone, 309, 311, 311f, 313f naltrexone, 201, 311–12 natural female features, 514–16, 515f naturalistic conditioned stimuli, 527 natural killer (NK) cells, 196 aversive stimuli and, 200 poly I:C and, 199 NE. See norepinephrine “near-miss” appraisals, 114 need for cognition, 500 negative conditioned stimulus (CS-), 50 attentional bias and, 64 PTSD and, 140 negative feedback decremental error correction and, 312 in fear conditioning, 309–12 model, of fear conditioning, 310f Pavlovian conditioning and, 306 neophobia, 141, 290 nervous system, 192 neural endophenotype, 63 neural plasticity, drug-induced, 235 neuroanatomical evidence anxiety and, 47–49 for classical conditioning, 457–61 of fear extinction, 32–33 PTSD in contrast to learned helplessness, 141–43 for social cognition, 461–64 threat attention and, 63–64 neurobiological processes in anxiety disorders, 44


565 behavioral control and, 222 for evaluative conditioning, 407 fear and, 47–49 in learned helplessness effects, 131–38 of social fear learning, 471–73 neuroimaging anxiety and, 52–53 dispositional attributions and, 367–71 drug cues and, 203 fear conditioning and, 51–52 fear extinction and, 33 instructed fear learning and, 469 social judgments and, 372 neuroimmune interactions, 192–93 neurons dopamine, 314–15 extinction, 33 fear, 33 neurotransmitter release, conditioned, 274 neutralizing behavior, 109 nicotine, 239 acquired excitatory properties and, 279–80 antagonists, 253–54 discriminated goal tracking and, 276–77 between drug conditioning and, 273 learned expectancies and, 230 patch, 251 Pavlovian features of, 271 replacement therapy, 250–52, 330 social support as substitute for, 330–31 nitric oxide, 202–4 NK cells. See natural killer cells NMDA. See N-methyl-D-asparate N-methyl-D-asparate (NMDA), 32–33, 35 N-methylnaltrexone, 201 nominal target stimulus (CS-0), 155 automatic processing of, 160 LI theory and, 159–60 nonanxious individuals, 65–66 nonreinforcement, 281–82 nonresponses, 10–11 N-opponents, 429–31, 434 noradrenergic system, 131 norepinephrine (NE) bupropion and, 254 drug-induced neural plasticity and, 235 immunostimulatory agents and, 199 learned helplessness and, 134–35 in PTSD, 31 normal behavior, 152 North Karelia program, 330 nucleus accumbens, 203, 221

obesity eating motivations and, 294 portion size and, 297 responsiveness to food cues and, 291–92

2/16/2011 9:31:22 AM

566 object recognition, 107 observation anxiety reactions and, 110 audience, 424 fear learning, in animals, 464–65 fear learning, in humans, 465–67 food preferences and, 291 functional knowledge and, 401–2 learning and, 114 observational conditioning, 57 observing response procedures, 182–83 obsessive compulsive disorder learned expectancies and, 230 neutralizing behaviors and, 109 stop-signal model of behavioral control and, 223 occasion setting, 7 in evaluative conditioning, 403–4 transfer of, 283f O-LIFE. See Oxford-Liverpool Inventory of Feelings and Experiences omission training, 3, 8 as negative contingency, 11 with sexual conditioning, 509 operant conditioning. See instrumental conditioning opioids antagonists, 311 antipain mechanisms and, 306 aversive stimuli and, 200–201 conditioned immunomodulation and, 201–5 contingency management for, 254–55 learned helplessness and, 134–35 withdrawal, 252 orientation attentional bias and, 62 spatial-cueing task and, 61 outcomes, 236. See also appetitive outcomes; aversive outcomes devaluation, 9, 13 expectancies and, 220 learned associations and, 238f modulating, 242 preexposure effects, 7 in response learning, 242–43 reward, 220–21 value of, 9 OVA. See ovalbumin ovalbumin (OVA), 199 overeating, 295, 298 overgeneralization, 28 overjustification effect, 331–32 overlearning, 248 overselectivity, 173 autism spectrum disorder and, 177–78 in comparator model, 179 extinction and, 179 extinction to combat, 183–84 interventions, 182–83 MTS procedure and, 176


INDEX sensitive comparator and, 181–82 subastymptotic performance and, 180–81 overshadowing, 7, 20n4 in advertising, 492–94 amphetamine and, 314 circuit selection and, 316 discounting and, 350 inverse, 445 negative feedback and, 311–12 one-trial, 314 overselectivity and, 180 reciprocal, 180–81 Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE), 158–59

PAG. See periaqueductal gray pain observational fear learning and, 466–67 perceptual-defensive-recuperative model and, 306–8 pain-inhibiting peptides, 306 palatability, 295–96, 298 panic, 46–47 panic disorder, 29, 58 parental modeling, 55 parsimony, 106, 483 pathological anxiety, 27 emotional processing theory and, 28 treatment of, 37 Pavlovian conditioning, 5, 79, 105, 399–401. See also classical conditioning analogs, 435–45, 446t fertilization success and, 522t with interoceptive stimuli, 285 interoceptive stimuli and, 270–71 negative-feedback and, 306 neuroimmune interactions and, 192–93 rules of correspondence and, 435–36, 436t US for, 508 Pavlovian features, 271 Pavlovian-instrumental transfer, 240–41 penile plethysmograph, 535, 535f pentobarbital, 272–73 perception of CS and US, in evaluative conditioning, 405–6 single-system propositional model and, 107 social, 468 of women, by men, 381–84 perceptual-defensive-recuperative (PDR) model, 306–8, 307f perceptual organizing blocking and, 387, 392–94, 392f, 393f category learning and, 377–81 congruence of, with category structure, 391–92, 392f in men’s learning study, 383f prototype-classification task data and, 390 spatial representation of, 378f

2/16/2011 9:31:22 AM

INDEX of women, by women, 384–95 performance comparator model and, 179 expectancy and, 104 learning in contrast to, 258 utilizing associative knowledge, 106 periaqueductal gray (PAG), 200, 308 peripheral neuropeptide Y, 205 personality advertising and, 498–501 anxiety conditioning and, 53 emotional learning and, 456 evaluative conditioning and, 406–7 fear conditioning and, 51–53 person impression formation, 351–52, 358–61, 359t assimilation and contrast and, 360–61, 361f recurrent network model of, 360f persuasive communication, 366–67 pervasive developmental disorder not otherwise specified, 168 PET. See positron emission tomography PFC. See prefrontal cortex pharmacokinetics, 250–51 pharmacological effects of cognitive enhancers, 85 LI and, 154 pharmacotherapy, 260–61 phased-learning task, 385–87, 386f, 386t prototype-classification task and, 388–89 phobias. See also specific phobias contextual cues and, 16 differences in learning, 54–59 nonassociative model of, 112 preparedness theory of, 56–57, 111 sensitization model of, 112 simple, 29 social, 45, 175 phobic objects, 80 photo stimuli, 388 picrotoxin, 136–37 placebos alcohol impairment and, 217–18 expectancy and, 215–16 R-O associations and, 259–60 place-conditioning apparatus, 280f plaque-forming cells, 196 PL region. See prelimbic region polydrug abuse, 280–81 poly I:C. See polyinosinic:polycytidylic polyinosinic:polycytidylic (poly I:C), 199 portion size, of food, 297–98 positive conditioned stimulus (CS+), 49–50 attentional bias and, 64 contextual conditioning, 57 PTSD and, 140 positivity effect, 487


567 positron emission tomography (PET), 141–42 posttraumatic stress disorder (PTSD), 29 amygdala and, 32 attentional biases and, 45 conditioning paradigms, 50 criteria for, 139 escape and, 141 evaluative conditioning and, 406–7 hippocampus and, 142–43 learned helplessness and, 121, 139–40 neophobia and, 141 neural bases of, 141–43 norepinephrine in, 31 preparedness theory of phobias and, 111–12 psychopathological applications of learned helplessness, 139 reminder cues, 129 single-system propositional model and, 110–11 sources of, 141 unpredictability and, 58 yohimbine in treatment of, 38 potentiation, 7 predatory imminence, 46 predictability, 130 prediction, 4 preexposure. See also latent inhibition CS, 492 effects, 7 in stimulus learning, 241–42 US, 491–92 prefrontal area, 53, 63 prefrontal cortex (PFC) in classical conditioning, 460–61 COMT and, 53 learned helpless effects and, 135–38 pregnancy, 332–33 prelimbic (PL) region, 135, 203 preparedness theory, of phobias, 56–57, 111–12 prior belief, 500–501 problem solving, 257, 385 product experience, 491 prohibition, 365, 366f prompting techniques, 173–74 propositional learning, 410–11. See also single-system propositional model prototype-classification task, 388–90 pseudoconditioning, 3, 18–19 psychoeducation, 258 psychological space, 378, 381 psychopathology early associative accounts, 79–80 learned helplessness, 138–43 psychostimulants, 517–19 PTSD. See posttraumatic stress disorder public policy incentives, 335

2/16/2011 9:31:22 AM

568 punishment, 3 of avoidance, 245 avoidance in contrast to, 11 contingencies, 12 instrumental contingencies of, 10t passive avoidance as, 11–12

quality of life, 335

radioligand binding, 192 Rape Justifiability Score, 382 RASHNL model, 380–81 sexual aggression of men and, 382–84 in women’s learning study, 387 ratio schedules, 12t, 13 reacquisition, 246, 277 reaction times (RTs), 60 reasoning associative learning and, 105 faulty, 110 learning as, 106 Wason selection task and, 110 recovery evaluative conditioning and, 89–90 of extinguished CR, 81, 83–84 reducing, after extinction, 84–96 spaced extinction sessions and, 94 recuperative behavior, 308–9 recurrent networks, 347f, 358, 360f reduction, 419 redundancy, in CS-US associations, 403 reexperiencing, 139 referential account, of evaluative conditioning, 410 reflexive theory, 111 reinforcement, 11 altruistic, 425 amount of, 329 in competition, 429–30 correlated analogies for, 422–23, 422f delay of, 329, 421–22, 422f, 429–30, 430f drug antagonists and, 253–54 drugs and, 235–36 habit learning and, 239 instrumental contingencies of, 10t intrinsic motivation and, 331–32 negative, 8, 10 nicotine replacement and, 250–51 of non-drug behaviors, 327–29 partial, 423, 430, 486–87 positive, 10 schedules, 12, 12t, 243, 323 self-administration of drugs and, 244 of speaking in reply, 421–23 reinforcers, 305, 322 reinstatement, 246 attentional bias and, 66f


INDEX of fear, 57–58 mental, 90 of treatment context, 84 relapse associative models of, 81–84 attenuating, 97 drug cues and, 201 experimental extinction and, 84 preventing, 84–85 single-system propositional model and, 115 through substitution, 283 triggers, 241 types of, 246–48 reminder cues, 128–29, 140–41 renewal, 247 AAC, 81–83, 85 ABA, 81–83 evaluative conditioning and, 90f massive extinction and, 85 preventing, 88 ABC, 81–83 extinction in multiple contexts and, 86–88 massive extinction and, 85 types of, 81–84 representation-mediated conditioning, 501 representations, 13–15, 28, 350–51 resampling approach, to statistical analysis, 391 Rescorla-Wagner model, 4 ABC renewal and, 82 anxiety and, 109 complex social behavior and, 418 compound conditioning, 17 delta learning algorithm and, 347 one-trial blocking and, 313 US processing and, 310 response-no-outcome (R-noO) associations, 7 response-outcome (R-O) associations, 3, 6 in animals, 9 drug antagonists in blocking, 253–54 expectancies about, 239 extinction, 260 smoking and, 236–38 treatment for, 259–60 responses conflict, 259 evaluative, 406 learned associations and, 238f positive contingencies and, 10–11 self-administration, 240 sexually conditioned, 516–17 response-stimulus (R-S) associations, 6 restrained eating, 292, 299–300 restricted access, to food, 291–92 resurgence, 247–48 retraining, of extinguished cues, 81 retrieval cues, 89–90, 249 discriminations and, 169

2/16/2011 9:31:22 AM

INDEX facilitation procedure, 5 second learning and, 82 Rett’s disorder, 168 return-of-deposit programs, 325 rewards. See also incentives centers, 221 dopamine neurons and, 315 learning, 458 outcomes, 220–21 smoking cessation and, 322 risk taking disinhibiting effects of alcohol and, 227–28 sexual, 541–42 R-noO associations. See response-no-outcome associations R-O associations. See response-outcome associations R-S associations. See response-stimulus associations RTs. See reaction times rules of correspondence, 418–19 instrumental escape conditioning and, 420, 421t Pavlovian conditioning and, 435–36, 436f

safety behaviors, 108–10 safety signals, 58, 244 salience category learning and, 381 of CS, with overshadowing, 493 of cues, 182 dimensional, 384 incentive, 239 perceived, of affect, in women, 394 perceived dimensional, 379 weights, 379–80 “Sally-Anne” task, 170 sample size acquisition and, 348–49 in causal attribution, 352–54 in dispositional attribution, 354–55 effect, 348 manipulating, 356 satiety, 293, 298 schizophrenia, 152–54 abnormal latent inhibition and, 159 distractibility and, 154 dopamine function and, 162 individual differences in cognitive processing and, 376 irrelevant stimuli and, 160 latent inhibition effects with, 156 LI and, 154–59 LI attenuation and, 161–62 LI potentiation and, 161–62 normal behavior in contrast to, 152 positive and negative symptoms, 152, 158–59, 161–62 smoking and, 324 schizotypality, 156–59 Schizotypal Personality Questionnaire (SPQ), 158–59 SCR. See skin conductance


569 S∆ , 7 S delta. See S∆ second excitor, 90–92 second-order conditioning, 15, 17, 180, 494–95 self-control disinhibited eating and, 299 modifying, 300 self-efficacy, 108 self-esteem, 139 self-injurious behavior, 183 self-judgments, 463 self-learning, 346 self-organization, 346 self-regulation, 228 self-reports, 456 sensitive comparator, 181–82 sensitization incentive, 238, 240–41 in learned helplessness, 140 model, for phobias, 112 PTSD, 140 single-stimulus presentation and, 18–19 tolerance in contrast to, 221 sensory preconditioning, 4 compound conditioning and, 17–18 within-compound associations and, 15 serotonin, 52–53, 133 serotonin transporter gene (5-HTT), 52–54 polymorphisms, 59 threat attention and, 63 SES. See Sexual Experience Scales sexual aggression individual differences in cognitive processing and, 376 of men, 382–84 sexual arousal, 535–36 in men, 537 models, 538 in women, 536–37 sexual behavior, 507–8 disinhibiting effects of alcohol and, 227–28 human, 532–33 psychostimulants and, 517–19 sexual conditioning and, 510–22 sexual compulsivity, 541–42 sexual conditioning, 275, 508–10 animal, 534–35 animal in contrast to human, 527 appetitive, 538 of approach behavior, 509f with artificial objects, 517 of body adornments, 514 contextual cues, 512–14, 512f CS factors in, 509f, 511–12 CS-US interval and, 523 effect of psychostimulants on, 517–19 fertilization success and, 520–22 field studies of, 539–40 human, 525–26, 535–36

2/16/2011 9:31:22 AM

570 sexual conditioning (Cont’d) individual differences in, 540–41 intertrial interval and, 526 learning and, 522–25 manifestations of behavior after, 523 modifications on interactions with sexual partner, 519–20 of natural female features, 514 nature of, 533–34 olfactory CS in, 539 range of responses to, 516–17 research, 534–42, 543 sexual behavior and, 510–22 signal learning and, 533–34 US factors in, 510–11 Sexual Experience Scales (SES), 540–41 Sexual Inhibition Scales /Sexual Excitation Scales (SIS/ SES), 540–41 sexual learning, 522–25 sexual partners, 519–20 sexual risk taking, 541–42 Sexual Sensation Seeking (SSS), 540–41 sheep red blood cells (SRBC), 194 shortcut strategies, 110 sickness behaviors, 192 signal learning, 410, 533–34 sign tracking. See autoshaping similarity-ratings paradigm, 377 single-stimulus presentation, 18–19 single-system propositional model, 106–16 SKF 38393, 314 skin conductance (SCR), 53 expectancy and, 107 Implicit Association Task and, 67 Skinner box, 510 sleep, 192 smokeless tobacco, 326 smoking aversive conditioning and, 252–53 combination treatment for, 251 comorbidity with alcohol, 281 coping and, 257 decline in, 321 denicotized, 260, 278–79 drugs in contrast to, 243 extinction and, 245–50 learned expectancies and, 230 massive extinction and, 248 with nicotine patch, 251 pregnancy and, 332–33 prevalence of, 321 rapid, 252, 260 reacquisition and, 246 reducing contexts for, 253, 260 R-O associations and, 236–38 scheduled reduction of, 253 schizophrenia and, 324 smokeless tobacco and, 326


INDEX spontaneous recovery and, 247 substance abuse treatment and, 250–61 smoking cessation, 236 aversive conditioning and, 252–53 bupropion for, 254 contests and lotteries for, 325–26 counseling, 257 deposit contingencies for, 325 effectiveness of, 321–22 incentives for, 324–25 programs, 333 refining incentives for, 334–35 taxes and, 326 treatment, 247 worksite interventions, 326–27 snacking, 295 S-noO associations. See stimulus-no-outcome associations S-O associations. See stimulus-outcome associations sobriety, 214. See also abstinence social analogs, 418–19 acquisition, 423 altruistic drive, 426–27 Campbell and Kraeling, 432–34, 433f causal relationship detection, 441–45 competition, 429–30 correlated reinforcement, 422–23, 422f CS-US association, 442 delayed reinforcement, 421–22, 422f dictionary of, 418–19 drive intensity, 423, 432 energization, 423–24 human agency, 441–45 instrumental conditioning, 420–35, 446t intermittent shock, 424 interpersonal attraction, 437–41 partial reinforcement, 423 Pavlovian conditioning, 435–45, 446t social cognition amygdala in, 461–62 emotional learning and, 455 frontal cortex in, 462–64 neuroanatomical bases for, 461–64 reflective in contrast to reflexive, 470 topics, 359t social connectionism, 346 social drinking, 216–17 social facilitation of eating, 297 N-opponents and, 430–31 social fear learning, 456–57 amygdala and, 472 neural model of, 459f–460f, 471–73 social judgments future research in, 371 neuroimaging and, 372 N-opponents and, 430–31 simulations, 358–70

2/16/2011 9:31:22 AM

INDEX studies, 351–58 social learning, 345–46, 371 artificial stimuli and, 396 sexual aggression and, 384 social motivation, 418 social neuroscience, 346 social perception, 468 social processes, 345–46, 371, 455 social reinforcement, 418 social skills, 171 social support, 330–31 societal status, 406 spaced training advertising and, 489 in extinction, 92–93, 93f spatial-cueing task, 61, 65 speaking in reply, 421–25 species-specific defense reactions (SSDRs), 307 specific phobias attentional biases and, 45 experimental extinction and, 80 infantile amnesia and, 112 preparedness theory of, 111 sperm depletion effect, 522 spleen denervation, 198 spontaneous recovery, 3, 7, 247 with second excitor, 91 theoretical accounts of, 83 SPQ. See Schizotypal Personality Questionnaire spreading activation models, 346 S-R associations. See stimulus-response associations SRBC. See sheep red blood cells S-S associations. See stimulus-stimulus associations SSDRs. See species-specific defense reactions SSRT. See stop signal reaction time SSS. See Sexual Sensation Seeking startle reflex, 47–49 contextually modulated, 59 fear-potentiated, 47, 57 potentiation of, 55f sensitization and, 140 statistical analysis, resampling approach to, 391 stereotype-inconsistent information, 364–65, 364f stereotyping, 361 stimulant effects, 283–84 stimuli. See also interoceptive stimuli artificial, 396 aversive, 199–201 category learning and, 380–81 contrast effects to, 243–44 differentiation, 173 dimensional metrics for, 378–79 drug states as, 271–78 environmental, 8 in evaluative conditioning, 404–6 exteroceptive, 274 fear, 84 irrelevant, 160


571 learned associations and, 238f local in contrast to global processing of, 177 naturalistic conditioned, 527 nondrug, during withdrawal, 251 Pavlovian, 271–78 perceptual organization of, 378 phobic, 111 photo, 388 pre-exposure, 241–42 prompts, 173–74 social in contrast to nonsocial, 440–41 threat-relevant, 44–45 trauma-related, 29 unattended, 162n3 stimulus-no-outcome (S-noO) associations, 7 stimulus-outcome (S-O) associations, 3, 6 cue exposure and, 255 expectancies about, 239 treatment for, 260 stimulus-response (S-R) associations, 3 addiction and, 240 in animals, 8–9 cue exposure and, 255 drugs and, 9, 215 learning and, 111 treatment and, 258–59 stimulus-stimulus (S-S) associations classical conditioning, 5–6 implicit misattribution in, 409 stop-signal model, of behavioral control, 222–23 stop signal reaction time (SSRT), 223 stress attenuated LI and, 159 controllability of, 128f corticosteroids and, 130 disinhibited eating and, 299–300 emotional learning and, 456 induced analgesia, 308 learned helplessness and, 130 responsiveness to food cues and, 292 time, 424 wound healing and, 200 stress-induced ulceration paradigms, 58 striatum, 458, 473 Stroop task, emotional, 60 substance abuse. See also addiction; alcohol; drugs memory consolidation and, 38–39 sexual compulsivity and, 542 treatment, 250–61 substitutability, 327–28 substitution. See also drug replacement therapy drugs, 327–28 functional in contrast to pharmacological, 282–83 nicotine replacement as, 330 social support as, 330–31 subtyping, 364–65 summation, 82 summation and retardation test, 284

2/16/2011 9:31:22 AM

572 superconditioning, 350 attraction, 439–40, 440f causal relationship detection and, 443–44, 443f human agency and, 443–44 support groups, 327 surprise, 310 symbolic representations, 350 sympathetic nervous system, 46

taste aversion, 193–94, 275 taxes, for smoking cessation, 326 teaching, 5 team competition, 327 temperament, 53 temporal weighting, 83 temporo-parietal junction (TPJ), 368 terminology, 48t, 237t testability, of dual-system model of learning, 105–6 test environments, 127–28 testosterone, 535 thalamus, 458 theory of mind, 170 threat appraisal, 108, 115 attentional biases for, 44–45, 59–64 cognitive dissonance and, 365 detection, 67 disengagement, 61 ego, 300 emotional learning and, 454 imminence, 47 prohibition and, 366f responsiveness, 63 stimuli, 44–45 threat beliefs, 104 reducing, 116 single-system propositional model and, 107–9 time stress, 424 tobacco. See smoking tolerance alcohol, 213, 218–21 conditioned drug, 273–74 distress, 257 drug, 221 trace conditioning, 252, 485 training interval in contrast to ratio, 243 reinforcement schedules and, 243 trait anxiety, 53 amygdala and, 63 anxiety disorders in contrast to, 69 attentional biases and, 59, 68–69 trait-implying behavior, 360 transfer affect, 496 data, 390 of occasion setting, 283f


INDEX Pavlovian-instrumental, 240–41 positive, 4 rules, 123 trauma coping with, 144 PTSD and, 29 treatment anxiety, 37, 113–15 autism spectrum disorder, 183–84 combination, 251 context, reinstatement of, 84 desensitizing, 16 drug replacement, 250–52 integrated addiction, 261–62 novel, 258 pharmacotherapy, 260–61 psychosocial, 256–58 PTSD, 38 for R-O associations, 259–60 smoking cessation, 247 for S-O associations, 260 spacing of, 247 for S-R associations, 258–59 substance abuse, 250–61 Treatment Episode Data Set, 281 triadic design, 122t, 125 triggers identifying, 257 relapse, 241 trinitrophenyl (TNP), 196 two-card discrimination procedure, 175f two-factor theory, 7–8

unconditioned stimuli (US) arrangement of, in time, 485–86 causal relationship detection and, 442 during extinction, 94–95, 95 extinction and, 15–16 factors, in sexual conditioning, 510–11 food as, 507 human agency and, 442 intrinsic relation of, with CS, 405–6 modality and semantic category of, 404 only, in evaluative conditioning, 403 in Pavlovian conditioning, 399, 507 postconditioning presentations of, 15–16 postextinction presentations of, 81 preexposure, in advertising, 491–92 presentation of, in evaluative conditioning, 406 processing, 310, 313–14 reinstatement, 16 relationship to CS of, 486 valence of, 404–5 unconscious processes, 107 unpredictability, 58–59 US. See unconditioned stimuli utilization coefficients, 380

2/16/2011 9:31:22 AM

INDEX vaginal photoplethysmograph, 535, 536f vaginal pulse amplitude (VPA), 535 Val158 allele, 53 valence, of US, 404–5 varenicline, 254 variety effect, 298–99 ventral medial prefrontal cortex (mPFCv), 135–36, 138f, 141–42 verbal instruction, 105 verbal interventions, 114. See also cognitive therapy veterans, 141–42 virtual reality exposure (VRE), 38 visual search tasks, 61, 67 vouchers, 326–28 VPA. See vaginal pulse amplitude VRE. See virtual reality exposure

Wason selection task, 110 withdrawal from drugs, 235–36


573 nicotine, 239 nondrug stimuli during, 251 opioids, 252 within-compound associations, 14–15 within-session habituation, 28 cognitive processes in, 36 efficacy of, 38 fear reduction and, 33–34 implications of, 37 women appetitive sexual conditioning in, 538 attentional shifts in, 394 eating disorders and, 384–85 men’s perceptions of, 381–84 perceived salience of affect in, 394 perception of women by, 384–95 sexual arousal in, 536–37

yohimbine, 31, 38, 261

2/16/2011 9:31:22 AM

Associative Learning and Conditioning Theory: Human and Non-Human Applications

Nonhuman DNA Typing: Theory and Casework Applications

New Directions in Human Associative Learning

Human Learning and Memory

Conditioning for Strength and Human Performance

Instantaneous Power Theory and Applications to Power Conditioning

The Psychology of Associative Learning

Human Behavior Learning and Transfer

Gestural Communication in Nonhuman and Human Primates (Benjamins Current Topics)

Air-conditioning and refrigeration

Gas conditioning and processing: Advanced techniques and applications (Volume 3)

UWB: Theory and Applications

Matrices: Theory and Applications

Graph Theory and Applications

Bifurcation Theory and Applications

Supermanifolds : theory and applications

Superconductivity – Theory and Applications

Rigidity theory and applications

Nucleation Theory and Applications

Multifractals. Theory and applications

Matrices theory and applications

Wavelets: Theory and applications

Graph Theory and Applications

Matrices: theory and applications

Herbicides, Theory and Applications

Multifractals: Theory and Applications

Action Learning and its Applications

Plasticity: Theory and Applications

Supermanifolds: theory and applications

Buildings: Theory and applications

Games, Theory and Applications

Associative Learning and Conditioning Theory: Human and Non-Human Applications

Nonhuman DNA Typing: Theory and Casework Applications

New Directions in Human Associative Learning

Human Learning and Memory

Conditioning for Strength and Human Performance

Instantaneous Power Theory and Applications to Power Conditioning

The Psychology of Associative Learning

Human Behavior Learning and Transfer

Gestural Communication in Nonhuman and Human Primates (Benjamins Current Topics)

Air-conditioning and refrigeration

Gas conditioning and processing: Advanced techniques and applications (Volume 3)

UWB: Theory and Applications

Matrices: Theory and Applications

Graph Theory and Applications

Bifurcation Theory and Applications

Supermanifolds : theory and applications

Superconductivity – Theory and Applications

Rigidity theory and applications

Nucleation Theory and Applications

Multifractals. Theory and applications

Matrices theory and applications

Wavelets: Theory and applications

Graph Theory and Applications

Matrices: theory and applications

Herbicides, Theory and Applications

Multifractals: Theory and Applications

Action Learning and its Applications

Plasticity: Theory and Applications

Supermanifolds: theory and applications

Buildings: Theory and applications

Games, Theory and Applications

Recommend Documents