Psychophysiology, 48 (2011), 437–440. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01084.x
BRIEF REPORT
Interstimulus intervals for skin conductance response measurement
ASSAF BRESKA, KEREN MAOZ, and GERSHON BEN-SHAKHAR Department of Psychology, The Hebrew University of Jerusalem, Jerusalem, Israel
Abstract This study examined, using the Concealed Information paradigm, whether interstimulus intervals (ISI) typically used for electrodermal measurement can be shortened. An ISI ranging from 16 s to 24 s (with a mean of 20 s) was compared with an ISI shortened by 50% using a within-participants design. It was demonstrated that this shortening had no effect on the differential skin conductance responses to the personally significant details and nearly identical detection efficiency was observed under the 2 ISIs. However, overall responses were attenuated with the shorter ISI. The implications of these results for various types of studies, using skin conductance responses, were discussed. Descriptors: Electrodermal responses, Concealed Information Test, Interstimulus intervals
they have to sit still and do nothing. In addition, using long ISIs is problematic for measuring SCR along with other measures that require much shorter ISIs (e.g., ERP, response time). Using these measures in the same experiment leads to suboptimal measurement of one or all measures (see Gamer & Berti, 2010). Recent attempts to overcome these difficulties and apply SCR measurement with much shorter ISIs (i.e., 2–3 s) have relied on time series analyses of the skin conductance signals, designed to separate the individual SCRs (Alexander et al., 2005) or on an attempt to model the SCR waveform by a combination of a sigmoid function and an exponential decay function, which were fitted to the data (Lim et al., 1997, 1999). However, these attempts rely on various assumptions that do not necessarily hold in many applications. Several studies measured SCRs with short ISIs in a go/no-go task. Barry and Rushby (2006) reported larger SCRs for target stimuli with an ISI of 1.1 s. No systematic comparison with longer ISIs was conducted in this study, but the authors reported that SCRs’ magnitudes were substantially smaller than is common in OR research. Recio, Schacht, and Sommer (2009) varied ISIs between 2, 5, and 8 s and found similar stimulus significance effect, independent of ISI. No comparison with standard ISIs was conducted in this study, but smaller SCR amplitudes were found in the 2-s ISI. In addition, using the Concealed Information Test (CIT) paradigm, Gamer, Bauermann, Stoeter, and Vossel (2007) and Gamer, Klimecki, Bauermann, Stoeter, and Vosser (in press) measured SCRs with ISIs ranging between 5.2 and 18.6 s and obtained the expected enhanced responses to the critical items. However, the authors did not examine whether their results were affected by ISI. The purpose of this short report is to conduct a systematic comparison between the standard ISI and a shorter one (8–12 s) and examine whether the shorter ISI interferes with proper SCR
Electrodermal measures, and in particular an increase in skin conductance level following the onset of a stimulus (skin conductance response [SCR]), have been used extensively in psychological and psychophysiological research for a variety of purposes. Specifically, SCRs have been used in the study of orienting responses (OR; e.g., Siddle, 1991; Sokolov, 1963); in research on autonomic conditioning (e.g., Dawson & Biferno, 1973), psychopathology, and, in particular, schizophrenia (e.g., Bernstein et al., 1988); and more recently to assess the role of emotions in decision making (e.g., Bechara, Tranel, Demasio, & Demasio, 1996). In addition, they have been extensively used in applied settings for the detection of concealed knowledge (e.g., Ben-Shakhar & Elaad, 2003; Lykken, 1960) and biofeedback (e.g., Birbaumer & Kimmel, 1979; Nagai, Goldstein, Fenwick, & Trimble, 2004). As changes in skin conductance level following stimulus onset are slowly developed and after reaching a peak slowly return to baseline, it has been suggested that long interstimulus intervals (ISIs) should be applied to properly separate between responses to consecutive stimuli. Typically, ISIs for SCR measurement range between 20 and 60 s (e.g., Dawson, Schell, & Filion, 2000). However, these long ISIs have various disadvantageous, as they lead to a significant lengthening of the experiments or, alternatively, they limit the number of stimuli that can be presented. Furthermore, participants in these experiments find them boring and sometimes even fall asleep during these long intervals where This research was funded by a grant from the Israel Science Foundation to Gershon Ben-Shakhar. We thank Tamar Pelet for her assistance in this research. Address correspondence to: Gershon Ben-Shakhar, Department of Psychology, The Hebrew University of Jerusalem, Jerusalem, Israel. E-mail:
[email protected] 437
438 measurement. We used the CIT as a convenient paradigm (BenShakhar & Elaad, 2003), where a great deal of information is available regarding the expected effects (SCR differences between critical and neutral control items). Specifically, the concealed items chosen for this experiment were autobiographical details (e.g., Ben-Shakhar & Elaad, 2002; Lykken, 1960). A within-subjects design, with two blocks (a block of short ISIs and a block of long ISIs) was adopted, with their order counterbalanced across participants. The criterion for comparing the two conditions was the size of the CIT effect measured by the standardized mean SCR difference between critical and neutral items. In addition, we examined whether general responsivity to all items is affected by the ISI.
Methods Participants Thirty-six Hebrew University students (19 women, mean age 25.3 years) participated in a 45-min experiment for course credit or payment. All participants had normal or corrected-to-normal vision and were native Hebrew speakers. They were briefed about the nature of the SCR measurement and signed an informed consent. Apparatus The experiment was conducted in a dimly lit, sound-attenuated room. Two Ag/AgCL electrodes (0.8-cm diameter) were filled with a conductive paste (K-Y jelly; Johnson & Johnson, France), and attached to the distal phalanges of the index and fourth fingers of the participant’s left hand. Skin conductance was recorded using a standard constant voltage system of 0.5 V (DAS-1; Atlas Researchers, Israel), and recordings were continuously digitized by an A/D converter with a sampling rate of 20 Hz. To minimize motion artifacts, participants’ hands rested on chair hand-rests, and responses were given vocally. Participants’ verbal responses were recorded by a small microphone attached to the collar of their shirt and connected to the computer. Stimuli were presented on a CRT monitor connected to a PC computer, which also recorded SCRs. Stimuli The stimuli were personally significant (PS) or neutral names from three categories (first names, family names, and mothers’ names). In each category, the PS stimulus was the participant’s own biographical item and the neutral-control stimuli were four names of the same category, matched in length to the PS stimulus. Stimuli were presented in the same spatial location in the center of the computer screen, on a black background. Each letter was approximately 0.7 cm in height by 0.6 cm in width. The font was Ariel, and the viewing distance was 80 cm. Design A within-subjects design with the following two conditions was employed: (1) the ‘‘long ISI’’ condition, where the interval between the offset of one stimulus and the onset of the next ranged between 16 s and 24 s (with a mean ISI of 20 s). This is the standard ISI that has been used in our laboratory for both CIT and other psychophysiological experiments that used electrodermal measures (e.g., Ben-Shakhar & Elaad, 2002; Ben-Shakhar & Gati, 2003; Gamer, Verscheuere, Crombez, & Vossel, 2008). (2)
A. Breska et al. The ‘‘short ISI’’ condition, where the interval between the offset of one stimulus and the onset of the next ranged between 8 s and 12 s (with a mean ISI of 10 s). The experiment was conducted in two blocks, such that each condition was run on the first block for half of the participants and on the second block for the other participants. Procedure Prior to the experiment, participants filled out a questionnaire on which they denoted a list of close family names and seven additional names of people who are meaningful for them. These names were disqualified from being used as neutral names in the experiment. Following attachment of the microphone and electrodes, the experiment began with a 2-min baseline recording during which participants were requested to sit quietly and relax. This was followed by two experimental blocks with a 1-min rest between them. In each block, 33 stimuli were presented. These stimuli were divided into the three categories, such that the order of the categories within each block was counterbalanced across participants, but for each participant the same order was used in the two blocks. Before presenting the individual stimuli of each category, an announcement of the category appeared on the center of the screen. Then, the five names were presented twice in a random order within each repetition, except that the significant name was never the first. A buffer name (a fifth neutral name) designed to absorb the initial orienting response was presented before the two repetitions of the five names. Thus each category included 11 presentations of names. Each name appeared on the screen for 5 s and the ISI was set according to the experimental condition. Participants were required to answer ‘‘NO’’ to each name. At the end of the second block, participants were released from the electrodes, debriefed, and paid. Response Scoring and Analysis The SCR was defined as the maximal conductance increase obtained from the examinee, from 1 s to 5 s after stimulus onset. To eliminate individual differences in responsivity and permit a meaningful summation of the responses of different participants, each examinee’s conductance changes were transformed into within-subjects standard scores (Ben-Shakhar, 1985), computed relative to the mean and standard deviation of the examinee’s response distribution within each block and each category.
Results As a preliminary analysis of the standardized SCRs indicated that the stimulus category factor had neither statistically significant main nor interaction effects with the other factors, the data were collapsed across the three categories. Table 1 displays the mean standardized SCRs as a function of stimulus significance, experimental condition, and block as well as effect size estimates, computed within each combination of ISI and block. The standardized SCRs to PS stimuli1 were subjected to a two-way mixed analysis of variance (ANOVA) with the group (participants for whom the short condition was administered in the first block vs. 1 Responses to neutral stimuli were not included in the analysis because of their inherent dependence with the personally significant stimuli following the standardization procedure. As in this design the dependent measure was the standardized SCR to the personally significant stimuli, the effect of stimulus significance is reflected by the size of the intercept.
Interstimulus intervals for SCR measurement
439
Table 1. Mean Standardized Responses (Standard Deviations in Parentheses) Elicited by Personally Significant (PS) and Neutral Control Stimuli and Effect Size Estimates (d) as a Function of Experimental Condition and Block Short ISI condition
Block 1 Block 2
Long ISI condition
PS
Neutral
d
PS
Neutral
d
0.93 (0.49) 0.55 (0.64)
" 0.29 (0.10) " 0.18 (0.12)
2.18
1.01 (0.65) 0.54 (0.56)
" 0.28 (0.16) " 0.14 (0.12)
1.68
0.97
1.06
Note: Effect size (d) defined as the mean standardized SCR difference between PS and neutral stimuli divided by the standard deviation of these SCR differences.
those for whom it was administered in the second) serving as a between-subjects factor and ISI (short vs. long) as a within-subjects factor. The results revealed that the PS stimuli elicited enhanced SCRs relative to the neutral stimuli as reflected by a positive and statistically significant intercept, F(1,34) 5 83.7, po.01, MSE 5 0.49, Cohen’s f 5 1.52. Crucially, this analysis did not yield a statistically significant main effect for the ISI, F(1,34) 5 0.15, MSE 5 0.20, p4.5, indicating that the relative responses to personally significant stimuli did not differ between the two ISIs. Furthermore, inspection of Table 1 reveals that, across blocks, the average effect sizes for the short (1.57) and long ISIs (1.37) are very similar. In addition, the ISI ! Group interaction yielded a statistically significant result, F(1,34) 5 16.6, MSE 5 0.20, f 5 0.47), indicating that relative responses to the significant stimuli decreased from the first to the second block (the average d values were 1.93 and 1.02 for the first and the second blocks, respectively). An ANOVA conducted on the raw SCRs for both PS and neutral stimuli revealed similar results. Specifically, there was a statistically significant main effect for the stimulus significance factor (the mean SCRs for the PS names and the neutral names were 1.14 and 0.54, respectively); F(1,34) 5 36.3, po.01, MSE 5 1.056, f 5 0.7. More importantly, the ISI ! Stimulus significance interaction produced Fo1 (in the short ISI, the mean SCRs for PS and neutral names were 0.97 and 0.43, respectively, and in the long ISI, they were 1.30 and 0.65); F(1,34) 5 0.89, p4.3, MSE 5 0.382). Furthermore, the effect sizes computed across blocks in the short (0.82) and long (0.96) ISI conditions were very similar. However, the overall SCRs to both types of stimuli were significantly smaller in the short (M 5 0.70) than in the long (M 5 0.97) ISI condition, F(1,34) 5 9.8, MSE 5 0.82, f 5 0.35.
Discussion The results of this experiment clearly indicate that the two types of ISI used lead to very similar detection efficiency. This conclusion is not based just on a failure to reject the null hypothesis at an accepted level of significance, but on the fact that the effect sizes of stimulus significance were very similar in the two conditions (1.57 and 1.37 for the short and long ISIs, respectively). These effect sizes are also very similar to the average effect size of 1.58 reported in the meta-analysis of CIT studies, with autobiographical details (Ben-Shakhar & Elaad, 2003), indicating that
the present results are compatible with many other experiments where a similar paradigm was used. In addition, although the differential responses to the critical items tended to habituate, this habituation was not affected by the ISI. This finding, which is inconsistent with earlier reports by Elaad and Ben-Shakhar (1997) and Ben-Shakhar and Elaad (2002), indicates that the within-blocks and categories standardization did not eliminate the effect of habituation. On the other hand, our results also demonstrate that the overall responding (regardless of stimulus type) was lower with the shorter ISI. This finding, which strengthens the conclusions drawn by Barry and Rushby (2006) and Recio et al. (2009), who used the go/no-go paradigm, implies that the choice of ISI should depend on the purpose of each study. When the focus of the study is on differential responding to one type of stimuli relative to others (e.g., novel stimuli, significant stimuli, conditioned stimuli, targets vs. nontargets), the ISI required for an efficient SCR measurement may be significantly shortened. Our findings indicate that in this type of study the experiments’ duration can be shortened by almost 50% or, more importantly, the number of stimuli presented can be doubled. Although this is definitely desirable, it is doubtful whether a 10-s ISI is sufficient for the simultaneous measurement of SCRs and ERPs. The use of shorter ISIs is important not just for laboratory experiments, but also for practical usages of electrodermal measurement. In the CITcontext, electrodermal measures have been demonstrated to provide the most efficient of all autonomic measures (e.g., Cutrow, Parks, Lucas, & Thomas, 1972; Gamer et al., 2008), and thus an optimal usage of SCRs for the detection of concealed information has important practical implications. As demonstrated in the meta-analysis reported by Ben-Shakhar and Elaad (2003), a large number of CIT questions and several repetitions of each question are required to achieve good sensitivity and specificity of CIT outcomes. However, the use of at least five different CIT questions (as recommended by BenShakhar & Elaad, 2003) with several repetitions of each question under the commonly applied ISIs may result in an unusually long test. Furthermore, it is doubtful whether the examinees’ attention can be kept for such long durations. Cutting the ISI by 50% would allow for a significant increase in the number of questions and repetitions and thus improve CIT detection efficiency. On the other hand, when the research is focused on overall SCR responding (e.g., comparing psychophysiological responding of different populations or under different emotional conditions), the use of shorter ISIs may attenuate the responses, and it is thus not recommended. Clearly, we cannot be certain as to whether the results of this experiment can be generalized to other paradigms where differential SCRs are the focus of interest, and this may need further research. On the other hand, the differences between the CIT paradigm, where a stimulus sequence, which includes both significant and neutral stimuli, is presented and paradigms such as conditioning (where both conditioned and neutral stimuli are used) and orienting habituation (where novel and nonnovel stimuli are used) do not seem to be great. Specifically, there is no reason to suspect that the SCR decay function will differ for personally significant, conditioned, and novel stimuli. The results obtained by both Barry and Rushby (2006) and Recio et al. (2009) with the go/ no-go paradigm provide some support for the generalizability of the present findings. Thus, we cautiously suggest that shortening the ISI may be helpful in various other paradigms in which differential SCR is used as an important dependent measure.
440
A. Breska et al. REFERENCES
Alexander, D. M., Trengove, C., Johnston, P., Cooper, T., August, J. P., & Gordon, E. (2005). Separating individual skin conductance responses in a short interstimulus-interval paradigm. Journal of Neuroscience Methods, 146, 116–123. Barry, R. J., & Rushby, J. A. (2006). An orienting reflex perspective on anterioristation of the P3 of the event-related potential. Experimental Brain Research, 173, 539–545. Bechara, A., Tranel, D., Demasio, H., & Demasio, A. (1996). Failure to respond autonomically to anticipated future outcomes following damage to the prefrontal cortex. Cerebral Cortex, 6, 215–225. Ben-Shakhar, G. (1985). Standardization within individuals: A simple method to neutralize individual differences in psychophysiological responsivity. Psychophysiology, 22, 292–299. Ben-Shakhar, G., & Elaad, E. (2002). Effects of questions’ repetition and variation on the efficiency of the guilty knowledge test: A reexamination. Journal of Applied Psychology, 87, 972–977. Ben-Shakhar, G., & Elaad, E. (2003). The validity of psychophysiological detection of deception with the Guilty Knowledge Test: A metaanalytic review. Journal of Applied Psychology, 88, 131–151. Ben-Shakhar, G., & Gati, I. (2003). The effects of serial position and frequency of presentation of common stimulus features on orienting response reinstatement. Psychophysiology, 40, 139–145. Bernstein, A. S., Riedel, J. A., Graae, F., Seidman, P., Steele, H., Connolly, J., et al. (1988). Schizophrenia is associated with altered orienting activity: Depression with electroderma (cholinergic?) deficit and normal orienting response. Journal of Abnormal Psychology, 97, 3012. Birbaumer, N., & Kimmel, H. D. (Eds.). (1979). Biofeedback and selfcontrol. Hillsdale, NJ: Erlbaum. Cutrow, R. J., Parks, A., Lucas, N., & Thomas, K. (1972). The objective use of multiple physiological indices in the detection of deception. Psychophysiology, 9, 578–587. Dawson, M. E., & Biferno, M. A. (1973). Concurrent measurement of awareness and classical conditioning. Journal of Experimental Psychology, 101, 55–62. Dawson, M. E., Schell, A. M., & Filion, D. L. (2000). The electrodermal system. In J. T. Cacioppo, L. J. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed, pp. 200–223). Cambridge, UK: Cambridge University Press.
Elaad, E., & Ben-Shakhar, G. (1997). Effects of item repetitions and variations on the efficiency of the guilty knowledge test. Psychophysiology, 34, 587–596. Gamer, M., & Berti, S. (2010). Task relevance and recognition of concealed information have different influences on electrodermal activity and event-elated brain potentials. Psychophysiology, 47, 355–364. Gamer, M., Bauermann, T., Stoeter, P., & Vossel, G. (2007). Covariation among fMRI, skin conductance and behavioral data during processing of concealed information. Human Brain Mapping, 28, 1287–1301. Gamer, M., Klimecki, O., Bauermann, T., Stoeter, P., & Vossel, G. (in press). fMRI-activation patterns in the detection of concealed information rely on memory-related effects. Social, Cognitive and Affective Neuroscience. Gamer, M., Verschuere, B., Crombez, G., & Vossel, G. (2008). Combining physiological measures in the detection of concealed information. Physiology & Behavior, 95, 333–340. Lim, C. L., Gordon, E., Rennie, C., Wright, J. J., Baharamali, H., Li, W. M., et al. (1999). Dynamics of SCR, EEG and ERP in an oddball paradigm with short interstimulus intervals. Psychophysiology, 36, 543–551. Lim, C. L., Rennie, C., Barry, R. J., Baharamali, H., Lazzaro, I., Manor, B., et al. (1997). Decomposing skin conductance into tonic and phasic components. International Journal of Psychophysiology, 25, 97–107. Lykken, D. T. (1960). The validity of the guilty knowledge test: The effects of faking. Journal of Applied Psychology, 44, 258–262. Nagai, Y., Goldstein, A. L., Fenwick, P. B. C., & Trimble, M. R. (2004). Clinical efficacy of galvanic skin response biofeedback training in reducing seizures in adult epilepsy: A preliminary randomized controlled study. Epilepsy and Behavior, 5, 216–223. Recio, G., Schacht, A., & Sommer, W. (2009). Effects of inter-stimulus interval on skin conductance responses and event-related potentials in a Go/NoGo task. Biological Psychology, 80, 246–250. Siddle, D. A. T. (1991). Orienting, habituation, and resource allocation: An associative analysis. Psychophysiology, 28, 245–259. Sokolov, E. N. (1963). Perception and the conditioned reflex. New York: Macmillan. (Received April 21, 2010; Accepted May 28, 2010)
Psychophysiology, 48 (2011), 441–452. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01118.x
Stressed out? Associations between perceived and physiological stress responses in adolescents: The TRAILS study
ALBERTINE J. OLDEHINKEL,a,b JOHAN ORMEL,a NIENKE M. BOSCH,a ESTHER M. C. BOUMA,a ARIE M. VAN ROON,c JUDITH G. M. ROSMALEN,a and HARRIE¨TTE RIESEa,d a Interdisciplinary Center for Psychiatric Epidemiology and Graduate School of Behavioral and Cognitive Neurosciences and for Health Research, University Medical Center, University of Groningen, Groningen, The Netherlands b Department of Child and Adolescent Psychiatry, Erasmus Medical CenterFSophia Children’s Hospital Rotterdam, Rotterdam, The Netherlands c Department of Internal Medicine, University Medical Center, University of Groningen, Groningen, The Netherlands d Unit of Genetic Epidemiology and Bioinformatics, Department of Epidemiology, University Medical Center, University of Groningen, Groningen, The Netherlands
Abstract Studies regarding the interrelation of perceived and physiological stress indices have shown diverging results. Using a population sample of adolescents (N 5 715, 50.9% girls, mean age 16.11 years, SD 5 0.59), we tested three hypotheses: (1) perceived responses during social stress covary with concurrent physiological stress responses; (2) high pretest levels of perceived stress predict large physiological responses; and (3) large physiological responses to social stress predict low posttest perceived stress levels. Perceived arousal, unpleasantness, and dominance were related to heart rate, respiratory sinus arrhythmia, and cortisol responses to a laboratory social stress test. Although effect sizes were small, the results suggest covariation of perceived stress and concurrent physiological stress responses in both the ANS and the HPA axis, as well as inverse associations between heart rate responsiveness and the subsequent appraisal of stress. Descriptors: Stress-reactivity, Heart rate, Cortisol, Self-report
Stress is an umbrella term which designates divergent symptoms such as rapid heartbeat, dizziness, pains, nervousness, agitation, irritability, worrying, concentration problems, and moodiness.
That all of these symptoms are referred to as stress suggests that they reflect a single underlying mechanism. The extent to which various stress indicators are actually related to each other determines the generalizability of a single stress measure to stress in a broader sense. Because physiological stress indices are harder to assess than psychological ones, perceived stress is often the initial or even only measure of states of stress, both in research and in clinical practice. It is therefore important to assess whether and how various psychological and physiological stress indices are interrelated. This study explores these interrelationships in a large sample of adolescents. In the first half of the twentieth century, Selye, often considered the father of stress research, discovered that a variety of different physical stimuli (e.g., cold, pain, toxic agents, extracts of organs) led to similar physical consequences, that is, degeneration of lymphatic structures, gastric ulceration, and increased activity of the adrenal cortex. He postulated these responses to be universal and non-specific, and called them the general adaptation syndrome or GAS (e.g., Selye, 1936). Selye’s notion of a universal stress response has been criticized for being an oversimplification of the reality. Mason (1968, 1971) and others after him (e.g., Dickerson & Kemeny, 2004; McCarty & Gold, 1996) noted that not all stress phenomena are nonspecific: some are only triggered if the stimulus requires specific demands to be met. In other words, stress systems may respond to variable degrees and in variable combinations to stressors, depending on
This research is part of the TRacking Adolescents’ Individual Lives Survey (TRAILS). Participating centers of TRAILS include various departments of the University Medical Center and University of Groningen, the Erasmus University Medical Center Rotterdam, the University of Utrecht, the Radboud Medical Center Nijmegen, and the Parnassia Bavo group, all in The Netherlands. TRAILS has been financially supported by various grants from the Netherlands Organization for Scientific Research NWO (Medical Research Council program grant GB-MW 940-38-011; ZonMW Brainpower grant 100-001-004; ZonMw Risk Behavior and Dependence grants 60-60600-98-018 and 60-60600-97-118; ZonMw Culture and Health grant 261-98-710; Social Sciences Council medium-sized investment grants GB-MaGW 480-01-006 and GBMaGW 480-07-001; Social Sciences Council project grants GB-MaGW 457-03-018, GB-MaGW 452-04-314, and GB-MaGW 452-06-004; NWO large-sized investment grant 175.010.2003.005; the Sophia Foundation for Medical Research (projects 301 and 393), the Dutch Ministry of Justice (WODC), the European Science Foundation (EuroSTRESS project FP-006), and the participating universities. We are grateful to all adolescents, their parents, and teachers who participated in this research and to everyone who worked on this project and made it possible. Address correspondence to: Albertine J. Oldehinkel, Interdisciplinary Center for Psychiatric Epidemiology, University Medical Center Groningen, CC72, P.O. Box 30.001, 9700 RB Groningen, The Netherlands. E-mail:
[email protected] 441
442 the nature of the stressor (Ulrich-Lai & Herman, 2009). In fact, there is increasing evidence that the two major stress systems of the body, the autonomic nervous system (ANS) and the hypothalamic-pituitary-adrenal (HPA) axis, are more dissociated than is often assumed: high ANS reactivity does not necessarily imply high HPA-axis reactivity (e.g., Gerra et al., 2001; Schommer, Hellhammer, & Kirschbaum, 2003), and vice versa. Selye was also criticized because he had excluded psychosocial stressors from his research, and ignored that a stressor may also evoke emotional arousal. Mason (1971) and Mikhail (1981) proposed that, rather than the stressor itself, the emotional response to the stressor generates stress phenomena. Lazarus and Folkman (Lazarus, 1966; Lazarus & Folkman, 1984) also focused on the psychological dimension of the stress response. They emphasized the importance of the appraisal of the situation and stated that physiological stress phenomena appear only if the situation is perceived as potentially damaging and hard to manage. Today, the psychological processes provoked by a (psychosocial) stressor are still believed to constitute the bridge between stressor and stress response (Van Praag, De Kloet, & Van Os, 2004). If the appraisal of the potentially stressful stimulus is the major determinant of the stress response, one might expect a strong positive association between the perceived stressfulness of a situation and the strength of the physiological stress responses. This hypothesis is consistent with the linkage of the ANS and HPAaxis with cortical and limbic structures, important mediators of subjectively experienced stress (e.g., Buijs & Van Eden, 2000; Schlotz et al., 2008). However, despite the intuitive and neurological plausibility of a close link between perceived stress and physiological stress responses, associations reported between the two are generally weak and divergent (Cohen et al., 2000; Hjortskov, Garde, Ørbæk, & Hansen, 2004; Lackschewitz, Hu¨ther, & Kro¨ner-Herwig, 2008; Schlotz et al., 2008). Schommer and colleagues noted that ‘‘this dissociation between subjective and biological indices of stress is most interesting from a psychosomatic point of view. Unfortunately, few experimental data are available to help explain why outflow from these different response levels hardly ever converges consistently’’ (Schommer et al., 2003, p. 458). Indeed, interrelationships between psychological and physiological stress indices have rarely been examined systematically, with a few notable exceptions. Al’ Absi et al. (1997) evaluated cardiovascular, HPA-axis, and psychological responses to public speaking and mental arithmetic, and found substantial correlations between psychological and HPAaxis responses, particularly during public speaking. By contrast, Gaab, Rohleder, Nater, and Ehlert (2005) reported that cortisol responses to social stress were particularly associated with anticipatory stress appraisal (perceived threat), not with (retrospective) ratings of perceived stress during the test. Schlotz et al. (2008) reported positive correlations between psychological stress measures and cortisol levels when psychological stress was assessed before cortisol, and negative correlations when the order was reversed. Though intriguing, these findings need replication and extension, not only because of the partly contradictory reports regarding temporal processes, but also because of methodological limitations of the studies. Al’ Absi et al.’s and Gaab et al.’s results were based on small (N 5 52 and N 5 81, respectively) samples of male volunteers, while all females (58%) in Schlotz’ study (total N 5 219) used oral contraceptives, which are known to affect cortisol responses (e.g., Bouma, Riese, Ormel, Verhulst, & Oldehinkel, 2009). Furthermore, Gaab et al.
A. J. Oldehinkel et al. and Schlotz et al. examined only the HPA-axis and no cardiac autonomic responses, and Gaab et al. used different measures for anticipatory versus retrospective stress appraisal. In other words, prior studies suggest interesting patterns of associations, but still with many gaps to be filled. The aim of the present study is to fill part of these gaps and so better understand how perceived stress relates to physiological stress. Associations between various perceived and physiological stress indices were investigated in 715 adolescents (351 boys, 364 girls, age 15–17) from the general population. Adolescents are a valuable population to study (psycho)physiological stress responses, because the prevalence of potentially confounding somatic disorders and medication use is relatively low at this age. Despite the fact that stress reactivity is affected by exposure to stressors earlier in life (e.g., Lupien, McEwen, Gunnar, & Heim, 2009), both perceived stress and physiological stress responses to psychosocial stress have been reported to be fairly invariant across age (e.g., Kudielka, Buske-Kirschbaum, Hellhammer, & Kirschbaum 2004; McManis, Bradley, Berg, Cuthbert, & Lang, 2001; Wood, Maraj, Lee, & Reyes, 2002), although it should be noted that the magnitude of heart rate responses tends to decrease with age (Carroll et al., 2000; Steptoe, Fieldman, Evans, & Perry, 1996). The adolescents included in this study participated in a series of behavioral tests including a social stress test (public speaking and mental arithmetic), which is considered a useful experimental paradigm to observe integrated psychological and physiological responses (Al’ Absi et al., 1997). The psychological stress indices used reflect bodily, affective, and cognitive dimensions of perceived stress; that is, subjective arousal, unpleasantness, and dominance (sense of being in control). The physiological measures, heart rate, respiratory sinus arrhythmia, and salivary cortisol, reflect (re)activity of two major physiological stress systems, the ANS and HPA-axis. The value of this study is not only its exceptionally large sample of adolescents, but also the fact that it examines various temporal patterns in the association between perceived and physiological stress. This is important, because Schlotz and colleagues (2008) showed that the direction of associations between psychological and physiological stress response may depend on the time lag between the measures. Based on associations found in the before-mentioned prior studies, three hypotheses were tested with regard to interrelations between perceived stress measures and physiological stress responses: 1. Perceived stress during a social stress test covaries with concurrent physiological stress responses; 2. High pretest levels of perceived stress predict large physiological responses to a social stress test; and 3. Large physiological responses to a social stress test predict low posttest perceived stress levels. The first hypothesis assumes an association between psychological and physiological stress during the social stress test, as compared to pretest levels, and is hence the most direct test of linkage between the various stress systems. Previous findings in favor of this hypothesis were reported by, among others, Al’ Absi et al. (1997), Roy (2004), and Thayer (1970). By comparing difference scores (that is, stress levels during exposure to a social stressor minus resting levels), it is possible to account for differences in response style, which can weaken estimated associations (e.g., Hjortskov et al., 2004). Response style refers to answer tendencies
Perceived and physiological stress responses that are unrelated to the content of the items, such as acquiescence. Because differences in on- and offsets of the stress responses may obscure covariations (Schlotz et al., 2008), the maximum stress response during the social stress test was used in this study, regardless of its timing. Justification for the second hypothesis is found in several studies suggesting that anticipatory appraisal processes predict physiological stress responses (e.g., Gaab et al., 2005; Rohrmann, Hennig, & Netter, 1999; Wirtz et al., 2006). Through various neural pathways, appraisal processes, such as perceived threat, provide input for the hypothalamic paraventricular nucleus, which plays a central role in the regulation of autonomic and endocrine stress responses (Gaab et al., 2005). It has been suggested that anticipation of stress, especially when the situation is perceived to be unpredictable and uncontrollable, may result in a state of vigilance toward events that are to occur and, consequently, in exaggerated stress responses (Schulkin, McEwen, & Gold, 1994). The third hypothesis, in a way, mirrors the second one. It was based on the intriguing phenomenon, observed in multiple studies, that high cortisol levels during stress may reduce post-stress anxiety, arousal, or fatigue (Het & Wolf, 2007; Reuter, 2002; Schlotz et al., 2008; Soravia et al., 2006; Tops, Van Peer, Wijers, & Korf, 2006). This suggests that, apart from normalizing the physiological stress systems, cortisol also regulates stress-induced negative emotions and perceived stress. Based on these findings, high cortisol levels during the social stress test were expected to predict low perceived stress levels afterwards in our study. As opposed to cortisol, autonomic stress responses have, to the best of our knowledge, not been investigated with regard to posttest perceived stress before, hence the analyses regarding heart rate and respiratory sinus arrhythmia were mostly exploratory in this respect. In sum, considering the wealth of data on psychological and physiological responses to stressful experiences, the relative scarcity of studies on the interrelation between the two is surprising and regrettable. The present study offers the opportunity to shed more light on this issue, because it involves cross-sectional and temporal associations between multiple perceived and physiological stress indices in a large general population sample of adolescents who were submitted to a social stress test. More knowledge about whether and how perceived stress predicts, follows, or covaries with cardiac and cortisol responses not only benefits theoretical stress models, but may also clarify the validity of perceived stress measures with respect to more general notions of stress.
Methods Participants The data were collected in a focus sample of TRAILS (TRacking Adolescents’ Individual Lives Survey), a large prospective population study of Dutch adolescents with bi- or triennial measurements from age 11 to at least age 25. Thus far, three assessment waves have been completed, running from March 2001 to July 2002 (T1), September 2003 to December 2004 (T2), and September 2005 to December 2007 (T3). During T1, 2230 children were enrolled in the study (response rate 76.0%, De Winter et al., 2005), of whom 1816 (81.4%) participated in T3. During T3, 744 adolescents were invited to perform a series of laboratory tasks (hereafter referred to as the experimental session) on top of the usual assessments, of whom 715 (96.1%) agreed to do so. The costly and labor-intensive nature of the
443 laboratory tasks precluded assessing the whole sample. Adolescents with a high risk of mental health problems had a greater chance of being selected for the experimental session. High risk was defined based on temperament (high frustration and fearfulness, low effortful control), lifetime parental psychopathology, and living in a single-parent family. In total, 66.0% of the focus sample had at least one of the above-described risk factors; the remaining 34.0% were selected randomly from the low-risk TRAILS participants. Please note that the focus sample still represented the whole range of problems seen in a normal population of adolescents, which made it possible to reproduce the distribution in the total TRAILS sample by means of sampling weights. Descriptive statistics of the focus sample (weighted estimates) are presented in Table 1. Procedure Experimental session. The experimental session consisted of a number of different challenges, listed here in chronological order: a spatial orienting task, a gambling task, a startle reflex task, and a social stress test. The session was preceded and followed by a 40-min period of rest. The participants filled out a number of questionnaires at the start and end of the session. Before, during, and after the experimental session, extensively trained test assistants assessed cardiovascular measures, cortisol, and perceived stress. Measures that were used in the present study are described more extensively below. The experimental sessions took place in sound-proof rooms with blinded windows at selected locations in the participants’ towns of residence. The total session lasted about 3 1/2 h, and started between 8:00 and 9:30 am (morning sessions, 50%) or between 1:00 and 2:30 pm (afternoon sessions, 50%). The protocol was approved by the Central Committee on Research Involving Human Subjects (CCMO). The social stress test. The social stress test was the last challenge of the experimental session. It involved a standardized protocol, inspired by (but not identical to) the Trier Social Stress Task (Kirschbaum, Pirke, & Hellhammer, 1993), for the induction of mild performance-related social stress. Socio-evaluative threats are highly salient challenges for adolescents and are known to be effective activators of various physiological stress systems, particularly in combination with uncontrollability; that is, in situations when negative consequences cannot be avoided (Dickerson & Kemeny, 2004). The participants were instructed to prepare a 6-min speech about themselves and their lives and deliver this speech in front of a video camera. They were told that their videotaped performance would be judged on content of speech as well as on use of voice and posture, and ranked by a panel of peers after the experiment. The participants had to speak continuously for the whole period of 6 min. The test assistant Table 1. Sample Characteristics (N 5 715) Variable Female gender Age Smoking (habitual) Physical exercisea Body mass index Use of oral contraceptives (% among girls)
Mean (SD) or percentage 50.7% 16.11 (0.60) 28.0% 3.26 (2.06) 21.45 (3.29) 34.4%
Note: Sampling weights were used to represent the distribution in the general population. a Number of days per week with at least 1 hr of physical exercise.
444 watched the performance critically, and showed no empathy or encouragement. The speech was followed by a 3-min interlude in which the participants were not allowed to speak. During this interval, which was included to assess cardiac autonomic measures that were not affected by speech, the participants were told that they had to wait for a moment because of computer problems, but that the task would continue as soon as these problems were solved. Subsequently, they were asked to perform mental arithmetic. The participants were instructed to repeatedly subtract the number 17 from a larger sum, starting with 13,278. A sense of uncontrollability was induced by repeated negative feedback from the test assistant (e.g., ‘‘No, wrong again, begin at 13,278’’; ‘‘Stop wiggling your hands’’; ‘‘You are too slow, we are running behind schedule’’). The mental arithmetic challenge lasted for 6 min, again followed by a 3-min period of silence, after which the participants were debriefed about the experiment. Measures Heart rate (HR). Cardiac autonomic function was assessed at the start of the experimental session (after 40 min of rest), as well as during and after the social stress test, in seven blocks: pretest (300 s), speech preparation (420 s), speech (360 s), silent interlude after speech (180 s), mental arithmetic (360 s), silent interlude after mental arithmetic (180 s), and posttest (300 s). A three-lead electrocardiogram was registered using 3M/RedDot Ag/AgCl electrodes (type 2255, 3M Health Care, Neuss, Germany), while the participant was sitting and breathing spontaneously. With a BIOPAC Amplifier-System (MP100, Goleta, CA), the signals were amplified and filtered before digitization at 250 samples/second. Dedicated software (PreCARSPAN, previously used in, e.g., Dietrich et al., 2007) was used to check signal stationarity, to correct for artifacts, to detect R-peaks, and to calculate the interbeat-interval (IBI) between two heartbeats. Blocks were considered invalid if they contained artifacts with a duration of more than 5 s, if the total artifact duration was more than 10% of the registration, or if the block length was less than 100 s (invalid blocks pretest: n 5 15, preparation: n 5 28, speech: n 5 27, interlude after speech: n 5 35, mental arithmetic: n 5 29, interlude after mental arithmetic: n 5 31, posttest: n 5 32). HR is inversely related to IBI by the equation HR 5 60000/IBI. HR was defined as the number of beats per minute (bpm). Respiratory sinus arrhythmia (RSA). Calculation of RSA was performed by power spectral analysis in the CARSPAN software program (Mulder, 1988) using estimation techniques based on Fourier transformations of IBI series (Robbe et al., 1987). RSA was defined as the power in the high-frequency (0.15–0.40 Hz) band, which is associated with the respiratory cycle, and expressed in ms2. RSA mainly results from centrally mediated cardiac vagal activity (Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology, 1996). Because the social stress test involved speech, which is known to interfere with analysis of RSA (e.g., Bernardi et al., 2000; Sloan, Korten, & Myers, 1991), the calculation of RSA was based on HR recordings during the 3-min interludes directly following the speech and mental arithmetic tasks, when the participants were not allowed to speak. The stress level remained relatively high during these interludes, because the participants expected that they had to continue any moment. Nevertheless, it was probably lower than during speech and mental arithmetic tasks and might not reflect the maximum response.
A. J. Oldehinkel et al. Cortisol. Cortisol levels were assessed just before the start of the social stress test (C1), directly after the end of the test (C2), 20 min after the test (C3), and 40 min after the test (C4). Considering the normal delay (20–25 min) in peak cortisol responses to experimental stressors (Kirschbaum, Read, & Hellhammer, 1992), all samples reflect stress reactions about 20 min earlier. Therefore, the samples were labeled as C1 5 pretest, C2 5 during test, C3 5 end of test (immediately after the test), and C4 5 posttest (20 min after the test). Cortisol was assessed from saliva by the Salivette sampling device (Sarstedt, Numbrecht, Germany). After the experimental session, the samples were placed in a refrigerator at 41C, and within a few days stored at ! 201C until analysis. All samples were analyzed with the same reagent, and all samples from a participant were assayed in the same batch. Cortisol was measured directly in duplicate in 100 ml saliva using an in-house radioimmunoassay (RIA) applying a polyclonal rabbit cortisol antibody and 1,2,6,7 3H Cortisol (Amersham International Ltd., Amersham, UK) as tracer. After incubation for 30 min at 601C, the bound and free fractions were separated using activated charcoal. The intra-assay coefficient of variation was 8.2% for concentrations of 1.5 nM, 4.1% for concentrations of 15 nM, and 5.4% for concentrations of 30 nM. The inter-assay coefficients of variation were, respectively, 12.6%, 5.6%, and 6.0%. The detection border was 0.9 nM. Missing samples (C1: n 5 12, C2: n 5 8, C3: n 5 10, C4: n 5 12) were due to detection failures in the lab (60%) or insufficient saliva in the tubes (40%). Cortisol levels above 5 standard deviations of the mean (C1: n 5 3, C2: n 5 6, C3: n 5 3, C4: n 5 4) were considered outliers and recoded into missing values. Perceived stress. Perceived stress was assessed by means of the Self-Assessment Manikin (SAM), a non-verbal pictorial assessment technique to measure the arousal, pleasure, and dominance (i.e., control) associated with a person’s affective reaction to a stimulus (Bradley & Lang, 1994). For each of the feelings assessed (i.e., arousal, unpleasantness, dominance), the subjective intensity could be indicated by choosing one out of nine ordered pictures. The pictures were translated into a nine-point scale (range 1–9) in such a way that high scores represented high levels of arousal, unpleasantness, and dominance. Perceived stress during the social stress test was assessed directly after the test, with a reference to the test (‘‘How did you feel during this test?’’) Pre- and posttest experiences were measured at the start (after 40 min of rest) and at the end of the experimental session (40 min after the social stress test), respectively. SAM ratings for arousal and unpleasantness have been shown to correlate almost perfectly (r " .95) with corresponding scales of the Semantic Differential Scale (Mehrabian & Russel, 1974), while the correlation was moderately high (r 5 .79) for dominance (Bradley & Lang, 1994). Other variables. Smoking, physical activity, and body mass index (BMI) were included as potential confounders of the associations under study. Smoking and physical exercise were assessed as part of the regular T3 questionnaire, which was filled out at school, on average 3.07 months (SD 5 5.12) before the experimental session. We distinguished between non-smokers and habitual smokers (i.e., at least one cigarette a day). Physical activity was operationalized as the number of days the respondent was physically active for at least 1 h. During the school assessments, length and weight were measured by trained test assistants. BMI is defined as the weight in kilograms divided by
Perceived and physiological stress responses the length in meters squared. Use of oral contraceptives (OC) was assessed by means of a checklist on current medication use administered at the start of the experimental session. In total, OC were used by 125 girls (34.4%). Analysis Adolescents with a high risk of mental health problems were overrepresented in the study sample. Therefore, sampling weights were used to reproduce the distribution in the total TRAILS sample in all analyses. Sampling weights denote the inverse probability that a subject is included in a sample. Missing data on any of the variables were handled by multiple imputation, using the ICE (Imputation by Chained Equations) approach available in the statistical package Stata (StataCorp, 2007). Five datasets with imputed missing values were created, given other variables in the dataset. Analyses were performed on each imputation, and subsequently combined into a single result using the Stata program MIM (Royston, 2005). The percentage of missing values was generally low and did not exceed 4.5% for any of the variables included in the analyses. Perceived and physiological stress responses were defined as the maximum level during (or immediately after) the test minus the minimum level before or after the test; for RSA and dominance, this equation was reversed in order to construct response measures that were positively associated with the strength of the response for all variables. Stress responses were defined in relation to either preor posttest levels instead of only pretest levels because prior research suggests that posttest stress levels make up better resting measures than pretest levels because posttest levels are not confounded by anticipation effects (Hansen, Johnsen, & Thayer, 2003). A two-sided p-value smaller than .05 was considered statistically significant. The first step was to calculate descriptive statistics of the (untransformed) variables used in this study, and to test differences between multiple assessments of the same variable by means of repeated measures analysis of variance. In case of significant within-subject changes, pairwise post hoc tests were performed to explore the nature of the differences, with Bonferroni correction for multiple testing. The analyses of variance were based on a single imputation dataset, because Stata’s multiple imputation procedures do not support repeated measures analysis of variance. The HR, RSA, and cortisol variables were log-transformed before analysis to obtain a more normal distribution. Before transformation, the skewness ranged from 0.53 to 0.97 for the HR variables, from 3.32 to 5.76 for the RSA variables, and from 1.68 to 2.70 for the cortisol variables. After transformation, the skewnesses were between ! 0.73 and 0.14, ! 0.17 and 0.07, and ! 0.17 and 0.96, respectively. Means and standard deviations were based on untransformed (raw) variables. Subsequently, the three hypotheses outlined in the introduction were tested by a series of linear regression analyses. The hypothesis that perceived stress covaried with concurrent physiological responses during the social stress test (hypothesis 1) was tested by analyses with HR, RSA, and cortisol responses as outcomes, and perceived stress responses (i.e., the difference between test and resting levels of arousal, unpleasantness, and dominance) as predictor variables. The hypothesis that high perceived stress levels at pretest predicted large physiological responses to the social stress test (hypothesis 2) was tested by using the pretest levels of arousal, unpleasantness, and dominance as predictor variables, and HR, RSA, and cortisol responses as outcomes. Finally, the hypothesis that large physiological stress responses
445 predicted low posttest perceived stress levels (hypothesis 3) was tested by regressing the difference between post- and pretest perceived stress levels on HR, RSA, and cortisol responses. All continuous variables were standardized to mean 0 and standard deviation 1 to obtain internally comparable regression coefficients. Partial Z2 was used as a measure of effect size. Gender, smoking, and physical exercise were included in all regression analyses as possible confounders. Furthermore, because there is ample evidence for gender differences in psychophysiological responses to stressful situations, both in previous studies (Biondi & Picardi, 1999; Kudielka, Hellhammer, & Wu¨st, 2009) and in the present dataset (Bouma et al., 2009), all effects under study were tested on gender differences. This was done by including interaction terms in the model, which were maintained if significant. A previous study by Bouma et al. (2009) on the effects of gender, menstrual phase, and use of oral contraceptives in the same sample had indicated that oral contraceptive users (34.4% of the girls) showed no cortisol response to the social stress test. Therefore, in the present study, oral contraceptive users were excluded from all analyses involving cortisol. This exclusion led to an overrepresentation of boys in the cortisol analyses, but not to a dramatic extent (59.6% boys versus 40.4% girls). Moreover, gender was included as covariate in all analyses, which prevented possible bias. Results Descriptive Statistics All stress measures changed significantly during the social stress test, with both psychological and physiological measures indicating that stress levels were higher during the social stress test than preceding or following it (Table 2). Please note that the pretest values of the perceived stress measures and HR and RSA reflect stress levels at the start of the laboratory session (after 40 min of rest), about 1 1/2 h before the start of the social stress test. Pretest RSA was exceptionally low, compared to RSA levels during and after the stress test. This is remarkable since pretest HR correlated ! .67 with pretest RSA, but was not exceptionally high. RSA levels after speech and mental arithmetic were relatively high compared to RSA during the preparation phase, probably because the speech and mental arithmetic values of RSA were assessed during silent interludes (directly) after the performance rather during the task itself. RSA levels during speech and mental arithmetic were lower indeed (speech: 1872, SD 5 2380; mental arithmetic: 1901, SD 5 2349), but may have been influenced by the respondents’ speaking at that time and are hence less trustworthy. Although RSA levels after speech and mental arithmetic were higher than during these stressors, they were still both significantly lower than posttest RSA. The cortisol statistics presented concern the pooled estimates across morning and afternoon sessions. Cortisol levels were higher in the morning (mean level morning 4.54 nM/L, SD 5 2.16; afternoon 3.62 nM/L, SD 5 1.98; t(588) 5 5.45, po.001), but the response patterns were comparable (Bouma et al., 2009), with significant within-changes in both mornings (F(3,288 5 41.4, po.001) and afternoons (F(3,295) 5 41.9, po.001). Correlations between subsequent assessments of stress measures were generally moderate to high (arousal: r 5 .32 to .47; unpleasantness: r 5 .18 to .32; dominance: r 5 .44 to .59; HR: r 5 .61 to .87; RSA: r 5 .69 to .87; cortisol: r 5 .47 to .87). Correlations between arousal, unpleasantness, and dominance were higher during stress (|r| 5 .41 to .54) than during rest (|r| 5 .17 to
446
A. J. Oldehinkel et al.
Table 2. Stress Measures Used in this Study, and Tests of Within-Subjects Changes Variable
Mean (SD)
Within-subject change
Significant differences
A. Arousal pretest B. Arousal during test C. Arousal posttest
2.68 (1.50) 4.19 (1.88) 2.37 (1.45)
F(2,713) 5 325.3, po.001
CoAoB
A. Unpleasantness pretest B. Unpleasantness during test C. Unpleasantness posttest
2.85 (1.23) 4.74 (1.89) 2.88 (1.76)
F(2,713) 5 367.1, po.001
AoB CoB
A. Dominance pretest B. Dominance during test C. Dominance posttest
6.46 (1.47) 5.39 (1.85) 6.97 (1.44)
F(2,713) 5 288.8, po.001
BoAoC
A. HR pretest (bpm) B. HR preparation (bpm) C. HR speech (bpm) D. HR mental arithmetic (bpm) E. HR posttest (bpm)
75.68 (11.13) 77.96 (11.12) 82.05 (13.20) 78.08 (11.49) 69.47 (9.96)
F(4,711) 5 457.2, po.001
EoAoBoC EoAoDoC
A. RSA pretest (ms2) B. RSA preparation (ms2) C. RSA after speech (ms2) D. RSA after mental arithmetic (ms2) E. RSA posttest (ms2)
1732 (2820) 2178 (3209) 2462 (3447) 2363 (3338) 2653 (3561)
F(4,711) 5 72.04, po.001
AoBoCoE AoBoDoE
A. Cortisol pretest (nM/L) B. Cortisol during test (nM/L) C. Cortisol end of testb (nM/L) D. Cortisol posttestc (nM/L)
3.43 (2.04)a 4.59 (2.85)a 4.46 (2.98)a 3.71 (2.12)a
F(3,586) 5 76.0, po.001
AoDoCoB
Note: Sampling weights were used to represent the distribution in the general population. Descriptives for HR, RSA, and cortisol data reflect untransformed data, while log-transformed data were used in the analyses. Analyses were based on single imputation data. Pairwise differences were adjusted for multiple testing (Bonferroni method). HR: heart rate, RSA: respiratory sinus arrhythmia. a Exclusive of girls using oral contraceptives. b Immediately after the social stress test. c 20 min after the social stress test.
.35). Similarly, HR and cortisol levels were significantly correlated (r 5 .08 to .17) during and immediately after the social stress test, but not before the test or 20 min afterwards (r 5 ! .06 to ! .05). Interestingly, pretest cortisol levels were inversely related to HR during and after the test. RSA was negatively associated with HR (r 5 ! .37 to ! .67), but not with cortisol. For an overview of all correlations, see the Appendix. Associations between psychological and physiological stress measures will be discussed in more detail below. Associations Between Perceived and Physiological Stress Measures Interrelations between perceived and physiological stress measures (adjusted for gender, smoking, BMI, and physical exercise) are shown in Tables 3–5. None of the effects were significantly different for boys and girls. The first hypothesis was that perceived responses during the social stress test would covary with concomitant physiological responses. As expected, changes in perceived arousal and unpleasantness responses were associated with changes in HR, RSA, and cortisol (Table 3). Changes in perceived dominance did not covary significantly with any of the physiological stress responses. Effect sizes (partial Z2) for arousal and unpleasantness ranged between .006 and .017, which correspond to Cohen’s d-values between 0.15 and 0.25 and thus signify small effects. To further illustrate the size of the effects, the sample was divided into three groups based on the perceived stress responses: low responders (limited change in perceived arousal and perceived unpleasantness, i.e., 0 or 1, 20.8%), high responders (large changes in perceived arousal or perceived unpleasantness of 5 or
more, 20.1%), and intermediate responders (all other adolescents, 59.1%). HR, RSA, and cortisol responses were plotted for each of these groups (Figures 1–3). Relative to the size of the stress response itself, the differences among the three perceived stress groups were considerably smaller for HR than for RSA and cortisol; Figure 1 suggests hardly any effect of perceived stressfulness on HR responses. This may seem inconsistent with the fact that both the effect sizes and the (standardized) regression coefficients were largely comparable for the three outcome measures. This seeming inconsistency can be explained by the small standard deviation of HR responses, compared to the size of the response. The graphs in Figures 1 and 3 show larger differences between high and intermediate responders than between low and intermediate responders, which could point to nonlinear effects. To test this (post hoc) hypothesis, we added quadratic effects of arousal and unpleasantness responses to the regression models predicting HR and cortisol, but none of these effects were statistically significant (all p-values 4.11). The patterns shown in Figures 1 and 3 may be due to the definition of the perceived stress response groups: perhaps the group of intermediate responders were on average more comparable to the low responders than to the high responders. The second hypothesis was that pretest perceived stress levels would be associated with physiological responses to the social stress test. No pretest levels of the perceived stress measures (arousal, unpleasantness, and dominance) predicted any subsequent HR, RSA, or cortisol responses (Table 4). With regard to the third hypothesis, that physiological responses would predict posttest perceived stress levels, we found that greater HR responses predicted less posttest unpleasantness
Perceived and physiological stress responses
447
Table 3. Perceived Concurrent Stress Responses as Predictors of Physiological Responses to the Social Stress Test
Table 5. Physiological Stress Responses as Predictors of Posttest Perceived Stress Posttest perceived stressa
Outcomes HR RSA Cortisol response (ln) response (ln) response (ln)a B (p) B (p) B (p)
Predictors Concurrent Arousal Concurrent Unpleasantness Concurrent Dominance
0.12 (.001) 0.09 (.03) 0.04 (.25)
0.08 (.02) 0.09 (.03) ! 0.00 (.97)
0.09 (.05) 0.12 (.008) 0.01 (.88)
Note: Sampling weights were used to represent the distribution in the general population. HR, RSA, and cortisol variables were log-transformed before analysis. Continuous variables were standardized to mean 0 and SD 1. All effects are adjusted for gender, smoking, BMI, and physical exercise. N 5 715. HR: heart rate, RSA: respiratory sinus arrhythmia, Response: difference between state during the test and pre- or posttest state. Bold: po.05. a Analyses exclusive of girls using oral contraceptives (N 5 589).
and more posttest dominance, as compared to pretest levels (Table 5), which lends partial support for the hypothesis that physiological stress responses predict posttest perceived stress levels. Large cortisol responses tended to be associated with low posttest unpleasantness as well (two-sided p 5 .06). RSA responses were not associated with any of the posttest perceived stress measures. Effect sizes were small, with partial Z2 values of around .006 for the (marginally) significant effects, corresponding to a Cohen’s d-value of 0.15. Discussion In this study, we explored the interrelation of perceived and physiological responses to a social stress test in a large sample of adolescents from the general population. The results suggest temporal covariation of psychological and physiological stress systems as well as limited associations between physiological stress responses and subsequent psychological measures. More specifically, perceived arousal and unpleasantness during the stress test covaried with all concurrent physiological stress responses (hypothesis 1), and large HR responses to social stress predicted low posttest unpleasantness and dominance, while a trend was found for an effect of cortisol responses on posttest
Table 4. Perceived Pretest Stress Responses as Predictors of Physiological Responses to the Social Stress Test Outcomes
Predictors Pretest Arousal Pretest Unpleasantness Pretest Dominance
HR response (ln) B (p)
RSA response (ln) B (p)
Cortisol response (ln)a B (p)
! 0.02 (.55) 0.06 (.10) ! 0.06 (.13)
! 0.02 (.62) 0.03 (.34) ! 0.08 (.09)
! 0.02 (.62) ! 0.04 (.32) ! 0.01 (.82)
Note: Sampling weights were used to represent the distribution in the general population. HR, RSA, and cortisol variables were log-transformed before analysis. Continuous variables were standardized to mean 0 and SD 1. All effects are adjusted for gender, smoking, BMI, and physical exercise. N 5 715. HR: heart rate, RSA: respiratory sinus arrhythmia, Response: difference between state during the test and pre- or posttest state. a Analyses exclusive of girls using oral contraceptives (N 5 589).
Predictors HR response RSA response Cortisol responseb
Arousal B (p)
Unpleasantness B (p)
Dominance B (p)
! 0.06 (.13) ! 0.03 (.35) ! 0.02 (.65)
! 0.09 (.05) ! 0.03 (.44) ! 0.07 (.06)
0.09 (.04) 0.06 (.14) 0.01 (.84)
Note: Sampling weights were used to represent the distribution in the general population. All effects are adjusted for gender, smoking, BMI, and physical exercise. N 5 715. HR: heart rate, RSA: respiratory sinus arrhythmia, Response: difference between state during the test and pre- or posttest state. Bold: po.05. a As compared to pretest perceived stress (difference scores). b Analyses exclusive of girls using oral contraceptives (N 5 589).
unpleasantness (hypothesis 3). There was no support for hypothesis 2, that high pretest perceived stress levels predict physiological responses to social stress. Hypothesis 1 Our results support the notion of covariation between perceived and physiological stress responses. Despite only weak correlations between HR and cortisol and no significant correlations between RSA and cortisol, most associations with perceived stress levels were largely comparable among the three physiological stress measures. The significant associations of perceived arousal and unpleasantness with both cardiac measures and cortisol suggest that perceived stress reflects, to a certain extent, activity of the HPA-axis as well as the autonomic nervous system. Our data do not allow conclusions about whether the perception of the stressfulness steered physiological responses or vice versa, but we assume bidirectional influences. On the one hand, it is obvious that psychosocial stressors need to be perceived and evaluated as such in order to trigger a stress response (e.g., Ulrich-Lai & Herman, 2009), on the other hand, physiological reactions (e.g., heart pounding) may be interpreted as signs of the apparent stressfulness of the situation and hence inflate perceived stress scores. Effect sizes were small according to Cohen’s conventions (1988), but still considerable compared to the size of the stress responses, specifically for RSA and cortisol (see Figures 2 and 3). Taking into account that this study involved a normal-population sample of adolescents and a mild brief stressor, and that both psychological and physiological stress responses are influenced by a multitude of only partially overlapping factors, we feel that high effect sizes could not be expected. Furthermore, as shown in several meta analyses (e.g., Ioannidis, Trikalinos, Ntzani, & ContopoulosIoannidis, 2003), published effect sizes based on large samples are, on average, considerably smaller than those based on small samples. This is probably due to publication bias: in studies with a limited sample size, small effects are usually not statistically significant and therefore less likely to be submitted and accepted for publication (Easterbrook, Berlin, Gopalan, & Matthews, 1991). Changes in perceived dominance were not significantly related to physiological stress responses. This seems inconsistent with previous reports of uncontrollability as a predictor of the cortisol response (Dickerson & Kemeny, 2004). The Dominance scale of the Self-Assessment Manikin (Bradley & Lang, 1994) depicts a series of schematic figures, ranging from very small
448
A. J. Oldehinkel et al. 85
5.5 5
cortisol (nM / L)
HR (bpm)
80
75
70 max. change 5+ max. change 2 – 4 max. change 1
65
0
4.5 4 3.5
max. change 5+ max. change 2 – 4 max. change 1
3
pretest preparation speech arithmetic posttest
0
pretest
during test
end of test
posttest
Figure 1. HR responses to the social stress test, by responsiveness level. Responsiveness level is based on a composite index of perceived arousal and perceived competence.
Figure 3. Cortisol responses to the social stress test, by responsiveness level. Responsiveness level is based on a composite index of perceived arousal and perceived competence.
(being controlled, submissive) to very large (being in control, powerful). This measure may not be specific enough to measure feelings of uncontrollability. It is also possible that the assumed effects of uncontrollability on cortisol responses relate to objective task characteristics rather than individual differences in perceived controllability. Apart from the above-described methodological issues, there may also be a more substantive reason why arousal and unpleasantness, but not dominance, covary with physiological stress responses. Unpleasantness and arousal reflect the desire to change the situation, and the intensity of this desire, respectively. These are primitive motivational parameters integrated in subcortical areas (e.g., Lang, Bradley, & Cuthbert, 1992), which have been associated with various physiological responses (e.g., Lang, Greenwald, Bradley, & Hamm, 1993). Dominance reflects the
perceived possibilities to change the situation, rather than the actual desire to do so. Consistent with this, dominance has been found to account for less variance in emotional judgments than arousal and pleasure (e.g., Bradley & Lang, 1994), and may hence also be more loosely linked to physiological stress responses.
3000 2900 2800 2700 2600 2500 RSA (ms2)
2400 2300 2200 2100 2000 1900 1800
max. change 5+
1700
max. change 2 – 4
1600
Hypothesis 2 Contrary to expectations, pretest perceived stress did not predict physiological responses to a social stress test. This could be due to the fact that the pretest perceived stress levels did not reflect how stressful the adolescents expected the social stress test to be, but rather how they felt in general at the start of the laboratory session. This general stress perception is probably not a very accurate measure of anticipatory appraisal, which has been found to predict physiological stress responses in previous studies (Gaab et al., 2005; Rohrmann et al., 1999; Wirtz et al., 2006). In addition, stress responses may have been affected by the considerable time lag between the pretest measure and the social stress test, as well as the various other tasks performed in between. It would thus be inappropriate to conclude that the anticipated stressfulness of a particular task is unrelated to physiological responses to that task. Although the pretest perceived stress measures used in this study may not assess anticipatory appraisal well, they have a validity of their own, as pretest resting levels. Hence, what can be concluded from the results is that resting levels of perceived arousal, unpleasantness, and dominance are not very informative with regard to subsequent physiological stress responses. In general, there does not seem to be much meaningful variance in stress measures assessed during rest, as is also illustrated by finding that correlations between various stress measures were higher during stress than pre- or posttest. This suggests that individual differences in stress responsiveness can best be ascertained under stressful conditions.
max. change 1 0
pretest
preparation
speech
arithmetic
posttest
Figure 2. RSA responses to the social stress test, by responsiveness level. Responsiveness level is based on a composite index of perceived arousal and perceived competence.
Hypothesis 3 The hypothesis that physiological stress responses predict posttest perceived stress levels was based on prior studies suggesting that high cortisol levels might prevent stressful experiences from inducing negative affect (Het & Wolf, 2007; Reuter, 2002; Sch-
Perceived and physiological stress responses lotz et al., 2008; Soravia et al., 2006; Tops et al., 2006). The marginally significant effect of cortisol responses on posttest perceived unpleasantness lends tentative support to this postulation. It seems contradictory that high cortisol levels reflect distress and high cortisol responses prevent it. Distinguishing between tonic and phasic cortisol levels might be relevant in this respect: high tonic cortisol levels have adverse effects on mood (e.g., Schmidt, Fox, Goldberg, Smith, & Schulkin, 1999; Wolkowitz et al., 1990), while high phasic cortisol levels (i.e., large responses) seem quite adaptive when measured in healthy adolescents. The functional effects of cortisol for regulating emotions are still unknown. Cortisol binds to (glucocorticoid and mineralocorticoid) receptors, which can be found in several brain areas, including prefrontal cortex and limbic areas (e.g., De Kloet, Vreugdenhil, Oitzl, & Joels, 1998), and can influence several catecholaminergic neurotransmitter systems (Joels, 2000). It has been proposed that cortisol modulates pathways of a neural network involving, among other things, the prefrontal cortex, amygdala, and HPA-axis. These networks play an important role in emotional processing (e.g., Davidson & Irwin, 1999; Dolan, 2002), in that the effects of cortisol on the prefrontal cortex reduce emotional responses to stress (Het & Wolf, 2007). HR responses, which have been suggested to reflect effort rather than distress (e.g., Arnetz & Fjellner, 1986; Peters et al., 1998), were more strongly associated with posttest perceived stress measures than cortisol responses. High HR responses predicted low posttest unpleasantness and high posttest dominance. A possible explanation for the association between HR responses and posttest perceived stress is that a high HR response is an adaptive mechanism to adequately cope with stressors. Assuming a positive association between the strength of the HR response and the amount of effort invested in the task (Arnetz & Fjellner, 1986; Peters et al., 1998), we could speculate that adolescents who invested a lot of effort performed better and hence felt more satisfied and in control afterwards. Otherwise stated, blunted stress responses may signal dysfunctional coping strategies, which in turn may increase feelings of discomfort and lack of control following the stressful experience. In fact, another study in the same sample indicated that adolescents with high effortful control (i.e., high self-regulation skills) had stronger HR responses to the social stress test (Oldehinkel, Hartman, Nederhof, Riese, & Ormel, submitted), which supports the idea that blunted stress responses may reflect poor coping with stress. Analogous to effects of physical exercise on emotional well-being (e.g., Sher, 1998; Yeung, 1996), a direct impact of physiological activity on subsequent subjective emotions is conceivable as well, such as through altered neurotransmitter release (Meeusen & Piacentini, 2001). Alternatively, high HR responses may not actually predict subsequent feelings, but rather mark adolescents who are still energetic and do not feel worn out and therefore report low levels of unpleasantness and uncontrollability at the end of the laboratory session. Why the effect of HR responses on posttest perceived stress was stronger for unpleasantness and low dominance than for arousal might be related to the fact that unpleasantness and uncontrollability are usually rated as negative emotions, while high arousal can be conceived of as either negative or positive. If high HR responses mark a satisfactory performance, as suggested above, this is likely to influence positive affect, but not necessarily relaxation. Hence, HR responses are possibly associated with posttest negative affect rather than (hyper)arousal. However, all these suggestions are highly tentative, and replication in an independent sample is
449 warranted before firm conclusions can be drawn regarding this association. RSA responses did not predict any of the posttest perceived stress measures. This could indicate that the effects of HR were mainly accounted for by sympathetic, and not vagal, activation. Prudence is called for, however, because HR and RSA measures during speech and mental arithmetic were not based on the same time periods. Practical Implications Given that our sample was large and representative of a normal population of adolescents, this study is particularly suitable to answer the practical question of whether, in clinical or research settings, self-reports of perceived arousal and unpleasantness during a stressful situation provide useful information about the magnitude of HR, RSA, or cortisol responses. Based on our findings, the answer to this question would have to be no. Due to substantial unexplained variance, measures of perceived stress provide only partial knowledge about the responsiveness of the autonomic system and HPA-axis. As suggested by Fahrenberg and Foerster (1982), a set of marker variables seems to be preferable to a single measure to assess individual differences in stress responsiveness, and we propose these marker variables should include both perceived and physiological stress indices. Strengths and Limitations The findings should be considered in light of a number of noteworthy strengths and limitations. A significant strength of the study is its very large sample size, compared to most other studies involving laboratory stress tests. This reduces the influence of single outliers and the probability of false-negative or false-positive results. The subjects were adolescents selected from the general population, whose perceived and physiological stress responses are less likely to be disturbed by medical conditions than those of older subjects or clinical patients. An additional strength is the repeated examination of stress indices across the testing session, a procedure which yields more clues about the direction of effects than single assessments. There are also limitations to this study. First, the social stress test was preceded by a spatial orienting task, a startle-response test, and a gambling task. We did not account for the perceived stressfulness of these challenges. The stress measures assessed during the social stress test could represent the cumulative effect of the prior experimental tasks rather than responses to the social stress test. A large systematic bias due to the experimental design is unlikely, however, because the order of the tasks was the same for all subjects. Hence, not only the exposure to social stress was standardized, but also the activities preceding the social stressor. Furthermore, the social stress test was by far the most stressful element of the session, both conceptually and in terms of subjectively experienced stress as measured by the Self-Assessment Manikin (data available upon request). Still, one cannot rule out effects of the preceding tasks on responses to the social stress test. Moreover, as mentioned before, pretest HR, RSA, and perceived stress measures reflect levels at the beginning of the laboratory session (after 40 min of rest) rather than levels immediately preceding the social stress task, which may have deflated the effects. A second limitation is that RSA was assessed during silent interludes following the periods wherein the participants were actively engaged in public speaking and mental arithmetic, to avoid interference with speech. Although the stress level during these silent interludes was relatively high because the participants anticipated near continuation of the test, it was still likely to be lower
450
A. J. Oldehinkel et al.
than during the performance. In most participants, the RSA responses reflected the difference between posttest RSA and RSA during the preparation phase, which may not be the maximum response. Third, respiration rate was not recorded in this study and hence could not be controlled for while analyzing RSA, as recommended by, for instance, Berntson et al. (1997). Because RSA was based on periods without speech in which the participants were sitting quietly, the task effects upon respiration rate were probably limited, which reduces the need for respiratory control (e.g., Grossman & Taylor, 2007; Houtveen, Rietveld, & De Geus, 2002), yet some confounding cannot be excluded. Finally, responses to social stress tests as used in laboratory experiments may not reflect responses to potentially pathogenic stressful experiences in real life. The social stress test used in our study lasted for less than half an hour, after which the adolescents were debriefed and could relax again. Real-life stressors and their aftermaths usually persist considerably longer than half an hour and are therefore likely to trigger more pervasive stress reactions. Conclusions Our findings suggest that perceived, autonomic, and HPA-axis responses to social stressors covary to some extent in adolescents.
Particularly on-task perceived arousal and unpleasantness may predict concurrent changes in HR, RSA, and cortisol levels. Dominance seems to have a specific, more cognitive role in adolescents’ stress appraisals, and to be less associated with physiological stress measures. Pretest resting perceived stress measures are not very informative with regard to physiological responses to stress. Furthermore, large physiological stress responses, notably HR responses, seem to reflect healthy, adaptive mechanisms, which might prevent post-stress negative affect. In sum, adolescents’ reported feelings of arousal and unpleasantness, but not dominance, to some degree reflect concurrent autonomic and HPA-axis activity. This could indicate thatFspecificFemotional responses to stressors generate physiological stress responses, as postulated in the introduction (Mason, 1971; Mikhail, 1981), be it to a limited extent. However, perceived stress levels do not seem to predict how adolescents will respond to later stressors, and should therefore be considered correlates rather than risk factors of physiological stress responses (Kraemer et al., 1997). This study also suggest that strong physiological stress responses, although perceived as arousing and unpleasant at the time being, can still be adaptive, in that they may increase feelings of pleasantness and dominance afterwards.
REFERENCES Al’ Absi, M., Bongard, S., Buchanan, T., Pincomb, G. A., Licinio, J., & Lovallo, W. R. (1997). Cardiovascular and neuroendocrine adjustment to public speaking and mental arithmetic stressors. Psychophysiology, 34, 266–275. Arnetz, B. B., & Fjellner, B. (1986). Psychological predictors of neuroendocrine responses to mental stress. Journal of Psychosomatic Research, 30, 297–305. Bernardi, L., Wdowczyk-Szulc, J., Valenti, C., Castoldi, S., Passino, C., Spadacini, G., et al. (2000). Effects of controlled breathing, mental activity and mental stress with or without verbalization on heart rate variability. Journal of the American College of Cardiology, 35, 1462–1469. Berntson, G. G., Bigger, J. T. Jr., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., et al. (1997). Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology, 34, 623–648. Biondi, M., & Picardi, A. (1999). Psychological stress and neuroendocrine function in humans: The last two decades of research. Psychotherapy and Psychosomatics, 68, 114–150. Bouma, E. M. C., Riese, H., Ormel, J., Verhulst, F. C., & Oldehinkel, A. J. (2009). Adolescents’ cortisol responses to awakening and social stress: Effects of gender, menstrual phase and oral contraceptives. The TRAILS Study. Psychoneuroendocrinology, 34, 884–893. Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: the SelfAssessment Manikin and the Semantic Differential. Journal of Behavior Therapy and Experimental Psychiatry, 25, 49–59. Buijs, R. M., & Van Eden, C. G. (2000). The integration of stress by the hypothalamus, amygdala and prefrontal cortext: Balance between the autonomic nervous system and the neuroendocrine system. In H. B. M. Uylings, C. G. Van Eden, J. P. C. De Bruin, M. P. G. Feenstra, & C. M. A. Pennartz (Eds.), Cognition, emotion and autonomic responses: The integrative role of the prefrontal cortex and limbic structures. Amsterdam: Elsevier. Carroll, D., Harrison, L. K., Johnston, D. W., Ford, G., Hunt, K., Der, G., & West, P. (2000). Cardiovascular Q2 reactions to psychological stress: The influence of demographic variables. Journal of Epidemiology and Community Health, 54, 876–877.
Cohen, J. (1988). Statistical power for the behavioral sciences (2nd. ed). Hillsdale, NJ: Erlbaum. Cohen, S., Hamrick, N., Rodriguez, M. S., Feldman, P. J., Rabin, B. S., & Manuck, S. B. (2000). The stability and intercorrelations among cardiovascular, immune, endocrine, and psychological reactivity. Annals of Behavioral Medicine, 22, 171–179. Davidson, R. J., & Irwin, W. (1999). The functional neuroanatomy of emotion and affective style. Trends in Cognitive Sciences, 3, 11–21. De Kloet, E. R., Vreugdenhil, E., Oitzl, M. S., & Joels, M. (1998). Brain corticosteroid receptor balance in health and disease. Endocrine Reviews, 19, 269–301. De Winter, A. F., Oldehinkel, A. J., Veenstra, R., Brunnekreef, J. A., Verhulst, F. C., & Ormel, J. (2005). Evaluation of non-response bias in mental health determinants and outcomes in a large sample of pre-adolescents. European Journal of Epidemiology, 20, 173–181. Dickerson, S. S., & Kemeny, M. E. (2004). Acute stressors and cortisol responses: A theoretical integration and synthesis of laboratory research. Psychological Bulletin, 130, 355–391. Dietrich, A., Riese, H., Sondeijker, F. E. P. L., Greaves-Lord, K., Van Roon, A. M., Ormel, J., et al. (2007). Externalizing and internalizing problems in relation to autonomic function: A population-based study in preadolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 378–386. Dolan, R. J. (2002). Emotion, cognition, and behavior. Science, 298, 1191–1194. Easterbrook, P. J., Berlin, J. A., Gopalan, R., & Matthews, D. R. (1991). Publication bias in clinical research. Lancet, 337, 867–872. Fahrenberg, J., & Foerster, F. (1982). Covariation and consistency of activation parameters. Biological Psychology, 15, 151–169. Gaab, J., Rohleder, N., Nater, U. M., & Ehlert, U. (2005). Psychological determinants of the cortisol stress response: The role of anticipatory cognitive appraisal. Psychoneuroendocrinology, 30, 599–610. Gerra, G., Zaimovic, A., Mascetti, G. G., Gardini, S., Zambelli, U., Timpano, M., et al. (2001). Neuroendocrine responses to experimentally-induced psychological stress in healthy humans. Psychoneuroendocrinology, 26, 91–107.
Perceived and physiological stress responses Grossman, P., & Taylor, E. W. (2007). Toward understanding respiratory sinus arrhythmia: Relations to cardiac vagal tone, evolution and biobehavioral functions. Biological Psychology, 74, 263–285. Hansen, A. H., Johnsen, B. H., & Thayer, J. F. (2003). Vagal influence on working memory and attention. International Journal of Psychophysiology, 48, 263–274. Het, S., & Wolf, O. T. (2007). Mood changes in response to psychosocial stress in healthy young women: Effects of pretreatment with cortisol. Behavioral Neuroscience, 121, 11–20. Hjortskov, N., Garde, A. H., Ørbæk, P., & Hansen, A˚. M. (2004). Evaluation of salivary cortisol as a biomarker of self-reported mental stress in field studies. Stress and Health, 20, 91–98. Houtveen, J. H., Rietveld, S., & De Geus, E. J. (2002). Contribution of tonic vagal modulation of heart rate, central respiratory drive, respiratory depth, and respiratory frequency to respiratory sinus arrhythmia during mental stress and physical exercise. Psychophysiology, 39, 427–436. Ioannidis, J., Trikalinos, T., Ntzani, E., & Contopoulos-Ioannidis, D. (2003). Genetic associations in large versus small studies: An empirical assessment. The Lancet, 361, 567–571. Joels, M. (2000). Modulatory actions of steroid hormones and neuropeptides on electrical activity in brain. European Journal of Pharmacology, 405, 207–216. Kirschbaum, C., Pirke, K. M., & Hellhammer, D. H. (1993). The ‘Trier Social Stress Test’FA tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology, 28, 76–81. Kirschbaum, C., Read, G. F., & Hellhammer, D. H. (1992). Assessment of hormones and drugs in saliva in biobehavioral research. Seattle: Hogrefe & Huber. Kraemer, H. C., Kazdin, A. E., Offord, D. R., Kessler, R. C., Jensen, P. S., & Kupfer, D. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54, 337–343. Kudielka, B. M., Buske-Kirschbaum, A., Hellhammer, D. H., & Kirschbaum, C. (2004). HPA axis responses to laboratory psychosocial stress in healthy elderly adults, younger adults, and children: Impact of age and gender. Psychoneuroendocrinology, 29, 83–98. Kudielka, B., Hellhammer, S. H., & Wu¨st, S. (2009). Why do we respond so differently? Reviewing determinants of human salivary cortisol responses to challenge. Psychoneuroendocrinology, 34, 2–18. Lackschewitz, H., Hu¨ther, G., & Kro¨ner-Herwig, B. (2008). Physiological and psychological stress responses in adults with attention-deficit/hyperactivity disorder (ADHD). Psychoneuroendocrinology, 33, 612–624. Lang, L. J., Bradley, M. M., & Cuthbert, B. N. (1992). A motivational analysis of emotion: Reflex-cortex connection. Psychological Science, 3, 44–49. Lang, L. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology, 30, 261–273. Lazarus, R. S. (1966). Psychobiological stress and the coping process. New York: McGraw-Hill. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. New York: Springer. Lupien, S., McEwen, B. S., Gunnar, M., & Heim, C. (2009). Effects of stress throughout the lifespan on the brain, behaviour and cognition. Nature Reviews Neuroscience, 10, 434–445. Mason, J. W. (1968). A review of psychoendocrine research on the pituitaryadrenal cortical system. Psychosomatic Medicine, 30, 576–607. Mason, J. W. (1971). A re-evaluation of the concept of ‘non-specificity’ in stress theory. Journal of Psychiatric Research, 8, 323–333. McCarty, R., & Gold, P. E. (1996). Catecholamines, stress and disease: A psychobiological perspective. Psychosomatic Medicine, 58, 590–597. McManis, M. H., Bradley, M. M., Berg, W. K., Cuthbert, B. N., & Lang, P. J. (2001). Emotional reactivity in children: Verbal, physiological, and behavioral responses to affective pictures. Psychophysiology, 38, 222–231. Meeusen, R., & Piacentini, M. (2001). Exercise and neurotransmission: A window to the future? European Journal of Sports Science, 1, 1–12. Mehrabian, A., & Russel, J. A. (1974). An approach to environmental psychology. Cambridge, MA: MIT. Mikhail, A. (1981). Stress: A psychophysiological conception. Journal of Human Stress, 7, 9–15. Mulder, L. J. M. (1988). Assessment of cardiovascular reactivity by means of spectral analysis. PhD thesis. Groningen, The Netherlands: University of Groningen.
451 Oldehinkel, A. J., Hartman, C. A., Nederhof, E., Riese, H., & Ormel, J. Effortful control as predictor of adolescents’ psychological and physiological responses to a social stress test. The TRAILS study. Submitted for publication. Peters, M. L., Godaert, G. L. R., Ballieux, R. E., Van Vliet, M., Willemsen, J. J., Sweep, F. C. G. J., et al. (1998). Cardiovascular and endocrine responses to experimental stress: Effects of mental effort and controllability. Psychoneuroendocrinology, 23, 1–17. Reuter, M. (2002). Impact of cortisol on emotions under stress and nonstress conditions: A pharmacological approach. Neuropsychobiology, 46, 41–48. Robbe, H. W., Mulder, L. J., Ruddel, H., Langewitz, W. A., Veldman, J. B., & Mulder, G. (1987). Assessment of baroreflex sensitivity by means of spectral analysis. Hypertension, 10, 538–543. Rohrmann, S., Hennig, J., & Netter, P. (1999). Changing psychobiological stress reactions by manipulating cognitive processes. International Journal of Psychophysiology, 33, 149–161. Roy, M. P. (2004). Patterns of cortisol reactivity to laboratory stress. Hormones and Behavior, 46, 618–627. Royston, P. (2005). Multiple imputation of missing values: Update of ICE. Stata Journal, 5, 527–536. Sher, L. (1998). The endogenous euphoric reward system that reinforces physical training: A mechanism for mankind’s survival. Medical Hypotheses, 51, 449–450. Schlotz, W., Kumsta, R., Layes, I., Entringer, S., Jones, A., & Wu¨st, S. (2008). Covariance between psychological and endocrine responses to pharmacological challenge and psychological stress: A question of timing. Psychosomatic Medicine, 70, 787–796. Schmidt, L. A., Fox, N. A., Goldberg, M. C., Smith, C. C., & Schulkin, J. (1999). Effects of acute prednisone administration on memory, attention and emotion in healthy human adults. Psychoneuroendocrinology, 24, 461–483. Schommer, N. C., Hellhammer, D. H., & Kirschbaum, C. (2003). Dissociation between reactivity of the hypothalamus-pituitaryadrenal axis and the sympathetic-adrenal-medullary system to repeated psychosocial stress. Psychosomatic Medicine, 65, 450–460. Schulkin, J., McEwen, B., & Gold, P. W. (1994). Allostasis, amygdala, and anticipatory angst. Neuroscience and Biobehavioral Reviews, 18, 385–396. Selye, H. (1936). A syndrome caused by diverse nocuous agents. Nature, 138, 32. Sloan, R. P., Korten, J. B., & Myers, M. M. (1991). Components of heart rate reactivity during mental arithmetic with and without speaking. Physiology & Behavior, 50, 1039–1045. Soravia, L. M., Heinrichs, M., Aerni, A., Maroni, C., Schelling, G., Ehlert, U., et al. (2006). Glucocortocoids reduce phobic fears in humans. Proceedings of the National Academy of Sciences of the USA, 103, 5585–5590. StataCorp. (2007). Stata Statistical software: Release 10. College Station, TX: StataCorp LP. Steptoe, A., Fieldman, G., Evans, O., & Perry, L. (1996). Cardiovascular risk and responsivity to mental stress: The influence of age, gender and risk factors. Journal of Cardiovascular Risk, 3, 83–93. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology (1996). Heart rate variability: Standards of measurement, physiological interpretation and clinical use. Circulation, 93, 1043–1065. Thayer, R. E. (1970). Activation states as assessed by verbal report and four psychophysiological variables. Psychophysiology, 7, 86–94. Tops, M., Van Peer, J. M., Wijers, A. A., & Korf, J. (2006). Acute cortisol administration reduces fatigue in healthy women. Psychophysiology, 43, 653–656. Ulrich-Lai, Y. M., & Herman, J. P. (2009). Neural regulation of endocrine and autonomic stress responses. Nature Reviews Neuroscience, 10, 397–409. Van Praag, H. M., De Kloet, E. R., & Van Os, J. (2004). Stress, the brain and depression. Cambridge: Cambridge University Press. Wirtz, P. H., Ehlert, U., Emini, L., Ru¨disu¨li, K., Groessbauer, S., Gaab, J., et al. (2006). Anticipatory cognitive stress appraisal and the acute procoagulant stress response in men. Psychosomatic Medicine, 68, 851–858. Wolkowitz, O. M., Rubinow, D., Doran, A. R., Breier, A., Berretini, W. H., Kling, M. A., et al. (1990). Prednisone effects on neurochemistry and behavior: Preliminary findings. Archives of General Psychiatry, 47, 963–968.
1. – .32n .47n .17n .12n .04 ! .29n ! .22n ! .21n .07 .05 .05 .03 .06 ! .04 ! .03 ! .02 ! .03 ! .05 ! .02 ! .02 .00 .01 2. – .35n .14n .41n .09n ! .21n ! .54n ! .17n .02 .04 .06 .05 ! .00 .05 .00 .03 .04 .04 ! .03 .06 .09n .08 3. – .12n .10n .19n ! .14n ! .19n ! .30n ! .04 ! .08n ! .10n ! .10n ! .05 .02 .08n .04 .05 .04 ! .03 ! .04 ! .05 ! .00 4. – .32n .29n ! .35n ! .19n ! .26n .04 .05 .08n .04 .04 ! .00 ! .02 .01 .03 .02 ! .02 ! .02 ! .04 ! .05 5. – .18n ! .24n ! .49n ! .21n ! .02 .05 .02 ! .01 ! .01 .05 ! .02 .04 .06 .05 ! .04 .07 .08 .06 6. – ! .08n ! .08 ! .20n .01 ! .07 ! .05 ! .07 ! .02 .00 .06 .03 .05 .04 ! .01 ! .07 ! .09n ! .10n 7. – .47n .59n ! .12n ! .13n ! .12n ! .07 ! .08n ! .01 .02 ! .02 ! .03 ! .03 .03 .02 .03 .06 8. – .44n ! .03 ! .09n ! .09n ! .06 ! .05 ! .03 .02 ! .03 ! .05 ! .03 .04 .02 .02 .03 9. – ! .06 ! .03 ! .04 ! .01 ! .04 ! .03 .01 ! .00 ! .01 ! .01 .07 .03 .06 .06 10. – .73n .61n .70n .81n ! .67n ! .57n ! .50n ! .51n ! .52n ! .05 ! .02 ! .00 ! .04 11. – .82n .83n .81n ! .41n ! .67n ! .42n ! .44n ! .45n ! .13n .04 .07 .01 12. – .87n .70n ! .37n ! .56n ! .37n ! .40n ! .40n ! .14n .09n .14n .08 13. – .78n ! .43n ! .49n ! .45n ! .48n ! .45n ! .13n .08n .17n .09n 14. – 15. ! .53n – ! .64n .69n ! .56n .73n ! .59n .72n ! .63n .74n ! .13n ! .02 ! .02 .02 .00 .03 ! .06 .03
16. – 17. .79n – 18. .79n .87n – n n .79 .85 .87n .08 .02 .03 .02 ! .03 .00 .00 ! .03 ! .00 .04 .02 .03
Note: Sampling weights were used to represent the distribution in the general population. HR, RSA, and cortisol variables were log-transformed before analysis. n po.05. a Exclusive of girls using oral contraceptives (N 5 589). b Immediately after the social stress test. c 20 min after the social stress test.
1. Arousal pretest 2. Arousal during test 3. Arousal posttest 4. Unpleasantness pretest 5. Unpleasantness during test 6. Unpleasantness posttest 7. Dominance pretest 8. Dominance during test 9. Dominance posttest 10. HR pretest 11. HR preparation 12. HR speech 13. HR mental arithmetic 14. HR posttest 15. RSA pretest 16. RSA preparation 17. RSA speech 18. RSA mental arithmetic 19. RSA posttest 20. Cortisol pretesta 21. Cortisol during testa 22. Cortisol end of testa,b 23. Cortisol posttesta,c
Appendix. Pearson Correlations Between the Various Stress Indices Before, During, and After the Social Stress Test
19. – .01 .01 .01 .03
20. – 21. .55n –. 22. .47n .87n – .51n .76n .87n
452 A. J. Oldehinkel et al.
Psychophysiology, 48 (2011), 453–461. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01122.x
Rumination in the laboratory: What happens when you go back to everyday life?
CRISTINA OTTAVIANI,a DAVID SHAPIRO,b and LEAH FITZGERALDc a
Department of Psychology, University of Bologna, Bologna, Italy Department of Psychiatry, University of California, Los Angeles, Los Angeles, California, USA c School of Nursing, University of California, Los Angeles, Los Angelos, California, USA b
Abstract Rumination has been suggested to mediate the physiological consequences of stress on health. We studied the effects of rumination evoked in the laboratory and subsequent changes over 24 h. Heart rate (HR) and systolic and diastolic blood pressure (SBP, DBP) were monitored in 27 male and 33 female participants during baseline, reading, an anger recall interview, and recovery. Half of the sample was assigned to a distraction condition. The lab session was followed by a 24-hour ambulatory (A)HR and BP recording and self-reports of moods and rumination. Rumination was associated with higher SBP, DBP, and HR and increased negative mood compared to distraction. Rumination during the day was a strong predictor of AHR, ABP, and mood. BP reactivity in the laboratory and increases in ABP during rumination were related. The effects of negative cognition on health go far beyond the recovery periods usually measured in the laboratory, thus playing a pathogenic role. Descriptors: Rumination, Distracter, Ambulatory, Moods, Cardiovascular
negative effects of rumination on sustained physiological activation (Rusting & Nolen-Hoeksema, 1998). It is likely that this recurrent and prolonged activation, rather than a single spike of activation, is more relevant as a potential risk factor in the gradual development of cardiovascular disease. One experimental task that has been demonstrated particularly effective in evoking ruminative thoughts is the anger recall interview (Ironson et al., 1992). Although instructions for the interview vary across studies, all of them report greater cardiovascular arousal and negative affect (Greeson et al., 2009; Prkachin, Mills, Zwaal, & Husted, 2001; Suarez, Saab, Llabre, Kuhn, & Zimmerman, 2004) compared to the effects of other commonly used laboratory stressors. The anger recall interview is an effective tool for evaluating emotional and physiological response differences, but it poses methodological issues that must be taken into consideration when evaluating sympathetic nervous system arousal. With the exceptions of the speech stressor (Saab, Matthews, Stoney, & McDonald, 1989) and mental arithmetic (Glynn et al., 2002), the anger recall interview differs significantly from traditional laboratory stressors in the degree of vocalization required from the participants. Vocalization alone, in the absence of emotional content, has been shown to elicit significant hemodynamic responses (Girdler, Turner, Sherwood, & Light, 1990; Lynch, Long, Thomas, Malinow, & Katcher, 1981). To account for the degree of sympathetic arousal associated with vocalization, in the first phase of the study we compared the effects of anger recall with the effects of a neutral speaking control condition, a reading task. A commonly used way to manipulate rumination after the recall task is to distract half of the sample (Gerin et al., 2006; Glynn et al., 2002; Rusting & Nolen-Hoeksema, 1998). In the
The ‘‘reactivity hypothesis’’ posits that people who are at risk for cardiovascular diseases are likely to be hyperreactive to stressors (Schwartz et al., 2003). A recent meta-analysis of prospective evidence (Chida & Steptoe, 2010) showed that greater reactivity to stress is associated longitudinally with poor cardiovascular status, including elevated blood pressure (BP), hypertension, left ventricular mass, subclinical atherosclerosis, and clinical cardiac events. Recent findings, however, suggest the need to improve this concept. It is now considered adaptive to react when we have to face a stressful event, but hyperreactivity becomes maladaptive if it continues when the source of stress is no longer present (Brosschot, Gerin, & Thayer, 2006). A clear example is provided by the recurrent physiological activation that has been associated with perseverative cognitions evoked by past stressful or angerprovoking events, that is, rumination (Rusting & Nolen-Hoeksema, 1998). Gerin, Davidson, Christenfeld, Goyal, and Schwartz (2006) proposed a theoretical model in which prolonged anger may promote ruminative thoughts, and increased autonomic arousal may prolong anger, with these two processes operating as a feed-forward process. This hypothesis is consistent with the findings of slower physiological recovery following personally relevant negative stressors (Glynn, Christenfeld, & Gerin, 2002) and with the findings that a distracter reduces the The authors thank Sarah Lee, Dave Bugayong, and Xiao Chen, Statistical Consulting Group, UCLA Academic Technology Services for their assistance. Address correspondence to: Cristina Ottaviani, Dipartimento di Psicologia, viale Berti Pichat, 5 – 40127 Bologna, Universita` degli Studi di Bologna, Italy. E-mail:
[email protected] 453
454 present study, the experimenter pretended to receive a phone call right after the interview as a distracter. We considered this more realistic and ecological compared to the distracters that have been usually employed, such as puzzles or graphic effects on a computer screen. Thus, the first aim of this study was to replicate the important findings of others that a distracter after anger recall is effective in speeding cardiovascular recovery. Our second and main purpose was to examine the consequences of perseverative cognition on cardiovascular responses outside the laboratory. In fact, in order to be pathogenic, rumination has, indeed, to be associated with sustained physiological activity in the laboratory but primarily with this sustained activity in everyday life. A major shortcoming in reactivity-based research is the failure to capture laboratory-to-life generalizability (Carroll et al., 2001). Cardiovascular reactivity in the laboratory, together with the recovery that immediately follows the end of a negative event, provides us with a small window inside a wider and chronic pattern of stress response. Portable instrumentation makes the recording of cardiovascular changes possible in everyday life and allows us to capture the presence of physiological alterations exactly at the moment when they are likely to happen, that is, in response to environmental demands. The diary is the instrument that allows us to get information on these environmental stimuli and the real-time subjective response to them. Although laboratory studies have yielded suggestive evidence that slow cardiovascular recovery after emotional stress is due to worry or rumination, only two studies have tested this hypothesis in everyday life (Brosschot, Van Dijk, & Thayer, 2007; Pieper, Brosschot, van der Leeden, & Thayer, 2007). The first study showed the direct effects of worry on cardiac activity during both waking and subsequent nocturnal sleep, thus demonstrating that these responses extend even into periods in which ‘‘concrete’’ stressors are absent. The second study further confirmed that worry in daily life has substantial cardiac effects in addition to the immediate effects of stressful events, especially during work-related and anticipatory stress. By using an ambulatory session that immediately followed the laboratory phase, we were able (1) to test if participants who are characterized by greater reactivity to the anger recall task and to the ruminative thoughts evoked in the laboratory also show higher cardiovascular reactivity to rumination during the day and (2) to examine the relationship between daily rumination and ambulatory BP and heart rate (HR). As previous ambulatory studies on rumination focused on cardiac activity, this is the first study to study the effect of daily rumination on BP in an ecological environment. Moreover, in the recent literature on ruminative thoughts and cardiovascular mechanisms, the link between the tendency to ruminate and concurrent mood has not been studied. If rumination is accompanied by negative affect, we can expect it to have an impact on the moods experienced during the day. Considering the role played by daily experiences of negative moods in determining higher BP levels (Shapiro, Jamner, Goldstein, & Delfino, 2001), we can hypothesize that the tendency to ruminate affects ambulatory BP by worsening daily mood. Thus, we examined the relationship between mood states and ruminative thoughts occurring during the day following the laboratory session. Finally, we tested if the tendency to ruminate as a personality trait (Porter, Stone, & Schwartz, 1999) has an effect on daily frequency and duration of rumination episodes, frequency of rumination episodes about the content of the laboratory task, and ambulatory cardiovascular parameters.
C. Ottaviani et al. Methods Sample Description Participants were recruited in the general community using ads and flyers and via weekly updated postings on Craig’s List, an online classified advertisement system. Exclusionary criteria were psychiatric disorders, diagnosis of hypertension or heart disease, history of cancer, pulmonary problems, active chronic infections, autoimmune diseases (e.g., rheumatoid arthritis, multiple sclerosis), diabetes, endocrine disorders, immunosuppression resulting from a disease (e.g., malignancy, HIV infection), use of drugs or medications that might affect cardiovascular function and/or catecholamines, obesity (body mass index 432 kg/m2), menopause, use of oral contraceptives during the previous 6 months, and pregnancy or childbirth within the last 12 months. The sample was composed of 27 men (mean age 5 31.8 ! 10.1 years; age range 21–54 years), 5 Asian, 13 Caucasian, 2 African, and 7 Latino Americans, and 33 women (mean age 5 34.7 ! 8.6 years; age range 20–54 years), 7 Asian, 10 Caucasian, 15 African, and 1 Latino American. Participants were paid $100. The protocol was approved by the University of California, Los Angeles, Institutional Review Board.
Procedure The laboratory session took place between 7:00 a.m. and 12:00 p.m. Participants were asked to refrain from drinking alcohol, tea, or coffee and strenuous exercise the morning of testing. After providing written informed consent, participants were seated in a comfortable chair and instrumented for electrocardiogram (ECG) and continuous BP monitoring. The laboratory protocol consisted of an initial 10-min baseline period, followed by a 5-min reading task in which participants were instructed to read aloud a passage about the ocean using ‘‘a normal tone of voice and normal rate of speech.’’ To ensure minimal emotional arousal, participants were informed that they would not be evaluated for reading style or comprehension. The reading task was followed by a 10-min rest period and a 5-min anger recall interview. Participants were asked to verbally describe a personal event that occurred within the last 3 to 4 months that elicited anger and ‘‘when thinking about it today’’ continues to arouse anger (Ironson et al., 1992). After a 1-min period for preparation, participants were asked to verbally describe the event to the experimenter for approximately 5 min. To assist the participant in recalling the event, the experimenter used prompts, such as ‘‘How did that make you feel when it happened?’’ The session ended with a 10-min resting period. During the first 2 min of this resting period, half the sample was randomly assigned to the distraction condition, in which the laboratory phone rang, the experimenter went into an adjacent room leaving the door open, and spoke loudly about job issues for approximately 2 min. For the nondistracted subjects the experimenter left the room and closed the door. Affect ratings were collected at baseline and at the end of each recovery period. At the end of the laboratory session, participants were instructed about the use of the diary and the Accutracker II (Suntech Medical Instruments, Raleigh, NC) ambulatory BP and HR device. The electrodes and apparatus were attached, and the participants left the laboratory. The next morning, the participants returned the diary and apparatus to the laboratory, filled out the personality questionnaire, were debriefed, and received monetary compensation.
Anger rumination: An ambulatory study Psychophysiological Assessment ECG was monitored with a multitrace recorder (AcqKnowledge, Biopac System, Santa Barbara, CA) and a standard electrode configuration (right clavicle and precordial site V6). Three disposable Ag-AgCl electrodes (ConMed Corp.) were used. Beat-to-beat BP was measured noninvasively using a Finapres Continuous NIBP Monitor (Ohmeda, Englewood, CO) via a finger cuff attached to the third finger of the nondominant hand. The Finapres has been shown to be a suitable device for reliable tracking of changes in BP (Imholz, Wieling, van Montfrans, & Wesseling, 1998). Affect Ratings Participants’ affect ratings were collected immediately following baseline and after the recovery periods following the reading and anger recall tasks. Participants were asked to rate their level of arousal using a 5-point Likert scale with 1 representing not at all and 5 representing very strong feelings for the following affects: stressed, happy, irritated, sad, frustrated, relaxed, depressed, optimistic, tired, anxious, annoyed, calm, aggravated, cool, and angry. State and Trait Rumination At the end of the recovery period following the anger recall, participants were asked to report if they had been thinking back about the content of the interview throughout the 10-min period following the task. This information (yes/no) was used as an indicator of the presence of rumination as a state. The Stress-Reactive Rumination Scale (SRRS) was administered at the end of the ambulatory session as a measure of the tendency to ruminate after stressful events. The scale was designed to measure rumination in a manner that is not confounded with depressive symptoms, a limitation of many other self-report rumination scales (Robinson & Alloy, 2003). The scale has the following subscales: Negative Inferential Style, Hopelessness, and Active Problem-Solving. Ambulatory Assessment Ambulatory 24-h systolic and diastolic BP (SBP, DBP) were obtained during a work day. The Accutracker II has been widely used with established reliability and validity in clinical and research studies (Jyothinagaram, Watson, & Padfield, 1990). The Accutracker II was programmed to operate at varying intervals approximately every 20 min during waking hours and once an hour during sleep. Ambulatory data were edited for artifacts based on Accutracker reading error codes, insufficient electrocardiogram or Korotkoff sounds, and extreme values (4200/120 or o70/40 mm Hg). We obtained a mean number of 41.2 (SD: 8.8, median: 42, range: 26–61) readings per subject. Given the reasonable number of readings, all subjects were included in the analysis. The average number of rumination readings per subject was 12.2 (SD: 6.9, median: 11, range: 2–34). Diary Participants were provided with a paper-and-pencil diary that had the definition of rumination as ‘‘the process of thinking perseveratively about one’s feelings and problems’’ on the first page. Each time they felt the BP cuff inflate during waking hours, participants were asked to complete the diary. Each entry asked for the presence and duration of rumination, stressors, or both during the preceding entry period and information on factors that may affect BP, including posture, physical activity, and food, caffeine, nicotine, and alcohol consumption since the last
455 diary report. Ruminative thought frequency was measured by the number of times the presence of intrusive ruminative thoughts were reported divided by the total number of readings. Rumination duration was noted by the participant for each rumination episode using the following scale: 0–1 min, 1–5 min, 5– 20 min, or more than 20 min. Participants reported the last option (more than 20 min) if they did not stop ruminating since the previous BP measurement. If participants reported ruminating, they were also asked to report if it was on the content of the laboratory task. Stressors were assessed by asking participants whether they experienced one or more annoying or disturbing events in the preceding period (for a positive answer, they were also asked to describe the events). On each cuff inflation, participants also rated moods (stressed, happy, irritated, sad, frustrated, relaxed, depressed, optimistic, tired, anxious, annoyed, calm, aggravated, cool, and angry) using a 5-point scale from not at all to very much. Statistical Analyses All data are expressed as means (SD). Differences at po.05 were regarded as significant. Laboratory data processing and data analyses were performed with the software modules of Systat 9.0 (Systat Software Inc., Richmond, CA). Reactivity change scores (D) were computed by subtracting the initial baseline average value from each average task value. Recovery scores were determined by subtracting the mean level obtained during the baseline from the average level measured during the recovery period after each task. Averages for baseline, tasks, and recovery were computed for each entire time period (10, 5, and 10 min, respectively). Raw change scores were used instead of residuals (Llabre, Spitzer, Saab, Ironson, & Schneiderman, 1991). To have a more reliable measure, recovery was also computed according to the method suggested by Christenfeld, Glynn, and Gerin (2000; ‘‘Curve Fitting Technique’’). With regard to moods, five representative moods (happy, stressed, tired, anxious, and angry) were selected following the methodology used by Shapiro et al. (2001). Based on exploratory principal components analysis, the authors chose one negative word (stressed), one positive word (happy), and one indicator of energy level (tired). We also included anxious and angry because of the commonly explored role of anxiety and anger in BP regulation. For the same reason, the mood ‘‘sad’’ was added because of the role played by depression in rumination, cardiovascular activity, and health. To evaluate the effects of sociodemographic factors, Pearson correlations were performed between BMI, age, physical activity, caffeine, alcohol, nicotine consumption, and baseline levels of each physiological variable. Differences due to gender and ethnicity were analyzed by t test and analysis of variance (ANOVA). To control for the presence of preexisting differences between the distraction and the nondistraction subgroups, the groups were compared by t tests for the following variables: age, rumination tendencies (total and subscale scores of the SRRS), habitual consumption of caffeine, nicotine, and alcohol, and baseline levels of SBP, DBP, and HR. Chi square comparisons were conducted for gender, ethnicity, and moods at baseline. To determine the effectiveness of the anger recall interview in inducing psychophysiological activation compared to baseline and to the neutral reading task that acted as a control for vocalization, ANOVAs with task as a repeated measure (baseline, anger recall, reading) were conducted for HR, SBP, DBP, and
456 each mood. To evaluate the effectiveness of the distracter condition in determining greater recovery to baseline levels after the anger recall task, t tests by group were performed on change scores for each physiological and mood variable. To test the relationship between cardiovascular reactivity and rumination in the laboratory and in daily life, Pearson correlations were performed between laboratory SBP, DBP, and HR reactivity and recovery change scores and ambulatory SBP, DBP, and HR change scores (rumination periods minus nonrumination periods) for the nondistracted participants. To test the effect of the distraction condition outside the laboratory, t tests by group were performed on frequency and duration of daily rumination episodes, frequency of rumination about the anger recall topic, and ambulatory SBP, DBP, and HR. To determine the effects of trait rumination, Pearson correlations were performed between scores on the SRRS (total score and subscales) and frequency and duration of daily rumination episodes and frequency of rumination about the anger recall topic. Random effects regression models are the most appropriate methods of analysis for the relationship between daily rumination and ambulatory BP and HR, as the ambulatory measures consist of repeated measures of SBP, DBP, HR, and diary variables (Shapiro et al., 2001). PROC MIXED (SAS Institute) was the program employed for general linear mixed modeling. This approach is particularly suitable, as the periodicity of rumination periods, moods, and physiological measurement is likely to be highly heterogeneous, and it also deals with missing values. Because it models each participant as a random effect, using this procedure accommodates interindividual variation in rumination–mood–BP or rumination–mood–HR relationships. At this step, only the biobehavioral variables that had a significant bivariate correlation with a given cardiovascular variable were entered, because the number of measured biobehavioral variables was so large that entering them all would greatly decrease the degrees of freedom for the present sample size. As to posture, under conditions controlled for activity level, sitting and standing BP differ only slightly (Goldstein & Shapiro, 1988), and there is no basis for assuming that mood is related to BP as a function of specific bodily position. Slight differences in sample sizes are due to missing values for some of the variables. First, rumination was related to each dependent variable: SBP, DBP, and HR during wake. Then, rumination was related to each single mood. To derive the variance of daily BP, HR, and moods accounted for by rumination in each model, the random effects regression models (Proc Mixed) required the use of the ‘‘Pseudo R-Squared method,’’ as recommended by Singer and Willett (2003). To correct for multiple comparisons, Pearson correlations, post hoc tests, and t tests have been performed with the use of the ‘‘Bonferroni correction’’ option provided by Systat. Adjusted p values are presented. By default, PROC MIXED adjusts all pairwise differences.
Results Sociodemographic and Trait Characteristics Pearson correlations showed an association between BMI and baseline SBP (r 5 .32; p 5 .01). No associations were found for age, caffeine, alcohol, nicotine, exercise, and baseline physiological levels. Baseline differences between the distracted and the
C. Ottaviani et al. Table 1. Sociodemographic Characteristics
Age (years) BMI (kg/m2) Education (years) Incomea Caffeine (cups/day) Alcohol (glasses/week) Physical activity (h/ week)b Smoking status
Distracted (n 5 30)
Nondistracted (n 5 30)
34.1 (9.8) 25.3 (5.1) 15.1 (3.2) 2.4 (1.5) 1.0 (1.3) 3.6 (3.7) 11.7 (7.9)
32.7 (9.3) 24.8 (6.2) 15.9 (3.3) 2.1 (1.1) 0.8 (1.2) 4.4 (6.5) 8.9 (6.8)
20 never, 5 past, 5 present
20 never, 6 past, 4 present
a 1 5 o$20,000; 2 5 $20,000–$35,000; 3 5 $35,000–$50,000; 4 5 $50,000–$65,000; 5 5 $65,000–$80,000; 6 5 $80,000–$95,000; 7 5 4$95,000. b Subjects were asked to indicate the types and amounts of exercise they did (hours/week) among the following: household chores; golf, softball, & baseball; yoga & stretching; tennis, handball, & other active sports; walking & dancing; jogging, running, & swimming; weight lifting; other.
nondistracted groups were not significant for the examined variables (see Table 1). No gender differences emerged. Significant ethnicity differences appeared for ambulatory SBP, F(3,56) 5 3.95, p 5 .01, and DBP, F(3,56) 5 4.84, p 5 .001, during wake. Post hoc comparisons showed that African Americans had higher ambulatory SBP during wake (126.2 mm Hg) than Caucasians (118.4 mm Hg) and Asians (110.4 mm Hg), but no differences emerged with Latinos (124.8 mm Hg). African Americans had also higher ambulatory DBP during wake (75.8 mm Hg) compared to Asians (66.2 mm Hg). Trait rumination did not have an effect on frequency or duration of rumination episodes or frequency of rumination episodes about the content of the anger recall interview or ambulatory BP or HR. Laboratory Findings The repeated measures ANOVAs revealed a significant effect of task for SBP, F(2,58) 5 164.77, po.0001, DBP, F(2,58) 5 143.68, po.0001, HR, F(2,58) 5 42.15, po.0001 (see Figure 1). Pairwise t tests indicated that the anger interview led to greater activation compared to baseline for SBP, D 5 18 mm Hg, t(1,59) 5 ! 13.13, po.0001, DBP, D 5 13 mm Hg, t(1,59) 5 ! 13.03, po.0001, and HR, D 5 6 bpm, t(1,59) 5 ! 9.79; po.0001, and compared to the reading task for SBP,D 5 18 mm Hg, t(1,59) 5 ! 13.54, po.0001, DBP, D 5 13 mm Hg, t(1,59) 5 ! 12.61, po.0001, and HR, D 5 3 bpm, t(1,59) 5 ! 4.04, po.0001. With regard to moods, the repeated measures ANOVAs revealed a significant effect of task for the following moods: happy, F(2,58) 5 15.64, po.0001, stressed, F(2,58) 5 15.95, po.0001, angry, F(2,58) 5 31.02, po.0001, tired, F(2,58) 5 5.66, po.001, and sad, F(2,58) 5 7.63, po.001 (see Figure 2). Anxious, F(2,58) 5 0.40, p 5 .67, did not yeld a significant effect. Specifically, participants were less happy and more stressed, angry, tired, and sad after the anger recall interview compared to baseline and to the reading task. No differences between baseline and the reading task emerged. Table 2 shows means and standard deviations for each mood in the three conditions. Twenty-eight out of 30 participants (93.3%) assigned to the distracter condition reported having not thought about the event during the recovery period following the anger recall, whereas all
Anger rumination: An ambulatory study
457
SBP (mm Hg)
Baseline
Task
DBP (mm Hg)
Recovery Baseline
Anger recall (distracter)
Task
HR (bpm)
Recovery Baseline
Anger recall (no distracter)
Task
Recovery
Reading
Figure 1. Effects of task (anger recall, reading) and distracter on change scores from baseline for HR, DBP, and SBP.
participants who were assigned to the nondistracter condition thought back about that specific event during the 10 min following the interview. Thus, self-reported state rumination during recovery depended on the presence/absence of the distracter. The distracter condition was effective in determining greater recovery to baseline values (smaller change scores) compared to the nondistracter condition for SBP, t 5 9.31, po.0001, DBP, t(2,58) 5 9.20, po.0001, and HR, t(2,58) 5 6.94, po.0001. Figure 1 shows the effects of task (anger recall, reading) and distracter on change scores from baseline for each of the examined physiological variables. The use of the curve fitting technique did not significantly change the results. The distracter condition was effective in determining a change in moods compared to baseline values (change scores) compared to the nondistracter condition for angry, t(2,58) 5 5.12, po.0001, happy, t(2,58) 5 ! 4.20, po.0001, and stressed, t(2,58) 5 4.79, po.0001 (see Figure 2) and was not effective for tired, anxious, and sad. Specifically, there was an increase in happy levels ( ! 0.1 vs. ! 1.2) and a decrease in angry (0.3 vs. 1.5) and stressed ( ! 0.1 vs. 1.2) levels. Ambulatory Findings Table 3 shows mean ambulatory SBP, DBP, and HR, mood intensity, stressor frequency, and rumination frequency and duration. Preliminary analyses showed significant effects only for age and BMI in determining ambulatory cardiovascular measures. Consequently, only these two biobehavioral variables were included as covariates in the random effects regression models. Pearson correlations showed an association between SBP and DBP reactivity during the anger interview and ambulatory SBP and DBP change scores between rumination and nonrumination periods, r 5 .38, p 5 .04, and r 5 .61, po.0001, respectively. For the nondistracted subsample, partial correlation analyses were performed to control for the effect of reactivity on the relationship between SBP and DBP recovery after the interview and Anger Recall (distracter) 1.4 1.2 1 0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1 −1.2 −1.4
Anger Recall (no distracter)
Reading
ambulatory SBP and DBP change scores between rumination and nonrumination periods. Rumination about the task was higher in nondistracted participants (50%) compared to distracted participants (30%), but the difference was not significant. Among all rumination episodes, the daily mean percentage of those related to the content of the anger interview was 43%. Daily moods and ambulatory SBP, DBP, and HR were not significantly different for distracted and nondistracted participants. The first random effects regression model tested the effect of daily state rumination on ambulatory SBP, DBP, and HR during wake. Age was not a significant predictor in the model. BMI had a significant effect on SBP and DBP, F(1,2381) 5 22.13, po.0001, and F(1,2381) 5 12.08, p 5 .0005, respectively. After controlling for the effects of age and BMI, rumination was a significant predictor in the model, F(1,2381) 5 1541.9, po.0001, DR2 5 .29 for SBP, F(1,2381) 5 594.3, po.0001, DR2 5 .18 for DBP, and F(1,2375) 5 22.13, po.0001, DR2 5 .05 for HR. Figure 3 shows relationships between rumination, SBP, DBP, and HR. The second random effects regression model tested the effect of daily state rumination on daily moods. Rumination was a significant predictor of moods for levels of stressed, F(1,2375) 5 831.27, po.0001, R2 5 .18, happy, F(1,2375) 5 503.67, po.0001, R2 5 .09, sad, F(1,2375) 5 166.13, po.0001, R2 5 .04, anxious, F(1,2375) 5 115.66, po.0001, R2 5 .05, and angry, F(1,2375) 5 1315.08, po.0001, R2 5 .35. Specifically, participants rated themselves as more stressed, angry, anxious, and sad and less happy during rumination compared to nonrumination periods (Figure 4).
Discussion Our study replicated previous results about the effectiveness of a distracter, in this case a simple one such as occurs in everyday lifeFa phone callFto stop rumination and speed cardiovascular recovery. Moreover, the findings showed that rumination Table 2. Mean and Standard Deviation of Each Mood during Baseline and after Recovery from the Reading Task and the Anger Recall Interview
Happy
Angry
Stressed
Figure 2. Effects of task (anger recall, reading) and distracter on mean mood rating (change scores from baseline).
Happy Stressed Angry Anxious Sad Tired
Baseline
Reading
Anger
3.4 (1.0) 2.1 (1.0) 1.2 (0.5) 2.2 (1.3) 1.4 (0.8) 2.7 (1.4)
3.4 (1.0) 1.8 (1.0) 1.2 (0.6) 2. 1 (1.2) 1.4 (0.8) 2.7 (1.4)
2.7 (1.2) 2.6 (1.3) 2.1 (1.1) 2.1 (1.3) 1.8 (1.1) 2.4 (1.3)
458
C. Ottaviani et al.
0.6 (0.4) 0.3 (0.2) 0.4 (0.4)
0.1–2.1 0–0.7 0–0.8
3.1 (0.9) 1.9 (0.7) 1.5 (0.4) 1.8 (0.7) 1.3 (0.5) 2.2 (0.9)
1.1–4.8 1.0–4.6 1.0–2.9 1.0–4.3 1.0–2.8 1.0–4.8
Happy
Note: Rumination duration is the average duration for the times participants were ruminating before each reading (1 5 0–1 min; 2 5 1–5 min; 3 5 5–20 min; 4 5 more than 20 min). Frequencies for stressful events and rumination periods were computed by the number of times the event/ rumination occurred by the total number of readings. Frequency for rumination on task was computed by dividing the number of episodes dedicated to rumination on the laboratory task by the total number of rumination periods. Moods ratings are the means of the intensity ratings for each mood (1 5 not at all; 5 5 very much).
3
2.5 2
1.5
1
1
3.5
3.5
3
3
2.5
2.5
2
1.5
1
1
3.5
3.5
3
3
2.5
2.5
2
1.5
1
1
With regard to laboratory findings, reading aloud determined a lower level of activation compared to the effects of the anger recall interview, as shown by the direct statistical comparison between the two tasks for change scores in cardiovascular variables. We can, therefore, conclude that vocalization by itself did not account for the cardiovascular reactivity to the anger interview and subsequent delayed recovery. The use of a simulated phone call as a distraction condition was effective in stopping rumination. In fact, 93% of participants who were distracted reported not having thought about the recalled event during the poststress period. In contrast, 100% of
80
80
HR (bpm)
85
DBP (mm Hg)
85
75 70
Rumination
75 70 65
65
110
No
Figure 4. Mean mood rating for the periods that participants were ruminating or not ruminating.
135
115
2
1.5
90
120
2
1.5
90
125
2
1.5
140
130
2.5
Yes
outside the laboratory was associated with large increases in ambulatory SBP and DBP, 19 mm Hg and a 11 mm Hg, respectively, after statistical adjustments for age and BMI. This result is particularly intriguing if we consider the prognostic value of BP for the development of cardiovascular disease (Boggia et al., 2007), and that a 10 mm Hg change is the target to evaluate the efficacy of antihypertensive medications (Ishikawa, Carroll, Kuruvilla, Schwartz, & Pickering, 2008). We also showed a relationship between BP reactivity to the anger interview in the laboratory and ambulatory BP increases from nonrumination to rumination periods in daily life. The present data suggest that individuals have a cardiovascular reaction of about the same magnitude when they ruminate about an emotional episode as when they talk about it in detail. Thus, in terms of cardiovascular arousal, thinking about a negative event is as stressful as the event itself.
SBP (mm Hg)
3
Anxious
51.9–93.5 48.4–91.0 99.6–148.1 82.2–149.3 57.9–85.8 44.5–84.5 0–0.8
3.5
Angry
76.3 (8.9) 66.3 (10.7) 119.9 (11.3) 106.9 (15.5) 71.3 (6.4) 60.0 (7.6) 0.2 (0.2)
3.5
Sad
Range
Stressed
HR wake HR sleep SBP wake SBP sleep DBP wake DBP sleep Stressful events (freq.) Rumination Duration Frequency Task Mood Happy Stressed Angry Anxious Sad Tired
Mean (SD)
Tired
Table 3. Mean, Standard Deviation and Range for Ambulatory Variables
60
60
Rumination YES
Rumination
NO
Figure 3. Mean ambulatory SBP, DBP, and HR for the periods that participants were ruminating or not, adjusted for age and BMI.
Anger rumination: An ambulatory study those who were not distracted had intrusive cognitions about it. It has to be acknowledged, however, that the interpretation of the effects of a distracter has been questioned by a few studies (Gerin et al., 2006; Key, Campbell, Bacon, & Gerin, 2008), in which rumination in the nondistracter condition did not explain or only partially and inconsistently (i.e., moderated by trait rumination) explained the slower cardiovascular recovery in that condition. In our study, however, besides the effects at a physiological level, participants who ruminated in the laboratory also became less happy and more stressed, angry, tired, and sad, although rumination/distraction did not have any effect on anxiety levels. This confirms the theoretical distinction between worry and rumination, where the first regards a concern for future events and is associated with anxiety- and depression-related symptoms, and the second is related to past events and seems to be characterized only by a depressed mood (Hong, 2007). Moreover, as previously noted for anger experience (Rusting & Nolen-Hoeksema, 1998), the distracter turns out to be efficient in reducing stress and anger levels and to increase self-reported happiness levels but is not sufficient to diminish sadness and tiredness levels evoked by the interview. The phone call distraction simulation had only an immediate effect in stopping rumination: As soon as participants went back to everyday life, rumination frequency, mood, BP, and HR levels were not affected by having been distracted or not during the laboratory session. Our result of only a short-term effect of the distracter in stopping rumination is consistent with previous findings for shorter time lags (Gerin et al., 2006). In fact, these authors found that as soon as they removed the distracter during the laboratory setting, BP increased again, although not as much as during the stressful task itself. It seems plausible that these participants started to ruminate again when the distracter was no longer there. Taken together, these results have therapeutic implications, such as the ineffectiveness of distraction as a clinical practice to treat rumination. Contrary to results from our previous study (Ottaviani, Shapiro, Davydov, Goldstein, & Mills, 2009), which indicated a stronger vagal withdrawal in women compared to men, but in agreement with other authors’ findings (Glynn et al., 2002; Suarez et al., 2004), gender differences in autonomic activation were not shown during the anger recall task. It has to be noted that our previous study only required participants to think about the anger episode, whereas in the present and other studies that got comparable results, participants had to talk about this episode (Glynn et al., 2002; Suarez et al., 2004). Therefore, inconsistencies in the results could be explained by this differences in methodology. With regard to the individual propensity to ruminate, results fail to show gender differences or associations with physiological variables. The lack of trait rumination effects found in this study could be explained by the questionnaire used to evaluate this dispositional characteristics. We chose SRRS because it is the only questionnaire that distinguishes rumination from other depressive symptoms; however, it should be noted that previous studies that showed gender differences or correlations between trait rumination and cardiovascular parameters used different questionnaires. The present study did not show differences in the BP and HR effects of rumination due to ethnicity. Likewise, Suarez et al. (2004) failed to demonstrate ethnicity effects on cathecolamines levels but did show higher BP during rumination for African Americans. Moreover, difficulties in cardiovascular recovery af-
459 ter a stressful task for this ethnic group were previously highlighted (Dorr, Brosschot, Sollers, & Thayer, 2007). Again, the divergence could be due to task differences, considering that the authors used the simulation of two debates to elicit rumination, and one of them was on ethnic issues. As the recalled episodes did not have any kind of ethnic connotation, the neutralization of differences between ethnic groups becomes plausible. There are only two studies that have monitored the effects of anger induction after a time lag (Glynn, Christenfeld, & Gerin, 2007; Wimalaweera & Moulds, 2008). Wimalaweera and Moulds demonstrated the importance of focusing on one event in order to see the negative consequences of rumination after 24 h of time. Glynn et al. (2007) showed that being harassed during a mental arithmetic task leads to BP increases of the same degree when recalled after 30 min and an entire week. In both cases, subjects were required to go back to the laboratory and recall the episode during psychophysiological monitoring. Responses were not monitored during the day. Again, only two studies (Brosschot et al., 2007; Pieper et al., 2007) monitored the relationship between spontaneous episodes of worry and ambulatory HR and HR variability. Unlike in the present study, however, rumination was not induced in the laboratory, and ambulatory BP was not recorded. Consistent with the noxious effects of worry on cardiac activity observed by those studies, we extended the results to BP, showing that rumination is responsible for relevant changes in ambulatory BP. Moreover, we found preliminary evidence of a possible link between BP response to emotional stress in the laboratory and to perseverative cognition in daily life. The present study further supports the notion of negative mood as a fundamental component of rumination (Lyubomirsky & Nolen-Hoeksema, 1995). In fact, participants reported higher levels of sadness, stress, and anger and lower levels of happiness during ruminative thoughts in both the laboratory and the ambulatory sessions. Contrary to what we observed in the laboratory, rumination in everyday life was associated with a concomitant increase in anxiety levels. A possible explanation comes from the findings of lower self esteem and less confidence associated with rumination in doing routine activities (Ward, Lyubomirsky, Sousa, & Nolen-Hoeksema, 2003). It is plausible that the consequences on anxiety are more evident in daily life, when people face ordinary events. Results accord with previous findings on the perpetuation of negative mood, assessed on a daily basis, by the recall of a past negative feeling (Verduyn et al., 2009). Negative life events, rumination, and moods were likewise registered for an entire week, showing that daily stress level, obtained by a composed score of anxiety, sadness, and irritation levels, was mostly determined by ruminative thoughts (Moberly & Watkins, 2008). Present results further extend these observations, showing the crucial role of rumination in mediating the relationship between moods and ambulatory BP. Several limitations have to be acknowledged. First, the use of distraction has been sometimes criticized because it has been considered more appropriate to compare rumination with, for example, reappraisal, that is, an equivalent mental process without the negative component that characterizes rumination (Mauss, Bunge, & Gross, 2007). Second, cardiovascular activity outside the laboratory has been recorded only during a single work day, whereas Kamarck et al. (2002) recommended a period of recording that includes, at least, one nonwork day. We used one day because cardiovascular measures between a work and a nonwork day did not differ (Ottaviani, Shapiro, Goldstein,
460
C. Ottaviani et al.
James, & Weiss, 2006; Ottaviani, Shapiro, Goldstein, & Mills, 2007). Third, a binary variable (yes/no) was used for ruminative thoughts: The use of a continuous measure derived from composite information (How many? How often? How long? How negative?) could have led to a more precise prediction and is warranted for future studies. Finally, several prospective studies examined the relationship between delayed poststress recovery and the development of essential hypertension (Steptoe & Marmot, 2005; Stewart, Janicki, & Kamarck, 2006) and indicated poststress recovery as an independent risk factor for the development of hypertension and coronary diseases (Steptoe, Donal, O’Donnell, Marmot, & Deanfield, 2006; Bigi, Gregori, Cortigiani, Colombo, & Fiorentini, 2005; Heponiemi et al., 2007; Steptoe & Wardle, 2005) but, up to now, there are no studies on the noxious effect of rumination that obtained a follow-up objective measure of health status. A few exceptions are the demonstration of the consequences of worry on myocardial infarction (Kubzansky et al., 1997) and another study showing that worry mediates long-term cardiovascular effects of a major stressor (Holman et al., 2008). Another prospective study showed a relationship between worry and somatic complaints and the possibility of reducing both the latter by a worry reduction intervention (Brosschot & Van Der Doef, 2006). Our study does not represent an exception, given that the consequences of prolonged activation can only be hypothesized on
the basis of theoretical knowledge on the effects of impaired cardiovascular recovery, ambulatory BP, and moods. Moreover, our participants were relatively young and generally healthy. The phenomenon of delayed cardiovascular recovery to emotionally stressful tasks may have quite different prognostic implications in patients with significant coronary disease. However, the ambulatory effects that we obtained are striking and larger compared to any other study in the field. As a possible bias, it has to be acknowledged that the text of our postingF‘‘Want to know what happens in your body when you get angry?’’Fmight have attracted people who are particularly prone to experience anger in their life, thus affecting the magnitude of the effects. To conclude, in order to be pathogenic, rumination has, indeed, to be effective in determining sustained physiological activity outside the laboratory in everyday life. Present data further confirmed this hypothesis, further suggesting a relationship between cardiovascular reactivity to an emotional stressor in the laboratory and BP increases during negative thoughts. It is therefore becoming clear that not only are stressful life events pathogenic but also thoughts related to stressful events. Given these findings, we need to conduct prospective studies to demonstrate the long-term health consequences of rumination on health and the subsequent development of therapeutic approaches to the control of stress rumination.
REFERENCES Bigi, R., Gregori, D., Cortigiani, L., Colombo, P., & Fiorentini, C. (2005). Stress recovery index for risk stratification of asymptomatic patients following coronary bypass surgery. Chest, 128, 42–47. Boggia, J., Li, Y., Thijs, L., Hansen, T. W., Kikuya, M., Bjo¨rklundBodega˚rd, K., et al. (2007). Prognostic accuracy of day versus night ambulatory blood pressure: A cohort study. Lancet, 370, 1219–1229. Brosschot, J. F., Gerin, W., & Thayer, J. F. (2006). The perseverative cognition hypothesis: A review of worry, prolonged stress-related physiological activation and health. Journal of Psychosomatic Research, 60, 113–124. Brosschot, J. F., & van der Doef, M. (2006). Daily worrying and somatic health complaints: Testing the effectiveness of a simple worry reduction intervention. Psychology & Health, 21, 19–31. Brosschot, J. F., Van Dijk, E., & Thayer, J. F. (2007). Daily worry is related to low heart rate variability during waking and the subsequent nocturnal sleep period. International Journal of Psychophysiology, 63, 39–47. Carroll, D., Smith, G. D., Shipley, M. J., Steptoe, A., Brunner, E. J., & Marmot, M. G. (2001). Blood pressure reactions to acute psychological stress and future blood pressure status: A 10-year follow-up of men in the Whitehall II study. Psychosomatic Medicine, 63, 737–743. Chida, Y., & Steptoe, A. (2010). Greater cardiovascular responses to laboratory mental stress are associated with poor subsequent cardiovascular risk status: A meta-analysis of prospective evidence. Hypertension, 55, 1026–1032. Christenfeld, N., Glynn, L. M., & Gerin, W. (2000). On the reliable assessment of cardiovascular recovery: An application of curve-fitting techniques. Psychophysiology, 37, 543–550. Dorr, N., Brosschot, J. F., Sollers III, J. J., & Thayer, J. F. (2007). Damned if you do, damned if you don’t: The differential effect of expression and inhibition of anger on cardiovascular recovery in Black and White males. International Journal of Psychophysiology, 66, 125–134. Gerin, W., Davidson, K. W., Christenfeld, N. J., Goyal, T., & Schwartz, J. E. (2006). The role of anger rumination and distraction in blood pressure recovery from emotional arousal. Psychosomatic Medicine, 68, 64–72. Girdler, S. S., Turner, J. R., Sherwood, A., & Light, K. C. (1990). Gender differences in blood pressure control during a variety of behavioral stressors. Psychosomatic Medicine, 52, 571–591.
Glynn, L. M., Christenfeld, N., & Gerin, W. (2002). The role of rumination in recovery from reactivity: Cardiovascular consequences of emotional states. Psychosomatic Medicine, 64, 714–726. Glynn, L. M., Christenfeld, N., & Gerin, W. (2007). Recreating cardiovascular responses with rumination: The effects of a delay between harassment and its recall. International Journal of Psychophysiology, 66, 135–140. Goldstein, I. B., & Shapiro, D. (1988). Cardiovascular responses to mental arithmetic and handgrip during different conditions of postural change. Psychophysiology, 25, 127–136. Greeson, J. M., Lewis, J. G., Achanzar, K., Zimmerman, E., Young, K. H., & Suarez, E. C. (2009). Stress-induced changes in the expression of monocytic beta2-integrins: The impact of arousal of negative affect and adrenergic responses to the Anger Recall Interview. Brain, Behavior, and Immunity, 23, 251–256. Heponiemi, T., Elovainio, M., Pulkki, L., Puttonen, S., Raitakari, O., & Keltikangas-Ja¨rvinen, L. (2007). Cardiac autonomic reactivity and recovery in predicting carotid atherosclerosis: The cardiovascular risk in young Finns study. Health Psychology, 26, 13–21. Holman, E. A., Silver, R. C., Poulin, M., Andersen, J., Gil-Rivas, V., & McIntosh, D. N. (2008). Terrorism, acute stress, and cardiovascular health: A 3-year national study following the September 11th attacks. Archives of General Psychiatry, 65, 73–80. Hong, R. Y. (2007). Worry and rumination: Differential associations with anxious and depressive symptoms and coping behaviour. Behavior Research and Therapy, 45, 277–290. Imholz, B. P., Wieling, W., van Montfrans, G. A., & Wesseling, K. H. (1998). Fifteen years experience with finger arterial pressure monitoring: Assessment of the technology. Cardiovascular Research, 38, 605–616. Ironson, G., Taylor, C. B., Boltwood, M., Bartzokis, T., Dennis, C., Chesney, M., et al. (1992). Effects of anger on left ventricular ejection fraction in coronary artery disease. American Journal of Cardiology, 70, 281–285. Ishikawa, J., Carroll, D. J., Kuruvilla, S., Schwartz, J. E., & Pickering, T. G. (2008). Changes in home versus clinic blood pressure with antihypertensive treatments: A meta-analysis. Hypertension, 52, 856–864. Jyothinagaram, S. G., Watson, D., & Padfield, P. (1990). Suntech Accutracker ambulatory blood pressure monitorFClinical validation. Journal of Ambulatory Monitoring, 3, 63–67.
Anger rumination: An ambulatory study Kamarck, T. W., Janicki, D. L., Shiffman, S., Polk, D. E., Muldoon, M. F., Liebenauer, L. L., et al. (2002). Psychosocial demands and ambulatory blood pressure: A field assessment approach. Physiology & Behavior, 77, 699–704. Key, B. L., Campbell, T. S., Bacon, S. L., & Gerin, W. (2008). The influence of trait and state rumination on cardiovascular recovery from a negative emotional stressor. Journal of Behavioral Medicine, 31, 237–248. Kubzansky, L. D., Kawachi, I., Spiro, A., Weiss, S. T., Vokonas, P. S., & Sparrow, D. (1997). Is worrying bad for your heart? A prospective study of worry and coronary heart disease in the normative aging study. Circulation, 95, 818–824. Llabre, M. M., Spitzer, S. B., Saab, P. G., Ironson, G. H., & Schneiderman, N. (1991). The reliability and specificity of delta versus residualized change as measures of cardiovascular reactivity to behavioral challenges. Psychophysiology, 28, 701–711. Lynch, J. J., Long, J. M., Thomas, S. A., Malinow, K. L., & Katcher, A. H. (1981). The effects of talking on the blood pressure of hypertensive and normotensive individuals. Psychosomatic Medicine, 43, 25–33. Lyubomirsky, S., & Nolen-Hoeksema, S. (1995). Effects of self-focused rumination on negative thinking and interpersonal problem solving. Journal of Personality and Social Psychology, 69, 176–190. Mauss, I. B., Bunge, S. A., & Gross, J. J. (2007). Automatic emotion regulation. Social & Personality Psychology Compass, 1, 146–167. Moberly, N. J., & Watkins, E. R. (2008). Ruminative self-focus, negative life events, and negative affect. Behavior Research and Therapy, 46, 1034–1039. Ottaviani, C., Shapiro, D., Davydov, D. M., Goldstein, I. B., & Mills, P. J. (2009). The autonomic phenotype of rumination. International Journal of Psychophysiology, 72, 267–275. Ottaviani, C., Shapiro, D., Goldstein, I. B., James, J. E., & Weiss, R. (2006). Hemodynamic profile, compensation deficit, and ambulatory blood pressure. Psychophysiology, 43, 46–56. Ottaviani, C., Shapiro, D., Goldstein, I. B., & Mills, P. J. (2007). Vascular profile, delayed recovery, inflammatory process, and ambulatory blood pressure: Laboratory-to-life generalizability. International Journal of Psychophysiology, 66, 56–65. Pieper, S., Brosschot, J. F., van der Leeden, R., & Thayer, JF. (2007). Cardiac effects of momentary assessed worry episodes and stressful events. Psychosomatic Medicine, 69, 901–909. Porter, L. S., Stone, A. A., & Schwartz, J. E. (1999). Anger expression and ambulatory blood pressure: A comparison of state and trait measures. Psychosomatic Medicine, 61, 454–463. Prkachin, K. M., Mills, D. E., Zwaal, C., & Husted, J. (2001). Comparison of hemodynamic responses to social and nonsocial stress: Evaluation of an anger interview. Psychophysiology, 38, 879–885. Robinson, M. S., & Alloy, L. B. (2003). Negative cognitive styles and stress-reactive rumination interact to predict depression: A prospective study. Cognitive Therapy and Research, 27, 275–292.
461 Rusting, C. L., & Nolen-Hoeksema, S. (1998). Regulating responses to anger: Effects of rumination and distraction on anger mood. Journal of Personality and Social Psychology, 74, 790–803. Saab, P. O., Matthews, K. A., Stoney, C. M., & McDonald, R. H. (1989). Premenopausal and postmenopausal women differ in their cardiovascular and neuroendocrine responses to behavioral stressors. Psychophysiology, 26, 270–280. Schwartz, A. R., Gerin, W., Davidson, K. W., Pickering, T. G., Brosschot, J. F., Thayer, J. F., et al. (2003). Toward a causal model of cardiovascular responses to stress and the development of cardiovascular disease. Psychosomatic Medicine, 65, 22–35. Shapiro, D., Jamner, L. D., Goldstein, I. B., & Delfino, R. J. (2001). Striking a chord: Moods, blood pressure, and heart rate in everyday life. Psychophysiology, 38, 197–204. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press. Steptoe, A., Donal, A. E., O’Donnell, K., Marmot, M., & Deanfield, J. E. (2006). Delayed blood pressure recovery after psychological stress is associated with carotid intima-media thickness. Whitehall Psychobiology Study. Arteriosclerosis, Thrombosis and Vascular Biology, 26, 2547–2551. Steptoe, A., & Marmot, M. (2005). Impaired cardiovascular recovery following stress predicts 3-year increases in blood pressure. Journal of Hypertension, 23, 529–536. Steptoe, A., & Wardle, J. (2005). Cardiovascular stress responsivity, body mass and abdominal obesity. International Journal of Obesity, 29, 1329–1337. Stewart, J. C., Janicki, D. L., & Kamarck, T. W. (2006). Cardiovascular reactivity to and recovery from psychological challenge as predictors of 3-year change in blood pressure. Health Psychology, 25, 111–118. Suarez, E. C., Saab, P. G., Llabre, M., Kuhn, C. M., & Zimmerman, E. (2004). Ethnicity, gender, and age effects on adrenoceptors and physiological responses to emotional stress. Psychophysiology, 41, 450–460. Verduyn, P., Delvaux, E., Van Coillie, H., Tuerlinckx, F., & Van Mechelen, I. (2009). Predicting the duration of emotional experience: Two experience sampling studies. Emotion, 9, 83–91. Ward, A., Lyubomirsky, S., Sousa, L., & Nolen-Hoeksema, S. (2003). Can’t quite commit: Rumination and uncertainty. Personality and Social Psychology Bulletin, 29, 96–107. Wimalaweera, S. W., & Moulds, M. L. (2008). Processing memories of anger-eliciting events: The effect of asking ‘why’ from a distance. Behavior Research and Therapy, 46, 402–409.
(Received January 21, 2010; Accepted June 29, 2010)
Psychophysiology, 48 (2011), 462–469. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01121.x
Selective suppression of the incorrect response implementation in choice behavior assessed by transcranial magnetic stimulation
CHRISTOPHE TANDONNET,a,b MICHAEL I. GARRY,a and JEFFERY J. SUMMERSa a
School of Psychology, Human Movement and Neuroscience Laboratory, University of Tasmania, Hobart, Australia Laboratoire de Psychologie Cognitive, Aix-Marseille Universite´ and CNRS, Marseille, France
b
Abstract Selecting the adequate alternative in choice situations may involve an inhibition process. Here we assessed response implementation during the reaction time of a between-hand choice task with single- or paired-pulse (3 or 15 ms interstimulus intervals [ISIs]) transcranial magnetic stimulation of the motor cortex. The amplitude of the single-pulse motor evoked potential (MEP) initially increased for both hands. At around 130 ms, the single-pulse MEP kept increasing for the responding hand and decreased for the nonresponding hand. The paired-pulse MEP revealed a similar pattern for both ISIs with no effect on short intracortical inhibition and intracortical facilitation measures. The results suggest that the incorrect response implementation was selectively suppressed before execution of the correct response, preventing errors in choice context. The results favor models assuming that decision making involves an inhibition process. Descriptors: Action selection, Human, Motor control, RT, Two-alternative forced choice
type, the pure race models, evidence for each alternative is integrated independently in two accumulators (e.g., Cohen, Dunbar, & McClelland, 1990). In contrast, in another type, evidence for each alternative is not integrated independently. Indeed, for the diffusion models, the difference between the two alternatives is integrated in a single accumulator (e.g., Ratcliff, Van Zandt, & McKoon, 1999). For the competition with inhibition models, the evidence for each alternative is integrated in two accumulators with an inhibition process modulating the integration performed by the accumulators (e.g., Usher & McClelland, 2001). In these models, the inhibition process acting on an accumulator can come directly from the other accumulator (lateral or mutual inhibition) or come from an element upstream of the two accumulators (feed-forward inhibition). According to Bogacz, Brown, Moehlis, Holmes, and Cohen (2006), all models except the pure race models can be formally reduced to a drift diffusion model, meaning that the diffusion models and the competition with inhibition models cannot be distinguished based on their behavioral predictions. The response competition models, although assuming that evidence for both alternatives is initially integrated, differ according to the way the competition takes place thereafter (i.e., by involving or not an inhibition process). In the pure race models, the increase of evidence for the selected alternative (i.e., the first that reaches the threshold) is accompanied by an increase of evidence for the nonselected alternative that become less pronounced with time. In contrast, in the competition with inhibition models, the evidence for the nonselected alternative decreases before the evidence for the
One of the most distinctive features of human motor control is the ability to adapt behavior quickly to environmental demands. In our everyday life, we constantly receive external stimuli to which we do or do not have to react. However, the relation between a stimulus and a response is not one to one: The same stimulation can lead to a different response depending on the context, our current goals, or both. Thus goal-direct responses involve the selection of the appropriate behavior and the organization of the operations in a coherent way so that the stimulus will be efficiently translated into action. These operations can be investigated in the choice reaction time (RT) paradigm. In this situation, the task explicitly requires the selection of one response among response alternatives. Models of forced-choice tasks principally rely on choosing between two alternatives. Such models assume that evidence favoring one alternative is integrated over time with a discrete or continuous accumulation until a threshold is reached that triggers the selected response (e.g., Ratcliff & Smith, 2004). A main difference between the models is the way that evidence for one alternative is integrated relative to the other alternative. In one
This research was supported under the Australian Research Council’s Linkage International Fellowship funding scheme (project number LX0667174 to C.T. and J.J.S). Address correspondence to: C. Tandonnet, Aix-Marseille Universite´ and CNRS, Laboratoire de Psychologie Cognitive, Faculte´ St. Charles, Baˆtiment 9 Case D, 3 place Victor Hugo, 13331 Marseille, France. E-mail:
[email protected] 462
Suppression of incorrect response implementation selected alternative reaches the threshold. An open question is whether such an inhibition process can affect the response implementation. The present study investigates whether the motor implementation of the nonselected response is initiated and then suppressed. To this end, the neural dynamics of motor implementation for both the selected and the nonselected responses was assessed by transcranial magnetic stimulation (TMS) of the motor cortex. TMS can be used to assess excitatory and inhibitory intracortical circuits within the motor cortex (see Reis et al., 2008, for a review). Pulses passing through a wire coil placed over the scalp induce brief electrical currents to the brain. When applied to the primary motor cortex, single-pulse TMS can cause a small twitch in the muscle controlled by the stimulated area. The physiological effect of TMS can be quantified by measuring the motorevoked potential (MEP) with surface electromyographic (EMG) techniques. The MEP amplitude obtained with single-pulse TMS reflects the net effect of excitatory and inhibitory inputs to the corticospinal pathway. During tonic voluntary muscle contraction, TMS can suppress the EMG activity; the duration of the late part of this silent period is thought to reflect cortical inhibitory mechanisms. In the paired-pulse TMS paradigm, two separate pulses are delivered to the motor cortex through the same TMS coil (Kujirai et al., 1993). Short interstimulus intervals (ISIs; 2–5 ms) are used to assess short intracortical inhibition (SICI), the first subthreshold (conditioning) pulse being thought to recruit intracortical inhibitory interneurons that reduce the MEP amplitude produced by a second suprathreshold (test) pulse (Di Lazzaro et al., 1998; Kujirai et al., 1993; Nakamura, Kitagawa, Kawaguchi, & Tsuji, 1997). Paired-pulse TMS with longer ISIs (10–15 ms) may be used to provide a measure of intracortical facilitation (ICF; Kujirai et al., 1993). The inhibition process has been investigated in tasks where participants are instructed to respond to a ‘‘go’’ stimulus by producing a particular movement and to withhold that movement when a ‘‘no-go’’ or ‘‘stop’’ stimulus is presented. In these go/no-go tasks, corticospinal excitability has been shown to increase after the go stimulus and decrease after the no-go stimulus around 150 ms following stimulus presentation (Hoshiyama et al., 1996, 1997). The amplitude of the MEP evoked with paired-pulse TMS (2-ms ISI) is also reduced in a no-go situation (Sohn, Wiltz, & Hallett, 2002) and in the voluntary termination of a planned movement (Coxon, Stinear, & Byblow, 2006), consistent with increased activity in intracortical inhibitory circuits. When a stop signal is presented shortly after the go stimulus on some trials (stop tasks), corticospinal excitability has been shown to decrease around 180 ms after the stop signal (van den Wildenberg et al., 2010). Moreover, a lengthening of silent period duration evoked by single-pulse TMS was found following the instruction to stop, suggesting an increased recruitment of intracortical inhibitory circuits within the motor cortex (van den Wildenberg et al., 2010). Thus there is evidence that different intracortical inhibitory circuits play a role in tasks involving explicit stopping of ongoing behavior. In choice RT tasks, although there is no explicit instruction to stop a movement, previous studies suggest involvement of an inhibition process (see Burle, Vidal, Tandonnet, & Hasbroucq, 2004, for a review). The H-reflex was found to increase for the responding hand and to decrease for the nonresponding hand before the movement initiation in a between-hand choice reaction time task (Hasbroucq, Akamatsu, Burle, Bonnet, & Possamaı¨ , 2000). This pattern has also been observed in EMG
463 activity, indicating that suppression can occur on the motoneuronal pool (Tandonnet, Burle, Vidal, & Hasbroucq, 2005). This suppression observed at the spinal level is compatible with inhibitory inputs to the corticospinal pathway. Event-related potential (ERP) studies using surface Laplacian estimation to enhance the spatial resolution of the recorded activity over the sensorimotor cortices have provided evidence for cortical inhibition. These studies revealed that the negativity developing over the cortex contralateral to the response is accompanied by a positivity developing over the ipsilateral cortex (Tandonnet, Burle, Vidal, & Hasbroucq, 2003; Taniguchi, Burle, Vidal, & Bonnet, 2001; Vidal, Grapperon, Bonnet, & Hasbroucq, 2003), a pattern focused over the primary sensorimotor cortex (Tandonnet, Burle, Hasbroucq, & Vidal, 2005). The negativity preceding the voluntary movement likely reflects the implementation of the correct response, and the activity with an opposite polarity developing in the same time range and in the same location area may reflect a suppression of the incorrect response implementation (Burle et al., 2004; see Vidal et al., 2003, for a review). In one study, the silent period evoked by single-pulse TMS was found to increase for the nonresponding hand following stimulus presentation, reflecting involvement of cortical inhibitory circuits within the motor cortex (Burle, Bonnet, Vidal, Possamaı¨ , & Hasbroucq, 2002). These previous studies suggest that an inhibition process may play a key role in choice RT tasks by modulating motor implementation of the responses and preventing the incorrect response from occurring. If the motor implementation of the nonselected response is suppressed during between-hand choice tasks, it should be visible in the variations of excitability of the final motor pathway. However, previous studies using single-pulse TMS of the motor cortex during between-hand choice RT tasks did not support this notion. The MEP elicited by single-pulse TMS was found to increase in either hand after stimulus presentation (Burle et al., 2002; McMillan, Ivry, & Byblow, 2006). The MEP was also found to increase more for the responding hand than for the nonresponding hand at some point during the reaction time period (Duque & Ivry, 2009; Koch et al., 2006; Leocani, Cohen, Wassermann, Ikoma, & Hallett, 2000). These patterns are compatible with the view that, after an initial phase in which both possible responses are implemented, the correct response receives further activation than the incorrect one. Thus previous TMS studies found no decrease of the MEP for the incorrect response, providing no clear evidence for a suppression of the incorrect motor implementation. The objective of the present study was to address whether the motor implementation of the nonselected response can be suppressed during a between-hand choice task. To this end, we assessed motor implementation of both the responding hand and the nonresponding hand with TMS. We used MEP amplitude obtained with single-pulse TMS of the motor cortex as a global measure of the ongoing response implementation representing the net effect of excitatory and inhibitory synaptic inputs on the final motor pathway. We used MEP amplitude obtained with paired-pulse TMS with 3 ms and 15 ms ISI (Kujirai et al., 1993) as a measure of SICI and ICF. To allow tracking of the dynamics, TMS was delivered at several possible intervals between stimulus presentation and response execution. We hypothesized that (i) if both responses are implemented in a first phase, the single-pulse MEP amplitude would increase for both hands, (ii) if the motor implementation of the incorrect response is suppressed in a second phase, the single-pulse MEP amplitude would decrease for
464
C. Tandonnet et al.
the nonresponding hand prior to movement initiation, and (iii) if such suppression involves an increase of SICI or a decrease of ICF, the decrease in amplitude of the single-pulse MEP for the nonresponding hand would be more pronounced for the conditioned MEPs (3 ms or 15 ms ISI).
Methods Participants Twelve participants (8 women and 4 men, aged 18–36 years, mean 5 25, SD 5 7) were healthy volunteers with self-reported right-handedness and normal or corrected-to-normal vision. Informed written consent was obtained according to the Declaration of Helsinki, and the study was approved by the University of Tasmania Human Research Ethics Committee. Behavioral Setup Participants were seated in a comfortable chair in a darkened room with supports for forearms and hands. The hands were held in a neutral position with the thumbs resting on the top of two vertical cylinders with mounted force sensors fixed on the table approximately 30 cm in front of participants. Participants faced a black panel, 80 cm distant at eye level. A horizontal row of three light-emitting diodes (LEDs) were positioned at the center of the panel; the two outer LEDs were 4 cm apart. The central LED (green) served as a fixation point and the two outer LEDs (yellow) were the response signals. Trial Events Each trial started with illumination of the central fixation LED. After 500 ms, the fixation was switched off and the imperative stimulus consisting of one of the two outer yellow LEDs was switched on for 500 ms. Participants were instructed to execute as quickly as possible the isometric thumb press that was spatially compatible with the stimulus location (e.g., left press for a stimulus on the left of fixation). After termination of the stimulus, a feedback signal was presented. If the first press exceeding 4 N was on the correct side during the 500 ms following stimulus presentation, the feedback was a short auditory ‘‘click’’ (1000 Hz, 50 ms in duration); otherwise it was a longer ‘‘buzz’’ (400 Hz, 200 ms in duration). The intertrial interval was 800 ms. Design Participants performed a training session without TMS and then an experimental session with TMS. Each block comprised 56 trials in which each imperative stimulus (left/right) occurred 28 times in a random sequence. In the training session, EMG was recorded and blocks of trials were performed until the error rate was below 0.05 and the coefficient of variation of response latencies was below 0.15 during two consecutive blocks. In the experimental session of 12 blocks, TMS was delivered at the stimulus presentation or at four other possible times individually determined from the EMG onset latencies of the responses performed with the left hand (i.e., when the right hand was not responding) in the last two blocks of the training session. One TMS time was at the imperative stimulus onset (t1) and the four other times were around the first decile of the EMG onset distribution: at one third before (t2), at one sixth before (t3), at the first decile (t4), and at one sixth after (t5; see Figure 1 for the TMS times relative to the EMG onset distribution of the no-
Figure 1. Distribution of EMG onset latencies (left and right hands) for the no-TMS trials as a function of poststimulus time (in milliseconds). Each point represents a decile of the EMG onset distribution. The arrows (bottom) indicate the four times of TMS delivery; t1 time corresponds to imperative stimulus onset.
TMS trials in the experimental session). The last four TMS times were, respectively, at 105 ms (SD 5 19), 131 ms (SD 5 24), 157 ms (SD 5 29), and 183 ms (SD 5 34) after stimulus onset. Within a block, 8 no-TMS trials and 48 TMS trials (i.e., 16 trials for each TMS condition: single-pulse, paired-pulse with 3-ms ISI, pairedpulse with 15-ms ISI) occurred in a random sequence. Time of TMS delivery occurred in random sequence with two repetitions for the first two TMS times and four repetitions for the last three TMS times for each block. Transcranial Magnetic Stimulation TMS was delivered to the left motor cortex using a 90-mm circular coil connected to two Magstim 200 magnetic stimulators via a BiStim module (Magstim, Whitland, Dyfed, UK). The two Magstim stimulators were configured to deliver paired pulse stimulation with an interstimulus interval of 3 ms or 15 ms to assess, respectively, SICI and ICF (Kujirai et al., 1993). The coil was oriented tangentially to the scalp to deliver induced current in a postero-anterior direction in the left motor cortex. The coil was initially placed at the vertex and then carefully positioned until the lowest threshold spot for activating the right flexor pollicis brevis (FPB) muscle was reached. The closest standard location corresponding to the mean stimulation site was the C3 site in the 10/20 system (on the central midline 6 cm to the left of the vertex). This spot was marked on the scalp with a felt-tip pen to allow consistent coil placement during the experiment. To adjust the TMS intensity, participants were asked to maintain the position of their thumbs on the response device and relax their muscles during the delivering of TMS. The minimal TMS intensity needed to evoke an MEP larger than 50 mV peak-to-peak in at least 5 of 10 consecutive trials was used as resting motor threshold. The TMS intensity for single pulse was set at the resting motor threshold. For the paired pulse, the TMS intensity of the test stimulus was set at 120% of resting motor threshold, and the TMS intensity of the conditioning stimulus (ISI of 3 ms) was initially set at 70% of resting motor threshold and then adjusted downward until the MEP was approximately half reduced though still present on every trial. In accordance with the recommendation of Garry and Thomson (2009), test stimulus
Suppression of incorrect response implementation intensity was held constant throughout the experiment. Mean resting motor threshold was 47% (SD 5 9) of the maximal stimulator output. Mean test stimulus intensity was 56% (SD 5 11) of the maximal stimulator output. Mean conditioning stimulus intensity was 30% (SD 5 6) of the maximal stimulator output, corresponding to 63% (SD 5 8) of resting motor threshold. EMG and Force Recordings Surface EMG activity was recorded from paired Ag/AgCl electrodes fixed over the FPB muscle in a belly tendon montage. The EMG signal was amplified with a gain of 1000, filtered using a 50- Hz notch filter, 20-Hz high-pass and 500-Hz low-pass (Butterworth, 12 dB/octave) filters, and digitized online at a sampling rate of 2 kHz (16-bit resolution; CED 1902, Cambridge Electronic Design, UK). The force signal was filtered using a 50-Hz notch filter and a 100-Hz low-pass (Butterworth, 12 dB/ octave) filter, and digitized online at 2 kHz (16-bit resolution; CED 1902, Cambridge Electronic Design, UK). Signal Processing The response latencies and the EMG onset latencies were measured relative to the onset of the imperative signal for the noTMS trials. The mechanical response onset used for the reaction time corresponded to a press exceeding 4 N on one of the force sensors. The EMG onset latencies were scored by the experimenter (Burle et al., 2002; Davranche et al., 2007; Van Boxtel, Geraats, Van den Berg-Lenssen, & Brunia, 1993). The trials presenting side errors (1.2%, SD 5 1.6) or latencies exceeding 500 ms (4.3%, SD 5 10.9) were discarded. For EMG onset latencies, trials with EMG activity before the voluntary EMG burst were also discarded (15.3%, SD 5 8.7). The peak-to-peak amplitude of the raw MEPs in the right FPB muscle was measured within a 50-ms analysis window from 15 ms after TMS onset for each trial whether the right hand was responding or not. Trials with EMG activity before the TMS delivery were discarded. The mean rejection percentage per TMS time was 6.3% (SD 5 1.0), 14.8% (SD 5 0.8), 26.2% (SD 5 3.2), 41.9% (SD 5 4.3), and 61.5% (SD 5 4.7), respectively. As the fifth TMS time led to no data for some participants, this TMS time was discarded for subsequent analyses. Because of large intersubject variability, raw MEP amplitudes were converted into z scores before being submitted to statistical analyses (Burle et al., 2002; Davranche et al., 2007; van den Wildenberg et al., 2010). The individual computation of mean and standard deviation included all experimental conditions, that is, both TMS type and time of TMS delivery. For paired-pulse TMS, raw MEP amplitudes were also expressed as ratios (MEP amplitude for paired-pulse TMS divided by MEP amplitude of single-pulse TMS) for both SICI and ICF measures, as this computation was commonly used to assess the size of the conditioned MEP relative to a control size (e.g., Kujirai et al., 1993). The amount of background EMG activity preceding TMS was analyzed in order to assess whether this activity could be related to any MEP modulations. The root mean square (RMS) of the EMG activity was computed within a 100-ms analysis window prior to TMS delivery for the same trials as for the MEP analysis. Raw RMS values were also converted into z scores before being submitted to statistical analyses. Statistical Analysis The RT and the EMG onset latencies were submitted to paired Student t tests. The MEP amplitudes (z scores) were submitted to
465 univariate repeated-measures analysis of variance (ANOVA) involving three within-subject factors: TMS type (single, paired-3 ms, paired-15 ms), involvement of the right hand in the response (responding, nonresponding), and TMS time (four times from the imperative stimulus). The MEP ratios (SICI, ICF) were submitted to univariate repeated-measures ANOVAs involving two within-subject factors: involvement of the right hand in the response (responding, nonresponding) and TMS time (four times from the imperative stimulus). The RMS values (z scores) of background EMG activity were submitted to an ANOVA with the same design as the one used for the analysis of MEP amplitudes. Planned comparisons were used to test the effect of time separately for the responding and the nonresponding conditions in order to assess the MEP dynamics for both the selected and the nonselected responses. Huynh-Feldt correction was used for univariate repeated-measures ANOVA tests involving more than one degree of freedom, in which case the uncorrected degrees of freedom, the corrected p value, and the e value were reported. Newman–Keuls was used for post hoc tests.
Results Response Latency There was no difference between left and right hands on either response latencies, t(11) 5 1.17, p 5 .27, or EMG onset latencies, t(11)o1. The mean reaction time for the no-TMS trials was 286 ms (SD 5 57). The mean EMG onset latency was 189 ms (SD 5 29); the deciles of these distributions are presented with respect to the times of TMS delivery (Figure 1). MEP Amplitude The mean amplitude of the raw MEPs for single-pulse TMS and paired-pulse TMS with 3-ms and 15-ms ISIs was 0.81 mV, 0.49 mV, and 0.91 mV, respectively. The mean MEP amplitudes (z scores) are presented in Figure 2. There was an effect of time, F(3,33) 5 27.07, po.001, e 5 .85, showing that corticospinal excitability tends to increase during the reaction time period. The MEP was larger for the responding condition than for the nonresponding condition, F(1,11) 5 16.38, po.01. This effect confirmed that the execution of the selected movement is preceded by an increase of corticospinal excitability for the response contralateral to the stimulated cortex. There was an effect of TMS type, F(2,22) 5 29.59, po.001, e 5 .76, indicating that paired-pulse TMS with a 3-ms ISI elicited smaller MEP amplitude than the MEP obtained with single-pulse TMS. The two-way interaction between time and responding hand, F(3,33) 5 23.46, po.001, e 5 .74, revealed that the effect of time was different when the right hand was responding than when it was the nonresponding hand. The effect of time was linear for the responding condition (po.01), and both the linear and the quadratic components were significant for the nonresponding condition (pso.05). Post hoc tests revealed that the MEP amplitudes for the nonresponding condition significantly increased until around 130 ms (TMS times t1 vs. t2, p 5 .03; t2 vs. t3, po.01) and then decreased before movement initiation (i.e., between t3 and t4, p 5 .03). Importantly, there was no interaction between TMS type and the two other factors, time, F(6,66)o1, e 5 .92, or hand, F(2,22)o1, e 5 1.00, and the three-way interaction was not significant, F(6,66)o1, e 5 .71, revealing that the MEP pattern obtained with paired-pulse TMS (3-ms and 15-ms ISIs) was similar to that obtained with single-pulse TMS.
466
C. Tandonnet et al.
Figure 2. MEP amplitude (z scores) as a function of poststimulus time (in milliseconds). Filled dots correspond to the responding condition and empty dots to the nonresponding condition. Left: single-pulse TMS. Middle: paired-pulse TMS with 3-ms interstimulus interval (ISI). Right: paired-pulse TMS with 15-ms ISI.
MEP Ratio The paired-pulse MEP amplitudes expressed relative to the single-pulse MEP amplitudes revealed no significant modulation (Figure 3). The ratio for the paired-pulse TMS with a 3-ms ISI showed no difference between responding and nonresponding conditions, F(1,11)o1, no effect of time, F(3,33) 5 1.11, p 5 .36, e 5 1.00, and no interaction, F(3,33) 5 1.69, p 5 .19, e 5 1.00. Similar statistical results were obtained for the 15-ms ISI, F(1,11)o1; F(3,33) 5 1.36, p 5 .28, e 5 .76; F(3,33)o1, e 5 1.00, respectively.
Background EMG Activity The RMS of the EMG activity in the pre-TMS interval showed no significant modulation. There was no difference between responding and nonresponding conditions, F(1,11)o1, no effect of time, F(3,33) 5 2.09, p 5 .12, e 5 1.00, and no effect of TMS type, F(2,22) 5 2.32, p 5 .12, e 5 1.00. There were no interactions: responding condition and TMS type, F(2,22)o1, e 5 1.00, responding condition and time, F(3,33)o1, e 5 1.00, TMS type
Figure 3. MEP ratio (conditioned MEPs expressed relative to test MEPs) as a function of poststimulus time (in milliseconds). Filled dots correspond to the responding condition and empty dots to the nonresponding condition. Left: paired-pulse TMS with 3-ms interstimulus interval (ISI). Right: paired-pulse TMS with 15-ms ISI.
and time, F(6,66) 5 1.06, p 5 .40, e 5 1.00, and no three-way interaction, F(6,66)o1, e 5 .76. Discussion The variations in amplitude of the MEP obtained with singlepulse TMS showed that corticospinal excitability increased first nonspecifically and then either kept on increasing when the right hand was about to respond or decreased when the right hand was not responding. The early nonspecific MEP increase is in line with previous studies (Burle et al., 2002; McMillan et al., 2006). These results are compatible with the notion that implementation of both responses are initiated in between-hand choice reaction time tasks. The dissociation between the responding and nonresponding conditions observed later in the reaction time period is also consistent with previous TMS studies (Duque & Ivry, 2009; Koch et al., 2006; Leocani et al., 2000). The new result of the present study is that the MEP was found to decrease for the nonresponding condition around 130 ms after stimulus presentation. The MEP decrease likely reflects a selective mechanism (i.e., response-specific) that keeps increasing the activation of the selected response and decreases the activation of the nonselected response. This mechanism can be viewed as an ‘‘active’’ suppression of the ongoing implementation of the incorrect response in two-choice reaction time tasks or may reflect a ‘‘passive’’ suppression following the initial activation (i.e., like a return to baseline). These two possible interpretations of the suppression mechanism are discussed in the following. We first present how the MEP suppression observed with single-pulse TMS extends previous studies, then discuss the possible neural mechanisms involved in the suppression mechanism based on the paired-pulse MEP modulations and on previous results, and finally assess to what extent the present results can challenge the current models of choice tasks. Selective MEP Suppression during Choice Reaction Time The MEP dissociation between the two hands has been observed in previous studies using choice tasks, but, to our knowledge, no significant MEP decrease for the nonresponding hand was reported during RT (Duque & Ivry, 2009; Koch et al., 2006; Leocani et al., 2000). The bilateral MEP increase in the early RT has been observed in one previous study (Burle et al., 2002). The
Suppression of incorrect response implementation apparent discrepancy in the literature may be resolved by the present study, where both nonselective and selective contributions were observed. These MEP modulations suggest that two mechanisms act concurrently on corticospinal excitability. If the nonselective activation is superimposed on the suppression component of the selective activation/suppression mechanisms, the nonselective component may potentially mask the selective one, depending of the relative weight and timing of the two components. This potential confound may explain why the MEP for the nonresponding hand was not found to decrease during RT in the above mentioned previous studies. Note also that even in the study showing a clear bilateral increase during RT, there was a nonsignificant decrease for the nonresponding hand for the last stimulation (Burle et al., 2002, p. 213, Figure 6). Thus one cannot exclude that a lack of statistical power in that previous study led to the apparent discrepancy with the present results. A more critical factor may be the specific times of TMS delivery. Indeed, the density of stimulation times preceding the EMG onset was higher in the present study than in the Burle et al. (2002) study. As for the nonresponding hand, the MEP tends to first increase quite sharply (between t2 and t3) and then decrease with the same sharpness (between t3 and t4, see Figure 2), it seems likely that the higher density of TMS times just before response execution in the present study allowed observation of the dynamics of such activation/suppression modulations. Importantly, such rapid changes in the dynamics favor the interpretation of these modulations as an active suppression mechanism. Possible Neural Mechanisms Involved in the Selective Suppression A suppression of the incorrect motor output in choice tasks is compatible with a previous study showing a reduction in EMG activity (Tandonnet, Burle, Vidal, et al., 2005). The brief reduction of tonic EMG activity in the hand involved in the incorrect response suggests that the suppression mechanism can affect the activity of the motoneuronal pool. A reduction of the H-reflex has also been found for the incorrect response in a between-hand choice task, perhaps reflecting modulation of presynaptic inhibition on the motoneurons’ somesthetic afferents (Hasbroucq et al., 2000). However, such modulations may involve a population of motoneurons different from that involved in the MEP (Petersen, Pyndt, & Nielsen, 2003). Hence the H-reflex modulations may not necessarily reflect the same mechanism responsible for the MEP variations observed in the present study. In the present study, the mean MEP amplitude in the nonresponding hand was found to decrease before the mean EMG onset derived from no-TMS trials, suggesting that the suppression of the incorrect response implementation can be initiated before the execution of the correct response starts. To test the involvement of different cortical circuits within the motor cortex, we used paired-pulse TMS. The paired-pulse MEP with 3-ms and 15-ms ISIs revealed a pattern similar to that exhibited by single-pulse MEP, with both SICI and ICF measures showing no significant modulation. Note that test TMS intensity was maintained constant at 120% RMT throughout the experiment to assess SICI modulations in accordance with the recommendation of Garry and Thomson (2009). Thus, in the present experiment, the paired-pulse TMS assessed the dynamics of the same cortical circuits for all stimulation times, likely including those generating the late I-waves (I2, I3; Di Lazzaro et al., 1998; Garry & Thomson, 2009). The lack of difference in SICI for the responding and nonresponding conditions suggests that the observed corticospinal modulations did not involve the
467 intracortical circuits assessed with paired-pulse TMS (3- and 15-ms ISIs). The present results, therefore, suggest that decreased SICI or increased ICF for the responding hand is not a prerequisite for the implementation of the selected response. Likewise, it also appears that the suppression mechanism may not involve increased SICI or decreased ICF for the nonresponding hand. However, previous ERP studies have provided evidence for cortical inhibitory mechanisms operating during choice RT tasks. By using surface Laplacian estimation to enhance spatial resolution, these studies have revealed an activation/suppression pattern over the sensorimotor cortices prior to movement initiation (Tandonnet, Burle, Hasbroucq, et al., 2005; Tandonnet et al., 2003; Taniguchi et al., 2001; Vidal et al., 2003). Evidence for suppression of the incorrect response has also been shown in silent period duration, suggesting the involvement of intracortical inhibitory circuits (Burle et al., 2002). As excitatory and inhibitory inputs to the corticospinal pathway influence MEP amplitude, we can speculate that the MEP pattern observed in the present study may reflect cortical modulations like those observed through variation of the silent period duration. This interpretation is in line with the explanation offered to account for the dissociation between response activation and stopping processes observed in a stop task (van den Wildenberg et al., 2010). That is, it was proposed that the recruitment of intracortical inhibitory circuits as revealed by silent period modulations was responsible for the subsequent decline in corticospinal excitability in stop tasks (van den Wildenberg et al., 2010). Cortical inhibition is thought to be influenced by gamma-aminobutyric acid (GABA)-ergic inhibition with modulation of the cortical silent period duration being mediated by GABAB receptors (Siebner, Dressnandt, Auer, & Conrad, 1998; Werhahn, Kunesch, Noachtar, Benecke, & Classen, 1999) and SICI by GABAA receptors (Ziemann, Lo¨nnecker, Steinhoff, & Paulus, 1996). ICF may be influenced by GABA-ergic inhibition through GABAA receptors and glutamate-ergic facilitation through N-methyl-daspartate (NMDA) receptors (see Ziemann, 2004, for a review). The present results raise the possibility that the corticospinal suppression observed involves different cortical circuits than SICI and ICF circuits. Rather the suppression may specifically involve other intracortical inhibitory circuits like those mediated by GABAB receptors. The corticospinal reduction may also be linked to changes in connectivity between the dorsal premotor cortex and the motor cortex, as these changes can be independent of intracortical circuits involved in SICI and ICF within the motor cortex (Koch et al., 2006). This leads to the speculation that the inhibition process involved in choice reaction time tasks may be more specific than those involved in tasks explicitly involving the stopping of an ongoing behavior (go/no-go and stop tasks). This possibility suggests that the physiological mechanisms involved in response inhibition critically depend on the specific context of the task. Implication of the Selective Suppression for the Models of Choice Tasks In what follows, we assess how the present data can challenge the different models of choice tasks. We consider more particularly how the dynamic of the motor implementation of the responses can constrain both the way the evidence in favor of one alternative is accumulated and the link between this decision process and the motor implementation of the responses. As stated in the introduction section, we assume that the MEP modulations
468
C. Tandonnet et al.
reflect the net effect of excitatory and inhibitory inputs to the corticospinal pathway and thus can be used as an index of the motor implementation of the responses. The nonselective increase of MEP in the early RT is assumed to reflect an initial bilateral activation of the responses. For models with two accumulators, the minimalist assumption that the motor implementation of the responses depends only on the accumulation of evidence for each alternative predicts this MEP pattern. Indeed, an increase of evidence for each alternative would lead to a motor implementation of both responses. In contrast, models with a single accumulator assume by definition that only evidence for the difference between the two alternatives can increase, which cannot lead to motor implementation of both responses without further assumption. An assumption compatible with the present data would be that the motor implementation of the responses depends also on a response activation process that is independent of the decision process, possibly triggered by the mere occurrence of the imperative stimulus. This assumption is in line with dual-route architectures (De Jong, Liang, & Lauber, 1994; Kornblum, Hasbroucq, & Osman, 1990). In this framework, the stimulus conveys both the attribute relevant for selecting the correct alternative and other attributes irrelevant for the task. The relevant attribute would activate the correct response via a controlled route, whereas the irrelevant attributes may activate the incorrect response via an automatic route. Thus the nonselective increase of MEP observed in the present study is compatible with two-accumulator models and may also be compatible with single-accumulator models with the additional assumption of a response activation process independent of the decision process. The specific MEP decrease obtained only for the nonresponding hand later in the RT period is assumed to reflect a selective suppression of the activation of the nonselected response. Note that this reduction may a priori be viewed either as an ‘‘active’’ suppression of the ongoing response implementation or as a ‘‘passive’’ suppression following the initial activation. Models with two independent accumulators assume an increase of evidence for the nonselected alternative that would lead to an increase of activation of the nonresponding hand. In contrast, in models with two accumulators assuming an inhibition process, evidence for the nonselected alternative can decrease, leading to a suppression of the motor implementation of the nonselected response. For models with a single accumulator, the predictions depend on the link between the decision process and the motor implementation of the responses. If an increase of evidence for
one alternative leads to an activation of the implementation of the selected response, it cannot lead to any decrease in the activation of the nonselected response. However, the decision process may also lead to a suppression of the initial activation of the nonselected response through an inhibition process. The specific MEP decrease for the nonresponding hand revealed in the present study nicely fits the predictions of the models assuming an inhibition process but is difficult to reconcile with models assuming two independent accumulators and models assuming a single accumulator without an inhibition process. Thus the dynamics of the response implementation as reflected by MEP modulations provide some constraints on models of choice tasks concerning the way the evidence is integrated and on the link between this decision process and the motor implementation of the responses. The present data are compatible with decision-making models assuming two accumulators with an inhibition process but not with models assuming two independent accumulators (pure race models). These data are also compatible with single accumulator models (diffusion models) with the following assumptions: (i) a nonselective activation process independent of the decision process and (ii) an inhibition process of the implementation of the nonselected response. The inhibition process assumed by the different models takes place at different levels. For the models assuming two accumulators with inhibition, the inhibition process acts on the accumulators (decision process), whereas for the single accumulator models the inhibition process acts on the response implementation. Further research is needed to address to what extent the inhibition process can be specific to decision making or response implementation processes. Importantly, the fact that the present data are compatible only with the predictions of models assuming an inhibition process favor an interpretation of the MEP decrease as an active suppression mechanism.
Conclusion The present results showed that MEP amplitude first increases during RT for both the responding and the nonresponding hand and then decreases in the nonresponding hand. These results suggest that after an initial facilitation of both responses, ongoing implementation of the incorrect response was selectively suppressed before initiation of the correct response execution. This selective suppression may secure the correct response implementation by preventing errors in choice behavior. These results favor models of choice tasks assuming an inhibition process.
REFERENCES Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks. Psychological Review, 113, 700–765. Burle, B., Bonnet, M., Vidal, F., Possamaı¨ , C., & Hasbroucq, T. (2002). A transcranial magnetic stimulation study of information processing in the motor cortex: Relationship between the silent period and the reaction time delay. Psychophysiology, 39, 207–217. Burle, B., Vidal, F., Tandonnet, C., & Hasbroucq, T. (2004). Physiological evidence for response inhibition in choice reaction time tasks. Brain and Cognition, 56, 153–164. Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97, 332–361.
Coxon, J., Stinear, C., & Byblow, W. (2006). Intracortical inhibition during volitional inhibition of prepared action. Journal of Neurophysiology, 95, 3371–3383. Davranche, K., Tandonnet, C., Burle, B., Meynier, C., Vidal, F., & Hasbroucq, T. (2007). The dual nature of time preparation: Neural activation and suppression revealed by transcranial magnetic stimulation of the motor cortex. European Journal of Neuroscience, 25, 3766–3774. De Jong, R., Liang, C. C., & Lauber, E. (1994). Conditional and unconditional automaticity: A dual-process model of effects of spatial stimulus-response correspondence. Journal of Experimental Psychology: Human Perception and Performance, 20, 731–750. Di Lazzaro, V., Restuccia, D., Oliviero, A., Profice, P., Ferrara, L., Insola, A., Mazzone, P., et al. (1998). Magnetic transcranial stimulation at intensities below active motor threshold activates intracortical inhibitory circuits. Experimental Brain Research, 119, 265–268.
Suppression of incorrect response implementation Duque, J., & Ivry, R. B. (2009). Role of corticospinal suppression during motor preparation. Cerebral Cortex, 19, 2013–2024. Garry, M. I., & Thomson, R. H. S. (2009). The effect of test TMS intensity on short-interval intracortical inhibition in different excitability states. Experimental Brain Research, 193, 267–274. Hasbroucq, T., Akamatsu, M., Burle, B., Bonnet, M., & Possamaı¨ , C. A. (2000). Changes in spinal excitability during choice reaction time: The H reflex as a probe of information transmission. Psychophysiology, 37, 385–393. Hoshiyama, M., Kakigi, R., Koyama, S., Takeshima, Y., Watanabe, S., & Shimojo, M. (1997). Temporal changes of pyramidal tract activities after decision of movement: A study using transcranial magnetic stimulation of the motor cortex in humans. Electroencephalography and Clinical Neurophysiology, 105, 255–261. Hoshiyama, M., Koyama, S., Kitamura, Y., Shimojo, M., Watanabe, S., & Kakigi, R. (1996). Effects of judgement process on motor evoked potentials in go/no-go hand movement task. Neuroscience Research, 24, 427–430. Koch, G., Franca, M., Del Olmo, M. F., Cheeran, B., Milton, R., Alvarez Sauco, M., & Rothwell, J. C. (2006). Time course of functional connectivity between dorsal premotor and contralateral motor cortex during movement selection. Journal of Neuroscience, 26, 7452–7459. Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: Cognitive basis for stimulus-response compatibilityFA model and taxonomy. Psychological Review, 97, 253–270. Kujirai, T., Caramia, M. D., Rothwell, J. C., Day, B. L., Thompson, P. D., Ferbert, A., Wroe, S., et al. (1993). Corticocortical inhibition in human motor cortex. Journal of Physiology, 471, 501–519. Leocani, L., Cohen, L. G., Wassermann, E. M., Ikoma, K., & Hallett, M. (2000). Human corticospinal excitability evaluated with transcranial magnetic stimulation during different reaction time paradigms. Brain, 123, 1161–1173. McMillan, S., Ivry, R. B., & Byblow, W. D. (2006). Corticomotor excitability during a choice-hand reaction time task. Experimental Brain Research, 172, 230–245. Nakamura, H., Kitagawa, H., Kawaguchi, Y., & Tsuji, H. (1997). Intracortical facilitation and inhibition after transcranial magnetic stimulation in conscious humans. Journal of Physiology, 498, 817–823. Petersen, N. T., Pyndt, H. S., & Nielsen, J. B. (2003). Investigating human motor control by transcranial magnetic stimulation. Experimental Brain Research, 152, 1–16. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333–367. Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and diffusion models of reaction time. Psychological Review, 106, 261–300. Reis, J., Swayne, O. B., Vandermeeren, Y., Camus, M., Dimyan, M. A., Harris-Love, M., Perez, M. A., et al. (2008). Contribution of
469 transcranial magnetic stimulation to the understanding of cortical mechanisms involved in motor control. Journal of Physiology, 586, 325–351. Siebner, H. R., Dressnandt, J., Auer, C., & Conrad, B. (1998). Continuous intrathecal baclofen infusions induced a marked increase of the transcranially evoked silent period in a patient with generalized dystonia. Muscle & Nerve, 21, 1209–1212. Sohn, Y. H., Wiltz, K., & Hallett, M. (2002). Effect of volitional inhibition on cortical inhibitory mechanisms. Journal of Neurophysiology, 88, 333–338. Tandonnet, C., Burle, B., Hasbroucq, T., & Vidal, F. (2005). Spatial enhancement of EEG traces by surface Laplacian estimation: Comparison between local and global methods. Clinical Neurophysiology, 116, 18–24. Tandonnet, C., Burle, B., Vidal, F., & Hasbroucq, T. (2003). The influence of time preparation on motor processes assessed by surface Laplacian estimation. Clinical Neurophysiology, 114, 2376–2384. Tandonnet, C., Burle, B., Vidal, F., & Hasbroucq, T. (2005). An electromyographic examination of response execution and inhibition in between-hand choice reaction time. Journal of Psychophysiology, 19, 330–336. Taniguchi, Y., Burle, B., Vidal, F., & Bonnet, M. (2001). Deficit in motor cortical activity for simultaneous bimanual responses. Experimental Brain Research, 137, 259–268. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550–592. Van Boxtel, G. J., Geraats, L. H., Van den Berg-Lenssen, M. M., & Brunia, C. H. (1993). Detection of EMG onset in ERP research. Psychophysiology, 30, 405–412. Vidal, F., Grapperon, J., Bonnet, M., & Hasbroucq, T. (2003). The nature of unilateral motor commands in between-hand choice tasks as revealed by surface Laplacian estimation. Psychophysiology, 40, 796–805. Werhahn, K. J., Kunesch, E., Noachtar, S., Benecke, R., & Classen, J. (1999). Differential effects on motorcortical inhibition induced by blockade of GABA uptake in humans. Journal of Physiology, 517, 591–597. van den Wildenberg, W. P. M., Burle, B., Vidal, F., van der Molen, M. W., Ridderinkhof, K., & Hasbroucq, T. (2010). Mechanisms and dynamics of cortical motor inhibition in the stop-signal paradigm: A TMS study. Journal of Cognitive Neuroscience, 22, 225–239. Ziemann, U. (2004). TMS and drugs. Clinical Neurophysiology, 115, 1717–1729. Ziemann, U., Lo¨nnecker, S., Steinhoff, B. J., & Paulus, W. (1996). The effect of lorazepam on the motor cortical excitability in man. Experimental Brain Research, 109, 127–135. (Received January 19, 2010; Accepted June 8, 2010)
Psychophysiology, 48 (2011), 470–478. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01119.x
Distinct neural generators of sensory gating in schizophrenia
TERRANCE J. WILLIAMS,a KEITH H. NUECHTERLEIN,b KENNETH L. SUBOTNIK,b and CINDY M. YEEa,b a
Department of Psychology, University of California, Los Angeles, Los Angeles, California, USA Department of Psychiatry, University of California, Los Angeles, Los Angeles, California, USA
b
Abstract Although malfunctioning of inhibitory processes is proposed as a pathophysiological mechanism in schizophrenia and has been studied extensively with the P50 gating paradigm, the brain regions involved in generating and suppressing the P50 remain unclear. The current investigation used EEG source analysis and the standard S1-S2 paradigm to clarify the neural structures associated with P50 gating in 16 schizophrenia patients and 14 healthy subjects. Based on prior research, the superior temporal gyrus, hippocampus, dorsolateral prefrontal cortex, thalamus, and their dipole moments were evaluated. In modeling the P50, a neural network involving all four brain regions provided the best goodness-of-fit across both groups. In healthy subjects, the P50 ratio score correlated positively with the hippocampal dipole moment ratio, whereas a significant association with the DLPFC dipole moment ratio was observed in schizophrenia patients. In each instance, the neural structure was found to account for unique variance in explaining the P50 ratio, along with some suggestion of DLPFC involvement in healthy subjects. Descriptors: Schizophrenia, Sensory gating, Inhibitory deficits, EEG source analysis, Hippocampus, Dorsolateral prefrontal cortex
The predominant method for demonstrating impaired sensory gating in humans is the paired-stimulus P50 paradigm, during which a brief, initial stimulus (‘‘Stimulus 1’’) activates an inhibitory mechanism to minimize the disruptive effects of an identical second stimulus (‘‘Stimulus 2’’) that occurs 500 ms later (Adler et al., 1982; Freedman et al., 1987, 1996; Freedman, Adler, Waldo, Pachtman, & Franks, 1983). Patients with schizophrenia have consistently been shown to exhibit deficient inhibitory processing relative to healthy individuals with poor suppression of P50 to Stimulus 2, although methodological differences may contribute to some heterogeneity in results (Bramon et al., 2004; de Wilde, Bour, Dingemans, Koelman, & Linszen, 2007a; Heinrichs, 2001; Patterson et al., 2008). The objectives of this study were to utilize high-density electroencephalogram (EEG) source analysis and the standard P50 paradigm to evaluate potential neural generators associated with P50 generation and suppression in healthy subjects and patients with schizophrenia. Multiple lines of animal research implicate the CA3 region of the hippocampus in P50 suppression, with a particular role of the alpha7-nicotinic receptor (Adler et al., 1998; Bickford-Wimer et al., 1990). This receptor is particularly abundant in the CA3 region of the rat hippocampus (Cullum et al., 1993), with blockade of the receptor resulting in loss of suppression of the animal analogue of the human hippocampal P50 wave (LuntzLeybman, Bickford, & Freedman, 1992). Involvement of the hippocampus in sensory gating has also been demonstrated in
Deficits in the ability to filter relevant from irrelevant information are thought to contribute to many of the difficulties experienced by individuals with schizophrenia (Freedman et al., 1996; Turetsky et al., 2007; Venables, 1964), ranging from an inability to maintain focus during a conversation to feeling bombarded and overwhelmed by the physical environment. Although malfunctioning of inhibitory processes is proposed as a pathophysiological mechanism in schizophrenia (Freedman et al., 1996) and has been studied extensively with the P50 sensory gating paradigm (see Bramon, Rabe-Hesketh, Sham, Murray, & Frangou, 2004), the brain regions involved and their associated neural dynamics remain unclear. Some progress has been made, but results from brain mapping research are quite varied and often deviate from those obtained with invasive studies of humans and animals. Such discrepancies may reflect procedural differences or reliance on neuroimaging techniques that are sensitive to different and potentially non-overlapping aspects of the sensory gating process.
This study was supported in part by grants MH57322, MH37705, and Center grant P50 MH066286 from the National Institute of Mental Health, Bethesda, MD. We thank Marilyn Kesler/West, Ph.D. for assistance with study procedures. Address correspondence to: Cindy M. Yee-Bradbury, Ph.D., Department of Psychology, University of California, Los Angeles, 1285 Franz Hall, Box 951563, Los Angeles, CA 90095-1563. E-mail:
[email protected] 470
Distinct neural generators of sensory gating in schizophrenia earlier studies using depth electrodes with human epilepsy patients (Goff, Williamson, VanGilder, Allison, & Fisher, 1980; Wilson, Babb, Halgreen, Wang, & Crandall, 1984), although some more recent investigations have questioned hippocampal contributions to the P50 (e.g., Boutros et al., 2008; Rosburg et al., 2008). One constraint associated with studies relying on patients with suspected hippocampal pathologies is that they may not be representative of normal processes in healthy individuals or even abnormal functioning in schizophrenia patients. Another candidate region is the nucleus reticularis thalami, an additional area with an abundance of alpha7-nicotinic receptors (Freedman, Adams, & Leonard, 2000) involved in regulating inhibitory feedback control of thalamic to cortical pathways (Scheibel, 1997). Notably, Court et al. (1999) observed a 25% reduction of the alpha7-nicotinic receptors in this neural structure in schizophrenia patients compared to healthy subjects. Further support for thalamic involvement comes from studies with cats (Hinman & Buchwald, 1983), given the correspondence between human and cat midlatency event-related potential (ERP) responses (Erwin & Buchwald, 1987). Using trains of auditory stimuli, magnetoencephalogram (MEG) studies with non-psychiatric populations have generally localized the M50, the MEG analogue of P50, to the superior temporal gyrus (STG) or the nearby primary auditory cortex (Huotilainen et al., 1998; Kanno, Nakasato, Murayama, & Yoshimoto, 2000; Makela, Hamalainen, Hari, & McEvoy, 1994; Onitsuka, Ninomiya, Sato, Yamamoto, & Tashiro, 2003; Reite, Teale, Zimmerman, Davis, & Whalen, 1988a; Yoshiura, Ueno, Iramina, & Masuda, 1995). Godey, Schwartz, de Graaf, Chauvel, and Liegeois-Chauvel (2001) confirmed these neural sources using MEG as well as intracerebral depth electrodes implanted in the auditory cortex of epilepsy patients. Sources in the STG or the primary auditory cortex also have been demonstrated in patients with schizophrenia in response to trains of auditory stimuli (Reite et al., 1988b) as well as in response to paired stimuli (Huang et al., 2003; Thoma et al., 2003). Involvement of prefrontal brain areas is suggested by evidence of P50 suppression deficits in neurological patients with lesions in this region (Knight, Scabini, & Woods, 1989). Covering temporal and frontal lobe brain areas with intracranial microelectrode grids, Korzyukov and colleagues (2007) detected significant contributions to P50 generation from both regions in a significant proportion of epilepsy patients. Utilizing EEG source analysis with the paired-stimulus paradigm in healthy individuals, Weisser and colleagues (2001) obtained evidence to corroborate contributions from the mid-frontal cortex in P50 gating, although a specific neural structure was not specified. Synthesis of these divergent findings is provided by recent functional magnetic resonance imaging (fMRI) studies, suggesting that a neural network subserves the generation of P50 and sensory gating. Relying on a modified P50 gating paradigm, Tregellas and colleagues (2007) determined that the STG, hippocampus, thalamus, and dorsolateral prefrontal cortex (DLPFC) are all associated with the P50 response. Relative to healthy individuals, schizophrenia patients exhibited greater activation in the hippocampus, thalamus, and DLPFC but no group difference in the STG during the modified gating paradigm. Across healthy participants and schizophrenia patients, the P50 ratio score was found to be positively correlated with activation in the hippocampus, thalamus, and DLPFC. The authors noted that failure to detect group differences may have resulted from reliance on a modified P50 paradigm and on a
471 summed hemodynamic response. These associations were largely replicated in a subsequent study using urban white noise as the stimulus (Tregellas, Ellis, Shatti, Du, & Rojas, 2009). Similarly, Mayer and colleagues (2009) showed distinct contributions from auditory cortices, prefrontal cortex, and thalamus when evaluating the hemodynamic response in healthy subjects. Results from these human brain mapping and animal studies highlight several key brain regions that may contribute to P50 and its suppression as well as the P50 gating deficit in schizophrenia. The present investigation aimed to build on these earlier findings and to utilize high-density EEG source analysis and the paired-stimulus P50 paradigm to further evaluate the potential neural generators associated with P50 gating in healthy subjects and patients with schizophrenia. EEG source analysis provides a unique opportunity to evaluate the association between a neural network involving the STG, hippocampus, DLPFC, and thalamus, and EEG-derived measures of P50 that constitute much of the research literature on sensory gating in schizophrenia.
Methods Participants Sixteen patients with schizophrenia were entered into the study. All patients were clinically stable as reflected by the Brief Psychiatric Rating Scale (BPRS) (Ventura et al., 1993) and receiving antipsychotic medication at the time of testing. Thirteen schizophrenia patients were treated with second-generation antipsychotic medications, two patients received first-generation antipsychotics, and one patient received both. Antiparkinsonian medications were discontinued at least 24 h before testing due to potential anticholinergic effects on the dependent measures. Patients were drawn from participants originally recruited for the UCLA Developmental Processes in Schizophrenic Disorder project (Nuechterlein et al., 1992). Diagnoses were made with the Structured Clinical Interview for DSM-IV (SCID; Ventura, Liberman, Green, Shaner, & Mintz, 1998). Seventeen healthy comparison subjects were recruited from the community, screened with the SCID, and excluded if a past or current major psychiatric disorder was reported for the participant or a first-degree relative. Exclusion criteria for both groups included substance or alcohol abuse in the last 3 months, a history of head trauma, a major medical condition or loss of consciousness for more than 5 min, and mental retardation. Data were excluded from 2 healthy participants due to an insufficient number of artifact-free trials. Data from an additional comparison subject were excluded because the P50 to Stimulus 1 was below 0.5 microvolts (mV) and could not be measured reliably. Thus, the final sample consisted of 14 healthy individuals. After providing a complete description of the study, written informed consent was obtained from all participants. Demographic and clinical characteristics are summarized in Table 1. Psychophysiological Recording Methods and Apparatus EEG recordings were obtained using an elastic cap containing 124 Ag-AgCl sintered electrodes (Falk Minow Services, Herrsching, Germany) with an equidistant layout. All electrode sites, including the right earlobe, were referenced to the left earlobe during data acquisition and re-referenced offline to averaged earlobes. The electrooculogram (EOG) was recorded by placing electrodes above and below the right eye and near the outer
472
T. J. Williams et al.
Table 1. Demographic and Clinical Characteristics of Participants
Age (years) Education (years) Parental education (years) Antipsychotic medication dosage (CPZ equiv in mg) Duration of illness (years) BPRS 24-item total score
Gender Male Female Race Asian African American Hispanic or Latino/a White Mixed Diagnosis Schizophrenia Schizoaffective Schizophreniform
Healthy Comparison subjects (n 5 14)
Schizophrenia patients (n 5 16)
M
SD
M
SD
Statistic
p
23.4 14.6 14.3 NA NA NA
4.4 1.4 2.3a
27.6 13.5 12.2 263 4.8 34.2
7.0 1.5 4.3 145 6.0 9.1
F(1,28) 5 3.62 F(1,28) 5 3.91 F(1,25) 5 2.13 – – –
.07 .06 .16 – – –
X2(1) 5 .20
.65
X2(4) 5 5.89
.21
–
–
n
n
9 5
9 7
1 5 4 3 1
2 1 6 7 0
NA NA NA
13 2 1
Note: CPZ equiv: Chlorpromazine equivalents, BPRS: Brief Psychiatric Rating Scale. a n 5 11.
canthi of the eyes. All impedances were below 10 kOhms. Signals were collected with a SynAmps system (Neuroscan, Charlotte, NC) at a sampling rate of 1000 Hz and using a bandpass of 0.5 Hz to 200 Hz. The EEG was amplified 2,500 times and EOG signals were amplified 500 times with a resolution of .03 mV and .17 mV per least significant bit, respectively. Auditory stimuli. Stimuli were delivered using the STIM presentation unit (Neuroscan) and presented through foam-insert earphones. Threshold levels for each ear were determined separately, and stimuli were delivered at 55-dB sound pressure level (SPL) above each subject’s sound threshold. Stimuli were 3 ms in duration and presented in pairs with an interstimulus interval of 500 ms. The ITI between pairs of stimuli varied between 9 and 12 s. Procedure. To verify normal hearing, audiometric screening was conducted by presenting participants with a range of sound intensities (40 to 0 decibels [dB] in 5-dB decrements) at different frequencies (250, 1000, 2000, 4000, and 8000 Hz). All participants were able to detect sounds above 30 dB SPL at each frequency with each ear. After instructing participants to sit quietly in a sound-attenuated room, they were presented with 80 trials of paired stimuli, with a 30-s rest period after 40 trials. Waveform and scalp component analysis. After single trials were screened to exclude movement artifact, blind source separation by extended Infomax independent component analysis (ICA) was performed in Matlab (The Mathworks, Inc., Natick, MA) to correct for eye movement and heart rate artifact (Jung et al., 2001; Lee, Girolami, & Sejnowski, 1999). On average, 5.94% (SD 5 2.56%) and 4.79% (SD 5 1.87%) of the data were removed from healthy subjects and schizophrenia patients, respectively, with no significant difference between the groups (F 5 1.99, df 5 1, 28, p 5 .17). Non-cerebral artifacts were iden-
tified and removed according to standardized criteria by technicians blind to diagnostic status. Possible artifact activity in an independent component was required to correspond temporally to artifact activity visible in the raw data prior to removal. All trials were then filtered at 10–50 Hz for measuring the P50 ERP while minimizing contributions from N100. A prestimulus baseline of ! 200 to 0 ms was subtracted from each average waveform before ERP scoring, but not prior to EEG source analysis. A minimum of 60 trials was included in each ERP average. For standard ERP component scoring, a computer algorithm was implemented. P50 was measured at Cz and identified as the most positive peak between 35 and 75 ms after stimulus onset. P50 amplitude was measured as the difference between the P50 amplitude and the N40 amplitude. The N40 was determined as the most negative peak between the P30 and P50 latency. The P30 was identified as the most positive component between 20 and 40 ms after stimulus onset. As noted above, one healthy subject was excluded from all analyses because P50 amplitude to Stimulus 1 was less than .5 mV, and it is difficult to discriminate such a small signal from noise in the data. In the absence of a discernable P50 peak in response to Stimulus 2, it was scored as having zero amplitude and interpreted as reflecting complete suppression. All scoring was subsequently verified by trained raters who were blind to group membership. The P50 suppression ratio was calculated as P50 amplitude to Stimulus 2 divided by P50 amplitude to Stimulus 1. EEG Source Analysis. All source analysis computations were carried out using CURRY V5.0 software (Neuroscan). To reconstruct the generators of the measured scalp data, a realistically shaped boundary element model (BEM) was generated using the standardized MRI dataset available in CURRY (Fuchs, Kastner, Wagner, Hawes, & Ebersole, 2002). To co-register EEG data with the standardized MRI dataset from the
Distinct neural generators of sensory gating in schizophrenia Montreal Neurological Institute (MNI), three-dimensional coordinates of the electrode positions and three fiducial landmarks (the nasion, and the left and right preauricular points) were determined for each participant with an Isotrak spatial digitizer. For source localization, landmarks and electrode positions were then aligned with the standardized MRI image. The BEM model contains 3 layers, which represent the scalp, skull, and brain. Standard conductivities of 0.33, 0.0042, and 0.33 S/m were used for the cerebrospinal fluid, skull, and skin, respectively. For dipole fitting, a time interval was selected for each individual by including activity between the two time points of the filtered and trial-averaged P50 waveform that corresponded to a signal amplitude of 50% of the P50 peak amplitude relative to the value of the preceding and subsequent negative peaks. The average strength of a source or ‘‘dipole moment’’ was determined over this latency range. Due to the restrictive filter settings, risk of tapping into the N100 should be minimal. Fixed dipoles were seeded based on prior empirical findings as described above. Specifically, dipoles were seeded bilaterally and allowed to vary within a 1-cm radius around the center of the seeded position (right and left x, y, z coordinates provided in MNI space) at a) STG (right: 49.2, 4.9, 52.4; left: ! 51.1, ! .6, 54.3), hippocampus (right: 27.4, 7.3, 28.6; left: ! 27.6, 7.3, 28.6), DLPFC (right: 36.3, 71.1, 59.8; left: ! 35.5, 67.7, 59.8), and thalamus (right: 10.9, 7.9, 65.0; left: ! 10.2, 7.9, 65.0). Dipoles were fitted simultaneously, and dipole orientations were allowed to vary. Considering the increased risk of producing an overdetermined model when seeding a number of dipoles, the goodness-offit (Gof) of the proposed model was validated by comparing it to a model using 8 theoretically unrelated dipoles. If the Gof of the proposed dipole model is due solely to the seeding of a large number of dipoles rather than the capture of a valid solution, there should be no difference in the amount of variance explained by the two models. If, however, the proposed model explains more of the variance than the model consisting of theoretically unrelated dipoles, it can be inferred that the proposed neural regions are more likely involved in P50 generation. Using the same seeding procedure described above, dipoles for the second model were seeded at (x,y,z MNI coordinates) dipole 1 ( ! 18.2, ! 69.0, 58.0), dipole 2 (28.8, ! 42.0, 12.4), dipole 3 (0.0, 63.1, 27.2), dipole 4 ( ! 40.3, 16.1, 100.2), dipole 5 (17.3, ! 66.5, 62.5), dipole 6 (28.8, ! 39.4, 97.0), dipole 7 (34.5, 31.7, 102.3), and dipole 8 ( ! 16.3, 36.6, 81.4). To gain confidence in whether each neural region is a component of the proposed neural network, the individual contributions of the STG, hippocampus, thalamus, and DLPFC also were examined by comparing each region against theoretically unrelated bilateral dipoles. The fidelity of the proposed source model was further evaluated by examining the number of dipoles localized to the edge of the 1-cm search radius. A dipole was considered to have reached the edge when any of its MNI coordinates (i.e., x, y, z) fell within .5 mm of the outer search window. This analysis was conducted separately for each group and stimulus. Data analysis. Analyses of variance (ANOVA) were conducted to examine traditional EEG indices of P50 suppression. To determine if group differences were present, the P50 ratio score was examined with one between-subjects factor, group (healthy comparison subjects versus schizophrenia patients). P50 amplitude data were subjected to ANOVA using one within-subjects fully crossed factor, stimulus (Stimulus 1 versus Stimulus 2), and
473 one between-subjects factor, group, to provide a difference score measure of suppression. Greenhouse-Geisser corrected p values were used throughout (Geisser & Greenhouse, 1958). The Gof of the dipole models was expressed as the percentage of explained variance that the dipole solution fit the observed P50 to Stimulus 1 and Stimulus 2. To test for group differences, a repeated-measures ANOVA was applied with one within-subjects factor, stimulus, and one between-subjects factor, group. A one-way ANOVA was calculated for each stimulus separately to compare the Gof of (1) the proposed dipole model with the model consisting of 8 theoretically unrelated dipoles, and (2) each neural region (i.e., STG, hippocampus, DLPFC, and thalamus) individually with bilateral theoretically unrelated dipoles. To examine potential changes in dipole moments between Stimulus 1 and Stimulus 2, a repeated-measures ANOVA was conducted with two within-subjects factors, neural structure (STG vs. hippocampus vs. thalamus vs. DLPFC) and stimulus, and one between-subjects factor, group. Post hoc analyses were performed throughout using t-tests at a 95% level of confidence. An analog to the P50 suppression ratio was calculated for each neural structure (dipole moment 2/dipole moment 1) in order to assess the relationships between dipole moments and the P50 suppression ratio. Pearson correlations were computed to evaluate these relationships. Results Demographic and Clinical Characteristics As shown in Table 1, the schizophrenia patient group was matched to the healthy comparison group on level of parental education, sex, and race. The patient sample tended to be somewhat older and their education levels were slightly lower than the healthy group, as reflected by statistical trend effects. Average ratings on the BPRS indicate that the patients exhibited low symptom levels overall, as might be expected of a clinically stabilized sample. P50 Suppression Ratio and P50 Amplitudes Grand-average ERP waveforms are presented in Figure 1. As expected, schizophrenia patients exhibited impaired P50 suppression (M 5 .59, SD 5 .31) relative to healthy comparison subjects (M 5 .35, SD 5 .20) on the ratio measure (F 5 6.26, df 5 1, 28, po.05). P50 suppression ratios and amplitudes obtained for the two groups are illustrated in Figure 2. Although the group by stimulus interaction was not statistically significant (F 5 .47, df 5 1,28, p 5 .50), the significant P50 ratio score difference between groups can be attributed largely to differences in Stimulus 2 amplitude (schizophrenia patients: M 5 2.51, SD 5 1.44; healthy comparison subjects: M 5 1.55, SD 5 1.16; F 5 3.92, df 5 1,28, p 5 .058) as amplitude differences to Stimulus 1 were not evident between the groups (schizophrenia patients: M 5 4.56, SD 5 2.63; healthy comparison subjects: M 5 4.06, SD 5 2.46; F 5 .29, df 5 1,28, p 5 .60). Evaluation of Dipole Model To assess Gof of the proposed dipole model, the total amount of variance explained was examined. A considerable percentage of the variance was accounted for in healthy subjects: (Stimulus 1 5 94.89%, Stimulus 2 5 94.39%) and patients with schizophrenia (Stimulus 1 5 93.70%, Stimulus 2 5 91.02%). No effects involving group or stimulus approached statistical significance. To validate the adequacy of the proposed model, an anatomically unconstrained dipole was added but was not found
474
T. J. Williams et al.
Figure 1. Grand average event-related potential waveforms at the Cz recording site. Waveforms are unfiltered, and the N40 and P50 components are indicated with arrowheads.
to contribute substantial power in explaining the total variance (i.e., less than .5%). Moreover, the individual contribution of each region within the proposed neural network accounted for a significantly greater proportion of the total variance than that of theoretically unrelated dipoles (all p-values less than .05). The proposed dipole model was also found to explain significantly more of the total variance when compared to the model consisting of 8 theoretically unrelated dipoles for Stimulus 1 (Ms 5 94.26% vs. 91.90%, respectively; F 5 10.07, df 5 29, po.01) and Stimulus 2 (Ms 5 92.59% vs. 88.90%, respectively; F 5 16.67, df 5 29, po.001). Lastly, relatively few dipoles were localized at the edge of the search window. Across the x, y,
Figure 2. Mean P50 suppression ratios and P50 amplitudes to paired stimuli for healthy normal comparison subjects and schizophrenia patients.
Figure 3. Mean dipole moments to paired stimuli for the hippocampus, thalamus, dorsolateral prefrontal cortex, and superior temporal gyrus for healthy normal comparison subjects and schizophrenia patients. mAmm: micro Ampere per millimeter, HC: hippocampus, TH: thalamus, DLPFC: dorsolateral prefrontal cortex, STG: superior temporal gyrus.
and z coordinates, 91.35% of Stimulus 1 and 88.46% of Stimulus 2 dipoles did not reach the search radius edge for healthy individuals. Similarly, 95.31% of Stimulus 1 and 85.16% of Stimulus 2 dipoles fell below the search radius limit for patients with schizophrenia. Evaluation of Dipole Moments Comparison of Stimulus 1 and Stimulus 2 dipole moments across the STG, hippocampus, thalamus, and DLPFC revealed a significant main effect for neural structure (F 5 42.50, df 5 3,84, po.001, e 5 .43), with the hippocampus and thalamus showing the greatest source strength across both groups (see Figure 3). Consistent with the inverse-square law, this pattern of findings is to be expected as deep structures require large current flows to be detected at distant scalp sites. There were no significant interaction effects involving group, neural structure, or stimulus. Examination of the relationship between the ratio of the dipole moment of each neural structure and P50 suppression for healthy comparison subjects revealed that the hippocampal dipole moment ratio significantly correlated with the P50 suppression ratio (r 5 .69, po.01; see Figure 4, top panel). Although not statistically significant, there also was some suggestion of an association between the DLPFC dipole moment ratio and the P50 suppression ratio (r 5 .46, p 5 .10). STG and thalamic dipole moment ratios were not significantly correlated with the P50 suppression ratio (r 5 .41, p 5 .15; r 5 .23, p 5 .44, respectively). A somewhat different pattern of associations emerged for schizophrenia patients. In contrast to healthy subjects, patients did not exhibit a significant correlation between the hippocampal dipole moment ratio and the P50 suppression ratio (r 5 .21, p 5 .43; see Figure 4, bottom panel). A significant association was observed between the DLPFC dipole moment ratio and the P50 suppression ratio (r 5 .61, po.05). Similar to healthy individuals, correlations between the P50 ratio and the STG or the
Distinct neural generators of sensory gating in schizophrenia
475 including only STG and thalamus (F change (1,10) 5 7.90, po.05). When the DLPFC dipole moment ratio was entered during the second step, a statistical trend was observed (F change (1,10) 5 3.94, p 5 .08). For schizophrenia patients, the regression model including the STG, thalamus, and hippocampus was not significant relative to the model involving only STG and thalamus (F change (1,12) 5 1.37, p 5 .26). However, the DLPFC dipole moment ratio accounted for unique variance when added to the STG and thalamic dipole moment ratios (F change (1,12) 5 7.29, po.05). Discussion
Figure 4. Associations between the P50 suppression ratio and the hippocampal dipole moment ratio for healthy comparison subjects and schizophrenia patients.
thalamic dipole moment ratios were not statistically significant (r 5 .06, p 5 .82; r 5 .00, p 5 .99, respectively). Planned comparisons were conducted to examine whether the groups differed statistically in the strength of the relationships between the dipole moment ratio for each neural source and the P50 ratio score. For the association between the hippocampal dipole moment ratio and the P50 suppression ratio, the group difference fell just short of conventional levels of statistical significance (p 5 .06, one-tailed). There were no significant group differences in the strength of relationships between DLPFC, STG, or thalamus and the P50 ratio (all ps4.18, one-tailed). Among healthy subjects, the strength of the associations also did not differ reliably between the four neural structures (ps4.15 one-tailed), with the possible exception of a marginally significant effect involving the hippocampus and thalamus (p 5 .08, one-tailed). For patients with schizophrenia, the P50 ratio score correlated more strongly with the DLPFC dipole moment ratio than with the STG or thalamic dipole moment ratios (all pso.05). In order to assess whether hippocampal and DLPFC dipole moment ratios accounted for unique variance in the P50 ratio score, hierarchical linear regression analyses were performed separately for the two groups. In each of the regression analyses, STG and thalamic dipole moment ratios were entered in the first step because both neural structures are implicated in basic auditory processing. As a second step, the hippocampal or DLPFC dipole moment ratio was entered. For healthy subjects, the regression model including the STG, thalamus, and hippocampus accounted for significantly more variance than the model
The present study used EEG source localization to evaluate the neural processes associated with a distributed neural network, involving the STG, hippocampus, DLPFC, and thalamus, which has been proposed to be involved in the generation of P50 and its suppression in healthy individuals and schizophrenia patients. Prior research has produced diverse findings regarding the neural generators associated with P50. The current results provide additional evidence to support an integrated perspective, consistent with results from non-traditional P50 paradigms that were developed for use with fMRI (Mayer et al., 2009; Tregellas et al., 2007, 2009). Diverging from results obtained from fMRI, however, the present data suggest that schizophrenia patients and healthy comparison subjects differ in the relative contributions made by different neuronal sources in the suppression of the P50. Because the correlation coefficients between dipole moment ratios and the P50 gating ratio did not differ significantly across all four neural structures in healthy individuals, it appears likely that the STG, hippocampus, DLPFC, and thalamus all contributed to the generation of P50 and its suppression. Nonetheless, it was noteworthy that the hippocampal dipole moment ratio may have been more strongly associated with P50 suppression in healthy subjects. These results corroborate previous research implicating the CA3 region of the hippocampus in P50 suppression (Bickford-Wimer et al., 1990; Cullum et al., 1993; LuntzLeybman et al., 1992). Specifically, Adler and colleagues (1998) describe a neurophysiological pathway for P50 suppression with the hippocampus as the central site of neural activation. In response to Stimulus 1, projections to the CA3 region appear to stimulate inhibitory interneurons through the opening of alpha7nicotinic receptors. Release of GABA (gamma-aminobutyric acid) from the interneurons then facilitates inhibition of postsynaptic pyramidal cells, particularly due to long-lasting inhibition via GABAB receptors on postsynaptic cells. Thus, when excitation as a result of Stimulus 2 arrives in the CA3 region, it is automatically suppressed through the lasting inhibition produced by Stimulus 1. A different pattern of relationships between P50 suppression and dipole moments emerged for schizophrenia patients. For this group, the hippocampal dipole moment ratio was not found to be significantly associated with P50 suppression. A statistical trend for a group difference in correlation coefficients further suggests that hippocampal functioning may have had a smaller impact on P50 suppression in schizophrenia patients relative to healthy subjects. However, the DLPFC dipole moment ratio was significantly associated with the P50 ratio in patients. Although not statistically significant, a moderate correlation between the DLPFC dipole moments and the P50 suppression ratio also was observed in healthy participants. The absence of group
476
T. J. Williams et al.
differences in correlation coefficients further indicates that both groups likely relied to some extent on DLPFC for P50 suppression. Consistent with such an interpretation, frontal areas have been implicated in P50 suppression in several studies involving non-psychiatric subjects (Boutros et al., 2008; Knight et al., 1989; Korzyukov et al., 2007; Weisser et al., 2001). While still speculative, a possible interpretation of these findings is that the DLPFC may be particularly critical for P50 suppression in schizophrenia patients. Such an association is further suggested by the finding of a significantly stronger relationship between the DLPFC dipole moment ratio and the P50 ratio score as compared to associations involving STG or the thalamus. However, given poorer P50 suppression in schizophrenia, this pattern may also reflect the DLPFC’s general inefficiency in suppressing the P50. Such possibilities are consistent with the ability of schizophrenia patients to transiently normalize deficits in P50 suppression when directing voluntary attention to Stimulus 1 and presumably further engaging prefrontal activity (Yee et al., 2010). These inferences do need to be considered in light of sample sizes that may have been insufficient for detecting significant correlations between STG or thalamic dipole moment ratios and the P50 suppression ratio. The absence of significant differences in correlation coefficients between dipole moment ratios and the P50 suppression ratio across neural structures in healthy subjects, for example, suggests that the STG and thalamus may also be contributing to the P50 suppression effect. With greater statistical power, it should then be possible to determine the degree to which healthy subjects and patients with schizophrenia are relying upon the same or different brain regions during P50 gating. The absence of an association between the ratio computed from hippocampal dipole moments and the P50 suppression ratio in the patient group is consistent with hippocampal dysfunction in schizophrenia. Hippocampal volume reductions have been documented in schizophrenia patients across numerous studies (Narr et al., 2004; Szeszko et al., 2003; Thoma et al., 2008; Velakoulis et al., 1999). Moreover, research suggests that the hippocampus provides important inputs to the DLPFC and vice versa (Goldman-Rakic, Selemon, & Schwartz, 1984), with schizophrenia patients exhibiting abnormalities in the neural connections between these two structures (Friston, 1998). While highly speculative, the present results may reflect a similar pattern of dysfunctional connectivity between two structures that appear to contribute to P50 suppression. To assess the possibility that the proposed dipole solution was merely the result of an overdetermined model, a comparison was conducted with a model consisting of theoretically unrelated dipoles. If the Gof represents solely non-meaningful variance, one would expect to observe comparable Gof values between the two models. Our results indicated significant differences between the models and validated the proposed model as reflecting the neural network underlying P50 suppression. Moreover, systematic evaluation of the amount of variance explained by the four brain regions under consideration indicated they fitted the data better, individually and collectively, than theoretically unrelated sources. Another consideration is the prevailing view that EEG is unlikely to detect signals from deeper brain structures, including the
hippocampus and thalamus. However, there is now substantial research to dispute this perspective. Relying on MEG source analysis, multiple empirical studies have provided evidence for its capacity to detect hippocampal activity (Breier, Simos, Zouridakis, & Papanicolaou, 1999; Gordon, Rennie, & Collins, 1990; Hanlon et al., 2003, 2005; Horiguchi, Ohta, & Nishikawa, 2003; Ioannides et al., 1995; Kikuchi et al., 1997; Kimora, Ozaki, & Hashimoto, 2008; Luo, Holroyd, Jones, Hendler, & Blair, 2007; Maestu et al., 2003; Miller, 2008; Moses et al., 2009; Nishitani, 2003; Nishitani et al., 1999; Nishitani, Nagamine, Fujiwara, Yazawa, & Shibasaki, 1998; Okada, Kaufman, & Williamson, 1983; Rogers, Basile, Papanicolaou, & Eisenberg, 1993; Simos, Basile, & Papanicolaou, 1997; Tesche, 1996, 1997; Tesche & Karhu, 1999, 2000; Tesche, Karhu, & Tissari, 1996). To date, however, MEG studies of sensory gating in schizophrenia have not focused on the hippocampus (Huang et al., 2003; Thoma et al., 2003). Utilizing EEG, the present study relied upon a 124-channel recording montage that provided coverage of most of the scalp, including electrodes positioned a few centimeters below the temporal lobe area, and should facilitate detection of activity from deeper brain sources. Taken together, there is a basis for inferring that neural activity ascribed to the hippocampus and thalamus can be detected with densely arrayed EEG sensors. Beyond sample size, other methodological limitations need to be considered. Dipoles were seeded and, consequently, created a confirmatory bias. Without a theoretical or empirical basis for seeding additional anatomically constrained dipoles, it is possible that other key neural regions were overlooked. It also is the case that associations detected between P50 ratio scores and dipole sources were correlational and should be interpreted with some caution. Because the present study sought to confirm prior source analyses, filter settings were implemented that have been the standard for P50 research. Recent studies, however, support the utility of examining a broader range of EEG frequencies (Hong et al., 2008; Jansen, Hedge, & Boutros, 2004), and they would be complemented by source analysis. A statistically significant group-by-stimulus interaction effect on P50 amplitude also was not detected. Despite the absence of this interaction, the presence of a significant group difference in the P50 ratio measure suggests a suppression deficit in the schizophrenia patients. It is possible that our ability to distinguish statistically between the patient and control samples with the amplitude measure was constrained by the beneficial effects of some second-generation antipsychotic medications on P50 gating (Adler et al., 2004; Light, Geyer, Clementz, Cadenhead, & Braff, 2000; Yee, Nuechterlein, Morris, & White, 1998). Additionally, the average duration of illness in the present sample was less than 5 years, and there is evidence indicating that the P50 deficit may be less pronounced during the early course of schizophrenia (de Wilde, Bour, Dingemans, Koelman, & Linszen, 2007b; Yee et al., 2010). In sum, current results suggest that a dynamic interplay exists between brain regions involved in P50 gating, with treatment implications for targeting the DLPFC and attention (Yee et al., 2010) to compensate for inhibitory deficits in schizophrenia. More broadly, our findings support a neural connectivity approach to understanding the regulation of sensory gating in future investigations of schizophrenia.
REFERENCES Adler, L. E., Olincy, A., Cawthra, E. M., McRae, K. A., Harris, J. G., Nagamoto, H. T., et al. (2004). Varied effects of atypical neuroleptics
on P50 auditory gating in schizophrenia patients. American Journal of Psychiatry, 161, 1822–1828.
Distinct neural generators of sensory gating in schizophrenia Adler, L. E., Olincy, A., Waldo, M., Harris, J. G., Griffith, J., Stevens, K., et al. (1998). Schizophrenia, sensory gating, and nicotinic receptors. Schizophrenia Bulletin, 24, 189–202. Adler, L. E., Pachtman, E., Franks, R. D., Pecevich, M., Waldo, M. C., & Freedman, R. (1982). Neurophysiological evidence for a defect in neuronal mechanisms involved in sensory gating in schizophrenia. Biological Psychiatry, 17, 639–654. Bickford-Wimer, P. C., Nagamoto, H., Johnson, R., Adler, L. E., Egan, M., Rose, G. M., & Freedman, R. (1990). Auditory sensory gating in hippocampal neurons: A model system in the rat. Biological Psychiatry, 27, 183–192. Boutros, N. N., Mears, R., Pflieger, M. E., Moxon, K. A., Ludowig, E., & Rosburg, T. (2008). Sensory gating in the human hippocampal and rhinal regions: Regional differences. Hippocampus, 18, 310–316. Bramon, E., Rabe-Hesketh, S., Sham, P., Murray, R. M., & Frangou, S. (2004). Meta-analysis of the P300 and P50 waveforms in schizophrenia. Schizophrenia Research, 70, 315–329. Breier, J. I., Simos, P. G., Zouridakis, G., & Papanicolaou, A. C. (1999). Lateralization of cerebral activation in auditory verbal and non-verbal memory tasks using magnetoencephalography. Brain Topography, 12, 89–97. Court, J., Spurden, D., Lloyd, S., McKeith, I., Ballard, C., Cairns, N., & Kerwin, R. (1999). Neuronal nicotinic receptors in dementia with Lewy bodies and schizophrenia: Alpha-bungarotoxin and nicotine binding in the thalamus. Journal of Neurochemistry, 73, 1590–1597. Cullum, C. M., Harris, J. G., Waldo, M. C., Smernoff, E., Madison, A., Nagamoto, H. T., et al. (1993). Neurophysiological and neuropsychological evidence for attentional dysfunction in schizophrenia. Schizophrenia Research, 10, 131–141. de Wilde, O. M., Bour, L. J., Dingemans, P. M., Koelman, J. H., & Linszen, D. H. (2007a). A meta-analysis of P50 studies in patients with schizophrenia and relatives: Differences in methodology between research groups. Schizophrenia Research, 97, 137–151. de Wilde, O. M., Bour, L. J., Dingemans, P. M., Koelman, J. H., & Linszen, D. H. (2007b). Failure to find P50 suppression deficits in young first-episode patients with schizophrenia and clinically unaffected siblings. Schizophrenia Bulletin, 33, 1319–1323. Erwin, R. J., & Buchwald, J. S. (1987). Midlatency auditory evoked responses in the human and the cat model. Electroencephalogry and Clinical Neurophysiology Supplement, 40, 461–467. Freedman, R., Adams, C. E., & Leonard, S. (2000). The alpha7-nicotinic acetylcholine receptor and the pathology of hippocampal interneurons in schizophrenia. Journal of Chemical Neuroanatomy, 20, 299–306. Freedman, R., Adler, L. E., Gerhardt, G. A., Waldo, M., Baker, N., Rose, G. M., et al. (1987). Neurobiological studies of sensory gating in schizophrenia. Schizophrenia Bulletin, 13, 669–678. Freedman, R., Adler, L. E., Myles-Worsley, M., Nagamoto, H. T., Miller, C., Kisley, M., et al. (1996). Inhibitory gating of an evoked response to repeated auditory stimuli in schizophrenic and normal subjects. Archives of General Psychiatry, 53, 1114–1121. Freedman, R., Adler, L. E., Waldo, M. C., Pachtman, E., & Franks, R. D. (1983). Neurophysiological evidence for a defect in inhibitory pathways in schizophrenia: Comparison of medicated and drug-free patients. Biological Psychiatry, 18, 537–551. Friston, K. J. (1998). The disconnection hypothesis. Schizophrenia Research, 30, 115–125. Fuchs, M., Kastner, J., Wagner, M., Hawes, S., & Ebersole, J. S. (2002). A standardized boundary element method volume conductor model. Clinical Neurophysiology, 112, 701–712. Geisser, S., & Greenhouse, S. W. (1958). An extension of Box’s results on the use of the F distribution in multivariate analysis. Annals of Mathematics and Statistics, 29, 886–891. Godey, B., Schwartz, D., de Graaf, J. B., Chauvel, P., & LiegeoisChauvel, C. (2001). Neuromagnetic source localization of auditory evoked fields and intracerebral evoked potentials: A comparison of data in the same patients. Clinical Neurophysiology, 112, 1850–1859. Goff, W. R., Williamson, P. D., VanGilder, J. C., Allison, T., & Fisher, T. C. (1980). Neural origins of long latency evoked potentials recorded from the cortical surface of the brain in man. In: J. E. Desmedt (Ed.), Clinical uses of cerebral, brain stem and spinal somatosensory evoked potentials (pp. 126–145). New York: S. Krager. Goldman-Rakic, P. S., Selemon, L. D., & Schwartz, M. L. (1984). Dual pathways connecting the dorsolateral prefrontal cortex with the
477 hippocampal formation and parahippocampal cortex in the rhesus monkey. Neuroscience, 12, 719–743. Gordon, E., Rennie, C., & Collins, L. (1990). Magnetoencephalography and late component ERPs. Clinical and Experimental Neurology, 27, 113–120. Hanlon, F. M., Weisend, M. P., Huang, M., Lee, R. R., Moses, S. N., Paulson, K. M., et al. (2003). A non-invasive method for observing hippocampal function. NeuroReport, 14, 1957–1960. Hanlon, F. M., Weisend, M. P., Yeo, R. A., Huang, M., Lee, R. R., Thoma, R. J., et al. (2005). A specific test of hippocampal deficit in schizophrenia. Behavioral Neuroscience, 119, 863–875. Heinrichs, R. W. (2001). In search of madness: Schizophrenia and neuroscience. New York: Oxford University Press. Hinman, C. L., & Buchwald, J. S. (1983). Depth evoked potential and single unit correlates of vertex midlatency auditory evoked responses. Brain Research, 264, 57–67. Hong, L. E., Summerfelt, A., Mitchell, B. D., McMahon, R. P., Wonodi, I., Buchanan, R. W., & Thaker, G. K. (2008). Sensory gating endophenotype based on its neural oscillatory pattern and heritability estimate. Archives of General Psychiatry, 65, 1008–1016. Horiguchi, T., Ohta, K., & Nishikawa, T. (2003). An MEG study of P300 activity during a color discrimination task 2: Source localization study. Brain Development, 25, 241–244. Huang, M. X., Edgar, J. C., Thoma, R. J., Hanlon, F. M., Moses, S. N., Lee, R. R., et al. (2003). Predicting EEG responses using MEG sources in superior temporal gyrus reveals source asynchrony in patients with schizophrenia. Clinical Neurophysiology, 114, 835–850. Huotilainen, M., Winkler, I., Alho, K., Escera, C., Virtanen, J., Ilmoniemi, R. J., et al. (1998). Combined mapping of human auditory EEG and MEG responses. Electroencephalography and Clinical Neurophysiology, 108, 370–379. Ioannides, A. A., Liu, M. J., Liu, L. C., Bamidis, P. D., Hellstrand, E., & Stephan, K. M. (1995). Magnetic field tomography of cortical and deep processes: Examples of ‘‘real-time mapping’’ of averaged and single trial MEG signals. International Journal of Psychophysiology, 20, 161–175. Jansen, B. H., Hedge, A., & Boutros, N. N. (2004). Contribution of different EEG frequencies to auditory evoked potential abnormalities in schizophrenia. Clinical Neurophysiology, 115, 523–533. Jung, T. P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., & Sejnowski, T. J. (2001). Analysis and visualization of single-trial event-related potentials. Human Brain Mapping, 14, 166–185. Kanno, A., Nakasato, N., Murayama, N., & Yoshimoto, T. (2000). Middle and long latency peak sources in auditory evoked magnetic fields for tone bursts in humans. Neuroscience Letters, 293, 187–190. Kikuchi, Y., Endo, H., Yoshizawa, S., Kait, M., Nishimura, C., Tanaka, M., et al. (1997). Human cortico-hippocampal activity related to auditory discrimination revealed by neuromagnetic field. NeuroReport, 8, 1657–1661. Kimora, T., Ozaki, I., & Hashimoto, I. (2008). Impulse propagation along thalamocortical fibers can be detected magnetically outside the human brain. Journal of Neuroscience, 28, 12535–12538. Knight, R. T., Scabini, D., & Woods, D. L. (1989). Prefrontal cortex gating of auditory transmission in humans. Brain Research, 504, 338–342. Korzyukov, O., Pflieger, M., Wagner, M., Bowyer, S., Rosburg, T., Sundaresan, K., & Elger, C. (2007). Generators of the intracranial P50 response in auditory sensory gating. NeuroImage, 35, 814–826. Lee, T. W., Girolami, M., & Sejnowski, T. J. (1999). Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation, 11, 417–441. Light, G. A., Geyer, M. A., Clementz, B. A., Cadenhead, K. S., & Braff, D. L. (2000). Normal P50 suppression in schizophrenia patients treated with atypical antipsychotic medications. American Journal of Psychiatry, 157, 767–771. Luntz-Leybman, V., Bickford, P. C., & Freedman, R. (1992). Cholinergic gating of response to auditory stimuli in rat hippocampus. Brain Research, 587, 130–136. Luo, Q., Holroyd, T., Jones, M., Hendler, T., & Blair, J. (2007). Neural dynamics for facial threat processing as revealed by gamma band synchronization using MEG. NeuroImage, 34, 839–847.
478 Maestu, F., Arrazola, J., Fernandez, A., Simos, P. G., Amo, C., GilGregorio, P., et al. (2003). Do cognitive patterns of brain magnetic activity correlate with hippocampal atrophy in Alzheimer’s disease? Journal of Neurological and Neurosurgical Psychiatry, 74, 208–212. Makela, J. P., Hamalainen, M., Hari, R., & McEvoy, L. (1994). Wholehead mapping of middle-latency auditory evoked magnetic fields. Electroencephalography and Clinical Neurophysiology, 92, 414–421. Mayer, A. R., Hanlon, F. M., Franco, A. R., Teshiba, T. M., Thoma, R. J., Clark, V. P., & Canive, J. M. (2009). The neural networks underlying sensory gating. NeuroImage, 44, 182–189. Miller, G. A. (2008). Localization for hippocampal abnormality: Hunting deep structures with MEG. Presentation at 4-D Neuroimaging 3rd Annual International Users Group Meeting, Barcelona. Moses, S. N., Ryan, J. D., Bardouille, T., Kovacevic, N., Hanlon, F. M., & McIntosh, A. R. (2009). Semantic information alters neural activation during transverse patterning performance. NeuroImage, 46, 863–873. Narr, K. L., Thompson, P. M., Szeszko, P., Robinson, D., Jang, S., Woods, R. P., et al. (2004). Regional specificity of hippocampal volume reductions in first-episode schizophrenia. NeuroImage, 21, 1563–1575. Nishitani, N. (2003). Dynamics of cognitive processing in the human hippocampus by neuromagnetic and neurochemical assessments. NeuroImage, 20, 561–571. Nishitani, N., Ikeda, A., Nagamine, T., Honda, M., Mikuni, N., Taki, W., et al. (1999). The role of the hippocampus in auditory processing studied by event-related electric potentials and magnetic fields in epilepsy patients before and after temporal lobectomy. Brain, 122, 687–707. Nishitani, N., Nagamine, T., Fujiwara, N., Yazawa, S., & Shibasaki, H. (1998). Cortical-hippocampal auditory processing identified by magnetoencephalography. Journal of Cognitive Neuroscience, 10, 231–247. Nuechterlein, K. H., Dawson, M. E., Gitlin, M. J., Ventura, J., Goldstein, M. J., Snyder, K. S., et al. (1992). Developmental processes in schizophrenic disorders: Longitudinal studies of vulnerability and stress. Schizophrenia Bulletin, 18, 387–425. Okada, Y. C., Kaufman, L., & Williamson, S. J. (1983). The hippocampal formation as a source of the slow endogenous potentials. Electroencephalography and Clinical Neurophysiology, 55, 417–426. Onitsuka, T., Ninomiya, H., Sato, E., Yamamoto, T., & Tashiro, N. (2003). Differential characteristics of the middle latency auditory evoked magnetic responses to interstimulus intervals. Clinical Neurophysiology, 114, 1513–1520. Patterson, J. V., Hetrick, W. P., Boutros, N. N., Jin, Y., Sandman, C., Stern, H., et al. (2008). P50 sensory gating ratios in schizophrenics and controls: A review and data analysis. Psychiatry Research, 158, 226–247. Reite, M., Teale, P., Zimmerman, J., Davis, K., & Whalen, J. (1988a). Source location of a 50 msec latency auditory evoked field component. Electroencephalography and Clinical Neurophysiology, 70, 490–498. Reite, M., Teale, P., Zimmerman, J., Davis, K., Whalen, J., & Edrich, J. (1988b). Source origin of a 50-msec latency auditory evoked field component in young schizophrenic men. Biological Psychiatry, 24, 495–506. Rogers, R. L., Basile, L. F., Papanicolaou, A. C., & Eisenberg, H. M. (1993). Magnetoencephalography reveals two distinct sources associated with late positive evoked potentials during visual oddball task. Cerebral Cortex, 3, 163–169. Rosburg, T., Trautner, P., Ludowig, E., Helmstaedter, C., Bien, C. G., Elger, C. E., & Boutros, N. N. (2008). Sensory gating in epilepsyFeffects of the lateralization of hippocampal sclerosis. Clinical Neurophysiology, 119, 1310–1319. Scheibel, A. B. (1997). The thalamus and neuropsychiatric illness. The Journal of Neuropsychiatry and Clinical Neurosciences, 9, 342–353. Simos, P. G., Basile, L. F., & Papanicolaou, A. C. (1997). Source localization of the N400 response in a sentence-reading paradigm using evoked magnetic fields and magnetic resonance imaging. Brain Research, 762, 29–39. Szeszko, P. R., Goldberg, E., Gunduz-Bruce, H., Ashtari, M., Robinson, D., Malhotra, A. K., & Lencz, T. (2003). Smaller anterior hippocampal formation volume in antipsychotic-naive patients with
T. J. Williams et al. first-episode schizophrenia. American Journal of Psychiatry, 160, 2190–2197. Tesche, C. D. (1996). MEG imaging of neuronal population dynamics in the human thalamus. Electroencephalography and Clinical Neurophysiology Supplement, 47, 81–90. Tesche, C. D. (1997). Non-invasive detection of ongoing neuronal population activity in the normal human hippocampus. Brain Research, 749, 53–60. Tesche, C. D., & Karhu, J. (1999). Interactive processing of sensory input and motor output in the human hippocampus. Journal of Cognitive Neuroscience, 11, 424–436. Tesche, C. D., & Karhu, J. (2000). Theta oscillations index human hippocampal activation during a working memory task. Proceedings of the National Academy of Sciences USA, 97, 919–924. Tesche, C. D., Karhu, J., & Tissari, S. O. (1996). Non-invasive detection of neuronal population activity in human hippocampus. Brain Research. Cognitive Brain Research, 4, 39–47. Thoma, R. J., Hanlon, F. M., Moses, S. N., Edgar, J. C., Huang, M., Weisend, M. P., et al. (2003). Lateralization of auditory sensory gating and neuropsychological dysfunction in schizophrenia. American Journal of Psychiatry, 160, 1595–1605. Thoma, R. J., Hanlon, F. M., Petropoulos, H., Miller, G. A., Moses, S. N., Smith, A., et al. (2008). Schizophrenia diagnosis and anterior hippocampal volume make separate contributions to sensory gating. Psychophysiology, 45, 926–935. Tregellas, J. R., Davalos, D. B., Rojas, D. C., Waldo, M. C., Gibson, L., Wylie, K., et al. (2007). Increased hemodynamic response in the hippocampus, thalamus and prefrontal cortex during abnormal sensory gating in schizophrenia. Schizophrenia Research, 92, 262–272. Tregellas, J. R., Ellis, J., Shatti, S., Du, Y. P., & Rojas, D. C. (2009). Increased hippocampal, thalamic, and prefrontal hemodynamic response to an urban noise stimulus in schizophrenia. American Journal of Psychiatry, 166, 354–360. Turetsky, B. L., Calkins, M. E., Light, G. A., Olincy, A., Radant, A. D., & Swerdlow, N. R. (2007). Neurophysiological endophenotypes of schizophrenia: The viability of selected candidate measures. Schizophrenia Bulletin, 33, 69–94. Velakoulis, D., Pantelis, C., McGorry, P. D., Dudgeon, P., Brewer, W., Cook, M., & Desmond, P. (1999). Hippocampal volume in firstepisode psychoses and chronic schizophrenia: A high-resolution magnetic resonance imaging study. Archives of General Psychiatry, 56, 133–141. Venables, P. H. (1964). Input dysfunction in schizophrenia. In B. A. Maher (Ed), Progress in Experimental Personality Research (Vol. 1, pp. 1–47). New York: Academic Press. Ventura, J., Liberman, R., Green, M., Shaner, A., & Mintz, J. (1998). Training and quality assurance with the Structured Clinical Interview for DSM-IV (SCID-I/P). Psychiatry Research, 79, 163–173. Ventura, J., Lukoff, D., Nuechterlein, K. H., Liberman, R. P., Green, M. F., & Shaner, A. (1993). Brief Psychiatric Rating Scale (BPRS), Expanded Version (4.0): Scales, anchor points, and administration manual. International Journal of Methods in Psychiatric Research, 3, 227–243. Weisser, R., Weisbrod, M., Roehrig, M., Rupp, A., Schroeder, J., & Scherg, M. (2001). Is frontal lobe involved in the generation of auditory evoked P50? NeuroReport, 12, 3303–3307. Wilson, C. L., Babb, T. L., Halgreen, E., Wang, M. L., & Crandall, P. H. (1984). Habituation of human limbic neuronal response to sensory stimulation. Experimental Neurology, 84, 74–97. Yee, C. M., Nuechterlein, K. H., Morris, S. E., & White, P. M. (1998). P50 suppression in recent-onset schizophrenia: Clinical correlates and risperidone effects. Journal of Abnormal Psychology, 107, 691–698. Yee, C. M., Williams, T. J., White, P. M., Nuechterlein, K. H., Wirshing, D. A., & Subotnik, K. L. (2010). Attentional modulation of the P50 suppression deficit in recent-onset and chronic schizophrenia. Journal of Abnormal Psychology, 119, 31–39. Yoshiura, T., Ueno, S., Iramina, K., & Masuda, K. (1995). Source localization of middle latency auditory evoked magnetic fields. Brain Research, 703, 139–144. (Received August 14, 2009; Accepted June 15, 2010)
Psychophysiology, 48 (2011), 479–487. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01120.x
Cerebral cortical dynamics and the quality of motor behavior during social evaluative challenge
JEREMY C. RIETSCHEL,a,b RONALD N. GOODMAN,a,b BRADLEY R. KING,a LI-CHUAN LO,a JOSE L. CONTRERAS-VIDAL,a,b and BRADLEY D. HATFIELDa,b a
Department of Kinesiology, School of Public Health, University of Maryland at College Park, College Park, Maryland, USA Neuroscience and Cognitive Sciences Program, University of Maryland at College Park, College Park, Maryland, USA
b
Abstract To determine the influence of arousal on cerebral cortical dynamics and motor behavior, 58 channels of EEG were recorded in 13 college-age men (n 5 6) and women during an aiming task performed alone and in a social evaluation condition. Moderate arousal, as measured by heart rate, skin conductance, and self-reported mood, was induced during the social evaluation. In accord with the Yerkes–Dodson Hypothesis, which posits optimal performance during moderate arousal, improved performance (i.e., quality of the aiming trajectories) was observed. During social evaluation, changes in electroencephalogram dynamics included decreased coherence between the motor planning (Fz) and right temporal region (T4), increased coherence in the sensorimotor networks subserving the task, and increased local processing (gamma, 30–44 Hz) in the temporal regions. The results imply that moderate arousal promotes specific alterations in cortical dynamics that facilitate motor performance. Descriptors: Arousal, Motor performance, EEG coherence, Social evaluation, Competition
views have been expressed such as the Attention Control Theory, the Catastrophe Model, and the Individual Zone of Optimal Functioning (Eysenck, Derakshan, Santos, & Calvo, 2007; Hanin, 2007; Hardy, 1990; Kerr, 1985; Robazza et al., 2004), recent data provided by Arent and Landers (2003) support the inverted-U relationship between arousal and performance. In their study, multiple levels of cardiovascular arousal were induced in participants, motivated via monetary reward, who performed varying workloads of cycle ergometry (demanding 20%– 90% of heart rate reserve) while challenged with a reaction time task. The participants’ best performances (i.e., fastest reaction times) were observed during the moderately arousing workload. From a cognitive neuroscience perspective, other investigators have advocated the importance of cerebral cortical processes in their attempts to explain superior cognitive–motor performance (Babiloni et al., 2008; Del Percio et al., 2009; Hatfield & Kerick, 2007). Specifically, Hatfield and Hillman (2001) suggested networking efficiency as an important contributor to superior motor performance in the form of refinements that ‘‘streamline’’ cortico-cortical communication between association and motor regions of the cerebral cortex. The refinement (i.e., reduced networking) is also postulated to promote a reduction in the variability of motor planning processes, which is ultimately reflected in efficient limb coordination and more consistent (i.e., less variable) motor performance. On the other hand, excessive networking may decrease performance by way of introducing nonessential inputs of ‘‘neuromotor noise’’ to the motor planning regions, introducing variability into the production of skilled motor behavior (Deeny, Haufler, Saffer, & Hatfield, 2009). Within the framework of the inverted-U relationship,
Skilled motor behavior results from continued practice and dedication (Ericsson & Smith, 1991), but despite an overall improvement in the level of performance as a result of learning, significant fluctuations in motor performance can occur. For example, the social environment affects human performance via changes in arousal (Zajonc, 1965). In this regard, competition and social evaluation can exert marked influence on the quality of performance (Hanin, 2007; Robazza, Pellizzari, & Hanin, 2004; Zaichkowsky & Baltzell, 2001). However, the manner by which such social influence affects motor performance is unclear. One possibility that merits investigation is the impact of arousal on central motor processes (i.e., activity in premotor and motor cortex) as these areas essentially underlie the quality of the peripheral motor processes (i.e., motor unit activity and limb movement). Accordingly, the present study investigated the influence of social evaluation (via competition) induced arousal on cerebral cortical dynamics and the expression of skilled motor performance. Yerkes and Dodson (1908) proposed an inverted-U relationship between arousal and performance that describes an optimal level of arousal as a prerequisite for high-quality performance, with lower and higher levels of arousal, relative to the optimum, resulting in a reduction of performance. Although additional
Ronald N. Goodman is now with Maryland Exercise and Robotics Center of Excellence, Baltimore VA Medical Center. The first two authors contributed equally to the completion of this work. We would like to thank Mark Saffer and Melissa Pangelinan for technical assistance. Address correspondence to: Bradley D. Hatfield, Department of Kinesiology, SPH Building, University of Maryland, College Park, MD 20742-2611. E-mail:
[email protected] 479
480 both high and low levels of arousal (and the subsequent poor performance) may be associated with nonessential networking, whereas optimal arousal is associated with refined/efficient networking between motor and nonmotor regions, resulting in consistent motor performance. Electrophysiological recordings of brain activity provide a high-resolution and objective measure of cerebral cortical dynamics during motor performance. More specifically, Hatfield, Haufler, Hung, and Spalding (2004) reviewed a number of studies that collectively revealed relatively higher left temporal (T3) alpha power in expert marksmen compared to novices during the aiming period just prior to trigger pull (Hatfield, Landers, & Ray, 1984; Hatfield, Landers, Ray, & Daniels, 1982; Haufler, Spalding, Santa Maria, & Hatfield, 2000). Electroencephalogram (EEG) alpha power (8–13 Hz) is reputedly known as the ‘‘idling frequency’’ of the human brain, and elevations in alpha power are commonly used to infer reduced activation (Pfurtscheller, Stancak, & Neuper, 1996), particularly in regards to visuomotor tasks (Hatfield et al., 2004). Other interpretations regarding the functional significance of alpha band power in cortical processing have been offered by von Stein and Sarnthein (2000), in which alpha power is inversely related to task engagement and Pfurtscheller and Lopes da Silva (1999), in which low alpha frequencies (8–10 Hz) index general cortical arousal and high alpha frequencies (10–13 Hz) index task-relevant cortical arousal. Additionally, alpha power is inversely related to gamma power (30–44 Hz; Oakes et al., 2004), which is positively related to local processing (von Stein & Sarnthein). Employment of EEG in training studies, in which participants practiced a visuomotor task over several sessions, revealed a marked increase in both T3 and T4 alpha power as a result of practice during the preparatory aiming period of target shooting (Kerick, Douglass, & Hatfield, 2004). Because the left temporal lobe has been robustly linked to language and analytical processes via lesion and neuroimaging studies and the right temporal lobe has been associated with visuospatial integrative processes (Springer & Deutsch, 1998), these observations support the position that heightened levels of visuomotor performance result from practice-induced decreases in any nonessential verbal-analytic and spatial-integrative processing. In addition to spectral measures of power to determine activation, refinement of neural networking may be inferred through EEG measures such as coherence. Coherence is a statistical measure of the degree of linear correlation between the power spectral densities of two separate electrodes. Thus, coherence values are an index of the correlation between the power estimates for a specific bandwidth (e.g., alpha) derived from the time series recorded from two electrode sites (e.g., T3-Fz or T4Fz). High coherence implies functional communication between these areas, whereas low coherence posits relative independence. In a classic study, Busk and Galbraith (1975) supported the notion of refinement of cortical networking during motor skill acquisition, as they observed significant decreases in EEG coherence between cortical regions with known neuroanatomical connections. More recently, Deeny et al. (2009) observed lower EEG coherence across the scalp topography during the aiming period of a visuomotor task in expert marksmen relative to novices. Importantly, Deeny et al. (2009) also observed a positive relationship between EEG coherence and the variability of the aiming trajectories in the experts. That is, lower corticocortical communication between a number of recording sites and the motor planning region (Fz) was associated with less vari-
J.C. Rietschel et al. ability of the aiming trajectory (i.e., higher quality of the aiming pattern), which may be indicative of a reduction in ‘‘neuro-motor noise.’’ In addition, Deeny, Hillman, Janelle, and Hatfield (2003) classified marksmen with comparable shooting experience into two groups, those that performed at a high level in competition and those that performed at a lower level in competition, and observed group differences in EEG coherence. The superior performers exhibited lower coherence between the left temporal and motor planning regions during shooting, whereas all other electrode pairings across the scalp topography with the motor planning region were undifferentiated between the groups. Again, these findings support the position that refinement of specific cortical networks underlies superior performance. Deeny et al. (2003) implicated networking specifically between the left temporal and motor planning regions as underlying superior performance of a visuomotor aiming task. Whereas they investigated group differences based on competitive history, the present study examined the impact of arousal induced by social evaluation on the cortical dynamics associated with motor planning and on motor behavior relative to a control condition. Specifically, the influence of arousal on distributed cortical networks during the motor planning period (2 s prior to movement onset) of a precision visuomotor aiming task was investigated to provide a mechanism to support the Yerkes–Dodson Hypothesis. In the event that participants exhibited a moderate increase in arousal during social evaluation, we predicted lower EEG coherence between the left temporal and motor planning regions of the cerebral cortex. Such a decrease in coherence, relative to that observed when performing alone, would be accompanied by superior performance. Conversely, in the event of excessive social evaluation-induced arousal, we predicted higher coherence between the left temporal and motor planning regions and degraded motor performance relative to performing alone. Even though motor planning and execution involve widely distributed cortical and subcortical networks, employment of Fz as representative of activity in cortical regions related to motor planning is supported by many EEG-based cognitive–motor investigations of motor learning and performance (Deeny et al., 2003, 2009; Hatfield & Hillman, 2001; Hatfield et al., 2004). Finally, in light of the studies mentioned above, coherence between the motor planning region (Fz) and the distributed scalp topography (i.e., frontal, central, right temporal, parietal, and occipital) was examined in an exploratory manner.
Methods Participants A total of 18 volunteers were recruited from undergraduate classes at a large university in the eastern United States. The data from 5 individuals were excluded either due to impedances that were not below threshold or because they did not complete testing. The 13 remaining participants consisted of 7 women and 6 men (mean age 5 23.78 years, SD 5 4.42). Only right-handed participants, as determined by the Edinburgh Handedness Inventory (EHI), were included in the study and were ipsilateral eye dominant. No participant reported any exclusionary health condition (e.g., neurological disorders, concussion, psychotropic medication, etc.) as assessed by the Health Status Questionnaire (HSQ). In addition, all participants reported refraining from alcohol, caffeine, and nicotine for at least 24 h and from food or
Arousal, performance, and coherence large quantities of water (41 quart) for at least 75 min before psychophysiological testing began. All participants provided informed consent. The EHI, HSQ, and informed consent form, including the dietary restriction advisory were administered 1 day prior to EEG testing. Task Participants completed a visuomotor pointing task, which consisted of center-out movements, from a centrally located start circle to one of four peripheral, circular targets (Contreras-Vidal & Kerick, 2004). Movements were made with an indicator pen compatible with a 12-in. ! 12-in. digitized drawing tablet (Intuos Graphics Tablet, WACOM Co., model GD-1212-R) which tracked X-Y movement coordinates during task performance at 100 Hz. Participants were seated in a comfortable chair without arm rests, approximately 20 in. away at eyelevel from a 15-in. Gateway monitor (model FPD1520). Their right hand was occluded from view via an 18-in. ! 18-in. upright board placed adjacent to the participant’s axillary fossa (see Figure 1A). A chin rest was positioned to minimize head movement artifact. Five circles, 1 cm in diameter, were presented to the participants via the Gateway monitor (see Figure 1B for pictorial description of the layout). Initially, subjects viewed only the center, red (home) circle; then, as participants moved the indicator pen into the home circle, the four peripheral blue circles (targets) appeared. Participants were instructed to keep the pen within the center circle for a minimum of 2 s; otherwise, the targets would dis-
481 appear and the trial must be repeated. If the minimum time constraint was met (42 s), then participants moved to a self-selected peripheral target. Once participants entered a peripheral target, all target circles disappeared, marking the end of that particular trial. Participants then returned to the home circle in order to begin the next trial. Visual feedback of pen movement trajectories was provided via real time tracings on the monitor. Task consisted of two trial types: visual consistent (congruent), where feedback was veridical with pen trajectory, and visually distorted (incongruent), where visual feedback was rotated 601 clockwise, thereby provoking a visual-proprioceptive mismatch (see Figure 1B). The incongruent trials were used as novel stimuli, eliminating initial biases in skill level, and caused subjects to internalize a new visuomotor map. Subsequently, these incongruent trials were the only trials used during the second day of testing. This design ensured that all participants came to the task with the same degree of task experience, thus eliminating any confounds due to exposure. Psychophysiological Recordings EEG acquisition. Scalp electroencephalographic data were collected using tin electrodes housed within a stretchable lycra cap (Electrode-Cap Instrumentation, Inc.). Data were recorded from 58 monopolar sites, labeled in accordance with the modified International 10-20 system (Jasper, 1958). At all sites of interest, impedances were maintained below 10 kO, and the EEG was referenced to linked earlobes and a common ground (FPz). All channels were amplified 1000 times using Neuroscan Synamps 1, linked to Neuroscan 4.3 acquisition/edit software on a Gateway Pentium computer running Windows XP operating system. Bandpass filters were set at 1–100 Hz with a 60-Hz notch filter, and the sampling rate was set to 1000 Hz. Autonomic acquisition. All autonomic measures were recorded from the left hand using a Thought Technology (TT) Procomp Infiniti system (encoder model SA7500). For autonomic measures of heart rate (HR) and skin conductance (SC), HR was sampled at 2048 Hz through a blood volume pulse (BVP) sensor (model SA9308M). The BVP sensor placement was on the second digit of the index finger. SC (model SA9309M) was sampled at 256 Hz, and sensors were attached to the second digit of the second and fourth fingers. Event marker. An electronic event marker was transmitted into the Neuroscan Synamps 1 amplifiers and simultaneously into the TTencoder as the hand-held pen left the home circle (i.e., at movement onset). The event marker served the purpose of time-locking task events with EEG and autonomic recordings.
Figure 1. A: Visual depiction of experimental setup: Participants’ writing hand was occluded while they received visual feedback about their movements via the monitor. B: A pictorial description of the target layout: The center circle was the start location and the peripheral circles were the targets. The arrows represent the distortion between the pen trajectory and the visual feedback. C: The shaded area represents deviation (RMSE) from the optimal trajectory.
Procedures The experiment entailed two days of testing. The purpose of Day 1 was to familiarize participants with the psychophysiological recording procedures and to achieve a stable state of task performance such that the effect of social evaluation on quality of motor behavior could be assessed on the second day of testing. Specifically, on the initial testing day, participants completed the informed consent form, EHI, and HSQ. Although psychological signals were not recorded on Day 1, subjects were fit with EEG caps and autonomic sensors to limit the novelty effect during Day 2 data acquisition. The participants then completed the visuomotor task. Instructions were provided that emphasized moving as quickly and accurately as possible to the peripherally
482 located targets. The experimental session consisted of 400 trials of task training (40 trials congruent, 360 incongruent). Previous research has demonstrated that 360 incongruent trials are sufficient to obtain performance stability (Contreras-Vidal & Kerick, 2004; Krakauer, Pine, Ghilardi, & Ghez, 2000). On the second day, participants completed both a performance alone and a social evaluation condition while the aforementioned psychophysiological measures were recorded. Participants were fitted with the EEG cap. Omni-prep conducting gel was applied to all 58 sites via a blunt tipped medical syringe. Additionally HR and SC sensors were attached as described above. Subjects were given the same task instructions as on the first day. When impedances reached the specified levels, participants entered a sound-proofed room and began the visuomotor task under one of the conditions, and the order was counterbalanced across the subjects. Each condition consisted of 60 incongruent trials. Regardless of condition, at the 30th trial subjects completed a visual analog scale (VAS) assessing stress, relaxation, confidence, and competitiveness. Subjects were allowed a 10-min rest period between the two conditions. A technician was present to monitor equipment and administer inventories in both conditions; however, the technician did not interact with the participant otherwise. The primary focus of the current study is on this second day of testing. Social evaluation condition. Social evaluation was induced through a competition, and throughout this condition two confederates stood immediately behind the participant. One confederate held a clipboard and both recorded and verbalized false starts (exiting center circle before 2 s had elapsed). Additionally, two video cameras were aimed at the participant’s face, one directly in front and the other within their peripheral visual field. Finally, the participants were told the film would subsequently be analyzed by others. The competition included a monetary reward to the participant who performed the quickest, most accurate trajectories with the least number of false starts. Participants were reminded at the start of the competition that a cash prize ($150) would be awarded to the ‘‘winner.’’ Performance-alone condition. This condition was identical to the social evaluation condition except that there was no socialevaluation induction (i.e., no competition, confederates, cameras, or monetary reward). However, the participants were reminded they should be as quick and accurate as possible while performing the visuomotor task. Data Processing Arousal manipulation validation. Average HR (in beats per minute [bpm]) within each condition was based on the successive interbeat intervals. To reveal the temporal dynamic of SC, the entire time series per condition was divided into three equal segments, and the mean for each segment was generated. Each of the VAS questions was scored by measuring the distance of the subjects’ response on a 100-mm line anchored by adjectives consistent with the dimensions listed above. EEG signal processing. EEG data reduction was performed using Neuroscan 4.3 edit/acquire software on electrodes of interest. Data were visually inspected, artifact reduced, and band passed filtered from 1 to 50 Hz with a 24 dB/octave rolloff. The 2-s epoch prior to movement onset for each trial was visually inspected, baseline corrected, and spline fit (2048 data points).
J.C. Rietschel et al. Coherence was calculated between Fz (motor planning region) and 10 (topographically distributed) electrodes (F3, F4, C3, C4, T3, T4, P3, P4, O1, and O2) in 0.5-Hz bins and averaged across the frequency bandwidths (alpha 8–13 Hz, beta 13–30 Hz), which are postulated to reflect midrange cortical distances (von Stein & Sarnthein, 2000). All coherence values were subjected to a Fisher z-transformation prior to statistical analysis to ensure normal distribution. Finally, a spectral average was computed by averaging the 0.5-Hz bins in the low-alpha (8–10 Hz), high-alpha (10–13 Hz), and gamma (30–44 Hz) bands at F3, F4, C3, C4, T3, T4, P3, P4, O1, and O2. These frequencies provide an index of general cortical arousal, task-relevant cortical arousal, and local processing respectively (Pfurtscheller & Lopes da Silva, 1999; von Stein & Sarnthein, 2000). These averages were then natural log transformed prior to statistical analyses. Kinematics of the aiming trajectory. Kinematic data processing was completed using MATLAB (version 6.1). All Cartesian position data was dual passed filtered (eighth order Butterworth, 10-Hz cutoff). Movement onset and offset were determined using an algorithm employed in previous research (Contreras-Vidal, Bo, Boudreau, & Clark, 2005). Subsequently, each trial was visually inspected to ensure accurate movement onset and offset; any trial containing artifacts was excluded from further analysis. The resultant trajectory segments (path between movement onset and offset) were compared to the optimal trajectory for each given target (shortest line segment between home and target circles) to calculate root mean square error (RMSE; see Figure 1C). RMSE is an index of the deviation from the optimal trajectory, thus providing an index of the quality of performance (Kagerer, Contreras-Vidal, & Stelmach, 1997). Statistical Analysis To ensure that performance during the visuomotor task was stable during the second day of testing, a one-way, repeated measures analysis of variance (ANOVA) on RMSE was conducted on five blocks of trials: (1) baseline (trials 21–40), (2) Day 1 ‘‘early’’ incongruent (trials 41–60), (3) Day 1 ‘‘middle’’ incongruent (trials 221–240), (4) Day 1 ‘‘late’’ incongruent (trials 380– 399), and (5) Day 2 ‘‘early’’ incongruent (first 20 trials of second testing session). The early, middle, and late incongruent blocks were selected to assess performance improvement during the training session. The first 20 trials of the second day of testing, regardless of condition, were included in the design to assess stability in performance across the two days. The comparison of the blocks of trials yielded evidence of performance stability as a result of practice. In this regard, a significant effect of block was found for the repeated measures ANOVA across the two days of testing, F(4,48) 5 26.99, po.0001. Tukey’s HSD post hoc analysis revealed that RMSE was significantly larger during the early Day 1 incongruent trial (Block 2) compared to all other blocks. The other four blocks of trials were undifferentiated (see Figure 2). Importantly, this finding suggests that the performance of the participants was stable, such that the effect of social evaluation could be unambiguously assessed. HR and the scores derived from all psychological inventories (VAS and STAI) were subjected to a series of 2 ! 2 (Order ! Condition) ANOVAs with the between-subjects factor of Order (performance alone followed by social evaluation and social evaluation followed by performance alone) and the withinsubjects factor of Condition. SC values were entered into to a 2 ! 2 ! 3 mixed design ANOVA (Order ! Condition ! Aver-
Arousal, performance, and coherence
483 tailed tests of significance. Finally, Tukey’s HSD were employed for all post hoc comparisons. Results
Figure 2. Performance stability. RMSE is shown for three experimental phases: (1) Day 1 baseline, (2) Day 1 incongruent condition, and (3) early Day 2 incongruent condition. Bar plots depict means and standard errors for 20-trial blocks. Day 1 incongruent condition contains three bar plots, representing early (trials 41–60), middle (221–240), and late (380–400) training. To provide a detailed characterization of performance improvement across the training session, the thick curved line is the estimated double-exponential trajectory during the Day 1 incongruent condition and the thin, curved lines depict upper and lower 95% prediction intervals. Only the Day 1 early block differed significantly from all other 20 trial blocks. Notably, the final block of Day 1 was not different from the early Day 2 block, indicative of a stable level of performance. nnpo.01.
aged Segment). Both the spectral averages and coherence values for specified bandwidths during the motor planning (the 2-s epoch prior to movement onset) period were subjected to 2 ! 2 ! 2 ! 5 (Order ! Condition ! Hemisphere ! Region) ANOVAs. RMSE was subjected to a 2 ! 2 (Order ! Condition) ANOVA to assess the impact of social evaluation on performance. Conventional degrees of freedom are reported throughout, and the Greenhouse–Geisser correction is provided when sphericity was violated. Furthermore, the probabilities reported for each effect are based on the corrected degrees of freedom and effect sizes using Cohen’s d are presented. All statistical analyses were conducted using two-
Arousal Manipulation A main effect of condition was detected, F(1,11) 5 12.22, po.01, e 5 .36, d 5 0.52, for HR such that HR was higher during social evaluation (M 5 75.87) relative to that observed during performance alone (M 5 70.99; see Figure 3A). In addition, a Condition ! Time interaction was revealed for SC, F(2,22) 5 5.56, po.05, e 5 .62. Post hoc analysis revealed an effect due to condition, po.01, d 5 0.20, indicating higher skin conductance during the social evaluation condition compared to performance alone for the third segment (see Figure 3B). Statistical analysis of the self-reported psychological states revealed main effects due to condition for the VAS competitive, F(1,11) 5 18.36, po.001, d 5 0.79, and VAS stressed, F(1,11) 5 7.33, po.05, d 5 0.32, variables, which were both elevated during social evaluation (see Figure 3C). The examination of state anxiety (STAI-Y1), VAS confidence, and VAS relaxation revealed no difference between the conditions. Corresponding mean values across the performance alone and social evaluation conditions were 27.94/29.38, 71.63/74.01, and 71.25/ 65.53, respectively. Electrophysiological Results Grand average spectra are provided in Figure 4. The 2 ! 2 ! 2 ! 5 (Order ! Condition ! Hemisphere ! Region) ANOVA applied to gamma frequency (30–44 Hz) revealed a significant Condition ! Region interaction, F(4,8) 5 9.41, po.01, characterized by a bilateral increase in the temporal region during social evaluation, relative to that observed during performance alone, po.01, whereas no such differences were evident in the other regions (see Figure 5A). No significant effects were observed for high- or low-alpha frequencies. A Condition ! Hemisphere ! Region interaction was revealed for beta coherence, F(4,8) 5 3.89, po.05. Post hoc analysis revealed lower coherence between motor planning (Fz) and the right temporal region (T4) during social evaluation, relative to that observed during performance alone, po.05. In addition,
Figure 3. A: Physiological indicators of arousal manipulation: Increase in heart rate (in beats per minute) during social evaluation compared to performance alone. B: Higher SC during social evaluation relative to performance alone. C: The change in self-reported (VAS) psychological variables (competitiveness and stressed). npo.05; nnpo.01; nnnpo.001.
484
J.C. Rietschel et al. mance alone, F(1,11) 5 4.95, po.05, d 5 0.45, with no interaction due to order (see Figure 6A). This result indicates less spatial variability in motor behavior (i.e., higher quality of performance) during social evaluation, regardless of the order of the conditions. Examples of the observed differences between conditions for this variable are provided in Figure 6B. Discussion
Figure 4. Grand averaged spectra across the topography for each condition.
higher beta coherence was observed between motor planning (Fz) and the left central region (C3) during social evaluation as well as bilateral increases between motor planning and the parietal (P3 and P4) and occipital (O1 and O2) regions, po.05 (see Figure 5B). Finally, no significant effects were observed for alpha coherence. Kinematic Results Movement spatial variability measured via RMSE was significantly reduced during social evaluation compared to perfor-
Figure 5. A: Spectral results. Significant gamma (30–44 Hz) synchrony during social evaluation as compared to performance alone in bilateral temporal regions. B: Results of the beta (13–30 Hz) coherence analysis indicating a reduction in coherence between right temporal and the motor-planning regions and increases in coherence between motor planning and left central (as expected in a right-handed motor task), bilateral parietal, and bilateral occipital regions, during social evaluation versus performance alone. npo.05.
Indices of autonomic activity and self-reported psychological states revealed a moderate level of arousal during social evaluation. Consistent with the inverted-U hypothesis and our prediction as stated for moderate arousal, we observed an improvement in motor performance. However, contrary to prediction, the enhanced performance was coupled with a refinement in communication between the right temporal and motor planning regions, as opposed to the left-temporal region. It is plausible that after learning this simple task (center-out pointing vs. precision aiming) that both conditions (characterized by lower arousal vs. moderate arousal) warrant similar levels of engagement from verbal-analytic areas (T3-Fz coherence). The refinement of visuomotor integrative processes (i.e., lower T4-Fz coherence during social evaluation), accompanied by enhanced sensorimotor networking between areas putatively relevant to motor performance (i.e., higher coherence between Fz and, left central, bilateral parietal, and occipital regions), suggests that these changes in networking serve an adaptive purpose in light of the improved kinematics (i.e., decreased RMSE). The pattern of improvement observed during the practice period (Day 1) supports that the participants had mastered this task. Recent investigation of visuomotor distortion of a 601 rotation, as employed herein, revealed a plateau in performance with less than 240 trials (Contreras-Vidal & Kerick, 2004; Krakauer et al., 2000). In the present study such stability was achieved after a comparable number of trials and, importantly, was retained on the second day of testing. The stability of performance achieved on Day 1, coupled with the novelty of the task (controls for experience), allowed for isolation of the influence of social evaluation-induced arousal on cortical dynamics and motor performance on Day 2 while minimizing confounds associated with the effects of motor learning. The experimental manipulation was successful, as indicated by the elevation in the self-reported competitiveness. Physiological and psychological measures of the experimental manipulation indicate a moderate level of arousal. Specifically, selfreported stress was higher, skin conductance was elevated during the social evaluation relative to the performance alone condition, and heart rate increased 6 bpm. This elevation in heart rate was statistically significant, but considerably less than the heart rates recorded during highly arousing conditions such as those examined in Fenz’s (1972) classic work with skydivers. Although the physiological indices of arousal in the present study were elevated, self-reported anxiety, as measured by STAI-Y1, showed no difference between conditions. In tandem, these results suggest an arousal-induced change in performance, as opposed to an anxiety-induced change. Consistent with the Yerkes–Dodson Hypothesis, enhanced performance accompanied this moderate change in arousal. Furthermore, the EEG results support modulation in cortical dynamics as the neurobiological underpinnings of this improvement in performance. The observed increase in cortical activation, as indexed by higher gamma (local processing) in bilateral temporal regions,
Arousal, performance, and coherence
485
Figure 6. A: Decreased variability of the aiming trajectory (RMSE), indicative of improved performance during social evaluation. B: Exemplar of one subject’s performance during the two conditions. npo.05.
is consistent with the somatic and self-report indices of increased arousal during social evaluation. The lack of difference in frontal activation between conditions may be related to the common need for active inhibition (i.e., executive function) of existing motor programs, as both conditions involved a visual distortion. In this regard, Shadmehr and Holcomb (1999) exposed participants to two consecutive dynamic force fields and reported that the ventral prefrontal cortex is involved in inhibiting a previously acquired motor memory. Additionally, we observed no differences in activation in the left-central, parietal, and occipital regions between conditions. This similarity in activation across conditions may be a result of fixed demands on primary sensorimotor processes. Similarly, no changes in coherence were observed between the frontal regions and motor planning across conditions, as the need for executive, inhibitory input was relatively constant during both conditions. However, as described above, we did observe differences in networking between sensorimotor and the motor-planning regions. Changes in coherence during social evaluation were significant only in the beta bandwidth (13–30 Hz). This finding is in accord with the view of von Stein and Sarnthein (2000), as they suggested a ‘‘relation between the size and distance of an (neural) interaction and the frequency of synchronization’’ (p. 308). In other words, the more distant the distributed networks of localized and functionally connected neural populations are, the lower the frequency necessary to coordinate activity between these regions (Nunez & Srinivasan, 2005). The idea that higher frequen-
cies (i.e., gamma) reflect local processing and that beta oscillations facilitate midrange cortico-cortical communication derives from the same conceptual framework. Specifically, von Stein and Sarnthein (2000) observed increased beta coherence over midrange topographic distances during multimodal integration of visually coherent stimuli (pattern vs. nonpattern), whereas the present study investigated multimodal task-relevant versus task-irrelevant processing over comparable interelectrode distances. Busk and Galbraith (1975) observed practice-induced decreases in coherence while subjects learned a motor skill. Additionally, Bell and Fox (1996) tracked EEG coherence across three distinct motor development stages of crawling acquisition (i.e., precrawling, during early crawling, and proficiency) in a longitudinal study of infants. A curvilinear relationship between development and cortico-cortical communication was observed such that coherence between sensorimotor and visual areas was relatively low (precrawling), became higher likely due to excessive synaptic connectivity (during early crawling), and then ultimately was reduced/refined due to synaptic pruning after children achieved proficiency in this skill. Further, in support of these notions of streamlining, Gentili, Bradberry, Hatfield, and Contreras-Vidal (2009), using the same visuomotor rotation task as the present study, observed decreased coherence between multiple brain regions and the motor planning cortices as learning progressed. These studies demonstrate that a refinement in networking between task-relevant regions underlies improvements in motor performance as a function of learning. In the
486
J.C. Rietschel et al.
present study the decreased coherence between T4-Fz and improved performance observed during social evaluation extends this relationship to the arousal-performance domain. However, we observed increased coherence between motor planning and primary sensorimotor regions, left central (righthanded participants), bilateral parietal, and occipital during social evaluation. These increases seem paradoxical given the motor-learning and performance literature described above, which posits an inverse relationship between coherence and performance. We observed both increases and decreases in coherence depending on the regions of interest. In light of the improved performance the elevated coherence between central, parietal, and occipital regions with the midfrontal region may indicate increased sensorimotor input to guide the motor-planning process. Moreover, the reduction in coherence between right temporal and the midfrontal regions may imply an adaptive reduction in nonessential cortico-cortical communication. Collectively, these cortical processes illustrate the complex dynamics underlying the relationship between arousal and the quality of motor performance. Consideration of the mediating influences of task-relevant cortical dynamics may be useful in understanding the arousal
performance relationship. Critchley (2005) describes a network of sympathetically driven noradrenergic projections from brain stem nuclei to thalamic and cortical systems as implicated in mediating beneficial changes in central arousal, subsequently enhancing environmentally salient (sensorimotor processing) stimuli while inhibiting irrelevant stimuli. In light of both the improved performance and the moderate level of arousal observed in the present study, these changes in cortical dynamics likely reflect a fine-tuning of task-relevant networks subsequently decreasing the variability in movement trajectories and enhancing the quality of movement. The previous motor learning literature demonstrated that a reduction in cortical networking is experience dependent (progression of learning) and consistent with notions of efficiency. Importantly, the current study extends this literature to the arousal-performance relationship. Specifically, the combination of increased primary sensorimotor input and refinement of higher order visuospatial input to the motor-planning region resulted in superior performance during advantageous increases in arousal. These findings provide an account of the neural underpinnings that mediate the relationship between arousal and cognitive–motor performance.
REFERENCES Arent, S. M., & Landers, D. M. (2003). Arousal, anxiety, and performance: A reexamination of the inverted-U hypothesis. Research Quarterly for Exercise and Sport, 74, 463–444. Babiloni, C., Del Percio, C., Iacoboni, M., Infarinato, F., Lizio, R., Marzano, N., et al. (2008). Golf putt outcomes are predicted by sensorimotor cerebral EEG rhythms. Journal of Physiology, 586, 131–139. Bell, M. A., & Fox, N. A. (1996). Crawling experience is related to changes in cortical organization during infancy: Evidence from EEG coherence. Developmental Psychobiology, 29, 551–561. Busk, J., & Galbraith, G. C. (1975). EEG correlates of visual-motor practice in man. Electroencephalography and Clinical Neurophysiology, 38, 415–422. Contreras-Vidal, J. L., Bo, J., Boudreau, P., & Clark, J. E. (2005). Development of visuomotor coordination and adaptation in children. Experimental Brain Research, 162, 155–164. Contreras-Vidal, J. L., & Kerick, S. E. (2004). Independent component analysis of dynamic brain responses during visuomotor adaptation. NeuroImage, 21, 936–945. Critchley, H. D. (2005). Neural mechanisms of autonomic affective, and cognitive integration. Journal of Comparative Neurology, 493, 154–166. Deeny, S. P., Haufler, A. J., Saffer, M., & Hatfield, B. D. (2009). EEG coherence during visuomotor performance: A comparison of corticocortical communication in expert and novice marksmen. Journal of Motor Behavior, 41, 106–116. Deeny, S. P., Hillman, C. H., Janelle, C. M., & Hatfield, B. D. (2003). Cortico-cortical communication and superior performance in skilled marksman: An EEG coherence analysis. Journal of Exercise and Sport Psychology, 25, 188–204. Del Percio, C., Babiloni, C., Bertollo, M., Marzano, N., Iacoboni, M., Infarinato, F., et al. (2009). Visuo-attentional and sensorimotor alpha rhythms are related to visuo-motor performance in athletes. Human Brain Mapping, 30, 3527–3540. Ericsson, K. A., & Smith, J. (1991). Prospects and limits of the empirical study of expertise: An introduction. In K. A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 1–39). Cambridge, England: Cambridge University Press. Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007). Anxiety and cognitive performance: Attentional control theory. Emotion, 7, 336–353. Fenz, W. D. (1972). Coping mechanisms and performance under stress. In D. M. Landers (Ed.), Psychology of sport and motor behavior (Vol.
11, pp. 3–24). University Park, PA: Pennsylvania State University Press. Gentili, R. J., Bradberry, T. J., Hatfield, B. D., & Contreras-Vidal, J. L. (2009, September). Brain biomarkers of motor adaptation using phase synchronization. Paper presented at the annual meeting of International IEEE EMBS Conference, Minneapolis, MN. Hanin, Y. L. (2007). Emotions in sport: Current issues and perspectives. In G. Tenenbaum & R. C. Eklund (Eds.), Handbook of Sport Psychology (3rd ed., pp. 84–107). Hoboken, NJ: John Wiley & Sons, Inc. Hardy, L. (1990). A catastrophe model of performance in sport. In J. G. Jones & L. Hardy (Eds.), Stress and performance in sport (pp. 81–106). Chichester, UK: John Wiley & Sons, Inc. Hatfield, B. D., Haufler, A. J., Hung, T.-M., & Spalding, T. W. (2004). Electroencephalographic studies of skilled psychomotor performance. Journal of Clinical Neurophysiology, 21, 1–13. Hatfield, B. D., & Hillman, C. H. (2001). The psychophysiology of sport. In R. N. Singer, H. A. Hausenblaus, & C. M. Janelle (Eds.), Handbook of research on sport psychology (2nd ed., pp. 362–386). New York: John Wiley. Hatfield, B. D., & Kerick, S. E. (2007). The psychology of superior sport performance: A cognitive and affective neuroscience perspective. In G. Tenenbaum & R. C. Eklund (Eds.), Handbook of Sport Psychology (3rd ed., pp. 84–107). Hoboken, NJ: John Wiley & Sons, Inc. Hatfield, B. D., Landers, D. M., & Ray, W. J. (1984). Cognitive processes during a self-paced motor performance: An electroencephalographic profile of skilled marksman. Journal of Sport Psychology, 6, 42–59. Hatfield, B. D., Landers, D. M., Ray, W. J., & Daniels, F. S. (1982). An electroencephalographic study of elite rifle shooters. American Marksmen, 7, 6–8. Haufler, A. J., Spalding, T. S., Santa Maria, D. L., & Hatfield, B. D. (2000). Neuro-cognitive activity during a self paced visual-spatial task: Comparative EEG profiles in marksmen and novice shooters. Biological Psychology, 53, 131–160. Jasper, H. H. (1958). The ten-twenty electrode system of the international system federation. Electroencephalography and Clinical Neurophysiology, 10, 371–375. Kagerer, F. A., Contreras-Vidal, J. L., & Stelmach, G. E. (1997). Adaptation to gradual as compared with sudden visuo-motor distortions. Experimental Brain Research, 115, 557–561. Kerick, S. E., Douglass, L., & Hatfield, B. D. (2004). Cerebral cortical adaptations associated with visuomotor practice. Medicine & Science in Sport & Exercise, 36, 118–129.
Arousal, performance, and coherence Kerr, J. H. (1985). The experience of arousal: A new basis for studying arousal effects in sports. Journal of Sport Sciences, 3, 169–179. Krakauer, J. W., Pine, Z. M., Ghilardi, M., & Ghez, C. (2000). Learning of visuomotor transformations for vectorial planning of reaching trajectories. Journal of Neuroscience, 20, 8916–8924. Nunez, P. L., & Srinivasan, N. (2005). Electric fields of the brain. New York: Oxford University Press. Oakes, T. R., Pizzagalli, D. A., Hendrick, A. M., Horras, K. A., Larson, C. L., Abercrombie, H. C., et al. (2004). Functional coupling of simultaneous electrical and metabolic activity in the human brain. Human Brain Mapping, 21, 257–270. Pfurtscheller, G., & Lopes da Silva, F. (1999). Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clinical Neurophysiology, 110, 1842–1857. Pfurtscheller, G., Stancak, A. Jr., & Neuper, C. (1996). Event-related synchronization (ERS) in the alpha bandFAn electrophysiological correlate of cortical idling: A review. International Journal of Psychophysiology, 24, 39–46. Robazza, C., Pellizzari, M., & Hanin, Y. L. (2004). Emotion self-regulation and athletic performance: An application of the IZOF model. Psychology of Sport and Exercise, 5, 370–404.
487 Shadmehr, R., & Holcomb, H. H. (1999). Inhibitory control of competing motor memories. Experimental Brain Research, 126, 235–251. Springer, S. P., & Deutsch, G. (1998). Left brain, right brain: Perspective from cognitive neuroscience (5th ed.). New York: Freeman. von Stein, A., & Sarnthein, J. (2000). Different frequencies for different scales of cortical integration: From local gamma to long range alpha/ theta synchronization. International Journal of Psychophysiology, 38, 301–313. Yerkes, R. M., & Dodson, J. D. (1908). The relation of strength of stimulus to rapidity of habit-formation. Journal of Comparative Neurology and Psychology, 118, 459–482. Zaichkowsky, L. D., & Baltzell, A. (2001). Arousal and performance: The psychophysiology of sport. In R. N. Singer, H. A. Hausenblaus, & C. M. Janelle (Eds.), Handbook of research on sport psychology (2nd ed., pp. 319–339). New York: John Wiley. Zajonc, R. B. (1965). Social facilitation. Science, 149, 269–274.
(Received October 13, 2009; Accepted June 15, 2010)
Psychophysiology, 48 (2011), 488–494. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01086.x
Efficient and cost-effective estimation of the influence of respiratory variables on respiratory sinus arrhythmia
VICTORIA B. EGIZIO,a MICHAEL EDDY,b MATTHEW ROBINSON,b and J. RICHARD JENNINGSb a
Department of Psychology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
b
Abstract Researchers are interested in respiratory sinus arrhythmia (RSA) as an index of cardiac vagal activity. Yet, debate exists about how to account for respiratory influences on quantitative indices of RSA. T. Ritz, M. Thons, and B. Dahme (2001) developed a within-individual correction procedure by which the effects of respiration on RSA may be estimated using regression models. We replicated their procedure substituting a spectral high-frequency measure of RSA for a time-domain statistic and a respiratory belt’s relative measure of tidal volume for the direct assessment provided by a pneumotachograph. The standardized slopes from the respiratory belt and pneumotachography-derived regression equations (estimated across a 6-min paced breathing protocol) were positively correlated (r 5 0.93, po.001); correlations were similar across 2- and 4-min time courses parsed from the 6-min protocol. Our results offer methodological alternatives to the research community. Descriptors: Individual differences, Normal volunteers, Heart rate
the longest and the shortest heart rates within a given respiratory cycle. Although these methods of RSA quantification are generally accepted, questions remain regarding the optimal manner in which to account for respiratory influences on such indices of RSA. The relationship of respiratory rate and depth to RSA has been a topic of substantial discussion (e.g., Denver, Reed, & Porges, 2007; Grossman & Taylor, 2007; Ritz & Dahme, 2006). Of particular concern, some have advocated that measures of RSA be ‘‘corrected’’ or ‘‘unconfounded’’ from respiratory effects (e.g., Grossman & Taylor, 2007; Ritz, 2009). Respiratory phase and duration (rate), though, essentially defines RSA. RSA is the result of cardiac–respiratory interaction. An understanding of this interaction permits the investigator to define clearly a measurement strategy that defines the dependent measure(s) appropriate to the question being posed. The nature of the cardiac– respiratory interaction is relatively well understood and has been described elsewhere in some detail (Berntson, Cacioppo, & Quigley, 1993; Berntson et al., 1997; Eckberg, 2003; Grossman & Kollai, 1993; Grossman & Taylor, 2007). Figure 1 summarizes the understanding and is closely related to the description presented earlier by Berntson et al. (1993). In the absence of any neural input, an intrinsic heart rate is present that represents a characteristic of that person’s cardiac sinus node pacemaker (e.g., Noble, 1979). Heart rate level is typically set by a combination of vagal inhibition and sympathetic activation acting on the sinus node pacemaker. Vagal control of heart rate is accepted as predominant, particularly at rest, although some, relatively minor sympathetic nervous system control is also present.
Respiratory sinus arrhythmia (RSA) is the rhythmic fluctuation in heart rate at the respiratory frequency. The psychophysiological research community has primarily become interested in RSA as an index of cardiac vagal activity. Commonly used quantitative RSA measurement techniques derived from electrocardiogram recordings have been previously described in detail (Berntson et al., 1997; Camm et al., 1996). In this report, we focus on a frequency-domain measure of RSA, high frequency spectral power. High frequency spectral power, a common measure of heart rate variability, can be derived using spectral methods such as fast Fourier transform or autoregressive modeling. These methods decompose the total variation in a heart rate or interbeat interval time series into frequency components. The high frequency band is considered to be between 0.15 and 0.4 Hz in adult participants; because younger participants breathe at faster rates, the frequency range may need to be reconsidered for such samples (Berntson et al., 1997; Camm et al., 1996). An alternative measure is the peak-to-valley statistic, a time-domain measure of RSA. This statistic produces a breath-by-breath index of heart rate fluctuations that reflects the difference between
Research support was provided by the Pittsburgh Mind-Body Center (National Institutes of Health Grant HL 076852/076858) and by National Institutes of Health Grant T32HL007560. We thank Charles Atwood, M.D., and the University of Pittsburgh Department of Pulmonary Medicine for their assistance and loan of the pneumotachograph. Address correspondence to: Victoria Egizio, Western Psychiatric Institute and Clinic, 3811 O’Hara St., Room E1329, Pittsburgh, PA 15213, USA. E-mail:
[email protected] 488
Respiratory variables and RSA
489
Figure 1. Diagrammatic view of likely sources of vagal influence on the heart, omitting central influences and emphasizing areas related to respiratory function.
Variation over time in heart rate at respiratory frequencies is attributed to a vagal gating phenomenon, located in medullary nuclei known to channel vagal efference, that is, the nucleus ambiguous and possibly the dorsal motor nucleus (Gilbey, Jordan, Richter, & Spyer, 1984). Vagal gating results in a modulation of heart rate such that heart rate decelerates as vagal firing occurs during expiration, whereas, during inspiration, gating of such vagal activation results in cardiac acceleration. Importantly, as illustrated in Figure 1, the gating typically modulates vagal input to the sinus node pacemaker of the heart rather than the gate completely inhibiting vagal influence during inspiration. Thus, if the aim of the investigator is to assess vagal influences on the heart, then two components are presentFone component that is modulated by respiration and another not altered by this modulation (see, e.g., Grossman, Karemaker, & Weiling, 1991, for experimental manipulation of these components). Finally, respiratory rate and depth are illustrated in Figure 1 as providing afferent information that can moderate the cyclic output from neural centers and thus contribute variance to the modulatory component of heart rate. Despite some disagreement (Denver et al., 2007), respiratory frequency and depth typically vary the degree of vagal modulation (for review, see Eckberg, 2003). Fast, shallow breathing is related to less vagal modulation than slow, deep breathing. This variation may be, in part, intrinsic to medullary cardiac–respiratory control and, in part, due to peripheral factors, for example, slower respiratory rate permits greater time for vagal/ cholinergic effects on the sinus node to be expressed (cf. Cooper, Parkes, & Clutton-Brock, 2003; Strauss-Blasche et al., 2000). Measures of RSA, for example, power in the respiratory band or peak-to-trough measures, assess the amplitude of the modulationFsimilar to the variance due to respiratory gating. As measures of modulation, they do not measure the nonmodulated component of vagal influence. The modulatory portion is, however, subject to moderation by respiratory rate and depth. This moderation can be assessed and is the focus of the current article. Degree of moderation becomes a separate measure of modulations, as RSA typically cannot be completely predicted by respiratory rate and depth (Ritz, Thons, & Dahme, 2001). The
measurement of the nonmodulated component is not straightforward, but use of average heart rate less a measure of the modulated component has been suggested (Grossman et al., 1991; Ritz et al., 2001). Interpretation is somewhat clouded, however, by the known possibility of sympathetic influences on average heart rate as well as individual differences in intrinsic heart rate. Note that both the modulated and nonmodulated components represent vagal influences on the heart. Given this, ‘‘correcting’’ or ‘‘unconfounding’’ does not discard nonvagal influences. Depending on the aims of an investigator, the modulated, nonmodulated, or combined vagal effects may be of interest. Acceptance of the use of respiratory measures in conjunction with heart rate variability measures has been slowed both by arguments over unconfounding as well as the cost in time and money of adding respiratory measurement. The current experiment was directed at testing a simple, inexpensive respiratory assessment that would ease the latter concern. We view the technique as permitting a reasonably easy means of assessing the modulatory components of vagal control and, as such, under some circumstances, to further permit assessment of nonmodulatory and combined vagal control. Note that we have avoided the physiologically outdated term ‘‘vagal tone.’’ It remains unclear whether ‘‘vagal tone’’ refers specifically to vagal effects on the sinoatrial node of the heart and, even if so, whether it refers to the modulated, nonmodulated, or combined vagal influence on the heart. This vagueness of the ‘‘vagal tone’’ terminology seems to contribute to the confusion over respiratory–cardiac interactions. Given this, we prefer to avoid the term (and suggest that others do so as well). Based on earlier work in the field (e.g., Grossman & Kollai, 1993), Ritz et al. (2001) have proposed a within-individual procedure by which respiratory rate and depth’s influence on RSA may be estimated. The procedure calls for a baseline paced breathing protocol in which measures of respiratory rate, tidal volume, and an electrocardiogram-derived index of RSA are obtained. From these measures, the index of RSA can be apportioned for respiratory influences by regressing RSA divided by tidal volume on a measure of respiratory cycle length (Ritz &
490 Dahme, 2006; Ritz et al., 2001). Following this baseline period, participants breathe freely during the experimental session. The predicted value of the index of RSA is obtained for the experimental data using the regression equation derived during the paced breathing period. The predicted value from the equation provides a measure of the modulatory component that is not linearly related to breathing frequency and depth; the predicted less observed measure assesses variance due to breathing frequency and depth. This method has been tested by Ritz et al. (2001). Their protocol employed the peak-to-valley statistic as an index of RSA and utilized pneumotachography to measure respiratory rate and tidal volume. The current study aimed to determine whether this correction procedure could be approximated using high frequency–heart rate variability (HF-HRV) to assess RSA and a respiratory belt to assess thoracic contraction and expansion. HF-HRV was substituted because spectral measures of heart rate variability are used widely. The equivalency of the peak-to-valley and HF-HRV methods of RSA analysis has been discussed elsewhere (e.g., Grossman, van Beek, & Weintjes, 1990). Indeed, the multiple time- and frequency-domain measurements of RSA, of which the peak-to-valley and HF-HRV methods are only two, are highly correlated (between- and within-subjects correlations range from 0.58 to 0.98; e.g., Allen, Chambers, & Towers, 2007). Similarly, although the initial purchase cost and participant burden (i.e., breathing through a tube while nasal passages are blocked) associated with pneumotachography is somewhat prohibitive, the respiratory belt offers cost-effective, minimally invasive assessment of respiration. In a typical measurement using respiratory belts, a combination of two respiratory belts spaced across the thorax is used to assess respiratory rate. Such two-belt systems can provide measures of tidal volume when precalibrated using a device that measures tidal volume directly (e.g., a pneumotachograph; Konno & Mead, 1967; Sackner, 1996). Instead, we tested the ability of a simply designed, single uncalibrated respiratory belt to provide a relative, not quantitative, measure of tidal volume comparable to that obtained with pneumotachography. The belt tested was commercially available, relatively inexpensive, and wider than those previously used and possibly responsive to both chest and diaphragmatic breathing. Finally, whereas Ritz and colleagues’ protocol called for 3-min blocks of paced breathing, we examined the accuracy of our methodology during 6-, 4-, and 2-min blocks. These 4- and 2-min blocks represent the first 4 and 2 min of the full 6-min block. If the correction procedure is validated using various timescales, the flexible, relatively inexpensive technique would encourage routine implementation of respiratory assessment during evaluation of heart rate variability.
Method Participants Participants were 20 college students (10 male, 10 female; mean age 5 20.8 years old, SD 5 1.7). Seventy-five percent of the sample was Caucasian. Participants abstained from drinking caffeine as well as from consuming nicotine for at least 2 h prior to their participation in the study. All received monetary compensation for their participation. The University of Pittsburgh Institutional Review Board approved all procedures and all participants provided their informed consent.
V.B. Egizio et al. Materials The respiratory flow curve was measured with a pneumotachograph (47304 A Flow Transducer, Hewlett Packard, Palo Alto, CA). Participants breathed through an oral tube connected to the transducer; participants’ noses were occluded to ensure airflow solely through the tube. The calibration of the pneumotachograph was checked using a standard 3-l syringe approximately twice per week throughout the study. The signal from the pneumotachograph, as well as the respiratory belt and electrocardiogram signal, was digitized at 512 Hz using the software and 12-bit analog-to-digital converter supplied with the respiratory belt (Vernier, Beaverton, OR). The pneumotach data were visualized to check for artifacts after being digitized using a locally developed Matlab program. Artifacts were rare and, when present, did not interfere with the determination of respiratory peak and trough. A Visual BASIC program was developed to calculate tidal volume from the pneumotach/flow-transducer. For each 2-min segment, all flow values 40 were summed to indicate volume, and this volume was divided by the number of breaths to form a tidal volume measure. Respiratory rate was calculated directly from the pneumograph readings and also verified by the rate estimate from the Mindware (Ghana, OH) Heart Rate Variability Scoring Module, which separately analyzed the respiratory signal obtained from the respiratory belt (see below). Respiratory Belt A single respiratory belt (Respiration Monitor Belt, Vernier, Beaverton, OR) measured the changes in pressure associated with the thoracic expansion and contraction accompanying participants’ breathing. The belt was situated such that it covered the area between and including the fifth and eighth ribs, and it was wrapped tightly enough so that only the experimenter’s index and middle fingers could fit underneath. The air bladder of the belt was pumped to approximately 100 kPa before each participant began the protocol. A Visual BASIC program was created to calculate peak-to-trough values from the respiratory belt data. Zero crossings in the pneumotach data (derived as described in the previous section using the Visual BASIC program) were used to identify the peaks and troughs within the pressure data. Peakto-trough differences in the pressure data were summed and divided by the number of breaths to provide an indicator of tidal volume. Electrocardiogram HF-HRV was obtained via electrocardiogram (ECG). A modified lead II electrode placement with three Ag-AgCl electrodes was used. The ECG signal was digitized (12 bit), sampled (again with the Vernier software at 512 Hz), and stored for off-line processing. R wave markers in the ECG signal, indications of each heartbeat, were assessed for artifacts by visual inspection and by an artifact detection algorithm in a commercial software package (Mindware Heart Rate Variability Scoring Module, version 2.16; Mindware Technologies Ltd., Columbus, OH). After the correction of artifacts, 2-min estimates of heart rate and HF-HRV were established. Specifically, for each 2-min period, a time series was created with an appropriately weighted conversion of the interbeat interval (the time in milliseconds between sequential ECG R spikes), creating a value for each 250-ms period within the 2-min period. This time series was linearly detrended, mean-centered, and tapered using a Hamming window. Then spectral power values were calculated (in milliseconds squared per hertz) via fast Fourier transformations, and the
Respiratory variables and RSA
491
Table 1. Means (SD) of Cardiac and Respiratory Parameters During Paced Breathing Parameter TTOT (s) VT Pneumo (ms/l) VT Belt (ms/kPa) HP (ms) HF-HRV (ms) lnHF-HRV (ms) HF-HRV/VT Pneumo (ms/l) lnHF-HRV/VT Pneumo (ms/l) HF-HRV/VT Belt (ms/kPa) lnHF-HRV/VT Belt (ms/kPa)
8 cycles/min
10.5 cycles/min
13 cycles/min
18 cycles/min
7.54 (0.09) 0.59 (0.28) 0.73 (0.47) 901 (115.76) 6140 (3441) 8.50 (0.71) 14,272 (15,735) 19.40 (11.23) 12,008 (9434) 17.97 (13.50)
5.71 (0.04) 0.49 (0.23) 0.67 (0.35) 887 (123.63) 4058 (3678) 7.95 (0.89) 10,793 (11,621) 20.30 (9.37) 8015 (7027) 15.98 (8.16)
4.72 (0.50) 0.44 (0.21) 0.60 (0.34) 893 (117.04) 2622 (2034) 7.53 (0.96) 8012 (7896) 20.66 (8.44) 5880 (4913) 16.31 (8.18)
3.34 (0.03) 0.36 (0.15) 0.56 (0.24) 887 (119.24) 1991 (3045) 6.98 (0.97) 5645 (6729) 22.42 (9.12) 3531 (3444) 15.33 (8.12)
Note: Values for TTOT do not include those for the participant who breathed at an aberrant frequency. Including this participant, the TTOT values for each tone are as follows: 7.29 (1.70), 6.00 (1.31), 5.17 (1.45), 3.51 (0.75).
power values in the 0.15- to 0.40-Hz spectral bandwidth were identified (in milliseconds squared). Both raw and natural logtransformed HF-HRVvalues were examined. Using the ShapiroWilk test, 35% of participants’ had raw HF-HRV values that were skewed, whereas 25% of participants had skewed HF-HRV values after natural log transformation (raw HF-HRV W-statistic range 5 0.69–0.96; lnHF-HRV W-statistic range 5 0.82– 0.97). Procedure Upon arrival, participants were given an overview of the experiment. ECG electrodes and the respiratory belt were fitted to participants, who remained seated for the duration of the protocol. Data from the pneuomotachograph, respiratory belt, and ECG were collected simultaneously throughout the session. To provide a pacing mechanism for respiration, a 300-Hz tone was amplitude modulated at each of the necessary respiration rates. The tones were copied to audio CD for playback via speakers in the subject chamber. The respiration rates were 8, 10.5, 13, and 18 breaths/min for 6 min each. Participants first engaged in a practice trial of 1–2 min to ensure that they breathed accurately in synchrony with the frequency variation of the pacing tone. Following the practice trial, participants breathed for 6 min at each of the rates presented in random order. Breaks of approximately 3 min were given in between each rate. Data Reduction and Statistical Analyses A measure of tidal volume (VT; in liters) was calculated based on the calibration of the pneumotachography signal. Raw and natural log-transformed HF-HRV values were divided by VT (HFHRV/VT Pneumo in milliseconds per liter). Raw and natural logtransformed HF-HRV values were also divided by the output from the respiratory belt (HF-HRV/VT Belt in milliseconds per kilopascal). Average respiratory cycle length in seconds (TTOT ) was also calculated by dividing the respiration rate for 2 min into 120 s. Separate within-individual regression equations were calculated regressing HF-HRV/VT Pneumo on TTOT and regressing HF-HRV/VT Belt on TTOT. These equations included all breathing rates from the paced breathing protocol. The resultant standardized slopes (b values) from the pneuomotachography and respiratory belt regressions were correlated to determine whether both respiration monitoring devices similarly estimated the effects of respiration on HF-HRV. The regression equations and standardized slope correlations were calculated using data from the entire 6 min of each auditory activity from the paced
breathing protocol from the first 4 min of each activity and from the first 2 min of each activity.
Results Effects of Paced Breathing Protocol Participants followed the paced breathing signal correctly, resulting in four distinct respiratory frequencies (e.g., 8 breaths/ min should result in 60/8 5 7.5 cycles/second; experimental mean value 5 7.54 cycles/second; Table 1). One participant did not follow the paced breathing protocol as directed; however, this individual did breathe reliably at twice the prescribed frequency. Thus, this individual was still included in the final sample. HFHRV and lnHF-HRV varied across the respiratory rates such that HF-HRV and lnHF-HRV decreased as respiratory frequency increased, F(1,19) 5 13.19, p 5 .000001, Z2 5 .41, and F(1,19) 5 36.27, p 5 .00, Z2 5 .66, respectively. The heart period (HP) did not vary significantly across the different breathing frequencies, F(1,19) 5 1.17, p 5 .33, Z2 5 .06. VT decreased as the frequency of respiration increased, showing the usual physiologic relationship between tidal volume and HF-HRV for both pneumotachography measures of VT, F(1,19) 5 8.20, p 5 .00013, Z2 5 .30, and respiratory belt measures of VT, F(1,19) 5 4.26, p 5 .0088, Z2 5 .18. All of these values are shown in Table 1. Finally, a correlation matrix including values for raw and lnHFHRV, heart period, pneumotachography and respiratory belt measures of VT, and TTOT is shown in Table 2. Within-Individual Regressions Separate within-individual regressions were calculated regressing each HF-HRV, lnHF-HRV, HF-HRV/VT Pneumo, and HFHRV/VT Belt (dependent variables) on TTOT (the independent variable; see Table 3). Ritz et al. (2001) found that the standardized
Table 2. Correlation Matrix of Key Variables HFHRV HF-HRV lnHF-HRV Heart period TTOT VTPneumo po.05; nnpo.001.
n
lnHFHRV
Heart period
.55
.53 .37
n
n
TTOT
VTPneumo
VTBelt
.20 ! .24 .16
! .39 ! .95nn ! .37 .30
! .08 ! .23 ! .41 .13 .31
492
V.B. Egizio et al.
Table 3. Mean Slopes and Intercepts of the Regressions of HF-HRV, lnHF-HRV, HF-HRV/VT Pneumo, lnHF-HRV/VT Pneumo, HF-HRV/VT Belt, and lnHF-HRV/VT Belt on TTOT
HF-HRV Slope Intercept lnHF-HRV Slope Intercept HF-HRV/VT Pneumo (6 min) Slope Intercept lnHF-HRV/VT Pneumo (6 min) Slope Intercept HF-HRV/VT Pneumo (4 min) Slope Intercept lnHF-HRV/VT Pneumo (4 min) Slope Intercept HF-HRV/VT Pneumo (2 min) Slope Intercept lnHF-HRV/VT Pneumo (2 min) Slope Intercept HF-HRV/VT Belt (6 min) Slope Intercept lnHF-HRV/VT Belt (6 min) Slope Intercept HF-HRV/VT Belt (4 min) Slope Intercept lnHF-HRV/VT Belt (4 min) Slope Intercept HF-HRV/VT Belt (2 min) Slope Intercept lnHF-HRV/VT Belt (2 min) Slope Intercept
Mean
SD
0.69 ! 1478
0.44 5340
0.34 5.96
0.18 1.39
0.59 ! 1667
0.48 11,205
! 0.76 25.05
1.71 10.88
0.61 1702
0.48 10,896
! 0.78 25.31
1.57 10.65
0.66 2955
0.48 11,922
! 0.68 25.60
1.48 10.38
0.63 ! 3354
0.42 9704
0.51 13.68
2.99 15.42
0.62 ! 3406
0.44 10,344
0.42 14.21
2.86 14.42
0.65 ! 3372
0.45 10,373
0.20 15.74
2.58 13.27
were no significant differences between standardized slopes resulting from 6- and 2-min-per-auditory activity estimations of the influence of respiration on HF-HRV as measured by either the pneumotachograph (t 5 0.50, df 5 19, p 5 .62) or respiratory belt (t 5 0.20, df 5 19, p 5 .85). The observed standardized slopes from the pneumotachography and respiratory belt regression equations estimated across 6 min of the paced breathing protocol were positively correlated (r 5 .93, p 5 .00; see Figure 2). The correlations between the pneumotachography and respiratory belt standardized slopes derived from regression equations examining the first 4 or 2 min of pacing were also positively correlated (4-min estimate: r 5 .91, p 5 .00; 2-min estimate: r 5 .95, p 5 .00). When we conducted duplicate analyses using the natural log-transformed HF-HRV values, the relationship between belt and pneumotachograph b values was reduced, as can be seen in Table 3. Additionally, the 6-min correlation of .93 between belt and pneumotachograph was reduced to .36 when the natural log values were employed. Examination of the distributions and summary statistics showed that the variance relative to the mean (the coefficient of variation) was increased substantially with the natural log transformation. This factor rather than any change in seeming outliers appeared to account for the reduction in the correlation. Thus, for this particular belt measure, raw values performed better than natural log-transformed values. It also is important to note that (a) our raw HF-HRV values were not extremely skewed and (b) the skew of those raw values was only improved minimally by natural log transformation.
Discussion
slope for the regression of the peak-to-valley statistic on TTOT was notably larger than the standardized slope that resulted from the regression of the peak-to-valley statistic as corrected for pneumotachography-derived tidal volume on TTOT. Comparing the standardized slopes for the regressions of HF-HRV on TTOT and of HF-HRV/VT Pneumo on TTOT for 6, 4, and 2 min per activity of the paced breathing protocol, we also found that our standardized slope for HF-HRV was larger than the standardized slopes obtained using the respiratory-corrected measures. Moreover, the respiratory belt (in combination with HFHRV) yielded results essentially equivalent to those found using the pneumotachograph. Note that, because of the large difference in measurement units between the peak-to-valley statistic and HF-HRV and between the pneumotachtograph and respiratory belt output, our standardized slope and intercept values cannot be expected to estimate directly the values obtained by Ritz et al. (2001). However, the relative values of tidal volume from the respiratory belt show a relationship with HF-HRV that is similar to that derived by Ritz and colleagues. Finally, there
We found that the standardized slopes of the equations regressing HF-HRV/VT Pneumo on TTOT and HF-HRV/VT Belt on TTOT were highly correlated. This result indicates that the procedure for correcting respiratory influences on quantitative indices of RSA presented by Ritz et al. (2001) can be closely approximated when HF-HRV is substituted for the peak-to-valley statistic and when a respiratory belt replaces a pneumotachography device. Additionally, we demonstrated that our methods can be used effectively over 6-, 4-, and 2-min time courses. As such, the correction procedure appears useful with as little as three pacing
Figure 2. Correlation between the slopes from the regressions of HFHRV/VT Pneumo and HF-HRV/VT Belt on TTOT. r 5 .93, po.01.
Respiratory variables and RSA
493
periods of 2 min each, rendering it an efficient addition to psychophysiological research designs assessing RSA. It is notable that these results were strongest when raw HF-HRV values, not those that were natural log-transformed, were used in the analyses. Thus, this correction procedure may be better suited for use with raw HF-HRV values that are not extremely skewed. Regarding the respiratory belt itself, we emphasize that our results do not establish that the belt provided a quantitative measure of tidal volume. Rather, they demonstrate that data from the respiratory belt can be used in the within-individual (not betweensubjects) respiratory correction procedure comparably to direct measures of tidal volume. Although this study does support the use of a respiratory belt in the measurement of relative tidal volume, it remains unclear what effects, if any, body habitus may have on the accuracy of its measurement. Although measures of body habitus such as body mass index or waist circumference were not taken, such factors may influence the position and fit of the belt, altering its measurement of thoracic expansion and contraction. Future research should address how such variations in participants’ physical characteristics may influence respiratory belt assessment. We also note that additional research is needed to validate our HF-HRV and respiratory belt substitution method against Ritz and colleagues’ original peak-to-valley statistic and pneumotachograph method. We acknowledge that the ratio of HF-HRV/VT Belt to HF-HRV/VT Pneumo varies somewhat as a function of breathing rate. To further assess their comparability, both methodologies should be conducted using the same participant sample so that the resulting within-individual regression models can be directly compared. Finally, such a study might incorporate the use of multilevel regression analyses, another technique often used to assess individual differences in within-subjects intercepts and standardized slopes. Despite these limitations, our findings are intriguing. They are particularly notable as there are certain advantages to the use of HF-HRVand the respiratory belt over the peak-to-valley statistic and pneumotachography, respectively. For example, several commercial software packages for automated spectral analysis have been validated and are in common use, standardizing the popular frequency-domain measurement. Correspondingly, the use of a respiratory belt can be seen as more cost effective and less invasive than pneumotachography. Generally, our demonstration that this respiratory correction procedure can be executed using HF-HRV and a respiratory belt provides methodological alternatives to the research community, increasing the ease with which it can be implemented. Naturally, different applications such as ambulatory versus laboratory studies may require different instrumentation. We cannot guarantee that our results will generalize to all situations, but we hope they are reasonably representative of noninvasive respiration and typical heart rate variability measures.
We recognize that there is some debate concerning whether correcting or unconfounding quantitative measures of RSA for respiratory influences is appropriate or necessary. Both centrally mediated cardiac vagal activity and respiration appear to contribute to the generation of RSA (Berntson et al., 1993, 1997; Denver et al., 2007; Grossman & Kollai, 1993; Grossman, Stemmler, & Meinhardt, 1990; Grossman & Taylor, 2007; Grossman et al., 1991; Martinmaki, Rusko, Kooistra, Kettunen, & Saalasti, 2006; Medigue et al., 2001; Penttila et al., 2001; Pyetan, Toledo, Zoran, & Akselrod, 2003). And, given one’s research question, the respiratory-modulated, nonrespiratory– modulated, or combined vagal effects may be of interest. The current experiment examined a simple, inexpensive respiratory assessment technique that we view as permitting a relatively easy means of assessing the modulatory components of vagal control. Given the developing nature of this field, researchers interested in using quantitative measures of RSA may initially find it appropriate to reflect on how respiratory changes might relate to their research question. For example, if it can be assumed that changes in respiration are related to the psychological concept being studied, they may be valid experimental effects. However, if changes in respiration do not follow from the psychological concept being investigated, they may warrant statistical removal of this portion of the vagal modulation. In such cases, it might be prudent to compare RSA values produced using the correction procedure described in this article to raw values. Attention should also be paid to whether there are systematic changes in respiratory rate and/or tidal volume as a function of experimental manipulations. This said, uncorrected RSA can also be conceptualized as an index of cardiorespiratory control with empirical relationships to dimensions of interest such as depression or physical fitness. This conceptualization does not attempt to infer vagal function, but considers HRV a meaningful index of the central and peripheral control of both cardiac and respiratory function. In short, any confounding is viewed as intrinsic to this index, and separation of respiratory and vagal influences may be inappropriate, compromising predictive validity (see elaboration of the general point in Jennings & Gianaros, 2007). Separation of these influences may, of course, be of conceptual interest despite the predictivity of the confounded index. In conclusion, we report results demonstrating that the procedure for correcting respiratory influences on quantitative indices of RSA developed by Ritz et al. (2001) can be modified for use with a spectrally derived measure of RSA and with a respiratory belt used to provide a relative measure of tidal volume. This methodology can be implemented over 6-, 4-, and 2-min time courses. Given these findings, we present an efficient and minimally invasive technique for respiratory correction.
REFERENCES Allen, J. J. B., Chambers, A. S., & Towers, D. N. (2007). The many metrics of cardiac chronotropy: A pragmatic primer and a brief comparison of metrics. Biological Psychology, 74, 243–262. Berntson, G. G., Bigger, J. T. Jr., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., et al. (1997). Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology, 34, 623–648. Berntson, G. G., Cacioppo, J. T., & Quigley, K. S. (1993). Respiratory sinus arrhythmia: Autonomic origins, physiological mechanisms, and psychophysiological implications. Psychophysiology, 30, 183–196.
Camm, A. J., Malik, M., Bigger, J. T., Breithardt, G., Cerutti, S., Cohen, R. J., et al. (1996). Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. European Heart Journal, 17, 354–381. Cooper, H. E., Parkes, M. J., & Clutton-Brock, T. H. (2003). CO2dependent components of sinus arrhythmia from the start of breath holding in humans. American Journal of Physiology. Heart and Circulatory Physiology, 285, H841–848. Denver, J. W., Reed, S. F., & Porges, S. W. (2007). Methodological issues in the quantification of respiratory sinus arrhythmia. Biological Psychology, 74, 286–294.
494 Eckberg, D. L. (2003). The human respiratory gate. Journal of Physiology, 548(Pt 2), 339–352. Gilbey, M. P., Jordan, D., Richter, D. W., & Spyer, K. M. (1984). Synaptic mechanisms involved in the inspiratory modulation of vagal cardio-inhibitory neurones in the cat. Journal of Physiology, 356, 65–78. Grossman, P., Karemaker, J., & Wieling, W. (1991). Prediction of tonic parasympathetic cardiac control using respiratory sinus arrhythmia: The need for respiratory control. Psychophysiology, 28, 201–216. Grossman, P., & Kollai, M. (1993). Respiratory sinus arrhythmia, cardiac vagal tone, and respiration: Within- and between-individual relations. Psychophysiology, 30, 486–495. Grossman, P., Stemmler, G., & Meinhardt, E. (1990). Paced respiratory sinus arrhythmia as an index of cardiac parasympathetic tone during varying behavioral tasks. Psychophysiology, 27, 404–416. Grossman, P., & Taylor, E. W. (2007). Toward understanding respiratory sinus arrhythmia: Relations to cardiac vagal tone, evolution and biobehavioral functions. Biological Psychology, 74, 263–285. Grossman, P., van Beek, J., & Wientjes, C. (1990). A comparison of three quantification methods for estimation of respiratory sinus arrhythmia. Psychophysiology, 27, 702–714. Jennings, J. R., & Gianaros, P. J. (2007). Methodology. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (pp. 812–833). New York: Cambridge University Press. Konno, K., & Mead, J. (1967). Measurement of the separate volume changes of rib cage and abdomen during breathing. Journal of Applied Physiology, 22, 407–422. Martinmaki, K., Rusko, H., Kooistra, L., Kettunen, J., & Saalasti, S. (2006). Intraindividual validation of heart rate variability indexes to measure vagal effects on hearts. American Journal of Physiology. Heart and Circulatory Physiology, 290, H640–647. Medigue, C., Girard, A., Laude, D., Monti, A., Wargon, M., & Elghozi, J. L. (2001). Relationship between pulse interval and respiratory sinus
V.B. Egizio et al. arrhythmia: A time- and frequency-domain analysis of the effects of atropine. Pflugers Archiv, 441, 650–655. Noble, D. (1979). The initiation of the hearbeat (2nd ed). Oxford, England: Clarendon Press. Penttila, J., Helminen, A., Jartti, T., Kuusela, T., Huikuri, H. V., Tulppo, M. P., et al. (2001). Time domain, geometrical and frequency domain analysis of cardiac vagal outflow: Effects of various respiratory patterns. Clinical Physiology, 21, 365–376. Pyetan, E., Toledo, E., Zoran, O., & Akselrod, S. (2003). Parametric description of cardiac vagal control. Autonomic Neuroscience, 109, 42–52. Ritz, T. (2009). Studying noninvasive indices of vagal control: The need for respiratory control and the problem of target specificity. Biological Psychology, 80, 158–168. Ritz, T., & Dahme, B. (2006). Implementation and interpretation of respiratory sinus arrhythmia measures in psychosomatic medicine: Practice against better evidence? Psychosomatic Medicine, 68, 617–627. Ritz, T., Thons, M., & Dahme, B. (2001). Modulation of respiratory sinus arrhythmia by respiration rate and volume: Stability across posture and volume variations. Psychophysiology, 38, 858–862. Sackner, M. A. (1996). A simple and reliable method to calibrate respiratory magnetometers and Respitrace. Journal of Applied Physiology, 81, 516–517. Strauss-Blasche, G., Moser, M., Voica, M., McLeod, D. R., Klammer, N., & Markti, W. (2000). Relative timing of inspiration and expiration affects respiratory sinus arrhythmia. Clinical and Experimental Pharmacology and Physiology, 27, 601–606.
(Received June 1, 2009; Accepted June 9, 2010)
Psychophysiology, 48 (2011), 495–506. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01080.x
Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification
PIA KNOEFERLE,a THOMAS P. URBACH,b,c and MARTA KUTASb,c,d,e a
Cognitive Interaction Technology Excellence Cluster, Bielefeld University, Bielefeld, Germany Department of Cognitive Sciences, University of California San Diego, La Jolla, California c Center for Research in Language, University of California San Diego, La Jolla, California d Department of Neurosciences, University of California San Diego, La Jolla, California e Kavli Institute for Brain and Mind, University of California San Diego, La Jolla, California b
Abstract To re-establish picture-sentence verificationFdiscredited possibly for its over-reliance on post-sentence response time (RT) measuresFas a task for situated comprehension, we collected event-related brain potentials (ERPs) as participants read a subject-verb-object sentence, and RTs indicating whether or not the verb matched a previously depicted action. For mismatches (vs. matches), speeded RTs were longer, verb N400s over centro-parietal scalp larger, and ERPs to the object noun more negative. RTs (congruence effect) correlated inversely with the centro-parietal verb N400s, and positively with the object ERP congruence effects. Verb N400s, object ERPs, and verbal working memory scores predicted more variance in RT effects (50%) than N400s alone. Thus, (1) verification processing is not all post-sentence; (2) simple priming cannot account for these results; and (3) verification tasks can inform studies of situated comprehension. Descriptors: Picture-sentence verification, Situated comprehension, Event-related brain potentials
temporally coordinated interplay: e.g., language directs attention to objects and events (or representations thereof in visuo-spatial working memory) and can, in turn, receive rapid feedback from scene-based mental representations (Knoeferle & Crocker, 2006; 2007; Mayberry, Crocker, & Knoeferle, 2009, ‘Coordinated Interplay Account’). The focus in the field of psycholinguistics to date has been on the ‘‘facilitative’’ effects of visual contexts (scenes) on language processing, e.g., how visual context might incrementally disambiguate temporary linguistic and/or referential ambiguities (e.g., Altmann, 2004; Chambers et al., 2004; Knoeferle et al., 2005; Knoeferle & Crocker, 2007; Sedivy et al., 1999; Spivey et al., 2002). However, visual context apparently can have effects on language processing even when the accompanying language and/ or reference is unambiguous (e.g., Stroop, 1935). Thus, it is important to examine the nature and timing of visual context and language comprehension interactions more generally, even when the two information sources are not completely in accord, which, upon careful consideration, is often the case. Inconsistencies can range from outright mismatches (e.g., when a passenger alerts the driver to a red traffic light, which has turned green by the time they actually look), mismatches due to lexically different but semantically similar terms, to subtle nuances in how different individuals describe the same object or event (e.g., one person sees and thinks ‘‘couch’’ which another might refer to as ‘‘sofa’’ or ‘‘a place to relax’’). Socio-economic factors (e.g., age, gender,
Although humans can readily understand sentences about events that are displaced in space and time without being present at the scene, language users often are physically ‘‘situated’’ in the scene. Indeed, information from a co-present or recently experienced visual environment has been found to affect auditory sentence comprehension with a relatively short lag. This has been evidenced by the continuous monitoring of eye gaze in a visual context (act-out tasks: e.g., Chambers, Tanenhaus, & Magnuson, 2004; Sedivy, Tanenhaus, Chambers, & Carlson, 1999; Spivey, Tanenhaus, Eberhard, & Sedivy, 2002; Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995; passive listening comprehension: e.g., Altmann, 2004; Knoeferle, Crocker, Scheepers, & Pickering, 2005; Knoeferle & Crocker, 2007), and by event-related brain potentials (ERP) recordings during concurrent utterance comprehension and visual scene inspection (e.g., Knoeferle, Habets, Crocker, & Mu¨nte, 2008). Processing accounts of situated utterance comprehension accommodate this
This research was funded by a postdoctoral fellowship to PK awarded by the German research foundation (DFG) and by NIH grants HD22614 and AG-08313 to MK. The present study was conducted while the first author was at the Center for Research in Language, at UC San Diego, La Jolla, California. Address correspondence to: Pia Knoeferle, Cognitive Interaction Technology Excellence Cluster, BLB Building, Morgenbreede 39, D-33615 Bielefeld, Germany. E-mail:
[email protected] 495
496 social status) also may influence not only how people perceive their world but also how they talk about it. An incident that a witness describes as ‘‘slightly prodding a friend’’ may be expressed as ‘‘aggressively shoving a man’’ by another individual. In short, utterances and written text or signs may often be a lessthan-perfect match for a language user’s current representation of the non-linguistic visual environment. Despite its obvious import for theories of sentence comprehension and situated processing accounts, however, little is known about how inferences about the time course of incremental comprehension might extend to situations when visual context clashes with the linguistic input. Morever, such findings would need to be theoretically accommodated, if observed (e.g., Knoeferle & Crocker, 2006, 2007). Models of how people understand and evaluate the truth or falsity of sentences against visual contexts have generally proposed a serial comparator mechanism between corresponding picture and sentence constituents, e.g., between dots and the noun ‘‘dots’’ (Carpenter & Just, 1975). These types of models aim to account for the general finding that verification times tend to be shorter when visually depicted information and linguistically expressed information match (vs. not). At first blush, then, verifying the truth of a sentence against a picture might appear well suited for developing cognitive models of language comprehension. In light of the burgeoning interest in situated and embodied language comprehension, the verification paradigm also would seem to be a fruitful approach for investigating the time course of language comprehension in visual contexts. Its utility and the validity of the proposed serial comparison mechanism have, however, been sharply criticized (e.g., Tanenhaus, Carroll, & Bever, 1976; see Carpenter & Just, 1976, for a reply). In particular, it has been argued that the key dependent measure (i.e., response verification time) may reflect scene-sentence comparison processes that take place only after sentence comprehension has been completed rather than the effects of pictorial information on ongoing language comprehension processes: ‘‘At best, such studies may provide specific examples of how subjects can verify sentences they have already understood against pictures they have already encoded’’ (Tanenhaus et al., 1976). In addition, a few failures to find the typical congruence effect (longer response latencies for a picture-sentence mismatch than match) with serial picture-sentence presentation have led to concerns about the generality of the paradigm. Despite its occasional use to study language comprehension (e.g., Goolkasian, 1996; Reichle, Carpenter, & Just, 2000; Singer, 2006; Underwood, Jebbett, & Roberts, 2004), then, the fact remains that insights obtained with this paradigm have had minimal impact on psycholinguistic theories of online sentence comprehension or of situated sentence comprehension. Perhaps the conclusions by Tanenhaus and colleagues are justified. After all, punctate measures such as verification response times and accuracy scores obtained at the end of sentences may merely reflect post-sentence verification processes and say nothing about online incremental language processing. Alternatively, however, they mayFto an important extentFreflect (visual context effects on) at least some aspects of online language processingFa possibility that can be better assessed if the offline verification time measures are combined with a continuous online processing measure such as ERPs (see, e.g., Kounios & Holcomb, 1992). The study of language processing benefits from combining these two measures (verification response times [RTs] and sen-
P. Knoeferle et al. tence ERPs) to the extent that ERPs in verification tasks reflect comprehension and/or verification processes rather than mere mismatch detection (since, in that case, ‘‘sentence or word processing’’ ERPs can be related to ‘‘verification’’ RTs, operationally defined). Critically, manipulations of semantic context that modulate sentence processing are known to be associated with a greater negativity (N400) starting within 200 ms of stimulus onset for semantically anomalous or unexpected words relative to semantically congruous, expected words (Kutas & Hillyard, 1980). Moreover, ample data show that such N400 effects are related to comprehension rather than strictly mismatch or pure verification processes (Fischler, Bloom, Childers, Roucos, & Perry, 1983), do not necessarily index plausibility (Federmeier & Kutas, 1999), and are dissociable from RTeffects (Fischler et al., 1983). Existing ERP evidence indicates that picture-sentence congruence effects can appear incrementally during sentence processing in the form of N400 effects and, sometimes, P600 effects (e.g., Vissers, Kolk, Van de Meerendonk, & Chwilla 2008; Wassenaar & Hagoort, 2007), among other possibilities. For example, some picture-sentence congruence manipulations during sentence comprehension were found to elicit a negativity akin to an N2b component previously observed in response to adjectival color mismatches (object: red square; linguistic input: green square; token test) and taken to index attentional detection of a mismatch rather than language processing (D’Arcy & Connolly, 1999; Vissers et al., 2008). In other cases, however, picture-sentence congruence manipulations yielded N400s effects that resemble those in language comprehension tasks (e.g., auditory N400: Holcomb & Neville, 1991; McCallum, Farmer, & Pocock, 1984), consistent with the view that visual contexts modulate language comprehension processes (e.g., Wassenaar & Hagoort, + 2007). Szucs, Solte´sz, Czigler, and Cse´pe (2007) reported both N2b and N400 activity; specifically, they observed N2bs to mismatches when the second of a pair of items (letter, digit) mismatched on color or category membership, while only category mismatches also elicited a centro-parietal N400. Clearly, there is much to be learned about the nature of the relationship between verification and comprehension, and language processing more generally. The present study thus examines whether, and if so how, participants’ picture-sentence congruence effects during sentence processing (that resemble ERP effects in strictly language comprehension tasks) relate to their end-of-sentence response latency verification times. On each trial, a few-second inspection of a clip art scene (e.g., a gymnast punching a journalist) was followed by word-by-word presentation of a written sentence; the verb either matched (e.g., punches) the depicted agent-action-patient event or not (e.g., applauds). Our primary dependent measures were the ERPs elicited by individual words and the post-sentence verification response times, traditionally taken to reflect the ease or difficulty of processing the match or mismatch between a scene and sentence in a verification task (e.g., Clark & Chase, 1972; Gough, 1965). Consistent with the literature at large, we expect mismatches to be verified more slowly than matches. With evidence in hand that processing of congruous versus incongruous sentence-picture pairs differs as expected, we will examine whether the ERP manifestations of these processing differences during the sentence resemble canonical ERP congruence effects in sentence comprehension tasks (e.g., Kutas & Hillyard, 1984; Van Berkum, Hagoort, & Brown, 1999). Based on two published reports (e.g.,
Comprehending visual context influences + et al., 2007; Wassenaar & Hagoort, 2007), we do expect to Szucs see a centro-parietal N400 effect (greater negativity to the mismatch) at the earliest possible mismatch point between the picture and sentenceFnamely, the verb. One open question is whether or not there will be any other signs of the picture-sentence discord during the course of sentence processing. Verb-action mismatch ERP effects could be quite local (restricted to the verb and the response). This would be expected on, for example, a ‘priming’ account according to which a representation of the depicted action primes the processing of the matching (relative to mismatching) verb as reflected in ERPs (see Barrett & Rugg, 1990; Kutas & Van Petten, 1988; Meyer & Schvaneveldt, 1971). Perhaps because there are no other discrepancies between the depicted elements and the sentence constituents, no other ERP congruence effects would be obtained. However, since participants were asked to indicate whether the sentence was true or false with regard to the immediately preceding picture, we would also expect to see a congruency effect in the post-sentence verification times. On a straightforward priming account, ERP and RTcongruence effects would both reflect a similar (priming) process, and thus should co-vary directly (i.e., a large ERP congruence effect should be associated with a large verification time congruence effect and vice versa). Alternatively, there may be multiple online effects of visual context on sentence processing, which may be manifest with variable and/or complex relations to the offline verification time effects. Referential verb-action mismatches are a good test case for adjudicating between these alternative accounts, since the locus of the mismatch is restricted to the verb. Finding more than one ERP congruence effect during the course of these sentences and/or the absence of direct variation between ERP picture-sentence congruence effects and RT verification effects, for example, would undermine any simple priming account. Moreover, to the extent that the collection of online ERP congruity effects as a whole were able to account for significant variance in the offline verification times, we could infer that verification was not simply something that happened after sentence comprehension was complete. In addition, to facilitate interpretation of any observed
497 congruence effects, we examined whether, and if so how, the nature or time course of visual context effects on online sentence comprehension was modulated by participants’ verbal working memory and/or visuo-spatial abilities. To that end, we obtained visual (Extended Complex Figure Test: Motor-independent version; Fasteneau, 2003) and verbal (Daneman & Carpenter, 1980, reading span test) working memory scores for our participants.
Methods Participants Twenty-four students of the University of California San Diego (UCSD) (11 men, 13 women, aged 19–24, mean 5 20.04 years) were paid for participation. All participants were native English speakers, right-handed according to the Edinburgh Handedness Inventory, and had normal or corrected-to-normal vision. All gave informed consent in accordance with the Declaration of Helsinki; the experiment protocol was approved by the UCSD Institutional Review Board. Materials An example image pair for one item shows a gymnast punching a journalist (Figure 1a) and a gymnast applauding a journalist (Figure 1b). These were paired with one of the following sentences: (1a) The gymnast punches the journalist. (1b) The gymnast applauds the journalist. The experiment was a 1-factor (congruence) within-subjects design with two levels (incongruent versus congruent verb-action relation): Sentence (1a) The gymnast punches the journalist is congruent when following Figure 1a and incongruent following Figure 1b. The materials were counterbalanced to ensure that any congruency-based ERP differences were not spuriously due to stimuli or to their presentation: (1) Each verb (e.g., punches/ applauds) and corresponding action (punching/applauding)
Figure 1. Example image set for the item sentences in (1a) and (1b).
498 occurred once in a congruent and once in an incongruent condition; (2) Each verb and action occurred in two different items (with different first and second nouns); and (3) Directionality of the actions (the agent standing on the left vs. the agent standing on the right) also were counterbalanced. We conducted two pre-tests on 168 image and sentence stimuli to determine whether the characters and depicted actions were recognizable as well as the degree of sentence-picture correspondence. Twenty participants (different from those in the ERP study) named the characters and action in each of the 168 scenes. The average accuracy in naming the agent, action, and patient was 81.84%, 85.17%, and 88.75%, respectively. Mean naming accuracy was above 50% for all items. Participants also rated the extent to which each image matched its corresponding sentence on a 5-point scale (1 indicating a poor match and 5 indicating very good congruence). Mean ratings of 4.31 (SD 1.0) in the congruent condition, and 2.04 (SD 1.14) in the incongruent condition revealed that the congruence manipulation was effective (po.001). Only images for which participants consistently named the characters and the action, and sentences for which there was a greater than or equal to 1.4-point difference between the congruent and incongruent pairs, were selected for the ERP recording. On the basis of the pretest, we selected 160 images and sentences and constructed 80 item sets, each consisting of 2 sentences and 2 images (such as Figure 1a and 1b, and sentences (1a) and (1b)), which, combined with the within-subject counterbalancing, yielded 8 experimental lists. Each participant saw an equal number of matching and mismatching trials, only one occurrence of an item (i.e., sentence/image pairing), and an equal number of left-to-right and right-to-left action depictions. Assignment of item-condition combinations to a list followed a Latin Square. As a result of the counterbalancing (each verb and action appeared in two different items), cross-modal repetition occurred within half of the items in a list: in other words, a given verb (e.g., ‘‘punches’’) occurred in one item sentence (together with, e.g., a mismatching applauding action), and a depicted punching action occurred in another item (together with a mismatching verb ‘‘applauds’’). We think this repetition was not noticed since it involved a relatively small number (40) of the overall (240) trials. When asked during the pen-and-paper debriefing whether anything caught their attention (‘‘patterns, anything strange, or surprising’’), participants did not report noticing verb-action repetition. Each list also contained 160 filler items, of which half included one of several different types of mismatches: ‘full mismatch’ (scene and sentence were entirely unrelated, ensuring that sentence comprehension was not always contingent on the scene), noun-object reference mismatches, mismatches of the spatial layout of the scene, and mismatches of color adjectives. These filler sentences had a variety of different syntactic structures including negation, clause-level and noun-phrase coordination, as well as locally ambiguous reduced relative clause constructions. They included ‘‘combinatory’’ mismatches (i.e., requiring combinatory processing to determine sentence truth). Accordingly, we were relatively confident that participants had to perform combinatory sentence comprehension in the verification task. Procedure Each trial consisted of an image (scene) followed by a sentence presented one word at a time in central vision. Images were presented on a 21-inch CRT monitor at a viewing distance of
P. Knoeferle et al. approximately 150 cm. Participants first inspected the image for a minimum of 3000 s, terminated by the participant via right thumb button press. Next, a fixation dot was presented for a random duration between 500 and 1000 ms, followed by a sentence one word at a time, each presented for 200 ms duration with a word onset asynchrony of 500 ms. Note that in picture-sentence verification research, the sentence is often presented as a whole. Rapid serial visual presentation was chosen based on its successful use in ERP-verification studies. Participants were encouraged to focus on comprehension during inspection of both picture and sentence: They were instructed to attend to the picture in order to understand what was depicted, and to comprehend the sentences in the context of the preceding image. To assess comprehension, participants were also required to indicate via a button press as quickly and accurately as possible after each sentence whether it matched (‘‘true’’) or did not match (‘‘false’’) the preceding image (only post-sentential responses were ‘‘counted’’ as accurate). Allocation of response hand to ‘‘true/ false’’ responses was counterbalanced across lists. Once a participant had responded, there was a short inter-trial interval varying between 500 and 1000 ms, to break the rhythm of the fixed rate of sentence presentation. In addition to response latencies, we assessed each participant’s verbal working memory (VWM) and visual-spatial abilities from a standard working memory test (Daneman & Carpenter, 1980), and the brief, motor-independent version of the extended complex figure test (ECFT-MI, Fasteneau, 2003), respectively. The ECFT-MI recognition trials provide a measure of visual-spatial encoding and recognition processes, and its matching trials verify that a participant’s visual-perceptual functions are intact. ERP Recording and Analysis ERPs were recorded from 26 electrodes embedded in an elastic cap (arrayed in a laterally symmetric pattern of geodesic triangles approximately 6 cm on a side and originating at the intersection of the inter-aural and nasion-inion lines as illustrated in Figure 2) plus 5 additional electrodes referenced online to the left mastoid, amplified with a bandpass filter from 0.016 to 100 Hz, and sampled at 250 Hz. Recordings were re-referenced offline to the average of the activity at the left and right mastoid. Eye movement artifacts and blinks were monitored via the horizontal (through two electrodes at the outer canthus of each eye) and vertical (through two electrodes just below each eye) electrooculogram. Only trials with a correct response were included in the analyses. All trials were scanned offline for artefacts, and contaminated trials were excluded from further analyses. For 18 of the participants, blinks were corrected with an adaptive spatial filter (Dale, 1994). After blink correction, no more than 25% of the correct trials per condition for a given participant were rejected. All analyses (unless otherwise stated) were conducted relative to a 500-ms pre-stimulus baseline. Of the 80 items, 21 had first and/or second noun phrases that were composite (e.g., ‘the volleyball player’) while the remaining 59 items had simple noun phrases (e.g., ‘the gymnast’). This means that for the 21 items, the first and second nouns have one extra region, rendering analyses and reporting somewhat complicated (i.e., it is unclear which noun region ‘volleyball’ or ‘player’ or both to report). Furthermore, as results did not differ substantially as a function of whether analyses were performed across all 80 items or only for the homogeneous subset of 59 items, below we report those for the latter.
Comprehending visual context influences
499
Figure 2. Grand average ERPs (mean amplitude) for all 26 electrodes, right-lateral, left-lateral, right-horizontal, left-horizontal (‘rle,’ ‘lle,’ ‘rhz,’ and ‘lhz’), and the mastoid (‘A2’) at the verb position. Waveforms were subjected to a digital low-pass filter (10 Hz) for visualization. A clear negativity emerges for incongruent relative to congruent sentences at the verb when the mismatch between verb and action becomes apparent. The ERP comparison at the mid-parietal (‘MiPa’) site is shown enlarged.
Analyses of Behavioral Data For response latency analyses, any individual score ! 2 standard deviation from the mean response latency of a participant was removed prior to further analyses. Mean response latencies time-locked to the sentence-final word (the second noun) were summarized by participants (t1) and items (t2) and analyzed via paired sample t-tests with the congruence factor (mismatch vs. match). For the analysis of working memory scores from the Daneman and Carpenter (1980) working memory test, we computed the proportion of items for which a given participant recalled all the elements correctly (Conway et al., 2005) as a proxy for VWM scores. For the extended complex figure test, we followed the scoring procedure for the motor-independent ECFTMI described in Fasteneau (2003). Analyses of ERP Data Analyses of variance (ANOVAs) were conducted on the mean amplitudes of the average ERPs elicited by the verb (e.g., punches) since this is where the verb-action (in)congruence first becomes evident. We measured the N400 between 300–500 ms post verb onset, as is common for visual N400s, and also from 300–600 ms, since it became apparent that the N400 lasted circa 100 ms into the post-verbal determiner. Based on visual inspection and traditional sensory evoked potential epochs, we also analyzed three earlier time windows (0–100 ms; 100–300 ms;
180–220 ms to explore the P2) in the ERPs to the verb. In addition, we analyzed ERPs elicited by the first nouns (gymnast) where we expected to see no effect of congruence and second determiners (the) and nouns (journalist) where a post-verb effect of congruence was possible in the same time windows. For the second determiner, we analyzed ERPs from 100–300 and 300– 500 ms with a baseline from " 500 to " 300 ms to ensure ERP effects at the verb didn’t modulate the baseline. The first 100 ms of the second determiner (0–100 ms) were analyzed as part of the verb ERPs (300–600 ms) since visual inspection showed a continuation of verb ERP effects into the early second determiner. We also analyzed the ERP after the offset of the second noun (500–1000 ms; 1000–1500 ms time-locked to the second noun) to gain insight into processing of the verb-action mismatch from sentence end up to the verification response. We performed omnibus repeated measures ANOVAs on mean ERP amplitudes (averaged by participants for each condition at each electrode site) with congruence (incongruent vs. congruent), hemisphere (left vs. right electrodes), laterality (lateral vs. medial), and anteriority (5 levels) as factors. Interactions were followed up with separate ANOVAs for left lateral (LLPf, LLFr, LLTe, LDPa, LLOc), left medial (LMPf, LDFr, LMFr, LMCe, LMOc), right lateral (RLPf, RLFr, RLTe, RDPa, RLOc), and right medial (RMPf, RDFr, RMFr, RMCe, RMOc) electrode sets (henceforth ‘slice’) that included con-
500 gruence (match vs. mismatch) and anteriority (5 levels). Greenhouse-Geisser adjustments to degrees of freedom were applied to correct for violation of the assumption of sphericity. We report the original degrees of freedom in conjunction with the corrected p-values. For pair-wise t-tests that were also conducted to follow up complex interactions, we report p-values after Bonferroni adjustments for multiple comparisons (20 comparisons) unless otherwise stated. Linear Regression Analyses We investigated the relationship between ERPs and verification response times. To that end, we conducted correlation, simple, and multiple linear regression analyses to see to what extent end-of-sentence verification times can be predicted from verb ERPs alone, or in combination with subsequent ERP congruence effects. For these analyses, we computed each participant’s mean congruence effects (incongruent minus congruent ERP amplitude), and each participant’s congruence effect for verification response latencies (incongruent minus congruent latencies). The analyses are thus between two sets of difference scores. Difference scores have been much discussed (and sometimes criticized, e.g., Cohen & Cohen, 1983; Griffin, Murray, & Gonzalez, 1999; Johns, 1981). We think they are informative in our study: the difference scores provide a measure of the extent to which the processing of congruous and incongruous trials differs both at the verb and at the end of the sentence. For response latencies, a positive number indicates longer verification times for incongruous than congruous times and a negative number indicates the converse. For ERPs, a negative number means that incongruous trials were relatively more negative (or less positive) than congruous trials, with the absolute value of the negative number indicating the size of the difference. For the correlation and regression analyses, we relied upon congruence ERP difference scores summarized using the mean across the electrode sites in the four slices used for the ANOVA analyses (e.g., left lateral: LLPf, LLFr, LLTe, LDPa, LLOc; Bonferroni adjustment for 4 analyses in a given window, 0.05/4). Expectations of which ERP scores should be good predictors in the regression model were influenced by the presence and scalp topography of congruence effects in the ANOVA, the presence of reliable correlations with the verification times, and by published findings showing larger and more robust N400 effects over right (than left) hemisphere sites (see Federmeier & Kutas, 2002). While linear regression is typically used to predict a dependent measure from an independently manipulated variable, we justify its current application (prediction of a dependent variable from another dependent variable) on the view that response latencies result from preceding brain activity. We screened scatter plots of residuals (errors) against predicted values of the dependent variable to ensure that assumptions of normality, linearity, and homoscedasticity between predicted scores of the dependent variable and errors of prediction were met. There were no multivariate outliers (Mahalanobis distance). We also inspected the data for multicollinearity of the independent variables in the multiple regression models (see Tabachnik & Fidell, 2007); multicollinearity was not an issue for the independent variables in our linear regression models (ERPs from 300–500 ms at the verb and second noun). For multiple regressions, we report adjusted R2s to ensure that increasing the number of independent variables does not artificially (because of chance variation of the independent variables) increase the accounted-for variance in the response verification congruence
P. Knoeferle et al. effect. Last but not least, to examine whether there was any reliable difference between two models (e.g., a model with ERP congruence scores at the verb and second noun vs. a nested model with ERP congruence effects at the verb), we computed a partial F (see Hutcheson & Sofroniou, 1999). Results Behavioral Participants’ verification accuracy was relatively good (83.84%) overall, suggesting participants understood the sentences and images and their interrelationships, albeit better for congruous (86.93%) than incongruous (80.75%) trials (t1(1,23) 5 1.41, p 5 .17; t2(1,58) 5 2.36, po.05). These accuracy values are somewhat lower than those in prior research (e.g., Vissers et al., 2008; Wassenaar & Hagoort, 2007) as we, unlike these others, included a range of picture-sentence filler combinations with reduced relative clause structures, and negated sentences. It is likely that the inclusion of these fillers made it more difficult for participants to verify the picture-sentence mismatch, and thus led to longer response times and lower accuracy levels, than if we had tested exclusively verb-action mismatches. RT analyses also revealed reliably faster verification times for congruent (by participants: 1104 ms, SD 5 271.72) than incongruent (by participants: 1273 ms, SD 5 361.06) sentences (t1(1,23) 5 ! 3.14, po.01, t2(1,58) 5 ! 4.125, po.001). Participants’ average VWM score in the Daneman and Carpenter working memory test was 0.31 (the score reflects the proportion of test elements for which all of the items were recalled correctly; range 0.13–0.65, see Conway et al., 2005). Their average score in the extended complex figure test was 12.13 out of 18 (range 7–18) for the recognition and 9.71 out of 10 for the matching part (range 8–10). ERPs: Analyses of Mean Amplitudes Figure 2 displays the grand average ERPs (N 5 24) at all 26 electrode sites in the picture-congruent versus incongruent conditions time-locked to verb onset. Figure 3 shows the grand average ERPs at prefrontal, parietal, temporal, and occipital sites together with the spline-interpolated topographies of the mean ERP amplitude difference (mismatch minus match) both between 300–500 ms post-verb onset, lasting into the post-verbal determiner, and between 300–500 ms at the second noun (patient). These figures illustrate a broadly distributed negativity (N400) at the verb, maximal at posterior recording sites, and a right lateral negativity at the second noun (patient), both of which are reliably larger for picture-incongruous than picturecongruous verbs. Tables 1 and 2 present the corresponding ANOVA results for the main effect of congruence and interactions between congruence, hemisphere, laterality, and anteriority at the verb and second noun. ANOVAs of ERPs to the first noun (e.g., gymnast) and the second determiner (100–300 ms, 300–500 ms) revealed neither reliable main effects nor interactions of picture-sentence congruence (all Fso2.53). Verb For the verb N400, ANOVAs indicated larger congruence effects over medial (left medial sites: partial eta squared Z2 5 0.50; right medial sites: Z2 5 0.43) than lateral (left: Z2 5 0.38; right: Z2 5 0.42, for corresponding F-values see Table 1) sites, confirming the typical N400 distribution, maximal over centro-parietal sites. Congruence effects also interacted with anteriority, being smaller over anterior (e.g., LMPf, Z2 5 0.29; RMPf, Z2 5 0.27,
Comprehending visual context influences
501 congruence effect n.s.) than central sites (LMCe, Z2 5 0.56; RMCe, Z2 5 0.46; pso.05). Complex interactions between congruence, hemisphere, and laterality are brought about by a large difference between congruence effects over left lateral (Z2 5 0.38) versus left medial (Z2 5 0.50, difference 5 0.12, see Table 1) sites relative to similar effect sizes for lateral (Z2 5 0.42) and medial (Z2 5 0.43, difference 5 0.01) sites over the right hemisphere. Additional ANOVAs on mean amplitudes in earlier time windows (0–100; 100–300) at the verb revealed no other reliable effects of congruence. Analyses from 300–600 ms at the verb revealed the same pattern of main effects and interactions as from 300–500 ms and are thus not reported in greater detail. Second Noun ERPs ERPs for the second noun revealed a reliable interaction between congruence and anteriority from 0–100 ms (Figures 2 and 3), which follow-up analyses showed were reliable over left (but not right) lateral or medial sites (Table 2). For the 300–500 ms (N400) time window, analyses revealed a reliable congruence by hemisphere interaction: None of the main effects of congruence in separate ANOVAS for left/right lateral and medial electrode sites were reliable (Table 2), but effect sizes were still larger over right (lateral Z2 5 0.1; medial Z2 5 0.06) than left (lateral Z2 5 0.00; medial: Z2 5 0.01) hemispheric sites.
Figure 3. Grand average ERPs (mean amplitude) across the sentence at prefrontal, parietal, temporal, and occipital sites. A clear negativity emerges at the verb when the mismatch between verb and action becomes apparent and, later, at the second noun (patient). Spline-interpolated difference scores at the verb and the second noun illustrate the centroparietal (verb) and right lateral (second noun) distribution of these two ERP effects.
Correlation Analyses Analyses showed reliable correlations of the response time congruence effect with mean ERP congruence effects summarized over right lateral, medial, and left medial slices at the verb: the larger a participant’s verb N400 congruence effect, the smaller her/his response time congruence effect (Figure 5). These analyses with ERPs summarized over the four slices were corroborated at individual electrode sites where we found reliable correlations of the response time verification congruence effects with verb N400 (300–500 ms) congruence effects at RMCe, RDPa, RMOc, RLTe according to Pearson r (r 5 .58, r 5 .59, r 5 .59, r 5 .66, respectively, at these sites, see Figure 4). The only other behavioral measure correlated with (right medial) ERP congruence effects at the verb was verification accuracy: the
Table 1. Statistical Results Mean Amplitude of ERPs at the Verb Sentence position Verb
Time window 0–100 100–300 180–220 300–500
Factors
Overall ANOVA
C CH CL CA CHL CHA CLA CHLA
F F F 21.85nnn F 12.99nn 4.29n 6.61n 3.73n F 3.51n
Left lateral sites
Left medial sites
Right lateral sites
Right medial sites
13.96nn
23.26nnn
16.70nnn
17.26nnn
3.89n
5.39n
3.79n
2.26
Note: Columns 3–4 show the results of the overall ANOVA electrode sets at the verb (20 electrode sites); columns 5–8 show results of separate follow-up ANOVAS for left lateral (LL: LLPf, LLFr, LLTe, LDPa, LLOc), left medial (LM: LMPf, LDFr, LMFr, LMCe, LMOc), right lateral (RL: RLPf, RLFr, RLTe, RDPa, RLOc) and right medial (RM: RMPf, RDFr, RMFr, RMCe, RMOc) electrode sets that included congruence (match vs. mismatch) and anteriority (5 levels). Given are the F-values; we report main effects of congruence (C) and interactions of congruence with hemisphere (H), laterality (L), and anteriority (A); main effects of factors hemisphere, laterality, and anteriority are omitted for the sake of brevity; degrees of freedom df(1,23) expect for CA, CHA, CLA, CHLA, df(4,20). n po.05; nnpo.01; nnnpo.001.
502
P. Knoeferle et al.
Table 2. Statistical Results Mean Amplitude of ERPs at the Second Noun Sentence position
Time window
2nd noun (patient)
0–100
100–300 300–500
400–600
Factors
Overall ANOVA
Left lateral sites
Left medial sites
Right lateral sites
Right medial sites
C CH CL CA CHL CHA CLA CHLA F C CH CL CA CHL CHA CLA CHLA C CH CL CA CHL CHA CLA CHLA
F F F 3.85n F F F F
F
F
F
F
5.38n
F
F
F
F 5.05n F F F F F F F 5.18n F F F F F F
Note: See Table 1 for legend. n po.05.
higher the mean accuracy scores, the larger the N400 congruence effect (Table 3 and Figure 6b). For the negativity at the second noun (300–500 ms), there was a trend showing that the larger the frontolateral N400 effect, the larger the response latency congruence effect. This relation was reliable for a slightly later (400–600 ms) time window, over the right lateral slice only (see Table 3), and not at individual electrode sites ( ! 0.37oro0.11, see Figure 4). Correlations for ERPs to the second noun with either verb ERPs, or with the other behavioral measures, were not reliable.
Scalp distribution of correlations between Mismatch-Match N400 (300-500 ms) and verification response time effects
In sum, participants differ systematically as to when they process the congruence: those who exhibit a large response time verification effect (e.g., 3, 10, 17, 19, 15, 22, and 9, see Figure 5) exhibit a small, or even reverse, congruence N400 at the verb (positive numbers on the x-axis reflect participants for whom incongruous verbs were more positive than congruous verbs). In contrast, participants that exhibit a small response time verification effect (e.g., 20, 6, 23, 8, 11, 14, 21) display a medium to large congruence N400 (i.e., a negative number on the x-axis; mismatching trials were more negative than matching trials) at the verb. Participants with large ‘‘early’’ congruence effects also tended to have higher accuracy (Figure 6b). Post-verbal ERP congruence effects were positively correlated with the response congruence effects: Participants that exhibit a large congruence
µV 0.07 −0.07
0.41
0.37 0.39 0.48 0.45
0.49 0.12
0.49
0.37
0.10
0.66
0.00
0.59 0.33 0.59
VERB
0.49
−0.18 −0.20
−0.22
−0.16 −0.24
−0.18
0.58
0.36 0.05
−0.23
0.23
−0.17 −0.14
−0.37
−0.28
−0.34
−0.19 −0.29
−0.15 −0.22
−0.37
−0.34
SECOND NOUN (PATIENT)
Figure 4. The r-values for correlations end-of-sentence response latency difference scores (mismatch minus match) and mean ERP amplitude difference scores at 20 electrodes (mismatch minus match) for the verb and the second noun in the 300–500 time windows. Lighter shades indicate larger positive correlations (i.e., a negative relationship) between verb and verification time congruence effects; darker shades depict negative correlations between second noun and verification time congruence effects. The r-values between individual sites were estimated by spherical spline interpolation.
Figure 5. Scatter plots of correlation coefficients between response latency difference scores and ERPs summarized for each participant: right lateral slice N400 verb.
Comprehending visual context influences
503
Table 3. Correlation Analyses for the Verb and Second Noun ERP congruence effect Left medial sites Verb (300–500 ms) Verification mean accuracy RT congruence effect VWM scores ECFT scores Second noun (400–600 ms) Verification mean accuracy RT congruence effect WVM scores ECFT scores
F 0.49n F F F F F F
Right lateral sites F 0.58nn F F F ! 0.49n F F
Behavioral measures Right medial sites
RT congruence effect
VWM scores
! 0.47n 0.57nn F F
! 0.45n
0.38# ! 0.47n
! 0.47n F
F
F F F F
Note: #po.1. n po.05; nnpo.01.
N400 at the second noun exhibit a large response time congruence effect; participants with a small congruence effect at the second noun exhibit a small response time congruence effect. The time course and accuracy with which a participant processes congruence further appears to be a function of his/her verbal working memory scores: the higher a participant’s verbal working memory score, the smaller his/her response latency
congruence effect and vice versa (see Table 3 and Figure 6a); and individuals with higher mean accuracy scores had a small, if any, response time congruence effect while those with lower mean accuracy scores had a large one. A participant’s verbal working memory score also correlated with his/her mean accuracy (the higher the VWM score, the higher mean accuracy, Table 3). Visual-spatial (ECFT) test scores, in contrast, were not reliably correlated with any ERP or verification time congruence effects. Standard Simple and Multiple Regression Analyses Next, we computed simple regression analyses (e.g., right lateral verb N400s), and also multiple regression analyses with ERPs to the verb and post-verbal regions (e.g., right lateral verb N400s, and right lateral ERP effects at the second noun) to examine how well these variables predict the response time congruence effect. We systematically explored a number of models (different combinations of mean amplitude ERP difference scores at the verb, object noun, and verbal working memory scores) and computed partial Fs to find the best-fitting model. The best-fitting model was one that combined right (but not left) lateral verb N400s and second noun N400s with VWM scores as predictors (right lateral: (F1(3,20) 5 11.11, po.001, r 5 0.79, adjusted R2 5 0.57, coefficient verb: t 5 2.82, po.05; coefficient second noun: t 5 ! 3.04, po.05; coefficient VWM: t 5 ! 2.66, po.05). This model accounted for substantially more variance than combinations of right lateral verb and second noun models (adjusted R2 5 0.44, partial F(1,21) 5 4.64, po.05), and the latter accounted again for more variance than simple (e.g., right lateral) verb models (r2 5 0.34, adjusted R2 5 0.31, partial F(1,21) 5 4.88, po.05). Follow-up ANOVAs: VWM To more fully examine the relationship between ERP and response latency congruence effects, we analyzed data based on a median split of participants’ VWM scores (informed by visual inspection).1 Specifically, we performed repeated measures
Figure 6. Scatter plots of correlation coefficients (a) between response latency difference scores and verbal working memory scores; and (b) between (right medial) verb ERP congruence effects and percentage of mean response accuracy for incongruous and congruous trials.
1 VWM score for one participantFwhile just below the median of ! 0.4876Fclustered (by eyeball and k-means clustering) with the above rather than below VWM median scores. Since k-means clustering results depend on the ordering of the scores, we performed clustering on 10 randomizations of participants’ VWM scores. Membership of participants in the high versus low VWM score cluster was highly consistent across multiple runs on randomized scores. The ‘low VWM’ cluster always had 11 participants and a final center at around ! 0.79; the ‘high VWM’ cluster contained scores from the remaining 13 participants with the final center around ! 0.40. The difference between the two groupings
504
P. Knoeferle et al.
Figure 7. Grand average ERPs (mean amplitude) at the verb and post-verbal regions for participants with high and low verbal working memory scores.
ANOVAs on the ERP and verification congruence scores separately for each VWM group. For the ANOVAs, we ensured that the data in the two groups were approximately normally distributed by inspecting skew and kurtosis values both separately for each group and across the two groups.
ANOVAS for low and high VWM groups. Analyses revealedFas expectedFa reliable verification time congruence effect only for the low VWM group (F1(1,10) 5 8.58, po.02, Z2 5 0.46; mean time congruous 1104 ms: mean time incongruous: 1337 ms; high VWM group: p4.1, Z2 5 0.17; mean time congruous: 1105 ms mean time incongruous: 1218 ms). Across conditions, the high VWM group’s average response time was faster (1162 ms) than the low VWM group’s average response time (1221 ms). Visual inspection of grand average difference waves (mismatch minus match) for the two VWM subgroups suggested a larger P2 difference for the high than for the low VWM group. The N400 congruence effect at the verb was only slightly larger for the high VWM group but seemed to arise earlier, and the more negative-going waveforms for the higher VWM participants increased during the post-verbal determiner and second noun especially over frontal and lateral sites (see Figure 7). In the ERP analyses, a reliable main effect of congruence between 180–220 ms, (the P2 region), for the high (F1(1,12) 5 8.5, po.02, Z2 5 0.41) but not for the low (Fo1) VWM group confirmed the visual impression. Both groups showed reliable verb N400 congruence effects between 300–500 ms (for low VWM, F1(1,10) 5 5.77, po.05, Z2 5 0.37; for high VWM, F1(1,12) 5 17.28, po.01, Z2 5 0.59). However, only the high VWM group showed a reliable main effect of congruence in the early half of the verb N400 (300–400 ms: F1(1,12) 5 8.72, po.02, Z2 5 0.42). For the second noun (300–500 ms), analyses revealed a reliable congruence by hemisphere interaction for the high VWM group only (F1(1,12) 5 5.00, po.05, Z2 5 0.29; low VWM group, Fo1). (clustering vs. median split) concerned only the one participant that was just below the median VWM but grouped more naturally with the above VWM group.
Discussion In this study, we assessed the suitability of the verification taskFinsofar as it reveals visual context influences on sentence comprehensionFas a paradigm for investigating situated language comprehension. We recorded ERPs along with post-sentence RTs during a picture-sentence verification task in which participants judged whether a simple subject-verb-object sentence, presented visually one word at a time, matched the preceding depicted action event involving two characters. We examined (a) whether the effect of the visual scene on sentence processing is just immediate and local (i.e., limited to the verb), and thus explicable in terms of simple priming, or manifest at multiple time points during sentence processing; (b) the relationships between verification time and ERP congruence effects (via correlations and linear regressions); and (c) the relationship of each of these congruence effects with VWM scores. From these, we determine the nature of the relationship between visual context influences on online comprehension and end-of-sentence verification times, in order to infer the time course of verification-related processes. Overall, the observed data pattern provides strong evidence against a simple priming account. First, we observed multiple reliable ERP congruence effects over the course of the sentence: At the earliest possible point in the sentence, namely the verb (300–500 ms), N400 amplitudes over centro-parietal scalp were larger for verb-action mismatches than matches. The centro-parietal N400 effect was indistinguishable from that of lexico-semantic anomalies or low cloze probability words in sentences read for comprehension (e.g., Kutas, 1993; see also Kutas, Van Petten, & Kluender, 2006; Otten & van Berkum, 2007; Van Berkum et al., 1999). We thus conclude that the verb N400 in the present study reflects visual context effects on language comprehension. Crucially, we also observed picture-sentence mismatch effects post-verbally, in the form of a negativity in response to the object noun (between 300–500 and 400–600 ms). Compared to the N400 at the verb, this negativity had a more anterior scalp distribution. While the functional significance of this post-verbal negativity is unclear (see Wassenaar & Hagoort, 2007 for comparison), the topographic difference between the verb and object noun ERP effects implicates at least two distinct congruence
Comprehending visual context influences
505
effects during the processing of these short sentences. It also provides some evidence against any simple action-verb priming account, which by itself provides no principled explanation for multiple, distinct ERP effects. Visual context effects also emerged, as expected, in post-sentence RTs, when participants gave their verification responses, with longer latencies for picture-sentence mismatches than matches. These multiple congruence effectsFthe verb N400 (reflecting visual context effects on comprehension), the object noun negativity (indexing further aspects of congruence processing), and the post-sentence RTs (widely presumed to index verification processes)Fcould, in principle, reflect the same (e.g., priming) processes; if so, we would expect these measures to co-vary directly. The pattern of correlations, however, is more complex than any single mechanism (e.g., priming) could readily accommodate: (1) there is an inverse correlation between the verb N400 congruence effect and the verification RTcongruence effect: the larger the N400 congruence effect, the smaller the RTcongruence effect; (2) the object noun ERP congruence effect, on the other hand, does correlate directly with the RT congruence effect: the larger the object noun congruence effect, the larger the RT congruence effect; and (3) moreover, the verb N400 and object noun congruence effects do not reliably correlate with each other, as might be expected if they were both consequences of the same cause. Multiple regression analyses corroborate the case against a simple priming account. The best predictor of verification time effects in a linear regression proved to be a combination of the verb N400 congruence effect plus the second noun congruence ERP effect (from right but not left hemisphere sites) plus verbal (and not visual) working memory scores. These three measures together account for 57% of the variance in verification RT effects. This is significantly greater than the variance accounted for either by the simple verb N400 effect (! 31%) or even by the significantly greater variance accounted for by the combination of the verb N400 congruence effect plus the second noun congruence effect (! 44%). Verbal working memory scores on their own also were a reliable predictor of the RT effects, but not of the N400 congruence effects. Nonetheless, separate analyses confirmed earlier congruence effects in the high span comprehenders (P2 and early N400 region) relative to the low span comprehenders (late N400 region and in the RTs). In the absence of any correlations with visual-spatial scores from the Extended Complex Figure Test, there is thus no evidence that the time course differences are related to participants’ differential abilities in recognizing the depicted actions. We speculate that the high VWM group may have used the visual scene more rapidly as a source of contextual information. This account is compatible with the proposal that individuals with high verbal working memory may be more motivated and/or attentive (see Chwilla, Brown, & Hagoort, 1995). To reiterate, these results taken all together cannot be accounted for by any simple priming account. Furthermore, they also run counter to any view that relegates all verification-related
processes to some time separate from, and after, comprehension (Tanenhaus et al., 1976). The exact timing of verification-related processes that contribute to the post-sentence verification times does seem to vary across individuals as a function of (at least) their verbal working memories. These individual differences not withstanding, the data are clear in demonstrating that verification-related processes, like comprehension processes, are distributed in time and incremental in nature. Indeed, the first ERP indicant of a picture-sentence mismatch not only occurs at the earliest moment it could (at the verb) in a component that is known to co-vary with comprehension (i.e., posterior N400) but is also predictive of post-sentence verification times. A qualitatively different contribution to verification RTs is evidenced in the ERP congruity effects at the second noun. Our findings thus support the methodological criticism that end-of-sentence verification times alone underdetermine models of the time course of congruence processing in such verification tasks (see Kounios & Holcomb, 1992; Tanenhaus et al., 1976). It is only the combination of continuous measures (e.g., eye tracking or ERPs) and RTs that permits us not only to validate the paradigm but to better delineate the time course of visual context effects. The multiple systematic relationships between a participant’s incremental ERP congruence effects and their verification response time pattern at sentence end indicate that overt picturesentence verification decisions post sentence are some function of how picture-sentence congruence is processed during incremental sentence comprehension, at least when people are encouraged to understand both the pictures and sentences. The variation of these effects with verbal working memory scores shows further that individual differences play a systematic role in the cognitive processes at work here, as elsewhere in sentence comprehension (e.g., Daneman & Carpenter, 1980; Daneman & Merikle, 1996; Just & Carpenter, 1992; Waters & Caplan, 2001). A better understanding of comprehension-verification relationships is important for theories of (situated) language comprehension, since verification (overt responses on correspondence between what is said and how things are) appears to be part and parcel of routine language communication, and is thus a potentially pervasive mechanism. Positive verification is evident in expressions of agreement (‘‘So I heard,’’ ‘‘No doubt’’) while failures to verify may be inferred from corrections and expressions of disbelief, uncertainty, requests for clarification, and the like (e.g., ‘‘Well no, actually what happened was . . .’’, ‘‘Are you sure?’’). In sum, the present data re-establish the validity of the picture-sentence verification paradigm for the study of online language comprehensionFprovided post sentence RTs are complemented by continuous measuresFthereby paving the way for investigations of comprehenders’ reliance on representations of linguistic and non-linguistic visual context even when they do not perfectly match those of the current linguistic input.
REFERENCES Altmann, G. T. M. (2004). Language-mediated eye movements in the absence of a visual world: The blank screen paradigm. Cognition, 93, B79–B87. Barrett, S. E., & Rugg, M. D. (1990). Event-related potentials and the semantic matching of pictures. Brain and Cognition, 14, 201–212. Carpenter, P. A., & Just, M. A. (1975). Sentence comprehension: A psycholinguistic model of verification. Psychological Review, 82, 45–76.
Carpenter, P. A., & Just, M. A. (1976). Models of sentence verification and linguistic comprehension. Psychological Review, 83, 318–322. Chambers, C. G., Tanenhaus, M. K., & Magnuson, J. S. (2004). Actions and affordances in syntactic ambiguity resolution. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 687–696. Chwilla, D. J., Brown, C., & Hagoort, P. (1995). The N400 as a function of the level of processing. Psychophysiology, 32, 274–285.
506 Clark, H. H., & Chase, W. G. (1972). On the process of comparing sentences against pictures. Cognitive Psychology, 3, 472–517. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum. Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user’s guide. Psychonomic Bulletin & Review, 12, 769–786. Dale, A. M. (1994). Source localization and spatial discriminant analysis of event-related potentials: Linear approaches. Unpublished doctoral dissertation, University of California, San Diego. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450–466. Daneman, M., & Merikle, P. M. (1996). Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin & Review, 3, 422–433. D’Arcy, R. C. N., & Connolly, J. F. (1999). An event-related brain potential study of receptive speech comprehension using a modified Token Test. Neuropsychologia, 37, 1477–1489. Fasteneau, P. S. (2003). Extended complex figure test. Los Angeles, CA: Western Psychological Services. Federmeier, K. D., & Kutas, M. (1999). A rose by any other name: Longterm memory structure and sentence processing. Journal of Memory and Language, 41, 469–495. Federmeier, K. D., & Kutas, M. (2002). Picture the difference: Electrophysiological investigations of picture processing in the cerebral hemispheres. Neuropsychologia, 40, 730–747. Fischler, I., Bloom, P. A., Childers, D. G., Roucos, S. E., & Perry, N. W. J. (1983). Brain potentials related to stages of sentence verification. Psychophysiology, 20, 400–409. Goolkasian, P. (1996). Picture–word differences in a sentence verification task. Memory and Cognition, 24, 584–594. Gough, P. B. (1965). Grammatical transformations and speed of understanding. Journal of Verbal Learning & Verbal Behavior, 4, 107–111. Griffin, D., Murray, S., & Gonzalez, R. (1999). Difference score correlations in relationship research: A conceptual primer. Personal Relationships, 6, 505–518. Holcomb, P. J., & Neville, H. J. (1991). Natural speech processing: An analysis using event-related brain potentials. Psychobiology, 19, 286–300. Hutcheson, G., & Sofroniou, N. (1999). The multivariate social scientist: Introductory statistics by using generalized linear models. Thousand Oaks, CA: Sage. Johns, G. (1981). Difference score measures of organizational behavior variables: A critique. Organizational Behavior and Human Performance, 27, 443–463. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122–149. Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the immediate visual context on incremental thematic role assignment: Evidence from eye movements in depicted events. Cognition, 95, 95–127. Knoeferle, P., & Crocker, M. W. (2006). The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking. Cognitive Science, 30, 481–529. Knoeferle, P., & Crocker, M. W. (2007). The influence of recent scene events on spoken comprehension: Evidence from eye movements. Journal of Memory and Language, 57, 519–543. Knoeferle, P., Habets, B., Crocker, M. W., & Mu¨nte, T. F. (2008). Visual scenes trigger immediate syntactic reanalysis. Cerebral Cortex, 18, 789–795. Kounios, J., & Holcomb, P. J. (1992). Structure and process in semantic memory: Evidence from event-related brain potentials and reaction times. Journal of Experimental Psychology: General, 121, 459–479. Kutas, M. (1993). In the company of other words: Electrophysiological evidence for single word versus sentence context effects. Language and Cognitive Processes, 8, 533–572. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203–205.
P. Knoeferle et al. Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307, 161– 163. Kutas, M., & Van Petten, C. (1988). Event-related brain potential studies of language. In P. K. Ackles, J. R. Jennings, & M. G. H. Coles (Eds.), Advances in psychophysiology (Vol. 3, pp. 139–187). Greenwich, Connecticut: JAI Press. Kutas, M., Van Petten, C. K., & Kluender, R. (2006). Psycholinguistics electrified II. In M. A. Gernsbacher & M. Traxler (Eds.), Handbook of psycholinguistics (2nd edition, pp. 659–724). New York: Elsevier Press. Mayberry, M., Crocker, M. W., & Knoeferle, P. (2009). Learning to attend: A connectionist model of situated language comprehension. Cognitive Science, 33, 449–496. McCallum, W., Farmer, S. F., & Pocock, P. V. (1984). The effects of physical and semantic incongruities on auditory event-related potentials. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section, 5, 477–488. Meyer, D. E., & Schvaneveldt, R. W. (1971). Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 90, 227–234. Otten, M., & Van Berkum, J. J. A. (2007). What makes a discourse constraining? Comparing the effects of discourse message and scenario fit on the discourse-dependent N400 effect. Brain Research, 1153, 166–177. Reichle, E. D., Carpenter, P. A., & Just, M. A. (2000). The neural basis of strategy and skill in sentence-picture verification. Cognitive Psychology, 40, 261–295. Sedivy, J. C., Tanenhaus, M. K., Chambers, C. G., & Carlson, G. N. (1999). Achieving incremental semantic interpretation through contextual representation. Cognition, 71, 109–148. Singer, M. (2006). Verification of text ideas during reading. Journal of Memory and Language, 54, 574–591. Spivey, M., Tanenhaus, M. K., Eberhard, K., & Sedivy, J. (2002). Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45, 447–481. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643–662. + D., Solte´sz, F., Czigler, I., & Cse´pe, V. (2007). ElectroencephaloSzucs, graphy effects to semantic and non-semantic mismatch in properties of visually presented single-characters: The N2b and the N400. Neuroscience Letters, 412, 18–23. Tabachnik, B., & Fidell, L. S. (2007). Using multivariate statistics. Pearson Allyn & Bacon: Boston. Tanenhaus, M. K., Carroll, J. M., & Bever, T. G. (1976). Sentencepicture verification models as theories of sentence comprehension: A critique of Carpenter and Just. Psychological Review, 83, 310–317. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634. Underwood, G., Jebbett, L., & Roberts, K. (2004). Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search. The Quarterly Journal of Experimental Psychology, 56, 165–182. Van Berkum, J. J. A., Hagoort, P., & Brown, C. M. (1999). Semantic integration in sentences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience, 11, 657–671. Vissers, C., Kolk, H., Van de Meerendonk, N., & Chwilla, D. (2008). Monitoring in language perception: Evidence from ERPs in a picturesentence matching task. Neuropsychologia, 46, 967–982. Wassenaar, M., & Hagoort, P. (2007). Thematic role assignment in patients with Broca’s aphasia: Sentence-picture matching electrified. Neuropsychologia, 45, 716–740. Waters, G. S., & Caplan, D. (2001). Age, working memory, and online syntactic processing in sentence comprehension. Psychology and Aging, 16, 128–144.
(Received November 23, 2009; Accepted May 24, 2010)
Psychophysiology, 48 (2011), 507–514. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01081.x
Why humans deviate from rational choice
JOHANNES HEWIG,a,d NORA KRETSCHMER,a RALF H. TRIPPE,a HOLGER HECHT,a MICHAEL G. H. COLES,b CLAY B. HOLROYD,c and WOLFGANG H. R. MILTNERa a
Department of Biological and Clinical Psychology, Friedrich–Schiller–University, Jena, Germany Donders Centre for Cognitive Neuroimaging, Nijmegen, The Netherlands c Department of Psychology, University of Victoria, Victoria, British Columbia, Canada d Department of Psychology I, Julius–Maximilians–University Wu¨rzburg, Wu¨rzburg, Germany b
Abstract Rational choice theory predicts that humans always optimize the expected utility of options when making decisions. However, in decision-making games, humans often punish their opponents even when doing so reduces their own reward. We used the Ultimatum and Dictator games to examine the affective correlates of decision-making. We show that the feedback negativity, an event-related brain potential that originates in the anterior cingulate cortex that has been related to reinforcement learning, predicts the decision to reject unfair offers in the Ultimatum game. Furthermore, the decision to reject is positively related to more negative emotional reactions and to increased autonomic nervous system activity. These findings support the idea that subjective emotional markers guide decision-making and that the anterior cingulate cortex integrates instances of reinforcement and punishment to provide such affective markers. Descriptors: Decision-making, Microeconomics, Feedback negativity, Somatic markers, Ultimatum game
offers since receiving at least some money is always preferable to receiving no money. However, empirical evidence shows that humans often deviate from rational choice to varying degrees in such experiments (Camerer, 2003; Gu¨th, Schmittberger, & Schwarze, 1982). While some participants act rationally and accept all offers, other participants reject offers that deviate even slightly from an equal distribution. The latter might be motivated by negative affective responses to unfairness (Nowak, Page, & Sigmund, 2000; van ‘t Wout, Kahn, Sanfey, & Aleman, 2006). Indeed, previous research indicates the importance of affective processes in Ultimatum decisions. For example, in a study by Pillutla and Murnighan (1996), participants who reported more anger rejected more offers. More recently, Harle and Sanfey (2007) showed that participants in the Ultimatum game who were exposed to a sadness induction before playing rejected relatively more fair offers, demonstrating that decision-making can be influenced by negative emotions that are unrelated to the task at hand. Furthermore, in another study, the skin conductance response to unfair offers, an autonomic measure of affect, also predicted rejection of unfair offers in the Ultimatum game (van ‘t Wout et al., 2006). Skin conductance responses have been repeatedly associated with the reaction to aversive stimuli (e.g., Hajcak, McDonald, & Simons, 2004; Meriau, Wartenburger, Kazzer, Prehn, Villringer, et al., 2009). Furthermore, electrodermal activity is believed to be related to physiological arousal elicited by the behavioral inhibition system (Fowles, 1980), which, in turn, is supposed to be the biological basis of punishment and negative affect (Gray, 1982, 1994). In addition, unfair offers in the Ultimatum game activate brain regions that are associated with negative emotional reactions, such as the insula (Sanfey, Rilling, Aronson, Nystrom, & Cohen, 2003; Tabibnia,
In the Ultimatum and Dictator games, a ‘‘proposer’’ and a ‘‘responder’’ play against each other. In each case, the proposer is instructed to divide 12 cents into 2 shares, the proportion of which can range between 6:6 and 11:1. In the Ultimatum game, the responder is required to decide whether or not to accept the proposer’s offer. If the responder accepts the partitioning, then each player receives the money offered by the proposer. If the responder rejects the offer, then neither player receives any money. Accordingly, this provides an opportunity for the receiver to punish the proposer for unfair offers. In the Dictator game, the proposer’s role is the same but the responder cannot reject the offer. Thus the responder has no option to reject unfair offers and the money is always apportioned between the proposer and the responder as the proposer dictates. In repeated games, one and the same ‘‘proposer’’ and ‘‘responder’’ interact several times whereas, in one-shot interactions, each responder is confronted with a certain proposer only once. In the one-shot Ultimatum game, economic rationality of utility as derived from classical game theory (von Neumann & Morgenstern, 1953) predicts that the responder should accept all This research was funded by a grant from the Friedrich–Schiller– Universita¨t Jena and a Schumpeter-Fellowship by Volkswagen Foundation to Johannes Hewig. The Alexander von Humboldt Foundation supported Michael Coles’ participation in this project. Thanks to John Allen for constructive comments on the manuscript. The authors thank Anett Siebenmorgen, Pino Nagel, Annelie Tuchscherer, Saskia Scho¨ler, and Melanie Spate for their assistance in the execution of the study. Address correspondence to: PD Dr. rer. nat. Johannes Hewig, Lehrstuhl fu¨r Biologische und Klinische Psychologie, Friedrich–Schiller– Universita¨t Jena, Am Steiger 3, Haus 1; D-07743 Jena, Germany. E-mail:
[email protected] 507
508 Satpute, & Lieberman, 2008). Moreover, the fMRI data from Sanfey and colleagues show that anterior cingulate cortex is activated by unfair offers, which is consistent with the very recent report of an increased negativity in the event-related potential (ERP) between 240 and 320 ms in response to unfair Ultimatum offers for receivers in a repeated Ultimatum Game against one and the same proposer (Polezzi, Daum, Rubaltelli, Lotto, Civai, et al., 2008). This negativity may be classified as a feedback negativity because of its topography and timing (Holroyd & Coles, 2002; Miltner, Braun, & Coles, 1997). Most importantly, we suggest that participants will generate expectations in each trial about the offer to be made by the other person. The presentation of the actual offer represents feedback as to whether the monetary outcome is equal to, or better or worse than, the participants’ expectation. The feedback negativity (FN, also known as the feedback error-related negativity/feedback ERN) is a negative deflection in the ERP, the maximum amplitude of which is recorded at the scalp over frontal brain regions at about 250–300 ms following negative as compared to positive performance feedback (Holroyd & Coles, 2002; Miltner et al., 1997) or losses as compared to wins in gambling situations (Hewig, Trippe, Hecht, Coles, Holroyd & Miltner, 2007; Nieuwenhuis, Yeung, Holroyd, Schurger, & Cohen, 2004; Yeung & Sanfey, 2004). Holroyd and Coles (2002) have suggested that the process manifested by the FN is involved in reinforcement learning. In terms of reinforcement learning theory, more negative amplitudes to losses are related to punishment when events are worse than expected. Punishment leads to negative affective responses and is linked with the concept of habitual or trait-like differences in negative affect (Gray, 1982, 1994). Thus, greater FN amplitudes indicating punishment should evoke stronger negative affect. In contrast, more positive amplitudes are thought to be related to a reinforcement response when upcoming events are better than expected (Holroyd, Pakzad-Vaezi, & Krigolson, 2008). Amplified FN amplitudes in response to aversive events have been found for participants who are sensitive to aversive stimuli or negative affect, such as those high in neuroticism, in trait negative affect, or behavioral inhibition (Boksem, Tops, Wester, Meijman, & Lorist, 2006; Hajcak et al., 2003; Hajcak, McDonald, & Simons, 2004; Luu, Collins, & Tucker, 2000; Pailing & Segalowitz, 2004). In the context of the Ultimatum game, greater FN amplitudes to unfair offers should reflect more negative responses and should lead to an increased likelihood of remedial action in terms of rejection of these unfair offers. In the present study, we adopted a multilevel approach to examine both basic mechanisms and individual differences in decision-making in the one-shot Ultimatum and Dictator Games. We collected data on behavior, subjective affect, and central and autonomic nervous system activity to reveal the contribution or influence of multiple levels of affective processing to decision-making in these games. Specifically, we expected that greater negative affective valence, larger skin conductance responses, and larger FN amplitudesFall indicating negative affective processingFwould predict larger deviations from rational choice. This finding would provide direct evidence that individual differences in affective processing contribute to differences in economic decision-making. It would further allow us to analyze the relative contribution of, and the relationship among, different aspects of affective processing that have been shown to contribute significantly to decision-making in the Ultimatum Game (Polezzi et al., 2008; van ‘t Wout et al., 2006).
J. Hewig et al. Methods Participants Thirteen participants were recruited from the student population of the Friedrich Schiller University. The data of one participant were excluded because he/she did not believe in the cover story of the experiment and suspected that he/she had played the games against a computer instead of a real human player. All others denied having any such suspicion when asked at debriefing. The remaining 12 participants (8 females and 4 males; mean age: 21.6 years, SD 5 ! 1.5 years, range 20–25 years) were paid h6 per hour plus an extra bonus that varied between h10.01 and h12.87 (Mean 5 11.81; SD 5 ! 0.98) according to their decisions in the games. After receiving verbal instructions about the experiment, participants gave written consent for participation. Task and Procedure Each participant played both the Ultimatum and the Dictator games repeatedly in a series of one-shot trials as a proposer and as a receiver. In each trial, a proposer is instructed to divide a fixed amount of money (here 12 cents) into two shares: one for him- or herself and the other for the responder. In the Ultimatum game, the responder is prompted to decide whether or not he or she accepts the offer of the proposer. If the responder accepts the offer, then each player receives the money assigned by the proposer. If the responder rejects the offer, then no money is given to either player. In the Dictator game, the responder cannot reject the offer and the money is always assigned to both players as dictated by the proposer. First, the participants acted in the role of the proposer in both games. Then they switched roles and became responders. In the role of the proposer, participants made 40 Ultimatum and 10 Dictator proposals by typing in their proposals via a PC. They were told that their proposals would be stored and used for future participants and that they would receive only one offer from a particular proposer when they later played the role of responder. In addition, a photograph was taken of each participant, which was used to enhance the plausibility of the cover story. This picture and that of the virtual proposer were presented following the feedback in the responder games. Figure 1 shows a single trial in the responder condition of the Ultimatum game. After playing the game as proposers, the participants were prepared for recording of electroencephalogram (EEG) and skin conductance (see section on EEG and skin conductance responses (SCR) recording and quantification). Then, participants played the games in the role of the responder and received a randomized series of 240 Ultimatum game offers (40 for each of 6 conditions: 6:6, 5:7, 4:8, 3:9, 2:10, 1:11). Each trial started with the presentation of a fixation cross (750 ms). This was followed by a divided color bar (Figure 1) that indicated the amount of the offer. The length of the blue portion indicated the amount offered by the proposer, while the red portion indicated the amount retained by the proposer. Four hundred milliseconds later, a tone (100 ms duration, 800 Hz) prompted the participants to respond and to either accept or reject the offer within 2 s. Immediately after their button press response, the amount of money that the participant would receive on that trial was presented for 600 ms. Finally, a photo of the participant and the pseudo-proposer on that trial were presented for 1 s together with the amount of money received by each one of them and the cumulative amount of the participant’s winnings. On each trial, a different proposer was presented, chosen randomly from a set of photographs either
Irrational decisions
Figure 1. Timeline of a single trial in the Ultimatum game.
taken from preceding participants or from an archive of face images. No proposer’s image was presented more than once throughout the entire experiment. The picture was shown after each trial in order to avoid confounding influences of the gender of the opponent on decision-making (Solnick & Schweitzer, 1999). The Ultimatum game was followed by 60 trials of the Dictator game with 10 trials for each of the 6 conditions. The timing of each trial was the same as in the Ultimatum game. In addition, after the participants finished all trials of each game, they completed a subjective rating of the emotional valence (from 1/negative to 9/positive) that they had experienced in each condition (6:6 through 11:1) while playing the role of responder. EEG and Skin Conductance Recording and Quantification EEG and SCRs were measured when participants were in the role of the responder. Participants were seated individually in an electrically shielded, dimly lit, and temperature controlled EEG cabin, and Ag/AgCl electrodes were applied for the measurement of electro-oculogram (EOG) and EEG. The EEG montage of electrodes was realized by the Brain-Cap MR 128-channel electrode system (EasyCap, Munich Germany) and included all electrodes according to the extended 10–20 system (128 electrode sites) referenced to vertex (Cz). Additionally, vertical EOG activity was recorded from an electrode fixed under the left eye. All electrode sites were cleaned with alcohol and gently abraided prior to electrode application to keep the impedances of electrodes below 5 kO, and the differences of impedance between homologous sites below 1 kO. EEG and EOG were amplified with four 32-channel DC BrainAmp MR plus amplifiers (Brain Products, Munich, Germany; input impedance: 10 MO). Bandpass filter was set to 0.015–250 Hz; the signals were digitized online at 500 Hz and stored on hard disk for later off-line analyses. After data acquisition, EOG and EEG recordings were subjected to off-line ocular correction, and automatic artifact correction procedures were performed using the Vision Analyzer software (BrainProducts). Trials with response times greater than 2000 ms were discarded from all analyses (mean: 3.75, SD: 5.72). Data for each electrode were filtered (high cut-off: 20 Hz), epoched from ! 150 ms to 1400 ms following stimulus onset (presentation of the color bar), and baseline corrected using the average activity of the 100 ms preceding the offer onset. Finally, EEG waveforms were averaged separately for each participant, each experimental condition, and each electrode. The peak amplitude of the FN to the presentation of the offer was defined as the average between 280 and 320 ms at electrode Fz and was determined for each participant and each exper-
509 imental condition. Additionally, difference waves between fair (6:6) and unfair (11:1) offers were calculated as the mean of the difference in the time window of 280 to 320 ms. In addition, P3 amplitude was defined as the average amplitude between 350 and 450 at electrode Pz (Coles & Rugg, 1995). The analysis of variance (ANOVA) of EEG data (FN amplitudes) associated with offer presentation included the factors Fairness (6 levels: 6:6 to 11:1) and Game (Ultimatum versus Dictator). Additionally, two factors for the topography of brain electrical activity were used (AnteriorF5 levels: frontal, frontocentral, central, centroparietal, parietal; and LateralityF5 levels: lateral left, left, midline, right, lateral right), which included the following 25 channels: F3, F1, Fz, F2, F4, FC3, FC1, FCz, FC2, FC4, C3, C1, Cz, C2, C4, CP3, CP1, CPz, CP2, CP4, P3, P1, Pz, P2, and P4, all referenced to linked mastoids. For topographical analyses, all electrodes were used. Huynh-Feldt correction was applied as appropriate to protect against violations of sphericity. Skin conductance was recorded from the sole of the left foot with Ag/AgCl electrodes (6 mm diameter and 0.28 cm2 recording area) during application of a constant voltage of 0.5 Volt using a VARIOPORT-C skin conductance amplifier (Becker Meditech, Karlsruhe, Germany). The measurement range was set to 50 mS with a resolution of 0.002 mS. The data were high-pass filtered (0.1 Hz). The single trial data were baseline corrected ( ! 1 to 0 s pre-stimulus). The maximum of the skin conductance response was detected automatically between 1.5 and 7.5 s after the presentation of the offers. Only positive values were accepted as valid changes of electrodermal activity (EDA). A Fairness (6 levels; 6:6 to 11:1) " Game (Ultimatum versus Dictator) ANOVA was performed on the skin conductance responses. Results Behavior We first examined the general effect of the fairness of offers by the proposer on the responders’ behavior in the Ultimatum game. Fairness was defined in terms of the proportional offer by the proposer with 11:1 being most unfair and 6:6 being most fair. An ANOVA revealed a main effect of Fairness (F(5,55) 5 31.37, po.001, Z2 5 .74). Figure 2A indicates that the probability of the responder accepting an offer in the Ultimatum game decreased as a function of unfairness. Offers of 9:3 were accepted significantly more often (p 5 .005) and offers of 11:1 were rejected more often (p 5 .004) than offers of 10:2. Offers of 10:2 were accepted by responders on about 50% of trials and were associated with the highest variance across participants, in accordance with previous results (Camerer, 2003). Affect Participants provided ratings of valence (on a Likert Scale, from very negative 1 to very positive 9) to the monetary offers in both games. The inclusion of the Dictator Game made it possible to examine whether the absence (Dictator) or presence (Ultimatum) of the response choice influenced the dependent variables. An ANOVA with the factors Game (2 levels; Ultimatum versus Dictator) and Fairness (6 levels; 6:6 to 11:1) on these ratings revealed a significant effect of Fairness on the valence ratings (F(5,55) 5 82.92, po.001, Z2 5 .88) and no significant main effect of game or interactions with game (Ultimatum versus Dictator, all values of p4.282). Accordingly, when confronted with increasingly unfair offers in both games, participants reported more negative emotional reactions (Figure 2B).
510
J. Hewig et al. higher skin conductance responses as compared to extremely fair (6:6) offers (p 5 .015). Data were collapsed across games because of an absence of significant main or interaction effects involving Game (all values of p4.131). Electrophysiology An ANOVA on FN amplitudes elicited by the offer with the factors Fairness, Game, and Electrode Position revealed a significant interaction between Fairness and Electrode Position (Fairness ! Anterior: F(20,220) 5 3.04, p 5 .011, Z2 5 .22). Post-hoc tests of this interaction showed a linear trend with more negative amplitudes for increasingly unfair offers at more anterior sites indicating stronger FN amplitudes for unfair as compared to fair offers (F(1,11) 5 6.20, p 5 .030, Z2 5 .36). Data were collapsed across games because of an absence of significant main or interaction effects involving Game (all values of p4.2). Figure 3A depicts the ERP waveforms for extremely unfair (11:1) versus fair (6:6) offers, averaged across games. The associated difference wave between unfair and fair offers, illustrated in Figure 3A, reflects the pure effect of fairness on FN and, consistent with classical definitions of FN, exhibits a frontocentral topographical distribution over the scalp as shown in Figure 3B. An ANOVA on the P3 data revealed a main effect of Fairness (F(5,55) 5 3.36, p 5 .011, Z2 5 .23) and a quadratic trend in post-hoc contrasts (F(1,11) 5 5.38, p 5 .041, Z2 5 .33). The P3 was largest for extremely unfair offers (11:1; M 5 5.04 mV), and declined with fairness showing a minimum for 9:3 offers (M 5 2.26 mV) and then increased again with increasing fairness (for 6:6; M 5 3.01 mV). Thus, P3 showed a different effect pattern as compared to FN. There were no other significant main or interaction effects of Game or Fairness for the P3 data (all values of p4.062).
Figure 2. Unfairness is related to rejections of offers, negative emotions, and the amplitude of the skin conductance response. (A) Mean probabilities of ‘‘accept’’ decisions across all participants and for all decisions in the Ultimatum game show a decrease in acceptance with increasing unfairness. (B) Subjective ratings of emotional valence for each level of fairness across Ultimatum and Dictator games show more negative emotional responses with increasing unfairness (1 extremely negative to 9 extremely positive). (C) Skin conductance responses for each level of fairness across Ultimatum and Dictator games show an increase in the response with increasing unfairness.
Electrodermal Responses An analysis of the skin conductance responses revealed a marginally significant main effect of fairness (F(5,55) 5 2.65, p 5 .072, Z2 5 .19) and a significant linear contrast effect of fairness (F(1,11) 5 5.24, p 5 .043, Z2 5 .32), indicating greater skin conductance responses as a function of increasing unfairness (see Figure 2C). In particular, extremely unfair offers (11:1) elicited
Individual Difference Analyses The previous analyses showed that rejection rates, negative emotional reactions, FN amplitudes, and skin conductance responses were all inversely related to the fairness of the offers. Subsequent correlation analyses investigated the factors associated with individual differences in the responders’ choice behavior in the Ultimatum game. Hypotheses about correlations with other measures were tested with one-tailed tests. Because the betweenparticipant variability in the decision to accept the 10:2 offers was largest, it most clearly reflects individual differences in decisionmaking. According to rational choice theory, the more likely a person rejects these offers the greater is the deviation from rationality. The data associated with this condition were evaluated in subsequent individual difference analyses. In these analyses, for each participant, we averaged the ratings of subjective valence across the Fairness and Game conditions to provide an overall measure of each individual’s affective response to outcomes in general, since the ratings in different conditions were highly correlated. The reliability of the aggregated measure was .91 (Cronbach’s alpha). Because skin conductance responses in each condition were also highly correlated across the Ultimatum and Dictator games, we also used an aggregated measure for subsequent correlation analyses (Cronbach’s alpha: .97). This aggregated measure reflected the responsiveness of the autonomic nervous system of each participant. We further utilized individual differences in FN to examine its relationship with decision-making and affect. For the analysis of the FN, the FN was measured at its maximum at channel Fz by evaluating the ERPs elicited by fair (6:6) and unfair offers (11:1) as well as their difference in the
Irrational decisions
511
Figure 4. Results of a multiple regression in which the number of rejections of the 10:2 offer is predicted by ERP amplitudes to fair offers, subjective emotional ratings (Valence), and skin conductance responses (EDA). These variables together explain 84% of the variance in number of rejections (for more details, see Table 1).
Figure 3. The effect of fairness on event-related potentials and their topography after the presentation of fair versus unfair offers across games. (A) ERPs for 11:1 and 6:6 offers and the difference waveform. Data for frontal (Fz), central (Cz), and parietal (Pz) electrodes are presented. A greater negativity at frontal sites is present for unfair as compared to fair offers at 290 ms. (B) The topographic current source density map of the peak of the difference waveform shows the distribution of the feedback negativity over the cortex. The data were band-pass filtered to exclusively show FN related theta activity for the figure only (3–8 Hz).
Ultimatum Game. Recent evidence (Holroyd et al., 2008) shows that the main effect in ERP analyses of FN amplitudes is due to the reduction of FN after good outcomes (fair offers in the present study) rather than increased FN to bad outcomes (here unfair). Thus, we examined the separate contribution of FN responses to extremely fair (6:6) and unfair (11:1) offers in subsequent analyses in addition to the difference amplitudes. The correlation analyses addressed the question of the relationship between subjective emotional responses, FN amplitudes, and skin conductance responses, on the one hand, and the degree of participant rationality, as defined in terms of their responses to the 10:2 offers. Results show that the responders who more frequently rejected the 10:2 offer rated the proposers’ offers as emotionally more negative in general (r 5 ! .61, p 5 .017). Larger skin conductance responses also predicted higher rejection rates (r 5 .56, p 5 .028). Those who tended to reject the 10:2 offer also tended to show larger FN amplitudes to unfair (11:1) versus fair (6:6) offers in the Ultimatum game (r 5 ! .44, p 5 .078). While FN amplitudes to 11:1 offers were not related to rejection rate (r 5 ! .04), reduced FN amplitudes to 6:6 offers in the Ultimatum Game were significantly related to the rejection rate, with high rejection rates being associated with smaller FN amplitudes (r 5 .58, p 5 .023). For FN amplitudes to offers of 10:2, 9:3, 8:4, and 7:5, there was no significant correlation with rejections (all values of p4.17). Accordingly, in subsequent multiple regression analyses the FN amplitude to fair offers was used. As shown in Figure 4, a multiple regression analysis on individual differences in 10:2 rejection rates revealed that FN amplitudes to fair offers, SCR amplitude, and emotional valence ratings together accounted for a large proportion of variance of the behavioral data (R2 5 .84, F(3,8) 5 13.7, p 5 .002). The stepwise introduction of each of these three predictors revealed that each predictor explained a substantial independent amount of variance (see Table 1). Emotional ratings and skin conductance responses explained 38% and 35% of variance, respectively. FN introduced as the third predictor further explained a marginally significant but substantial amount of additional variance (11%). In order to further verify these results, we used a Jackknife method repeating the multiple regression analysis 12 times, omitting a different participant in each analysis (Tukey, 1958). The
512
J. Hewig et al.
Table 1. Results of the Multiple Regression Analysis Change statistics Model a b c
R
R2
corrR2
.614 .377 .855 .731 .915 .837
.315 .672 .776
SE
DR2
DF
df 1 df 2 p of DF
12.56 .377 6.05 8.69 .354 11.87 7.18 .106 5.21
1 1 1
10 9 8
.034 .007 .052
Notes. Model a. Variables: (constant), Valence. Model b. Variables: (constant), Valence, EDA. Model c. Variables: (constant), Valence, EDA, ERP at Fz for 6:6 in Ultimatum. Dependent variable: Rejections at 10:2 offers. R and R2 multiple correlation and explained variance (corrR2 5 corrected); SE 5 standard error, DR2 5 change in R2 specific for this variable, DF 5 change in F, df 5 degrees of freedom (nominator and denominator), p of DF 5 significance of change in DF.
standardized beta weights were consistent for all analyses. The weights varied between ! .52 and ! .74 for the influence of the subjective emotional ratings, between .37 and .66 for the SCR scores, and between .13 and .45 for the ERP amplitudes (see Table 2). Exploratory analyses of the relationship between P3 amplitudes and variables discussed in this section failed to reveal any significant effects. Further, analysis of the behavior of the participant when acting as proposer failed to reveal any significant relations with their behavior as responder or any of the variables mentioned above (all values of p4.5).
Discussion We have shown that unfair offers in the one-shot Ultimatum game were rejected more frequently, evoked more negative subjective emotional ratings, led to greater SCRs, and elicited larger FNs, than fair offers. Moreover, individual differences in the number of rejected 10:2 offers in the Ultimatum Game could be explained to a large extent by a combination of subjective emotional ratings, SCRs, and reduced FN amplitudes to fair offers. Our results are consistent with previous work that has reported finding similar relationships between emotional responses and rejection rate (Pillutla & Murnighan, 1996), between SCRs and rejection rate (van ‘t Wout et al., 2006), and between FN amplitudes and rejection rate in a repeated Ultimatum game (Polezzi et al., 2008). In particular, by measuring these three different correlates of rejection rates within the same experimental context,
Table 2. Correlation Between Variables and beta Weights in the Multiple Regression
we have shown that smaller FNs to fair offers, more negative emotional ratings, and larger SCRs predicted more rejections, that is, larger deviations from rational choice. The multiple regression analysis indicated that each of these three indicators of affective processing contributed an independent additional portion of variance (see Table 1). The physiological variables were uncorrelated with the subjective emotional responses (see Table 2), indicating that conscious emotional experience can be independent from physiological responses.1 On the level of the physiological variables, the regression analysis might suggest that two systems involved in affective processing contribute independently to influence irrational decision-making. First, the autonomic nervous system as reflected by SCRs has been associated with activation of the amygdala (e.g., Davis, 1992; Furmark, Fischer, Wik, Larsson, & Fredrikson, 1997; Phelps & LeDoux, 2005). The amygdala isFamong other structuresFsaid to control SCRs through the regulation of the sympathetic nervous system. For example, Furmark et al. (1997) found a significant relation between regional cerebral blood flow in the amygdala and electrodermal fluctuations. It has to be noted that SCRs and feedback negativity were correlated (see Table 2). This is in line with findings that anterior cingulate is also involved in electrodermal control (Fredrikson, Furmark, Olsson, Fischer, Andersson, & Langstrom, 1998). However, the regression analyses revealed that the FN explained an additional independent amount of variance. This suggests that the activities of a reinforcement learning system, as reflected in the FN and associated with the midbrain dopamine system and the anterior cingulate cortex, contributes independently to the explanation of variability in decision-making (Hewig, Straube, Trippe, Hecht, Kretschmer, et al., 2009; Hewig et al., 2007; Holroyd & Coles, 2002; Holroyd, Nieuwenhuis, Yeung, Nystrom, Mars, et al., 2004; Jocham & Ullsperger, 2009; Ullsperger & von Cramon, 2003). Mathematical models of reinforcement learning (Sutton & Barto, 1998) suggest that individuals learn from the detection of differences between the expected and actual reinforcement. Thus, any outcome of a decision or a behavioral act can be either better or worse than expected, and the size and valence of the discrepancy is proposed to be reflected in the amplitude of the FN. In line with this proposal, FN amplitude is smaller when an outcome is better than expected and larger when the outcome is worse than expected, (e.g., Hewig et al., 2007; Holroyd & Krigolson, 2007). Conversely, outcomes that are better than expected lead to an increase in the activity of the midbrain dopamine system, inhibition of the apical dendrites of motor neurons in the ACC, and production of small amplitude FNs or of an outcome positivity (OP) or a feedback-related positivity (Holroyd et al., 2008). Within this theoretical framework, anterior cingulate cortex is thought to integrate reinforcement history to guide voluntary behavior (Holroyd & Coles, 2008). It has further
Multiple Regression Correlations EDA
Valence EDA FN
FN
Rej10:2 beta
T
p–value
.050 ! .054 ! .614 ! .616 ! 4.303 .003 .427 .564 .440 2.785 .024 .582 .361 2.282 .052
beta range
! .522 to ! .742 .371 to .661 .133 to .456
Notes. Valence 5 emotional ratings; EDA 5 electrodermal activity/skin conductance responses; ERP at Fz for 6:6 in Ultimatum 5 FN, Rej10:2 5 number of rejections for 10:2 offers. Beta 5 standardized beta weight in the multiple regression analysis; T and p-values of the standardized beta weight, beta range 5 range of beta values in the 12 Jackknife regression analyses.
1 It might be argued that using several dependent variables increases the number of statistical tests and thus may lead to a type I error inflation. However, for the present study most of the effects for each single variable had been found previously and were clearly predicted by directed hypotheses. Thus, they are conceptual replications, which is one of the most important tools to oppose type I error problems. Moreover, a type I error correction does increase the probability of a type II errorFthe probability not to detect an effect that is present in the population. Accordingly, replicating previous results is hindered significantly if type II error probability is increased because the chance of a successful replication is reduced. Hence, we decided to avoid an increase in type II error and did not use a type I error correction.
Irrational decisions
513
been suggested that FN amplitude may reflect somatic markers of reinforcement and punishment (Holroyd, Nieuwenhuis, Yeung, & Cohen, 2003). The somatic marker hypothesis (e.g., Bechara & Damasio, 2005; Bechara, Damasio, & Damasio, 2000) holds that somatic markers, which are autonomic signals that indicate the positive and negative consequences of experienced stimuli, guide decision-making. Positive somatic markers associated with an action increase the likelihood of its selection, whereas negative somatic markers of an action decrease the likelihood of its selection. Accordingly, the present findings suggest that affective somatic markers may contribute to rejections in the Ultimatum Game. Such markers may also be an important source of motivation for altruistic punishment against other players who defect, do not cooperate, or show egoistic behavior (Fehr & Camerer, 2007; Fehr & Gachter, 2002). The proposed neuroanatomical basis for the somatic marker is the ventromedial prefrontal cortex (VMPFC), which includes the orbitofrontal, the medial frontal, and parts of the anterior cingulate cortex. Taken together, these findings suggest that a reinforcement learning mechanism involving anterior cingulate cortex and VMPFC uses affective markers to guide behavior in complex decision-making situations. We hypothesized that increased FN would be related to heightened rejection rate. However, the present data indicate that the number of rejections was related to more positive amplitudes to fair offers rather than to more negative amplitudes to unfair offers. The present data cannot provide an explanation for this finding. On the one hand, it may be argued that lower FN amplitudes to fair offers would indicate that participants expect proposers to make unfair offers and hence are positively surprised when the offers are fair. In line with this reasoning, negative views of others and negative expectations concerning others’ intentions might at the same time lead to more rejections of unfair offers. On the other hand, the fact that participants showed increased positivity towards fair offers might suggest that the participants were particularly sensitive to reward. For example, pathological gamblers show more positive amplitudes to reinforcing events (Hewig, Kretschmer, Trippe, Hecht, Coles, et al., 2010). According to this idea, more reward-sensitive par-
ticipants might be disappointed by unfair offers and reject them more often. Future research will be necessary to shed more light on this result. The statistical analyses revealed no significant effects of Game in the sense that there were no differences in the EEG, EDA, or subjective responses between offers in the Ultimatum as compared to the Dictator game. Accordingly, our findings indicate that the processes we have identified are related primarily to an evaluation of unfairness and the motivation to oppose unfairness, rather than the preparation or initiation of remedial action against unfairness. Thus, the data also indicate that FN might not primarily reflect direct behavioral reinforcement learning here but rather a more general form of reinforcement learningFthe learning of action values or the learning of the motivational value of a situation (e.g., Holroyd & Coles, 2002, 2008; Kennerley, Walton, Behrens, Buckley, & Rushworth, 2006; Walton, Croxson, Behrens, Kennerley, & Rushworth, 2007). In accordance with this suggestion, previous studies have shown that a FN is present in the absence of response choice (Yeung, Holroyd, & Cohen, 2005), which is in line with the observed FN in the Dictator game in the present study. In addition, it may be noted that, in the Dictator game and in the Ultimatum game (accepted offer trials), the presentation of the offers indicates the exact monetary feedback that can be expected. Moreover, for all low offers in the Ultimatum game, the presentation of the offer already signals the low outcome of the current trial (either being 1 or 2 cents upon accept and 0 cents upon reject). Since FN seems to migrate back to the earliest indicator of reinforcement, as shown by Dunning and Hajcak (2007), the offerFin terms of reinforcement learningFalready implies the likely decision and thus the final outcome. Thus, the presentation of the offer conveys very similar information about outcome in both games. Taken together, this might explain the absence of differences between the Ultimatum and Dictator games. In summary, our data corroborate previous findings indicating the importance of emotional processes in decision-making. Our data further reveal the presence of several independent sources of variance that each contribute to human decision-making in the Ultimatum game.
REFERENCES Bechara, A., & Damasio, A. R. (2005). The somatic marker hypothesis: A neural theory of economic decision. Games and Economic Behavior, 52, 336–372. Bechara, A., Damasio, H., & Damasio, A. R. (2000). Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex, 10, 295–307. Boksem, M. A., Tops, M., Wester, A. E., Meijman, T. F., & Lorist, M. M. (2006). Error-related ERP components and individual differences in punishment and reward sensitivity. Brain Research, 1101, 92–101. Camerer, C. F. (2003). Behavioral game theory: Experiments in strategic interaction. New York, NY: Russell Sage Foundation. Coles, M. G. H., & Rugg, M. D. (1995). Event-related brain potentials: An introduction. In M. D. Rugg & M. G. H. Coles (Eds.), Electrophysiology of mind: Event-related brain potentials and cognition. Oxford: Oxford University Press. Davis, M. (1992). The role of the amygdala in fear and anxiety. Annual Review of Neuroscience, 15, 353–375. Dunning, J. P., & Hajcak, G. (2007). Error-related negativities elicited by monetary loss and cues that predict loss. NeuroReport, 18, 1875–1878. Fehr, E., & Camerer, C. F. (2007). Social neuroeconomics: The neural circuitry of social preferences. Trends in Cognitive Sciences, 11, 419–427. Fehr, E., & Gachter, S. (2002). Altruistic punishment in humans. Nature, 415, 137–140.
Fowles, D. C. (1980). The three arousal model: Implications of Gray’s two-factor learning theory for heart rate, electrodermal activity and psychopathy. Psychophysiology, 17, 87–104. Fredrikson, M., Furmark, T., Olsson, M. T., Fischer, H., Andersson, J., & Langstrom, B. (1998). Functional neuroanatomical correlates of electrodermal activity: A positron emission tomographic study. Psychophysiology, 35, 179–185. Furmark, T., Fischer, H., Wik, G., Larsson, M., & Fredrikson, M. (1997). The amygdala and individual differences in human fear conditioning. NeuroReport, 8, 3957–3960. Gray, J. A. (1982). The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system. Oxford: Oxford University Press. Gray, J. A. (1994). Personality dimensions and emotion systems. In P. Ekman & R. J. Davidson (Eds.), The nature of emotion. New York: Oxford University Press. Gu¨th, W., Schmittberger, R., & Schwarze, B. (1982). An experimental analysis of ultimatum bargaining. Journal of Economic Behaviour and Organization, 3, 367–388. Hajcak, G., McDonald, N., & Simons, R. F. (2003). Anxiety and errorrelated brain activity. Biological Psychology, 64, 77–90. Hajcak, G., McDonald, N., & Simons, R. F. (2004). Error-related psychophysiology and negative affect. Brain & Cognition, 56, 189–197.
514 Harle, K. M., & Sanfey, A. G. (2007). Incidental sadness biases social economic decisions in the Ultimatum Game. Emotion, 7, 876–881. Hewig, J., Kretschmer, N., Trippe, R. H., Hecht, H., Coles, M. G., Holroyd, C. B., & Miltner, W. H. (2010). Hypersensitivity to reward in problem gamblers. Biological Psychiatry, 67, 781–783. Hewig, J., Straube, T., Trippe, R. H., Hecht, H., Kretschmer, N., Coles, M. G. H., & Miltner, W. H. R. (2009). Decision-making under risk: An fMRI study. Journal of Cognitive Neuroscience, 21, 1642–1652. Hewig, J., Trippe, R. H., Hecht, H., Coles, M. G. H., Holroyd, C. B., & Miltner, W. H. R. (2007). Decision-making in Blackjack: An electrophysiological analysis. Cerebral Cortex, 17, 865–877. Holroyd, C. B., & Coles, M. G. (2008). Dorsal anterior cingulate cortex integrates reinforcement history to guide voluntary behavior. Cortex, 44, 548–559. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the errorrelated negativity. Psychological Review, 109, 679–709. Holroyd, C. B., & Krigolson, O. E. (2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology, 44, 913–917. Holroyd, C. B., Nieuwenhuis, S., Yeung, N., & Cohen, J. D. (2003). Errors in reward prediction are reflected in the event-related brain potential. NeuroReport, 14, 2481–2484. Holroyd, C. B., Nieuwenhuis, S., Yeung, N., Nystrom, L., Mars, R. B., Coles, M. G., & Cohen, J. D. (2004). Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nature Neuroscience, 7, 497–498. Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45, 688–697. Jocham, G., & Ullsperger, M. (2009). Neuropharmacology of performance monitoring. Neuroscience & Biobehavioral Reviews, 33, 48–60. Kennerley, S. W., Walton, M. E., Behrens, T. E., Buckley, M. J., & Rushworth, M. F. (2006). Optimal decision making and the anterior cingulate cortex. Nature Neuroscience, 9, 940–947. Luu, P., Collins, P., & Tucker, D. M. (2000). Mood, personality, and selfmonitoring: Negative affect and emotionality in relation to frontal lobe mechanisms of error monitoring. Journal of Experimental Psychology: General, 129, 43–60. Meriau, K., Wartenburger, I., Kazzer, P., Prehn, K., Villringer, A., van der Meer, E., & Heekeren, H. R. (2009). Insular activity during passive viewing of aversive stimuli reflects individual differences in state negative affect. Brain and Cognition, 69, 73–80. Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997). Eventrelated brain potentials following incorrect feedback in a time-estimation task: Evidence for a ‘‘generic’’ neural system for error detection. Journal of Cognitive Neuroscience, 9, 788–798. Nieuwenhuis, S., Yeung, N., Holroyd, C., Schurger, A., & Cohen, J. D. (2004). Sensitivity of electrophysiological activity from medial frontal
J. Hewig et al. cortex to utilitarian and performance feedback. Cerebral Cortex, 14, 741–747. Nowak, M. A., Page, K. M., & Sigmund, K. (2000). Fairness versus reason in the ultimatum game. Science, 289, 1773–1775. Pailing, P. E., & Segalowitz, S. J. (2004). The error-related negativity as a state and trait measure: Motivation, personality, and ERPs in response to errors. Psychophysiology, 41, 84–95. Phelps, E. A., & LeDoux, J. E. (2005). Contributions of the amygdala to emotion processing: From animal models to human behavior. Neuron, 48, 175–187. Pillutla, M. M., & Murnighan, J. K. (1996). Unfairness, anger, and spite: Emotional rejections of ultimatum offers. Organizational Behavior and Human Decision Processes, 68, 208–224. Polezzi, D., Daum, I., Rubaltelli, E., Lotto, L., Civai, C., Sartori, G., & Rumiati, R. (2008). Mentalizing in economic decision-making. Behavioural Brain Research, 190, 218–223. Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300, 1755–1758. Solnick, S. J., & Schweitzer, M. E. (1999). The influence of physical attractiveness and gender on ultimatum game decisions. Organizational Behavior and Human Decision Processes, 79, 199–215. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press. Tabibnia, G., Satpute, A. B., & Lieberman, M. D. (2008). The sunny side of fairness: Preference for fairness activates reward circuitry (and disregarding unfairness activates self-control circuitry). Psychological Science, 19, 339–347. Tukey, J. W. (1958). Bias and confidence in not-quite large samples. Annals of Mathematical Statistics, 29, 614–614. Ullsperger, M., & von Cramon, D. Y. (2003). Error monitoring using external feedback: Specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. Journal of Neuroscience, 23, 4308–4314. van ‘t Wout, M., Kahn, R. S., Sanfey, A. G., & Aleman, A. (2006). Affective state and decision-making in the Ultimatum Game. Experimental Brain Research, 169, 564–568. von Neumann, J., & Morgenstern, O. (1953). Theory of Games and Economic Behavior. Princeton: Princeton University Press. Walton, M. E., Croxson, P. L., Behrens, T. E., Kennerley, S. W., & Rushworth, M. F. (2007). Adaptive decision making and value in the anterior cingulate cortex. NeuroImage, 36(Suppl 2), T142–154. Yeung, N., Holroyd, C. B., & Cohen, J. D. (2005). ERP correlates of feedback and reward processing in the presence and absence of response choice. Cerebral Cortex, 15, 535–544. Yeung, N., & Sanfey, A. G. (2004). Independent coding of reward magnitude and valence in the human brain. Journal of Neuroscience, 24, 6258–6264. (Received February 25, 2010; Accepted May 17, 2010)
Psychophysiology, 48 (2011), 515–522. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01083.x
Repetitive exposure: Brain and reflex measures of emotion and attention
VERA FERRARI,a MARGARET M. BRADLEY,a MAURIZIO CODISPOTI,b and PETER J. LANGa a
Center for the Study of Emotion and Attention, University of Florida, Gainesville, Florida, USA Department of Psychology, University of Bologna, Bologna, Italy
b
Abstract Effects of massed repetition on the modulation of the late positive potential elicited during affective picture viewing were investigated in two experiments. Despite a difference in the number of repetitions across studies (from 5 to 30), results were quite similar: The late positive potential continued to be enhanced when participants viewed emotional, compared to neutral, pictures. On the other hand, massed repetition did prompt a reduction in the late positive potential that was most pronounced for emotional pictures. Startle probe P3 amplitude generally increased with repetition, suggesting diminished attention allocation to repeated pictures. The blink reflex, however, continued to be modulated by hedonic valence, despite massive massed repetition. Taken together, the data suggest that the amplitude of the late positive potential during picture viewing reflects both motivational significance and attention allocation. Descriptors: Emotion, Attention, Repetition
hypothesis in two experiments by repeatedly presenting the same picture over and over with no intervening stimuli between repetitions. If it is motivational significance that mediates the persistent modulation found for emotionally engaging pictures despite repetition, we expected to continue to find differences in LPP amplitude between emotional and neutral pictures; to the extent that the LPP primarily indexes heightened initial attention allocation, we expected the differences to disappear. We employed two other reliable measures of attention and emotion in picture viewing to aid in interpreting changes in the late positive potential with massed repetition. Presentation of an acoustic startle probe during affective picture viewing modulates both the reflexive eyeblink and the amplitude of the P3 component of the event-related potential to the probe. Probe P3 amplitude is smaller for startle stimuli presented in the context of emotional, compared to neutral, picture viewing. One interpretation is that heightened resource allocation during affective picture processing results in fewer resources available for processing the secondary acoustic probe (Bradley, Codispoti, & Lang, 2006; Bradley, Cuthbert, & Lang, 1999; Cuthbert, Schupp, Bradley, McManis, & Lang, 1998). This interpretation is consistent with data finding that reaction times to secondary probes are also slower when people view emotional, compared to neutral, pictures (Bradley et al., 1999; De Cesarei & Codispoti, 2008). If repetitive contiguous presentation decreases attention allocation to the picture, we expected the difference in probe P3 amplitude between emotional and neutral pictures to decrease with massed repetition. Moreover, the specific comparison between modulation of the LPP and the probe P3 as a function of repetition is also quite informative: If the LPP during picture viewing and the probe P3 index the same attentional process, we expected that the
Affect reliably modulates the magnitude of a late positive potential (LPP) measured over centro-parietal sensors, with the largest LPPs elicited when people view either pleasant or unpleasant, compared to neutral, pictures (e.g., Cacioppo, Crites, & Gardner, 1996; Codispoti, Mazzetti, & Bradley, 2009; Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Johnston, Miller, & Burleson, 1986; Palomba, Angrilli, & Mini, 1997). Several recent studies have found that, despite a decrease in overall amplitude of the LPP with repeated presentation of the same picture, emotional pictures continue to elicit a larger late positive potential than neutral pictures (Codispoti, Ferrari, & Bradley, 2006, 2007). One interpretation is that the LPP reflects, in part, motivational significance, defined as activation of cortico-limbic appetitive and defensive systems that mediate the sensory and motor processes that support perception and action (Bradley, 2009). In our previous studies, however, repeated pictures were always presented intermixed among other (repeated) pictures. Thus, although the pictures were highly familiar, initial encoding and attention allocation processes were still necessary for picture identification on each presentation, and it may be these processes that underlie continued modulation of the LPP. We tested this This research was supported in part by a grant from the National Institute of Mental Health (P50 MH 72850) to the Center for the Study of Emotion and Attention (CSEA) at the University of Florida. Vera Ferrari is now at the University of Bologna, Italy. Address correspondence to: Margaret Bradley, Center for the Study of Emotion and Attention, Box 112766, University of Florida, Gainesville, FL 32611, USA, or Vera Ferrari, Department of Psychology, University of Bologna, Viale Berti Pichat, 5, 40127 Bologna, Italy. E-mail:
[email protected] or
[email protected] 515
516 effects of picture repetition on affective modulation of these electrocortical potentials would be similar. Unlike the probe P3, the reflexive blink response to a startle probe presented during picture perception is modulated by hedonic valence, with larger reflexes elicited when people view unpleasant, compared to pleasant, pictures (e.g., Bradley et al., 1999; Vrana, Spence, & Lang, 1988). When affective pictures are repeatedly presented intermixed with other (repeated) pictures, the modulatory effects of emotion remain intact (Bradley, Lang, & Cuthbert, 1993). On the other hand, the reflexive blink response is significantly smaller shortly after picture onset, compared to later in the viewing interval, suggesting blink magnitude is inhibited when pictures are novel and attention getting (Bradley et al., 2006). If repetition decreases this inhibitory effect on blink magnitude, startle reflexes are expected to be generally smaller for novel, compared to repeated, pictures. Nonetheless, because the blink reflex is differentially modulated by pleasant and unpleasant content (i.e., unlike the LPP or the P3), we expected affective modulation to remain with massed repetition.
EXPERIMENT 1 Method Participants Participants were 25 students (13 women) from the University of Florida introductory psychology courses who participated for course credit. Of these, 2 subjects (1 man and 1 woman) were excluded from analysis of startle data because of virtually no startle responses. Materials and Design Stimuli were 168 pictures selected from the International Affective Picture System (Lang, Bradley, & Cuthbert, 2008), consisting of 56 pleasant, 56 neutral, and 56 unpleasant pictures. Of these, 144 pictures were presented only once throughout the study (‘‘novel’’ presentation), whereas each of the remaining 24 pictures (8 for each hedonic content) was repeatedly presented 5–8 times in a row (‘‘massed’’ repetition); block length for massed pictures was variable to reduce predictability. In total, there were six blocks each of 5, 6, 7, or 8 massed repetitions (156 trials). Novel pictures were arranged such that there were two pictures of each hedonic content in each block of 6. Blocks of novel pictures were alternated with blocks of massed repetitions, for a total of 300 trials (144 novel pictures1156 repeated pictures). The acoustic startle probe consisted of a 98-dB, 50-ms burst of white noise with instantaneous rise time. The stimulus was generated by a Coulbourn S81-02 white-noise generator and presented binaurally over earphones (Eartone A3 Audiometric Insert Earphones, Aearo Company, Indianapolis, IN). A total of 48 startle probes were presented (1200 ms following picture onset) in the study, 24 during viewing of novel pictures (8 of each hedonic content) and 24 during viewing of massed repetitions (8 of each hedonic content). During massed presentation, a startle probe could be presented at either the third, the fourth, or the fifth repetition of the picture. Using the same 168 pictures, three presentation orders were constructed that varied, across participants, the specific pictures presented in the novel and massed conditions as well as the pictures that were probed. Each trial consisted of a fixation cross presented at the center of the screen for 500 ms prior to picture onset. Each picture was
V. Ferrari et al. displayed for 2 s, followed by 2 s of intertrial interval. Pictures were presented on a 19-in. CRT monitor, situated approximately 100 cm from the participant. Physiological Recording and Data Reduction Electroencephalographs were collected from the scalp using a 129-channel system (Electrical Geodesics, Inc., Eugene, OR) running NetStation software on a Macintosh computer. Scalp impedance for each sensor was kept below 50 kO. The electroencephalogram (EEG) was recorded continuously with a sampling rate of 250 Hz, the vertex sensor as reference electrode, and online bandpass filtered from 0.01 to 100 Hz. EEG data were analyzed off-line using a MATLAB-based program (Jungho¨fer, Elbert, Tucker, & Rockstroh, 2000) in which continuous EEG data were low-pass filtered at 40 Hz using digital filtering, and artifact detection was performed by means of a dedicated algorithm that uses statistical parameters to determine trials with artifacts (Jungho¨fer et al., 2000). For both picture onset and startle probe ERP epochs, a 100-ms prestimulus data segment was subtracted as baseline. Processed data were averaged by hedonic content and novel/massed presentation1 for each participant. The average number of trials per condition employed was approximately 40 for picture onset ERPs and approximately 8 for the startle probe ERPs. Based on previous studies (Bradley, Hamby, Loew, & Lang, 2007; Codispoti et al., 2006) and ERP visual inspection, statistical analyses were performed on mean amplitude values computed in a 400– 700-ms window for the late positive potential and a 250–350-ms window for the probe P3. For both analyses, amplitudes were measured over a group of centro-parietal sensors (sensor number 7, 32, 38, 54, 55, 61, 62, 68, 79, 80, 81, 88, 107, 129). The eyeblink component of the startle response was measured by recording electromyogram (EMG) activity from the orbicularis oculi muscle beneath the left eye. The raw EMG signal was amplified (! 30,000), and frequencies below 90 Hz and above 250 Hz were filtered with a Coulbourn S75-01 bioamplifier. The raw signal was rectified and integrated using a Coulbourn S76-01 contour-following integrator using a normal time constant of 123 ms. The blink response was sampled at 1000 Hz for 50 ms prior to the onset of the startle probe and for 250 ms after probe onset. The startle data were reduced off-line using a VPM program (Cook, 1997) that implements a peak-scoring algorithm (Balaban, Losito, Simons, & Graham, 1996) that scores the peak response for onset latency and amplitude. Trials with artifacts were rejected whereas trials with no responses were scored as zero magnitude blinks. Procedure After their arrival at the laboratory, participants signed an informed consent form. Participants were then seated in a recliner in a small, sound-attenuated, dimly lit room, and recording electrodes for the blink and the EEG sensor net were attached. The subject was instructed that a series of pictures would be presented in which each picture should be viewed the entire time it was on the screen and that brief noises heard over the headphones could be ignored. 1 Based on a previous analysis (Ferrari, Bradley, Codispoti, & Lang, 2010), trials that signaled a change in the experimental structure (i.e., the first novel picture following a block of massed repetitions and the first repetition of a picture in the block of massed repetitions) were excluded from the averaging because these trials prompt an enhanced P3 amplitude that reflects stimulus meaning, regardless of repetition.
Repetitive exposure
517
Figure 1. Experiment 1. Grand average ERP waveforms (centro-parietal sensor group) when subjects viewed pleasant, neutral, and unpleasant pictures in the massed and novel repetition conditions. Insets are the top view of the scalp distribution of the difference in the 400–700-ms window between emotional (pleasant and unpleasant) and neutral picture processing.
An initial 30-trial practice block, consisting of the same neutral picture repeatedly displayed, was presented together with 12 startle probes in order to facilitate the basic circuit habituation of the startle reflex (Bradley et al., 1993). Following this initial habituation phase, 300 pictures were presented consisting of novel pictures and massed repetitions. After the picture series was finished, subjects completed a postexperimental questionnaire and were debriefed. Data Analysis Each measure (LPP, probe P3, blink magnitude) was analyzed in a repeated measure analysis of variance (ANOVA) using stimulus repetition (2: massed, novel) and hedonic content (3: pleasant, neutral, unpleasant) as factors. Greenhouse–Geisser corrections were applied where relevant. The eta squared statistic (Z2), indicating the proportion between the variance explained by one experimental factor and the total variance, has been calculated and is reported. Results Late Positive Potential Figure 1 illustrates the ERP waveforms averaged over centroparietal sensors for pleasant, neutral, and unpleasant pictures in the massed and novel repetition conditions.2 Significant main effects of hedonic content, F(2,48) 5 43.7, po.0001, Z2 5 .64, and repetition, F(1,24) 5 24.7, po.0001, Z2 5 .51, were accompanied by a significant interaction between hedonic content and repetition, F(2,48) 5 14.9, po.0001, Z2 5 .38. When subjects viewed novel pictures, hedonic content modulated the magnitude of the centro-parietal late positive potential, as expected, F(2,48) 5 42.1, po.0001, Z2 5 .64, with larger positivity when they viewed pleasant or unpleasant pictures, compared to neutral 2 Analysis of the large N2 component in the ERP to pictures that is dramatically present for all of the novel pictures and greatly attenuated for all of the massed pictures (i.e., independent of hedonic content) is presented in a separate report (Ferrari et al., 2010).
contents, Fs(1,24)426, pso.0001, Z24.52, replicating many previous studies. Viewing massed repetitions of emotional pictures also prompted significantly larger LPPs than when viewing neutral pictures, F(2,48) 5 17.7, po.0001, Z2 5 .42; pleasant and unpleasant versus neutral, Fs(1,24)419.3, po.0001, Z24.45. The interaction primarily indicates that, whereas the LPP when viewing novel and repeated neutral pictures did not differ, the LPP for emotional pictures was smaller when the pictures were repeated, F(1,24) 5 10.5, po.001, Z2 5 .31 for pleasant, F(1,24) 5 59.1, po.0001, Z2 5 .71 for unpleasant. Probe P3 Figure 2 illustrates the ERP waveforms to the startle probe (centro-parietal sensors) presented during viewing of pleasant, neutral, and unpleasant pictures in the massed and novel repetition conditions. Repetition of the same picture over and over significantly increased probe P3 amplitude, with larger P3s when viewing repeated, compared to novel, pictures, novel versus massed, F(1,24) 5 18.6, po.0001, Z2 5 .44. A main effect of hedonic content, F(2,48) 5 3.8, po.05, Z2 5 .14, indicated smaller P3 amplitude for emotional, compared to neutral, pictures, quadratic effect, F(1,24) 5 4.1, p 5 .055, Z2 5 .15, as illustrated in Figure 2. Although the interaction of hedonic content and repetition did not reach significance (p 5 .31), a priori tests exploring modulation for novel and massed pictures indicated that whereas probe P3 was significantly modulated by picture content during the viewing of novel pictures, F(2,48) 5 6.34, po.01, Z2 5 .2, it was not reliably modulated by emotional content when subjects viewed repeated pictures (p 5 .3). Startle Reflex Startle blink magnitude was significantly smaller when subjects viewed novel, compared to repeated, pictures, F(1,22) 5 5.2, po.05, Z2 5 .19 (see Table 1). Nonetheless, blink magnitude varied with hedonic content, F(2,44) 5 13.88, po.0001, Z2 5 .39, regardless of picture repetition (Hedonic Content ! Stimulus Repetition, p4.05). Blinks were larger when subjects viewed unpleasant, compared to pleasant or neutral, pictures,
518
V. Ferrari et al.
Figure 2. Experiment 1. Grand average ERP waveforms (centro-parietal sensor group) to the startle probe during viewing of pleasant, neutral, and unpleasant pictures in the massed and novel repetition conditions. Insets are the top view of the scalp distribution of the difference in the 250–350-ms window between emotional (pleasant and unpleasant) and neutral picture processing.
F(1,22) 5 22.12, po.0001, Z2 5 .5, F(1,22) 5 26.9, po.0001, Z2 5 .55, respectively.
EXPERIMENT 2 In Experiment 1, massed repetition affected all measuresFdecreasing the LPP following picture onset and increasing both the amplitude of the P3 to the startle probe during picture viewing as well as startle blink magnitude. Nonetheless, LPP amplitude continued to be significantly larger when subjects viewed emotional, compared to neutral, pictures, even following multiple massed repetitions of the same picture. Similarly, affective modulation of the reflexive blink remained significant despite massed repetition of the pictures. On the other hand, although mean probe P3 amplitude was attenuated when subjects viewed emotional, compared to neutral, novel pictures, the difference clearly decreased and was not statistically reliable for pictures that were repeatedly presented, consistent with a hypothesis that probe P3 indexes attention allocation. In Experiment 2, we extended the number of massed repetitions to 30 contiguous repetitions in order to more completely attenuate attentional resource allocation to emotional pictures. As in Experiment 1, we measured the magnitude of the LPP Table 1. Means (Standard Error) of Late Positive Potential (LPP), the Amplitude of the P3 Component to the Startle Probe (Probe P3), and Startle Blink Magnitude for Each Hedonic Content in Experiment 1 LPP (400–700 ms) Massed Unpleasant Neutral Pleasant
Novel
Probe P3 (250–350 ms) Massed
Novel
Startle blink magnitude Massed
Novel
1.5 (0.4) 3.4 (0.4) 6.6 (0.9) 5.2 (0.7) 8.1 (1) 7.7 (0.1) 0.2 (0.4) 0.4 (0.3) 7.0 (0.9) 5.9 (0.9) 7.2 (1) 6.3 (0.9) 1.5 (0.3) 2.6 (0.4) 6.1 (0.8) 4.2 (0.7) 7.1 (1) 6.3 (0.9)
Note: All values are expressed in microvolts.
following picture onset as well as the amplitude of the probe P3 and the startle blink reflex. Moreover, startle probes were also presented in the interpicture interval in Experiment 2. If probe P3 amplitude primarily indexes attention allocation, as hypothesized, we expected not only that its modulation by emotion would be eliminated by massive, massed repetition, but that probe P3 amplitude following repetition would not differ from that elicited when there was not even a picture in the foreground. To the extent that the startle blink reflects motivational significance, we expected modulation to remain intact regardless of the number of repetitions. Of particular interest are the effects of massive repetition on the late positive potential: If the heightened positivity primarily reflects differences in attention allocation, we expected it to be eliminated with massive, massed repetition. To the extent it reflects motivational significance, we expected to continue to find a larger LPP for emotional, compared to neutral, pictures. Rather than comparing massed repetition to novel picture presentation, in Experiment 2 we compared massed repetition to a distributed repetition condition in which pictures were repeated across the experiment, but intermixed with other (repeated) pictures. In this comparison, effects specifically due to massed repetition should be highlighted. Method Participants Participants were 25 students (13 women) from the University of Florida introductory psychology courses who participated for course credit. Of these, 3 subjects (2 men and 1 woman) were excluded from analysis of startle data because of virtually no startle responses. Materials and Design Thirty-six color pictures depicting 12 pleasant (erotic couple and romance), 12 unpleasant (threat and mutilation), and 12 neutral (objects and people) scenes were selected from the International Affective Picture System (Lang et al., 2008). Of these, 6 pictures
Repetitive exposure were presented 30 times in a row (massed) and 30 were presented intermixed with other pictures and repeated 6 times across the study (distributed). Each massed repetition was followed by the series of 30 distributed pictures, resulting in a total of 360 trials. Six presentation orders were constructed that varied, across participants, the specific pictures presented in the massed and distributed blocks such that each of the 36 pictures was presented in the massed and distributed condition across participants. The order of pictures in the distributed condition was counterbalanced such that there were two pictures of each hedonic content in each subblock of six. As in Experiment 1, each trial consisted of a fixation cross presented at the center of the screen for 500 ms prior to picture onset. Each picture was displayed for 2 s, followed by a 2-s intertrial interval. Pictures were presented on a 19-in. CRT monitor, situated approximately 100 cm from the participant. Across each of the 30 massed repetitions, three startle probes (one every 10 trials) were delivered during picture viewing (1200 ms after picture onset) as well as in the postpicture interval (1200 ms after picture offset). Startle probes were similarly presented in the distributed condition.
Procedure The procedure was equal to that described in Experiment 1.
Physiological Recording and Data Reduction The startle probe, and measurement of the blink response, and electroencephalographic activity as well as data reduction were as described in Experiment 1. Blink responses and probe ERPs were averaged across the 30 massed repetition block, separately for probes delivered during picture viewing and those prompted in the postpicture interval (6 trials per condition). For the LPP (400–700-ms window), the 30 massed repetitions were averaged into two subblocks of 15 trials (Massed 1–15, Massed 16–30). Each measure was analyzed in a repeated measures ANOVA involving picture hedonic content (3: pleasant, neutral, unpleasant) and stimulus repetition (2: massed, distributed).
519 Results Late Positive Potential Figure 3 illustrates the ERP waveforms for pleasant, neutral, and unpleasant pictures in the massed and distributed repetition conditions over centro-parietal sensors. A main effect of hedonic content, F(2,48) 5 104, po.0001, Z2 5 .81, again indicated greater positivity for both pleasant and unpleasant pictures, compared to neutral content, F(1,24) 5 120, po.0001, Z2 5 .83, F(1,24) 5 175, po.0001, Z2 5 .88, respectively. As illustrated in Figure 3, emotional (pleasant and unpleasant) pictures prompted a larger LPP than neutral pictures both in the massed repetition condition, Fs(1,24)419.6, pso.0001, Z24.45, and in the distributed repetition condition, Fs(1,24)482.9, pso.0001, Z24.78. For massed repetition, in fact, the magnitude of affective modulation was similar when assessed either during the first or last 15 repetitions in the massed condition, F(1,24)o1 (see Figure 3, inset). Again, effects of repetition, F(1,24) 5 138.3, po.0001, Z2 5 .85, and a significant interaction between repetition and hedonic content, F(2,48) 5 19.8, po.0001, Z2 5 .45, indicated a larger decrease in the LPP following massed repetition for emotional, compared to neutral, pictures, resulting in significantly smaller affective modulation for massed, compared to distributed, repetitions, F(1,24) 5 32, po.0001, Z2 5 .57. Probe P300 As in Experiment 1, probe P3 amplitude was significantly larger for massed repetitions, F(1,24) 5 9.9, po.005, Z2 5 .3. Unlike Experiment 1, however, massive massed repetition now resulted in a significant interaction between repetition and hedonic content, F(2,48) 5 6.25, po.005, Z2 5 .21. As illustrated in Figure 4 (top), probe P3 amplitude was significantly attenuated when subjects viewed emotional, compared to neutral, pictures, when repetitions were distributed, similar to the pattern found for novel pictures in Experiment 1, F(2,48) 5 10, po.0001, Z2 5 .29. Massed repetition, however, again eliminated the difference in probe P3 amplitude when subjects viewed emotional and neutral pictures, F(2,48)o1. Whereas the type of repetition did not affect the amplitude of the probe P3 when subjects viewed neutral pictures, massed repetition significantly increased probe P3 ampli-
Figure 3. Experiment 2. Grand average ERP waveforms (centro-parietal sensor group) to pleasant, neutral, and unpleasant pictures in the massed (averaged over 30 repetitions) and distributed repetition conditions. Insets are the top view of the scalp distribution of the difference in the 400–700-ms window between emotional (pleasant and unpleasant) and neutral picture processing. Massed condition also shows the mean amplitude of the late positive potential (400–700 ms) for massed repetitions averaged over two subblocks of 15 repetitions each.
520
V. Ferrari et al. pictures. A significant main effect of hedonic content, F(2,42) 5 13.8, po.0001, Z2 5 .39, indicated significant startle potentiation when subjects viewed unpleasant pictures, F(1,21) 5 9.9, po.01, Z2 5 .32, and significant attenuation when they viewed pleasant pictures, F(1,21) 5 7.9, po.05, Z2 5 .27, compared to neutral pictures. Similar to Experiment 1, affective modulation of blink magnitude was intact even in the massed condition, in which the same picture was repeated 30 times in a row, F(2,42) 5 8.87, po.01, Z2 5 .29. Moreover, the type of repetition (massed vs. distributed) did not affect overall blink magnitude.3 GENERAL DISCUSSION
Figure 4. Experiment 2. Top: Probe P3 amplitude in the 250–350-ms window over centro-parietal sensors for pleasant, neutral, and unpleasant pictures in the massed and distributed repetition conditions. Bottom: Blink magnitude for startle probes presented during the viewing of pleasant, neutral, and unpleasant pictures in the massed and distributed repetition conditions.
tude when subjects viewed emotional pictures, consistent with an interpretation of less attention allocation as repetition increased, Fs(1,24)45, po.05. Z24.2. The amplitude of the probe P3 in the interpicture interval differed for massed and distributed repetitions, consistent with an interpretation that probe P3 indexes attention allocation: Probe P3 amplitude in the picture viewing (5.05 mV) and postpicture interval (4.48 mV) did not differ for massed repetitions. On the other hand, probe P3 amplitude was smaller (3.04 mV) during picture viewing, compared to the interpicture interval (4.49 mV) when the repetition was not massed, F(1,24) 5 24.8, po.0001, Z2 5 .5. Startle Reflex Figure 4 (bottom) illustrates blink magnitude for startle probes presented during viewing of pleasant, neutral, and unpleasant
Effects of massed repetition on the modulation and the amplitude of the LPP during affective picture viewing were investigated. Despite a difference in the absolute number of contiguous repetitions in two studies (from 5 to 30), results were strikingly similar: Even following many contiguous repetitions of the same picture, larger late positive potentials were associated with viewing emotional, compared to neutral, pictures. These data disconfirm a hypothesis that intermixing pictures in previous studies (Codispoti et al., 2006, 2007) was responsible for the sustained modulation following repetition. Rather, modulatory differences remained intact even following the massive massed repetition in Experiment 2, in which the same picture was presented 30 times in a row. In both studies, a reduction in the amplitude of the late positive potential with massed repetition was most pronounced for emotional, compared to neutral, pictures. Thus, although viewing emotional pictures consistently prompted a larger late positive potential than viewing neutral pictures, the absolute amplitude of the late positive potential was greatly reduced for emotional pictures following massed repetition. Two additional indices of attention allocation and motivational significance shed light on this pattern of LPP findings. The amplitude of the P3 component to a secondary startle probe presented during picture viewing, a measure of attention allocation, significantly increased with repetition in both studies, consistent with the hypothesis that more attention is available for processing the secondary startle probe when the picture has been previously processed. Indeed, with the massive massed repetition in Experiment 2, probe P3 amplitude during picture viewing was no longer different from that elicited in the interpicture interval, implying equivalent processing of the startle probe regardless of whether a (repeated) picture was even present in the visual foreground. Moreover, the increase in probe P3 amplitude with repetition was specifically related to emotional picture processingFrepetition did not change probe P3 amplitude when subjects viewed neutral pictures, suggesting these stimuli attract few attentional resources even when novel. Thus, whereas probe P3 amplitude is typically attenuated when subjects viewed novel emotional, compared to neutral, pictures (Cuthbert et al., 1998), massed repetition effectively eliminated affective modulation of this electrocortical component in both studies. Taken together then, the probe P3 data suggest that ‘‘motivated attention’’ (Lang, Bradley, & Cuthbert, 1997)Fthe heightened attention alloca3 A follow-up test indicated that startle potentiation during the viewing of unpleasant pictures was even larger during massed repetition, compared to the viewing of distributed unpleasant pictures, F(1,21) 5 4.8, po.05, Z2 5 .19.
Repetitive exposure
521
tion directed toward motivationally significant (pleasant or unpleasant) stimuliFdecreases with massed repetition. Affective modulation of the reflexive eyeblink to the secondary startle probe, on the other hand, was preserved despite picture repetition. Even after 30 repetitions, blink reflexes were significantly potentiated for startle probes presented during unpleasant picture viewing (relative to neutral) and reduced when subjects viewed pleasant pictures. Again, the persistence of affective modulation of the startle reflex, also found previously for distributed repetitions (Bradley et al., 1993), is particularly striking in Experiment 2, given repetition of the same picture 30 times in a row. Consistent with a multiprocess model of affective blink modulation (Bradley et al., 2006), startle magnitude when viewing novel pictures (Experiment 1) was significantly smaller overall than those elicited when viewing repeated pictures, supporting the thesis that attention allocation can generally inhibit reflex magnitude. Unlike the probe P3 data, however, the attenuating effects of repetition on blink magnitude were not confined to emotional stimuli, and, in Experiment 2, in which all pictures were repeated (massed or distributed), the type of repetition did not differentially affect blink magnitude. These data suggest that perceptual novelty may be the critical mediator of repetition effects on the overall magnitude of the blink reflex, rather than motivated attention. In any case, it is clear that the blink reflex continues to reflect motivational significance and that this modulation is not affected by repetition. Taken together, these data support a two-process account of LPP modulation during affective picture viewing, in which the amplitude of the late positive potential reflects both motivational significance and differences in initial motivated attention. Thus, like the blink reflex, affective modulation of the LPP persists despite massed repetition, suggesting it indexes motivational significanceFthe activation of fundamental appetitive and defensive motivational systems that are the foundation of emotion (Lang et al., 1997). On the other hand, like probe P3 amplitude, massed repetition affected the amplitude of the LPP specifically for emotional stimuli, reflecting a decrease in attention allocation. A dual process interpretation is consistent with our previous study in which the amplitude of the LPP was independently affected by task relevance and motivational significance (Ferrari, Codispoti, Cardinale, & Bradley, 2008). In that study, when emotional pictures were targets in a categorization task, the LPP was heightened, compared to emotional nontargets. On the other hand, LPP amplitude continued to be enhanced for
emotional, compared to neutral, pictures, regardless of target status. In the current study, attention was manipulated by stimulus repetition, rather than by task relevance, and the data are consistent in showing independent effects of motivational significance and attention allocation. Taken together, the data indicate that the ‘‘natural selective attention’’ that is engaged by the presentation of emotionally evocative cues in the absence of overt tasks or specific instructions (Bradley, 2009; Lang et al., 1997) is greatest for novel pictures and decreases with massed repetition. We have suggested that cues acquire motivational significance from their associations to appetitive and defensive neural circuits that mediate attention, arousal, and action in the service of protecting and sustaining life (e.g., Lang & Bradley, 2008; Lang et al., 1997). For pictures, the similarity of perceptual (visual) features to real-world objects and events serves as the cues that activate the relevant motivational circuit (Bradley & Lang, 2007). It is not surprising that sheer repetition alone is not sufficient to destroy these long-standing associations. Indeed, it has long been recognized that extinctionFrepetitive presentation of cues with existing appetitive or defensive associationsFdoes not eliminate these associations, as evidenced by spontaneous recovery, reinstatement, and their reappearance in a different context (see Myers & Davis, 2007, for an overview). In the present studies, the data suggest that despite repeated massed exposure to an aversive or appetitive picture, affective cues continue to activate the neural circuits mediating appetitive or defensive motivation. That these motivational associations are indexed, in part, by the late positive potential is supported by its continued modulation despite repetition. We have suggested that the activation of fundamental motivational systems mediating appetitive and defensive behavior is the foundation of emotion, giving events and objects ‘‘motivational significance’’ and prompting heightened attention in the service of action (Ferrari et al., 2010; Lang & Bradley, 2008; Lang et al., 1997). The probe P3 data indicate that the initial orienting and heightened attention allocation to emotional cues can be attenuated by repetitive exposure. The persistence of startle modulation, however, is strong evidence that motivational activation continues despite massive repetition. Taken together, then, the data suggest that motivational significance remains following repetitive massed exposure, whereas motivated attention is attenuated or eliminated, and that the amplitude of the late positive potential is sensitive to both emotion and attention.
REFERENCES Balaban, M. T., Losito, B. D. G., Simons, R. F., & Graham, F. K. (1996). Off-line latency and amplitude scoring of the human reflex eye blink with Fortran IV. Psychophysiology, 23, 612. Bradley, M. M. (2009). Natural selective attention: Orienting and emotion. Psychophysiology, 46, 1–11. Bradley, M. M., Codispoti, M., & Lang, P. J. (2006). A multi-process account of startle modulation during affective perception. Psychophysiology, 43, 486–497. Bradley, M. M., Cuthbert, B. N., & Lang, P. J. (1999). Affect and the startle reflex. In M. E. Dawson, A. Schell, & A. Boehmelt (Eds.), Startle modification: Implications for neuroscience, cognitive science and clinical science (pp. 157–183). Cambridge, UK: Cambridge University Press. Bradley, M. M., Hamby, S., Loew, A., & Lang, P. J. (2007). Brain potentials in perception: Picture complexity and emotional arousal. Psychophysiology, 44, 364–373.
Bradley, M. M., & Lang, P. J. (2007). Emotion and motivation. In J. T. Cacioppo, L. G. Tassinary, & G. Berntson (Eds.), Handbook of psychophysiology (3rd ed, pp. 581–607). New York: Cambridge University Press. Bradley, M. M., Lang, P. J., & Cuthbert, B. N. (1993). Emotion, novelty and the startle reflex: Habituation in humans. Behavioral Neuroscience, 107, 970–980. Cacioppo, J. T., Crites, S. L. Jr., & Gardner, W. L. (1996). Attitudes to the right: Evaluative processing is associated with lateralized late positive event-related brain potentials. Personality and Social Psychology Bulletin, 22, 1205–1219. Codispoti, M., Ferrari, V., & Bradley, M. M. (2006). Repetitive picture processing: Autonomic and cortical correlates. Brain Research, 1068, 213–220. Codispoti, M., Ferrari, V., & Bradley, M. M. (2007). Repetition and ERPs: Distinguishing early and late processes in affective
522 picture perception. Journal of Cognitive Neuroscience, 19, 577–586. Codispoti, M., Mazzetti, M., & Bradley, M. M. (2009). Unmasking emotion: Exposure duration and emotional engagement. Psychophysiology, 46, 731–738. Cook, E. W. III (1997). VPM reference manual. Birmingham, Alabama: Author. Cuthbert, B. N., Schupp, H. T., Bradley, M. M., Birbaumer, N., & Lang, P. J. (2000). Brain potentials in affective picture processing: Covariation with autonomic arousal and affective report. Biological Psychology, 52, 95–111. Cuthbert, B. N., Schupp, H. T., Bradley, M. M., McManis, M., & Lang, P. J. (1998). Probing affective pictures: Attended startle and tone probes. Psychophysiology, 35, 344–347. De Cesarei, A., & Codispoti, M. (2008). Fuzzy picture processing: Effects of size reduction and blurring on emotional processing. Emotion, 8, 352–363. Ferrari, V., Bradley, M. M., Codispoti, M., & Lang, P. J. (2010). Detecting novelty and significance. Journal of Cognitive Neuroscience, 22, 404–411. Ferrari, V., Codispoti, M., Cardinale, R., & Bradley, M. M. (2008). Directed and motivated attention during processing of natural scenes. Journal of Cognitive Neuroscience, 20, 1753–1761. Johnston, V. S., Miller, D. R., & Burleson, M. H. (1986). Multiple P3s to emotional stimuli and their theoretical significance. Psychophysiology, 23, 684–694.
V. Ferrari et al. Jungho¨fer, M., Elbert, T., Tucker, D. M., & Rockstroh, B. (2000). Statistical control of artifacts in dense array EEG/MEG studies. Psychophysiology, 37, 523–532. Lang, P. J., & Bradley, M. M. (2008). Appetitive and defensive motivation as the substrate of emotion. In A. Elliott (Ed.), Handbook of approach and avoidance motivation (pp. 51–66). Mahwah, NJ: Erlbaum. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. Gainesville, FL: University of Florida. Lang, P. J., Bradley, M. M., & Cuthbert, M. M. (1997). Motivated attention: Affect, activation and action. In P. J. Lang, R. F. Simons, & M. T. Balaban (Eds.), Attention and orienting: Sensory and motivational processes. Hillsdale, NJ: Erlbaum. Myers, K., & Davis, M. (2007). Mechanisms of fear extinction. Molecular Psychiatry, 2, 120–150. Palomba, D., Angrilli, A., & Mini, A. (1997). Visual evoked potentials, heart rate responses and memory to emotional pictorial stimuli. International Journal of Psychophysiology, 27, 55–67. Vrana, S. R., Spence, E. L., & Lang, P. J. (1988). The startle probe response: A new measure of emotion? Journal of Abnormal Psychology, 97, 487–491.
(Received April 1, 2010; Accepted May 30, 2010)
Psychophysiology, 48 (2011), 523–531. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01082.x
Parafoveal perception during sentence reading? An ERP paradigm using rapid serial visual presentation (RSVP) with flankers
HORACIO A. BARBER,a,n SHIR BEN-ZVI,b,n SHLOMO BENTIN,b,c and MARTA KUTASd,e,f a
Cognitive Psychology Department, University of La Laguna, La Laguna, Spain Psychology Department, Hebrew University of Jerusalem, Jerusalem, Israel c Interdisciplinary Center of Neural Computation, Hebrew University of Jerusalem, Jerusalem, Israel d Department of Cognitive Science, University of California, San Diego, San Diego, California, USA e Department of Neurosciences, University of California, San Diego, San Diego, California, USA f Center for Research in Language, University of California, San Diego, San Diego, California, USA b
Abstract We describe a new procedure using event-related brain potentials to investigate parafoveal word processing during sentence reading. Sentences were presented word by word at fixation, flanked 21 bilaterally by letter strings. Flanker strings were pseudowords, except for the third word in each sentence, which was flanked by either two pseudowords or a pseudoword and a word, one on each side. Flanker words were either semantically congruent or incongruent with the sentence context. P2 (175–375 ms) amplitudes were less positive for contextually incongruent than congruent flanker words but only with flanker words in the right visual field for English and in the left visual field in Hebrew. Flankered word presentation thus may be a suitable method for the electrophysiological study of parafoveal perception during sentence reading. Descriptors: Cognition, Language/speech, EEG/ERP
that parafoveal perception is used for preprocessing of upcoming words, both saccadic programming and the asymmetrical allocation of attentional resources to the hemifields should be influenced by a directional preference based on reading habits and the side on which new information is located. Important contributions of eye-tracking techniques notwithstanding, much remains to be determined about the nature and amount of linguistic information garnered from the parafovea, under what circumstances, and how this information is integrated with foveal information in real time. In this report we introduce a new method for using event related brain potentials (ERPs) associated with parafoveal perception during sentence reading, which we tested in two groups of participants reading sentences with opposite reading directions (Hebrew, English).
Visual acuity varies across the retina due to the heterogeneous concentration of visual receptors, maximal at the fovea with a diameter of about 21 of the visual field around fixation, smaller parafoveally (between 21 and 51), and minimal in the periphery (beyond 51). This characteristic of the visual system has critical implications for reading. A basic finding is that reading (in English) is reliably slower when information to the right of the fixated word is not available than when it is (Rayner, Well, Pollatsek, & Bertera, 1982). However, parafoveal word perception is not equivalent across the two visual hemifields, with the asymmetry depending on the direction in which letters and words are scanned. For scripts in which reading is from left to right, as in Western writing systems, there is a right visual field (RVF) advantage (Rayner, Well, & Pollatsek, 1980). By contrast, the opposite asymmetry is found when reading is from right to left as in Hebrew (Deutsch, Frost, Pelleg, Pollatsek, & Rayner, 2003; Pollatsek, Bolozky, Well, & Rayner, 1981). On the assumption
Electrophysiological Signatures of Words Perceived in the Parafovea ERPs have become an increasingly popular tool in the study of language comprehension in general and in reading in particular (for reviews, see Barber & Kutas, 2007; Kutas, Van Petten, & Kluender, 2006). One of the limitations of this technique, however, is that electroencephalographic recording is also affected by field potentials caused by eye movements; indeed, such activity can produce undesirable artifacts in the average response (see Berg & Scherg, 1991). For this reason, psycholinguistic researchers typically study reading with ERPs in the absence of
Horacio A. Barber was funded by the ‘‘Ramo´n y Cajal’’ program and Grant SEJ2007-67364 of the Spanish Ministry of Science. Marta Kutas was supported by Grant HD22614 from the U.S. National Institute of Child Health and Human Development, and Grant AG08313 from the National Institute of Aging. n These authors equally contributed to this work. Address correspondence to: Horacio A. Barber, Departamento de Psicologı´ a Cognitiva, Campus de Guajara, Universidad de La Laguna, 38205-Tenerife, Spain. E-mail:
[email protected] 523
524 lateral movements by presenting sentences one word at a time at a single (usually fixation) point, asking the reader to minimize eye movements (e.g., Kutas & Hillyard, 1980). ERP studies using this rapid serial visual presentation (RSVP) method have provided considerable insights on word processing in sentential contexts. This paradigm, however, is non-ecological and cannot be used to investigate the role of parafoveal perception during reading. One potentially promising approach to the study of parafoveal perception during reading relies on the simultaneous recording of eye movements and ERPs. Pioneer attempts by Marton and Szirtes (1988) and Marton, Szirtes, and Breuer (1985), for example, demonstrated that brain responses could be time-locked to the onset of a saccade, leading to the ‘‘presentation’’ of the sentence final word. With the advent of new signal processing techniques, there has been a resurgence of interest in this type of saccade–ERP approach (Baccino & Manunta, 2005; Hutzler et al., 2007; Kretzschmar, Bornkessel-Schlesewsky, & Schlesewsky, 2009; Simola, Holmqvist, & Lindgren, 2009). Baccino and Manunta, for instance, recorded eye-fixation-related potentials (EFRPs) to French word pairs, one at fixation and the other in the right parafovea. They reported effects on early components (N1 and P140) contingent on the lexical status (word/ nonword) of the parafoveal stimulus and on subsequent components (P2), reflecting the associative relationship between the two words. Simola and colleagues extended this design to include words either in the right or in the left visual field and showed P2 lexical effects for target words in right but not left visual field. Kretzschmar et al. examined saccadic-locked ERPs to sentence final words and found a dissociation between foveal and parafoveal processes: Predictability affected the foveal processing of the final word whereas context congruency modulated the responses to the previous word. Clearly, this is a very promising line of investigation. The co-registration of EEG and eye-tracking measures during sentence reading, however, raises several methodological difficulties. In addition to those imposed by ocular artifacts, it is not easy to disentangle overlapping signals obtained when words are read at the fast rates characteristic of natural reading even with certain assumptions. The Present Study We thus describe a complementary method for examining parafoveal processing based on a modification of the canonical wordby-word sentence presentation (i.e., RSVP) procedure, in which sentences are presented foveally as usual, but each word is flanked bilaterally by letter strings, one string on each side. In the two ERP experiments reported herein, short sentences were presented one word at a time on a computer screen at fixation, flanked 21 bilaterally by letter strings. All but the third word in each sentence was flanked by two pseudowords, one on each side. The third word in each sentence was flanked bilaterally either by two pseudowords or by a pseudoword on one side and a word on the other. The flanker word appeared either on the left or on the right side and was either semantically congruent with the sentence context (identical to the upcoming fourth word) or not. For example, in ‘‘Chatty barbers trim beards while talking,’’ the critical third word ‘‘trim’’ was flanked on one side by a six-letter pseudoword and on the other side either by the semantically congruent (upcoming) word ‘‘beards’’ or the semantically incongruent word ‘‘crises.’’ Based on the (small) extant literature, we expected that semantic congruence of the parafoveal (flanker) word would modulate the P2 and perhaps the N400 components
H.A. Barber et al. of the ERPs to target items or to the following triad, although any reliable ERP effect would be evidence of parafoveal influence. The P2 is a sensory component sensitive to manipulations of visual feature extraction, attention, and contextual constraint (Federmeier & Kutas, 1999, 2002; Luck & Hillyard, 1994; Skrandies, 2003). Therefore, to the extent that congruent flanker words, compared to incongruent flanker words, would show more positive P2 amplitudes we would infer facilitation of the word recognition and/or integration processes triggered by the critical triad. The subsequent N400 component of the ERP is sensitive to both semantic congruity and word level associative/ semantic priming (for a recent review, see Kutas et al., 2006). Accordingly, smaller negative N400 amplitudes for the congruent versus incongruent word flankers would be taken to indicate activation of the semantic system triggered by parafoveal information. As our primary aim was to demonstrate the utility of this flankered RVSP procedure for the study of parafoveal processing, we conducted two different experiments with two different languages that differed in their orthography and reading direction (from left to right in English and from right to left in Hebrew). In so doing, whatever the nature of the specific ERP effects, we expected to see evidence of parafoveal perception during sentence reading interacting with the visual field in which the critical information was presented. For English, read from left to right, we expected to see parafoveal influences driven by words appearing in the left but not right visual field, whereas for Hebrew, read from right to left, we expected to see the reverse: parafoveal influences from words in the left but not the right visual field. Method Two similar experiments were conducted, one using English sentences as stimuli and English native speakers as participants and the other with native Hebrew speakers reading Hebrew sentences. As experimental procedures were kept as similar as possible for both experiments, they are jointly described below. Participants In the English experiment, the participants were 24 overseas students at the Hebrew University of Jerusalem aged 19 through 38 (mean age 21.5 years, 5 men) who were native English speakers; likewise, in the Hebrew experiment the participants were 24 students at the Hebrew University of Jerusalem aged 22 through 34 (mean age 25.8 years, 10 men), who were native Hebrew speakers. All of the participants were right-handed and all reported normal or corrected-to-normal sight and no history of neurological disorders. They received experimental credits or payment for their participation in the study. Informed consent conforming to the requirements of the Hebrew University experimental ethics committee was obtained after the experimental procedures were explained to them. Stimuli Sentences with a similar structure were used in the English and Hebrew experiments. Each sentence comprised 5 to 10 words. The Hebrew sentences comprised 5 words, whereas English sentences had between 6 and 10 words (mean 5 7.17). The third word (the ‘‘critical’’ word) was always a verb, and the fourth was always a noun. The words of the sentence were presented sequentially, one at a time at fixation on a CRT monitor, each flanked by two letter strings. The distance between fixation and
ERPs and parafoveal perception in reading the most medial external letter of the flankers was 21. Note that, whereas in English this was the first letter of the right flanker and the last letter of the left flanker, in Hebrew the order is reversed. For all words in the sentence but the critical word (the third), the flankers were pseudowords. The critical word was flanked either by two pseudowords (like the rest of the words in the sentence) or by a word (noun) in the left or in the right side and a pseudoword in the contralateral side. The nature of the stimuli flanking the third (critical) word defined a 2 ! 2 factorial design, with the factors Flanker position (left vs. right), and Condition (semantically congruent vs. incongruent). This design thus yielded four conditions: (1) In the congruent-right condition, the right flanker was a noun identical to the subsequent fourth word in the sentence (hence, semantically congruent with the first three words of the sentence). (2) In the congruent-left condition, the left flanker was identical to the fourth word. (3) In the incongruent-right condition, the right flanker was a noun different from and unrelated to either the critical word or the subsequently presented fourth word (and it was also semantically incongruent with the sentence). (4) In the incongruent-left condition, the left flanker was an unrelated noun. Table 1 gives examples of the stimuli in both languages. It should be noted that indefinite plurals were used in English as subjects (e.g., ‘‘Chatty barbers’’) in order to equate the number of words in noun phrases across two languages, because Hebrew determiners (e.g., ‘‘the’’ or ‘‘a’’) are morphologically integrated in the nouns. Also, note that although noun phrases were composed of a noun and an adjective in both languages, the canonical word order in the two differs (adjective preceding the noun in English and adjective following the noun in Hebrew). Lexical frequency of congruent and incongruent flankers was matched for the stimulus lists of the Hebrew experiment. The average written word frequency of the Hebrew flankers (Hebrew Word Frequency Database: http://homepages.inf.ed.ac.uk) was 32 (SD 5 129.89) per million for congruent flankers and 16.95 (SD 5 65.10) per million for incongruent flankers. A one-way analysis of variance (ANOVA) carried out on both lists confirmed that they were not significantly different, F(1,159) 5 2.57, p 5 .1. The average written word frequency of the English flankers (CELEX database; Baayen, Piepenbrock, & Gulikers, 1995) was 54.42 (SD 5 184.14) per million for congruent flankers and
525 17.66 (SD 5 47.46) for incongruent flankers. A one-way ANOVA carried out on the congruent–incongruent lists showed that this difference was statistically significant, F(1,159) 5 5.97, po.05. Pseudowords of different lengths were created in English and in Hebrew to be used as flankers in each experiment. The English pseudowords were taken from the ARC Nonword Database (Rastle, Harrington, & Coltheart, 2002); the Hebrew ones were generated by a group of native Hebrew speakers. Pseudowords were defined as letter strings that fulfill the orthographic and phonological rules of the respective language but with no meaning associated with them. To avoid attentional biases due to flanker length, the two flankers for any specific word were matched in length (number of letters) to the subsequent central word in the sentence, and the length of the final flankers was determined by the final central word. The Hebrew as well as the English words varied in length from 2 to 11 letters. The average number of letters per word at the critical position was 4.3 in the Hebrew sentences and 5.7 in the English sentences. The average number of letters per flanker at the critical position was 4.5 in the Hebrew sentences and 6.6 in the English sentences. Semantically congruent and incongruent flanker words were matched in length. It should be noted that all sentences at the central position were semantically plausible and grammatically correct, and participants were instructed to read only those central words. One hundred sixty experimental sentences in English and in Hebrew were created by native speakers, and four different sentence lists were constructed for each experiment in order to counterbalance the different conditions. Across participants, each sentence was presented in the four conditions, whereas within participants each sentence was presented only once. Additionally, there were 80 filler sentences for each experiment in which the critical word also was flanked by two pseudowords. Procedure The experimental procedure was identical in both experiments. Participants were seated comfortably in a semi-darkened soundattenuated booth after being fit with an electrode cap. All stimuli were presented on a high-resolution monitor (1024 ! 768 pixels) positioned at eye level 70 cm in front of the participant. All the string letters were displayed in black lowercase against a white
Table 1. Example Stimuli (Words at the Critical Position Are Underlined for Illustration) Congruent flanker
Incongruent flankers
cereal pies tails
rhymes surf veins
The rough pipe empties a round pool.
pool
notebook
The industrious scientist operated an anesthetized rabbit.
rabbit
pencil sharpener
The beautiful singing gave me hope.
hope
chain
English sentences My cousin eats cereal for breakfast Jolly clowns toss pies in the circus Excited puppies wag tails and bark Hebrew sentences and English translation
526
H.A. Barber et al.
Figure 1. Sentence presentation procedure.
background. A single trial consisted of the presentation of the fixation point (a red dot) for a random duration between 1250 and 1400 ms, followed by the sentence presentation in 5 to 10 displays, each consisting of three letter strings as described above exposed for 260 ms, with an interdisplay duration of 60 ms (Figure 1). Participants were asked to read the sentences silently and be ready to answer comprehension questions related to sentence meaning. The questions were presented randomly at the end of the sentence on about 25% of the trials and required a yes/no answer via one of two button presses. These questions were to ensure that subjects read the sentences for meaning. Participants did not report any difficulties comprehending the sentences, and the number of errors was less than 5% in both experiments. Although participants were not explicitly asked to ignore flankers, they were told that lateral information was irrelevant for task performance (and sentence meaning) and were asked to maintain focus on the center of the screen and to avoid eye movements and blinks during the interval spanning the fixation point until the end of the trial. The interval between trials varied randomly between 1.5 and 2 s. The experiment was divided into six blocks of 40 sentences each, with a short rest between blocks. The sentences were presented in a different random order for each participant. Twelve practice trials with characteristics similar to those of the experimental trials were presented at the beginning of the session and were repeated when necessary. EEG Recording EEG was recorded via 64 Ag-AgCl electrodes attached to an elastic electrode cap (ECI Inc., Eaton, OH) according to the extended 10–20 system (see Figure 2), and 3 external electrodes, two placed at the two mastoids and a third one on the tip of the nose that was used as online reference. Eye movements and blinks were monitored via 4 additional external electrodes providing bipolar recordings of the horizontal and vertical electrooculogam (EOG): Two electrodes were located at the outer can-
thus of the right and left eyes and two at the infraorbital and supraorbital regions of the right eye. Both EEG an EOG were sampled at 256 Hz using the Biosemi Active II digital 24-bit amplification system (http://www.biosemi.com) with an active input range of ! 262 mV to 1262 mV per bit and a low-pass filter of 64 Hz to avoid aliasing. The digitized EEG was saved and processed off-line. Data Analysis Raw data were bandpass filtered between 0.1 and 30 Hz (dB) and re-referenced to the average of the two mastoid electrodes before analysis. Ocular artifacts were corrected using independent components analysis, and remaining artifacts exceeding " 100 mV in amplitude or containing a transient of over 100 mV in a period of 100 ms were rejected along with an epoch of 300 ms symmetrical around the event. Following this procedure, average ERPs resulted from individual segments starting 100 ms before and ending 500 ms after the critical triad onset (word/pseudowordFword 3Fword/pseudoword),1 separately for each of the four conditions, each electrode, and each participant. The baseline was adjusted by subtracting the mean amplitude of the prestimulus activity from all the data points in the epoch. Twelve separate regions of interest were computed from 48 lateral electrodes, each comprising the mean of 4 electrodes (Figure 2). There were six electrode groups in each hemisphere: two in each of the anterior, posterior, and central scalp areas, one in the lateral, and one in the medial position of the hemisphere: left anterior lateral (F7, F5, FT7, FC5), left anterior medial (F3, F1, FC3, FC1), left central lateral (T7, C5, TP7, CP5), left central medial (C3, C1, CP3, CP1), left posterior lateral (P7, P5, P9, PO7), left posterior medial (P3, P1, PO3, O1), right anterior medial (F2, F4, FC2, FC4), right anterior lateral (F6, F8, FC6, 1 Analyses of the ERPs time-locked to the onset of the following triad were also performed but not reported here because there were no significant effects.
ERPs and parafoveal perception in reading
527
Figure 2. Schematic flat representation of the 64 electrode positions from which EEG activity was recorded. The grouped electrodes are those analyzed in the 12 critical regions.
FT8), right central medial (C2, C4, CP2, CP4), right central lateral (C6, T8, CP6, TP8), right posterior medial (P2, P4, O2, PO4), and right posterior lateral (P6, P8, PO8, P10). The mean
amplitude of two different epochs compromising the P2 and N400 components (175–375 and 375–475 ms, respectively) was analyzed using a mixed-model ANOVA with Language as a be-
ENGLISH
Left flankers
Right flankers
F5
F1
F2
F6
F5
F1
F2
F6
C5
C1
C2
C6
C5
C1
C2
C6
P5
P1
P2
P6
P5
P1
P2
P6
P2 Heog
congruent incongruent
Heog
µV
–3 0
200 400 ms
Figure 3. ERPs showing congruent and incongruent flanker words presented to the left visual field in the English sentences.
528
H.A. Barber et al. HEBREW
Left flankers F5
F1
C5
C1
P5
P1
F2
Right flankers
F6
F5
F1
F2
F6
C6
C5
C1
C2
C6
P6
P5
P1
P2
P6
P2 C2
P2
Heog congruent incongruent
Heog
µV
–3 0
200 400
ms
Figure 4. ERPs showing congruent and incongruent flanker words presented to the left visual field in the Hebrew sentences.
tween-subjects factor and Flanker position (left, right), Condition (congruent, incongruent), and Area (12 electrode groups) as within-subjects factors. In cases where the sphericity assumption was violated, Greenhouse–Geisser-corrected degrees of freedom and p values are reported. Effects of the Area factor will be reported only when it interacts with the experimental manipulations. In addition, post hoc Sida´k contrasts (Sida´k, 1967) were performed after interactions or main effects of Flanker Word to control for Type I error in multiple comparisons. Results ERPs time-locked to the onset of the critical triads included a series of negative and positive peaks during the first 500 ms identified as N1, P2, and N400 components (even when the N400
Figure 5. Mean amplitudes of the P2 component (175–375 ms) in the right posterior medial region for both groups and both visual fields.
partially overlaps with the N1-P2 complex of the following triad). Figures 3 and 4 show the grand average waveforms corresponding to the congruent and incongruent conditions in the experiment in English (Figure 3) and in the experiment in Hebrew (Figure 4). Twelve representative electrodes are plotted, corresponding with the analyzed electrode groups. ERPs elicited by congruent and incongruent flanker words revealed amplitude differences starting at about 175 ms after stimulus onset and lasting for about 200 ms. During that epoch the P2 elicited in the congruent condition was larger (more positive) than that elicited in the incongruent condition. Critically, this effect is unilateral, appearing on opposite sides for Hebrew and English. Conforming to reading direction, in English sentences the effect emerged when relevant information appeared in the right parafovea (see Figure 3) whereas, in Hebrew, it emerged when information appeared in the left parafovea (see Figure 4). This interaction can be observed in Figure 5, in which P2 mean amplitudes in the right posterior medial region (C2, C4, CP2, CP4) are graphed. Figure 6 represents the topographical distribution of the P2 differences across the scalp (incongruent minus congruent ERPs). The effect in English is localized at posterior areas; the effect in Hebrew shows a broader distribution maximum at frontal electrodes. These observations were corroborated by the mixed-model ANOVA. P2 Time Window: 175–375 ms The analyses in this time window showed a significant three-way interaction of Flanker position ! Condition ! Area, F(11,506) 5 2.95, po.05; e 5 .34; MSE 5 2.02; Z2 5 .06, which was further modulated by a four-way interaction with Language F(11,506) 5 2.51, po.05; MSE 5 2.02; Z2 5 .05. The differential pattern of effects in English and Hebrew (as substantiated by the four-way interaction) was further investigated by separate Flanker position ! Condition ! Area ANOVAs for each language. For English sentences, the ANOVA resulted in a significant second-order interaction between the three factors, F(11,253) 5 4.14, po.01; e 5 .29; MSE 5 2.8; Z2 5 .15. Post hoc tests revealed that in the right parafovea,
ERPs and parafoveal perception in reading
529
Figure 6. Topographical maps of the P2 differences over the scalp (incongruent minus congruent conditions).
congruent flanker words produced more positive mean values than incongruent flanker words, but only in the right posterior lateral area, F(1,23) 5 5.01, po.05. No significant differences were found in any other area, confirming the posterior distribution of the effect in this experiment. Additionally, no significant differences were obtained when the flanker manipulation occurred in the left parafovea (Fo1). This analysis supports a modulation of ERP amplitude associated with lexical-semantic information displayed in the right parafovea in English, which is read from left to right. For Hebrew sentences, an ANOVA showed a significant Position ! Condition interaction, F(1,23) 5 4.44, po.05; MSE 5 16.05; Z2 5 .16. Post hoc tests revealed that the mean amplitude elicited in the congruent-left condition was bigger than in the incongruent-left condition, but only for flankers presented in the left parafovea, F(1,23) 5 4.6, po.05. There were no significant differences when the flanker words were presented in the right parafovea (Fo1). Although the interaction with the factor area did not reach the level of significance (Fo1), post hoc tests showed significant differences for specific areas (mostly at right hemisphere sites): right central medial, F(1,23) 5 4.6, po.05, right posterior medial, F(1,23) 5 4.45, po.05, right posterior lateral, F(1,23) 5 5.88, po.05, and also at left anterior medial, F(1,23) 5 4.7, po.05. This analysis supports a modulation of the ERP amplitude when lexical-semantic information is displayed in the left parafovea in Hebrew, which is read from right to left.
N400 Time Window: 375–475 ms As can be seen in Figure 4, the congruency effect of the left flankers in the Hebrew sentences lasts beyond the analyzed window for the P2 component. These differences remain visible in the time window of the N400 component, which peaks around 425 ms across conditions. For this reason, additional ANOVAs were performed on the mean amplitude values between 375 and
475 ms after stimulus onset. However, these analyses did not yield any significant interaction involving the Condition factor. Discussion The present study explored the processing effects of words presented in the parafovea during the (foveal) reading of Hebrew or English sentences using ERPs. To that end, we assessed the efficacy of a new stimulus presentation procedure. Specifically, we recorded ERPs as sentences appeared one word at a time at fixation, flanked on either side by letter strings. Of experimental interest were the ERP modulations to letter string flankers timelocked to the third word of each sentence, as these included a word that was congruent or incongruent with the sentence context on either the right or left side of the word at fixation. In brief, our findings revealed that this paradigm may indeed be a useful way to study the effects of parafoveal information on word reading. We found that ERPs to the manipulated triads (left visual field, center, right visual field) were sensitive to the nature of the parafoveal information. Specifically, amplitudes of the P2 component (measured 175–375 ms after the onset of the critical triads) of the ERP were larger when the flanker word was congruent with the overall sentence context than when it was incongruent. Moreover, this parafoveal effect interacted with the visual field in which the flanker word appeared, with the pattern of the interaction varying with reading direction: For English, normally read from left to right, the flanker effect was reliable only when the contextually incongruent flanker word appeared in the right parafovea. In contrast, for Hebrew, read from right to left, the effect of the flanker was reliable only when the contextually incongruent flanker word appeared in the left parafovea, that is, the reverse. Future studies will need to replicate these findings and delve into the differences in the scalp topography of the P2 effects in the two languages. Distributional differences notwithstanding, the ERP measures reveal an impact of parafoveal information on central word processing. Importantly, an effect of reading direction obtained despite the fact the sentences
530 were presented word by word in the same location, that is, with no horizontal scanning as in natural reading. Although eye tracking was not implemented in these experiments, the early onset of the effect (175 ms) is inconsistent with the interpretation of this effect as due to foveal stimulus perception consequent to lateral eye movements. The earliest reliable reported ERP effects associated with lexical variables, when one single word is presented at fixation, ranges also between 100 and 200 ms (see Barber & Kutas, 2007). It is thus relatively unlikely that two words can be sequentially perceived by means of a saccade, producing ERP effects in a similar time range. In sum, with this RSVP flanker paradigm, we found P2 amplitude modulations by parafoveal information similar to those reported with event fixation related potentials, namely, when participants are moving their eyes (Baccino & Manunta, 2005; Simola et al., 2009). Although our P2 modulation is consistent with semantically driven contextual effects (e.g., Federmeier & Kutas, 1999, 2002), with the present design and data we can conclude only that words in the parafovea were processed, at least at a form level. We did not observe any reliable and consistent effects on the N400 component of the ERP to the critical (third) word triad or to the following word. The absence of N400 modulation by flankers’ semantic congruence suggests that the flankers probably did not activate the semantic system extensively. This result is not without precedent. Some eye-tracking studies of reading have consistently failed to demonstrate semantic effects in the parafovea, leading to the conclusion that words outside fixation are processed only at the level of form (e.g., length or orthography: for review, see Rayner, White, Kambe, Miller, & Liversedge, 2003). In contrast, however, one study using concurrent EEG and eye-movement recordings reported a semantic N400 effect that the authors attributed to parafoveal perception (Kretzschmar et al., 2009). For the moment, different pattern of effects are inexplicable. As the primary goal of our experiment was not to determine the specific information extracted from the parafovea but rather to assess the RSVP-flanker paradigm, the words in our semantically congruent and incongruent conditions were not carefully matched in all lexical or sublexical variables (e.g., lexical frequency2 or orthographic regularities). Our ERP data, thus, cannot help resolve these inconsistencies, though our findings demonstrate that the presentation method we introduce could be informative, in principle. In the present study, the use of (meaningless) pseudowords as flankers for almost all the central words could reasonably have led our participants to adopt an (unconscious) strategy of focusing their processing resources on the analysis of the formal aspects of the flankers (e.g., orthographic regularities) at the expense of meaning construction and contextual integrationF processes to which the N400 has been variously linked. Our flanker RVSP method, however, can be modified such that all flankers for central words at every sentential position are words, thereby making flanker information more ecologically valid as well as methodologically more relevant for inferences about reading (and parafoveal) processes.
2 However, it is important to note that the parafoveal effect was larger in the Hebrew experiment, where mean lexical frequent values of congruent and incongruent flankers were not statistically different. Therefore, it is unlikely that lexical frequency accounts for the reported effects.
H.A. Barber et al. In line with eye-movement research on sentence reading in both writing systems (Deutsch et al., 2003; Pollatsek et al., 1981; Rayner et al., 1980) as well as psychophysiological word pair experiments (Pernet, Uusvuori, & Salmelin, 2007; Simola et al., 2009), we observed an interaction of the parafoveal effects with the flanker’s position to the left or the right side of the fixated word in a direction that varied with the direction typical of reading in each of the different orthographies. In English, the flanker effect was reliable only when the flanker word was positioned to the right of the fixated word whereas the opposite pattern obtained in Hebrew. Previous studies have shown left-hemisphere superiority for language processing even in languages with leftto-right reading such as Hebrew (Bentin, 1981; Nazir, BenBoutayab, Decoppet, Deutsch, & Frost, 2004; Smolka & Eviatar, 2006). However, cerebral specialization cannot account for the visual field effects in the present study, opposite in Hebrew and English. Similarly, the assumption that parafoveal effects are related to the programming for the upcoming saccade, which has been entertained by eye-tracking studies, is not supported by these ERP data. Although eye movements were not monitored in the present study, participants were instructed to refrain from moving the eyes, and the sequential presentation of the sentence words at fixation coupled with noninformative pseudowords in the parafovea did not encourage systematic saccades. The visual field effects in the present study could reasonably be explained by an asymmetry in the natural deployment of visual attention during reading, imposed by reading direction. Behavioral studies have shown that the typical RVF advantage for visual word recognition can be reduced or even eliminated when lateralized words are precued. Precues are presumed to guide exogenous spatial attention mechanisms, thereby leading to more efficient lexical processing in the precued visual field (Ducrot & Grainger, 2007; Ortells & Tudela, 1996). Along with this account, as well as with the present pattern, effects of covert attention (in the absence of gaze shifting) on parafoveal lexical processing have been demonstrated in several priming studies in which lateral eye movements were controlled or eliminated (Calvo & Nummenmaa, 2008; Hyo¨na¨ & Koivisto, 2006; Marzouki & Grainger, 2008). It is also worth noting that parafoveal information did not play any role in the specific task demands for our participants, and the majority of the parafoveal stimuli were meaningless pseudowords, which did not add to the sentence meaning. Indeed, most participants reported being unaware that any words were presented in the parafovea. Therefore, the observed flanker effect in the present study seems most reasonably associated with uncontrolled (or at least veiled controlled; Shiffrin & Schneider, 1977) processing of sublexical/lexical information in the parafovea, rather than with task goals or reading strategies. These speculations clearly call for more controlled studies using this new presentation paradigm. In summary, we show that ERP data collected using an RSVP paradigm with letter string flankers, in the absence of directional scanning, can offer reliable evidence of the word processing effects of at least some word form information appearing in the parafovea. This appears to be the case when the flankers appear in locations consistent with reading location. We suggest that this effect may be accounted for by attentional factors resulting from reading habits, and we encourage the adoption of this flankerRSVP ERP methodology to complement other approaches to the investigation of the influences of parafoveal information during reading.
ERPs and parafoveal perception in reading
531 REFERENCES
Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). The CELEX Lexical Database (Release 2) [CD-ROM]. Philadelphia: University of Pennsylvania, Linguistic Data Consortium. Baccino, T., & Manunta, Y. (2005). Eye-fixation-related potentials: Insight into parafoveal processing. Journal of Psychophysiology, 19, 204–215. Barber, H. A., & Kutas, M. (2007). Interplay between computational models and cognitive electrophysiology in visual word recognition. Brain Research Reviews, 53, 98–123. Bentin, S. (1981). On the representation of a second language in the cerebral hemispheres of right handed people. Neuropsychologia, 19, 599–603. Berg, P., & Scherg, M. (1991). Dipole models of eye movements and blinks. Electroencephalography & Clinical Neurophysiology, 79, 36–44. Calvo, M. G., & Nummenmaa, L. (2008). Lateralised covert attention in word identification. Laterality, 5, 1–18. Deutsch, A., Frost, R., Pelleg, S., Pollatsek, A., & Rayner, K. (2003). Early morphological effects in reading: Evidence from parafoveal preview benefit in Hebrew. Psychonomic Bulletin & Review, 10, 415–422. Ducrot, S., & Grainger, J. (2007). Deployment of spatial attention to words in central and peripheral vision. Perception & Psychophysics, 69, 578–590. Federmeier, K. D., & Kutas, M. (1999). Right words and left words: Electrophysiological evidence for hemispheric differences in meaning processing. Cognitive Brain Research, 8, 373–392. Federmeier, K. D., & Kutas, M. (2002). Picture the difference: Electrophysiological investigations of picture processing in the two cerebral hemispheres. Neuropsychologia, 40, 730–747. Hutzler, F., Braun, M., Vo˜, M. L., Engl, V., Hofmann, M., Dambacher, M., et al. (2007). Welcome to the real world: Validating fixationrelated brain potentials for ecologically valid settings. Brain Research, 1172, 124–129. Hyo¨na¨, J., & Koivisto, M. (2006). The role of eye movements in lateralised word recognition. Laterality, 11, 155–169. Kretzschmar, M., Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2009). Parafoveal versus foveal N400s dissociate spreading activation from contextual fit. NeuroReport, 20, 1613–1618. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203–205. Kutas, M., Van Petten, C., & Kluender, R. (2006). Psycholinguistics electrified II (1994–2005). In M. A. Gernsbacher & M. Traxler (Eds.), Handbook of psycholinguistics (2nd ed, pp. 659–724). New York: Elsevier Press. Luck, S. J., & Hillyard, S. A. (1994). Electrophysiological correlates of feature analysis during visual search. Psychophysiology, 31, 291– 308. Marton, M., & Szirtes, J. (1988). Context effects on saccade-related brain potentials to words during reading. Neuropsychologia, 26, 453–463.
Marton, M., Szirtes, J., & Breuer, P. (1985). Electrocortical signs of word categorization in saccade-related brain potentials and visual evokedpotentials. International Journal of Psychophysiology, 3, 131–144. Marzouki, Y., & Grainger, J. (2008). Effects of prime and target eccentricity on masked repetition priming. Psychonomic Bulletin & Review, 15, 141–148. Nazir, T. A., Ben-Boutayab, N., Decoppet, N., Deutsch, A., & Frost, R. (2004). Reading habits, perceptual learning, and recognition of printed words. Brain & Language, 88, 294–311. Ortells, J. J., & Tudela, P. (1996). Positive and negative semantic priming of attended and unattended parafoveal words in a lexical decision task. Acta Psychologica, 94, 209–226. Pernet, C., Uusvuori, J., & Salmelin, R. (2007). Parafoveal-on-foveal and foveal word priming are different processes: Behavioral and neurophysiological evidence. NeuroImage, 38, 321–330. Pollatsek, A., Bolozky, S., Well, A. D., & Rayner, K. (1981). Asymmetries in the perceptual span for Israeli readers. Brain and Language, 14, 174–180. Rastle, K., Harrington, J., & Coltheart, M. (2002). 358,534 nonwords: The ARC nonword database. Quarterly Journal of Experimental Psychology, 55A, 1339–1362. Rayner, K., Well, A. D., & Pollatsek, A. (1980). Asymmetry of the effective visual field in reading. Perception & Psychophysics, 27, 537–544. Rayner, K., Well, A. D., Pollatsek, A., & Bertera, J. H. (1982). The availability of useful information to the right of fixation in reading. Perception & Psychophysics, 31, 537–550. Rayner, K., White, S. J., Kambe, G., Miller, B., & Liversedge, S. P. (2003). On the processing of meaning from parafoveal vision during eye fixations in reading. In J. Hyo¨na¨, R. Radach, & H. Deubel (Eds.), The mind’s eye: Cognitive and applied aspects of eye movement research (pp. 213–234). Amsterdam: Elsevier Science. Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychological Review, 84, 127–190. Sida´k, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62, 626–633. Simola, J., Holmqvist, K., & Lindgren, M. (2009). Right visual field advantage in parafoveal processing: Evidence from eye-fixationrelated potentials. Brain and Language, 111, 101–113. Skrandies, W. (2003). Evoked potentials studies of visual information processing. In A. Zani & A. Proverbio (Eds.), The cognitive electrophysiology of mind and brain (pp. 71–92). New York: Academic Press. Smolka, E., & Eviatar, Z. (2006). Phonological and orthographic visual word recognition in the two cerebral hemispheres: Evidence from Hebrew. Cognitive Neuropsychology, 23, 972–989. (Received April 21, 2010; Accepted May 28, 2010)
Psychophysiology, 48 (2011), 532–546. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01085.x
FN400 potentials are functionally identical to N400 potentials and reflect semantic processing during recognition testing
JOEL L. VOSSa AND KARA D. FEDERMEIERa,b a
Beckman Institute for Advanced Science and Technology, University of Illinois Urbana–Champaign, Urbana, Illinois, USA Department of Psychology, University of Illinois Urbana–Champaign, Urbana, Illinois, USA
b
Abstract The ‘‘F’’ in FN400 denotes a more frontal scalp distribution relative to the morphologically similar N400 componentFa distinction consistent with the hypothesized distinct roles of FN400 in familiarity memory versus N400 in language. However, no direct comparisons have substantiated these assumed dissimilarities. To this end, we manipulated short-term semantic priming during a recognition test. Semantic priming effects on N400 were indistinguishable from memory effects at the same latency, and semantic priming strongly modulated the ‘‘FN400,’’ despite having no influence on familiarity memory. Thus, no evidence suggested either electrophysiological or functional differences between the N400 and FN400, and findings were contrary to the linking of the ‘‘FN400’’ to familiarity. Instead, it appears that semantic/conceptual priming (reflected in the N400) occurs during recognition tests and is frequently (mis)labeled as FN400 and attributed to familiarity. Descriptors: Learning/memory, Language/speech, Normal volunteers, EEG/ERP
recollection whenever these potentials are identified, that is, make reverse inferences of familiarity and recollection based on FN400 and late-parietal potentials (e.g., Czernochowski, Mecklinger, & Johansson, 2009; Ecker, Arend, Bergstrom, & Zimmer, 2009; Klonek, Tamm, Hofmann, & Jacobs, 2009; Mecklinger, Brunnemann, & Kipp, 2010; Nyhus & Curran, 2009; Opitz & Cornell, 2006; Speer & Curran, 2007). However, as we review below, recent work has called into question the mapping between the FN400 and familiarity and has suggested that the FN400 may not be different from the N400 component. The N400 has been characterized primarily within the domain of language comprehension, but also more broadly, and has been linked to implicit semantic access processes. The FN400 bears striking similarity to the N400 in morphology, timing, and response pattern; indeed, effects that are now referred to as FN400s were originally characterized as N400s in the early literature looking at cognitive electrophysiological responses in memory paradigms. Yet, to our knowledge, no empirical work has attempted to elicit both responses in tandem to directly compare them. We therefore set out to answer fundamental and yet untested questions regarding FN400 potentials: Are they in fact N400 potentials present during recognition tests, and are they related to semantic processing rather than to familiarity? The N400 component was first characterized by Kutas and Hillyard (1980), who found larger negative ERP amplitudes to unexpected relative to expected words completing sentences.
Familiarity and recollection are two hypothesized processes that contribute to performance in tests of recognition memory (Mandler, 1980; Yonelinas, 2002). Many investigators interpret two distinct ERP effects obtained during recognition tests as unique neural correlates of familiarity and recollection: FN400 old/new effects and late-parietal old/new effects, respectively (Mecklinger, 2000; Paller, Voss, & Boehm, 2007; Rugg & Curran, 2007). The validity of these hypothesized associations is of substantial theoretical importance. For instance, the dissociation between the FN400 and late-parietal ERPs has been widely taken as reification of the constructs of familiarity and recollection and, thus, as justification for launching more detailed neuroanatomical studies of these hypothesized processes (e.g., Eichenbaum, Yonelinas, & Ranganath, 2007). Moreover, investigators increasingly cite the strength of associations between the FN400 and familiarity and late-parietal potentials and recollection as justification to infer the involvement of familiarity and/or
We thank Brian Gonsalves for providing access to his EEG equipment, Ashley Galvan for help collecting data, and Ken Paller for commenting on an earlier draft of this article. Research support was provided by a Beckman Institute Postdoctoral Fellowship award and by National Institutes of Health Grants K99-NS069788-01 to J.L.V. and R01-AG026308 to K.D.F. Address correspondence to: Joel L. Voss, Beckman Institute, 405 N. Mathews Avenue, Urbana, IL 61801, USA. E-mail: joelvoss@ illinois.edu 532
Why the ‘‘F’’ in FN400? Three decades of empirical work (reviewed by Kutas & Federmeier, in press) have shown that the N400 is part of the normal electrophysiological response to not only words in all modalities, but also to pictures, faces, sounds, mathematical symbols, and so forthFessentially all meaningful stimuli. Moreover, the amplitude of the N400 is reduced by many variables that have in common that they ease semantic processing. These variables include associative and semantic priming and the fit between the N400-eliciting word and sentence/discourse context information. One notable feature of the N400 is that its latency is remarkably invariant, with a peak at approximately 375 ms in young adults. The distribution of the N400 is broad and, although typically characterized as having a centro-posterior maximum based on the original studies of sentential congruity, is actually variable across input types (as discussed below). It is also notable that the N400 has been found across many paradigms to manifest with high ‘‘automaticity,’’ meaning without the subject’s intent to process stimulus meaning. Likewise, the N400 is elicited even when subjects show no awareness for the eliciting stimulus (e.g., Vogel, Luck, & Shapiro, 1998), and N400 modulations occur with repetition even in patients with amnesia who show impaired explicit memory for repetition (e.g., Olichney et al., 2000). Thus, the N400 has been characterized as a neural manifestation of relatively automatic and implicit semantic processing of a variety of meaningful stimulus categories (Kutas & Federmeier, in press). Several developments were instrumental in the initial adoption of the N400 as a dependent measure by the memory research community. Repetition of the N400-eliciting stimulus was shown to strongly modulate the amplitude of the N400, with reduced amplitudes (i.e., more positive ERPs) to repeated stimuli (Neville, Kutas, Chesney, & Schmidt, 1986; Paller & Kutas, 1992; Rugg, 1990; Young & Rugg, 1992). In the context of a body of literature delineating the functional sensitivity of the N400 (e.g., Kutas & Van Petten, 1990), these repetition effects could be logically associated with the greater ease/fluency of semantic processing that occurs with repetition, known as conceptual priming when measured in an indirect priming test. At the time these discoveries were being made, the memory research community was substantially influenced by the theories of memory proposed by Craik, Lockhart, and Tulving (Craik & Lockhart, 1972; Craik & Tulving, 1975), which emphasized the central importance of semantic/conceptual processing for recognition. Thus, the N400 seemed promising as a marker of processing relevant to recognition memory. Besson, Kutas, and Van Petten (1992) and Friedman (1990) were among the first to describe N400 effects during recognition tasks. The distribution of the N400 repetition effects observed in the context of recognition memory has been characterized as being more frontal than that for N400 effects in language processing tasks. However, it is important to note that this distributional difference may have been more apparent than real. The distribution of the N400 is broad, and a number of factors have now been found to modulate it, including item type differences: for example, N400 responses are more centro-posterior for abstract words but more frontal for concrete words and pictures (e.g., Ganis, Kutas, & Sereno, 1996; Kounios & Holcomb, 1994). Thus, the distribution of the FN400 typically described in memory tasks is, in fact, compatible with the distribution of effects observed on the N400 in language processing studies, especially because recognition memory paradigms often use relatively concrete single words or pictures. Furthermore, other ERP
533 effects that overlap in time with N400, such as the late posterior complex (LPC), are elicited during memory tasks, but more rarely occur during language tasks. These contemporaneous effects might also skew the observed distribution of the N400 in studies of memory relative to studies of language.1 Nevertheless, the suggestion of a distributional difference, combined with differences in the types of processes researchers in the memory and language domains set out to characterize with their tasks, led to a split between the literatures, with the FN400 and N400 coming to be treated as functionally distinct entities in studies of memory versus studies of language. In particular, Smith (1993) provided early evidence for an association between the FN400 and familiarity-based recognition memory, as distinct from recollection-based recognition. Familiarity and recollection are hypothesized episodic memory processes that differ in the quality of the phenomenological experience associated with recognition and/or with the type of information that is retrieved (Mandler, 1980; Yonelinas, 2002). Recollection entails the retrieval of contextual detail from the prior occurrence, including retrieval of spatiotemporal context or other episodic details, and is associated with the awareness of retrieval or ‘‘mental time travel’’ to the prior event. In contrast, familiarity entails unsubstantiated feelings of familiarity coupled with a lack of retrieval of specific detail from the prior occurrence, as can occur when only item-specific information is retrieved during recognition. Smith showed that the magnitude of the FN400 repetition effect was similar for memory judgments that entailed recollection or only a feeling of familiarity (i.e., FN400 effects were observed during memory tasks but were not uniquely associated with recollection). Later, in a pivotal study, Curran (2010) investigated FN400 effects using a plurality reversal manipulation that reduced recollection but had little effect on familiarity. This manipulation had no effect on FN400 repetition effects or on familiarity, but did influence recollection as well as late-parietal ERPs (LPC). This was also taken as support for the association between the FN400 and familiarity and between the LPC and recollection. Many studies followed using similar techniques to associate FN400 potentials with familiarity and lateronset LPC-family potentials with recollection (Rugg & Curran, 2007). However, several findings present striking challenges to the case for linking the FN400 specifically with familiarity and, correspondingly, for dissociating the FN400 from the N400. When experimental circumstances differentially influence familiarity and semantic processing, FN400 potentials often show properties more characteristic of N400 effects and semantic processing rather than of familiarity. For instance, strong familiarity-based recognition of stimuliFbut an absence of corresponding FN400 effectsFhas been found for a range of stimuli without preexisting conceptual representations and therefore little ability to support conceptual/semantic processing. These stimuli include novel faces (MacKenzie & Donaldson, 2007; Yovel & Paller, 2004, but see Donaldson & Curran, 2007), geometrical patterns (Voss & Paller, 2009b), and novel symbols (De Chastelaine, Friedman,
1 It is important to note, as discussed in more detail below, that the strong hypothesis linking FN400 to familiarity (e.g., Rugg & Curran, 2007) does not propose that FN400 effects are just slight anterior shifts in the observed distribution of the N400, but rather that these are distinct effects that would not be expected to manifest with mostly overlapping distributions.
534 Cycowicz, & Horton, 2009).2 Furthermore, at very short delays, when familiarity should be maximal, repetition of stimuli with semantic value produces FN400 effects whereas repetition of stimuli without semantic value does not (Danker et al., 2008). These results seem to point to a stronger link between FN400 elicitation and meaning-based processing (as would be expected for an N400 modulation) than between the FN400 and familiarity. Indeed, Paller and colleagues (2007) argued that there is a fundamental and pervasive error in the reasoning that equates FN400 with familiarity. They found that manipulations purportedly linking FN400 to familiarity in over 25 published studies in fact showed instead that the FN400 is not related to recollectionFnot that it is related to familiarity. For example, the Curran (2000) results showed that FN400 was produced by repetition and was unaffected by a manipulation (plurality reversal) that reduced recollection. This indicates that the FN400 indexes some repetition-related process that is not recollectionFbut does not show that this process is, in fact, familiarity. Paller et al. suggested that most evidence linking the FN400 to familiarity (including that cited by Rugg & Curran, 2007) has suffered from the same logical flaw and is, therefore, equally amenable to the interpretation that FN400 effects instead reflect conceptual/semantic primingFand, thus, in fact, the same kind of processing as has been associated with the N400. Some recent research has endeavored to determine if the FN400 is better characterized as an N400, elicited by repeating meaningful stimuli during recognition tests, rather than as a specific index of familiarity. These studies measured both conceptual priming and familiarity in highly similar circumstances in order to identify ERPs that vary with one versus the other. For instance, Voss and Paller (2006) found that priming the conceptual information associated with celebrity faces led to FN400 effects that covaried with the magnitude of conceptual priming. In contrast, familiarity for the same faces was unrelated to FN400 modulations and was instead associated with late-parietal LPC potentials. A follow-up study using functional magnetic resonance imaging (fMRI; Voss, Reber, Mesulam, Parrish, & Paller, 2008) provided further support for the notion that the FN400 reflected conceptual priming, given that the same conceptual priming manipulation found to modulate the FN400 was associated with left inferior prefrontal fMRI response reductions characteristic of the control processes associated with conceptual priming (Schacter, Wig, & Stevens, 2007). Other, related studies have dissociated ERP correlates of conceptual priming and familiarity by examining variations in both during priming and recognition tests. For instance, abstract geometric shapes vary in the extent to which individuals find them to be meaningful (as in the Rorschach test) and thus in the extent to which they can support conceptual priming with repetition. Repeating them also produces different amounts of familiarity, similar to repeating the kind of word stimuli that are 2 De Chastelaine et al. (2009) also found little evidence to associate FN400 with semantic/conceptual processing, but their analyses were limited, as they were based on ratings of semantic content collected after subjects finished the lengthy experiment, which, therefore were likely not well yoked to the level of conceptual processing engaged on a trial-by-trial basis during the experiment. Furthermore, the validity of the ad hoc semantic ratings was not assessed by including, for example, tests of conceptual priming analyzed according to the ratings. However, trial-by-trial estimates of recognition were straightforward and convincing and unequivocally showed no FN400 effects despite familiarity-based recognition.
J.L. Voss & K.D. Federmeier more typically used in recognition memory tasks. Voss and Paller (2007) found that repeating relatively meaningful abstract shapes produced conceptual priming, whereas repeating relatively meaningless shapes did not. During a recognition test, meaningful and meaningless shapes were both recognized with equal levels of familiarity, which was associated with LPC potentials for both meaningfulness levels. However, only meaningful shapes produced FN400 effects, indicating that it is the potential for conceptual priming, not for familiarity, that can be associated with the FN400. A follow-up study substantiated this finding by indicating that the magnitude of conceptual priming for meaningful shapes is strongly correlated with the size of FN400 effects (Voss, Schendan, & Paller, 2010). Furthermore, obscure words were also found to vary in their ability to produce conceptual priming during a recognition test, and these variations in conceptual priming were linked to the FN400 (Voss, Lucas, & Paller, 2010). These studies collectively argue against the functional distinctions that have motivated the differentiation of the FN400 from the N400. Instead, they indicate that the semantic processing reflected in the N400 is also active during recognition memory tasks and can be modulated by repetition, meaning that N400 variations may be incorrectly attributed to familiarity memory if not exhaustively characterized (Voss & Paller, 2008a). Although this recent work has built a case for reintegrating the FN400 with the more general N400 literature, a stronger test of the notion that the FN400 is actually an N400 would be to conduct a direct, within-subjects comparison of the two effects by jointly manipulating factors associated with each. To that end, in the current study we manipulated short-term semantic priming within a recognition test. Preceding a target word with a semantically related prime (e.g., doctor–nurse) is an established as a means of facilitating (reducing) N400 amplitude (Kutas & Federmeier, 2010), presumably because prior access to shared/linked information eases semantic access for the target concept. We therefore repeated ‘‘target’’ words during a continuous recognition paradigm and preceded each target by either a semantically related prime or an unrelated word. Subjects explicitly made recognition memory decisions and reported on phenomenological feelings of familiarity, recollection, and confidence. Thus, ERP correlates of familiarity-based recognition in the absence of semantic priming could be identified by focusing on repetition of targets with unrelated preceding words (as in a standard recognition memory paradigm). Likewise, ERP correlates of semantic priming could be identified by focusing on targets with related primes, independent from repetition (as in a standard semantic priming paradigm). The hypothesis that FN400 potentials are distinct from N400s and are unique correlates of familiarity would predict an association between familiarity-based recognition and FN400 effects, separable from N400 effects associated with semantic priming. That is, an anterior FN400 effect should correlate with familiarity and a more posterior N400 effect should correlate with semantic priming. In contrast, on the hypothesis that FN400 effects and N400 effects share a common function and neural source, then we would expect to find nearly identical effects for familiarity-based recognition and for semantic priming. Furthermore, this hypothesis would predict the FN400/ N400 effect to old items to be sensitive to semantic priming, but not strongly yoked to familiarity-based recognition. We also predicted that recognition would be associated with late-parietal old/new effects on what is known as the late-positive complex (LPC). LPC effects have been associated with self-
Why the ‘‘F’’ in FN400? reported recollection, accurate source memory, and the amount of information retrieved during recollection and therefore are widely accepted as ERP correlates of processes that support recognition memory (Friedman & Johnson, 2000; Mecklinger, 2000; Paller et al., 2007; Rugg & Curran, 2007; Voss & Paller, 2008a,b). Some evidence indicates that LPC effects can be fractionated, with left-lateralized LPC effects signaling recollection and more widespread LPC effects signaling confidence (e.g., Vilberg & Rugg, 2009). We therefore predicted that LPC effects would be associated with recognition and that this relationship would not be influenced markedly by semantic priming. We also included direct tests of laterality to examine whether the LPC effects we observe are lateralized.
Methods Participants Behavioral and electrophysiological data are reported for 16 right-handed, native speakers of English (ages 18–27 years, 9 female) recruited from the University of Illinois Urbana–Champaign community. All subjects had normal or corrected-to-normal vision and were native speakers of English. The experiment was terminated early for an additional 3 subjects because of failure to control eye blinking, and these data were discarded. All procedures were approved by the Institutional Review Board of the University of Illinois Urbana–Champaign. Subjects were paid $30 for their participation. Stimuli Verbal stimuli included 600 common English words (3–8 letters in length) that were divided into 200 triplets. Each triplet comprised a semantically related word pair (e.g., doctor–nurse; forward and backward free association strengths range 5 0.08–0.25 according to University of South Florida Free Association Norms; Nelson, McEvoy, & Schreiber, 1998), and the third word was unrelated to the pair (e.g., kettle, not listed as an associate in the University of South Florida Free Association Norms). Unrelated words were matched, on average, for length and written frequency (Kucera & Francis, 1967) to related words. Words in 70% of the triplets described common objects, and normalized concreteness ratings for all words was greater than 400 (on a scale of 100 to 700). An additional 100 common English words served as filler stimuli and were chosen to be as unrelated as possible to all words in the 200 triplets. Words were presented at the center of a computer monitor and subtended average approximate visual angles of 0.51 ! 5.01. A central fixation cross was present during interstimulus intervals (ISI). Behavioral Procedures Subjects performed a continuous recognition test for words. Targets were one member of each of the 200 semantically related word pairs. Each target was shown twice, with the second presentation at a delay of 15–25 intervening targets from the first presentation (mean 5 20, delay range 5 83–138 s). Each target trial was preceded by either its semantically related prime (e.g., doctor–nurse) or by the corresponding semantically unrelated word (e.g., kettle–nurse). Half of the targets were preceded by the related prime on first presentation and then by the unrelated word on second presentation and vice versa for the other half of the targets (Figure 1). The other 100 filler words were inter-
535 spersed in order to maintain the average delay between consecutive presentations of a target. Order of presentation of the pairs (i.e., Figure 1A,B vs. Figure 1C,D) was counterbalanced across subjects, as was the word from each semantically related pair that served as the target item. During each trial, one word was presented for 500 ms. Subjects were instructed to immediately make a judgment about the word’s emotional valence in order to encourage semantic processing. The subject pressed one button to indicate if the word was of positive valence and another button to indicate if the word was of neutral or negative valence. No words had actual strong affective valence. At a delay of 2500 ms from word onset, a recognition prompt appeared on the screen (the letter ‘‘R’’). This prompt signaled to the participant to make a memory judgment, whereby the participant attempted to indicate if the word was old or new. The memory judgment was made using a modified remember/know procedure (Gardiner & Java, 1991; Tulving, 1985b) that captured recollection, familiarity, and confidence. Subjects pressed one button to indicate ‘‘remember old,’’ another button to indicate ‘‘know old,’’ another button to indicate ‘‘guess old,’’ and another button to indicate ‘‘new.’’ Remember, know, and guess responses were only correct for the second presentation of a target. The correct response to all other stimuli was ‘‘new.’’ The next trial immediately followed, with a randomized 2500– 3500-ms ISI between the recognition prompt and the onset of the next word. The semantic judgment response and the recognition memory judgment response were made with different hands, alternated across subjects. To become familiarized with the experiment, subjects completed a practice session with 50 additional targets (25 preceded by related primes and 25 preceded by unrelated words), which were not used again during the main experiment. ERP Procedures Electroencephalogram (EEG) was sampled continuously from 64 scalp locations conforming to the extended International 10– 20 positioning system (Chatrian, Lettich, & Nelson, 1988) using a BioSemi Active II system (BioSemi Instrumentation, Amsterdam). Four additional channels were used to monitor horizontal and vertical eye movements and two additional channels recorded from left and right mastoids. The digitization rate was 1024 Hz. The recording bandpass was of 0.01 to 120 Hz, and no additional filtering was performed. Recordings were rereferenced off-line to averaged mastoids. ERPs time-locked to the onset of each word were calculated for each condition of interest in 900ms epochs, beginning 100 ms prior to stimulus onset. Baseline correction was performed using the mean amplitude of the prestimulus interval. Epochs contaminated by artifacts were discarded. ERPs were averaged over latency intervals and electrode clusters for statistical assessment, determined based on a priori hypotheses regarding FN400 and late-posterior potentials. Latency intervals included 300–500 ms and 600–800 ms. Four electrode clusters included mid-frontal (F1, Fz, F2, FC1, FCz, FC2, C1, Cz, C2), mid-posterior (CP1, CPz, CP2, P1, Pz, P2, POz), left-posterior (TP7, CP5, CP3, P7, P5, P3, PO7, PO3), and right-posterior (TP6, CP6, CP4, P8, P6, P4, PO8, PO4). ERP values averaged over clusters were used for statistical assessment. Repeated-measures analyses of variance (ANOVAs) incorporated Greenhouse–Geisser correction when appropriate. Only trials with correct recognition memory responses were included in ERP averages (i.e., remember, know, and guess responses for old targets and new responses for new targets and all primes).
536
J.L. Voss & K.D. Federmeier
Figure 1. Schematic diagram of the experiment design. Half of the targets in a continuous recognition paradigm were presented for the first time following semantically unrelated words (A) and were presented again after a delay following semantically related primes (B). The other half of the targets were presented first following semantically related primes (C) and then again after a delay following unrelated words (D). The coloration of these conditions is used for reference in subsequent figures.
The mean number of trials per condition, after excluding trials contaminated by artifact and trials with incorrect behavioral responses, was 64 (range 61–69). Results Behavioral Findings Hit rates for the recognition responses and response times (RTs) for the valence judgments are summarized in Table 1. It is clear that RTs for the valence judgments did not vary reliably for any two conditions, and no pairwise comparisons were significant. Therefore, the inclusion of valence judgments to each word for the intent of encouraging semantic analysis of words did not introduce RT differences between conditions that could potentially confound ERP comparisons. It is also clear from Table 1 that subjects succeeded at correctly rejecting related primes and unrelated words that preceded targets. False alarm rates for both conditions were uniformly low for remember, know, and guess responses, and correct rejection rates were uniformly high. There was a tendency for slightly more false alarm responses to primes that preceded the second presentations of targets (Figure 1B) compared to unrelated words that preceded the second presentation of targets (Figure 1D), Condition ! Response Type interaction, F(3,45) 5 19.5, po.001. This tendency was most pronounced for remember responses, t(15) 5 4.2, po.001, but was also evident for know responses, t(15) 5 2.3, p 5 .04, and guess responses, t(15) 5 2.2, p 5 .05. Because these primes were semantically related to the target and the target had been viewed previously (Figure 1A,B), this indicates that subjects occasionally mistook primes to second target presentations as the targets themselves and therefore endorsed them as old. Given the very small number of trials comprising this bias, it is reasonable to conclude that subjects rarely made this error.3 Subjects successfully discriminated repeat targets (old) from first-seen targets (new) both for targets preceded by related primes and for targets preceded by unrelated words: Repetition (old/new) ! Recognition Response Type (remember/know/ guess/new) interactions, F(3,45) 5 109.7, po.001, and 3 ERP correlates of false alarm trials included positive shifts in frontal N400 waveforms similar to those reported below for other experimental conditions, but we do not report these ERP effects in detail given that it is unclear whether they were related to semantic priming or to familiarity. That is, because the semantically related target appeared previously, ERPs could conceivable reflect either semantic priming due to the target or episodic familiarity for the target. The analyses described below were suitable for separately assessing these possible influences on ERPs.
F(3,45) 5 109.9, po.001, respectively. For targets preceded by related primes, hits to old items significantly outnumbered false alarms to new items for remember responses, t(15) 5 8.3, po.001, and know responses, t(15) 5 4.4, po.001. Likewise, correct rejections for new items significantly outnumbered misses for old items, t(15) 5 17.8, po.001. Guess responses did not significantly discriminate between old and new items, t(15) 5 1.0, p 5 .34. A similar pattern was identified for targets preceded by unrelated words: remember hits versus false alarms, t(15) 5 8.4, po.001; know hits versus false alarms, t(15) 5 4.5, po.001; correct rejections versus misses t(15) 5 16.8, po.001; guess hits versus false alarms t(15) 5 1.9, p 5 .07. Collapsing across remember, know, and guess response types, the mean discrimination sensitivity (d’) score was 2.5 for targets following related primes, t(15) 5 12.3, po.001 versus chance, and was 2.7 for targets following unrelated words, t(15) 5 11.5, po.001 versus chance. An important finding shown in Table 1 is that the semantic priming manipulation had no discernable influence on measures of familiarity-based recognition, Priming Status ! Repetition ! Recognition Response Type interaction, F(3,45) 5 1.5, p 5 .23. That is, for targets following related primes versus targets following unrelated words, response rates were virtually identical for remember responses, t(15) 5 0.2, p 5 .86, know responses, t(15) 5 0.7, p 5 .49, guess responses, t(15) 5 0.8, p 5 .43, and new responses, t(15) 5 0.5, p 5 .62. Some findings indicate that the remember/know procedure measures gradations in memory strength rather than separate recollection and familiarity processes (Rotello & Zeng, 2008; Wixted, 2007, 2009).4 Nonetheless, treating remember, know, guess, and new responses as a confidence scale (high to low) yields the same conclusions regarding the lack of influence of semantic priming on familiarity. The virtually matched prevalence of each response type would indicate highly matched distributions of memory strength 4 Indeed, correlations across subjects between remember response rates to old and to new items showed that subjects who made more remember responses to old items also made more remember responses to new items, r(14) 5 .49, p 5 .03, for targets following related primes and r(14) 5 .46, p 5 .04 for targets following unrelated words. These correlations are consistent with the hypothesis that subjects varied in terms of the threshold that was adopted for reporting recollection rather than the hypothesis that remember responses indicate a qualitatively unique state of phenomenological awareness of memory (Wixted & Stretch, 2004). That is, remember responses would be registered for old items if they truly reflected veridical recollection of specific details, and the correlation instead suggests that some subjects were more likely to make remember responses than other subjects, even to new items for which recollection should not be possible.
Why the ‘‘F’’ in FN400?
537
Table 1. Summary of Behavioral Data Recognition response type (hit rate) Stimulus type Targets First presentation (new) Related prime Unrelated prime Second presentation (old) Related prime Unrelated prime Primes First target presentation Related target Unrelated target Second target presentation Related target Unrelated target
Valence judgment RT (ms)
Remember old
Know old
Guess old
New
692 (55) 697 (76)
.03 (.01) .03 (.01)
.03 (.01) .03 (.01)
.07 (.02) .06 (.02)
.87 (.03) .88 (.03)
691 (56) 702 (58)
.57 (.07) .57 (.07)
.21 (.04) .23 (.04)
.09 (.02) .08 (.03)
.13 (.02) .12 (.03)
703 (57) 675 (55)
.02 (.01) .02 (.01)
.02 (.01) .02 (.01)
.06 (.02) .05 (.02)
.90 (.03) .91 (.03)
682 (57) 660 (58)
.05 (.01) .02 (.01)
.05 (.01) .03 (.01)
.09 (.02) .08 (.03)
.81 (.04) .87 (.03)
Note: Provided for each stimulus type is the mean valence judgment response time (RT) and the mean hit rate for each recognition response type. Targets are classified according to old/new status and according to whether the target was preceded be a semantically related or unrelated prime (Figure 1). Primes are classified according to the old/new status of the immediately following target and by the semantic relatedness to the target. No primes were repeated (all were ‘‘new’’). Parentheses indicate SEM.
for targets following related primes versus targets following unrelated words, and we are aware of no models of familiarity memory that would predict different levels of familiarity despite highly matched memory strength distributions (Wixted, 2007; Yonelinas, 2002). Semantic Priming ERP Effects for New Targets The first set of ERP analyses concerned differences between targets and the immediately preceding words as a function of whether the preceding words were related primes versus unrelated words. To identify effects of semantic priming without repetition (Figure 1C), ERPs are shown for two conditions that vary in the degree to which they produce semantic priming but are matched in repetition: (1) the first presentation of targets (new) and preceding unrelated words (Figure 2A) and (2) the first presentation of targets and preceding related primes (Figure 2B). Figure 2A shows that there were virtually no ERP differences between targets and their unrelated preceding words for any electrodes or for any latencies. In contrast, Figure 2B shows robust semantic priming effects on the N400 for targets preceded by related primes. Target ERPs were more positive than prime ERPs between 300 and 500 ms after stimulus onset, especially over central electrode sites. Statistical assessment of the 300–500-ms latency interval confirmed that target ERPs differed reliably from related prime ERPs for the mid-frontal, mid-posterior, left-posterior, and right-posterior electrode clusters, condition main effect, F(1,15) 5 12.8, p 5 .003. A marginal Condition ! Cluster interaction, F(1.8,27.6) 5 3.5, p 5 .05, substantiated the impression that the prime/target ERP differences were slightly more pronounced at mid-frontal and mid-posterior electrode clusters (mean 5 1.8 and 1.6 mV, respectively) versus left-posterior and right-posterior electrode clusters (mean 5 0.9 and 1.2 mV, respectively). Indeed, the 300–500-ms related prime/target difference was significantly more positive for the mid-frontal and mid-posterior electrode clusters relative to the left-posterior and right-posterior clusters (pairwise p values o.05) but did not differ for the mid-frontal and mid-posterior clusters (p 5 .22). Figure 1C shows that ERPs for targets following unrelated words, for the unrelated words preceding these targets, and for
primes are all well aligned, indicating that the relative positivity associated with semantic priming was selective for targets that were preceded by related primes. ERP effects occurring after 500 ms are described below in the last section of the Results. Semantic Priming and Recognition ERP Effects for Repeated Targets Figure 3 shows ERPs for the second presentations of targets (old) and also for the words immediately preceding the target as a function of whether the preceding words were related primes versus unrelated words (as in Figure 1B,D). Comparing target ERPs to preceding unrelated word ERPs identified reductions in negativity between 300 and 500 ms (FN400) and later posterior effects (Figure 3A,C,D), both of which are commonly associated with recognition memory. For the 300–500-ms latency interval, target ERPs were more positive than ERPs to unrelated preceding words, with significant variation across electrode clusters, condition main effect, F(1,15) 5 15.2, p 5 .001; Condition ! Cluster interaction, F(1.7,24.9) 5 9.4, p 5 .002. This 300–500-ms positive difference was significantly greater for the mid-frontal and mid-posterior electrode clusters relative to the left-posterior and right-posterior clusters (pairwise p values o.03) but did not differ for the mid-frontal and mid-posterior clusters (p 5 .46). For the 600–800-ms latency interval, targets were more positive than unrelated preceding words, with an apparent left-posterior topography (Figure 3D). However, a main effect of condition, F(1,15) 5 15.0, p 5 .002, and nonsignificant Condition ! Cluster interaction, F(3,45) 5 0.47, p 5 .71, indicated that the magnitude of the positive effect did not differ across the electrode clusters. Based on a priori hypotheses that the lateposterior effect would be left-lateralized, we directly compared the left-posterior and right-posterior clusters and found a trend towards left-lateralization of the positive difference between targets and unrelated preceding words, t(15) 5 2.2, p 5 .05. Very similar target/prime ERP effects were identified for old targets following related primes, but the 300–500-ms effects were strikingly more pronounced than for old targets following unrelated words (Figure 2B,C,D). For the 300–500-ms interval, a main effect of condition, F(1,15) 5 40.9, po.001, and Condition ! Cluster interaction, F(1.5,23.1) 5 10.4, p 5 .001, were
538
J.L. Voss & K.D. Federmeier
Figure 2. ERPs for the first presentation of targets. ERPs are shown for targets presented for the first time as well as for the words immediately preceding these targets. (A) ERPs for targets preceded by unrelated words and for the unrelated words. (B) ERPs for targets preceded by primes and for the primes. (C) Overlay of all four conditions at electrode Cz. Primed targets (P. Targets) are those that followed primes. Unprimed targets (U. Targets) are those that followed unrelated words. ERPs were matched except for primed targets, which showed positive N400 shifts. (D) The topography of ERP difference between primed targets and their primes is shown for the N400 interval, with difference amplitude values indicated by coloration.
due to reliably more positive prime/target differences for the midfrontal and mid-posterior electrode clusters relative to the leftposterior and right-posterior clusters (all pairwise p valueso.02), with no significant difference between the mid-frontal and midposterior clusters (p 5 .70). For the 600–800-ms interval, a main effect of condition, F(1,15) 5 8.1, p 5 .01, and nonsignificant Condition ! Cluster interaction, F(3,45) 5 1.6, p 5 .20, indicated that target ERPs were more positive than prime ERPs but did not differ across electrode clusters. Again, direct comparison between the left-posterior and right-posterior clusters indicated significant left laterality, t(15) 5 2.7, p 5 .02. The primary difference between prime/target ERP effects and ERP effects for targets following unrelated words thus appeared to be markedly greater positivity from 300 to 500 ms for prime/ target pairs (Figure 2C,D). We therefore directly tested the magnitude of the ERP differences between targets minus primes and targets minus preceding unrelated words. Indeed, prime/target
ERP differences were significantly more positive than differences between targets and unrelated preceding words from 300 to 500 ms, condition main effect, F(1,15) 5 5.9, p 5 .03; nonsignificant Condition ! Cluster interaction, F(3,45) 5 0.2, p 5 .87, but such differences were not reliable for the 600–800-ms interval, nonsignificant condition main effect, F(1,15) 5 0.3, p 5 .61; nonsignificant Condition ! Cluster interaction, F(3,45) 5 0.9, p 5 .45. To summarize, ERP effects for old targets compared to the immediately preceding word included 300–500-ms mid-frontal/ mid-posterior positivities and 600–800-ms left-posterior positivities. These effects were identified both for targets following unrelated words and for targets following related primes. Semantic priming modulated the magnitude of the 300–500-ms effect, which was greater for prime/target pairs than for target/ unrelated word pairs. In contrast, left-lateralized effects from 600 to 800 ms did not differ significantly for targets following primes versus targets following unrelated words. Semantic priming also
Why the ‘‘F’’ in FN400?
539
Figure 3. ERPs for the second presentation of targets. ERPs are shown for targets presented for the second time as well as for the words immediately preceding these targets. (A) ERPs for targets preceded by unrelated words and for the unrelated words. (B) ERPs for targets preceded by primes and for the primes. (C) Topographic maps are shown for the N400 interval for the ERP difference between targets and their preceding words, as a function of whether the preceding words were unrelated (no priming) or whether they were related primes (priming). (D) Topographic maps are provided using the same format for the later LPC interval.
did not modulate behavioral indices of familiarity memory (no effects on recognition accuracy or remember/know/guess response rates). The aforementioned analyses are thus consistent with an association between 300–500-ms effects at mid-frontal and mid-posterior sites and semantic priming and between 600– 800-ms left-lateralized posterior effects and recognition. Indeed, a very similar 300–500-ms semantic priming effect was also identified for the first presentation of a target following a related prime (Figure 2), as discussed later. ERP Old/New Effects In the previous analyses, the distribution of the 600–800-ms ERP effects for second presentations of targets appeared to be more broadly distributed for the comparison involving related primes than that with unrelated words (Figure 3D). One possible cause
for this difference is that ERPs might have differed for words immediately preceding the targets, which served as the comparison condition for the ERP effects. Indeed, the behavioral results indicated that subjects showed significantly more false alarms to the primes preceding targets presented for the second time (Figure 1B) than to unrelated words preceding targets presented for the second time (Figure 1D). In this section, therefore, we assessed old/new ERP effects, partly in order to provide a different baseline against which to compute repetition ERP effects immune to these small effects on prime ERPs. Figure 4 shows ERPs to old and new targets split by whether the targets were preceded by related primes or unrelated words. That is, targets were compared for Figure 1A (unrelated new) versus Figure 1D (unrelated old) and for Figure 1c (related new) versus Figure 1B (related old). The ERP old/new difference
540
J.L. Voss & K.D. Federmeier
Figure 4. ERP old/new effects for targets with and without semantic priming. ERPs are shown for new targets correctly rejected as new (new CR) and old targets correctly identified as old (old hit). (A) ERPs for old targets and new targets without semantic priming on either presentation, as targets were preceded by unrelated words on both occasions. (B) ERPs for old targets and new targets with semantic priming on both presentations, as targets were preceded by related primes on both occasions. (C) Topographic maps of the old/new ERP differences are shown for the N400 interval for the targets preceded by unrelated words on both occasions (no priming) and for targets preceded by related primes on both occasions (priming). (D) Topographic maps are provided using the same format for the later LPC interval.
(Figure 4) is highly similar to the comparison between targets and preceding words (Figure 3). One exception is that the distribution of the 600–800-ms old/new effects were clearly left-lateralized both for targets following related primes and for targets following unrelated words (Figure 4D), unlike 600–800-ms effects in Figure 3, which may have shown slight ERP differences due to choice of baseline conditions, as discussed above. Statistical assessment of the 300–500-ms latency interval indicated that old ERPs were significantly more positive than new ERPs for targets following unrelated words, condition main effect, F(1,15) 5 5.0, p 5 .04; nonsignificant Condition ! Cluster interaction, F(3,45) 5 1.7, p 5 .17, and for targets following related primes, condition main effect, F(1,15) 5 37.5, po.001; marginal Condition ! Cluster interaction, F(1.9,28.4) 5 3.2,
p 5 .06. For targets following related primes, old/new effects were marginally more positive for mid-frontal and mid-posterior clusters relative to left-posterior and right-posterior clusters (pairwise p values " .05), but did not differ for mid-frontal and mid-posterior clusters (p 5 .13). For the 600–800-ms latency interval, old ERPs were significantly more positive than new ERPs for targets following unrelated words, condition main effect, F(1,15) 5 6.7, p 5 .02; nonsignificant Condition ! Cluster interaction, F(3,45) 5 2.2, p 5 .10, and for targets following related primes, condition main effect, F(1,15) 5 5.6, p 5 .03; marginal Condition ! Cluster interaction, F(2.1,32.2) 5 3.0, p 5 .06. Assessments of posterior lateralization showed significantly greater ERP old/new differences for left-posterior versus right-posterior electrode clusters
Why the ‘‘F’’ in FN400?
541
both for targets following unrelated words, t(15) 5 2.8, p 5 .01, and for targets following related primes, t(15) 5 3.9, p 5 .001. The aforementioned analyses converged in indicating that positive shifts in ERP potentials from 600 to 800 ms with a leftlateralized posterior distribution were associated with recognition memory. These potentials were sensitive to repetition, in that they were greater for old targets relative to the immediately preceding words (Figure 3A,B,D) and for old targets relative to new targets (Figure 4A,B,D). Furthermore, recognition memory strength and accuracy were matched for targets following related primes versus targets following unrelated words, and the magnitude of the 600–800-ms ERP effect likewise was matched for these conditions. Left-lateralized 600–800-ms ERP effects and recognition memory were therefore both related to repetition and both insensitive to semantic priming. There were distinct effects on potentials from 300 to 500 ms showing a mid-frontal/central distribution, which appeared to be more anterior than 600–800ms effects. Moreover, these 300–500-ms effects were related to repetition in that they were evident for old targets following unrelated words relative both to the unrelated words (Figure 3A,C) and to the first presentation of the targets (i.e., old/new effects; Figure 4A,C). However, it appeared that the same 300–500-ms effect was strongly related to semantic priming in that it was much greater for targets following related primes both when targets were old (Figure 3B,C and Figure 4B,C) and when they were new (Figure 2B,C,D). ERP Topographic Comparisons We next report a series of analyses to determine how topographically distinct 300–500-ms effects were from 600–800-ms effects and if there were any topographical differences across the 300– 500-ms effects for any of the various conditions. Topographical/ distributional comparisons were made using the range normalization approach (McCarthy & Wood, 1985) to remove overall amplitude differences between the conditions being compared. In the first set of analyses, all electrodes were included. Overall distributional differences between two conditions would be indicated by a significant Condition ! Electrode interaction (although such differences do not necessarily implicate distinct neuroelectric generators; see Urbach & Kutas, 2002; Wilding, 2006). Topographical plots that correspond to these distributional analyses (that is, scaled individually according to minimum and maximum amplitude values such that distributions can be compared visually across plots) are provided in Figure 5. The first set of comparisons sought to determine if 300–500ms effects were topographically distinct from 600–800-ms effects for each condition separately. The analyses thus compared each plot in the left column of Figure 5 to the corresponding plot in the right column of Figure 5. Significant Condition ! Electrode interactions were obtained for every comparison, respectively for the five conditions shown in Figure 5A–E: F(4.7,70.2) 5 3.3, p 5 .01; F(3.8,56.9) 5 4.0, p 5 .007; F(4.0,60.1) 5 3.4, p 5 .01; F(3.9,58.5) 5 4.1, p 5 .005; F(4.3,64.8) 5 3.0, p 5 .02, thus substantiating the impression that 300–500-ms effects were more anterior and central than later effects. This distinction between 300–500-ms and later effects is consistent with the criteria generally used to distinguish so-called FN400 effects from late-posterior effects. We next sought to determine if the 300–500-ms semantic priming effect obtained by preceding a target with a related prime differed for the first time a target was viewed versus the second time a target was viewed. Comparing the related prime/target
Figure 5. Range-scaled topographic maps. Topographic maps are provided for the difference ERPs for the given conditions. Map coloration is based on the range of values for each condition and therefore corresponds to the topographic analyses described in Results. Primed targets are those preceded by semantically related primes. Unprimed targets are those preceded by unrelated words. Note that the same ERP topographic maps are shown without range scaling in Figures 2–4.
effect for the first presentation of a target (Figure 5A) to the related target/prime effect for the second presentation of the target (Figure 5B) yielded a nonsignificant interaction, F(4.0,59.7) 5 1.6, p 5 .20. Likewise, comparing the prime/target effect for the first presentation of a target (Figure 5A) to the old/ new effect for the second presentation of targets following related primes (Figure 5D) yielded a nonsignificant interaction, F(5.1,76.2) 5 1.3, p 5 .26. These results indicate that target
542 repetition did not significantly alter the influence of semantic priming on ERP effects from 300 to 500 ms. The next set of comparisons sought to determine if 300–500ms repetition effects differed topographically for targets following related primes compared to targets following unrelated words. The first comparison was made for the prime/target effect (Figure 5B) versus the effect for targets minus unrelated preceding words (Figure 5C) and yielded a nonsignificant interaction, F(4.1,62.1) 5 1.9, p 5 .12. The second comparison was made for the old/new effect for targets following related primes (Figure 5D) versus targets following unrelated words (Figure 5E) and yielded a nonsignificant interaction, F(1.0,15.0) 5 2.1, p 5 .16. To summarize, 300–500-ms ERP effects were always significantly more anterior than 600–800-ms ERP effects. However, there was no evidence to indicate that ERP correlates semantic priming differed spatially as a function of repetition (Figure 5A vs. 5B,D), nor was there evidence to indicate that ERP correlates of repetition differed as a function of semantic priming (Figure 5B vs. 5C and 5D vs. 5E). Semantic Priming ERPs versus Recognition ERPs The primary goal of the current study was to directly compare ERP correlates of semantic priming to ERP correlates of familiarity-based recognition in the same subjects and using the same experimental parameters. The final set of analyses therefore concerned (1) prime/target ERP effects for the first presentation of a prime/target pair (Figure 1C) and (2) old/new ERP effects for new targets preceded by unrelated words (Figure 1A) compared to old targets preceded by unrelated words (Figure 1D). Thus, the analysis of semantic priming ERPs was entirely independent of repetition and therefore independent of familiarity-based recognition, as in standard semantic priming paradigms. Likewise, the analysis of ERP old/new effects was entirely independent of semantic priming, as in standard recognition memory paradigms. The ERP effects for semantic priming without recognition are shown in Figure 2B,C,D and in Figure 5A, and the ERP effects for recognition without semantic priming are shown in Figure 4A,C,D and in Figure 5E. Note that topographic comparisons made in the previous section provided evidence that 300–500-ms ERP effects for both conditions were more anterior than corresponding ERP effects from 600 to 800 ms. We first sought to compare the topographies of the two effects for the 300–500-ms interval using the range normalization method described above for all electrodes (Figure 5A vs. 5E). A nonsignificant Condition ! Cluster interaction, F(63,945) 5 0.8, p 5 .83, indicated no reliable topographic difference. We then conducted a more specific test of any potential anterior ‘‘FN400’’ topographic differences for a restricted set of frontal electrodes. Only frontal electrodes near the midline were included (Fp1, Fpz, Fp2, AF3, AFz, AF4, F1, Fz, F2, FC3, FC1, FCz, FC2, and FC4), such that the anterior/posterior extent of the frontal topographies could be compared across conditions with high sensitivity. The Condition ! Cluster interaction for this more sensitive test of anterior distributional differences was also strikingly nonsignificant, F(15,225) 5 0.6, p 5 .90. Another sensitive test of any possible anterior/posterior differences between the distributions of the two effects was then conducted. For this analysis, we included nine clusters, each comprised of three electrodes, each of which included the three electrodes that straddled the midline, ranging from the most anterior (Fp1, Fpz, Fp2) to the most posterior (O1, Oz, O2; also Af3, Afz, and Af4; F1, Fz, and F2; FC1, FCz, and FC2; C1, Cz,
J.L. Voss & K.D. Federmeier and C2; CP1, CPz, and CP2; P1, Pz, and P2; and PO3, POz, and PO4). The Condition ! Cluster interaction on range-normalized values from these nine clusters was nonsignificant, F(8,120) 5 0.3, p 5 .96, indicating that there were no discernable anterior/posterior differences between the distributions of the two effects. We next sought to compare the topographies of the two effects from 600 to 800 ms, which appeared to be right-lateralized for semantic priming without repetition (Figure 5A) and left-lateralized for repetition without semantic priming (Figure 5E). A significant Condition ! Electrode interaction on range-normalized values substantiated this impression, F(4.4,66.2) 5 2.9, p 5 .03. As discussed above, recognition was associated with left-lateralized parietal effects from 600 to 800 ms for all conditions involving repetition (Figure 5B–E). Moreover, left-lateralized 600–800-ms effects were not modulated by semantic priming, just as behavioral indicators of recognition were not modulated by semantic priming. The current topographic dissociation further supports the association with recognition by indicating that these left-lateralized ERP correlates of recognition could be dissociated from effects of semantic priming on 600– 800-ms ERPs. These tests collectively provided no evidence consistent with a dissociation between N400 correlates of semantic priming and FN400 correlates of recognition, given that both modulated topographically indistinguishable ERPs from 300 to 500 ms. Furthermore, the significant topographic dissociation for the 600–800-ms interval as well as the topographic dissociations between the 300–500-ms interval and the 600–800-ms interval reported above indicate that the tests did not simply lack the sensitivity required to identify topographic differences across conditions.
Discussion Repetitions of target words in the absence of semantic priming (thus similar to standard recognition memory paradigms) were associated with ample familiarity-based recognition and elicited ERP old/new effects (Figure 4A,C,D and Figure 5E). In particular, in addition to left-lateralized late-parietal effects, these old/new effects included reductions in negativity between 300 and 500 ms with a more anterior distribution, consistent with extant descriptions of ‘‘FN400’’ old/new effects (Paller et al., 2007; Rugg & Curran, 2007). There were at least two possible causes of these FN400 effects: (1) familiarity-based recognition due to word repetition and (2) semantic/conceptual priming due to word repetition. The hypothesis that FN400 potentials index familiarity (Rugg & Curran, 2007) would predict correlations of this response primarily with familiarity (i.e., as opposed to semantic priming per se) and would also predict that semantic priming would be associated with distinct, posterior N400 repetition effects. By examining ERP prime/target effects for semantically related word pairs seen for the first time (Figure 2A,B,C and Figure 5A), we were able to isolate ERP correlates of semantic priming independent from recognition memory within the same experiment. Contrary to what would be predicted by the hypothesis that FN400 potentials are distinct from N400 potentials, semantic priming without recognition elicited ERP effects that were indistinguishable in time course or distribution from those evident for familiarity-based recognition without semantic prim-
Why the ‘‘F’’ in FN400? ing. In contrast, ERP effects from 600 to 800 ms were dissociated for these conditions, with left-lateralized ERP correlates of recognition memory distinct from less left-lateralized ERP correlates of semantic priming. To our knowledge, no previous study has directly compared ‘‘FN400’’ and N400 effects; instead, the hypothesized electrophysiological distinction between the two responses has been based on visual comparisons of ERP topographies obtained from different subjects participating in different experimental paradigms, often in different laboratories using different analysis strategies. Our results indicate that when these factors are controlled, there is no reason to conclude that ‘‘FN400’’ correlates of memory are distinct from N400 correlates of semantic priming. Other aspects of our results provide additional evidence for this conclusion. Preceding an old target with a semantically related prime had no discernable effects on recognition memory strength, accuracy, confidence, or self-reported familiarity relative to old targets preceded by unrelated words (and therefore not primed). However, old targets primed by related words were associated with strikingly larger modulations of anterior N400 (‘‘FN400’’) effects than were old targets preceded by unrelated words. It is difficult to reconcile the hypothesis that FN400 old/ new effects are a general index of familiarity when two conditions that are highly matched in familiarity nevertheless differ so markedly in FN400 amplitude. Instead, it was LPC effects that seemed to be more yoked to recognition than to semantic priming, consistent with the larger literature linking LPC amplitudes to recognition measures (Friedman & Johnson, 2000; Mecklinger, 2000; Paller et al., 2007; Rugg & Curran, 2007; Voss & Paller, 2008a,b). The magnitude of left-lateralized LPC repetition effects did not vary across any of the conditions we tested involving repetition (i.e., were completely insensitive to doctor–nurse semantic priming). Likewise, behavioral indicators of recognition memory and memory-strength distributions were matched for these conditions.5 Collectively, our data support the hypothesis that effects in recognition memory paradigms labeled as ‘‘FN400’’ are actually N400 effects and thus share both the electrophysiological characteristics (such as temporal stability) and functional sensitivity of the N400. As shown by the present findings and buttressed by the established literature on the N400 (see, e.g., Kutas & Federmeier, in press), this N400 response is sensitive to short-term conceptual priming arising from shared semantic features and/or association as well as to semantic facilitation that occurs when a word is repeated after a delay. This conclusion is consistent with findings that ERP correlates of recognition for semantically impoverished stimuli do not include N400-like effects (De Chastelaine et al., 2009; MacKenzie & Donaldson, 2007; Voss & Paller, 2009b; Yovel & Paller, 2004) even at very short delays (Danker et al., 2008) and that N400-like effects in recognition memory paradigms covary with the magnitude of conceptual/ semantic priming and with the ability for relatively low-meaning stimuli to support conceptual priming (Voss & Paller, 2006, 2007; Voss et al., 2009, 2010). More generally, our results thus suggest that semantic/conceptual priming operates during recognition memory testing whenever meaningful stimuli are repeated and 5 Of course, direct evidence for the link between recognition memory and LPC effects would require manipulations that affect both in a similar way, but establishing this relationship was not the goal of the current study.
543 N400-like (‘‘FN400’’) effects are observed (Paller et al., 2007; Voss & Paller, 2008a). Some recent findings appear to provide evidence counter to our conclusion that FN400 effects during memory testing are actually N400 correlates of semantic/conceptual priming rather than signals associated with familiarity-based recognition (Woodruff, Hayama, & Rugg, 2006; Yu & Rugg, 2010). In these studies using words and nameable pictures, self-reports of familiarity strength (confidence ratings for ‘‘know’’ judgments using the remember/know procedure) did show correlations with the magnitude of ‘‘FN400’’ effects, with higher familiarity strength associated with larger FN400 effects. Importantly, however, these studies did not attempt to dissociate familiarity confidence from variations in semantic facilitation, which we would argue are the actual basis of the observed correlation. Why might confidence decisions and semantic/conceptual priming be correlated for words, thus creating an apparent association between ‘‘FN400’’ effects and familiarity? Below we explicate one possible theoretical account that draws from the notion that decisions during a memory test are made on the basis of multiple, contemporaneous brain processing events. We start by pointing out that confidence ratings often provide valid measures of memory ‘‘strength’’ or accuracy (Heathcote, 2003; Yonelinas, 2001). However, confidence and memory strength/accuracy are not perfectly correlated and can be dissociated (Chandler, 1994; Dobbins, Kroll, & Liu, 1998; Heathcote, Freeman, Etherington, Tonkin, & Bora, 2009; Tulving, 1981; Voss, Baym, & Paller, 2008; Voss & Paller, 2009a; Wilimzig, Tsuchiya, Fahle, Einhauser, & Koch, 2008). Some evidence indicates that confidence ratings may be particularly sensitive to signals of perceptual and conceptual fluency (Jacoby & Whitehouse, 1989; Keane, Orlando, & Verfaellie, 2006; Verfaellie & Cermak, 1999; Whittlesea & Williams, 2000; Wolk et al., 2005). Indeed, all experiments referenced above as supporting dissociations between confidence and accuracy achieved these dissociations through manipulations of fluency. The influence of fluency on confidence does not necessarily need to derive from an object’s ‘‘oldness’’Fthat is, fluency need not be related to repetition during a memory test. For instance, Verfaellie and colleagues have shown that fluency can influence memory confidence, even for items that were never studied previously during the experiment (Verfaellie & Cermak, 1999). Thus, when episodic memory signals are weak, as is the case for items that garner ‘‘know’’ responses rather than ‘‘remember’’ responses, perceptual or conceptual fluency might exert stronger influences on confidence decisions than when memory is relatively strong. Indeed, some evidence indicates that fluency has little, if any, influence on memory decisions made with high confidence or strengthFthat is, fluency and confident recognition can be convincingly dissociated in many circumstances (e.g., Conroy, Hopkins, & Squire, 2005; Stark & Squire, 2000; Wagner, Gabrieli, & Verfaellie, 1997). Following this logic, the reason that confidence ratings for ‘‘know’’ items with relatively weak memory signals might covary with N400 signals of semantic/conceptual priming (mistaken as familiarity-based ‘‘FN400’’) is that these signals vary across items and influence confidence decisions accordingly. This kind of semantic/conceptual variation is plausible for many reasons. For instance, there is likely considerable variation across items in terms of how ‘‘deeply’’ words are semantically elaborated during encoding, and variations in semantic elaboration would be expected to correlate with the degree of later conceptual/semantic priming (Roediger, 1990). Furthermore, words vary in fluency
544
J.L. Voss & K.D. Federmeier
and conceptual priming based on factors such as the frequency with which they are encountered (Yap, Tse, & Balota, 2009), and how associated they are to other concepts stored in semantic memory (Griffiths, Steyvers, & Firl, 2007). Our proposal is simply that subjects use these signals when making confidence decisions, especially when memory is relatively weak (i.e., when ‘‘know’’ responses are registered; consistent with findings of Jacoby & Whitehouse, 1989; Keane et al., 2006; Verfaellie & Cermak, 1999; Wolk et al., 2005). Thus, just as fluency biases decisions in many circumstances (Alter & Oppenheimer, 2009), conceptual fluency biases recognition judgments, leading to covariation between confidence ratings and N400 correlates of semantic/conceptual priming. Our results can thus be taken as evidence consistent with multiple memory-systems theories that propose that memory behaviors are determined by the output from several, coactive memory systems, including a semantic system distinct from an episodic memory system (Henson & Gagnepain, 2010; Tulving, 1985a). Confidence ratings can be sensitive to the output from different systems in different circumstances, and manipulations such as those used in the present experiment are necessary to determine the dissociations and associations between these systems in producing a memory decision. In other words, relying on
confidence ratings alone will fail to accurately characterize neural substrates of memory, given that additional information is needed to determine which processes in which systems produced the ratings. In conclusion, then, we found evidence consistent with the notion that conceptual priming occurs during recognition memory tests and is the source of N400 old/new effects during these tests. These potentials have been (mis)labeled as ‘‘FN400’’ old/ new effects and assumed to have a specific, separable functional sensitivity and electrophysiological signature, despite having never been dissociated convincingly or empirically from N400 potentials. Rather than merely serving as a cautionary note in the search for ERP correlates of memory, we take our results as support for a multiple memory-systems account of recognition whereby a relatively implicit, automatic semantic system assesses all incoming information and thus produces semantic/conceptual fluency signals that can influence memory behaviors and decisions along with the output from other systems. This semantic analysis does not always provide accurate memory decisions, given that semantic fluency need not stem from true ‘‘oldness’’ or repetition, but we speculate that it provides an unobtrusive and reasonably accurate means of adapting to environmental stimuli based on recent experiences.
REFERENCES Alter, A. L., & Oppenheimer, D. M. (2009). Uniting the tribes of fluency to form a metacognitive nation. Personality and Social Psychology Review, 13, 219–235. Besson, M., Kutas, M., & Van Petten, C. (1992). An event-related potential (ERP) analysis of semantic congruity and repetition effects in sentences. Journal of Cognitive Neuroscience, 4, 132–149. Chandler, C. C. (1994). Studying related pictures can reduce accuracy, but increase confidence, in a modified recognition test. Memory & Cognition, 22, 273–280. Chatrian, G. E., Lettich, E., & Nelson, P. L. (1988). Modified nomenclature for the ‘‘10%’’ electrode system. Journal of Clinical Neurophysiology, 5, 183–186. Conroy, M. A., Hopkins, R. O., & Squire, L. R. (2005). On the contribution of perceptual fluency and priming to recognition memory. Cognitive, Affective & Behavioral Neuroscience, 5, 14–20. Craik, F., & Lockhart, R. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning & Verbal Behavior, 11, 671–684. Craik, F., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104, 268–294. Curran, T. (2000). Brain potentials of recollection and familiarity. Memory & Cognition, 28, 923–938. Czernochowski, D., Mecklinger, A., & Johansson, M. (2009). Age-related changes in the control of episodic retrieval: An ERP study of recognition memory in children and adults. Developmental Science, 12, 1026–1040. Danker, J. F., Hwang, G. M., Gauthier, L., Geller, A., Kahana, M. J., & Sekuler, R. (2008). Characterizing the ERP old-new effect in a shortterm memory task. Psychophysiology, 45, 784–793. De Chastelaine, M., Friedman, D., Cycowicz, Y. M., & Horton, C. (2009). Effects of multiple study-test repetition on the neural correlates of recognition memory: ERPs dissociate remembering and knowing. Psychophysiology, 46, 86–99. Dobbins, I. G., Kroll, N. E., & Liu, Q. (1998). Confidence-accuracy inversions in scene recognition: A remember-know analysis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1306–1315. Donaldson, D. I., & Curran, T. (2007). Potential (ERP) studies of recognition memory for faces. NeuroImage, 36, 488–489. Ecker, U. K., Arend, A. M., Bergstrom, K., & Zimmer, H. D. (2009). Verbal predicates foster conscious recollection but not familiarity of a
task-irrelevant perceptual featureFAn ERP study. Consciousness & Cognition, 18, 679–689. Eichenbaum, H., Yonelinas, A. P., & Ranganath, C. (2007). The medial temporal lobe and recognition memory. Annual Review of Neuroscience, 30, 123–152. Friedman, D. (1990). ERPs during continuous recognition memory for words. Biological Psychology, 30, 61–87. Friedman, D., & Johnson, R. Jr. (2000). Event-related potential (ERP) studies of memory encoding and retrieval: A selective review. Microscopy Research and Technique, 51, 6–28. Ganis, G., Kutas, M., & Sereno, M. (1996). The search for common sense: An electrophysiological investigation of the semantic analysis of words and pictures in sentences. Journal of Cognitive Neuroscience, 8, 89–106. Gardiner, J. M., & Java, R. I. (1991). Forgetting in recognition memory with and without recollective experience. Memory & Cognition, 19, 617–623. Griffiths, T. L., Steyvers, M., & Firl, A. (2007). Google and the mind: Predicting fluency with PageRank. Psychological Science, 18, 1069– 1076. Heathcote, A. (2003). Item recognition memory and the ROC. Journal of Experimental Psychology Learning, Memory, and Cognition, 29, 1210–1230. Heathcote, A., Freeman, E., Etherington, J., Tonkin, J., & Bora, B. (2009). A dissociation between similarity effects in episodic face recognition. Psychonomic Bulletin & Review, 16, 824–831. Henson, R. N., & Gagnepain, P. (2010). Predictive, interactive multiple memory systems. Hippocampus (in press). Jacoby, L. L., & Whitehouse, K. (1989). An illusion of memory: False recognition influenced by unconscious perception. Journal of Experimental Psychology: General, 118, 126–135. Keane, M. M., Orlando, F., & Verfaellie, M. (2006). Increasing the salience of fluency cues reduces the recognition memory impairment in amnesia. Neuropsychologia, 44, 834–839. Klonek, F., Tamm, S., Hofmann, M. J., & Jacobs, A. M. (2009). Does familiarity or conflict account for performance in the word-stem completion task? Evidence from behavioural and event-related-potential data. Psychology Research, 73, 871–882. Kounios, J., & Holcomb, P. J. (1994). Concreteness effects in semantic processing: ERP evidence supporting dual-coding theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 804–823.
Why the ‘‘F’’ in FN400? Kucera, H., & Francis, W. N. (1967). Computational analysis of presentday American English. Providence, RI: Brown University Press. Kutas, M., & Federmeier, K. (in press). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203–205. Kutas, M., & Van Petten, C. (1990). Electrophysiological perspectives on comprehending written language. Electroencephalography & Clinical Neurophysiology Supplement, 41, 155–167. MacKenzie, G., & Donaldson, D. I. (2007). Dissociating recollection from familiarity: Electrophysiological evidence that familiarity for faces is associated with a posterior old/new effect. NeuroImage, 36, 454–463. Mandler, G. (1980). Recognizing: The judgment of previous occurrence. Psychological Review, 87, 252–271. McCarthy, G., & Wood, C. C. (1985). Scalp distributions of event-related potentials: An ambiguity associated with analysis of variance models. Electroencephalography and Clinical Neurophysiology, 62, 203–208. Mecklinger, A. (2000). Interfacing mind and brain: A neurocognitive model of recognition memory. Psychophysiology, 37, 565–582. Mecklinger, A., Brunnemann, N., & Kipp, K. (2010). Two processes for recognition memory in children of early school age: An event-related potential study. Journal of Cognitive Neuroscience (in press). Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (1998). The University of South Florida word association, rhyme, and word fragment norms. Retrieved February 15, 2010, from http://www.usf.edu/ FreeAssociation/. Neville, H., Kutas, M., Chesney, G., & Schmidt, A. L. (1986). Eventrelated brain potentials during initial encoding and recognition memory of congruous and incongruous words. Journal of Memory & Language, 25, 75–92. Nyhus, E., & Curran, T. (2009). Semantic and perceptual effects on recognition memory: Evidence from ERP. Brain Research, 1283, 102–114. Olichney, J. M., Van Petten, C., Paller, K. A., Salmon, D. P., Iragui, V. J., & Kutas, M. (2000). Word repetition in amnesia. Electrophysiological measures of impaired and spared memory. Brain, 123, 1948–1963. Opitz, B., & Cornell, S. (2006). Contribution of familiarity and recollection to associative recognition memory: Insights from event-related potentials. Journal of Cognitive Neuroscience, 18, 1595–1605. Paller, K. A., & Kutas, M. (1992). Brain potentials during memory retrieval provide neurophysiological support for the distinction between conscious recollection and priming. Journal of Cognitive Neuroscience, 4, 375–391. Paller, K. A., Voss, J. L., & Boehm, S. G. (2007). Validating neural correlates of familiarity. Trends in Cognitive Sciences, 11, 243–250. Roediger, H. L. (1990). Implicit memory. Retention without remembering. American Psychologist, 45, 1043–1056. Rotello, C. M., & Zeng, M. (2008). Analysis of RT distributions in the remember-know paradigm. Psychonomic Bulletin & Review, 15, 825–832. Rugg, M. D. (1990). Event-related brain potentials dissociate repetition effects of high- and low-frequency words. Memory & Cognition, 18, 367–379. Rugg, M. D., & Curran, T. (2007). Event-related potentials and recognition memory. Trends in Cognitive Sciences, 11, 251–257. Schacter, D. L., Wig, G. S., & Stevens, W. D. (2007). Reductions in cortical activity during priming. Current Opinion in Neurobiology, 17, 171–176. Smith, M. E. (1993). Neurophysiological manifestations of recollective experience during recognition memory judgments. Journal of Cognitive Neuroscience, 5, 1–13. Speer, N. K., & Curran, T. (2007). ERP correlates of familiarity and recollection processes in visual associative recognition. Brain Research, 1174, 97–109. Stark, C. E., & Squire, L. R. (2000). Recognition memory and familiarity judgments in severe amnesia: No evidence for a contribution of repetition priming. Behavioral Neuroscience, 114, 459–467. Tulving, E. (1981). Similarity relations in recognition. Journal of Verbal Learning and Verbal Behavior, 20, 479–496. Tulving, E. (1985a). How many memory systems are there? American Psychologist, 40, 385–398.
545 Tulving, E. (1985b). Memory and consciousness. Canadian Journal of Psychology, 26, 1–12. Urbach, T. P., & Kutas, M. (2002). The intractability of scaling scalp distributions to infer neuroelectric sources. Psychophysiology, 39, 791–808. Verfaellie, M., & Cermak, L. S. (1999). Perceptual fluency as a cue for recognition judgments in amnesia. Neuropsychology, 13, 198–205. Vilberg, K. L., & Rugg, M. D. (2009). Functional significance of retrieval-related activity in lateral parietal cortex: Evidence from fMRI and ERPs. Human Brain Mapping, 30, 1490–1501. Vogel, E. K., Luck, S. J., & Shapiro, K. L. (1998). Electrophysiological evidence for a postperceptual locus of suppression during the attentional blink. Journal of Experimental Psychology: Human Perception and Performance, 24, 1656–1674. Voss, J. L., Baym, C. L., & Paller, K. A. (2008). Accurate forced-choice recognition without awareness of memory retrieval. Learning & Memory, 15, 454–459. Voss, J. L., Lucas, H. D., & Paller, K. A. (2010). Conceptual priming and familiarity: Different expressions of memory during recognition testing with distinct neurophysiological correlates. Journal of Cognitive Neuroscience, 22, 2638–2651. Voss, J. L., & Paller, K. A. (2006). Fluent conceptual processing and explicit memory for faces are electrophysiologically distinct. Journal of Neuroscience, 26, 926–933. Voss, J. L., & Paller, K. A. (2007). Neural correlates of conceptual implicit memory and their contamination of putative neural correlates of explicit memory. Learning & Memory, 14, 259–267. Voss, J. L., & Paller, K. A. (2008a). Brain substrates of implicit and explicit memory: The importance of concurrently acquired neural signals of both memory types. Neuropsychologia, 46, 3021–3029. Voss, J. L., & Paller, K. A. (2008b). Neural substrates of remembering: Electroencephalographic studies. In J. Byrne (Ed.), Learning and memory: A comprehensive reference (Vol. 3, pp. 79–97). Oxford: Elsevier. Voss, J. L., & Paller, K. A. (2009a). An electrophysiological signature of unconscious recognition memory. Nature Neuroscience, 12, 349–355. Voss, J. L., & Paller, K. A. (2009b). Remembering and knowing: Electrophysiological distinctions at encoding but not retrieval. NeuroImage, 46, 280–289. Voss, J. L., Reber, P. J., Mesulam, M. M., Parrish, T. B., & Paller, K. A. (2008). Familiarity and conceptual priming engage distinct cortical networks. Cerebral Cortex, 18, 1712–1719. Voss, J. L., Schendan, H. E., & Paller, K. A. (2010). Finding meaning in novel geometric shapes influences electrophysiological correlates of repetition and dissociates perceptual and conceptual priming. NeuroImage, 49, 2879–2889. Wagner, A. D., Gabrieli, J. D., & Verfaellie, M. (1997). Dissociations between familiarity processes in explicit recognition and implicit perceptual memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 305–323. Whittlesea, B. W., & Williams, L. D. (2000). The source of feelings of familiarity: The discrepancy-attribution hypothesis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26, 547–565. Wilding, E. L. (2006). On the practice of rescaling scalp-recorded electrophysiological data. Biological Psychology, 72, 325–332. Wilimzig, C., Tsuchiya, N., Fahle, M., Einhauser, W., & Koch, C. (2008). Spatial attention increases performance but not subjective confidence in a discrimination task. Journal of Vision, 8, 1–10. Wixted, J. T. (2007). Dual-process theory and signal-detection theory of recognition memory. Psychological Review, 114, 152–176. Wixted, J. T. (2009). Remember/know judgments in cognitive neuroscience: An illustration of the underrepresented point of view. Learning & Memory, 16, 406–412. Wixted, J. T., & Stretch, V. (2004). In defense of the signal detection interpretation of remember/know judgments. Psychonomic Bulletin & Review, 11, 616–641. Wolk, D. A., Schacter, D. L., Berman, A. R., Holcomb, P. J., Daffner, K. R., & Budson, A. E. (2005). Patients with mild Alzheimer’s disease attribute conceptual fluency to prior experience. Neuropsychologia, 43, 1662–1672. Woodruff, C. C., Hayama, H. R., & Rugg, M. D. (2006). Electrophysiological dissociation of the neural correlates of recollection and familiarity. Brain Research, 1100, 125–135.
546 Yap, M. J., Tse, C. S., & Balota, D. A. (2009). Individual differences in the joint effects of semantic priming and word frequency: The role of lexical integrity. Journal of Memory & Language, 61, 303. Yonelinas, A. P. (2001). Consciousness, control, and confidence: The 3 Cs of recognition memory. Journal of Experimental Psychology: General, 130, 361–379. Yonelinas, A. P. (2002). The nature of recollection and familiarity: A review of 30 years of research. Journal of Memory & Language, 46, 441–517. Young, M. P., & Rugg, M. D. (1992). Word frequency and multiple repetition as determinants of the modulation of event-related
J.L. Voss & K.D. Federmeier potentials in a semantic classification task. Psychophysiology, 29, 664–676. Yovel, G., & Paller, K. A. (2004). The neural basis of the butcher-on-thebus phenomenon: When a face seems familiar but is not remembered. NeuroImage, 21, 789–800. Yu, S. S., & Rugg, M. D. (2010). Dissociation of the electrophysiological correlates of familiarity strength and item repetition. Brain Research, 1320, 74–84. (Received April 6, 2010; Accepted June 7, 2010)
Psychophysiology, 48 (2011), 547–558. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01087.x
Ambiguous figures and binding: EEG frequency modulations during multistable perception
WERNER EHM,a MICHAEL BACH,b and JU¨RGEN KORNMEIERa,b a
Institute for Frontier Areas of Psychology and Mental Health, Freiburg, Germany Universita¨ts-Augenklinik, Freiburg, Germany
b
Abstract Ambiguous figures induce sudden transitions between rivaling percepts. We investigated electroencephalogram frequency modulations of accompanying change-related de- and rebinding processes. Presenting the stimuli discontinously, we synchronized perceptual reversals with stimulus onset, which served as a time reference for averaging. The resultant gain in temporal resolution revealed a sequence of time–frequency correlates of the reversal process. Most conspicuous was a transient right-hemispheric gamma modulation preceding endogenous reversals by at least 200 ms. No such modulation occurred with exogenously induced reversals of unambiguous stimulus variants. Postonset components were delayed for ambiguous compared to unambiguous stimuli. The time course of oscillatory activity differed in several respects from predictions based on binding-related hypotheses. The gamma modulation preceding endogenous reversals may indicate an unstable brain state, ready to switch. Descriptors: Binding problem, EEG, Gamma oscillation, Necker cube, Object perception, Temporal coding hypothesis
Research on multistable perception phenomena dates back to Necker’s initial paper (Necker, 1832). However, the neural processes underlying spontaneous perceptual reversals remain elusive (for reviews, see Blake & Logothetis, 2002; Long & Toppino, 2004). A better understanding of how the perceptual system changes spontaneously between two different representations of the same visual object could also shed light on another more general issue, the binding problem: How, in principle, does the brain integrate separately analyzed features to a coherent object representation (e.g., Livingstone & Hubel, 1988; Treisman & Gormican, 1988; Uhlhaas et al., 2009)? In the last two decades binding type problems have also been discussed in connection with the grouping of letters of a word or words within a sentence, with the match of memory contents and perceptual contents, with sensory-motor coupling, the dynamic integration of distant neural subsystems, and with learning and consciousness (e.g., Cosmelli et al., 2004; Herrmann, Munk, & Engel, 2004; Revonsuo, 1999; Uhlhaas et al., 2009). A widely favored but also criticized (e.g., Shadlen & Movshon, 1999) approach to the resolution of ‘‘the’’ binding problem relies on the temporal aspects of neural activity. It is hypothesized that neurons engaged in a common task such as the perceptual representation of some visual object are coordinated and grouped together through temporally synchronized rhythmic firing (Engel, Fries, Konig, Brecht, & Singer, 1999; Milner, 1974; von der Malsburg, 1981). This concept received support from studies with single cell recordings in animals (Eckhorn et al., 1988; Fries, Roelfsema,
Engel, Konig, & Singer, 1997; Gray, Konig, Engel, & Singer, 1989; Hirabayashi & Miyashita, 2005) and in humans (e.g., Tallon-Baudry, Bertrand, Henaff, Isnard, & Fischer, 2005). A version of the temporal coding hypothesis applicable to electroencephalogram (EEG) data was proposed by TallonBaudry and Bertrand (1999). According to their ‘‘representational hypothesis,’’ induced gamma band activity (iGBA; cf. Data Analysis) should play an important role in object representation: Characteristic iGBA modulations would accompany the formation and decay of a coherent percept and reflect, for example, the binding of spatially restricted bottom-up neural activity related to the visual input and top-down activity like rehearsal, retrieval, and utilization of an internal (memorized) representation (e.g., Herrmann et al., 2004; Lutzenberger, Pulvermuller, Elbert, & Birbaumer, 1995; for a review, see Tallon-Baudry, 2009). Spontaneous perceptual alternations in the absence of an external stimulus change should provide ideal test cases for the binding-by-synchrony/temporal coding and the related representational hypotheses (Borisyuk, Chik, & Kazanovich, 2009; Doesburg, Kitajo, & Ward, 2005; Engel et al., 1999; Engel, Fries, & Singer, 2001; Revonsuo, 1999; Tallon-Baudry & Bertrand, 1999; Varela, Lachaux, Rodriguez, & Martinerie, 2001). This is because such alternations should be accompanied by de- and rebinding processes and thus by changes in neural synchronization of a purely endogenous origin free from the confounding factors inevitable when the physical stimulus is changed.
Research was supported by grants from the Deutsche Forschungsgemeinschaft (BA 877-16). Address correspondence to: Ju¨rgen Kornmeier, Institute for Frontier Areas of Psychology and Mental Health, Wilhelmstra!e 3a, 79098 Freiburg, Germany. E-mail:
[email protected] Ambiguous Figures, EEG Oscillations, and the Time Reference Problem Perceptual reversals of ambiguous figures have been reported to be associated with a transient increase of (anterior right 547
548 hemispheric) gamma oscillatory activity (BaSar-Eroglu, Stru¨ber, Schu¨rmann, Stadler, & BaSar, 1996; Mathes, Struber, Stadler, & Basar-Eroglu, 2006; Stru¨ber, Basar-Eroglu, Hoff, & Stadler, 2000; Stru¨ber, Basar-Eroglu, Miener, & Stadler, 2001), a concurrent decrease of alpha activity, and a P300-like positive event-related potential (ERP) component (I˙Sog˘lu-Alkac¸ et al., 2000; Stru¨ber & Herrmann, 2002). A major problem with endogenous perceptual reversals is the lack of a suitable time reference for the not directly accessible reversal instant (Keil, Muller, Ray, Gruber, & Elbert, 1999; Kornmeier & Bach, 2004; Stru¨ber & Herrmann, 2002). Most of the above mentioned studies used the participant’s manual response for this purpose. However, the intra-individual temporal jitter of the reaction times makes it difficult to identify a physiological signature as pre-, peri-, or post-reversal, complicating the interpretation of the reported effects. Stimulus presentation events as time reference were used by Keil et al. and Muller, Federspiel, Fallgatter, and Strik (1999). The former group found reversal-related gamma modulation but could not clarify whether it precedes or follows the reversal. The latter group reported reversal-related modulations in the delta and alpha frequency ranges but did not analyze higher frequencies. Unambiguous stimuli for comparison purposes were considered in neither of the two studies. Recently Kornmeier et al. (Kornmeier & Bach, 2004; Kornmeier, Heinrich, Atmanspacher, & Bach, 2001) proposed another approach to the time reference problem applicable to any type of ambiguous figure. They drew upon Orbach, Ehrlich, and Heath’s (1963) and Orbach, Zucker, and Olson’s (1966) indication that presenting the Necker cube discontinuously with short interruptions should urge the reversal instant into a definite temporal relation with, and close to, stimulus onset. Kornmeier and Bach (2005) adopted this presentation mode and used the stimulus onset as time reference for averaging EEG trials. They reported an improved precision of this time reference with regard to the reversal instant (! 30 ms; Kornmeier & Bach, 2005) compared to reaction time (! 100 ms; Kornmeier & Bach, 2004). This also improved the temporal resolution of the underlying processes and allowed them to identify a chain of successive ERP components. In the present article, the gain in temporal precision is utilized to reliably distinguish, for the first time, pre- and peri/postreversal activity modulations in the frequency domain. This paves the way for testing some conceptions about binding and object representation related to the binding-by-synchrony and the representational hypotheses. Neural correlates of binding were most often reported for (and searched in) the gamma band, with a special focus on induced gamma activity (e.g., Roskies, 1999; Tallon-Baudry & Bertrand, 1999). Recent contributions to the temporal coding discussion also take lower frequencies into account (e.g., Fries, Nikolic, & Singer, 2007). Accordingly, we study amplitude modulations in the theta to beta bands in addition to the gamma band. Frequencies below 4 Hz (delta) would require longer analysis segments than the ones used here and are not considered. The data analyzed in the following is taken from Kornmeier and Bach (2004). Methods Participants Sixteen participants (aged 20 to 31 years, 9 women, 7 men) with normal or corrected visual acuity took part in the experiment.
W. Ehm et al. They gave informed written consent to participate and were naı¨ ve as to the specific experimental question. The experiments were performed in accordance with the ethical standards laid out in the Declaration of Helsinki (World Medical Association, 2000) and approved by the local review board. Stimuli Stimuli were perceptually ambiguous Necker lattices (a combination of nine Necker cubes; Figure 1c,d) and unambiguous variants with depth cues (shading, central perspective, OpenGL lighting model; Woo, Neider, & Davis, 1998) related to the two possible three-dimensional (3D) interpretations of the Necker lattices (Figure 1a,b). All stimuli were generated with a Macintosh G4 computer and presented on a Philips GD 402 monochrome monitor with a frame rate of 85 Hz under a viewing angle of 7.511 " 7.511. Luminance of the Necker lattice was 20 cd/m2; luminance of the unambiguous lattices was 20 cd/m2 averaged across all vertices. In all experimental blocks, successive stimuli were jittered in 3D space over ! 121 of both elevation and azimuth angle, resulting altogether in 14 unambiguous and 7 ambiguous variants, thus avoiding afterimages as trivial local cues. A small cross in the center of the screen served as a fixation target. EEG Recording EEG was recorded from nine gold-cup scalp electrodes at Oz, P3, P4, Pz, C3, C4, Cz, Fz, and FPz (American Clinical Neurophysiology Society, 2006), with averaged ears as reference. Vertical electrooculogram (EOG) electrodes controlled for eyeblinks. Signals were amplified, filtered (first-order analogous bandpass 0.3–70 Hz), digitized with a resolution of 12 bits at a sampling rate of 500 Hz, and streamed to disk. Procedure The Necker lattice is a two-dimensional projection of a 3D object. When viewed binocularly, a conflict arises between the 3D interpretation and the missing stereo disparity. Participants viewed the stimuli monocularly in order to prevent such a conflict. We ran two experiments, one with the ambiguous Necker lattices, the other one with the unambiguous lattices. In each experiment the stimuli were presented discontinuously, an 800ms presentation interval alternating with a 400-ms interstimulus interval (ISI) showing a blank screen. After ambiguity of the Necker stimuli was pointed out to the participants, they compared in a go/no-go task the perceived front–back orientation of the current stimulus with that of the preceding one in two experimental conditions. In one condition they pressed a key whenever the currently perceived orientation differed from the preceding one (reversal condition [RC]). The second condition was identical except that the task was the opposite: a key press whenever the perceived orientation of the stimulus remained the same (stability condition [SC]). Each key press, which had to be executed in the ISI following the task-relevant event, extended that ISI to 1000 ms and terminated the current trial. The first stimulus thereafter started the next trial, which again lasted until the next key press. A trial thus consisted of several (at least two) ISI1presentation interval pairs hereafter called segments. Because of the go/no-go paradigm, the number of such segments varied from trial to trial. The average number of segments within a trial differed also between conditions and stimuli. For instance, a trial usually included more segments (hence lasted longer) under condition RC than under SC, because a percept
Ambiguous figures and binding
549
Figure 1. Paradigm. Participants viewed in different experiments either unambiguous lattices (a,b) or ambiguous lattices (c,d) and compared the 3D perspective of successively presented stimuli. In separate experimental conditions they indicated in a go/no-go manner either a perceived perspective reversal (conditions RC in a and c) or perceived stability (conditions SC in b and d) across two successive stimulus presentations by a key press in the ISI following the respective perceptual event. Each key press extended the current ISI from 400 ms to 1000 ms.
reversal across two successive presentations was less likely than percept stability. In each of the four experiment–condition combinations, trials came in six blocks, with a break of about 1 min in between. Usually, a block was terminated after either 20 responses or 7 min duration. For a few participants with many artifacts or low reversal rate, block length was prolonged in order to get roughly 100 artifact-free trials (see below) per experiment and condition. The 2 ! 2 ! 6 5 24 blocks were counterbalanced across experiments and conditions in an ABCDDCBA scheme in order to reduce sequential effects. The entire experimental session lasted about 3 h with a break of about 10 min after half of the time. Our design disentangles trial epochs from task execution and motor activity. Furthermore, it introduces the stability condition as a suitable control for the reversal condition. In the latter, participants were instructed to report reversals only if they occurred with stimulus onset and to ignore reversals (if any) occurring later in the presentation interval. They reported after the experiment that such delayed reversals happened never or very rarely. Data Analysis EEG recordings were split into trials (cf. Procedure), and an analysis segment was chosen within each trial. The analysis segment consisted of the last (800 ms) presentation interval before a key press along with the (400 ms) interstimulus interval preceding it. The instant of stimulus onset in an analysis segment was chosen as the common time reference across trials, t 5 0. Because endogenous perceptual reversals might be initiated earlier than
400 ms before stimulus onset, we occasionally also considered extended analysis segments reaching from 1600 ms before to 800 ms past t 5 0. Analysis segments showing amplitude excursions exceeding "100 mV, typically related to blinks or eye movements, were automatically detected and discarded, along with the whole trial containing that segment. The minimum number of artifact-free analysis segments per participant, experiment, and condition was 64, the maximum 171, the average 113.1. We then computed the short time Fourier transform (STFT) across each artifact-free analysis segment, using a Hanning window (Papoulis, 1962) with four different window widths. The selected window widths were 160, 240, 320, and 480 ms, to be applied with different frequency ranges corresponding to 40–65 (upper gamma), 26–40 (lower gamma), 14–26 (beta), and 4–14 Hz (theta to alpha), respectively. The time region actually analyzed was restricted to the interval [ # 240, 480] ms in order to minimize boundary effects and because ‘‘late’’ components were not in our focus. Based on the single trial STFTs, a three-level hierarchy of (event-related) time–frequency charts (TFC) was defined as follows: (1) At the lowest level, for each subject, stimulus, experimental condition, electrode, and analysis segment, single trial time–frequency charts (TFCtrial) were obtained by taking absolute values of the respective STFT (amplitudes) and subtracting a frequency-specific baseline (more details below). (2) At the subject level, individual time–frequency charts (TFCsubj) were obtained by averaging the respective single-trial time–frequency charts across trials. (3) Finally, grand means of the individual time–frequency charts across participants yielded average time– frequency charts (TFCgrand).
550 For statistical testing, TFCs were compressed on the frequency axis and band-activity traces (BATtrial, BATsubj, BATgrand) were obtained by averaging the respective TFCs across various frequency bands in the theta to gamma range. To remove the activity related to stimulus onset and the processing of low-level stimulus features, we focused on differences of TFCs/ BATs under the reversal (test) and stability (control) conditions, namely, RC minus SC. A note on terminology: The TFCs/BATs as computed here represent what is often called induced activity. We adhere to this usage even though some authors prefer to call it total activity (e.g., Herrmann et al., 2004). More important is the delimitation of induced/total activity from evoked activity, which is computed from the ERP rather than from single trials. Evoked activity picks up only those features that are strictly phase-locked, whereas the present induced measures also collect activity jittering in time across trials, as it typically occurs before stimulus onset and at higher latencies or in endogenously generated processes, which are of particular interest here. Statistics Under the null hypothesis H0, ‘‘no systematic difference between the reversal condition (RC) and the stability condition (SC),’’ the time course of the difference of the respective band activity traces should only exhibit chance fluctuations around the zero line. Randomization tests were performed to check for systematic departures from H0 at the population level. The tests were based on the maximum and minimum values (separately) assumed by the difference of the average ( 5 average of differences) traces BATgrand(RC) ! BATgrand(SC) within a time region of interest (ROI). A reference null distribution for the observed extrema was obtained as follows. For each subject, the two BATsubj under study were randomly labeled SCn and RCn. The difference BATsubj(RCn) ! BATsubj(SCn) was then averaged across participants, and the extreme values were obtained in the resulting average trace. This was repeated 10,000 times, providing reference distributions, hence p values, for the actually observed maximum and minimum values. Such a randomization procedure yields a correct p value up to a random error that is largely negligible with 10,000 repetitions (Edgington & Onghena, 2007). To get a time-, frequency-, and space-resolved picture, the tests were carried out for a large number of (preselected) tfe cells: A tfe cell is understood as a triplet (T,F,E) resulting from the choice of a ROI T, a frequency band F, and an electrode E. ROIs T were chosen as overlapping time intervals of width 160 ms centered at delays ! 80, 0, 80, . . ., 400 ms with respect to stimulus onset. Frequency bands F in the range 4–65 Hz, such as 4–7 Hz, 8–13 Hz, and so forth, were selected jointly with suitable window widths of the STFT as indicated above.1 A significant result for the maximum (minimum) type test at tfe cell (T,F,E) 1
Frequency bands F were selected by the following criteria. They should (1) sparsely cover the range from 4 to 65 Hz, (2) fall roughly within some standard frequency range, (3) be neither too large (to allow for sufficient discrimination) nor too narrow (compared to the actual frequency resolution), and (4) overlap (because a peak/dip at the boundary of F is difficult to detect with average amplitudes across F). Given these criteria, we decided to select the following frequency bands: 4–7; 8– 13, 10–15; 14–20, 20–26, 14–26; 26–32, 32–40, 26–40; 40–50, 45–55, 50– 60, 55–65; 35–55, 40–60, 45–65 Hz. It should be noted that the classical frequency bands (e.g., alpha: 8–12/13 Hz) refer to ‘‘raw’’ signals. However, which frequency bands are relevant to the comparison RC versus SC is not a priori clear.
W. Ehm et al. indicates that the oscillatory activity for frequencies in band F at electrode E is distinctly higher (lower) somewhere within ROI T under the reversal than under the stability condition. Analogous tests for comparisons a and b were applied at the level of individual participants, based on the maximum (and minimum) of the difference trace BATsubj(RC) ! BATsubj(SC). Corresponding reference distributions were obtained similarly as above by shuffling the single trials. The trials from both conditions were pooled and then randomly reassigned to SC and RC, observing the original number of trials in the two conditions. Repeated 10,000 times, the procedure yielded a p value for the observed maximum as above, now at the individual level. The choice of a baseline interval presents special problems in our case. Usually in studies with evoked potentials, the baseline is taken across a short time interval before stimulus onset. The exact choice was nonessential if ‘‘nothing of interest’’ happened within this baseline interval, as is usually assumed. With ambiguous stimuli, however, endogenous perceptual reversals may well be ‘‘prepared’’ prior to stimulus onset within the typical range of baseline intervals, so that effects of interest may be confounded with baseline adjustment effects.2 To make significance criteria less dependent on the choice of the baseline, we considered three different baseline intervals, namely [ ! 100,0], [ ! 200,0], and [ ! 240,1480] ms, and reported as ‘‘three-significant’’ only those instances where the single tests were significant for all three baseline intervals simultaneously. Precisely, we considered as three-significant (nn) any test for which all three p values are o.01; and as three-significant (n) any test for which all three p values are o.05 and po.01 for at least one of the baseline intervals. We corroborated some results using a work-around for the baseline interval problem: the derivative, or increment, of a band activity trace with respect to time is independent of the baseline interval. Because local maxima of a trace are preceded (or followed) by maxima (or minima) of the derivative, the latter gives a clue to the onset (or offset) of a positive deflection of the trace, which can be of interest in itself. Tests for extrema of derivative traces were implemented in complete analogy to those for the original traces. Alpha error inflation due to the massive multiple testing involved was only accounted for by setting tight limits via the notion of three-significance. Arguments defending such an exploratory approach are given in the discussion. Results Behavioral results (e.g., reversal rates) and ERPs for the present data have already been reported by Kornmeier and Bach (2004). Prior to our main results we first present, purely phenomenologically, the raw time-frequency characteristics of the original EEG response before differencing, which are also of interest. Raw TFCs and BATs Grand average TFCs averaged across the experimental conditions (indication of perceptual reversal and stability) are shown in Figure 2a, separately for the two stimuli (ambiguous [A] and unambiguous [U]). The most conspicuous patterns common to both stimuli are (a) the initial activation in the alpha to beta range 2 The choice of the baseline interval determines what counts as a positive or negative deflection. For example, if the baseline interval is located around the global minimum of the trace, anywhere else one will find positive deflections.
Ambiguous figures and binding immediately after onset that quickly shifts to and then persists in the theta and lower alpha range (o10 Hz); (b) the sustained deactivation in the beta band starting about 100 ms after onset at 20 Hz and afterward spreading to the whole beta and upper alpha band (10–26 Hz); (c) the less pronounced, transient increase in gamma activity (50 Hz) roughly around 350 ms after onset at the frontopolar electrode (FPz); and (d) a fourth feature, not so obvious from Figure 2a, that is a transient enhancement of gamma activity within the ISI before onset. Remarkably, this pre-onset gamma enhancement essentially is confined to the ambiguous stimulus and to central and righthemispheric electrode positions. As another difference, the postonset gamma enhancement (feature c) apparently is delayed for ambiguous compared to unambiguous stimuli. Further interesting phenomena become visible when epochs are prolonged to the ‘‘past’’ beyond the ISI; see Figure 3. For ease of reference, let us call the intervals [ ! 400, 600] ms and [ ! 1400, ! 400] ms the last and the penultimate segments before the key press, abbreviated LS and PS, respectively. Alpha and beta band traces clearly exhibit deactivation with time, in a twofold sense. First, activity steeply goes down within both PS and
551 LS after a short ascent around the respective stimulus onset (which is at ! 1200 ms in PS) and then recovers steadily, though without returning to its initial level. Second, on a larger time scale, average amplitudes in the segment LS are distinctly decreased compared to those in the segment PS. In the theta band, activity strongly increases with stimulus onset and settles back roughly to its baseline after about 600 ms within both PS and LS. Gamma band traces exhibit more complex features (Figure 3, first three rows). Similar to the adjacent beta frequencies, there appears to be a tendency for gamma deactivation at a larger scale (i.e., across PS and LS). However, shapes differ from the staircase-like decrease in the alpha and beta bands and appear less homogeneous across conditions, stimuli, frequency bands, and electrodes. Comparison of Experimental Conditions RC versus SC The difference–TFCgrand in Figure 2b highlight the time– frequency regions of higher or lower activity under RC compared to SC. To single out the statistically significant excursions, tests based on maximal deflections of difference-BATgrand were carried out for a large number of tfe cells covering the time–
Figure 2. Time–frequency charts. The baseline interval was [! 100,0] ms. The color scale in panel a was chosen so as to emphasize small effects at higher frequencies, which otherwise were unnoticeable due to dominance of low frequency amplitudes. (Conversely, modulations within large effects at low frequencies become less distinguishable.) Dashed vertical lines mark the stimulus onset. TFCs for the ambiguous and unambiguous stimuli are stacked in pairs, ambiguous on top, unambiguous underneath, on a gray background. (a) Raw time–frequency charts. Grand mean time–frequency charts were obtained by averaging TFCgrand (cf. Data Analysis) across conditions and stimuli. (b) Differences of TFCgrand (reversal minus stability). Entries in b such as A1, U2, and so forth indicate the position in the time–frequency plane of significant deflections from zero (‘‘components’’; cf. Table 1). A: ambiguous; U: unambiguous; 1, 2, and so forth: temporal order; black/white coloring of the numbers and letters is only for saliency.
552
W. Ehm et al. Table 1. Listings of All Three-Significant Outcomes When Testing RC versus SC (Necker Stimulus) Extremum
Figure 3. Band activity traces BATgrand were obtained by averaging time–frequency charts TFCgrand over selected frequency bands. Solid and dashed lines stand for reversal and stability conditions, respectively, and gray-bold and black-thin for ambiguous and unambiguous stimuli, respectively. For presentation, traces were baseline corrected by subtracting average amplitudes.
frequency–electrode space (cf. Statistics). Table 1 lists all tfe cells that gave rise to a three-significant test result for the ambiguous stimulus, ordered according to peak time and grouped into components. Such listings, followed by a grouping according to peak time, frequency, and sign, underlie all features discussed in the following. For frequencies below or above 26 Hz, significance is understood as three-significant (nn) or three-significant (n), respectively. Only tfe cells with ROIs ending not later than 400 ms after onset are taken into account. In Figure 2b the significant components are indicated by numbers in the order of their appearance, accompanied by prefix A (ambiguous) in order to distinguish them from components U1, U2, and so on found with stimulus U (unambiguous). For the latter, components were obtained in the same way as for stimulus A, based on a list of three-significant test results. The difference charts represent grand means without any information about the variability across participants. Thus what appears prominent in Figure 2b need not necessarily be significant, nor do all significant components stand out clearly. Figure 4 summarizes the test results by indicating peak times, frequencies, positions, and signs of the components. The most conspicuous difference between stimuli A and U is the appearance of components A1 and A2 within the ISI before stimulus onset in the case of stimulus A. This pattern of activity has no counterpart with stimulus U, which underlines a similar obser-
ROI (ms)
Frequency Peak band (Hz) Electrode time (ms) Sign Component
[! 240, ! 80] [! 160, 0] [! 240, ! 80] [! 240, ! 80] [! 160, 0] [! 160, 0] [! 80, 80] [0, 160] [! 80, 80] [! 80, 80] [! 80, 80] [0, 160] [0, 160] [80, 240] [0, 160] [80, 240] [0, 160] [80, 240] [0, 160] [80, 240] [0, 160] [80, 240] [160, 320] [160, 320] [80, 240] [160, 320] [160, 320] [240, 400] [240, 400] [240, 400] [240, 400]
26–32 40–50
P4 C4 Cz Oz
35–55 55–65
C3
4–7
Cz Fz Oz
8–13
P3
10–15
Fpz
8–13
Oz
10–15
Fpz P3 C3 P3
45–55 35–55 14–26
C3 C3 Cz Cz C4
o ! 250 ! 160 ! 178 ! 166 ! 158 ! 166 26 26 34 34 74 74 134 134 134 134 146 146 146 146 158 158 170 170 194 194 194 258 242 250 338
1
A1
!
A2
!
A3
!
A4
!
A5
1
A6
1
A7
Note: In columns 2 to 6, any entry identical to the one above it is omitted for better readability.
vation made for the raw TFCs (feature d in the previous section). Component A1 represents a positive deflection of the differenceBATgrand at low gamma frequencies (26–32 Hz) about 200 ms before onset at right posterior and central positions. As a further check of the genuineness of the deflection, permutation tests for the derivatives of the difference traces were carried out (cf. Statistics). They resulted in significant (p o.01) minima at electrodes C4 and Cz for ROIs [ ! 240, ! 80], [ ! 160,0], and band 26–32 Hz located about 130 ms before onset, that is, in the right flank of the peak of the original difference traces. No significant maxima of derivatives were found in the left flank, maybe due to a less rapid ascent to the maximum than decay from it. Components A1, A2 are followed by a succession of three negative deflections starting immediately after onset at left central and frontal positions in the high gamma range (A3), and ending after about 200 ms at left hemispheric positions with frequencies in the range 8–15 Hz (A5). Differential gamma reduction early after stimulus onset also is observable for the unambiguous stimulus (U1), whereas the latter two features (A4 and A5) occur with ambiguous figures only. A feature shared by both stimuli is the positive higher gamma deflection at left central electrodes and 150 to 250 ms peak time (components A6, U2). In both cases it is followed by a positive deflection in the beta band (components A7, U3; see the gray underlain areas in Figure 4). However, there is a marked difference in the peak time of the components
Ambiguous figures and binding
553
Figure 4. Significance diagram: Summary of all significant differences (‘‘components’’ in Table 1). The closed polygons surrounding electrodes mark three-significant test results; color indicates the corresponding frequency range. Head positions on the time axes indicate peak time. Positive (negative) excursions appear above (below) the time axes. Top: Necker stimulus; bottom: unambiguous stimulus. Gray backgrounds highlight apparent analogies between ambiguous and unambiguous stimuli.
between stimuli, those for A being delayed by about 100 ms compared to U. Individual versus Group Results Tests on the participant level were carried out only for tfe cells relevant to the components identified in the group analysis. Interindividual variability was considered by allowing for slightly more flexibility regarding frequency bands if these appeared too narrow. For instance, component A1 was taken to consist of its deviation type (positive) together with all tfe cells obtained by free combinations of ROI [ ! 240, ! 80] ms, bands 26–32, 26–40 Hz, and electrodes P4, C4, Cz. Table 2 summarizes the individual test results by listing for each component the number of participants having a three-significant outcome for at least one of the relevant tfe cells. The term three-significance here is understood in a weakened sense: (n) and (nn) mean that po.05 and po.02 for all three baselines, respectively. Discussion Ambiguous figures allow for the investigation of EEG correlates of the decay and rebuilding of object representations in the absence of potentially confounding factors related to a change in visual stimulation. In the present study, we analyzed EEG correlates of spontaneous perceptual reversals of a Necker-type stimulus in the time–frequency domain. Using a discontinuous stimulus presentation mode, we synchronized perceptual reversals with stimulus onset and used the latter as a time reference for the reversal instant. Participants continually compared the perceived stimulus perspective between successive presentations. In two experimental conditions of the go/no-go type, they had to
indicate either perceptual reversal (reversal condition) or perceptual stability (stability condition). A second, entirely analogous experiment was run with unambiguous stimulus variants. There, the orientation reversals were induced by the stimulation program. This paradigm has a number of advantages. (1) The parallel consideration of an experiment with unambiguous stimuli allows for the identification of effects distinctive of endogenously generated percept reversals as compared to exogenously driven reversals. (2) Both experimental conditions, RC and SC, have identical stimulation protocols, use the same stimulus, and require preparation of a motor response. Therefore, EEG patterns related to such confounding factors should cancel when taking differences RC minus SC, and only the reversal-related activity should remain. Nevertheless, we also considered raw data before subtraction for better comparability with related EEG studies in which one or several confounders had not been, or could not be, handled symmetrically in the test (here: RC) and control (here: SC) conditions. (3) The discontinuous presentation paradigm yields a time reference, stimulus onset, that is more closely aligned to the reversal-related processes than a key press, the time reference commonly used with continuous presentation paradigms. Kornmeier and Bach (2006) reported a gain in temporal precision by a factor of more than 3, which helped uncover activity patterns that would otherwise be blurred by reaction time jitter. In particular, pre- and peri-/post-reversal events could be distinguished more reliably. The present approach also has its limitations and challenges. (i) Our results do not immediately carry over to the case where the stimulus is observed continuously. Still, there is some evidence that, in regard to the spontaneous reversal-related
554
W. Ehm et al.
Table 2. Summary of the Individually Three-Significant Test Results Component A1 A2 A3 A4 A5 A6 A7 U1 U2 U3 U4 No. of participants Significance
4 n
1 n
1 n
1
2
nn
nn
3 n
0
1 n
3
0
0
nn
po.05; nnpo.01.
n
processes, the difference between the two settings is not substantial (e.g., Kornmeier & Bach, 2004, 2006; Sterzer & Rees, 2008). (ii) By items 1–3 above, our approach opens up chances for uncovering so far inaccessible neural correlates. This, conversely, means that there is yet little knowledge about potential regions of interest. One is thus left with a large search space initially, which in turn suggests adopting an exploratory approach using heavy multiple testing. (3) Any attempt to deal with the ensuing problem of alpha error inflation, for example, by means of (then necessarily massive) Bonferroni adjustment, would require very small p values. These, however are out of reach with the randomization tests used here.3 Given these considerations it appeared to us more important at the current stage to collect possibly meaningful patterns than to prove they are relevant. Ultimately, the latter question will not be answered by a single experiment but by accumulation of evidence from diverse studies under varied conditions. We here report for the first time a sequence of time–frequency components tracking the processes underlying endogenous perceptual reversals of ambiguous figures with high temporal precision. This sequence has several similarities to but also important differences from the corresponding sequence for the experiment with unambiguous stimulus variants. Raw time-frequency characteristics (prior to differencing) ! Object presentation causes an initial amplitude increase in the alpha and beta frequency range (after stimulus onset) that quickly shifts to the lower alpha and theta band. Further, there is a massive, sustained beta reduction starting about 100 ms after stimulus onset. ! On a larger time scale, average amplitudes at alpha and beta frequencies are decreased in the last interval before response compared to the penultimate stimulus interval. These observations apply to both stimulus types and experimental conditions. Our main findings rely on test results for differences of band activity traces (BATs). Differential time-frequency characteristics: Reversal minus stability ! Lower gamma activity (26–32 Hz) was (differentially) increased roughly about 200 ms before stimulus onset, which deflection was followed by a decrease of higher gamma activity (35–55 Hz) at about " 160 ms. These 3 Very small p values easily surviving Bonferroni correction are attainable with the classical F-type tests, but presuppose overreliance on distributional assumptions. Here, we applied randomization tests with guaranteed validity practically independent of preconditions. However, for the latter, it is often impossible to determine very small p values with the necessary accuracy because the required number of Monte Carlo simulations is infeasibly large.
transient modulations of the difference traces showed a righthemisphic dominance and occurred with ambiguous Necker stimuli only. No prestimulus gamma modulation was observed in the case of exogenously induced reversals of unambiguous stimuli. ! For both stimulus types gamma band difference traces decreased shortly after stimulus onset. ! About 250 ms after stimulus onset, there was a transient increase of induced gamma activity for reversed compared to unchanged (stable) percepts in the case of ambiguous stimuli. A similar increase occurred about 100 ms earlier in the case of unambiguous stimuli. We associate it with what is commonly referred to as induced gamma band activity (e.g., TallonBaudry, 2009). Could Gamma Activity Modulations Be Explained by Miniature Saccades? In recent widely recognized studies, Yuval-Greenberg and Deouell (2009) and Yuval-Greenberg, Tomer, Keren, Nelken, and Deouell (2008) demonstrated that iGBA largely can be an artifact of a type of eye movement called miniature saccades. This fundamentally questions the origin of iGBA in gamma oscillations and, thus, its functional role in visual grouping and the related binding processes. We therefore checked whether miniature saccades could alternatively explain the present iGBA results. To summarize briefly, we found iGBA to be correlated with EEG spike clustering in the raw band activity traces (BATsubj). Quantitatively, spike clustering was less strong than in YuvalGreenberg et al.’s data. Moreover, the differential effects found when comparing the reversal and stability conditions were only marginally affected by spike rate modulations and less so for component A1 than for A6. Thus for our data, miniature saccades do not provide a sufficient explanation of these differential effects. Neural Oscillations and Temporal Aspects of Binding Our discussion will be organized around three predictions derived from the representational hypothesis (cf. Introduction). For clarity, let us state what, in the context of bistable perception of a Necker type stimulus, we tentatively considered as most relevant to binding: (i) the grouping of the single edges to a 3D object seen in a certain, well-defined perspective and (ii) the match of the incoming bottom-up sensory information with topdown memory contents (e.g., the memorized perspective of the stimulus). Prediction 1: Onset of gamma activity when a new percept is built up. If iGBA reflects binding in the sense of (i) and (ii) above, the buildup of a new stable percept should be accompanied by an onset of enhanced iGBA (Tallon-Baudry, 2009; Uhlhaas et al., 2009). Based on previous ERP and reaction time findings we expect the onset of such activity to be delayed for endogenous as compared to exogenously induced reversals by about 40 ms (Kornmeier & Bach, 2006). In any case, it should antecede neural activity reflecting the maintenance of the newly built percept. Enhancement of induced gamma activity at about 200–300 ms after stimulus onset of visual and auditory stimuli is an often reported finding. It was related to a variety of cognitive functions, often in connection with a binding interpretation (e.g., Engel et al., 2001; Herrmann et al., 2004; Tallon-Baudry & Bertrand, 1999; cf. Introduction) and recently also with miniature saccades (Yuval-Greenberg & Deouell, 2009; Yuval-Greenberg et al., 2008) as discussed above. For multistable perception,
Ambiguous figures and binding enhancement of induced gamma activity during spontaneous perceptual reversals was observed in a number of studies (e.g., BaSar-Eroglu et al., 1996; Mathes et al., 2006; Stru¨ber et al., 2000, 2001); however, the timing of this activity with respect to the reversal instant remained vague. The higher temporal resolution achieved in the present study revealed two instances of a differential gamma modulation, located about 200 ms before and after the onset of a perceptually reversed Necker stimulus. It appears plausible to assume that binding as understood here does not start until the stimulus is presented, leaving only the postonset gamma component as a possible correlate of binding. But how early or late is 200 ms in regard to binding? The earliest ERP correlate of an endogenous perceptual reversal of a Necker stimulus found by Kornmeier and Bach (e.g., Kornmeier & Bach, 2006) was a positivity about 130 ms after stimulus onset (‘‘reversal positivity’’). The subsequent chain of ERP components was delayed by about 40 ms compared to an otherwise largely identical ERP chain found with exogenously induced reversals of unambiguous stimulus variants. Kornmeier and Bach argued that this processing delay should account for the time necessary to disambiguate the ambiguous visual information from the Necker stimulus and, furthermore, that the ‘‘decision’’ about the perceptual outcome had taken place 260 ms after stimulus onset, at the latest, which is the temporal position of the (ERP) ‘‘reversal negativity’’ (Britz, Landis, & Michel, 2009; Kornmeier & Bach, 2004, 2006; Pitts, Nerger, & Davis, 2007). Let us, as in Kornmeier and Bach’s ERP analyses (e.g., Kornmeier & Bach, 2006), conceive of the succession of components displayed in Figure 4 as single steps of some underlying processing chain. Similarly here, the processing step underlying the iGBA enhancement occurs later with endogenously than with exogenously induced reversals (components A6, U2 in Figure 2b and Figure 4), and its temporal position at about 250 ms fits well with Kornmeier and Bach’s upper estimate for the completion of the perceptual decision process. The time delays, 100 ms here versus 40 ms in the ERPs, agree less well, which might partly be due to the lower temporal resolution of time–frequency analyses compared to ERPs. All in all, we may tentatively associate components A6 and U2 with Kornmeier et al.’s reversal negativity. Moreover, there might be a connection between the (hypothetical) disambiguation process starting at their reversal positivity and component A5. The temporal position of the reversal positivity, at 130 ms, coincides with the beginning of A5’s temporal extension (cf. Table 1). Along with A5’s broad spatial distribution, from occipital to frontopolar electrodes, and the fact that neither A5 nor the reversal positivity have an analogue with the unambiguous stimuli, this suggests associating component A5 with the disambiguation process involving recurrent activity between several brain areas. We speculate that during disambiguation the perceptual system tries to matchFor bindFthe visual information to one of the perceptual interpretations (pre)existing in memory. Visual information may be called ambiguous if more than one interpretation is possible and a kind of binding competition takes place. This could explain the time delay of the measured ERP and frequency components between the ambiguous and unambiguous stimuli. The assumption that this kind of binding has already taken place at around 130 ms fits well with recent results by Kirchner and Thorpe (2006), who found that natural scences with and without animals can be distinguished as early as 120 ms after stimulus onset. This would seem possible only if binding in the above sense has already taken place by that time.
555 According to these considerations, the transient gamma modulation (component A6) about 100 ms after the alpha modulation (component A5, between 130 ms and 190 ms) could mirror perceptual processes executing after disambiguation and, hence, after binding, in the sense described in connection with our predictions, has already been accomplished. This does not rule out a binding interpretation of component A6. It could reflect binding processes executing later on, such as binding necessary for a conscious percept or the binding of the perceptual outcome with the task and/or the related motor preparation and execution as recently suggested (Revonsuo, 1999; Uhlhaas et al., 2009) This, then, would mean that the processes from vision to action are implemented in a stepwise fashion, a supposition that would yet have to be verified. In any case, the discussion highlights the necessity of clearly specifying what is meant by ‘‘binding’’ when it comes to timing issues. Prediction (2): Sustained gamma activity during perceptual stability. If iGBA reflects the maintenance of a percept, it should continue during perceptually stable periods. Recent studies (e.g., Rolls, 2000; Sterzer & Rees, 2008) suggest that the neural activity underlying a stable object representation is maintained even when the stimulus is temporarily absent during short interstimulus intervals as used in the present paradigm. We only found transient gamma modulations. This is in line with a number of studies reporting a transient iGBA increase between 200 ms and 300 ms after stimulus onset (e.g., TallonBaudry, 2009). Induced gamma band activity was often interpreted as evidence for binding processes (e.g., Uhlhaas et al., 2009). However, the question of how the result of this bindingFthe final object representationFis held active during the period of the observer’s object awareness remains largely unanswered. If gamma oscillations were crucial in that respect, they should continue during periods of percept stability. In fact, sustained gamma activity could be observed with single cell ablations in primates (e.g., Fries et al., 1997; Fries, Schroder, Roelfsema, Singer, & Engel, 2002) and humans as well as in magnetoencephalography (MEG) data (e.g., Hoogenboom, Schoffelen, Oostenveld, Parkes, & Fries, 2006), though generally not with EEG data in humans (e.g., Tallon-Baudry et al., 2005); an exception is the study by Koch, Werner, Steinbrink, Fries, and Obrig (2009). In response to this issue. it was hypothesized that transient gamma in the EEG signifies the supposedly fast buildup of a coherent percept rather than its maintenance (e.g., Engel et al., 1999; Revonsuo, 1999; Uhlhaas et al., 2009). The question of how the percept is held active once built up was not addressed. According to another idea, gamma oscillations come in periodic bursts gated by low frequency oscillations (3–6 Hz) during periods of percept stability (Fries et al., 2007; see also Schroeder & Lakatos, 2009). The object representation would then be maintained by continued refreshment or cognitive update (e.g., VanRullen, Reddy, & Koch, 2006). A third possibility might result from too fuzzy a time reference. Even if gamma oscillations were sustained during the period of percept stability, that period could be shifted back and forth across trials relative to the time reference. Averaging across trials would then produce a peak in the center of the jitter region and, hence, the impression of a transient phenomenon. For stimulus-related activity, such is certainly more likely to happen when relying on a button press as a time reference than with the present paradigm. In contrast to the gamma band, we did observe sustained activity modulations at lower frequencies. In large parts of the presentation intervals theta activity was enhanced, whereas beta
556 was suppressed in the TFCgrand/BATgrand data before subtraction (Figures 2a and 3). In fact, a growing body of literature reports lower frequency oscillations correlated with several perceptual and cognitive brain functions (e.g., Gail, Brinksmeyer, & Eckhorn, 2004; Klimesch, Freunberger, Sauseng, & Gruber, 2008; Pfurtscheller & Lopes da Silva, 1999). For example, there is recent evidence for sustained oscillations in the alpha range correlated with motion perception of the ambiguous WagonWheel stimulus (VanRullen et al., 2006). This raises the question of whether sustained binding underlying stable object representation might be realized, at least in the present case, by synchronous neural activity in the theta frequency band concurrent with suppression of the beta band. Prediction (3): Decay of gamma activity prior to perceptual reversals. Psychophysical and physiological evidence suggests that an initially stable conscious percept gradually wanes up to an instant of maximal instability, after which an alternative object representation is built up (e.g., Long, Toppino, & Mondin, 1992; Roeber & Veser, 2009; Stru¨ber & Herrmann, 2002). Accordingly, gamma activity should decrease slowly prior to endogenous reversals and collapse suddenly in the case of exogenously induced reversals. Psychophysical evidence for decay processes preceding perceptual reversals recently was found by Roeber and Veser (2009) and Alais, Cass, O’Shea, and Blake (2009). In an MEG experiment with the Stroposcopic Alternative Motion (SAM) stimulus,4 Stru¨ber and Herrmann (2002) observed a slow decay of oscillatory activity during endogenous motion reversals and a fast drop after onset in the case of exogenously induced motion reversals of an unambiguous SAM variant. However, this finding pertained to the alpha band and not to the gamma band commonly regarded as relevant for object representation. Inspection of Figure 3 indicates that, on average, alpha activity decays from the penultimate ([! 1600, ! 400] ms) to the last segment ([! 400,800] ms) before the key press, in agreement with Stru¨ber and Herrmann’s results. Notably, this pattern is restricted neither to the unambiguous stimuli nor to the reversal condition. In fact, the overall shapes of the band activity traces BATgrand in the lower frequency bands exhibit a number of striking similarities. Such are prominent when comparing (1) the BATgrand across stimuli and conditions (theta, alpha, beta), (2) the respective time courses within the penultimate and the last segment before the key press (theta; in the alpha and beta bands up to an overall downward shift), and (3) the BATgrand for the alpha and beta bands (apart from the difference in magnitude of the amplitude deflections). Initially, this homogeneityFwhich also prevails at electrodes other than P4 and Cz (not shown in Figure 3)Frenders it unlikely that oscillations in the lower frequency bands (theta, alpha, beta) should carry important information specific to stimulus type or to perceptual reversal/stability. They rather may reflect processes triggered by the discontinuous presentation mode and slow state changes across time. The slow/fast decay phenomenon initially was hypothesized to occur in the gamma frequency range. Close examination of Figure 3 suggests that, in the reversal condition, the BATgrand 4 The SAM stimulus was introduced by von Schiller (1933) and consists of four dots placed at the corners of an imaginary square. Two diagonal dots flash together in synchrony and in alternation with the other diagonal pair. This presentation mode induces either apparent horizontal or vertical motion. Both directions of motion perception are mutually exclusive, instable, and alternate spontaneously with each other.
W. Ehm et al. decay after stimulus onset (at t 5 0) in the range 26–40 Hz may indeed be steeper at electrode Cz for the unambiguous than for the ambiguous stimulus. However, the evidence is weak. Individual versus Group Results Most of the time–frequency components determined on the basis of grand means could also be found in individuals, though generally for a few only, and for different onesFmaximally 4 out of 16 in the case of component A1 (Table 2). The reasons behind this gap can be manifold. In the best case, the trial-wise variability within participants is too large to allow for individual significance, but effects are homogeneous enough across participants to aggregate to population-wise significance. The latter may, however, also be due to the influence of a single subject and vanish if it is excluded from the population. Such is the case with component A4, for instance, where three-significance is missed upon exclusion of 1 of the 16 participants, even if the criterion is weakened to po.1 for all baselines. On a more fundamental note, grand average analyses certainly are appropriate if effects for the single participants are weak but unanimous, described as the best case above. However, they appear less appropriate if response patterns are dissimilar or oppositional across participants. Sometimes such may result from artifacts and can safely be ignored. Pattern classification would account for diversity but does not give way to one-sentence take-home messages. A purely statistical approach would declare deviant cases as outliers and focus on the bulk. In fact, at the lower sensory or motor processing levels, all human brains seem to work similarly, which affords the basis for neurological diagnostics. Here, the statistical approach appears suitable. However, the more cognitive neural processing becomes, the more likely it is that individuality prevails over commonness. From this perspective, interindividual discrepancies present less surprise than the often remarkable similarities of grand averages across different conditions, a circumstance that allows differential analyses to focus on a few components only. Does Gamma Predict Spontaneous Perceptual Reversals? The most conspicuous and robust result of our study was the pattern of increased and decreased gamma activity in the interstimulus interval preceding those ambiguous stimuli that were indicated as reversed. No such effects were observed in the case of unambiguous stimuli. Additional evidence from other studies indicates that this increase of right-hemispheric induced gamma activity at about 200 ms before onset and its subsequent occipital/parietal decrease may be highly specific to and even predictive for endogenous reversals. Thus BaSar-Eroglu et al. (1996) reported a right anterior gamma increase during endogenous perceptual reversals of the SAM stimulus. VanRullen et al. (2006) recently found higher gamma activity at right hemispheric central locations with illusory motion direction reversals of the Wagon-Wheel Illusion compared to real motion reversals. Lumer, Friston, and Rees (1998) found selective right hemispheric functional magnetic resonance imaging (fMRI) activation during perceptual transitions of binocular rivalry stimuli, but no such activity with exogenous transitions of unambiguous stimulus variants. Sterzer and Kleinschmidt (2007) reported an increased fMRI response in the right inferior frontal cortex with endogenous motion reversals of the SAM stimulus compared to exogenously induced reversals of unambiguous SAM variants. Similarly, Ilg et al. (2008) found posterior right hemispheric fMRI activity with spontaneous
Ambiguous figures and binding
557
motion direction reversals of the spinning wheel illusion (Wertheimer, 1912), but no such activity with exogenously induced reversals. Roeber et al. (2008) recently reported a right hemispheric ERP correlate of binocular rivalry of sine wave gratings. Muller et al. (2005) used the onset of the SAM stimulus immediately before a button press as the time reference for reversals of motion direction. They found changes in EEG activity about 300 ms before the reversal-related SAM flashes, that is, temporally close to our pre-onset gamma modulation. Spatial information was not provided. Britz et al. (2009) reported right-hemispheric EEG correlates anticipating spontaneous perceptual reversals. They used the same onset paradigm and type of ambiguous stimulus as in the present work, but did not consider unambiguous stimulus variants. In summary, all studies listed above reported right-hemispheric EEG activity accompanying endogenous perceptual reversals. Some of them provide a sufficient temporal resolution to locate this activity prior to endogenous reversals. Conclusions Single-cell recordings in primates and MEG studies in humans indicate that stable object representation is accompanied by sustained oscillatory activity in the gamma frequency range (e.g., Tallon-Baudry, 2009). In contrast, in the present study with EEG data, the periods of perceptual stability were
accompanied by a sustained increase of theta and decrease of beta activity. iGBA was transient only, as in many other EEG studies, and was delayed by more than 100 ms compared to the onset of lower frequency oscillations (beta, theta). The delay casts doubt on iGBA as a correlate of binding processes as unterstood here, which should begin early after stimulus onset. On the other hand, (differential) iGBA also could not be reduced to miniature saccades. Thus the role of the post-onset gamma modulations remains largely unclear in the present study. Transient right-hemispheric modulations of gamma activity were found in the interstimulus interval preceding an endogenous perceptual reversal by about 200 ms. Remarkably, similar prereversal right-hemispheric activations were observed with a variety of paradigms and ambiguous stimuli. A common feature of all these settings is spontaneous perceptual reversals and, along with them, endogenously generated transitions from one stable brain state to another. We speculate that prereversal gamma modulations might reflect a transient brain state of maximal instability reached in the interstimulus interval antedating a perceptual reversal or else the recognition of that state by some unconscious and so far unknown neural instance.
REFERENCES Alais, D., Cass, J., O’Shea, R. P., & Blake, R. (2009). Adaptation-related changes during binocular rivalry phases: Dominance sensitivity declines, suppression sensitivity improves. Perception, 38, 1–4. American Clinical Neurophysiology Society. (2006). Guideline 5: Guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology, 23, 107–110. BaSar-Eroglu, C., Stru¨ber, D., Schu¨rmann, M., Stadler, M., & BaSar, E. (1996). Gamma-band responses in the brain: A short review of psychophysiological correlates and functional significance. International Journal of Psychophysiology, 24, 101–112. Blake, R., & Logothetis, N. K. (2002). Visual competition. Nature Reviews Neuroscience, 3, 13–21. Borisyuk, R., Chik, D., & Kazanovich, Y. (2009). Visual perception of ambiguous figures: Synchronization based neural models. Biological Cybernetics, 100, 491–504. Britz, J., Landis, T., & Michel, C. M. (2009). Right parietal brain activity precedes perceptual alternation of bistable stimuli. Cerebral Cortex, 19, 55–65. Cosmelli, D., David, O., Lachaux, J. P., Martinerie, J., Garnero, L., Renault, B., et al. (2004). Waves of consciousness: Ongoing cortical patterns during binocular rivalry. NeuroImage, 23, 128– 140. Doesburg, S. M., Kitajo, K., & Ward, L. M. (2005). Increased gammaband synchrony precedes switching of conscious perceptual objects in binocular rivalry. NeuroReport, 16, 1139–1142. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., et al. (1988). Coherent oscillations: A mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biological Cybernetics, 60, 121–130. Edgington, E. S., & Onghena, P. (2007). Randomization Tests (4th ed). Boca Raton, FL: Chapman & Hall/CRC. Engel, A. K., Fries, P., Konig, P., Brecht, M., & Singer, W. (1999). Temporal binding, binocular rivalry, and consciousness. Consciousness and Cognition, 8, 128–151. Engel, A. K., Fries, P., & Singer, W. (2001). Dynamic predictions: Oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2, 704–716. Fries, P., Nikolic, D., & Singer, W. (2007). The gamma cycle. Trends in Neurosciences, 30, 309–316. Fries, P., Roelfsema, P. R., Engel, A. K., Konig, P., & Singer, W. (1997). Synchronization of oscillatory responses in visual cortex correlates
with perception in interocular rivalry. Proceedings of the National Academy of Sciences, USA, 94, 12699–12704. Fries, P., Schroder, J. H., Roelfsema, P. R., Singer, W., & Engel, A. K. (2002). Oscillatory neuronal synchronization in primary visual cortex as a correlate of stimulus selection. Journal of Neuroscience, 22, 3739–3754. Gail, A., Brinksmeyer, H. J., & Eckhorn, R. (2004). Perception-related modulations of local field potential power and coherence in primary visual cortex of awake monkey during binocular rivalry. Cerebral Cortex, 14, 300–313. Gray, C. M., Konig, P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338, 334–337. Herrmann, C. S., Munk, M. H., & Engel, A. K. (2004). Cognitive functions of gamma-band activity: Memory match and utilization. Trends in Cognitive Sciences, 8, 347–355. Hirabayashi, T., & Miyashita, Y. (2005). Dynamically modulated spike correlation in monkey inferior temporal cortex depending on the feature configuration within a whole object. Journal of Neuroscience, 25, 10299–10307. Hoogenboom, N., Schoffelen, J. M., Oostenveld, R., Parkes, L. M., & Fries, P. (2006). Localizing human visual gamma-band activity in frequency, time and space. NeuroImage, 29, 764–773. Ilg, R., Wohlschlager, A. M., Burazanis, S., Woller, A., Nunnemann, S., & Muhlau, M. (2008). Neural correlates of spontaneous percept switches in ambiguous stimuli: An event-related functional magnetic resonance imaging study. European Journal of Neuroscience, 28, 2325–2332. I˙Sog˘lu-Alkac¸, U¨., Basar-Eroglu, C., Ademoglu, A., Demiralp, T., Miener, M., & Stadler, T. (2000). Alpha activity decreases during the perception of Necker cube reversals: An application of wavelet transform. Biological Cybernetics, 82, 313–320. Keil, A., Muller, M. M., Ray, W. J., Gruber, T., & Elbert, T. (1999). Human gamma band activity and perception of a gestalt. Journal of Neuroscience, 19, 7152–7161. Kirchner, H., & Thorpe, S. J. (2006). Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Research, 46, 1762–1776. Klimesch, W., Freunberger, R., Sauseng, P., & Gruber, W. (2008). A short review of slow phase synchronization and memory: Evidence for
558 control processes in different memory systems? Brain Research, 1235, 31–44. Koch, S. P., Werner, P., Steinbrink, J., Fries, P., & Obrig, H. (2009). Stimulus-induced and state-dependent sustained gamma activity is tightly coupled to the hemodynamic response in humans, 29, 13962– 13970. Kornmeier, J., & Bach, M. (2004). Early neural activity in Necker-cube reversal: Evidence for low-level processing of a gestalt phenomenon. Psychophysiology, 41, 1–8. Kornmeier, J., & Bach, M. (2005). The Necker cubeFAn ambiguous figure disambiguated in early visual processing. Vision Research, 45, 955–960. Kornmeier, J., & Bach, M. (2006). Bistable perceptionFAlong the processing chain from ambiguous visual input to a stable percept. International Journal of Psychophysiology, 62, 345–349. Kornmeier, J., Heinrich, S. P., Atmanspacher, H., & Bach, M. (2001). The reversing ‘‘Necker Wall’’FA new paradigm with reversal entrainment reveals an early EEG correlate. Paper presented at the ARVO 2001 Annual Meeting, Fort Lauderdale, Florida, USA. Livingstone, M., & Hubel, D. (1988). Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science, 240, 740–749. Long, G. M., & Toppino, T. C. (2004). Enduring interest in perceptual ambiguity: Alternating views of reversible figures. Psychological Bulletin, 130, 748–768. Long, G. M., Toppino, T. C., & Mondin, G. W. (1992). Prime time: Fatigue and set effects in the perception of reversible figures. Perception & Psychophysics, 52, 609–616. Lumer, E. D., Friston, K. J., & Rees, G. (1998). Neural correlates of perceptual rivalry in the human brain. Science, 280, 1930–1934. Lutzenberger, W., Pulvermuller, F., Elbert, T., & Birbaumer, N. (1995). Visual stimulation alters local 40-Hz responses in humans: An EEG study. Neuroscience Letters, 183, 39–42. Mathes, B., Struber, D., Stadler, M. A., & Basar-Eroglu, C. (2006). Voluntary control of Necker cube reversals modulates the EEG deltaand gamma-band response. Neuroscience Letters, 402, 145–149. Milner, P. M. (1974). A model for visual shape recognition. Psychological Review, 81, 521–535. Muller, T. J., Federspiel, A., Fallgatter, A. J., & Strik, W. K. (1999). EEG signs of vigilance fluctuations preceding perceptual flips in multistable illusionary motion. NeuroReport, 10, 3423–3427. Muller, T. J., Koenig, T., Wackermann, J., Kalus, P., Fallgatter, A., Strik, W., et al. (2005). Subsecond changes of global brain state in illusory multistable motion perception. Journal of Neural Transmission, 112, 565–576. Necker, L. A. (1832). Observations on some remarkable optical phaenomena seen in Switzerland; and on an optical phaenomenon which occurs on viewing a figure of a crystal or geometrical solid. London and Edinburgh Philosophical Magazine and Journal of Science, 1, 329–337. Orbach, J., Ehrlich, D., & Heath, H. (1963). Reversibility of the Necker cube: I. An examination of the concept of ‘‘satiation of orientation’’. Perceptual and Motor Skills, 17, 439–458. Orbach, J., Zucker, E., & Olson, R. (1966). Reversibility of the Necker cube: VII. Reversal rate as a function of figure-on and figure-off durations. Perceptual and Motor Skills, 22, 615–618. Papoulis, A. (1962). The Fourier integral and its applications. New York: McGraw-Hill. Pfurtscheller, G., & Lopes da Silva, F. H. (1999). Event-related EEG/ MEG synchronization and desynchronization: Basic principles. Clinical Neurophysiology, 110, 1842–1857. Pitts, M. A., Nerger, J. L., & Davis, T. J. R. (2007). Electrophysiological correlates of perceptual reversals for three different types of multistable images. Journal of Vision, 7, 1–14. Revonsuo, A. (1999). Binding and the phenomenal unity of consciousness. Consciousness and Cognition, 8, 173–185. Roeber, U., & Veser, S. (2009). Depth of binocular-rivalry suppression reduces with time of suppression: Electrophysiological evidence. Perception, 38(suppl.), 154. Roeber, U., Widmann, A., Trujillo-Barreto, N. J., Herrmann, C. S., O’Shea, R. P., & Schro¨ger, E. (2008). Early correlates of visual
W. Ehm et al. awareness in the human brain: Time and place from event-related brain potentials. Journal of Vision, 8, 1–12. Rolls, E. T. (2000). Memory systems in the brain. Annual Review of Psychology, 51, 599–630. Roskies, A. L. (1999). The binding problem. Neuron, 247–9, 111–125. Schroeder, C. E., & Lakatos, P. (2009). The gamma oscillation: Master or slave? Brain Topography, 22, 24–26. Shadlen, M. N., & Movshon, J. A. (1999). Synchrony unbound: A critical evaluation of the temporal binding hypothesis. Neuron, 2467–77, 111–25. Sterzer, P., & Kleinschmidt, A. (2007). A neural basis for inference in perceptual ambiguity. Proceedings of the National Academy of Sciences, USA, 104, 323–328. Sterzer, P., & Rees, G. (2008). A neural basis for percept stabilization in binocular rivalry. Journal of Cognitive Neuroscience, 20, 389–399. Stru¨ber, D., Basar-Eroglu, C., Hoff, E., & Stadler, M. (2000). Reversalrate dependent differences in the EEG gamma-band during multistable visual perception. International Journal of Psychophysiology, 38, 243–252. Stru¨ber, D., Basar-Eroglu, C., Miener, M., & Stadler, M. (2001). EEG gamma-band response during the perception of Necker cube reversals. Visual Cognition, 8, 609–621. Stru¨ber, D., & Herrmann, C. S. (2002). MEG alpha activity decrease reflects destabilization of multistable percepts. Cognitive Brain Research, 14, 370–382. Tallon-Baudry, C. (2009). The roles of gamma-band oscillatory synchrony in human visual cognition. Frontiers in Bioscience, 14, 321–332. Tallon-Baudry, C., & Bertrand, O. (1999). Oscillatory gamma activity in humans and its role in object representation. Trends in Cognitive Science, 3, 151–162. Tallon-Baudry, C., Bertrand, O., Henaff, M. A., Isnard, J., & Fischer, C. (2005). Attention modulates gamma-band oscillations differently in the human lateral occipital cortex and fusiform gyrus. Cerebral Cortex, 15, 654–662. Treisman, A., & Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review, 95, 15–48. Uhlhaas, P. J., Pipa, G., Lima, B., Melloni, L., Neuenschwander, S., Nikolic, D., et al. (2009). Neural synchrony in cortical networks: History, concept and current status. Frontiers in Integrative Neuroscience, 3, 17. VanRullen, R., Reddy, L., & Koch, C. (2006). The continuous wagon wheel illusion is associated with changes in electroencephalogram power at approximately 13 Hz. Journal of Neuroscience, 26, 502–507. Varela, F., Lachaux, J. P., Rodriguez, E., & Martinerie, J. (2001). The brainweb: Phase synchronization and large-scale integration. Nature Reviews Neuroscience, 2, 229–239. von der Malsburg, C. (1981). The correlation theory of brain function. Internal Report No. 81-2. Go¨ttingen, Germany: Max-PlanckInstitut fu¨r biophysikalische Chemie. von Schiller, P. (1933). Stroboskopische Alternativversuche. Psychologische Forschung, 17, 179–214. Wertheimer, M. (1912). Experimentelle Studien u¨ber das Sehen von Bewegungen. Zeitschrift fu¨r Psychologie, 61, 161–265. Woo, M., Neider, J., & Davis, T. (1998). OpenGL Programming Guide. The Official Guide to learning OpenGL, Version 1.1 (2nd ed). Reading, MA: Addison-Wesley. World Medical Association. (2000). Declaration of Helsinki: Ethical principles for medical research involving human subjects. Journal of the American Medical Association, 284, 3043–3045. Yuval-Greenberg, S., & Deouell, L. Y. (2009). The broadband-transient induced gamma-band response in scalp EEG reflects the execution of saccades. Brain Topography, 22, 3–6. Yuval-Greenberg, S., Tomer, O., Keren, A. S., Nelken, I., & Deouell, L. Y. (2008). Transient induced gamma-band response in EEG as a manifestation of miniature saccades. Neuron, 58, 429–441.
(Received January 20, 2010; Accepted June 11, 2010)
Psychophysiology, 48 (2011), 559–568. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01115.x
Switch-specific and general preparation map onto different ERP components in a task-switching paradigm
FRINI KARAYANIDIS, ALEXANDER PROVOST, SCOTT BROWN, BRYAN PATON, and ANDREW HEATHCOTE Functional Neuroimaging Laboratory, School of Psychology and Centre for Brain and Mental Health Research, University of Newcastle, Callaghan, Australia
Abstract We examined whether the cue-locked centroparietal positivity is associated with switch-specific or general preparation processes. If this positivity (300–400 ms) indexes switch-specific preparation, faster switch trials associated with smaller RT switch cost should have a larger positivity as compared to slower switch trials, but no such association should be evident for repeat trials. We extracted ERP waveforms corresponding to semi-deciles of each participant’s RT distribution (i.e., fastest to slowest 5% of trials) for switch and repeat conditions. Consistent with a switch-specific preparation process, centroparietal positivity amplitude was linked to slower RT and larger RT switch cost for switch but not repeat trials. A later pre-target negativity (500–600 ms) was inversely correlated with RT for both switch and repeat trials, consistent with a general anticipatory preparation processes. Descriptors: Advance preparation, Task-switching, RT distribution, Single trial ERP
cesses. Even at long CSIs and RSIs, a residual RT switch cost remains, suggesting either that switch-specific preparation cannot be completed before stimulus onset (Rogers & Monsell, 1995) or that the stimulus triggers processes that interfere with applying the recently activated task-set (Meiran, 2000). In cued trials paradigms, event-related potential (ERP) waveforms time-locked to cue onset consistently show a large centroparietal positive shift for switch as compared with repeat trials (e.g., Kieffaber & Hetrick, 2005; Miniuissi, Marzi, & Nobre, 2005; Nicholson, Karayanidis, Poboka, Heathcote, & Michie, 2005; Nicholson, Karayanidis, Bumak, Poboka, & Michie, 2006; Rushworth, Passingham, & Nobre, 2005). This cue-locked differential switch-positivity emerges as early as 200 ms postcue. With long CSIs, this differential switch-positivity peaks and often dissipates prior to stimulus onset whereas, with short CSIs, it peaks after stimulus onset. Recent studies have found that the amplitude of the switch-positivity is related to behavioral indices of preparation (Kieffaber & Hetrick, 2005; Lavric, Mizon, & Monsell, 2008; but see Swainson, Jackson, & Jackson, 2006). Using temporal principal components analysis (PCA), Lavric et al. (2008) extracted a late slow parietal-positivity/anterior negativity component that maps onto the switch-positivity. This component was negatively correlated with a performance index of switch cost, supporting the argument that the switch-positivity is associated with preparation to switch task. Based on the assumption that fast RT responses are associated with greater preparation than slow RT responses (de Jong, 2000; Nieuwenhuis & Monsell, 2002), Lavric et al. (2008) averaged cue-locked ERPs separately for the fastest and the slowest third (terciles) of RT responses for each individual. A significant differential
Task-switching paradigms require shifting between two or more tasks using an alternating task sequence (i.e., alternating runs) or random sequences of task or transition cues (i.e., cued trials). Studies have consistently shown that response time (RT) is longer on trials that require a switch as compared to a repeat in task (e.g., Allport, Styles, & Hsieh, 1994; Rogers & Monsell, 1995). In cued trials paradigms, this RTswitch cost reduces as the interval between an informative cue and stimulus onset (cuestimulus interval: CSI) increases (e.g., Meiran, 2000)Fsuggesting the activation of control processes associated with task-set reconfiguration, e.g., activation of task goal and category-response rules relevant to the cued task (Meiran, 2000; Rogers & Monsell, 1995; Rubinstein, Meyer, & Evans, 2001) or activation of a task mediator cue (Logan & Schneider, 2006). We will use the term switch-specific preparation to refer to these processes. For a fixed CSI, RT switch cost also reduces with increasing response-stimulus interval (RSI), which has been attributed to passive processes associated with dissipation of the previously active task-set (Allport et al., 1994). Cued trials paradigms allow investigation of switch-specific processes by manipulating CSI, with fixed values of RSI controlling for passive dissipation proWe thank Rebecca Nicholson for access to the original data, Damien Mannion and Tony Kemp for programming support, Ross Fulham for access to EEGDisplay software, and Pat Michie for discussions related to this work. This work was supported by funding from the University of Newcastle Research Grants Committee and approved by the University of Newcastle Human Research Ethics Committee. Address correspondence to: Dr. Frini Karayanidis, School of Psychology, University of Newcastle, NSW 2308 Australia. E-mail:
[email protected] 559
560 switch-positivity was evident for fast but not slow RT terciles, supporting a relationship between the amplitude of cue-locked switch positivity and switch-specific preparation for an impending switch trial. Goffaux, Phillips, Sinai, and Pushkar (2006) showed that the cue-locked posterior positivity was larger for switch than repeat trials equated for RT, indicating that it likely reflects differences in preparation for the upcoming trial, rather than generalized speed differences. There is increasing evidence that a number of preparationrelated ERP components are superimposed within the cue-target interval. In Kieffaber and Hetrick (2005), spatiotemporal PCA resulted in three components within the latency range of the switch-positivity mapped to different aspects of preparation. Partially informative cues indicating a switch away from the current task but not specifying the identity of the upcoming task elicited an early cue-locked differential switch-positivity, whereas fully informative cues also elicited a later switch-positivity within the CSI (Karayanidis, Mansfield, Galloway, Smith, Provost, & Heathcote, 2009; Nicholson, Karayanidis, Davies & Michie, 2006). In cue-locked waveforms, a positivity can often be seen for both switch and repeat trials, although it is larger for switch trials (Goffaux et al., 2006; Kieffaber & Hetrick, 2005; Nicholson et al., 2005). Informative cues that validly signal either a switch or a repeat in task elicit an early posterior positivity relative to noninformative cues (that signal an upcoming target, but not the task), whereas informative switch cues also showed a later posterior positivity relative to informative repeat cues (Jamadar, Michie, & Karayanidis, 2010). Lavric et al. (2008) reported PCA components that temporally dissociated an earlier P3b from a later switch-specific positivity. A late frontal negative component has also been identified, more frequently with a common average reference (e.g., Astle, Jackson, & Swainson, 2008; Mueller, Swainson, & Jackson, 2007, 2009), that appears to be associated with response set preparation. In addition, with CSI of around 600–1000 ms, cuelocked ERP waveforms often show a pre-target negativity emerging for both switch and repeat trials, although it is usually larger for repeat trials (e.g., Nicholson et al., 2005). This pretarget negativity is likely to reflect anticipatory attention (i.e., stimulus-preceding negativity, SPN) and/or response readiness (i.e., contingent negative variation, CNV), which may also vary across the RT distribution and possibly affect RT switch cost (see Karayanidis, Jamadar, Ruge, Phillips, Heathcote, & Forstmann, 2010, for a recent review). At a behavioral level, there is increasing evidence that RT switch cost does not always reflect an increase in switch trial RT relative to a constant repeat trial RT baseline, but may also reflect variation in repeat trial RT. For example, in a study of sequence effects in task-switching, Koch and Philipp (2005) found that RT switch cost was eliminated on trials preceded by a no-go trial. However, this effect was largely due to an increase in repeat trial RT rather than a reduction in switch trial RT. Jamadar, Michie, and Karayanidis (2010) demonstrated that this effect is due to reduced response readiness on the repeat trials. Altmann (2004) posits a general preparatory process that reduces RT for both switch and repeat trials as CSI increases. This anticipatory preparation process is argued to reflect a primary process of memory retention that is strongest for repeat trials and maintains the previously active task-set. Altmann argues that any preparation for switch trials may or may not be subsumed under this general preparation process. A central aim of our study is to examine preparation effects separately on both switch
F. Karayanidis et al. and repeat trials. In particular, we examine whether variation over the range of the RT distribution affects the amplitude of the cue-locked positivity and the pre-target negativity for both switch and repeat trials.
Variation in ERP Components with RT ERP component amplitude varies in the range of 1–20 mV, whereas EEG ranges from ! 100 to 1100 mV (Regan, 1989; Handy, 2004). The most common method to extract ERPs involves time-locked signal averaging over many trials, so that large EEG ‘‘noise’’ fluctuations average to zero and the embedded ERP signal is extracted. Depending on the nature of the ERP component, the number of trials needed to obtain a satisfactory signal-to-noise ratio can vary considerably. Long-latency components associated with higher order cognitive processes, like the differential switch-positivity, often manifest as small variations between experimental conditions. These difference waveforms tend to produce relatively small ERP components (approx. 1–5 mV) with comparatively low signal-to-noise ratio, and so require averaging over a large number of experimental trials. Signal averaging relies on the assumption that the ERP signal remains constant across repeated presentations of the stimulus. However, there is ample evidence that this assumption is not always valid. Successive presentations of the stimulus may result in systematic changes in the ERP component over the course of the testing session, and in some instances can result in important component information being ‘‘averaged out’’ with noise (Spencer, 2004). In the context of most research questions addressed by ERP studies, violation of this assumption does not have a critical impact on the validity of the conclusions. In other contexts, however, variation in behavior and ERP components over time is central to the research focus (e.g., Karayanidis, Robaey, Bourassa, de Koning, Geoffroy, & Pelletier, 2000; Lorist, Klein, Nieuwenhuis, de Jong, Mulder, & Meijman, 2000; Shelley et al., 1991; Woestenburg, Verbaten, Van Hees, & Slangen, 1983). There is emerging evidence that switch-related preparation varies as a function of RT. As mentioned earlier, Lavric et al. (2008) found that ERPs derived by averaging over trials with broadly similar RTs (i.e., the fastest and slowest thirds of the RT distribution) differ in the morphology of cue-locked components associated with advance preparation. Here we focus on the variation of ERP components with trial-to-trial variations in RT. We took a different approach to Lavric’s, using a modified version of Woestenburg et al.’s (1983) orthogonal polynomial trend analysis (OPTA) technique (see the Methods section for details). In this approach, RTs for each trial are used as a covariate in a regression model of the observed EEG signals. The regression model can then be used to examine ERP components for any single observed RT value, enabling a fine-grained analysis of component changes with RT. The signal-to-noise issue is addressed because all trails contribute to the regression model. As changes in RT switch cost over the RT distribution (de Jong, 2000; Poboka, Heathcote, Karayanidis, & Nicholson, 2005) likely reflect, at least in part, variation in anticipatory preparation, the amplitude of the differential switch-positivity was expected to vary systematically across the RT distribution. We applied the OPTA analysis to data previously examined with conventional ERP averaging by Nicholson et al. (2005) and with RT distributional analyses by Poboka et al. (2005). In the current paper, we report results only from two conditions with a 600 ms CSI, as these conditions showed a substantial reduction in
Switch-specific and general preparation in cued task-switching switch cost relative to a short CSI condition (150 ms) and a plateau in residual switch cost relative to a longer CSI condition (1050 ms). Analyses were targeted to two ERP components evident in cue-locked waveforms: the centroparietal positivity over 300–400 ms and the frontocentral pre-target negativity over 500– 600 ms, both of which contribute to the differential switch-positivity. Variation in RT as a result of general preparation following cue onset would be expected to be evident on both switch and repeat trial ERPs, whereas variation associated with switch-specific preparation would be expected to occur for switch trials only. Based on evidence that the cue-locked positivity is related to processes associated with switch-specific preparation, faster RT trials that are associated with smaller RT switch cost were expected to show larger cue-locked positivity than slower RT trials after switch but not repeat cues. As the pre-target negativity is likely to be associated with general preparation processes such as anticipatory attention and response readiness, faster RT trials were expected to be associated with larger pre-target negativity than slow RT trials after both repeat and switch cues. Method Participants Twenty-four students from the University of Newcastle (18 to 30 years of age, mean 5 22.2 years, 15 female) enrolled in an introductory psychology course completed this study for course credit. Paradigm A rectangular box was divided into four equal quadrants and continuously displayed on a monitor at a viewing distance of 90 cm.1 Participants alternated randomly between two tasks. The letter classification task involved classifying a letter as consonant (G, K, M, R) or vowel (A, E, I, U). The number classification task involved classifying the digit as odd (3, 5, 7, 9) or even (2, 4, 6, 8). Responses were mapped to the left and right hand and response hand mapping was counterbalanced across participants. The letter task was assigned to the top two quadrants for half of the participants and to the right two quadrants for the other half in order to counterbalance the mapping between eye shift (vertical or horizontal) and trial type (switch or repeat). Stimuli were pairs of Times New Roman font characters. One of the characters was selected from the currently active task set (e.g., letters for the letter task). On one third of trials, the second character was a non-alphanumeric character (i.e., #, %, ?, n) that was not mapped to any response (neutral pairing). On the remaining trials, the task irrelevant character was selected from the other task (i.e., number for letter task) with half of these trials involving a congruent pairing (relevant and irrelevant characters mapped to the same response) and the other half involving an incongruent pairing (mapped to opposite responses). Character position (e.g., E#, #E) varied randomly across trials. Each trial began with a cue indicating the quadrant in which the next stimulus would be presented. The cue was a highlight of the line defining that quadrant (from gray to white) and remained on throughout the duration of the stimulus presentation. The stimulus remained on screen until a response was emitted or 5000 ms elapsed. The next cue appeared in one of the two adjacent quadrants (horizontal or vertical), and its position defined a switch or repeat trial. The first four trials of every block, incorrect response trials, the trial following an incorrect response, and any trial with 1
See Nicholson et al. (2005) for full details on the paradigm.
561 a response outside 200–2000 ms post-stimulus were not included in data analysis. RSI and CSI were manipulated across different blocks resulting in a total of six timing conditions presented in counterbalanced order. Only two timing conditions with CSI of 600 ms (blocked RSI of 750 ms and 1200 ms) will be presented in this paper (3 ! 100 trials per RSI). These two timing conditions were selected because they resulted in a large clear switch positivity that fully resolved within the CSI (Nicholson et al., 2005). EEG Recording and Data Analysis EEG was continuously sampled from 12 scalp electrodes according to the 10/20 system (Fz, Cz, Pz, F3, C3, P3, T5, F4, C4, P4, T6, Oz) with linked mastoids reference (500 Hz/channel; NeuroScan Acquire, Compumedics, Ltd., Abbotsford, Australia; Grass Neurodata system (Model 12), Grass Technologies, West Warwick, RI; bandpass: 0.01–30 Hz, " 6 dB down). Vertical and horizontal electrooculogram was recorded bipolarly from electrodes attached to the supraorbital and infraorbital ridges of the left eye and the outer canthi of each eye, respectively. Electroencephalogram (EEG) was corrected for eyeblink artifact (Semlitsch, Anderer, Schuster, & Presslich, 1986), and sections with movement artifact or channel saturation were excluded from further analyses. Cue-locked EEG epochs for trials associated with a correct response were extracted over an interval spanning 200 ms before and 1200 ms after cue onset (baseline # 50 ms). OPTA Analysis In order to examine variation in differential switch-positivity as a function of changes in RT switch cost over the RT distribution, we applied a revised version of the OPTA technique developed by Woestenburg et al. (1983). OPTA uses a polynomial regression model of EEG components in the frequency domain to estimate how that component changes with a covariate. The process involved transforming EEG data into the frequency domain and applying an orthogonal polynomial regression equation, using RT as the only covariate, to each component of the frequency profile. Polynomial coefficients up to and including the 5th order term were used in component estimation only when the regression analysis suggested that they explained a significant proportion of variance. Orthogonal polynomial trend analysis has been used previously to model a variety of time-varying neurophysiological phenomena (de Koning, Woestenburg & Elton, 2001; Karayanidis et al., 2000; Kenemans, Verbaten, Melis, & Slangen, 1992; van der Lubbe & Woestenburg, 1997, 1999, 2000; Woestenburg et al., 1983). For example, in a habituation study, Woestenburg et al. used OPTA to examine changes in ERPs as a function of trial. OPTA allows the estimation of ERP components on an individual trial-by-trial level, whereas conventional techniques require averaging over a large number of trials to improve signalto-noise ratio. Woestenburg et al. showed that OPTA can achieve about a tenfold improvement in the signal-to-noise ratio when ERP components change with some covariate, and that OPTA outperforms another quite sophisticated averaging technique (Wiener filtering). The OPTA procedure used here differs from the original algorithm implemented by Woestenburg et al. in that it uses RT as a covariate. We applied OPTA on epoched EEG data using software written in MATLAB (The MathWorks, Inc., Natick, MA). For each participant, RSI condition, and trial type, cue-locked epochs associated with correct responses were ranked based on RT: for example, the fastest RT recorded by the
562 participant for repeat trials in a particular RSI condition was assigned the covariate value ‘‘1,’’ the next fastest ‘‘2,’’ and so on. These covariates correspond to the order statistics of RT samples.2 The EEG epochs were then transformed using a fast Fourier transform (FFT). A polynomial regression equationF similar to a trend analysis in a conventional ANOVAFwas applied at each frequency, using the order statistics as the covariate and polynomial terms up to the fifth order. The proportion of analyses on which the highest-order polynomial coefficient (5th) explained significant variance was smaller than the Type I error rate in both conditions (4.2% for repeat, 3.7% for switch) indicating that analysis of even higher-order terms (e.g., 6th, 7th, etc.) was not necessary. After removing terms that failed to explain significant variance, the fitted polynomial functions were used to generate predicted waveforms in the frequency domain, corresponding to each trial. This was achieved by using the polynomial equation at each frequency to generate predicted power values for order statistics corresponding to the RT value of each trial for each participant. Finally, these RT-ranked predicted waveforms were transformed back into the time domain using the inverse FFT. This resulted in an RT ranked waveform for each trial for each participant. For each trial type, group average waveforms corresponding to 5th, 15th, etc., up to the 95th percentile of the RT distribution were created by averaging across participants, and these were used to depict the variation in cue-locked ERP waveforms over the RT distribution. For statistical analyses, RT ranked single-trial waveforms were averaged within each of the 20 semi-decile intervals for each participant (4–7 trials per semi-decile). This procedure resulted in 20 cue-locked waveforms per individual for each condition and trial type, which represent the averaged waveform corresponding to RT percentile values incrementing by 5%, starting at the 2.5th percentile. Data Analysis The mean RT for each semi-decile interval was analyzed using a 2 RSI condition (750, 1200 ms) ! 2 trial type (repeat, switch) ! 20 RT intervals repeated measures generalized linear model (GLM) in SPSS. Since we were specifically interested in examining variation in cue-locked waveforms as a function of RT, we targeted analyses at frontal, central, and parietal sites over the midline and right scalp, as initial visual inspection of the averaged data indicated that these locations showed the largest effects of decile. Based on the conventionally averaged data presented by Nicholson et al. (2005), we defined two mean amplitude windows to target the cue-locked positivity (300–400 ms) and the pre-target negativity (500–600 ms). ERPs were analyzed using a 2 RSI ! 2 trial type ! 20 RT intervals ! 3 coronal (frontal, central, parietal) ! 2 lateral (midline, right) repeated measures analyses at each of the two mean amplitude windows. Planned linear and quadratic polynomial comparisons are reported in all analyses of the RT interval factor. For both behavioral and ERP analyses, degrees of freedom for factors with more than two levels were adjusted using Greenhouse–Geisser correction for the violation of the assumption of sphericity (Vasey & Thayer, 1987). 2 We also conducted the analyses using RT magnitude rather than order statistics as the covariate. This analysis required a further modification of the original OPTA analysis, as it necessitated using orthogonal polynomial coefficients for unequally spaced covariates. The conclusions obtained with both types of covariate were the same.
F. Karayanidis et al. Results Figure 1 shows RT and mean amplitude measures from cuelocked waveforms for switch and repeat trials at each RT percentile interval. Note that each value represents the mean across a 5% range, so that the first value corresponds to the mean value under 5% (i.e., an average corresponding to the mid-point of the rank, 2.5th percentile), the second to the mean value between 5– 10%, etc. As shown in Figure 1 (left), RT was significantly faster for repeat than switch trials (F(1,23) 5 60.3, po.001). RT increased significantly across percentiles (F(19,437) 5 328, po.001) with significant linear (F(1,23) 5 340, po.001) and quadratic (F(1,23) 5 254, po.001) trends. More importantly, RT switch cost was affected by RT percentile rank (F(19,437) 5 36.8, po.001) increasing linearly from 26 ms at the first percentile interval to 157 ms at the last percentile interval (F(1,23) 5 96.2, po.001). Residual switch cost was significant even for the fastest 5% of RT responses (F(1,23) 5 41.6, po.001). There was no significant interaction between RSI condition and RT rank. Figure 2A shows cue-locked ERP difference waveforms derived by subtracting the repeat trial from the switch trial ERP waveform for corresponding percentile values. Overall, these difference waveforms show a typical centroparietally maximal differential switch positivity, emerging earliest centrally and extending beyond stimulus onset parietally. The variation in the amplitude of the switch-positivity across the RT distribution is striking, with a much larger switch-positivity for fast as compared to slow RT trials. Note that fast RT trials are associated with a prolonged centroparietal switch-positivity throughout most of the CSI, whereas slow RT trials show little differentiation between switch and repeat trials beyond 450 ms. In order to examine whether these effects are due to RT-related variation on switch and/or repeat trial ERPs, Figure 2B depicts ERP waveforms for repeat and switch trials separately. Both switch and repeat trial ERPs showed a centroparietally maximal positivity over 200–400 ms. This positivity was much larger for switch trials, especially over 300–400 ms, reflecting the differential switch positivity seen in many previous task-switching studies. This was followed by a fronto-central pre-target negativity emerging around 200 ms before target onset that tended to be larger for repeat trials. Two mean amplitude windows targeted the parietal positivity (300–400 ms) and the pre-target negativity (500–600 ms) where the effects of decile were most pronounced. Although both ERP mean amplitude windows showed a significant effect of RSI condition (300–400: F(1,23) 5 18.9, po.001; 500–600: F(1,23) 5 8.0, po.009), reflecting a greater positive drift in the entire waveform for the long as compared to short RSI, this factor did not interact with RT percentile rank in either 300–400 ms or 500–600 ms windows. Switch trials had more positive values than repeat trials in both epochs (Figure 2A), reflecting a larger centroparietal positivity over 300–400 ms followed by a smaller pre-target negativity over 500–600 ms (300–400: F(1,23) 5 41, po.001; 500– 600: F(1,23) 5 6.5, po.018). The OPTA procedure resulted in ERP waveforms that varied quite systematically from fast to slow RT deciles with little variation in the interval preceding cue onset. Note that the effect of RT percentile on the amplitude of the cue-locked waveforms varied across the CSI and for switch and repeat cues (Figure 2B). For repeat cues, the effect of RT variation was restricted to the amplitude of the stimulus-preceding negativity with faster RT percentile intervals showing larger
Switch-specific and general preparation in cued task-switching
563 Mean Amplitude
0
1400 1200 800
µV
msec
1000 600 400
300–400 Parietal
–8
2
–6
4
–4
µV
Reaction Time
6
–2
8
0
500–600 Central
200 0 RT Percentile
RT Percentile Repeat
RT Percentile
Switch
Figure 1. A. Mean RTat each of the 20 percentile ranks used in the analyses. B. Mean amplitude of the positivity averaged over midline and right parietal sites (300–400 ms) and pre-target negativity averaged over midline and right central sites (500–600 ms) across the range of the RT percentile ranks.
negativity. Switch cues showed a similar, though smaller, effect of RT variation on the stimulus-preceding negativity. In addition, switch cues showed the opposite effect on the amplitude of the centroparietal positivity with faster RT percentiles showing a larger positivity. While the RT percentile factor resulted in significant main effects and/or interactions with trial in both epochs, these did not reflect a uniform effect of percentile across both trial types, but rather a selective pattern of effects for repeat and switch trials on different ERP components. Figure 1 (right) shows mean amplitude of repeat and switch ERPs at each RT percentile for each mean amplitude window. Centroparietal Positivity (300–400 ms) The centroparietal positivity (300–400 ms) produced a significant RT interval ! trial type interaction (F(19,437) 5 10.1, po.001). Repeat trials showed a small and sustained positivity of around 4 mV parietally that did not vary significantly with increasing RT percentile rank (Figure 2B; Fo1). In contrast, switch trials showed a larger positivity than repeat trials (" 7.5 mV parietally) and the amplitude of this positivity varied with RT percentile rank. This positivity varied as a function of RT interval (F(19,437) 5 10.1, po.001) showing a significant linear decline with increasing RT (F(1,23) 5 14.2, po.001). So, fast switch responses that were associated with small RT switch cost (Figure 1 left) showed a larger overall positivity and a larger differential switch positivity as compared to slow switch responses that were associated with large RT switch cost. Note that, although the cue-locked positivity difference between switch and repeat trials reduced with increasing RT, the positivity was still significantly larger for switch as compared to repeat trials even at the slowest 5% of the RT distribution (i.e., 95th percentile; F(1,23) 5 20, po.001; see Figure 2C). We examined inter-individual variability in RT and cuelocked positivity by analyzing correlations between mean RTand mean amplitude of the cue-locked positivity over 300–400 ms averaged over RT percentile interval and RSI condition using one-tailed Pearson correlations. RT switch cost showed a very high and significant correlation with switch trial RT (r 5 .821, po.001) and a smaller correlation with repeat trial RT (r 5 .456, po.013), indicating that individual variation in RT switch cost was more strongly related to differences in switch rather than repeat trial RT. As expected, there was no significant correlation between repeat trial RTand cue-locked positivity for repeat trials
(p4.17). For switch trials, mean RT was negatively correlated with cue-locked positivity amplitude over the right parietal scalp (P4: r 5 # .405, po.025), as well as with the amplitude of the differential switch positivity over central and parietal scalp (Pz, C4, P4: # .409oro # .480, p 5 .024 to .009). These findings indicate that individuals with faster switch trial RT tended to elicit a larger positivity for switch trials and a larger switch-repeat differential positivity as well as a smaller RT switch cost. Together with the semi-decile analyses above, these results show effects of both intra- and inter-individual variability in cue-locked positivity affect performance, such that faster RT was associated with smaller RT switch cost, as well as with larger cue-locked differential positivity for switch as compared to repeat trials. Pre-target Negativity (500–600 ms) The effect of RT percentile rank was even more pronounced on the pre-target negativity, which showed a significant linear reduction with increasing RT (F(19,437) 5 22.3, po.001; linear contrast: F(1,23) 5 27.47, po.001). The effect of percentile was more pronounced centrally (F(38,874) 5 9.8, po.001) and over the midline (F(19,437) 5 9.3, po.001). Although the reduction in negativity with increasing RT percentile was evident for both switch and repeat trials, the effect was more pronounced in the latter (Figure 1; interval ! type: F(19,437) 5 3.9, po.035; interval ! type ! coronal: F(38,874) 5 4.5 po.008). Overall, repeat trials reduced by 1.8 mV and switch trials by 1.2 mV from the fastest to the slowest RT percentile. Repeat trials showed a linear reduction in negativity with increasing RT percentile (F(1,23) 5 34.9, po.001), whereas switch trials showed both linear and quadratic trend (F(1,23) 5 8.9, po.007; F(1,23) 5 8.4, po.008) reflecting a plateau during the second half of the RT distribution. So, for both repeat and switch trials, a larger pretarget negativity was associated with a faster response to the upcoming target. The pre-target negativity amplitude difference between switch and repeat trials reduced across the RT distribution (Figure 1, right). At central sites, where the pre-target negativity was largest, switch and repeat trials did not differ significantly beyond the 75th RT percentile (p4.10). We examined inter-individual variability in RTand pre-target negativity by analyzing correlations between mean RTand mean amplitude over 500–600 ms averaged over RT percentile interval and RSI condition using one-tailed Pearson correlations. Pre-
564
F. Karayanidis et al.
A
B
(Switch - Repeat) Cue –5 µV
Cue
Stimulus
Repeat
Switch Stimulus
–5µV Fz
5% 15% 25% 35% 45% 55% 65% 75% 85% 95%
Cz
Pz
0
200 400 600 800
Fz
Cz
Pz
0
C
5th vs 95th Percentile Cue Stimulus –5 µV
200 400 600 800
200 400 600 800
Switch and Repeat Equated for RT
D
Cue Fast Switch Slow Switch Fast Repeat Slow Repeat
Stimulus –5 µV
Fz
Fz
Cz
Cz
Pz
Pz
0
0
200 400 600 800
0
Switch Repeat
200 400 600 800
Figure 2. Cue-locked average waveforms for switch and repeat trials are presented here for ten trials that represent RT percentile ranks in 10% increments starting at the 5th percentile. (A) Difference waveforms derived by subtracting repeat trial from switch trial waveform at each RT percentile interval. (B) ERP waveforms for repeat and switch trials. (C) ERP waveforms superimposed for the 5th and the 95th RT percentile. (D) ERP waveforms superimposed for repeat and switch trials matched on RT ("550–560 ms).
target negativity amplitude was primarily related to RT for repeat trials. Specifically, on repeat trials, a large pre-target negativity and a large switch-repeat difference were associated with faster RT. These correlations were stronger centroparietally (repeat negativity and repeat RT: r 5 .364–.403, p 5 .04–.026; switch-repeat difference and repeat RT over right scalp: r 5 ! .437–.461, p 5 .012–.016). So, inter-individual variability in pre-target negativity was associated with variability on repeat trial RT. Analyses of Cue-Locked Waveforms for Switch and Repeat Trials with Equivalent RT Task switching paradigms produce a significant RT switch cost in most experimental conditions. So ERP waveforms for switch and repeat trials differ not only in trial type but also in mean RT.
Therefore, cue-locked ERP differences between switch and repeat trials could be confounded by non-specific factors affecting overall RT rather than processes specifically associated with anticipatory preparation for an impending change in task-set. The OPTA technique provides a unique capability to compare cuelocked ERP waveforms for switch and repeat trials equated for RT within a very narrow band (see Goffaux et al., 2006, for a related approach). Within each condition, we identified RT percentile ranks around the middle of the distribution that had very similar mean RT for switch and repeat trials. Specifically, repeat trial RT from 50th and 55th percentile from short and long RSI conditions, respectively, was closely matched with switch trial RT from 35th percentile in both RSI conditions (short RSI: 556 vs. 559 ms; long RSI: 551 vs. 556 ms, for repeat and switch, respectively; trial type and RSI condition effects Fo1). These
Switch-specific and general preparation in cued task-switching
565
data were entered into a 2 condition ! 2 trial type ! 3 coronal ! 2 lateral repeated measures GLM for each mean amplitude window. Over 300–400 ms, the main effect of type was highly significant (F(1,23) 5 36.6, po.001), indicating a larger cuelocked positivity for switch than repeat trials matched for mean RT (Figure 2D). This difference between switch and repeat trials showed the same centroparietal midline pattern seen when averaging across all trials (type ! coronal ! lateral: F(2,46) 5 9.2, po.001). In contrast, the pre-target negativity (500–600 ms) showed no main effect of trial type (Fo1) and a significant trial ! coronal interaction (F(2,46) 5 16.9, po.001), resulting largely from the slower resolution of the parietal positivity for switch trials. Comparing OPTA Findings Against Conventional Averaging A full methodological investigation of the benefit of the OPTA procedure applied in this study as compared to conventional averaging and other methods of small-trial ERP analyses is beyond the scope of the current paper (Provost, Brown, Heathcote, Karayanidis, in preparation). For the aims of the current study, we examined whether we would be able to obtain the same findings with more conventional signal averaging. For this purpose, we processed the ERP epoched data in an identical manner, but rather than applying the OPTA technique, we simply averaged over the ERP epochs within each of the semi-deciles. As expected, these conventional averages showed much greater variability than the OPTA-derived waveforms (Figure 3). This was confirmed by signal-to-noise measures, which showed that mean amplitude in the 300–400 ms window derived from OPTA waveforms were associated with 2.5 times greater signal-noise ratio than conventional averaging. In particular, a test of all 480 conditions gave an F(479,11017) 5 20.7 for OPTA vs. 8.4 for conventional averaging. When we ran the mean amplitude analyses on the conventional averages over the 20 semi-deciles, the pattern of results was largely the same as for the OPTA analysis, although the associated F-values were smaller and sometimes failed to reach statistical significance. Discussion In this paper we applied an OPTA analysis (Woestenburg et al., 1983), using individual trial RTs as a covariate, to examine cuelocked ERP waveforms in a task-switching paradigm. Our aim was to determine whether variability in anticipatory preparation as indexed by variation in RT switch cost across the RT distri-
bution is associated with variation in the cue-locked waveforms exclusively for switch trials or for both switch and repeat trials. At a relatively long preparation interval of 600 ms, increasing RT was associated with larger RT switch cost (de Jong, 2000; Nieuwenhuis & Monsell, 2002), with as much as a 5-fold increase in RT switch cost from the fastest to the slowest RT semi-deciles. We used the OPTA technique to derive ERP waveforms for twenty RT percentile rank values spread across the entire RT distribution. ERP waveforms showed a stable profile across RT percentile rank with a much higher signal-to-noise ratio than could be achieved by conventional averaging. Variability associated with RT percentile very selectively affected specific components of the ERP waveform differentially for switch and repeat trials. A cue-locked centroparietal positivity emerged around 200 ms, peaked around 350 ms and was larger for switch than repeat trials (Nicholson et al., 2005). OPTA analyses showed that, for repeat trials, this positivity did not vary in amplitude across the range of the RT distribution. However, for switch trials, the cuelocked positivity was significantly larger for fast as compared to slow RT trials. Therefore, the fastest end of the RT distribution, which showed the smallest RT switch cost and the most prepared responses, was associated with the largest centroparietal positivity for switch trials. The slowest end of the RT distribution, which displayed the largest RT switch cost and represented the least prepared switch responses, was associated with the smallest positivity for switch trials. As there was no significant variation in positivity for repeat trials over this latency range, the reduction in differential switch-positivity with increasing RT switch cost can be attributed to the variation in switch trial positivity. This is consistent with the finding that inter-individual variability in switch trial, but not repeat trial, RT was negatively correlated with the amplitude of the cue-locked positivity for switch trials and with the amplitude of the differential switch-positivity. Despite a progressive reduction in differential-switch positivity over the RT range, the cue-locked difference between switch and repeat trials for the slowest 5% of responses was still significant. In addition, significant differences in cue-locked ERPs between switch and repeat trials remained even when the RT difference that is normally obtained between these trials was eliminated (see also Goffaux et al., 2006). Taken together, the findings that (a) the most prepared responses show the largest switch-repeat differentiation (fastest 5%), (b) the amplitude of this positivity differs significantly between switch and repeat trials even at the lowest point of amplitude differentiation (slowest
OPTA
No OPTA 5% 15% 25% 35% –5µV 45% 55% 06 65% 75% 85% 95%
Cue
Stimulus
Figure 3. Cue-locked ERP waveforms for one participant depicted at Pz for switch trials. No-OPTA data (left) represent conventional ERP averaging over the same trial numbers as the OPTA processed data (right).
566 5%), and (c) the difference in positivity is evident even when repeat and switch trials are matched on RT suggest that the differential switch-positivity represents a component elicited specifically on switch trials. This component is superimposed on a more generic positivity that is also elicited for repeat trials, and its amplitude is associated with the amount of switch-specific preparation. This conclusion is also supported by the finding that the positivity for repeat trials showed little change over RT semideciles, despite the fact that repeat and switch RT varied over largely overlapping ranges (380–1250 ms for repeats, 405–1405 ms for switches). These findings suggest that cue-locked ERPs consist of a positivity for repeat and switch trials over 200–400 ms whose amplitude is not associated with variation in RTand a superimposed positivity, especially over the later part of that interval, that occurs exclusively for switch trials and whose amplitude is inversely related to RT and RT switch cost (see also Lavric et al., 2008). These findings are consistent with at least two preparatory processes: one that is common to both switch and repeat trials and another that, at least in the present context, is specifically elicited by switch cues. The positivity associated with both switch and repeat trials is consistent with the larger positivity found for repeat trials in a mixed-task block as compared to trials in a single-task block (i.e., mixing positivity; Goffaux et al., 2006; Jost, Mayr, & Rosler, 2008; Wylie et al., 2009; Ruge, Stoet, & Naumann, 2006) and for informative repeat cues compared to non-informative cues (Jamadar et al., 2010). Wylie et al. (2009) showed that this mixing positivity was associated with LAURA source activation in parietal areas. Jamadar, Hughes, Fulham, Michie, and Karayanidis (2010) found that variation in the amplitude of this mixing positivity was associated with fMRI contrast activation in dorsolateral prefrontal cortex and posterior cingulate. The switchspecific positivity is compatible with a process associated with anticipatory preparation that, at least with the current task parameters, is only engaged on switch trials.3 The gradual variation in switch-positivity amplitude and its association with RT switch cost suggests that the switch-specific reconfiguration process varies in efficiency across trials resulting in intra-individual variability in RT switch cost. The switch-specific positivity observed here is compatible in latency with the switch-specific activation shown over frontal sources in Wylie et al.’s Figure 5B. Lavric et al. (2008) found greater frontal (500–600 ms) followed by later posterior (600–800 ms) LORETA activity for the switch–repeat contrast. Jamadar et al. (2010) showed that the switch-repeat positivity was associated with fMRI contrast activation in posterior parietal cortex. While the latency and, to some degree, the morphology of these switch-positivities differ across these studies, this may be due to differences in EEG reference settings, complexity of the experimental paradigm, and amount of training. Although the positive relationship between RTand RT switch cost is generally compatible with de Jong’s (2000) intention-activation model’s stipulation of a mixture of both prepared and unprepared trials, the gradual variation in cue-locked switch positivity amplitude with increasing RT switch cost is not easily reconciled with an all-or-none process (see also Brown, Lehmann, & Poboka, 2006). Furthermore, we found a significant 3 Alternatively, as suggested by an anonymous reviewer, this switchpositivity may represent the different outcome of a process that is equally activated for both switch and repeat trials. As we cannot differentiate between these alternatives here, we do not discuss this alternative further.
F. Karayanidis et al. RT switch cost even for the fastest 5% of responses. This again argues against a binary division between fully prepared vs. unprepared states, and suggests either that anticipatory reconfiguration cannot be fully completed until after stimulus onset (Rogers & Monsell, 1995) or that S-R interference processes activated by the target (Allport & Wylie, 2000) affect RTeven for fully prepared trials. These findings suggest that switch-specific preparation may involve a gradual process or a set of sub-processes that may be partly or fully completed within the cue-target interval. This conclusion is compatible with other data suggesting a multi-component process of advance preparation (Jamadar, Michie, & Karayanidis, 2010; Karayanidis et al., 2009; Nicholson, Karayanidis, Davies, & Michie, 2006; Swainson et al., 2006) which are differentially activated depending on task parameters and strategy (see Karayanidis et al., 2010). In contrast to the finding that the amplitude of the cue-locked positivity varies across the RT distribution for switch trials only, the amplitude of the pre-target negativity was related to RT for both switch and repeat trials. The pre-target negativity was larger for repeat as compared to switch trials and for fast as compared to slow responses on both repeat and switch trials. Fast switch trials were associated with a larger positivity as well as a larger pre-target negativity as compared to slow switch trials. This indicates that RT had opposite effects on the switch waveform in these two epochs (Figure 2B) and suggests that the effect of RT percentile on the pre-target negativity does not simply reflect a carryover of the earlier effect on the switch positivity. The independence of cue-locked positivity and the pre-target negativity associations to RT is also supported by the fact that, although pre-target negativity amplitude varied across RT percentile for repeat trials, there was no variability in the cue-locked positivity for repeat trials. So, intra-subject variability showed that a larger pre-target negativity was associated with faster responding for both switch and repeat trials, whereas inter-subject variability showed that fast responders tended to show a larger pre-target negativity for repeat trials only. Note that eliminating RT differences between switch and repeat trials eliminated the difference in pre-target negativity amplitude between these trial types at central electrodes. The variation in pre-target negativity amplitude with RT is consistent with a more general arousal or attentionrelated process and suggests that the switch/repeat difference in pre-target negativity amplitude is more likely to be related to generic differences in preparation or task readiness. In conclusion, application of the OPTA regression methodology to cue-locked waveforms provided evidence for both switch-specific and general anticipatory preparation processes within the CSI. What in the cue-locked difference waveforms appeared as a centroparietal switch-positivity spanning most of the CSI was, in fact, the result of differential modulation of two underlying components. The peak of the centroparietal differential switch-positivity mapped onto a cue-locked positivity that was inversely related to RT for switch trials only. The slow return to baseline of this differential switch-positivity mapped onto the pre-target negativity that was directly related to RT for both switch and repeat trials. These findings support multi-component models of anticipatory preparation and suggest that care needs to be taken when interpreting results based solely on difference waveform analyses. The cue-locked positivity and the pre-target negativity are compatible with the conceptualization of advance preparation as a set of processes that may include central as well as modality-specific and/or task-specific components (Jennings & van der Molen, 2005). The finding that the cue-locked
Switch-specific and general preparation in cued task-switching positivity varied as a function of RT percentile only for switch trials suggests that, in the current context, this component reflects a switch-specific reconfiguration process. However, this does not
567 preclude the possibility that, given the right task parameters (e.g., low repeat probability), advance reconfiguration may also be helpful on repeat trials.
REFERENCES Allport, D. A., Styles, E. A., & Hsieh, S. (1994). Shifting intentional set: Exploring the dynamic control of tasks. In C. Umilta & M. Moscovitch (Eds.), Attention and Performance XV (pp. 421–452). Cambridge, MA: MIT Press. Allport, D. A., & Wylie, G. (2000). Task switching, stimulus-response bindings, and negative priming. In S. Monsell & J. Driver (Eds.), Attention and Performance XVIII (pp. 35–70). Cambridge, MA: MIT Press. Altmann, E. M. (2004). The preparation effect in task switching: Carryover of SOA. Memory & Cognition, 32, 153–163. Astle, D. E., Jackson, G. M., & Swainson, R. (2008). Fractionating the cognitive control required to bring about a change in task: A dense-sensor event-related potential study. Journal of Cognitive Neuroscience, 20, 255–267. Brown, S. D., Lehmann, C., & Poboka, D. (2006). A critical test of the failure-to-engage theory of task-switching. Psychonomic Bulletin & Review, 13, 152–159. de Jong, R. (2000). An intention-activation account of residual switch costs. In S. Monsell & J. Driver (Eds.), Attention and Performance XVIII (pp. 357–376). Cambridge, MA: MIT Press. de Koning, D., Woestenburg, J., & Elton, M. (2001). Visual spatial attention in migraine sufferers in postictal and interictal phases: An event-related potential study. Journal of Psychophysiology, 15, 22–34. Goffaux, P., Phillips, N. A., Sinai, M., & Pushkar, D. (2006). Behavioural and electrophysiological measures of task switching during single and mixed-task conditions. Biological Psychology, 72, 278–290. Handy, T. (2004). Basic principle of ERP quantification. In T. Handy (Ed.), Event-related potentials: A methods handbook (pp. 33–56). Cambridge, MA: MIT Press. Jamadar, S., Hughes, M., Fulham, W. R., Michie, P. T., & Karayanidis, F. (2010). The spatial and temporal dynamics of anticipatory preparation and response inhibition. NeuroImage, 51, 432–449. Jamadar, S., Provost, A., Fulham, W. R., Michie, P. T., & Karayanidis, F. (2010). Multiple sources underlie ERP indices of task-switching. In W. Christensen, E. Schier, & J. Sutton (Eds.), ASCS09: Proceedings of the 9th Conference of the Australasian Society for Cognitive Science (pp. 154–161). Sydney: Macquarie Centre for Cognitive Science. DOI: 10.5096/ASCS200924; Retrieved from http://www.maccs.mq. edu.au/news/conferences/2009/ASCS2009/html/jamadar.html. Jamadar, S., Michie, P. T., & Karayanidis, F. (2010). Sequence effects in cued task-switching modulate response preparedness and repetition priming processes. Psychophysiology, 47, 365–386. Jennings, J. R., & van der Molen, M. W. (2005). Preparation for speeded action as a psychophysiological concept. Psychological Bulletin, 131, 434–459. Jost, K., Mayr, U., & Rosler, F. (2008). Is task switching nothing but cue priming? Evidence from ERPs. Cognitive, Affective & Behavioral Neuroscience, 8, 74–84. Karayanidis, F., Jamadar, S., Ruge, H., Phillips, N., Heathcote, A., & Forstmann, B. U. (2010). Advance preparation in task-switching: Converging evidence from behavioral, brain activation, and modelbased approaches. Frontiers in Psychology, 1, 25. Karayanidis, F., Robaey, P., Bourassa, M., de Koning, D., Geoffroy, G., & Pelletier, G. (2000). ERP differences in visual attention processing between attention-deficit hyperactivity disorder and control boys in the absence of performance differences. Psychophysiology, 37, 319–333. Karayanidis, F., Mansfield, E. L., Galloway, K. L., Smith, J., Provost, A., & Heathcote, A. (2009). Anticipatory reconfiguration elicited by fully and partially informative cues that validly predict a switch in task. Cognitive, Affective & Behavioural Neuroscience, 9, 202–215. Kieffaber, P. D., & Hetrick, W. P. (2005). Event-related potential correlates of task switching and switch costs. Psychophysiology, 42, 56–71. Kenemans, J. L., Verbaten, M. N., Melis, C. J., & Slangen, J. L. (1992). Visual stimulus change and the orienting reaction: Event-related
potential evidence for a two-stage process. Biological Psychology, 33, 97–114. Koch, I., & Philipp, A. M. (2005). Effects of response selection on the task repetition benefit in task switching. Memory & Cognition, 33, 624–634. Lavric, A., Mizon, G., & Monsell, S. (2008). Neurophysiological signature of effective anticipatory task-set control: A task-switching investigation. European Journal of Neuroscience, 28, 1016–1029. Lorist, M., Klein, M., Nieuwenhuis, S., de Jong, R., Mulder, G., & Meijman, T. (2000). Mental fatigue and task control: Planning and preparation. Psychophysiology, 37, 614–625. Logan, G. D., & Schneider, D. W. (2006). Interpreting instructional cues in task switching procedures: The role of mediator retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 347–363. Meiran, N. (2000). Reconfiguration of stimulus task sets and response task sets during task switching. In S. Monsell & J. Driver (Eds.), Attention and Performance XVIII (pp. 377–399). Cambridge, MA: MIT Press. Miniussi, C., Marzi, C. A., & Nobre, A. C. (2005). Modulation of brain activity by selective task sets observed using event-related potentials. Neuropsychologia, 43, 1514–1528. Mueller, S. C., Swainson, R., & Jackson, G. M. (2007). Behavioural and neurophysiological correlates of bivalent and univalent responses during task switching. Brain Research, 1157, 56–65. Mueller, S. C., Swainson, R., & Jackson, G. M. (2009). ERP indices of persisting and current inhibitory control: A study of saccadic task switching. NeuroImage, 45, 191–197. Nicholson, R., Karayanidis, F., Poboka, D., Heathcote, A., & Michie, P. (2005). Electrophysiological correlates of anticipatory task-switching processes. Psychophysiology, 42, 540–554. Nicholson, R., Karayanidis, F., Bumak, E., Poboka, D., & Michie, P. T. (2006). ERPs dissociate the effects of switching task sets and task cues. Brain Research, 1095, 107–123. Nicholson, R., Karayanidis, F., Davies, A., & Michie, P. T. (2006). Components of task-set reconfiguration: Differential effects of ‘switch-to’ and ‘switch-away’ cues. Brain Research, 1121, 160–176. Nieuwenhuis, S., & Monsell, S. (2002). Residual costs in task switching: Testing the failure-to-engage hypothesis. Psychonomic Bulletin and Review, 9, 86–92. Poboka, D., Heathcote, A., Karayanidis, F., & Nicholson, R. (2005). An Investigation of Task Switch Costs: Preparation Activation, Timing and Readiness Decay. International Conference on Attentional Control, Jan 4–7, National Chung Cheng University, Taiwan. Provost, A., Brown, S., Heathcote, A. J., & Karayanidis, F. (in preparation). A revised orthogonal polynomial trend analysis (OPTA) procedure for single-trial ERP waveform estimation across the RT distribution. Unpublished manuscript. Regan, D. (1989). Human brain electrophysiology: Evoked potentials and evoked magnetic fields in science and medicine. New York: Elsevier. Rogers, R. D., & Monsell, S. (1995). Costs of predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124, 207–231. Rubinstein, J., Meyer, D. E., & Evans, J. E. (2001). Executive control of cognitive processes. Journal of Experimental Psychology: Human Perception and Performance, 27, 763–797. Ruge, H., Stoet, G. T., & Naumann, E. (2006). Attentional set mixing: Effects on target selection and selective response activation. Psychophysiology, 43, 413–421. Rushworth, M. F. S., Passingham, R. E., & Nobre, A. C. (2005). Components of attentional set-switching. Experimental Psychology, 52, 83–98. Semlitsch, H. V., Anderer, P., Schuster, P., & Presslich, O. (1986). A solution for reliable and valid reduction of ocular artefacts. Psychophysiology, 23, 695–703. Shelley, A. M., Ward, P. B., Michie, P. T., Andrews, S., Mitchell, P. F., Catts, S. V., et al. (1991). The effect of repeated testing on ERP
568 components during auditory selective attention. Psychophysiology, 28, 496–510. Spencer, K. (2004). Averaging, detection, and classification of single-trial ERPs. In T. Handy (Ed.), Event-related potentials: A methods handbook (pp. 209–228). Cambridge, MA: MIT Press. Swainson, R., Jackson, S. R., & Jackson, G. M. (2006). Using advance information in dynamic cognitive control: An ERP study of taskswitching. Brain Research, 1105, 61–72. van der Lubbe, R., & Woestenburg, J. (1997). Modulation of early ERP components with peripheral precues: A trend analysis. Biological Psychology, 45, 143–158. van der Lubbe, R., & Woestenburg, J. (1999). The influence of peripheral precues on the tendency to react towards a lateral relevant stimulus with multiple-item arrays. Biological Psychology, 51, 1–21. van der Lubbe, R., & Woestenburg, J. (2000). Location selection in the visual domain. Psychophysiology, 37, 662–676.
F. Karayanidis et al. Vasey, M. W., & Thayer, J. F. (1987). The continuing problem of false positives in repeated measures ANOVA in psychophysiology: A multivariate solution. Psychophysiology, 24, 479–486. Wylie, G. R., Murray, M. M., Javitt, D. C., & Foxe, J. J. (2009). Distinct neurophysiological mechanisms mediate mixing costs and switch costs. Journal of Cognitive Neuroscience, 21, 105–118. Woestenburg, J. C., Verbaten, M. N., Van Hees, H. H., & Slangen, J. L. (1983). Single trial ERP estimation in the frequency domain using orthogonal polynomial trend analysis (OPTA): Estimation of individual habituation. Biological Psychology, 17, 173–191.
(Received July 20, 2009; Accepted June 16, 2010)
Psychophysiology, 48 (2011), 569–577. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01116.x
Task requirements change signal strength of the primary somatosensory M50: Oddball vs. one-back tasks
THERESA GO¨TZ,a RALPH HUONKER,a WOLFGANG H. R. MILTNER,b OTTO W. WITTE,a KONRAD DETTNER,c and THOMAS WEISSb a
Friedrich Schiller University Jena, Department of Neurology, Biomagnetic Center, Jena, Germany Friedrich Schiller University Jena, Institute of Psychology, Department of Biological and Clinical Psychology, Jena, Germany University of Bayreuth, Department of Animal Ecology, Bayreuth, Germany
b c
Abstract Studies on attention to tactile stimuli have produced conflicting results concerning the possibility and/or direction of modulation of early somatosensory-evoked fields (SEFs). To evaluate sources of these conflicting results, the same subjects performed four different tasks in which the stimulation site, type, and intensity were kept constant. Twelve subjects performed an oddball-like tactile task, two different one-back tactile tasks, and a visual task, while two distal phalanges of the index and ring finger were stimulated. Task-dependent SEF modulations were found as early as 50 ms after stimulus onset (M50 component). Target/non-target ratios of M50 revealed enhanced values for the oddball-like tactile task, but decreased values for the tactile one-back task. This indicates that previously obtained conflicting results might be due to different central mechanisms induced by different task requirements. Descriptors: Somatosensory cortex, Magnetoencephalography (MEG), Somatosensory-evoked fields (SEFs), Intraand intermodal attention, Cortical representations
not found modulations of the early processing of somatosensory information due to attentional task requirements (Desmedt, Robertson, Brunko, & Debecker, 1977; Forss, Jousmaki, & Hari, 1995; Mima, Nagamine, Nakamura, & Shibasaki, 1998). Moreover, Mauguiere et al. (1997b) even found a tendency for reduced signal strength for early processing of somatosensory stimuli for attended stimuli. On a cross-modal level, many studies have investigated links in spatial attention between vision and audition (e.g., Eimer & Schro¨ger, 1998). Comparing visual and auditory ERPs within an attentional modulation task, these authors described the traditionally reported enhancement of early ERP components when attention was directed towards the relevant modality. Comparing vision and touch, similar effects were found when attention was directed towards the somatosensory modality (e.g., Michie, 1984; Garcia-Larrea et al., 1995). Employing neuromagnetic measuring techniques, Iguchi and co-authors (Iguchi, Hoshi, Tanosaki, Taira, & Hashimoto, 2005) found an enhanced component in the somatosensory-evoked field (SEF) 50 ms after the stimulus (M50) for the attended target finger compared with the non-target finger. The authors concluded that attention was able to regulate activity in primary somatosensory cortex (S1) by selectively enhancing the task-relevant information and filtering out other inputs. Braun and coauthors (2002) also found a modulation of the M50 component for similar attended tactile stimuli, but this was dependent on the type of task (detection of movement direction as indicated by the stimulation of successive fingers vs. detection of direction as indicated by a slight spatial displacement of stimulation on a single
Traditionally, it was thought that voluntary attention to the auditory (Hotting, Rosler, & Roder, 2003; Poghosyan & Ioannides, 2008), visual (Hillyard et al., 1987), or somatosensory (Desmedt, Huy, & Bourguet, 1983; Desmedt & Robertson, 1977; Mauguiere et al., 1997a, b) modality would lead to stronger event-related potentials (ERPs) or magnetic fields (ERFs) for the attended modality. Neurophysiologically, however, it seems that the effects of attention may actually depend on the nature of the experimental task employed. At the cellular level, selective attention has been shown to increase firing rates for attended stimuli (Buia & Tiesinga, 2006), to shrink the receptive field around an attended stimulus (Compte & Wang, 2006; Womelsdorf, Anton-Erxleben, Pieper, & Treue, 2006; Womelsdorf, Anton-Erxleben, & Treue, 2008), or to shift the tuning curves toward the task-relevant stimulus (Compte & Wang, 2006). At the macroscopic level, results have been conflicting, with some studies reporting attention-related increases of early somatosensory potentials, and others reporting decreases; some have even found no modulation at all. Within the ERP domain, for example, intramodal selective attention has been shown to increase the amplitude of ERP components as early as 45 ms after stimulus onset (Desmedt et al., 1983; Desmedt & Robertson, 1977). In contrast to these findings, there are a number of studies that have We thank Uwe Schulze for technical support. The first two authors contributed equally to this study. Address correspondence to: Theresa Go¨tz, University Hospital Jena, Department of Neurology, Biomagnetic Center, Erlanger Allee 101, D-07747 Jena, Germany. E-mail:
[email protected] 569
570 finger). In contrast, some studies have reported no modulation of the early processing of somatosensory information due to attentional task requirements (Desmedt et al., 1977; Forss et al., 1995; Mima et al., 1998). On a cross-modal level, Huonker, Weiss, and Miltner (2006) reported a reduced M50 in a somatosensory one-back task. In this study, subjects were requested to detect the repeated stimulation to any finger in a stream of tactile stimuli applied to two different fingers. The M50 to attended tactile stimuli was reduced compared to the response to the same stimuli when attention was directed to the visual modality (by a series of visual stimuli presented either with or between the tactile stimuli). While the enhancement of the early components of somatosensory-evoked potentials or fields can be interpreted as a filter mechanism that promotes one channel and inhibits all others, the interpretation of a reduction in these early components is less straightforward. Huonker et al. (2006), however, proposed an interpretation based on primary processing mechanisms at the level of cortical representations. In this model, selective attention leads to an increase of the inhibitory periphery of the receptive fields within S1 causing changes in the center-periphery-relation of the fingers’ cortical representations (CRs). Thus, signal strength is reduced, but noise is also reduced, and signal-tonoise ratio is thereby enhanced. A similar attentional modulation involving lateral inhibition has already been described for the visual system (Worgotter et al., 1998). Here the authors demonstrated enhancement of visual contrast by this mechanism, which resulted in improvements in recognition of distinct shapes and orientation. Applying these results to the somatosensory modality, one might expect the change in center-periphery relations for the receptive fields in the primary somatosensory cortex S1 to improve the ability to discriminate between fingers, a requirement for the successful performance of the task. We assume that an increased peripheral inhibition within a CR leads to a tightening of its inner excitatory part. At the same time, we assume that, when inhibited, the peripheral outer CR zone does not contribute to the overall magnetic signal strength (or even lowers the contribution of the excitatory part of CR). Assuming an additive (but not multiplicative) effect of inhibition on the signal produced by the stimulus and on noise, this mechanism would yield a higher signal-to-noise ratio. The sharpening, in the physiological sense, of the CR might improve discrimination ability for stimulus locations. Summing up the theoretical background, we propose that early modulations of processing of the somatosensory stimuli might not just depend on attention, but also on task requirements. More specifically, we hypothesized that somatosensory processing might depend on the difference between detecting a single, spatially constant target (i.e., in an oddball task) and detecting a spatially changing target, the target position of which is produced by a certain stimulus-constellation (e.g., the last stimulus location, i.e., one-back task). While the former oddball task might be associated with an increase of the signal to the target by a filter mechanism (i.e., the traditional view), the latter one-back task might be associated with a reduction of the signal to the target due to a change in center-periphery relations of CR in S1. To further investigate this intriguing possibility, we used the same tactile one-back task described above that has been shown to reduce early somatosensory signal strength during somatosensory attention (Huonker et al., 2006). This task requires discrimination of the stimulated fingers, retrieval of the last location stimulated, a comparison of the current stimulus location with the previous, and a response in the case of repeated stimulation.
T. Go¨tz et al. In order to perform the task, however, the key process within the first 80 ms (M50) following stimulus onset is presumably the correct discrimination of stimulus location. We therefore also used, for comparison, a visual task (that required no somatosensory localization or discrimination) as a distraction from the tactile stimuli. Our hypothesis predicts lower signal strength for the tactile tasks. Furthermore, if the hypothesis of intracortical inhibition is correct, then it is reasonable to further hypothesize that the level of inhibition will depend on the proximity of the stimulated sites: the closer the potential stimulation sites, the stronger the peripheral inhibition necessary (of the CR) for correct discrimination of the sites. We therefore included three somatosensory attention tasks, each requiring detection of a different set of somatosensory stimulus locations. We expected strength of early somatosensory responses to vary with stimulus proximity. Moreover, for one of these tasksFthe oddball taskFwe expected to see the normal increase of SEF activity due to the selective enhancement of targets by a different mechanism. Finally, we collected behavioral data, in the form of localization errors, to test the hypothesis that these would depend on task requirements and, in particular, the proposed level of inhibition. The aims of the study were to replicate the results of Huonker et al. (2006) and to extend them by testing for an inverse relationship between cortical inhibition and target proximity.
Materials and Methods Experimental Procedure Twelve female right-handed subjects (age 19–28 years, mean 23 ! 3 years) participated in the study, which was approved by the local Ethics Committee. According to the Declaration of Helsinki, all subjects were instructed verbally about all details of the experiment and their right to terminate participation at any time. They gave written informed consent prior to the experiment. No subject suffered from neurological disease, and all participants were free of medication. Prior to the experiment, all subjects were familiarized with the experimental procedure using some practice trials. Subjects were paid for participation (6h/hour) and were promised an additional bonus if the tasks during the experiment were performed as accurately as possible. For application of the tactile stimuli, an air puff stimulator (BTI) was used. Stimuli were applied to the two distal phalanges of the index (D21 for finger tip and D22 for middle phalanx of the index finger) and ring finger (D41, D42; indices analogous to the index finger) of the right hand (see Figure 1A). The tactile stimuli were presented in a pseudo-random order with an interstimulus interval (ISI) of 1251 ! 144 ms (mean ! SE). Tactile stimuli were presented with a stimulus rise time of 20 ms and stimulus duration of 300 ms. Simultaneously visual stimuli were presented in a pseudo-randomized order by one red or green flashing LED with an ISI of 1002 ! 116 ms (mean ! SE). The onset and duration of the visual stimuli were not time-locked to the tactile stimulation. During two experimental sessions on each of two days, subjects had to perform two tasks during each session. The task order was randomized across all subjects to ensure that no habituation or order effects affected the results. For each condition, different targets were defined. Subjects were asked to press a pneumatic button as quickly as possible with the left hand
Task requirements affect the SEF M50
571 on the proximity of the stimulus locations. Therefore, we expected the strongest inhibition during this task (see Figure 1B). One-back task for index finger stimulations as the target (DSIF). Subjects were requested to attend to any repeated stimulation of the index finger, regardless of which phalanx of the index finger was stimulated. This task was introduced as a ‘‘mixed’’ task between the oddball-like TIF and DS4F task. As it is comparable to the TIF task, we expected enhanced signal strengths for the SEFs after index finger stimulation (target) in comparison to the ring finger stimulation (non-target). Additionally, and this is the ‘‘mixed character’’ of this task, we expected enhanced discrimination ability of the stimulation sites on the index finger (target) as compared to the ring finger (nontarget).
Figure 1. (A) Sites of the air puff stimulation on the index (D21 and D22) and ring (D41 and D42) finger of the right hand. (B) Schematic description of our model of lateral inhibition and a representative channel. Circles indicate the inner and outer parts of the cortical representations (CRs) of D2 and D4; the black fill color indicates excitatory center, gray, the inhibitory surrounding. The visual distraction task (DC, left) is performed with standard CRs with a degree of overlap since no discrimination is required. This results in a certain (high) signal strength (black signal curve). In contrast, the one-back task including all four tactile stimulation sites as target locations (DS4F, right part of B) requires discrimination of the fingers. According to our model, DS4F is performed with narrowed CRs, i.e., only the inner part of the CR is active and no (or less) overlap occurs. As a result, performance of this task results in a reduced signal strength (gray signal curve).
whenever a target was detected. During the tasks, sudden breaks were introduced. During these breaks, subjects were asked to report the last stimulus location (mislocalization test). Each of the four tasks consisted of exactly 1200 tactile stimuli and lasted 20 min. The following tasks were performed (see Table 1): Tip of the index finger as target location (TIF). Subjects had to respond as quickly as possible whenever D21 (the tip of the index finger) was stimulated. In this oddball-like task, we expected enhanced signal strengths in response to D21 stimulation compared to the stimulation of the ring finger (D4). We assume that the enhanced signal strengths might be due to selective filter mechanisms for targets. Furthermore, we expected better discrimination for the tip of the index finger as compared to the other parts of the fingers. One-back task including all 4 tactile stimulation locations as target locations (DS4F). Subjects were instructed to pay attention to any repeated stimulation of one of the four stimulus locations (one-back task). Contrary to the other three tasks, all four stimulus locations were targets. Thus, the different target locations lay close to each other. As briefly explained in the introduction, we hypothesized that the level of inhibition depends
Visual stimuli as target (DC). This task was included to distract attention away from the tactile modality. Subjects were requested to pay attention to a repeated flash of the same LED color. On the one hand, this situation does not require any discrimination of the tactile stimulus location; therefore, we expected SEF amplitudes to be higher in DC than in DS4F because higher levels of inhibition in cortical representations were hypothesized for discrimination of the location of tactile stimulation. On the other hand, TIF was hypothesized to produce enhanced SEF amplitudes due to a selective filter and/or enhancement of the target location; therefore, we expected SEF amplitudes to be lower in DC than in TIF. Concerning the behavioral results from the mislocalization test in the sudden breaks, we expected the poorest discrimination of all tasks for DC because attention during this task is oriented to the visual modality (see Figure 1B). Targets occurred with a probability of 16% for DS4F, 21% for DSIF, and 25% for TIF, whereas target probability of LEDs was 35%. Behavioral Data The following three behavioral data sets were recorded and analyzed: percentage of target detection during a task (performance), mislocalization test, and the subject’s estimation of task difficulty (self-estimation). Performance was analyzed on the basis of button presses to targets. Performance was assessed on the basis of missed targets and false positive trials during a task. A mislocalization test was introduced to assess the influence of task requirements on the detection of the last localization of stimulation sites. In other words, discrimination ability was assessed in the form of localTable 1. Schematic Account of the Expected Outcome in Respect of the Amplitude of the M50 and the Number of Mislocalizations M50 – Amplitude
TIF DS4F DSIF DC
No. of mislocalizations
D21
D22
D41
D42
D21
D22
D41
D42
"" ## " ""
# ## " ""
# ## # ""
# ## # ""
# ## # ""
" ## # ""
" ## " ""
" ## " ""
Note: The arrows indicate an enhancement or a reduction of amplitude or error number depending on the task performed. TIF: oddball-like task with the tip of the index finger as target, DS4F: one-back task including all four stimulus locations as targets, DSIF: one-back task including the two phalanges of the index finger as targets, DC: visual distraction condition.
572 ization errors during all fours tasks, i.e., fewer mislocalizations on the target finger compared to the non-target finger. For this test, 13 randomly distributed breaks were introduced into each task, where subjects were requested to recall the position of the last tactile stimulus as accurately as possible by reporting finger and phalanx of the last tactile stimulus. Self-estimation was assessed after the two tasks on each of the two days using a 12-point scale (0: very easy, 12: extremely difficult).
Data Acquisition and Analysis We performed MEG measurement using a single Dewar system with 31 channels (Philips Deutschland GmbH, Hamburg, Germany). The array of the antennae had a diameter of 14 cm and contained first-order axial gradiometers of 2 cm diameter and 7 cm baseline arranged in a hexagonal grid with a 2.5 cm pitch. All p channels had an overall system noise level of less than 1 ffiffiffiffiffiffi fT= Hz. Before MEG was recorded, the coordinates of coils and anatomical landmarks (Nasion, Cz, preauricular points) were digitized with an ISOTRAK II digitizer (Polhemus Inc., Colchester, VT). The subject’s head was fixed during MEG recordings in relation to the Dewar by a vacuum cushion. For each subject, the Dewar was centered above the primary somatosensory cortex. This was performed by running a test series of electrical median nerve stimulations applied to the subject’s right wrist. Electrical stimulation of the median nerve activates the nerve fibers, directly evoking components like the N20 or P35 that are distinct from components received by tactile stimulation. While electrical stimulation helps to identify an optimal position of the Dewar faster than mechanical stimulation, it is an artificial stimulation that probably does not activate the mechanoreceptors. In contrast, tactile air puff stimulation activates cutaneous receptors and not the nerve itself. The air puffs used during the main experiments consisted of a dilatation of a membrane which formed a part of a finger clip attached to the phalanx. During the air influx, this membrane dilates and produces a slight pressure on a skin area of the stimulated phalanx. Therefore, the tactile stimulation used will evoke an earliest component at a latency of 50 ms, i.e., M50 (similarly to Braun et al., 2002; Huonker et al., 2006). Each subject’s head position was captured prior and after each MEG recording by localizing the positions of five functional landmarks (orthogonal coil sets) of the head positioning interface. The distance of the head surface to the coils was kept constant at 42.1 ! 1.7 mm (mean ! standard error SE). During MEG recordings, eye movements (EOG) and task performance were recorded simultaneously. MEG and EOG data were digitized at 1 kHz sampling rate. Data and source analysis were performed with Curry version 4.6 software. EOG artifacts were rejected at " 150 to 1150 mV. After visual inspection for movement artifacts, data were averaged for each stimulus location (phalanx) and task. We included 244 ! 12 artifact-free responses for each phalanx starting at " 100 ms prior to the tactile stimulus onset and ending at 400 ms post-stimulus. The datasets for each of the four tasks could be subdivided into four sub-datasets, one for each stimulus location. Overall, we obtained data from four tasks per subject and four stimulus locations per task. Further pre-processing included a common mode rejection (CMR) and a band pass filter with a third order Butterworth filter with a band width of 0.3 to 200 Hz. Additionally, data were baseline corrected from " 100 to 0 ms. Data were further pre-
T. Go¨tz et al. processed by a singular value decomposition (SVD), ignoring components below a signal-to-noise ratio of 1. Source reconstruction was performed using a spherical volume conductor model (Hari et al., 1990; Hoechstetter et al., 2001). The spherical volume conductor was fitted to the segmented cortex surface covering optimally the somatosensory cortex area, which was received from the isotropic 3D data set. Each subject’s T1-weighted data set of the head (192 slices, thickness 1 mm) was obtained from a 1.5T MR scanner (Siemens Magnetom Vision 1.5T, Erlangen, Germany). Dipole strength was calculated using a single equivalent current dipole model. The maximum mean global field power (MGFP) derived from the data averaged in the time window of 20–60 ms after stimulus onset was calculated. For further analysis, only dipoles with a goodness of fit over 85% were included (Salmelin & Hamalainen, 1995). The average goodness of fit was 91% for DS4F and DC, and 90% for DSIF and TIF. Additionally, the cortical representations of the phalanges in terms of polar angles were evaluated. The procedure for calculating polar angles was performed according to Braun et al. (2002). Statistical Data Analysis In a first analysis, only the non-repetitions (single tactile stimulus on one of the four phalanges; mainly non-targets except D21 in the TIF task) were used for analysis comparing all tasks including DC. Repetitions were excluded for the first analysis because they had a shorter ISI (leading to lower signal strength) and were different in nature, i.e., partly targets, partly non-targets. The subdivision of datasets resulted in four datasets per subject (four tasks) and four stimulus locations per task (four stimulated phalanges), i.e., 16 values of MGFP and 16 values of polar angles per subject were achieved. Statistical analyses of these data were performed using SPSS 13.0. A repeated measures multivariate analysis of variance (MANOVA) was performed using the factors Task (DS4F vs. TIF vs. DSIF vs. DC), Finger (D2 vs. D4), and Phalanx (distal vs. medial) and the dependent variables MGFP and polar angle. Posthoc tests were performed using separate MANOVAs for single factors and t-tests with Bonferroni corrections. When appropriate, results were e-corrected using the Greenhouse-Geisser method. An additional MANOVA with the factors Day (1 vs. 2) and Task (task 1 vs. task 2 on each day) was performed to assess habituation effects. The statistical analysis of the behavioral data (which were given in percentages) was performed using a Wilcoxon rank test after testing with the Friedman ANOVA. A second analysis tested more directly our main hypothesis, i.e., a different processing of targets in the oddball-like TIF task vs. the DS4F task. Thus, we used the data of the stimulations of D21, representing targets during each of the somatosensory attention tasks. For non-target signal strength, we used the mean value of all non-targets (note that TIF has no non-target at D21). We then compared a target/non-target ratio of MGFP for TIF and DS4F, respectively, because the traditional attention theory would predict an increase of this ratio for both types of tasks, while our hypothesis would predict an increase for TIF (according to the traditional view), but a decrease for DS4F (according to the inhibition hypothesis). We used t-tests for the comparison. Results Behavioral Data Behavioral data consisted of three categories: performance (percentage of correct target detection), the mislocalization test, and
Task requirements affect the SEF M50
573
Table 2. Results of Behavioral Data Analysis TIF-DS4F
Performance Self-estimation Mislocalization
TIF-DSIF
TIF-DC
DSIF-DS4F
DS-DS4F
DC-DSIF
z-value
p-value
z-value
p-value
z-value
p-value
z-value
p-value
z-value
p-value
z-value
p-value
" 2.32 " 2.98 " 0.17
.02 .003 .87
" 1.65 " 2.28 " 0.60
.094 .02 .55
" 3.06 " 3.06 " 2.30
.002 .002 .02
" 1.15 " 0.51 " 1.89
.25 .61 .059
" 3.06 " 3.06 " 2.40
.002 .002 .02
" 3.07 " 3.06 " 2.27
.002 .002 .02
Note: TIF: oddball-like task with the tip of the index finger as target, DS4F: one-back task including all four stimulus locations as targets, DSIF: oneback task including the two phalanges of the index finger as targets, DC: visual distraction condition. Bold numbers indicate significant comparisons.
subject’s estimation of task difficulty. The following results are shown in mean ! SE, except for the mislocalization test. All subjects were able to perform the tasks adequately. Thus, subjects detected 86.25 ! 1.85% (mean ! SE) of the DC trials correctly, while the other three tasks were accomplished with even higher accuracy (98.17 ! 0.37% for TIF, 96.75 ! 0.46% for DS4F, and 97.25 ! 0.45% for DSIF). Comparison between tasks revealed significant differences between DC and all other conditions (see Table 2). The mislocalization test revealed a small number of mistakes. While 85% (median) correct answers were given for DC, DSIF responses were 93% (median) correct, and TIF and DS4F were 100% (median) correct. Comparison between tasks revealed significant differences between DC and all other conditions as well as a trend for statistical significance between DSIF and DS4F (p 5 .059, z-value 5 " 1.89; see Table 2). Concerning the self-estimation of task difficulty, DC was perceived as the most difficult task (14.51 ! 3.13%) followed by DS4F (51.67 ! 5.86%), DSIF (51.88 ! 6.26%), and TIF (72.78 ! 4.49%). Comparison of self-estimation between tasks revealed significant differences between DC and all other conditions. MEG Data The air puff stimulation produced an early component over the primary somatosensory cortex for all 12 subjects at 50.03 ! 4.95 ms (mean ! SE) after stimulus onset (M50 component of the SEF). There was no latency difference between stimulation of D2 and D4. Dipoles from air puff stimulation were all generated in the somatosensory hand region (see Figure 2). There was no significant effect of factor Day on the signal strengths (F(1,47) 5 .06; p 5 .81) and no significant interaction of Day # Task (F(1,47) 5 1.59; p 5 .21). Consequently, we could exclude the factor Day from further analyses. Finger Representation The polar angles of the two fingers differed highly significantly (main effect of factor Finger: F(1,11) 5 77.5; po.001). The polar angle of D2 (48.8 ! 0.771; mean ! SE) was about 3.321 more medial than the polar angle of D4 (45.4 ! 0.671; mean ! SE). The factor phalanx showed only a trend toward a main effect (F(1,11) 5 3.83; p 5 .076). D21 (48.9 ! 1.081) and D22 (48.6 ! 1.11) did not differ, whereas D41 (45.8 ! 0.9) was found 0.91 more medial compared to D42 (44.9 ! 1). Factor Task did not influence polar angles (main effect of Task: F(3,33) 5 .18; p 5 .75). A significant interaction between Task and Finger was obtained (F(3,33) 5 3.48; p 5 .04), but post hoc comparisons did not reach significance. There were no other significant interactions.
M50 Mean Global Field Power For the analysis of the non-repetitions, MANOVA revealed a significant main effect of the factor Task on mean global field power (F(3,33) 5 4.79; p 5 .01) of the somatosensory M50 component. No significant main effects were found for factors Finger (F(1,11) 5 .36; p 5 .56) or Phalanx (F(1,11) 5 1.65; p 5 .23), nor did the interactions between factors Finger and Task (F(3,33) 5 1.82; p 5 .18) or Task and Phalanx (F(3,33) 5 .43; p 5 .67) reveal significant effects. Figure 3 shows that the main effect of factor Task resulted primarily from task DC. Post-hoc tests revealed significant differences between DC vs. DS4F (p 5 .02) and DC vs. DSIF (p 5 .02) (see Figure 2A). For the second analysis of target stimulation, ratios of target/ non-target for TIF vs. DS4F were compared at the D21 location. These ratios differed significantly (t(10) 5 2.51; p 5 .03) and yielded higher values for all except two subjects for the TIF (mean ! SE: 1.06 ! 0.09) whereas the target/non-target ratio of DS4F was lower for all subjects (mean ! SE: 0.86 ! 0.04). Using all targets (repeated stimulation and single D21 target location) in TIF, one could legitimately argue that the influence of ISI should be negligible when comparing TIF and DS4F. However, a comparison between TIF and DS4F using solely repeated stimulation did point in the same direction (t(10) 5 1.99; p 5 .074) (see Figure 2B). Discussion The present study examined changes of early SEF amplitudes evoked by different task requirements, with stimulation site, stimulation type, and stimulus intensity kept constant. Subjects were required to perform a one-back task either in the somatosensory or visual modality. Our data demonstrate that an intermodal switch of attention from visual distracters to somatosensory stimulation significantly modulates the somatosensory M50. Finger representation did not change significantly during the tasks. In comparison to visual distraction, attention to the somatosensory stimuli led to a significant decrease of MGFP of the SEFs for non-targets in the somatosensory modality. However, different intramodal somatosensory tasks produced modulations of early SEFs below statistical significance for these non-targets. Interestingly, we observed a significant difference in the target/non-target ratio of the signal strength for the oddballlike TIF task vs. the tactile one-back DS4F task. The latter result indicates a difference in somatosensory processing during different somatosensory tasks. As expected, we localized the somatosensory representations of the two fingers (index and ring fingers) in S1. The representation of the index finger was found more inferior compared to the ring finger. This finding is in line with the expected sensory
574
T. Go¨tz et al.
Figure 2. (A) Upper left: arrangement and placement of the Dewar with the 31 gradiometer channels over the left somatosensory cortex of a subject with three highlighted channels (16, 17, and 29). Lower right: Example of somatosensory-evoked fields (SEFs) of the channels 16, 17, and 29 for all four tasks in a time window of ! 100 to 300 ms. These example channels show only non-repetitions for all four tasks. (B) Example of SEFs for the tip of the index finger (D21) during the task with the tip of the index finger as target (TIF) and the one-back task including all four tactile stimulation sites as target locations (DS4F). SEFs are shown at channels 18 and 29 of one single subject in a time window from ! 50 to 200 ms for repetitive stimuli at D21 (i.e., targets) and non-repetitive stimuli.
finger somatotopy already found in former studies (Baumgartner et al., 1991; Weiss et al., 2000). Furthermore, we did not find significant short-term modulations of the S1 representation of each finger by the tasks. This result is in line with our hypothesis of a change in the centre-periphery relationship of the CRs. The hypothesis would predict stable centers of the functional cortical representations of each finger in S1 as only the area of the inhibitory CR periphery would be adapted according to the task, with the center remaining in the same location. Results of the analysis of non-targets showed that the visual distraction task (DC, intermodal attention), as compared to the other three somatosensory tasks, produced the highest MGFP values for the somatosensory M50 after stimulus onset in re-
sponse to finger stimulation. However, there were no significant differences for MGFP of the M50 between the three somatosensory tasks. The enhancement of the M50 during DC is in line with our hypothesis of changes of CRs in the primary somatosensory cortex S1. Given that normal tactile perception of the skin is based on certain CRs with an excitatory center and an inhibitory periphery, DC will be performed with rather ‘‘standard’’ CRs with a certain ratio of excitatory and inhibitory activity. It is well known that these CRs normally overlap to a substantial extent (Blake, Byl, & Merzenich, 2002; Lotze et al., 2000). However, when tactile discrimination is required, an overlapping of the CRs would be counterproductive, i.e., the discrimination of stimulus locations would be less reliable (which
Task requirements affect the SEF M50
Figure 3. Mean global field power (MGFP) of the non-repetitions of the somatosensory M50 (means ! standard errors) for the four tasks. DC: visual distraction condition, DS4F: one-back task including all four stimulus locations as targets, DSIF: one-back task including the two phalanges of the index finger as targets, TIF: index finger tip as target location.
is especially required during DS4F). Thus, the discrimination of precise localization of a tactile stimulus might be produced by a similar mechanism, i.e., lateral inhibition within CRs, as described for the visual system to enhance the contrast for the recognition of distinct shapes (Worgotter et al., 1998). The highest number of mislocalizations during DC supports this view. An alternative interpretation for the finding of reduced MGFP for non-targets in the somatosensory tasks derives from the traditional view. The results given above belong to non-targets, both in the attended and the non-attended modality. As such, these first results might also be explained as a stronger inhibition of non-targets in the attended modality for all somatosensory tasks, as the inhibition of non-targets would seem to be independent of the various task requirements. More interestingly, however, the second analysis of target/ non-target ratios revealed a difference for this ratio between TIF and DS4F. This result, first of all, clearly shows that task requirements are able to change the early somatosensory processing as early as 50 ms. Moreover, an analysis of the difference in ratios is revealing. On the one hand, with TIF there was an increase in the M50 MGFP for targets compared to non-targets (mean value for the ratio above 1). This result is in line with the traditional view of an oddball-like mechanism for the D21 signal strength enhancement in TIF. On the other hand, and in direct contrast, with DS4F there was a decreased M50 MGFP for targets compared to non-targets (mean value for the ratio below 1). This result is difficult to explain from the traditional viewpoint of increased signal with attention. This difference might be explained by the difference in the tasks between TIF and DS4F. While the TIF would profit from filtering out (and thereby enhancing) the input from the target position, the DS4F, in contrast, needs to analyze the exact location of the current stimulus because this will be the next target location (one-back task). Here, a discrimination of each stimulus location is required. Our hypothesis of a center-surround modulating mechanism would nicely explain the results for this task. Assuming an additive effect of signal and noise, the reduction of the same amount of signal and noise by inhibition would increase the signal-to-noise ratio, thereby increasing the possibility of discriminating stimuli,
575 but also reducing the signal from the targets, as reported here. However, it should be mentioned that we cannot directly extrapolate from the macroscopic SEF results to the level of a single cell. While the SEF signal, and thus the signal strength, can be interpreted as the sum activity from a volley of pyramidal cells (Hamalainen, Hari, Ilmoniemi, Knuutila, & Lounasmaa, 1993; Okada, Lauritzen, & Nicholson, 1987), it is clear that we cannot directly assess the inhibitory processes postulated by our main hypothesis. Inhibitory neurons mainly show a diffuse configuration and do not fulfill the open field criterion proposed by Okada, Wu, and Kyuhou (1997) and, therefore, cannot be measured with the gradiometers used in this study. Nevertheless, while the center-surround modulating mechanism is putative, it is obvious that the early somatosensory processing (M50) of an oddball-like task differs from that of a one-back task. Besides the proposed mechanism, several alternatives might explain or, at least, contribute to our result of different early somatosensory processing. A one-back task requires a certain amount of memorization, i.e., to fulfill the task requirements, it is necessary to remember the location of the last stimulus. Given that primary cortices are involved in short-term memory processes (Harris, Harris, & Diamond, 2001; Harris, Miniussi, Harris, & Diamond, 2002), one might speculate that all tactile tasks use resources of S1 for this short-term memory process while the visual task uses resources from the visual system. If the resources used for short-term memory cannot contribute to the processing of the next stimulus, then tactile stimuli would have fewer resources during the tactile tasks as compared to DC. This would explain why the DC produced higher SEF power than the tactile tasks. However, this alternative hypothesis would suggest that the short-term memory process is performed in S1 during the whole interstimulus interval of 1 s. Harris et al. (2002) investigated short-term memory to vibrotactile stimulation using transcranial magnetic stimulation (TMS) over the contralateral SI with different retention intervals to disrupt the tactile memory process. They found a significant decrease in discrimination ability for retention intervals of 300 and 600 ms, but not for 900 and 1200 ms. This indicates that S1 is a transient storage site for tactile stimuli, but in a time window below the interstimulus interval of our tactile tasks. In line with the above short-term memory explanation, memory load possibly also influences earlier SEF components. According to Lavie (2005), high memory load is able to increase the influence of distractors. Given that the same amount of memory load is used during all one-back tasks, differences in the visual distraction task and the somatosensory one-back tasks should not be influenced by memory load. However, this is not the case in TIF where the perception of the sole target location is sufficient to fulfill the task. Therefore, we cannot exclude a contribution of memory load in the observed differences between TIF and DS4F. Another factor that possibly influences earlier SEF components is perceptual load (Lavie, 2005; Schwartz et al., 2005). High perceptual load is able to reduce the influence of distractors. Within the visual modality, the study of Schwartz et al. (2005) showed that high perceptual load reduces the distractor interference in the primary visual cortex. In our study, perceptual load differs between the visual task, on the one hand, and somatosensory tasks, on the other hand. If it is the case that perceptual load has an influence, then it should only be able to explain differences between the DC and somatosensory tasks, and not those between somatosensory tasks.
576
T. Go¨tz et al.
Another related alternative results from differences in task difficulty for the tactile vs. the visual task. Indeed, given that task difficulty has an influence on resources such as arousal, it might follow that arousal is lower during the tactile tasks than during the visual task. This might even be more relevant for the biologically important visual modality than for the ‘‘rather primitive’’ (Gregory, 1967) somatosensory modality. Yet, the allocation of executive resources in a given task reflects both task difficulty and motivation (Buhle & Wager, 2010). However, Huonker et al. (2006) used similar one-back tasks in the tactile and somatosensory modality with slightly different interstimulus intervals. While their subjects rated the tactile condition as more difficult than the distraction condition, the changes of SEFs were similar to our results. In summary, the present study revealed three main results: (a) Task requirements are able to significantly modulate SEFs as early as 50 ms post-stimulus, i.e., during the primary processing
in S1; (b) modulations of SEFs to non-targets were less evident for changes within the tactile modality as compared to a crossmodal attentional shift to the visual modality; and (c) the ratio of M50 for target/non-targets was significantly higher in an oddball-like task than in a one-back task. The last finding indicates a selective enhancement of M50 for targets in the oddball-like task according to traditional views of attentional modulations, but a decrease of M50 for targets for the one-back task, indicating a different mechanism with this task. The result of reduced M50 target/non-target ratio in the one-back task is in line with our hypothesis that task requirements are able to modify the primary processing in SI even when all physical parameters of stimulation remain constant. These results are, at least in part, in line with our main hypothesis that task requirements, i.e., continuous discrimination at different locations of tactile stimuli at the fingers, might influence cortical representations in S1.
REFERENCES Baumgartner, C., Doppelbauer, A., Deecke, L., Barth, D. S., Zeitlhofer, J., Lindinger, G., & Sutherling, W. W. (1991). Neuromagnetic investigation of somatotopy of human hand somatosensory cortex. Experimental Brain Research, 87, 641–648. Blake, D. T., Byl, N. N., & Merzenich, M. M. (2002). Representation of the hand in the cerebral cortex. Behavioural Brain Research, 135, 179–184. Braun, C., Haug, M., Wiech, K., Birbaumer, N., Elbert, T., & Roberts, L. E. (2002). Functional organization of primary somatosensory cortex depends on the focus of attention. NeuroImage, 17, 1451–1458. Buhle, J., & Wager, T. D. (2010). Performance-dependent inhibition of pain by an executive working memory task. Pain, 149, 19–26. Buia, C., & Tiesinga, P. (2006). Attentional modulation of firing rate and synchrony in a model cortical network. Journal of Computational Neuroscience, 20, 247–264. Compte, A., & Wang, X. J. (2006). Tuning curve shift by attention modulation in cortical neurons: A computational study of its mechanisms. Cerebral Cortex, 16, 761–778. Desmedt, J. E., Huy, N. T., & Bourguet, M. (1983). The cognitive P40component, N60-component and P100-component of somatosensory evoked-potentials and the earliest electrical signs of sensory processing in man. Electroencephalography and Clinical Neurophysiology, 56, 272–282. Desmedt, J. E., & Robertson, D. (1977). Differential enhancement of early and late components of cerebral somatosensory evoked-potentials during forced-paced cognitive tasks in man. Journal of PhysiologyFLondon, 271, 761–782. Desmedt, J. E., Robertson, D., Brunko, E., & Debecker, J. (1977). Somatosensory decision tasks in manFEarly and late components of cerebral potentials-evoked by stimulation of different fingers in random sequences. Electroencephalography and Clinical Neurophysiology, 43, 404–415. Eimer, M., & Schro¨ger, E. (1998). ERP effects of intermodal attention and cross-modal links in spatial attention. Psychophysiology, 35, 313–327. Forss, N., Jousmaki, V., & Hari, R. (1995). Interaction between afferent input from fingers in human somatosensory cortex. Brain Research, 685, 68–76. Garcia-Larrea, L., Lukaszewicz, A. C., & Mauguiere, F. (1995). Somatosensory responses during selective spatial attention: The N120-to-N140 transition. Psychophysiology, 32, 526–537. Gregory, R. L. (1967). Origin of eyes and brains. Nature, 213, 369–372. Hamalainen, M., Hari, R., Ilmoniemi, R. J., Knuutila, J., & Lounasmaa, O. V. (1993). MagnetoencephalographyFTheory, instrumentation, and applications to noninvasive studies of the working human brain. Reviews of Modern Physics, 65, 413–497. Hari, R., Hamalainen, H., Hamalainen, M., Kekoni, J., Sams, M., & Tiihonen, J. (1990). Separate finger representations at the human 2nd somatosensory cortex. Neuroscience, 37, 245–249.
Harris, J. A., Harris, I. M., & Diamond, M. E. (2001). The topography of tactile working memory. Journal of Neuroscience, 21, 8262– 8269. Harris, J. A., Miniussi, C., Harris, I. M., & Diamond, M. E. (2002). Transient storage of a tactile memory trace in primary somatosensory cortex. Journal of Neuroscience, 22, 8720–8725. Hillyard, S. A., Woldorff, M. G., Mangun, G. R., & Hansen, J. C. (1987). Mechanisms of early selective attention in auditory and visual modalities. Electroencephalography and Clinical Neurophysiology, 39, 317–324. Hoechstetter, K., Rupp, A., Stancak, A., Meinck, H. M., Stippich, C., Berg, P., & Scherg, M. (2001). Interaction of tactile input in the human primary and secondary somatosensory cortexFA magnetoencephalographic study. NeuroImage, 14, 759–767. Hotting, K., Rosler, F., & Roder, B. (2003). Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli. Experimental Brain Research, 148, 26–37. Huonker, R., Weiss, T., & Miltner, W. H. R. (2006). Reduction of somatosensory evoked fields in the primary somatosensory cortex in a one-back task. Experimental Brain Research, 168, 98–105. Iguchi, Y., Hoshi, Y., Tanosaki, M., Taira, M., & Hashimoto, I. (2005). Attention induces reciprocal activity in the human somatosensory cortex enhancing relevant- and suppressing irrelevant inputs from fingers. Clinical Neurophysiology, 116, 1077–1087. Lavie, N. (2005). Distracted and confused?: Selective attention under load. Trends in Cognitive Sciences, 9, 75–82. Lotze, M., Erb, M., Flor, H., Huelsmann, E., Godde, B., & Grodd, W. (2000). fMRI evaluation of somatotopic representation in human primary motor cortex. NeuroImage, 11, 473–481. Mauguiere, F., Merlet, I., Forss, N., Vanni, S., Jousmaki, V., Adeleine, P., & Hari, R. (1997a). Activation of a distributed somatosensory cortical network in the human brain. A dipole modelling study of magnetic fields evoked by median nerve stimulation. 1. Location and activation timing of SEF sources. Evoked PotentialsF Electroencephalography and Clinical Neurophysiology, 104, 281–289. Mauguiere, F., Merlet, I., Forss, N., Vanni, S., Jousmaki, V., Adeleine, P., & Hari, R. (1997b). Activation of a distributed somatosensory cortical network in the human brain: A dipole modelling study of magnetic fields evoked by median nerve stimulation. 2. Effects of stimulus rate, attention and stimulus detection. Evoked PotentialsFElectroencephalography and Clinical Neurophysiology, 104, 290–295. Michie, P. T. (1984). Selective attention effects on somatosensory eventrelated potentials. Annals of the New York Academy of Sciences, 425, 250–255. Mima, T., Nagamine, T., Nakamura, K., & Shibasaki, H. (1998). Attention modulates both primary and second somatosensory cortical activities in humans: A magnetoencephalographic study. Journal of Neurophysiology, 80, 2215–2221.
Task requirements affect the SEF M50 Okada, Y., Lauritzen, M., & Nicholson, C. (1987). Meg source models and physiology. Physics in Medicine and Biology, 32, 43–51. Okada, Y. C., Wu, J., & Kyuhou, S. (1997). Genesis of MEG signals in a mammalian CNS structure. Electroencephalography and Clinical Neurophysiology, 103, 474–485. Poghosyan, V., & Ioannides, A. A. (2008). Attention modulates earliest responses in the primary auditory and visual cortices. Neuron, 58, 802–813. Salmelin, R. H., & Hamalainen, M. S. (1995). Dipole modelling of MEG rhythms in time and frequency domains. Brain Topography, 7, 251–257. Schwartz, S., Vuilleumier, P., Hutton, C., Maravita, A., Dolan, R. J., & Driver, J. (2005). Attentional load and sensory competition in human vision: Modulation of fMRI responses by load at fixation during taskirrelevant stimulation in the peripheral visual field. Cerebral Cortex, 15, 770–786.
577 Weiss, T., Miltner, W. H. R., Huonker, R., Friedel, R., Schmidt, I., & Taub, E. (2000). Rapid functional plasticity of the somatosensory cortex after finger amputation. Experimental Brain Research, 134, 199–203. Womelsdorf, T., Anton-Erxleben, K., Pieper, F., & Treue, S. (2006). Dynamic shifts of visual receptive fields in cortical area MT by spatial attention. Nature Neuroscience, 9, 1156–1160. Womelsdorf, T., Anton-Erxleben, K., & Treue, S. (2008). Receptive field shift and shrinkage in macaque middle temporal area through attentional gain modulation. Journal of Neuroscience, 28, 8934–8944. Worgotter, F., Suder, K., Zhao, Y. Q., Kerscher, N., Eysel, U. T., & Funke, K. (1998). State-dependent receptive-field restructuring in the visual cortex. Nature, 396, 165–168.
(Received July 10, 2009; Accepted June 17, 2010)
Psychophysiology, 48 (2011), 578–582. Wiley Periodicals, Inc. Printed in the USA. Copyright r 2010 Society for Psychophysiological Research DOI: 10.1111/j.1469-8986.2010.01117.x
BRIEF REPORT
Chronic stroke recovery after combined BCI training and physiotherapy: A case report
ANDREA CARIA,a,b CORNELIA WEBER,a DORIS BRO¨TZ,a ANDER RAMOS,a,c LUCA F. TICINI,d ALIREZA GHARABAGHI,e CHRISTOPH BRAUN,f and NIELS BIRBAUMERa,g a
Institute of Medical Psychology and Behavioural Neurobiology, University of Tu¨bingen, Tu¨bingen, Germany Department of Cognitive Sciences and Education, University of Trento, Trento, Italy c Fatronik-Tecnalia Germany, Tu¨bingen, Germany d Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany e Functional and Restorative Neurosurgery Unit, Department of Neurosurgery, and Neuroprosthetic Research Group, Werner Reichardt Centre for Integrative Neuroscience, University of Tu¨bingen, Tu¨bingen, Germany f CIMeC Center for Mind/Brain Sciences, University of Trento, Trento, Italy g Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico, Venezia–Lido, Italy b
Abstract A case of partial recovery after stroke and its associated brain reorganization in a chronic patient after combined brain computer interface (BCI) training and physiotherapy is presented. A multimodal neuroimaging approach based on fMRI and diffusion tensor imaging was used to investigate plasticity of the brain motor system in parallel with longitudinal clinical assessments. A convergent association between functional and structural data in the ipsilesional premotor areas was observed. As a proof of concept investigation, these results encourage further research on a specific role of BCI on brain plasticity and recovery after stroke. Descriptors: BCI, Chronic stroke, fMRI, DTI
ical orthosis affixed to the paralyzed hand, which moves in a hand-grasping or hand-opening fashion, depending upon sensorimotor rhythm activity (Buch et al., 2008). BCI training is hypothesized to induce brain reorganization by contingently linking motor commands to hand movements and thus providing afferent feedback to sensorimotor cortex. Here we report, as a proof of concept study, a case of partial recovery after stroke and its associated brain reorganization in a chronic patient after combined BCI training and physiotherapy. A multimodal neuroimaging approach was used to study functional and structural plasticity. In parallel with the longitudinal clinical assessments, we applied fMRI and psycho-physiological interaction (PPI) analysis to assess functional connectivity and diffusion tensor imaging (DTI) to evaluate the structural integrity of the cortico-spinal tract (CST).
Spontaneous recovery of motor function in chronic stroke patients is reduced and often negligible. Functional recovery mainly occurs in response to intensive therapeutic intervention and rehabilitation such as constrained induced movement therapy, bilateral arm training, or robot-assisted training (Wolf et al., 2006; Prange et al., 2006; Page & Levine, 2007). However, the success of rehabilitation programs in chronic patients with severe hemiparesis is limited. Moreover, most of these treatments rely on the existence of residual hand/arm functionality; hence, patients with no or minimal motor function are excluded. New technologies such as the development of brain computer interfaces (BCI) that utilize neurophysiologic or metabolic brain activity to drive external devices offer promising strategies to modulate neuroplasticity and motor behavior in stroke survivors (Birbaumer & Cohen, 2007; Buch et al., 2008). Recently, a non-invasive BCI based on neurophysiological signals has been proposed to apply in patients with severe motor disability resulting from stroke. Using BCI, chronic stroke patients learned to control a mechan-
Methods Patient The patientFaged 67 years, right handedFsuffered from severe hand paresis as a result of a single unilateral cerebrovascular incident of the right thalamus and the adjacent corticospinal tract of the internal capsule 14 months prior to study entry (Figure 1a). Neglect and motor extinction was excluded by extensive neurological exams and neuropsychological tests. He was not able to extend his fingers or to use his hand/arm for any relevant daily life activity. His paretic arm blocked him from standing up
This work was supported by the Deutsche Forschungsgemeinschaft (DFG SFB 550, DFG GH 94/2-1), Bernstein 01GQ0761, BMBF 16SV3783, European Union (ERC 227632, HUMOUR 231724), and Motorika (Cesarea, Israel). Address correspondence to: Andrea Caria, Institute of Medical Psychology and Behavioural Neurobiology, Eberhard-Karls-University of Tu¨bingen, Gartenstrasse 29, D-72074, Tu¨bingen, Germany. E-mail:
[email protected] 578
A case report of chronic stroke recovery after BCI training
579
Figure 1. a. T2 fluid-attenuated inversion recovery (FLAIR) axial slices. b. Lesion location. The patient had a single unilateral cerebrovascular incident of the right thalamus and the adjacent corticospinal tract of the internal capsule 14 months prior to study entry.
and walking. Physiotherapy was applied to the patient immediately after the stroke, but no substantial improvement of the hand functionality was observed after 14 months. Patient provided written informed consent, and the study was approved by the Ethical Committee of the Faculty of Medicine of the University of Tu¨bingen.
BCI Training The patient underwent two main rehabilitation trainings using magnetoencephalography [4 weeks MEG-BCI between month 14 (S1) and 18 (S2) after stroke] and electroencephalography based BCI [4 weeks EEG-BCI between month 18 and 22 (S3) after stroke] in combination with physiotherapy. MEG-BCI training consisted of 20 sessions, during which the patient learned, using hand/arm motor imagery, to modulate the m rhythm (synchronization and desynchronization) over the ipsilesional sensorimotor area by means of an online visual feedback representing the recorded signal. Depending on the signal amplitude, a mechanical orthosis attached to the paretic hand flexed and extended the patient’s fingers in a hand-grasping or hand-opening fashion (for a detailed description see Buch et al., 2008). Similarly, during EEG-BCI sessions (20), the patient was trained to modulate the m rhythm over the same region, but the learned control drove forward and backward movements of an arm robot (Motorika, Caesarea, Israel). BCI performance was assessed by measuring the proportion of trials in which the patient was successful in producing the requested m rhythm amplitude modulation. MEG-BCI based training increased over sessions from 53.50% to 86.85% (t 5 23.20, po0.001) whereas during EEG-BCI based training the performance remained stable around 76.96% ! 5.78 (mean ! SD).
After each session, the patient underwent 1 h of active and passive physiotherapy. Hand and arm motor functions were assessed during S1, S2, and S3, by using Fugl-Meyer Assessment for the arm (FMA), Wolf Motor Function Test (WMFT) functional ability, Modified Ashworth Scale, and Goal Attainment Score (GAS). The GAS task consisted in taking the cane with him in the paretic hand while grabbing a banister during climbing stairs. Clinical scores indicated a paretic hand/arm motor performance improvement of between 10.8–85.7% (Table 1). Magnetic Resonance Imaging Three fMRI and DTI datasets were collected during S1, S2, and S3, respectively. Neuroimaging data were acquired using a 1.5 Tesla Siemens MRI system. Functional MR images were acquired using a gradient-echo planar imaging (EPI) aligned in axial orientation: TR (repetition time) 5 2000 ms; TE (echo time) 5 46 ms; flip angle 5 901; FOV (field of view) 5 320 mm; matrix size 5 64; interslice gap 5 1 mm; slices 5 22; slice thickness 5 4 mm. A T1-weighted anatomical MR image was acquired using a 1 mm isotropic MPRAGE sequence with the following parameters: TR 5 1300 ms; TE 5 3.19 ms; TI 5 660 ms; flip angle 5 151; FOV 5 256 " 256; matrix size 5 256 " 256; number of slices 5 176; slice thickness 5 1 mm, bandwidth 5 190 Hz/Px (Figure 1B). Diffusion tensor images were acquired using 2 mm isotropic EPI sequence: number of slices 5 65; orientation 5 transversal; phase-encoding direction A4P; slice thickness 5 2 mm; TR 5 7500 ms; TE 5 71 ms; averages 5 3; b-value 5 800 s/mm2; dimensions 5 256 mm " 256 mm " 130 mm; EPI factor 5 128; bandwidth 5 1502 Hz/Px; noise level 5 40; number of diffusion directions 5 6. In order to reduce movements, two foam cushions immobilized the participant’s head.
Table 1. Clinical Assessment Motor S1 (baseline) S2 S3 % change from baseline
FMA passive
FMA sensory
FMA motor
WMFT functional ability
MAS
GAS
37 39 41 10.8
5 5 6 20
13 19 24 84.6
7 9 13 85.7
8 4 4 50
#2 12 #1
Note: FMA (passive movement and pain: 0 5 maximum disability, 48 5 normal; sensory loss: 0 5 maximum disability, 12 5 normal; motor function: 0 5 maximum disability, 66 5 normal), WMFT (0 5 maximum disability; 80 5 normal), MAS (spasticity, 0 5 normal; 36 5 maximum disability), GAS (# 2 5 outcome much less than expected, 0 5 program goal/expected outcome; 2 5 outcome much better than expected; as the patient scored 12 in S2, the S3 baseline was reset to allow detection of a further improvement).
580 fMRI Design and Analysis Each fMRI session consisted of four runs (190 volumes) of visually and auditorily cued, executed and imagined flexion– extension of the fingers (12 s), with either the affected (left) or the unaffected (right) hand, alternating with rest (12 s). fMRI data analysis was performed using SPM5. EPI volumes of the three fMRI sessions were realigned, slicetime corrected, anatomically coregistered, spatially normalized to the Montreal Neurological Institute (MNI) reference space, and smoothed (9-mm). Hemodynamic response amplitudes were estimated using standard regressors, constructed by convolving a boxcar function, for each of the three different conditions (actual movement, imagined movement, and rest), with a canonical hemodynamic response function using standard SPM5 parameters. The time series in each voxel were high-pass filtered at 1/128 Khz to remove low frequency drifts. Movement parameters were also included into the general linear model (GLM) as covariates to take into account head motion artefacts. Voxels were identified as significant if they surpassed a threshold of Po.001 family-wise error (FWE) corrected for multiple comparisons. Small volume correction analysis based on anatomically selected regions, MI, PMC, SMA SI, SII, and the cerebellum, allowed us to calculate, for each region of interest (ROI), a lateralization index (LI) during left and right hand movements. LI, expressed as the normalized difference between the number of active voxels in the left and the right hemisphere, approaches a value of 1 or ! 1 when the activity was either purely contralesional or ipsilesional. PPI Analysis Motor network reorganization was assessed using PPI analysis as implemented in SPM5. PPI is a simple fMRI-based brain connectivity method that determines whether a given region, the seed/ source, ‘predicts’ the activity in other brain regions as a function of a task/context specific factor with no need of anatomical pathways to be specified. In particular, it calculates the change in the interregional covariance using the difference in regression coefficients between the neuroimaging signals from two brain regions. Recently, Kim and colleagues demonstrated that fMRI based PPI analysis produces reliable results reflecting the underlying changes in neural interactions (Kim & Horwitz, 2008). The first eigenvector of a sphere of 6 mm radius centered on the right dorsal PMC was selected as a ‘seed’ region. This area was identified based on the peak maxima measured when the executed movement condition of the paretic hand in S3 was compared to S1. The ipsilesional PMC’s modulatory effect on the remaining brain areas was assessed over time. In order to consider contextual connectivity changes, PPI analysis (Po.05 FWE) was conducted comparing actual to imagined movement condition. PPI analysis was also conducted using a size/coordinates matched volume of interest in the healthy hemisphere corresponding to the left PMC for the task being performed with the healthy hand. We expected to observe no significant changes of the interregional covariance over sessions. DTI Analysis Diffusion-weighted raw data were first corrected for eddy current distortions and motion artifacts using the FMRIB Diffusion Toolbox (FSL). Diffusion images from multiple sessions were then realigned and co-registered to the skull-stripped TL-image. The diffusion tensor and fractional anisotropy (FA) maps were calculated using MedINRIA software. Tractography was carried out
A. Caria et al. using Diffusion Toolkit and TrackVis. Fiber tracking was performed using the Interpolated Streamline algorithm with a steplength of 0.5 mm and was terminated if FA was less than 0.15 or the tract angle between successive steps was greater than 351. Results fMRI Whole brain analysis revealed primarily contralateral activity in the primary sensorimotor areas (SMI) and in the secondary motor areas when the task was performed with the healthy hand (MNI peak maxima S1, S2, S3: (BA3, x, y, z 5 ! 42, ! 27, 60, t(708) 5 31.35; 35.60; 27.72; Table 2). These results were consistent over sessions, and no significant difference was measured over sessions. Highly distributed and bilateral brain activity was observed during the affected hand task (MNI peak maxima S1: BA40, x, y, z 5 45, ! 39, 60, t(708) 5 16.89; S2: BA40, x, y, z 5 45, ! 39, 60, t(708) 5 17.13; S3: BA6, x, y, z 5 30, ! 15, 68, t(708) 5 15.94; Table 2). A significant difference in the secondary motor areas, specifically in the bilateral SMA (BA6, x, y, z 5 ! 9, 27, 44, t(3540) 5 8.66; BA6, x, y, z 5 6, ! 12, 52, t(3540) 5 7.23) and in the ipsilesional PMC (BA6, x, y, z 5 36, ! 12, 60, t(3540) 5 6.85), was observed when the executed movement condition in S3 was compared to S1 (Figure 2a). The activity during motor imagery performed either with the healthy or the paretic hand was bilateral with a bias toward the contralateral regions, and no statistical difference emerged over sessions. Healthy and paretic hand movements were controlled during each condition: visually during S1 and S2 and in addition by parallel electromyographic (EMG) signal acquisition during S3. No movements were observed and recorded with EMG in S3 during motor imagery and rest condition, and no mirror movements or involuntary co-contraction were observed or measured during the actual movement condition. ROI analysis indicated contralateral activity over all sessions during the healthy hand task. The paretic hand task was associated with bilateral SMI activity shifted to the contralesional hemisphere during S1 and to the ipsilesional hemisphere during S2 and S3. The PMC showed a noteworthy reallocation to the ipsilesional side from S1 (LI 5 0.76) to S3 (LI 5 ! 0.10) (Figure 2c). Both whole brain and ROI analyses showed increased activity in the ipsilesional and contralesional SMI and an extensive bilateral recruitment of the PMC and SMA during S1 when the task was instructed to be performed with the paretic hand. During S3 a more focal bilateral activity was found, probably the result of less effort involved as a consequence of automatization and improved mastery of the skill, or both. Additionally, a shift to the ipsilesional hemisphere, quite considerable for the PMC, was observed. Functional Connectivity During left paretic hand movements, the right PMC activity positively co-varied with the ipsilesional primary and secondary sensorimotor regions across all sessions, with visual areas in S1 and S2 and with the contralesional secondary sensorimotor cortex during S3 (MNI peaks maxima S1: BA17, x, y, z 5 ! 3, ! 87, ! 12, t(708) 5 7.40; S2: BA18, x, y, z 5 9, ! 84, ! 16, t(708) 5 4.97; S3: BA40, x, y, z 5 ! 48, ! 39, 60, t(708) 5 8.93); Table 2 (see also Figure 2B). Left PMC showed a stable positive covariation with the left sensory and motor areas over all sessions when the task was performed with the right non-paretic hand (MNI peak maximum S1: BA3, x, y, z 5 ! 42, ! 30, 60, t(708) 5 11.02; S2: BA3, x, y, z 5 ! 42, ! 27, 60, t(708) 5 8.04; S3: BA3, x, y, z 5 ! 42, ! 30, 60, t(708) 5 9.79; Table 2).
A case report of chronic stroke recovery after BCI training
581
Table 2. fMRI and PPI Results During S1 and S3 Activated areas
fMRI results Healthy S1 Contralesional SI (BA3) MI (BA4) PMC (BA6 SMA Ipsilesional SII (BA40) PMC (BA6 Cerebellum Vermis S3 Contralesional SI (BA3) MI (BA4) PMC (BA6 SM Ipsilesional SII (BA40) MI (BA4) PMC (BA6) Cerebellum Paretic S1 Contralesional SII (BA40) MI (BA4) PMC (BA6) Cerebellum SMA Ipsilesional SII (BA40) MI (BA4) PMC (BA6) Cerebellum V2 (BA18) S3 Contralesional SII (BA40) MI (BA4) PMC (BA6) Cerebellum Ipsilesional SII (BA40) MI (BA4) PMC (BA6) Cerebellum SMA PPI results Healthy S1 Contralesional SMI (BA3) V2 (BA17) S3 Contralesional SMI (BA3) Paretic S1 Contralesional V2 (BA17) Mid front (BA8) Ipsilesional Thalamus SMA (BA8,6) Sup par gyrus (BA7) SMI (BA3) S3 Contralesional SII (BA40) Ipsilesional SMA (BA6) SMI (BA1,2) SMI (BA1,2)
MNI coordinates (x, y, z)
t-value
" 42, " 27, 60 " 36, " 24, 64 " 30, " 12, 60 " 3, " 12, 48
31.35 22.31 11.21 12.47
46, " 36, 60 30, " 12, 64 15, " 54, " 16 0, " 63, " 12
12.00 7.49 16.53 11.18
" 42, " 36, " 30, " 3,
" 27, 60 " 24, 64 " 12, 60 " 9, 52
27.72 22.65 7.95 10.23
42, " 39, 60 36, " 24, 64 33, " 15, 68 12, " 54, " 12
10.55 6.88 8.34 14.62
" 45, " 42, 56 " 36, " 24, 64 " 39, " 9, 56 " 30, " 63, " 20 " 3, " 15, 64
11.06 8.35 6.15 5.81 15.16
45, " 39, 60 39, " 24, 64 39, " 9, 56 30, " 57, " 24 3, " 102, 4
16.89 10.58 8.59 4.86 15.28
" 45, " 45, 60 " 36, " 27, 60 " 27, " 15, 64 " 30, " 63, " 20
12.19 7.08 7.22 5.30
42, " 42, 60 39, " 24, 64 30, " 15, 68 30, " 57, " 24 0, " 15, 56
15.09 15.64 15.94 5.96 14.97
" 42, " 30, 60 0, " 84, " 16
11.02 6.02
" 42, " 30, 60
9.79
" 3, " 87, " 12 " 33, 18, 48
7.40 7.13
0, " 15, 12 3, 15, 56 33, " 63, 60 39, " 36, 68
6.99 5.37 6.51 5.78
" 48, " 39, 60
8.93
0, " 12, 64 54, " 27, 48 39, " 42, 64
7.23 6.92 6.59
Figure 2. fMRI and PPI results during the paretic hand motor task execution. a. Activated areas during S1, S3, and comparing S3 to S1. b. Areas covarying with the right ipsilesional premotor cortex. c. Lateralization index (LI).
DTI DTI analysis and tractography were applied to assess the CST integrity and to localize potential preserved descending pathways. Using S1 data, a smaller number of fibers was reconstructed in the ipsilesional hemisphere (n 5 8) compared to the contralesional side (n 5 667). CST reconstruction overlaid on the MRI structural image disclosed fibers descending from the ipsilesional premotor region (Figure 3). FA index of a perilesional portion of the CST and of a corresponding area in the contralesional hemisphere was calculated over the three sessions. FA, representing on a scale from 0 to 1 the extent to which myelin sheets constrain diffusion of water molecules along a specific direction, provides an index of the tissue microstucture’s integrity. Fa, being influenced by several factors, such as axonal myelination, fibers’ diameter, density, and orientation, decreases when the CST is disrupted, and it indicates the degree of axonal loss and Wallerian degeneration (M!ller et al., 2007). A number of studies emphasized the CST integrity as a predictor of cortical reorganization and motor recovery in chronic stroke (Stinear et al., 2007). Voxelwise analysis (n 5 97) showed increased FA values in the perilesional region of the CST from 0.20 ! 0.12 (mean ! SD, S1) to 0.24 ! 0.12 (S3) (S3 vs S1 t 5 9.51 po.001) but not in the contralesional side: 0.75 ! 0.16 (S1), 0.74 ! 0.16 (S3) (S3 vs S1 ns). Recent studies on animal models demonstrated that FA is a good measure of white matter reorganization (Chen et al., 2002). Increased FA values in the perilesional area, as index of augmented density and directionality of myelinated fiber tracts, indicate a better outcome of neurological function after stroke (Schaechter et al., 2009).
582
A. Caria et al.
Figure 3. DTI based reconstruction of the ipsilesional (green) and contralesional (red) CST fibers superimposed on the T1 MPRAGE.
Discussion Although specific and individual effects of the different rehabilitative methods applied are not possible to discern, a significant and clinically substantial recovery of the hand motor function was observed over time. We believe this finding to have relevant implications for rehabilitation of chronic patients with no or minimal residual hand movements, who generally do not show significant improvements after rehabilitation programs and often are not eligible for physical therapies. The clinical outcome, as assessed by the Fugl-Meyer test, was higher than that obtained by chronic stroke patients with moderate and severe upper limb impairment after robot-aided therapy (Prange et al., 2006), and it
was twice the effect of 10 weeks of modified constraint-induced therapy in chronic stroke patients with minimal movement ability in the affected arm (Page & Levine, 2007). A similar improvement was reported after 4 weeks of computerized arm training in subacute stroke patients with severe upper limb paresis (Hesse et al., 2005). Therefore, the results of the proposed approach based on a combination of BCI training and physiotherapy, which applies to chronic patients with severe hand paresis, seem promising and call for a more structured and controlled clinical trial. Overall fMRI data reveal an increased lateralization toward the ispilesional sensorimotor regions, specifically, an enhanced activity in the dorsal premotor region and supplementary motor area after the training. These results are in line with previous studies indicating the adaptive and compensatory role of the ipsilesional PMC in hand movement after stroke and its participation as a substrate mediating functional recovery of executive motor function (Fridman et al., 2004). The evidence of preserved anterior fibers of the CST in the anterior part of the internal capsule originating either from anterior parts of MI or from the PMC may constitute the anatomical prerequisite for the observed cortical reorganization and behavioral improvement over time. Furthermore, structural analysis of the motor pathways indicated a small but significant increase of the fractional anisotropy in the ipsilesional CST, potentially indicating white matter reorganization. This finding would be in line with the postulated mechanisms involved in long-term plasticity (Chen, Cohen, & Hallett, 2002) and results in animal models and humans (Chen et al., 2002; Schaechter et al., 2009). However, further group studies on patients and controls are necessary to test these hypotheses. As a proof of concept investigation, these results encourage further research on a specific role of BCI on brain plasticity and recovery after stroke. Finally, our findings suggest the importance of fMRI and DTI investigations for tailoring specific rehabilitation programs in chronic stroke patients based on preserved functionality of the motor system.
REFERENCES Birbaumer, N., & Cohen, L. (2007). Brain-computer interfaces: Communication and restoration of movement in paralysis. Journal of Physiology, 579, 621–636. Buch, E., Weber, C., Cohen, L. G., Braun, C., Dimyan, M. A., Ard, T., et al. (2008). Think to move: A neuromagnetic BCI system for chronic stroke. Stroke, 39, 910–917. Chen, P., Goldberg, D. E., Kolb, B., Lanser, M., & Benowitz, L. I. (2002). Inosine induced axonal rewiring and improves behavioral outcome after stroke. Proceedings of the National Academy of Sciences USA, 99, 9031–9036. Chen, R., Cohen, L. G., & Hallett, M. (2002). Nervous system reorganization following injury. Neuroscience, 111, 761–773. Fridman, E. A., Hanakawa, T., Chung, M., Hummel, F., Leiguarda, L. C., & Cohen, L. G. (2004). Reorganization of the human ipsilesional premotor cortex after stroke. Brain, 127, 747–758. Hesse, S., Werner, C., Pohl, M., Rueckriem, S., Mehrholz, J., & Lingnau, M. L. (2005). Computerized arm training improves the motor control of the severely affected arm after stroke: A single-blinded randomized trial in two centers. Stroke, 36, 1960–1966. Kim, J., & Horwitz, B. (2008). Investigating the neural basis for fMRIbased functional connectivity in a blocked design: Application to interregional correlations and psycho-physiological interactions. NeuroImage, 26, 583–593. M!ller, M., Frandsen, J., Andersen, G., Gjedde, A., VestergaardPoulsen, P., & Østergaard, L. (2007). Dynamic changes of cortico-
spinal tracts after stroke detected by fibertracking. Journal of Neurological and Neurosurgical Psychiatry, 78, 587–592. Page, S. J., & Levine, P. (2007). Modified constraint-induced therapy in patients with chronic stroke exhibiting minimal movement ability in the affected arm. Physical Therapy, 87, 872–878. Prange, G. B., Jannink, M. J. A., Groothuis-Oudshoorn, C. G. M., Hermens, H. J., & Ijzerman, M. J. (2006). Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke. Journal of Rehabilitation Research and Development, 43, 171–184. Schaechter, J. D., Fricker, Z. P., Perdue, K. L., Helmer, K. G., Vangel, M. G., Greve, D. N., & Makris, N. (2009). Microstructural status of ipsilesional and contralesional corticospinal tract correlates with motor skill in chronic stroke patients. Human Brain Mapping, 30, 3461–3474. Stinear, C. M., Barber, P. A., Smale, P. R., Coxon, J. P., Fleming, M. K., & Byblow, W. D. (2007). Functional potential in chronic stroke patients depends on corticospinal tract integrity. Brain, 130, 170–180. Wolf, S. L., Winstein, C. J., Miller, J. P., Taub, E., Uswatte, G., Morris, D., et al. (2006). Effect of constraint-induced movement therapy on upper extremity function 3 to 9 months after stroke: The EXCITE randomized clinical trial. JAMA, 296, 2095–2104.
(Received January 15, 2010; Accepted June 21, 2010)
