Handbook of Psychological and Educational Assessment of Children: Personality, Behavior, and Context, 2nd Edition

reyn2-fm.qxd

6/6/2003

8:57 AM

Page i

HANDBOOK OF PSYCHOLOGICAL AND EDUCATIONAL ASSESSMENT OF CHILDREN Personality, Behavior, and Context

reyn2-fm.qxd

6/6/2003

8:57 AM

Page ii

This page intentionally left blank

reyn2-fm.qxd

6/6/2003

8:57 AM

Page iii

HANDBOOK OF PSYCHOLOGICAL AND EDUCATIONAL ASSESSMENT OF CHILDREN Personality, Behavior, and Context Second Edition

Edited by

Cecil R. Reynolds Randy W. Kamphaus

THE GUILFORD PRESS New York London

reyn2-fm.qxd

6/6/2003

8:57 AM

Page iv

© 2003 The Guilford Press A Division of Guilford Publications, Inc. 72 Spring Street, New York, NY 10012 www.guilford.com All rights reserved No part of this book may be reproduced, translated, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without written permission from the Publisher. Printed in the United States of America This book is printed on acid-free paper. Last digit is print number: 9 8 7 6 5 4 3 2 1

Library of Congress Cataloging-in-Publication Data Handbook of psychological and educational assessment of children : personality, behavior, and context / edited by Cecil R. Reynolds and Randy W. Kamphaus.— 2nd ed. p. cm. Includes bibliographical references and index. ISBN 1-57230-884-2 (alk. paper) 1. Psychological tests for children. 2. Achievement tests. I. Reynolds, Cecil R., 1952– II. Kamphaus, Randy W. BF722.H33 2003b 155.4⬘028⬘7—dc21 2003005957

reyn2-fm.qxd

6/6/2003

8:57 AM

Page v

This work is dedicated in recognition of the lifetime of work in the assessment, identification, and programming for gifted and talented children of Dr. E. Paul Torrance. An extraordinary scholar, pioneer, and mentor, his work has forever changed the landscape of gifted and talented education and how we think about children in general.

reyn2-fm.qxd

6/6/2003

8:57 AM

Page vi

This page intentionally left blank

reyn2-fm.qxd

6/6/2003

8:57 AM

Page vii

About the Editors

Cecil R. Reynolds, PhD, ABPN, ABPP, is Professor of Educational Psychology, Professor of Neuroscience, and a Distinguished Research Scholar at Texas A&M University. His primary research interests are in all aspects of psychological assessment, with particular emphasis on assessment of memory, emotional and affective states and traits, and issues of cultural bias in testing. Dr. Reynolds is the author of more than 300 scholarly publications and author or editor of 35 books, including The Clinician’s Guide to the Behavior Assessment System for Children (2002, Guilford Press), Clinical Applications of Continuous Performance Tests (2001, Wiley), Handbook of School Psychology (1999, Wiley), the Encyclopedia of Special Education (2000, Wiley), and the Handbook of Clinical Child Neuropsychology (1997, Plenum Press). He is the author of several widely used tests of personality and behavior, including the Behavior Assessment System for Children and the Revised Children’s Manifest Anxiety Scale. Dr. Reynolds is also senior author of the Reynolds Intellectual Assessment Scales, the Test of Memory and Learning, the Clinical Assessment Scales for the Elderly, and the forthcoming Elderly Memory Schedule, as well as coauthor of several computerized test interpretation systems. He has a clinical practice in Bastrop, Texas, where he treats trauma victims and individuals with traumatic brain injury. Randy W. Kamphaus, PhD, is Professor and Head of the Department of Educational Psychology at the University of Georgia. A focus on issues related to clinical assessment has led him to pursue research in classification methods, differential diagnosis, test development, and learning disability and attention-deficit/hyperactivity disorder (ADHD) assessment. Dr. Kamphaus has served as principal investigator, coinvestigator, or consultant on federally funded research projects dealing with early intervention and prevention, child classification methods, prevalency of ADHD and conduct disorder in Latin America, and aggression reduction in schools. As a licensed psychologist and a Fellow of the American Psychological Association (APA), he has contributed extensively to his profession, and he is past president of the APA’s Division of School Psychology. Dr. Kamphaus has also authored or coauthored five books, three psychological tests, more than 40 scientific journal articles, and more than 20 book chapters. He also participates in scholarship in the field through work as an editorial board member, associate editor, test reviewer, and newsletter editor.

vii

reyn2-fm.qxd

6/6/2003

8:57 AM

Page viii

This page intentionally left blank

reyn2-fm.qxd

6/6/2003

8:57 AM

Page ix

Contributors

Thomas M. Achenbach, PhD, Department of Psychiatry, University of Vermont, Burlington, Vermont Jerry C. Allen, PhD, Department of Educational Psychology, University of Georgia, Athens, Georgia Heather Applegate, PhD, Department of Psychiatry, University of Mississippi Medical Center, Jackson, Mississippi David W. Barnett, PhD, College of Education, University of Cincinnati, Cincinnati, Ohio Laura Bennett, MA, Department of Educational Psychology, University of Texas at Austin, Austin, Texas Karen L. Bierman, PhD, Children, Youth, and Families Consortium, Pennsylvania State University, University Park, Pennsylvania Patrice H. Butterfield, PhD, FPPR, FSMI, private practice, Savannah, Georgia Cindy I. Carlson, PhD, Department of Educational Psychology, College of Education, University of Texas at Austin, Austin, Texas Timothy A. Cavell, PhD, Department of Psychology, University of Arkansas, Fayetteville, Arkansas Louis A. Chandler, PhD, Department of Psychology in Education, University of Pittsburgh, Pittsburgh, Pennsylvania Ron Drabman, PhD, ABPP, Department of Psychiatry, University of Mississippi Medical Center, Jackson, Mississippi James L. Dupree, PhD, Department of Psychology, Humboldt State University, Arcata, California Samuel E. Fiala, MS, Department of Psychology, Texas A&M University, College Station, Texas Jennifer T. Freeland, PhD, Department of Educational and School Psychology, Indiana State University, Terre Haute, Indiana Paul J. Frick, PhD, Department of Psychology, University of New Orleans, New Orleans, Louisiana Christian P. Gruber, PhD, Western Psychological Services, Los Angeles, California ix

reyn2-fm.qxd

x

6/6/2003

8:57 AM

Page x

Contributors

Laura Guli, BA, Department of Educational Psychology, University of Texas at Austin, Austin, Texas Ruth Adlof Haak, PhD, Balcones Special Services Cooperative, Austin, Texas Jillayne Hollifield, PhD, Dorothea Dix Hospital, Raleigh, North Carolina Wayne H. Holtzman, PhD, Hogg Foundation for Mental Health, University of Texas at Austin, Austin, Texas Randy W. Kamphaus, PhD, Department of Educational Psychology, University of Georgia, Athens, Georgia Howard M. Knoff, PhD, National School Improvement, Tampa, Florida David Lachar, PhD, Department of Psychiatry and Behavioral Science, University of Texas– Houston Medical School, Houston, Texas Francis E. Lentz, Jr., PhD, College of Education, University of Cincinnati, Cincinnati, Ohio Bryan R. Loney, PhD, Department of Psychology, Florida State University, Tallahassee, Florida Gregg M. Macmann, PhD, Department of Educational and Counseling Psychology, University of Kentucky, Lexington, Kentucky Stephanie H. McConaughy, PhD, Department of Psychiatry, University of Vermont, Burlington, Vermont Barbara T. Meehan, MS, Department of Psychology, Texas A&M University, College Station, Texas Shari Neul, PhD, Learning Support Center, Texas Children’s Hospital, Houston, Texas Frances Prevatt, PhD, School Psychology Program, Department of Human Services, Florida State University, Tallahassee, Florida Cecil R. Reynolds, PhD, Department of Educational Psychology, College of Education, Texas A&M University, College Station, Texas Cynthia A. Riccio, PhD, Department of Educational Psychology, College of Education, Texas A&M University, College Station, Texas Kelly Robinson, MS, Department of Educational Psychology, University of Georgia, Athens, Georgia Ellen W. Rowe, MA, School Psychology Program, University of Georgia, Athens, Georgia Margaret Semrud-Clikeman, PhD, Department of Educational Psychology, University of Texas at Austin, Austin, Texas Edward S. Shapiro, PhD, Department of Education and Human Services, Lehigh University, Bethlehem, Pennsylvania Christopher H. Skinner, PhD, Department of Educational Psychology, College of Education, University of Tennessee, Knoxville, Tennessee Jon D. Swartz, PhD, private practice, Georgetown, Texas Jennifer Thorpe, MEd, Department of Educational Psychology, University of Georgia, Athens, Georgia Janet A. Welsh, PhD, FAST Track Project, Pennsylvania State University, University Park, Pennsylvania Anne Pierce Winsor, MS, School Psychology Program, University of Georgia, Athens, Georgia

reyn2-fm.qxd

6/6/2003

8:57 AM

Page xi

Preface

The general area of psychological testing and assessment continues to be, as it has been for decades, the most prolific of research areas in psychology, as is evident by its representation in psychological journals. Although always controversial, psychological testing has nevertheless grown in its application to include evaluation and treatment of children’s disorders of development, learning, and behavior. Tests continue to be published at an increasing rate. The scholarly literature on psychological testing of children has grown significantly over the past three decades and is rapidly becoming unmanageable. More than 40 different scholarly, refereed journals exist in North America alone that publish articles on psychological and educational assessment of children, making the task of the professor, the student, and the practitioner seem an impossible one. Hence, periodic comprehensive reviews of this massive literature seem necessary, albeit onerous. Such tasks require the work and the thoughts of many esteemed authors. In undertaking this task in the first edition, we endeavored to devise a work suitable for the professor as a reference, the student as a text, and the practitioner as a sourcebook and guide. In order to do this effectively, it seemed reasonable to separate the two major areas of assessment—intelligence and personality—into their own volumes. We have continued with this practice based upon the success in the first editions. This approach has allowed us the space for in-depth coverage, while retaining cohesion of topics in each book. The two volumes can thus be used in tandem or as separate units, depending on need. Our hope for this two-volume handbook was to develop a broad-based resource for those individuals who are charged with the assessment of children and adolescents. We also wanted to develop a comprehensive resource for researchers who are studying various aspects of children’s assessment and psychodiagnostics, and to provide breadth and depth of coverage of the major domains of children’s assessment in a single source. These volumes include such diverse areas as academic achievement, intelligence, adaptive behavior, personality, and creativity assessment. Individual tests, such as the Wechsler Intelligence Scale for Children—Third Edition, the Kaufman Assessment Battery for Children, and the Rorschach, are given their own treatments, in addition to some general methods such as projective storytelling techniques. In each volume, the theoretical foundations and the measurement limitations of current approaches to the assessment of these latent constructs are addressed. In order to ensure the volumes are authoritative, we sought out eminent scholars with a general command of assessment and a special expertise in research or practice in the area of their respective contributions. We have also sought new scholars, perhaps less well established, but whose thinking is clear, strong, and challenging on several fronts in formulating the second edition of each work. The chapters themselves purposely vary from an emphasis on specific applications in assessment to cutting-edge knowledge and critiques of research and statistical procedures. We hope that this scholarly emphasis will enhance the possibility of using the second edition of this two-volume handbook as a graduate-level text as it was so xi

reyn2-fm.qxd

xii

6/6/2003

8:57 AM

Page xii

Preface

frequently adopted in its first edition. Because of its breadth, we think this text could be useful for courses in intellectual and personality assessment, practica and internship coursework, and courses on psychodiagnostics, psychopathology, and special education. In the second edition, several chapters were added, a few deleted, and all but one revised to greater and lesser extents. Chapters on new instruments not in existence for the first edition, such as the Behavior Assessment System for Children and the Kaufman Adolescent and Adult Intelligence Test, are now treated in depth. We estimate the typical chapter contains one-third new material and some much more. With the various additions and deletions, more than half of the work is new. We intended to retain the best of the material from the first edition, and revise and update where new research and science so dictated. We are deeply indebted to a number of individuals for assisting us with this at times overwhelming project. First of all, we wish to thank the authors of the various chapters for their extraordinary talent and patience with this arduous effort. We wish them continued success in all of their professional activities. We owe a great debt to Sharon Panulla, our original editor at The Guilford Press who signed the first edition of this work, and her successor, Chris Jennison, who followed the second edition through to its completion. We greatly appreciate their faith in giving us the opportunity to produce this work. We also thank Editor-in-Chief Seymour Weingarten for his concurrence, as well as his early thoughts on the organization and development of the first edition of this work. We are very appreciative of the efforts of our staff and students, especially Justine Hair, who assisted us in many ways through their organizational contributions and many trips to the library! Finally, we wish to thank all of the researchers of the last century dating back to and including Sir Frances Galton and his modern-day counterpart, Arthur Jensen, as well as such clear thinkers in the field as John Carroll, John Horn, Anne Anastasi, and Raymond Cattell, for the great strides they have made in enhancing our ability to measure and consequently understand the nature of human behavior. To our common mentor, Alan S. Kaufman, we acknowledge a continuing debt for the superb model of scholarship that he continues to provide. However, it is to Julia and to Norma that we owe our greatest debts of gratitude. The strength they lend, the understanding they convey, and the support they give make our onerous schedules tolerable, and enable us to be so much more than we would be without them— thank you, again. CECIL R. REYNOLDS RANDY W. KAMPHAUS

reyn2-fm.qxd

6/6/2003

8:57 AM

Page xiii

Contents

I. GENERAL ISSUES 1. Personality Assessment Research: Applying Criteria of Confidence and Helpfulness David W. Barnett, Gregg M. Macmann, and Francis E. Lentz, Jr. 2. Procedural Issues Associated with the Behavioral Assessment of Children Christopher H. Skinner, Jennifer T. Freeland, and Edward S. Shapiro

3

30

II. PROJECTIVE METHODS 3. The Projective Hypothesis and the Development of Projective Techniques for Children Louis A. Chandler

51

4. Projective Storytelling Techniques James L. Dupree and Frances Prevatt

66

5. Evaluation of Projective Drawings Howard M. Knoff

91

6. The Sentence Completion as a Tool for Assessing Emotional Disturbance Ruth Adlof Haak

159

7. Using the Rorschach with Children and Adolescents: The Exner Comprehensive System Jerry C. Allen and Jillayne Hollifield

182

8. Use of the Holtzman Inkblot Technique with Children Wayne H. Holtzman and Jon D. Swartz

198

xiii

reyn2-fm.qxd

6/6/2003

8:57 AM

Page xiv

xiv

Contents

III. INTERVIEWING AND OBSERVATIONS 9. Using the Clinical Interview to Assess Children’s Interpersonal Reasoning and Emotional Understanding Janet A. Welsh and Karen L. Bierman

219

10. Structured Diagnostic Interviewing Bryan R. Loney and Paul J. Frick

235

11. Direct Behavioral Observation for Classrooms Anne Pierce Winsor

248

IV. SPECIFIC SYNDROMES AND SYMPTOMS 12. Assessment of Childhood Depression Margaret Semrud-Clikeman, Laura Bennett, and Laura Guli

259

13. The Assessment of Attention via Continuous Performance Tests Cynthia A. Riccio and Cecil R. Reynolds

291

14. Assessment of Attention-Deficit/Hyperactivity Disorder Shari Neul, Heather Applegate, and Ron Drabman

320

V. OBJECTIVE METHODS 15. Multisource and Multidimensional Object Assessment of Adjustment: The Personality Inventory for Children, Second Edition; Personality Inventory for Youth; and Student Behavior Survey David Lachar and Christian P. Gruber

337

16. The Minnesota Multiphasic Personality Inventory—Adolescent Ellen W. Rowe

368

17. The Behavior Assessment System for Children Jennifer Thorpe, Randy W. Kamphaus, and Cecil R. Reynolds

387

18. The Achenbach System of Empirically Based Assessment Thomas M. Achenbach and Stephanie H. McConaughy

406

VI. ASSESSMENT OF ADAPTIVE SKILLS/BEHAVIOR 19. Assessing Social Competence in Children and Adolescents Timothy A. Cavell, Barbara T. Meehan, and Samuel E. Fiala

433

20. Adaptive Behavior Scales Randy W. Kamphaus

455

reyn2-fm.qxd

6/6/2003

8:57 AM

Page xv

Contents

xv

VII. ADVANCED TOPICS 21. Assessing the Family Context Cindy I. Carlson

473

22. Issues in Child Custody Evaluation and Testimony Patrice H. Butterfield

493

23. Assessment of Childhood Anxiety Kelly Robinson

508

Index

527

reyn2-fm.qxd

6/6/2003

8:57 AM

Page xvi

This page intentionally left blank

reyn2-1.qxd

6/6/2003

8:59 AM

Page 1

PART I

GENERAL ISSUES

reyn2-1.qxd

6/6/2003

8:59 AM

Page 2

This page intentionally left blank

reyn2-1.qxd

6/6/2003

8:59 AM

Page 3

1 Personality Assessment Research: Applying Criteria of Confidence and Helpfulness

DAVID W. BARNETT GREGG M. MACMANN FRANCIS E. LENTZ, JR.

Personality assessment has had a tentative status within professional psychology for the past several decades and, as we indicated in the last edition, an important question for many professionals in light of the difficulties is “Why hold on?” Our answer was not optimistic—not with regard to the construct of personality per se but to the professional uses of constructs typically associated with personality measurement. The proliferation of seemingly shallow approaches to assessment and intervention is cause for concern, and perhaps all would agree that the ultimate criteria for the evaluation of clinical and educational decisions should rest on enhanced personal and social development (i.e., Seligman & Csikszentmihalyi, 2000) rather than the more easily measurable objectives ordinarily associated with test development. An underlying theme of the earlier chapter was the need for new research consistent with the goal of defining “an epistemology of practice” (Schön, 1983)—research that involves the analysis of successful practice in real-life situations. Maintaining that emphasis in this chapter, we offer an opinion about the kinds of research information that would be helpful in developing professional practices for the assessment of child and

adolescent personality. However, in a natural evolution of our views in regard to the most appropriate criteria for personality measurement, our current discussion of the issues is framed in terms of desirable measurement qualities for professional practice, namely, decision confidence and helpfulness in natural setting contexts.

AN OVERVIEW OF THE CHAPTER Despite the enormous controversies, personal and social development remains the broadest context for assessment and intervention design that serves to organize an array of professional practices involving clinical, educational, and vocational decisions about problems of children and adolescents. Guiding principles are clearly needed to enable effective outcomes in the difficult and ambiguous situations in which professionals need to apply the tools of their trade with confidence. However, almost since their inception, personality measures have been subject to extensive criticism as summarized in the following paragraphs. First, the difficulties in “knowing” the personality of an individual are fundamental 3

reyn2-1.qxd

4

6/6/2003

8:59 AM

Page 4

I. GENERAL ISSUES

and raise serious questions related to the degree of confidence that can be justifiably entertained in making decisions during problem solving—a process limited in part by error rates as well as by meaningful predictions. Competing theories and measurement differences make any consensus about appropriate assessment practices among professionals unlikely. The crucial procedure for practice becomes one of empirically demonstrating useful information through probabilistic statements—a question of helpfulness in decisions. However, the need to account for “person” variables is unassailable. This is an important distinction we return to later. Second, personality assessment for children has special difficulties (again, both theoretical and practical). The issues are made evident by research that deals with the continuity of personal and social development and the processes of psychosocial change. Evidence exists for both the coherence and continuity of development; yet the consistencies described confidently for groups are not easily defined or measured for individuals and individuals may change significantly (Block & Block, 1980; Lewis, 1997; Moss & Sussman, 1980). When viewed from the perspective of individual development, Emmerich’s (1966) review has stood the test of time: “Impressive as the evidence for the early determination of personality may seem, there are also grounds for believing that personality ordinarily remains open to change over extended periods” (p. 237). The task for professional practice is one of enhancing the developmental trajectory of high-risk or challenging children, and surprisingly little is known about the process. A major goal of personality research has been the identification of traits or person variables, or pathological markers, that would improve the selection of specific and effective treatments. Such attributes remain elusive thus far, and individual applications are tenuous, based on understanding problem situations. Intervention design is in its infancy, and the course of personal and social development is complexly determined by psychological, biological, and accidental events (Bandura, 1986; Plomin, DeFries, McClearn, & Rutter, 1997). A third issue has been the debate concerning the relative importance of situational versus personal determinants of behavior. The

focal points have been the obvious control that situations exert on behavior, and the relatively modest predictive power of many person variables. War was waged over this issue in the 1960s (Mischel, 1968; Peterson, 1968), but most have assumed an interactionist position (Magnusson & Endler, 1977; Plomin et al., 1997). Perhaps the debate has been put to rest: “Behavior, cognitive and other personal factors, and environmental influences all operate interactively as determinants of each other” (Bandura, 1986, p. 23). The relative influence of specific factors in any unique situation will certainly vary across persons, situations, and behaviors. It would be easy simply to state that more research is needed in personality assessment techniques. In the past 10 years, that is where the field has gone, but without much apparent gain. Research is still needed to verify useful constructs and to clarify the potential applications of personality measurement to individual children and intervention design. In other words, decisionmaking research guided by the natural questions of psychological practice, such as how to help in natural problem contexts, is still the most critical gap at present. In this chapter, we first analyze personality measurement and related topics of psychopathology within a unified validity framework that encompasses the traditional categories of reliability and validity. Our focus is on central theoretical and practical issues defined in terms of confidence and helpfulness of decision outcomes related to test use. Second, we present fundamental dilemmas for research and professional practices associated with personality measurement. Third, we make suggestions in the form of component strategies for personal and social assessment. The challenges of behavioral assessment also are integrated into the discussion because they are inherently related to child intervention outcomes.

CRITICAL CONCEPTS OF PSYCHOMETRIC QUALITY The psychometric qualities of measurement traditionally have been analyzed via two separate but intertwined constructs—reliability and validity (duly compartmentalized in terms of the “three c’s” of construct, con-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 5

1. Personality Assessment Research

tent, and criterion-related validity). These measurement qualities are likewise treated separately in the technical manuals of most tests. Following Messick (1989, 1995), we frame our discussion of psychometrics within a “unified” theory of validity. Consistent with that framework, the adequacy and appropriateness of measures can be evaluated through a series of pointed questions, as follows: 1. What is the meaning of an inference made from a test score? This is easily recognized as an aspect of validity evidence (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 1999) and is critical within the unified view of validity. Perhaps especially within the assessment of personality variables it is imperative that the meaning of any score derived from a test be clearly established to determine the appropriateness of its use to make assessment decisions. Score meanings can be supported through a number of related considerations such as the relevance and representativeness of test content, nomological relationships with other construct measures, and generalizability of scores within the content domain (as shown through reliability analyses). However, beyond their theoretical or substantive import, scores have social meaning as well (derived from the consequences of score use). We believe that the latter idea has not received sufficient emphasis in either test development or use. 2. Given an understanding of the meaning of a score, can you use it to meet some purpose (answer some assessment question)? All assessment should be driven by questions and purposes. This is the characteristic toward which traditional validity and reliability estimates have been somewhat loosely aimed. For example, the reliability of some test inference is important in evaluating whether you can consistently make a particular decision. Likewise, various means of assessing validity have to do with how well an assessment decision meets a defined purpose. 3. Should you use a metric to meet some defined purpose? Many methods for judging validity and reliability have been developed in regard to the first two sets of emphases (shown previously). This question

5

more clearly places emphasis on the consequences of psychological decision making in regard to desirable outcomes for clients and includes analyzing the values that accompany measurement (e.g., labels applied to a client because of test results). A set of critical assumptions underlie this perspective, as follows: 1. Perhaps the most important assumption is that validity can only be assessed within the boundaries of some clearly understood assessment purpose. This is (and has been) the position of major professional organizations in regard to appropriate test interpretation (AERA et al., 1999). That means that the concept of test validity per se is inappropriate, and that inferences from a test score(s) may be made validly or not given some particular purpose. If the intent is to use a test score for multiple purposes, these purposes must be articulated and examined separately. One of the biggest shortcomings of test manuals is that purposes intended for derived scores are only vaguely outlined, and evidence concerning validity is not typically tied to any specific purpose. This is a dangerous practice in that it may lull practitioners into ill-supported decisions about their use of test scores. 2. Treatment (or other) decisions are nearly always the result of multiple assessment activities. This is not the way validity has been analyzed within the psychometric literature, nor within test manuals where single instruments are analyzed for general psychometric qualities as opposed to qualities tied to a specific test purpose. 3. Even given multiple measures, decisions are based on the inferences from test scores. It is these inferences, not particular test scores, that ultimately must be validated. 4. Psychometric qualities such as reliability (a critical facet of unified validity) must be judged within coherent units of assessment decisions (e.g., diagnosis, screening, developing problem hypotheses, and selecting appropriate treatments), and based on what aspect(s) of a test will guide a particular decision. For example, if a profile of subtest scores is used to make a diagnostic inference, it is the reliability of that profile that is critical, not the set of subtest reliability estimates or the overall measure of test

reyn2-1.qxd

6/6/2003

8:59 AM

Page 6

6

I. GENERAL ISSUES

reliability. Using diagnosis as a continuing example, the meaningfulness of a diagnostic category, accuracy of classification decision, and the consequences of making the diagnosis must be analyzed to judge if one can and should use test data to make the diagnostic inference. If a single test item were used to make an inference, it is the reliability of that inference (and the single item) that is important. Decision-based reliability indices may bear little resemblance to traditional reliability estimates. 5. Judgments about client outcomes as the critical arbiters of validity are complex and must ultimately involve subjective analysis based on clearly described values. 6. Analysis of the contribution of any single measure within an array of measures used in making a decision is most closely related to what has been called incremental validity (Sechrest, 1963), and can be profitably analyzed. For example, if a single measure adds some “amount” to improved outcomes for children, it adds some increment of validity. Validity remains a unified overall construct. It is in this spirit of incremental validity that we turn to the traditional constructs of reliability and validity. Inferences derived from a personality instrument will still need to be consistent in order to decide that a score can be used to help make some decision. Thus, reliability as consistency can still be, and should be, analyzed as an important facet of validity. As we will see, traditional views of reliability leave something to be desired, even in consideration of incremental improvement to overall validity.

ESTIMATING THE RELIABILITY OF SINGLE SCORES “Reliability” has generally pertained to the consistency of measurement and enables an estimation of test error. As stated earlier, the real issue for practice is the estimation of the stability of an inference or a decision based on an inference. The reliability of a test or score is not the critical level of psychometric analysis, even though this is the way such data are nearly always presented. We see the construct of reliability as an important facet of validity, and most directly tied to the second evaluation question de-

fined previously (“Can you use a score?”). Unfortunately, classic numerical estimates of reliability can only indirectly inform a practitioner about the actual consistency of an inference or decision. The only direct information concerning the confidence one might place in a diagnostic inference concerns the probability of making the same inference or decision after a second measurement occasion with the same or different examiners, across practitioners given the same information, or other combinations of possibilities. These sorts of probability analyses are seldom provided to assessors by test developers. Further, if decision consistency became the basis of reliability, it would obviate many of the arguments about which reliability coefficient is appropriate. Given these caveats, it is still important to review the typical ways in which reliability is examined, and the issues which need to be understood by practitioners as they make decisions about instrument use.

Classic Construct of Test Reliability The classical reliability model developed by Spearman at the turn of the 20th century assumed that each score has two components: a “true score,” which is estimated from a person’s obtained or “observed score” on a measure, and “test error.” Traits are assumed to be stable, and error is assumed to be random or uncorrelated. The reliability of a test may be defined by the variation (or variance) in true scores divided by the variance in observed scores, or the ratio of truescore to observed-score variance (e.g., Thorndike, 1982). The definition of reliability from classical test theory typically makes the assumption of hypothetical repeated measurements of an individual’s performance on parallel tests, so that fluctuations in scores enable the estimation of error. Although intuitively attractive and amenable to statistical manipulations, the classical model has significant complexities that have been debated for nearly as long as the model has dominated measurement theory (Lord & Novick, 1968). The greatest concern is that there is no direct link between the actual observed score and the hypothetical true score; in psychology, the true score is always inferred, and for many traits critics have argued that the true score may

reyn2-1.qxd

6/6/2003

8:59 AM

Page 7

1. Personality Assessment Research

exist more vividly in the minds of test developers than in behavioral evidence. The definition of reliability based on classical test theory cannot be directly tested, and there are significant theoretical divisions in reliability theory. Further, there is no single method of estimating the reliability of a test score. A variety of methods to estimate reliability in a classical framework exists based on both logic and specific statistical and experimental procedures. Some think that the concept of true score should be abandoned (e.g., Lumsden, 1976). However, the classical model and its derivatives have dominated test theory, and therefore test development (Ghiselli, Campbell, & Zedeck, 1981). Thus, many contemporary practices associated with test use are based on the assumptions that stem from this model. For any test, reliability can be estimated in numerous ways; however, the methods account for different sources of error. Traditionally, experimental and statistical procedures are used to present reliability data in four ways: internal consistency, test–retest reliability, alternate forms, and analysis-ofvariance procedures. Many discussions stop at this point. However, practical problems are associated with the fact that different estimates of reliability address different aspects of the consistency of measurement. Perhaps the most important criticism related to any view of reliability as a quality of a score or as a quality of a decision (these are related) is that estimates of reliability based on a single procedure are nearly always underestimates of actual instability of both scores and inferences. As we discuss, realistic numerical estimates of score reliability must take into account different sources of error available only through simultaneous consideration of reasonably composed estimates of different reliability facets. In addition, estimates of the reliability of a decision based on a score can only be estimated by examining studies of various aspects of actual decision probabilities, not just errors associated with a score.

Estimation of Reliability as Internal Consistency The reliability coefficient of a scale can be estimated from a single administration. Sta-

7

tistical analyses are used to determine the degree of “internal consistency” of responses to test items. An early procedure was to (1) “split” the test into equivalent parts, (2) correlate the halves, and (3) “correct” the obtained correlation that resulted from the briefer scale to estimate the reliability coefficient for the full-length test. The Spearman–Brown prophecy formula, the basic method to estimate the resulting reliability from the test halves, has other uses as well and is discussed in a subsequent section concerned with aggregation (see Equation 3). A basic problem is that the resulting reliability coefficients derived from splitting the test in various ways (e.g., odd–even and first half–second half) will vary. Coefficient alpha (Cronbach, 1951) provides a more general solution for internalconsistency reliability, in effect representing the average of all possible item splits. It is derived from the variance of the items relative to the variance of the total test, or from the average interitem correlations, and establishes the “upper limit” for the test’s reliability based on content sampling (Nunnally, 1978, p. 230). If coefficient alpha for a measure is low, the items have little in common and/or are too few in number. Internal-consistency estimates are widely used in practice and are especially important for constructs hypothesized to change or fluctuate over time. Although it appears straightforward, coefficient alpha does present interpretative difficulties. Because coefficient alpha increases as a function of the magnitude of interitem correlations and number of correlated items, scales with similar reliabilities may have different properties. High reliabilities may be achieved through a small number of highly correlated items or through large numbers of items with modest or even low interitem correlations. Tests that have high estimates of internal consistency may be measuring a homogeneous trait, asking redundant questions, or asking highly related questions repeatedly.

Estimation of Reliability through Test–Retest and Alternative Forms Equally straightforward in description, the test–retest method involves a second administration of a scale within a specified inter-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 8

8

I. GENERAL ISSUES

val (usually about 2 weeks). The results of the two administrations are correlated; thus, an estimate of the stability of the measurement is provided. It has been argued that test–retest reliability is less important to personality measurement because of a number of confounding variables: the unknown effects of two administrations of the questions or test stimuli, unknown changes in life circumstances during the retest interval, changes that occur in the testing situations, and the unknown influence of these factors on estimates of the stability of the trait. Reliability estimates also can be derived from alternative or equivalent forms of a scale, although this method is less frequently used in personality measurement. Simply, two (or more) measures of a trait developed from an item pool are administered and the results correlated to produce reliability estimates. When this method is used to examine test–retest reliability, the effects of memory in responding to specific items are eliminated. From the viewpoint of professional practices, the stability estimates based on test– retest procedures are often important. Practitioners are quite concerned about potential fluctuations of trait estimates over brief periods, and the corresponding degree to which intervening variables may influence interpretations of various constructs and thus decisions that stem from test usage. Time-limited decisions are troublesome (even embarrassing) for many professional practices. A low test–retest reliability coefficient is only problematic if the behavior measured is presumed to be relatively stable. In this case, questions can be raised about the measurement procedure, the trait, or the power of intervening variables.

Estimation of Reliability through Analysis-of-Variance Procedures Related to the classical model, analysis-ofvariance procedures (Thorndike, 1967, 1982) and generalizability theory, advanced by Cronbach, Gleser, Nanda, and Rajaratnam (1972) have been proposed. Logically, the procedures enable questions to be raised about both the true-score and error components. Thorndike (1982) writes from a psychometric perspective: “What is included

under the heading ‘error variance’ depends on how the universe that the test score is presumed to represent is defined, with certain sources of variance being treated as error under one definition . . . and as true score under another definition” (p. 157). The realities are that when tests are administered on different occasions, under different conditions, or through different procedures, variation in true-score estimates is a rule rather than an exception. Generalizability theory recasts the reliability problem “into a question of accuracy of generalization” (Cronbach et al., 1972, p. 15). An analysis is made of how test scores that result from a particular measure generalize to other subjects and to the universe of possible conditions. The conditions that are potentially examined include tasks, time of observations, situations or settings, and the performance of different raters. Classical notions of test error have been challenged in a similar way by behaviorists. The questions raised are important because the criteria typically used are tied directly to issues of usefulness by way of intervention design. Concerns related to classical measurement assumptions are described by Cone (1981; see also Johnston & Pennypacker, 1993): “Error is just a blanket way of referring to a host of ‘don’t knows,’ none of which are random. . . . [T]he term ‘random’ is really a pseudonym for ‘haven’t found out yet’ . . . since all behavior is lawful whether we know the controlling variables or not” (p. 61). Analysis-of-variance procedures were for a time deemed of interest to behavioral assessors. However, their overall contribution is likely to be limited, as the central tenets are different and behaviors of concern may be idiosyncratic, may have a low rate of occurrence, and may be episodic (Hayes, Nelson, & Jarrett, 1986). The hallmark of behavioral assessment, functional analysis, requires that a wide range of antecedent and consequent events be potentially amenable to appraisal as well. Behavioral assessment approaches have not resolved the basic measurement dilemmas, but the topics are dissimilar. In general, the most critical reliability and validity questions in behavioral assessment apply to (1) problem identification, (2) treatment selection, (3) treatment adherence, and (4) outcome measurement

reyn2-1.qxd

6/6/2003

8:59 AM

Page 9

1. Personality Assessment Research

of the therapeutic process (Barrios & Hartmann, 1986). These are not well treated through analysis-of-variance procedures. However, some studies are quite useful. For example, Hansen, Tisdelle, and O’Dell (1985) used generalizability procedures to establish the overall equivalence of audio versus direct observation of parent–child interactions, although low-frequency behaviors likely to have clinical significance had to be dropped from the analysis (e.g., crying and destructiveness). Generalizability studies pertinent to professional practices are few (cf. Jones, Reid, & Patterson, 1975), and the methods based on analysis-of-variance procedures typically have rigorous design requirements that hamper many applications. Multifaceted studies require large samples, and clinical realities introduce the potential for confounding design factors that are difficult to control, especially with more than two facets (Thorndike, 1982). The conceptual issues, however, are clearly critical to individual assessment within the model of unified validity. Consistent with Messick’s (1989, 1995) perspective, with generalizability theory, the distinction between reliability and validity is deliberately blurred. The model enables questions to be raised about “error.” Sources of error become information from an experimental view. The study by Hansen and colleagues (1985) illustrates both the power of the techniques associated with analysis-of-variance procedures and their difficulties: General relationships may be studied, but specific behaviors require idiographic measurements.

Applying Estimates of Reliability Standards for Reliability Well-established conventions for reliability coefficients have been described as follows (Nunnally, 1978; Stagner, 1948). Reliabilities for research purposes should range from .70 to .80 to be deemed “acceptable.” For individual decisions, it is suggested that reliabilities should be .90 at a minimum, with coefficients of .95 the desirable standard. These standards for individual decisions are commonly ignored by scale developers and consumers. However, as it turns out, even scales that meet these conventional stan-

9

dards may not work as expected in applied decision contexts (see “Reliability of the Inference”). Reliabilities can be directly interpreted as explained (or true) variance. However, for individual decisions, the “standard error of measurement” generally provides a more meaningful way of representing measurement precision in practice (though often biased by incomplete representation of the sources of variability in scores). Conceptually, the standard error of measurement is the standard deviation or variability of observed scores, given a true score, based on the hypothetical repeated measurement of an individual with parallel tests. In practice, an individual’s true score is not known but is estimated from a fallible observed score. The appropriate statistic for this situation is the “standard error of estimation,” or standard deviation of true scores given an observed score (Dudek, 1979). The standard error of estimation is used to establish a “confidence interval,” which indicates the probability that an individual’s true score lies within a specified range over repeated “theoretical” test occasions; estimates of true score (based on observed scores) vary around the actual true score because of error, and thus presumably because of chance factors. The conceptualization of standard error is of more practical significance than a coefficient of reliability because it shows a practitioner something about score stability and, indirectly, the stability of a decision. One of many significant problems occurs when some sort of cut or criteria score is used to make a decision (school-based screening, for example). Such decisions are basically dichotomous and any sort of standard error is insufficient to describe the likelihood of making the same decision across practical facets (other examiners, forms, occasions, etc.). Although there is a well-described literature concerning this type of issue (and the related issue surrounding ideas of “false positives” and “false negatives”), information does not seem available for most common practices. As we discuss later, when there is information about decision analysis of cut score use, it often has been discouraging with respect to the adequacy of decisions. Three issues concerning standards for reliability are of primary importance for re-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 10

10

I. GENERAL ISSUES

search on professional practices. The first two are considered here; the third bridges the gap between traditional views of reliability and validity. First, confidence intervals that apply to the prediction of true scores should be established around the “estimated true score.” Nunnally (1978) points out the “obtained scores are biased estimates of true scores. Scores above the mean are biased upward, and scores below the mean are biased downward” (p. 217). More extreme scores contain more “error,” and thus more bias (i.e., the error component for a score that is 2 standard deviations from the mean is twice as large as the error component for a score that is 1 standard deviation from the mean). Obtained scores easily can be adjusted to enable the use of an estimated true score (x⬘) for creating confidence intervals, as follows: X) x⬘ = X 苶 + (rxx) (x – 苶

(1)

where X 苶 is the mean of X, rxx is the reliability of X, and x is the observed score. Although this issue is often trivial in a numerical and statistical sense (scores may be “adjusted” by small amounts), an important exception to the triviality occurs when specific cutting scores are used for professional decisions—implicitly or explicitly, a common practice. Second, the calculation of standard error of measurement should be placed within the context of a specific reliability estimate, intended for a specific function (Sechrest, 1984). Many test authors use internal estimates of reliability (e.g., coefficient alpha) to establish confidence intervals for test users. This type of reliability only takes into account the consistency and number of items within the domain as represented by the particular test developer. Confidence intervals to describe the stability of the behavior over time, and the generalizability of estimates across potential raters and alternative measures within the same domain (as defined by other test developers), are often unexamined. For these situations, the “standard error of prediction,” or standard deviation of expected scores across raters or occasions, is the appropriate statistic (Dudek, 1979; Lord & Novick, 1968; Schulte & Borich, 1985). Third, and most important, reliability must be viewed within a context of validity.

For example, the standard errors of prediction associated with independent estimates of the construct or behavior of interest need to be considered—a discussion resumed below in regard to multitrait–multimethod analyses. Reliability of Inferences from Test Scores The errors of measurement reported in manuals have a combined effect with regard to a score inference. Anastasi has pointed this out in many editions (e.g., Anastasi & Urbina, 1997). The focus on specific estimates of reliability, which may be high, tends to underestimate the problems of individual interpretation of scores. To estimate the extent of the problem, a theoretical approximation of the combined effect of measurement errors can be obtained through variance components analysis (L. Feldt, personal communication, September 26, 1991), in which the reliability coefficients for independent sources of variability in scores (e.g., content, time, and examiners) are represented by ryy (subscript 1 through n): 1

冢

1 – ryy ryy1

冣 冢

1 – ryy ryy2

冣

冢

1 – ryy ryyn

1 2 n 1 + ᎏᎏ + ᎏᎏ + . . . ᎏᎏ

冣 (2)

As an example, a popular rating scale that measures internalizing problems has a coefficient alpha reliability of .78 and a test–retest reliability of .76 for teacher ratings at the elementary school level. From the aforementioned formula, the combined effects reliability is .62, which would be the reliability most appropriately associated with an inference about internalizing behaviors for an individual child based on the score. Depending on the decision purpose, the scale user would then build a “confidence” interval that would show how much confidence there may be in an inference related to “internalizing.” However, this reliability estimate is still optimistic because other reliability facets could also be addressed (i.e., other potential raters), as well as comparisons with other instruments purporting to measure the same construct, a matter we will turn to again in the section on validity.

reyn2-1.qxd

6/6/2003

8:59 AM

Page 11

1. Personality Assessment Research

One would not have much confidence in the inference of “internalizing.” Aggregation to Improve Reliability of Inferences The fundamental method to increase the reliability of a measure is simply to increase the number of items. The relationship between test length and reliability is defined by the Spearman–Brown formula:

krxx 1 + (k – 1)rxx

r⬘xx = ᎏᎏ

(3)

where r⬘xx is the reliability of expanded test form, rxx is the original reliability of measure X, and k is the proportionate increase in test length. For example, in determining split-half reliability, k = 2. Through a series of widely cited studies, Epstein (1984) has applied this basic relationship to issues concerning personality measurement. Rushton, Brainerd, and Pressley (1983) also have reviewed the principle of aggregation and reached a similar conclusion: “Many weak or unconvincing correlational relationships found in the personality, social, and cognitive development literature are consequences of failure to aggregate” (p. 18). Epstein (1984) theorized that “personality coefficients,” a disparaging term applied to low but significant correlations of .20–.30 often found in personality research, might be the results of a failure to aggregate based on the analysis of units of behavior that were simply too small; these results might be similar to the results achieved when looking at item correlations. He summarized the effects of different aggregation strategies in the following manner: 1. Aggregation over stimuli and situations reduces error variance associated with the uniqueness of particular stimuli and situations. 2. Aggregation over occasions and over trials within occasions reduces error variance associated with changes over time. 3. Aggregation over judges or raters reduces error variance associated with individual differences among judges. 4. Aggregation over modes of responses reduces error variance associated with different response modes. . . . (pp. 260–261)

11

Through aggregation, Epstein was able to achieve respectable correlations (as high as .80 and .90) for traditional constructs associated with personality measurement. Epstein (1984) summarized the results in the following way: “Stable response dispositions can be demonstrated when responses are averaged over adequate samples of behavior but not necessarily when single instances of behavior are observed” (p. 214). The respectable stability coefficients obtained through aggregation of responses, he argued, lead to improved evidence for validity across a number of constructs and measures. Through aggregation, Epstein found support for the following statements: “(a) behavior is situationally specific, (b) behavior is general across situations, and (c) people have broad cross-situational response dispositions” (p. 263). Of course, nothing is this simple in personality measurement, and aggregation is not a panacea, as pointed out by Epstein (1984) and others (Bandura, 1986; McFall & McDonel, 1986; Mischel, 1984b; Mischel & Peake, 1982). First, aggregation leads to a corresponding loss of information. If observations are aggregated over any of the four dimensions listed previously, variance attributed to the dimensions is termed “error.” For example, if observations are aggregated over situations, then information related to the potential variance of situations may be less readily apparent. Second, Mischel (1984b) argues that Epstein’s research does not deal with the central issues of “the classic personality debate”: the manner in which individuals discriminate situations in relationship to their social behavior and the usefulness of inferring traits in predicting actions in a given context. In Bandura’s (1986) words, “Aggregation inflates correlations but yields indeterminate or empty predictions” (p. 10). Furthermore, a high degree of behavioral consistency logically cannot be used to argue for the existence of traits without other considerations pertaining to validity (see McFall & McDonel, 1986, for a discussion of these concerns in addition to other related problems). Finally, aggregation as discussed by Epstein (1984) relates to aggregation of observations of behavior, whereas personality has been much more typically measured in prac-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 12

12

I. GENERAL ISSUES

tice by administration of a test of one sort or another. Thus, the idea of aggregation in practice appears most useful in basic research regarding propensities to respond or possibly an analysis of construct facets. Although aggregation is perhaps useful in some conceptual or theoretical analyses, it would seem of little use in examining the reliabilities of decisions based on typical personality measures. Aggregation may elucidate one area of difficulty but, in doing so, obscure others. Although aggregation may better reveal the coherence in some behaviors related to personality constructs, social and cognitive behaviorists argue that personality processes should be linked to psychological processes that result in a better understanding of individual patterns of behavior within socially important contexts.

ESTIMATION OF THE RELIABILITY OF INFERENCES FOR DIFFERENCE SCORES AND PROFILES The error involved in interpreting a difference between two measures is larger than the error in each measure: More error is involved and the difference score itself has a separate meaning. The “difference-score” problem is widely known in the field of psychological and educational measurement (AERA et al., 1999, Standard 2.3). The basic question is this: Given two measures for an individual, how reliable is the difference between the two scores? The reliability of a difference is related to (1) the reliability (or error) of each measure, (2) the intercorrelations between measures, and (3) possible normative difference between the measures. As the average reliability of the two measures approaches the intercorrelation between the two measures, the reliability of the difference score (rDIFF) approaches zero:

[½(rxx + ryy)] – rxy 1 – rxy

rDIFF = ᎏᎏ

(4)

where rxx and ryy are the reliabilities of measures X and Y, respectively, and rxy is the correlation between X and Y. The problem of the reliability of differences has been discussed primarily in terms of learning disabilities (Salvia & Ysseldyke,

1998; Thorndike, 1963; Thorndike & Hagen, 1961), but it is also a serious problem in any profile interpretation (Barnett & Macmann, 1992b; Macmann & Barnett, 1985) in which the objective is to identify “severe discrepancies” among correlated variables, as discussed in the next section

The Reliability of Profiles Profile interpretation is the hallmark of many techniques, yet it raises a host of thorny issues because multiple difference scores are used to build personality profiles. Profile interpretation depends on scale construction employing appropriate methods and on validity studies, although neither may be sufficient to enable one to interpret individual profiles with great confidence. We define “profile interpretation” in two related ways: the meaning that can be attributed to the pattern of scores, and whether or not the pattern is related to a defined taxonomic group. A “taxonomic group” refers to “psychologically” similar individuals who occur with sufficient frequency in a population to permit ostensibly reliable predictions and thus meaningful comparisons (Wiggins, 1973). Three dimensions of profiles are important (Nunnally, 1978): (1) the level or elevation of scores, (2) the dispersion or scatter of scores, and (3) the shape or features of the profile. The following difficulties are associated with profile interpretation in professional practice (Barnett & Zucker, 1990): 1. The reliability of subscales, the validity of subscales, and the reliability and validity of patterns are likely to vary within a scale. Some profiles or patterns may ultimately be useful; others may require further research or may lack validity support. 2. The reliability of the difference between two correlated scores is lower than the reliability of either score alone (see the preceding section on the reliability of differences). This problem is severely compounded when one considers the relationship among many correlated scores. 3. The stability of the profile may be unknown. Because of all of the factors described above, the overall pattern may be quite unstable. To the extent that there is

reyn2-1.qxd

6/6/2003

8:59 AM

Page 13

1. Personality Assessment Research

4.

5. 6. 7. 8. 9.

an important general factor in a scale, a common finding (Macmann & Barnett, 1999), peaks and valleys in the profile may just connote trivial differences. As the number of comparisons increases, differences due to chance are more likely to occur. Furthermore, most comparisons are unplanned. Profile interpretations for individuals capitalize on chance occurrences of behavior. Extreme scores are most often interpreted; these have the most error. Subscales may have very different meanings (a) at different elevations, and (b) within unique or overall profile patterns. Unusual or idiographic patterns will not be represented in taxonomies. Support for the utility of profiles in intervention decisions badly needs critical empirical evaluation. Profile “classification” must also be understood in terms of error rates.

One of the best known examples of profile development is that of Achenbach and Edelbrock (1983). They write what amounts to a prototypical description of professional practices of this genre: “Practitioners should adapt our materials to their own situations and integrate them with other types of data. . . . The essence of clinical creativity is to synthesize diverse and imperfect tools and data into practical solutions suited for each individual case” (p. 113). However, the potential cumulative error rates for assessment inferences in this type of recommendation are not even estimable. Suffice it to say that the difficulties in personality measurement have not been redressed by profile development. Based on measurement principles, namely, the theoretical behavior of correlated variables (Macmann & Barnett, 1999), profiles basically will have unknown reliability when applied to individual children, except for the proviso that, to the extent that inferences derive from multiple difference score analyses, they will be much less than the reliabilities of the individual scales on which they are built. More important than the determination of statistical significance of profile types, and their reliability, the psychological meaning of profiles must be determined. Profile interpretation has a long history within personality

13

assessment; think, for example, of the extremely large number of interpretive profiles for the Minnesota Multiphasic Personality Inventory (MMPI). However, we argue here that the meaning of profiles or their contribution to assessment decisions is too often a moot issue because of the severe problems surrounding the stability of profile interpretations for individuals. If an individual’s profile is seen as “psychologically significant,” yet the profile is based on chance relationships among subtests for a single occasion, decisions cannot be made with confidence.

The Reliability of Projectives The term “projective” carries negative connotations for many, based on years of high hopes followed by disappointing research and subsequent reviews. Although the development of some projective techniques has yielded reliability estimates that approximate those of so-called objective measures, and the interest in covert processes remains lively, many challenges to their use in professional practice still exist. A review by Karon (1968) challenged the classical assumptions associated with the reliability of projective instruments. Karon argued that traditional notions of reliability have resulted in a “paradox”: Reliability estimates associated with temporal consistency are not applicable because the experiences are expected to fluctuate, and similar arguments may be applied to internal-consistency estimates. Thus, Karon argued that the validity of projective techniques is not necessarily bound by reliability. Despite the fact that some reliability estimates (e.g., interrater reliabilities) reach satisfactory levels, the aforementioned problems associated with the reliability of the inference have not been addressed by Karon or others. Moreover, the use of the information concerning reliability for intervention design is a necessary and independent criterion yet it remains unavailable. Although projectives have been widely criticized for their limited success in reliably assessing personality traits or structures, a gap still exists in the study of “private” thought processes and automatic (or in a different paradigm, unconscious) influences. Many constructs have been applied, such as depression or anxiety, as have terms such as

reyn2-1.qxd

6/6/2003

8:59 AM

Page 14

14

I. GENERAL ISSUES

“fantasies,” “affect,” and so on. Furthermore, there continues to be interest in subjective or private inner experiences and their function as potential causal factors, including motivational and emotional processes, from diverse sources (Bandura, 1986; Friman, Hayes, & Wilson, 1998; Mahoney, 1980; Meichenbaum & Gilmore, 1984). The primary technical problems are verifications of inner experiences (Johnston & Pennypacker, 1993 ) and, ultimately, utility. The potential contributions of private processes, including inner states not amenable to direct report, cannot be readily dismissed; to do so would be to deny the significance, as well as creative applications, of cognitive and affective theory (Bandura, 1986; Friman, et al., 1998). Major advances are less likely to be based on new instruments that follow tradition than on procedural safeguards and research-based methods for incorporating the information about private thoughts and those out of awareness into testable plans for professional practices. Mischel (1973) has described personality appraisal in a manner consistent with this direction and we review his recommendations in a later section.

CONFIDENCE IN PROFESSIONAL PRACTICE: DECISION RELIABILITY Although addressed by the Standards for Educational and Psychological Testing (AERA et al., 1999), more attention should be given to the reliability of decisions. “Decision reliability” (e.g., the reliability of a psychological classification or diagnosis) is amenable to study and merits formal practical consideration (e.g., Cronbach & Gleser, 1965; Livingston, 1977). To examine the reliability of professional decisions, one can approximate the professional or natural decision process as closely as possible (e.g., the child fits a classification or does not; a targeted variable is deemed important or not, and the child would benefit from intervention or would not). Studies of decision reliability for high incidence educational classifications and clinical profiles have produced alarming results (Barnett & Macmann, 1992a; Macmann & Barnett, 1999). As a principal finding, we

have found three to five cases of disagreement for every case of agreement across many types of inferences (i.e., learning disabled or not learning disabled), using different samples, computer simulations, constructs, and methods. Although some of these studies have focused on scales not traditionally thought of as “personality measures” (i.e., achievement) or are marginally associated with personality (i.e., intelligence as a “trait” or personal competence), the patterns of findings apply because they are based on measurement theory and principles, not on specific constructs. Also, many researchers and professionals anchor personal competence to these constructs as some traditional sources have created momentum to infer personality-related constructs from intelligence profiles (e.g., Kaufman, 1979). There are many reasons for these pessimistic decision reliability findings, and some are well traveled (i.e., approaches to norm development) or are discussed in other sections (i.e., base rate and selection rate, extreme scores, and difference score reliability, depending on the decision being analyzed). The procedures for analyzing the reliability of decisions, however, are basic and fit practical choices. Alternate forms, raters, observational systems, and decisions implied by test outcomes can be compared (e.g., parent vs. teacher ratings and parent vs. clinician ratings) with the outcomes dichotomized (e.g., eligible vs. not eligible for services, internalizing disorder or not, and attention-deficit/hyperactivity disorder or not). For example, Ronka and Barnett (1986) investigated the reliability of decisions based on different adaptive behavior instruments and raters and found kappa coefficients ranging from .00 to .51. In another example, the kappa coefficients revealed in comparisons of parents’ versus clinicians’ ratings of profiles and of internalizing and externalizing syndromes ranged from .30 to .90 (Achenbach & Edelbrock, 1983). Decisions based on multifactored evaluations can also be subjected to analysis. It is reasonable to require criterion levels for decision reliability that are equal to those for other areas of reliability (e.g., .90). We close our reliability section by simply pointing out that reliability is a far more vexing problem for professional practice

reyn2-1.qxd

6/6/2003

8:59 AM

Page 15

1. Personality Assessment Research

than typically implied in test manuals and many texts. The interpretation of reliability estimates reported in manuals generally is too optimistic: The error rates associated with inferences derived from scores are much higher than commonly discussed, even by many critics. Of course, reliability estimates limit validity inferences.

EVALUATING ASPECTS OF VALIDITY As stated earlier, validity is a unified concept tied to judgments about the quality of decisions and the inferences made from test scores. Following Messick (1989, 1995), we outlined validity as related to a sequence of decisions (what is being measured; can you use the measure to meet some purpose; should you use it). Research results can only validate a specific use for a test, not the test itself (Nunnally, 1978). In this conception, there are many facets of validity that can and should be explored separately and simultaneously. In a unified view of validity, many of the methods that have traditionally been seen as different types of validity (construct, content, criterion, etc.) are best conceptualized as related facets or considerations that define the meaning of a construct in both theory and practice. In Messick’s model, this broader notion of multiple facets of construct evidence is the first of several necessary decisions about overall validity. Any facet of validity can be examined conceptually, logically, and through experimental procedures. There is no consensus on the methods that are used to validate inferences derived from tests. Tests and test scores cannot have “high” or “low” validity in this general sense because it is the decisions and inferences based on scores that must be evaluated. General guidelines and principles for assessing various validity facets have, however, been developed and are easily incorporated into a unified validity concept. To be meaningful, validity information must be qualified and scrutinized through consideration of some specific purpose, test standardization procedures and samples, and the actual experimental validation procedures used by the test developers. Personality scales were intended as a short-cut method, to save time and money in contrast to other intensive methods (Pe-

15

terson, 1968). However, in considering major scales, one often finds that test development has occurred over decades through the efforts of many—a process clearly revealed through the history of any major personality measure (e.g., Schinka & Greene, 1997). It is an expensive endeavor (Burisch, 1984). Validation efforts are conducted not only by test developers but subsequently, even more important, by independent researchers. Therefore, validity evidence is subsumed by the body of research that follows the publication of a scale. However, Cronbach and Meehl (1955) pointed out that there is no one “scientific approach” that can completely legitimize a construct in a manner that would rule out scientific disputes and varying viewpoints. We discuss here the traditional validities called construct, criterion-related validity, and content as methods used to validate constructs within the unified validity model. Validating the meaning of a constructed measure that is used to make an inference relates to the first question we used to define unified validity—“What is the meaning of the construct being measured?” In terms of the second question (“Can you use a test score to make an inference to guide some decision?”), we discuss several facets that have typically been called incremental validity and criterion-related validity. In addressing the issue further, we extend our discussion of the relationship of reliability to unified validity. Given the criteria of confidence and helpfulness, we conclude with the ultimate defining quality of validity—the quality of client outcomes resulting from assessment (the facet of consequential validity) and the relationship to the practice of personality measurement. Unfortunately, the incorporation of the criteria of meaningful outcomes resulting from assessment decisions has been well developed and examined only within behavioral models (i.e., Baer, Wolf, & Risley, 1968; Wolf, 1978), not within traditional uses of personality assessment.

The Facets of Constructs Nomological Network as the Foundation Constructs are central to much of scale development and use. Test developers should

reyn2-1.qxd

16

6/6/2003

8:59 AM

Page 16

I. GENERAL ISSUES

place the construct measured by a scale within a conceptual framework that defines the meaning of the construct and its relationships to other constructs and observable behaviors or variables—a process referred to as the construction of a “nomological network” (Cronbach & Meehl, 1955; see also Messick, 1989). This network should explicitly translate theoretical constructs into verifiable relationships among operations. It is certainly dynamic during instrument development but should be coherent and well developed at the point of making an instrument available for clinical use. We believe that the nomological network is a critical construction if practitioners are to adequately use personality measures. A clearly described nomological network would allow developers to organize and present their data (to potential users) related to verifying the operational meaning of the constructs measured by an instrument and supporting a set of uses. Unfortunately, test manuals seldom, if ever, explicitly offer a proposed network. Furthermore, data collected during scale development are often presented in a fractured manner, related to separate validity concepts, and not clearly connected to meaningful inferences or to specific uses. Although there are many statistical approaches to scale construction that address aspects of validity (Schinka & Greene, 1997), we present two well-articulated methods typically used in construct validation studies that could easily fit within an explicit nomological network: the multitrait–multimethod matrix and factor analysis. The Multitrait–Multimethod Correlation Matrix Kenny (1995) wrote: “The MTMM matrix represents one of the most important discoveries in the social and behavioral sciences” (p. 123). A logical approach that stems from a classic paper by Campbell and Fiske (1959), the “multitrait-multimethod correlation matrix” (MTMM) can be employed whenever there are at least two constructs measured by two methods. An MTMM analysis includes two features of validity. First, it examines “convergent validity.” Logically, the correlations between the methods intended to measure the same trait should “converge” or be higher than

those developed to measure different traits. “Discriminant validity” provides an indication of the predicted divergence of the traits. Dissimilar traits should have lower intercorrelations than those found for similar traits. Each measurement procedure yielding a score is considered to be a “trait-method” unit in order to evaluate the possible contributions of various methods used to estimate the trait. “Method bias” is indicated when correlations between different traits (measured by the same method) are higher or similar to the correlations between different traits (measured by different methods). Correlations between supposedly related traits may be substantial because of shared method variance (e.g., self-report methods), although ideally the correlation between two traits should not be a function of the methods. In contrast, the use of distinctly different methods (e.g., observations and behavior ratings) may attenuate (or minimize) estimates of the same trait (Campbell & O’Connell, 1982). Validity coefficients are represented by correlations across methods for the same trait. To demonstrate evidence for constructs, same-trait or validity coefficients must be higher than correlations between different traits. A number of alternative statistical approaches to MTMM analysis have been proposed (Kenny, 1995; Millsap, 1995; Wothke, 1995), but none exactly match the rationale and logic of MTMM analyses. Separating trait from method remains challenging (Millsap, 1995). The MTMM analysis is quite appropriate to professional practice questions associated with the interpretation of psychological constructs. Given two (or more) methods of measuring various personality constructs (i.e., temperament, children’s self-concepts, or anxiety), a basic question pertains to the decisions that would be made depending on the various alternative assessment methods. Typically, the MTMM analysis has revealed a number of problems when employed (e.g., Macmann & Barnett, 1984). The validity coefficients, even when significant, may be low or moderate, and logical or expected relationships may not be upheld. The effects of undesirable or unpredicted method similarities and differences may be pronounced. Too frequently, scale developers have tried to measure constructs that do not differ

reyn2-1.qxd

6/6/2003

8:59 AM

Page 17

1. Personality Assessment Research

much from other constructs (Kenny, 1995). Practically speaking, the ambiguity of interpretation that is revealed through MTMM analyses is considerable (Poth & Barnett, 1988). In Fiske’s (1982) review of the frequent disappointments revealed by the MTMM procedure, he pointed out that convergentdiscriminant validation represented “modest criteria” (p. 90; emphasis added). The ultimate goal is convergence not only across different methods but also across different research strategies. Campbell and O’Connell (1982) argued that the MTMM matrix may overemphasize problems with traits and that measurement methods require the same scrutiny as traits when one is analyzing the disappointing results of construct validation. Factor Analysis “Factor analysis” represents a family of techniques that have as the common goal the simplification of complex correlation matrices, or interrelationships between items (or behaviors), in order to reveal the major dimensions that underlie a set of items. Factor analysis can also be used to test theory and clarify the meaning of personality constructs. The logic is easy to comprehend, but the use of the techniques requires a great deal of sophistication. Furthermore, the statistical approaches that are used rely on judgments and assumptions made in carrying out the necessary steps, and ultimately in interpreting and giving meaning to the results. A “factor” is a construct that seems to best represent the structure or relationship between variables or clusters of variables. Factors are named by the researcher or scale developer through a subjective labeling or judgment process. “Factor analysis is only a prelude to more systematic investigations of the constructs” (Nunnally, 1978, p. 330). Despite the vast array of techniques, there is increasing agreement on the basic procedures. Stevens (1996) recommends the use of principal-components analysis to first describe and “enumerate” the underlying constructs for a set of variables—a procedure that requires fewer assumptions but often yields results similar to other statistical procedures. Confirmatory methods such as

17

structural equation modeling (e.g., Crowley & Fan, 1997; Judd, Jessor, & Donovan, 1986), may be used to evaluate the degree of correspondence between the obtained factor structure and theory as can other factor-analytic methods. There are two major areas in which the use of factor analysis is relevant to the topics subsumed in the present chapter: research in personality and research in psychopathology. An overarching difficulty is the variation in terms (defined by the results of factor analysis with different samples, items, methods, and researchers) that have been used to describe similar behaviors (Sells & Murphy, 1984). One of the best examples of factor analysis for the purpose of personality description stems from decades of research by Cattell (e.g., Cattell, 1982; Dielman & Barton, 1983). In numerous studies, Cattell has factor-analyzed self-report inventories, biographical inventories, and observations. The work is based on Allport and Odbert’s (1936) research on basic descriptive adjectives that define human characteristics. Cattell has typically discussed 16 source traits of normal personality for adults and 14 for children. However, other researchers stress far fewer factors (e.g., Costa & McCrea, 1986; Goldberg, 1995), which they think describe fundamental features of personality in a reasonable manner. Although there are important sources of agreement, the descriptive research has been tangential to professional practices associated with the diagnosis or classification of psychopathology, and especially to research concerning intervention design. The study of child psychopathology through the use of factor-analytic procedures is exemplified by the work of Achenbach and Edelbrock (e.g., Achenbach, 1985; Achenbach & Edelbrock, 1978, 1983; Achenbach & McConaughy, 1997). (For other similar lines of research, see Reynolds & Kamphaus, 1992; Wirt, Lachar, Klinedinst, & Seat, 1984.) In a comprehensive review, Achenbach and Edelbrock (1978) examined the literature for empirical syndromes used in classifying childhood psychopathology. The studies included ratings by mental health workers, teachers, and parents. Syndromes with at least five items and with “factor loadings or average inter-

reyn2-1.qxd

18

6/6/2003

8:59 AM

Page 18

I. GENERAL ISSUES

correlations of .30 or higher” were included in the analysis (p. 1284). They found four “broad-band” syndromes that were replicated across studies, or that had “counterparts” in at least two studies. Sex differences were found to be important, with the findings most often based on boys. Empirically based syndromes found in two or more studies included the following: Overcontrolled, Undercontrolled, Pathological Detachment, and Learning Problems. In addition, 14 “narrow-band” syndromes were identified (e.g., Aggressive, Depressed, Hyperactive, and Anxious). Achenbach and Edelbrock (1983) developed the Child Behavior Checklist (CBCL) and the associated Child Behavior Profile to provide “standardized descriptions” of problems and social competencies of children. A principal-components analysis with a sample of clinically referred children (n = 2,300) yielded Internalizing and Externalizing syndromes, and narrow-band scales varying by age and sex (Depressed, Uncommunicative, Social Withdrawal, etc.). This methodology has also been extended to the analysis of teachers’ ratings and adolescent self-reports, as well as observations, into a complex assessment system (Achenbach & McConaughy, 1997). From this brief review, a number of professional practice questions concerning factor analysis become evident. The first involves determining the primary purpose of the factor analysis. Some purposes involve basic research and are tangential to professional practice issues. Second, the technical adequacy of the procedures should be evaluated. Whether the factor structure is stable and replicable is partly determined by the adequacy of the measure, but also by sample size and other sample characteristics. Evidence for cross-validation should be presented (replications on an independent sample). Third, the importance of each factor should be considered, estimated by the percentage of variance explained by the factor solution. Fourth, evidence to support various interpretations of the factors should be presented. This would include MTMM logic to consider the adequacy of the nomological network and relationship between similar and dissimilar constructs and measures. MTMM analyses are also important because most factor-analytic solutions are

based on rating scale techniques, but intervention design and evaluation are founded on direct observation. Finally, because scales derived from factor analysis often are used to derive profiles during assessment, the serious problems of profile reliability (see earlier) may severely limit their contributions to reliable decisions about children. What may appear to describe meaningful groupings of childhood problems based on research with groups may not yield information that provides reliable information in making decisions about individuals. Basic research in childhood psychopathology is clearly still needed, and empirical research will permit a better study of etiology and of the relationship among differentiable traits, empirical syndromes, or constellations of behaviors and well-developed interventions (Hibbs & Jensen, 1996; Kazdin, 1985a, 1985b). However, important factors have been traditionally omitted from factor-analytic solutions: “the stability or predictability of dysfunction” (Kazdin, 1985a, p. 38) and contextual appraisals of dysfunction. These are the factors that lead to confidence and helpfulness in decisions. In summary, factor solutions that result in glee for researchers may imply headaches for practitioners when the magnitude of support for various interpretations is translated into actual professional practice decisions and corresponding error rates. Although factor-analytic procedures represent an important contribution to childhood personality and psychopathology, applications to intervention design have not been directly addressed (see Haynes, 1986).

Criterion-Related Validity Methods within a Unified Validity Model Traditionally, “criterion-related validity” has focused attention on the degree to which a test predicts behavior (or classification status) on an independent criterion. The term “concurrent validity” is used to describe studies involving the child’s status at the time of testing (e.g., a diagnosis), whereas the term “predictive validity” is used to examine the relationship between a score and future status (e.g., temperament and later adjustment). “Postdictive studies” examine the relationship between a score

reyn2-1.qxd

6/6/2003

8:59 AM

Page 19

1. Personality Assessment Research

and past status on a criterion. Interventionists apply other criteria to outcome measurement (Komaki, 1998; Wolf, 1978). Criterion-related validity thus involves predictions between one set of variables (the predictor) and a second set (the criterion). We believe that in many cases, these relationships are most appropriately considered within the nomological network that gives operational meaning to a construct. It may even be difficult to distinguish methods subsumed under multimethod–multitrait procedures from criterion-related methods. However, there are important situations when methods associated with criterion-related validity facets have been used to evaluate some actual use of a test score. Correlations typically used to estimate the association between variables have been termed “validity coefficients,” although within our analyses this may be viewed as an overstatement. In addition to estimation of the correspondence between variables, the correlation coefficient permits an estimation of the standard error of prediction. The Pearson correlation, perhaps most often used, indicates the strength of a linear relationship. When squared, it reveals the proportion of accountable variation between two variables, or the variance that may be reliably predicted (Wiggins, 1973). Factors that limit validity coefficients are the same as those that influence correlations: whether the measures have a linear relationship, are normally distributed, and have equal variances throughout the range of scores (homoscedasticity). Furthermore, sample characteristics can limit (or exaggerate) the size of the correlations (Nunnally, 1978). The professional practice issues in studying criterion-related validity facets are formidable, but criterion-related validity lies at the heart of many professional practice questions. A fundamental issue may be referred to as the “criterion problem.” Wiggins’s (1973) warning still applies: “Criterion analysis has proved to be the most recondite and vexing issue confronting personality assessment” (p. 39). The reasons are obvious. Many of the inherent problems in scale construction have counterparts in criterion construction. The criterion itself must be evaluated in terms of its reliability and validity. Frequently, personality scale

19

developers have used similar rating scales for concurrent validity studies, although these fall into our analysis as being related to construct evidence. Even though this practice confounds shared method with validity and promotes circularity, results are often modest. For example, correlations of internalizing scores for two different rating scales on internalizing or externalizing disorders are likely to range from .3 to .7. Criteria often include such ambiguous and ill-defined behaviors as “adjustment,” “emotional disturbance,” or “risk” measured at one point in time. Some relationships have proven to be limited because of moderating variables apart from the predictor or criterion measure. For example, fantasy or other indirect indications of “aggressiveness” may not predict overt aggressive behaviors because of social sanctions or inner controls, not because they do not exist or because they are trivial. Moreover, forms of aggression may be varied and subtle in their expression. Because of these and other theoretical factors, modest correlations are often expected. Decision versus Criterion-Related Validity The criterion-related validity “problem” can positively be refocused to that of “decision accuracy and validity,” or the analysis of correct and incorrect decisions. This refocusing clearly is a better fit for a unified model of validity that ultimately hinges on the outcomes of assessment for clients. Decision theory is concerned not only with accuracy but also with the utility of decisions, and with values that can be tied to different outcomes. The decisions with respect to personal and social assessment are typically in the form of diagnostic systems: whether groups or individuals differ in measurable ways, and whether the groups or individuals should receive special “treatments.” The prototypical study of decision accuracy is depicted in a 2 × 2 matrix whereby valid positives are successfully screened, with high scores on both the screening measure and criterion; valid negatives are also successfully screened, with low scores on both measures; false positives have high scores on the screening measure but do not “succeed” according to the criterion measure; and false negatives have low scores on the screening

reyn2-1.qxd

6/6/2003

8:59 AM

Page 20

20

I. GENERAL ISSUES

measure but do “succeed” according to the criterion measure. “Cutting scores” on the test or predictor are used to guide decisions. The empirical basis for cutting scores should be clearly specified (AERA et al., 1999). As the cutting score is “moved” either higher or lower by the investigator, the type of errors change. Classic sources for decision theory include Cronbach and Gleser (1965), Cronbach and colleagues (1972), Meehl and Rosen (1955), Mischel (1968), and Wiggins (1973). Contemporary approaches to decision theory emphasize attempts to come closer to the natural process (i.e., context specific setting and organizational factors such as teaming and multiple goals, feedback loops, and realistic or high stakes) (Zsambok & Klein, 1997). Professionals’ situational awareness based on problem structuring is a key construct.

Among a number of factors, the incremental validity of an assessment procedure depends on base rates for the problem to be identified, evaluated within an appropriate population. Mischel (1968) offered the following example: “If 95 percent of the inpatients of a particular hospital are diagnosed ‘schizophrenic,’ a test-derived statement that predicts this label correctly 80 percent of the time is less useful than systematically calling every patient in the hospital schizophrenic” (p. 106). Thus, base rates limit the association that can be demonstrated between a predictor and criterion. Alternatively, modest validity coefficients may be useful in certain situations, because the “usefulness” depends on information that is actually gained through a measurement procedure. Content Validity

Additional Facets of Unified Validity Incremental Validity The correlations between a test and a criterion measure applied in the analysis of criterion-related validity can be misleading to a large degree. Other factors have to be taken into account. There should be clear benefits for using the targeted measure over alternative procedures in terms of cost, intrusiveness, acceptability, concordance with intervention design, and other factors. The outcomes of actual decisions based on a comparison of procedures should be assessed. Introduced earlier, “incremental validity” (Sechrest, 1963) suggests the need to evaluate the contributions that a test-based inference makes to a decision, over and above information that is already known or readily available. Incremental validity is concerned with the improvements in assessment decisions that occur by the use of a procedure, compared to those that can be made without it. The question is critical to personal and social assessment. For example, after interviewing parents, teachers, and child, and conducting observations across settings, the professional psychologist would ask: “What nonredundant information will be gained by adding other procedures?” and “Will adding these procedures help my client?”

Content validity is concerned with the adequacy of the items and is based on the systematic examination of content, selected from an identifiable “universe” of content. Typically, content validity has been applied to academic tests, or tests with similar properties, because of the difficulties of sampling items from a universe or domain related to personality or social constructs. When the point of emphasis is shifted to the functional analysis of behavior, however, principles associated with content validity can have considerable relevance for personal and social assessment (see Linehan, 1980). Subjective Validity Although “subjective validity” is not a traditional topic in measurement theory, the personal or subjective elements of test usage are considerable. There are many lines of reasoning necessary for the consideration of subjective validities, but most point toward an increasing ecological perspective (Cronbach, 1984, p. 571). A judgment that assessment leads to better outcomes for clients innately involves values and subjective viewpoints. Within a unified model of validity, these subjective decisions must be made and clearly stated. Messick (1980, 1989) pointed out that tests are evaluated on the basis of measurement characteristics, whereas test applica-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 21

1. Personality Assessment Research

tions require evaluation through potential social outcomes that include both normative and subjective comparisons. Social outcomes cannot be studied without reference to actual behaviors in real-life settings, followed over long periods. In assessment related to intervention design, social validity (Kazdin, 1977; Wolf, 1978) has been described through the identification of (1) socially significant problems for behavior change, and (2) socially acceptable methods with respect to the immediate social community. The results of the intervention should have practical significance (Baer et al., 1968) and should be evaluated over time (Schwartz & Baer, 1991) and in multiple ways (Willems, 1977). As broad and illdefined as they appear in contrast to other measurement topics, subjective validities are necessary and basic considerations in the assessment–intervention process.

Modern Test Theory “Modern test theory” involves the latenttrait approach to test development (e.g., Lord, 1980). The procedure involves the study of the relationship between individual items and an underlying (thus latent) trait through mathematical models. Two major aspects of latent-trait models include (1) the assumption of a unidimensional trait with homogeneous items, and (2) mathematical functions that depict the relationship between item choices and the trait or ability measured (Ghiselli et al., 1981). A “latent attribute” refers to “the hypothesized but unobserved characteristics that account for a particular set of consistencies within and differences among persons” (Thorndike, 1982, p. 5). The mathematical functions are expressed as item–characteristic curves. In test design, items are selected that satisfactorily are related to the latent attribute and represent different levels of the trait. Thorndike (1982; see also Wainer & Braun, 1988) presents an introductory but comprehensive description of the latent-trait approach to test construction. Although modern test theory has been used increasingly for test construction, the potential issues on a practical level remain unknown. Psychometrically, the theory has attractive features. Items developed through latent-trait procedures can be thought of as

21

“calibrated,” presumed to be a major advantage. Nunnally (1978) points out that although the theory is termed “modern,” its basic principles were developed in the 1950s, with the ideas present long before that period. We believe that if there are implications for personality research, they likely lie within the facet of validating personality constructs, but the potential use of the constructs for individual decisions must be addressed independently.

CONTINUING THEMES IN PERSONALITY MEASUREMENT The purpose of this section is to review major themes in personality measurement that have created dilemmas for both researchers and professional psychologists. In the final section, we turn to component strategies for sound professional practices associated with personal and social assessment.

Idiographic or Nomothetic? Unique life experiences combined with inherited characteristics lead to “idiographic” patterns of behavior. “Nomothetic” pertains to general lawfulness of behavior. Although some view the relationship between the two as conflicted (Nunnally, 1978), the differences (as in many “dilemmas” in psychology) often define various goals in research and professional practices, and not fundamental conflicts. Allport (1937) argued for the importance of both approaches. Although “nomothetic” defines basic goals of behavioral research, there has been a continued interest in idiographic approaches and their relationships to general laws. Furthermore, it is also viewed as a “false dichotomy” (McFall & McDonel, 1986). All assessments involve idiographic and nomothetic objectives and procedures. The Idiographic Approach An interest in the in-depth understanding of individuals is shared by psychodynamic, phenomenological, behavioral, and cognitive-social learning theorists. Mischel (1968, 1973, 1984a, 1984b) has developed the most coherent description of the idiographic approach to the study of personality from a

reyn2-1.qxd

6/6/2003

8:59 AM

Page 22

22

I. GENERAL ISSUES

social and cognitive viewpoint, as well as of its relationship to nomothetic principles. Mischel (1984b) wrote: Deeply impressed by George Kelly’s (1955) thinking, I was sensitive to the fact that clients-like other people don’t describe themselves with operational definitions. They invoke motives, traits, and other dispositions as ways of describing and explaining their experiences and themselves. Much of the assessor’s task . . . should be to help people in the search for such referents for their own personal constructs, instead of forcing the assessor’s favorite dispositional labels on them. (p. 280)

Mischel (1973, p. 265) suggested five “person” variables that serve as guidelines for assessment–intervention design: (1) the individual’s competencies to construct (generate) diverse behaviors under appropriate conditions, (2) the encoding or categorization of events, (3) expectancies about outcomes, (4) the subjective values of different outcomes, and (5) self-regulatory systems and plans. The Nomothetic Approach Two distinct lines of research can be identified within the nomothetic approach: those that have been concerned with the identification of traits (e.g., Costa & McCrea, 1986), and those that have been concerned with lawful behavioral principles (e.g., Cone, 1986; Mischel, 1968). We briefly describe the “trait” approach here. The applications from behavioral and from social and cognitive-behavioral approaches are integrated into the suggested components discussed in the final section of this chapter. From Allport’s early work, Stagner (1984) defines a trait in the following manner: “A trait is a consistent and persistent pattern of behavior and experience (cognitive and affective) characteristic of a particular individual” (p. 7). Theoretically, traits may be unique to the individual, may be shared with others, or may be common. They are hierarchically structured, and function to control lower-level processes in behavior. Thus, a trait is associated with various probabilities of occurrences of specific behaviors or classes of behaviors in situations. “Trait-situation interactions are considered to be the rule” (p. 34).

Although there are far too many traits (or factors) to review, convergence has been a recent goal of researchers (Costa & McCrea, 1986; Sells & Murphy, 1984). Reviewed briefly under the topic of factor analysis, a well-established Big Five taxonomy similar to one proposed a number of years ago by several researchers (e.g., Fiske, 1949; Norman, 1963; Tupes & Christal, 1961) is presented by Costa and McCrea (1986, p. 410; see also Costa & Widiger, 1994): Neuroticism (e.g., Calm–Worrying), Extraversion (e.g., Reserved–Affectionate), Openness (e.g., Down-to-Earth–Imaginative), Agreeableness (e.g., Ruthless–Softhearted), and Conscientiousness (e.g., Negligent–Conscientious). It is ironic that two very different Big Fives in personality measurement have stood the test of time (i.e., Mischel’s). Potential effects of a better understanding of traits include (1) more adequate research in personal and social development, (2) a better understanding of expressions and meanings of various symptoms, and (3) further insights into mechanisms of stability and change. Costa and McCrea (1986) argue for stability in personality development through adulthood, and for the proposition that “human nature is by no means easily changed” (p. 420). This “proposition” has been well analyzed (e.g., Lewis, 1997; Moss & Sussman, 1980), and although no one would disagree with the difficulties of change, others would argue that the complexities of intervention design and execution have hampered unequivocal statements regarding possibilities for change.

Interviews, Observations, or Ratings? Interviews? Many regard the interview as the most important “instrument” in assessment. An extensive and diverse literature on interviews stresses anything from communication skills, unstructured or structured clinical diagnosis, private experiences, and intervention roles, to therapeutic approaches. The topic has also been addressed with respect to interviews directed to children and to all others associated with the assessment-intervention process; these vary widely by function (Barnett & Zucker, 1990). Many ques-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 23

1. Personality Assessment Research

tions persist concerning the technical adequacy and use of interview procedures (Gresham, 1984). Despite problems in describing the interview in a way that would satisfy all uses, it may be best to state that it is integral to assessment and intervention and all stages therein. All sources of assessment error are potentially relevant as well. Peterson (1968) described a prototypical behavioral interview. He pointed out that the distinguishing feature of the interview is the relationship between interviewer and interviewee. Without the relationship, the assessment–intervention process is impaired and ultimately terminated. The future of the interview with respect to studies of technical adequacy rests with the study of the interview’s interrelated but specialized functions: (1) as a technique of behavior analysis (e.g., Kanfer & Grimm, 1977; Peterson, 1968), (2) as a formal diagnostic tool, (3) as a method to study private experiences, and (4) as a data-gathering technique necessary to the assessment–intervention process. Ratings or Observations? As a reaction against the use of both projective and objective techniques for the study of personality structures and processes, there has been renewed interest in behavior rating scales and direct observations. They are not alternative ways of assessing the same information, nor are they complements of each other; they have different objectives (Cairns & Green, 1979). Rating scales have been developed to assess a broad range of factors: personality, psychopathology, problem behavior, and social emotional functioning (Edelbrock, 1983). “Direct observations . . . can be the key for identifying how actual behaviors are elicited, maintained, and organized” (Cairns & Green, 1979, p. 222). Observations are essential for the analysis of interactions (Bakeman & Gottman, 1986). Cairns and Green (1979) have examined the issue in depth. In their words, The distinguishing characteristic of rating scales is that they involve a social judgment on the part of the observer, “or rater,” with regard to the placement of an individual on some psychological dimension. (p. 210)

23

The distinguishing property of behavior observations is that they involve an attempt to record the actual activities of children as opposed to offering a judgment about children’s personal dispositions or the quality of their relationships. (p. 213)

Professional Practice Issues In summarizing the topics presented in this section, one problem stands out: The use of all the techniques described earlier must be guided by a template that defines decisions related to use, including interpretations and outcomes. “Professional judgment” is the term used to bridge the gap between the knowledge base and actual decisions concerning the use of various techniques (Barnett, 1988). Professional judgment itself involves a host of unknowns and depends on a personal model of professional practice. Unfortunately, professional judgment has not served well as a template, because the decision process has not been successfully explicated and has not been amenable to study. Although concerns have been voiced most often about projectives, the same arguments apply in varying degrees to all techniques, including interviews, observations, “objectives,” and behavior rating scales.

CONFIDENCE AND HELPFULNESS: STRATEGIES FOR PERSONAL AND SOCIAL ASSESSMENTS Throughout the previous sections, many questions have been raised concerning personality measurement and professional practice issues. The final section addresses two underlying themes: that test or scale interpretation would be best guided by the criteria of confidence and helpfulness. We think that these are addressed by applying appropriate theory related to behavioral change, and by analyzing problem-solving procedures that serve as the basis for professional practices.

What Can Be Said with Confidence? Applying a Coherent Theoretical Model Which factors guide decisions in ambiguous and complex real-life assessment situations? Decisions are guided by theory that may be

reyn2-1.qxd

6/6/2003

8:59 AM

Page 24

24

I. GENERAL ISSUES

deliberate and formal, or may be internalized and personalized adaptations of a venerable theory (e.g., Freudian theory and social-cognitive behaviorism), or may be eclectic or ad hoc. To the degree that factors that guide the assessment of personality remain unspecified, the process cannot be studied. Within any endeavor related to the assessment of personal and social functioning from the interpretations of projectives, objectives, rating scales, interviews, and observations, the potential exists for significant judgmental differences that can lead to idiosyncratic outcomes for clients. Many personality “theories” remain viable in the literature or in the minds of professional psychologists (see Loevinger, 1987). For the purposes of this chapter, they warrant appraisal to the extent that they guide actual professional practice decisions. Bandura’s (1986) criterion is pertinent: “The value of a theory is ultimately judged by its usefulness as evidenced by the power of the methods it yields to effect psychological changes” (p. 4). Theories of behavioral change are different from trait theories in that different variables are considered. Rather than resolving theoretical debate, a more promising tactic is clarifying technical adequacy characteristics for the key variables used to guide intervention plans. Applying Psychometric Concepts to Professional Practices As discussed in prior sections and other work (i.e., Barnett, Lentz, & Macmann, 2000; Macmann & Barnett, 1999), professionals relying on test or scale use to improve professional judgments, defended by traditional information about traditional measurement quality but without regard to decision research, problem context, or consequence, may have unwarranted confidence in decisions. The data on psychometric quality presented in test manuals does not sufficiently fit the real-world decision contexts faced by professionals, parents, and children, nor do reliability and validity data collected during instrument development adequately generalize to questions of problem identification or classification, or problem solving. The intent of psychometric methods to improve decision making still must be met,

but in ways that lead to confidence in understanding problem context and decision consequence. Practitioners must consider the psychometric qualities of decisions across the sequence of problem solving. Technical adequacy questions may be addressed by directly collecting ongoing data on both problem variables and the quality of those measures in the natural problem settings that constitute a referral. In other words, behaviors, other key variables (environmental events and interventions), and their technical adequacy can be all measured or sampled in phases of problem solving. The fundamentals of psychometric quality should have to do with measuring (1) meaningful child-related variables, (2) within specific educational or other natural contexts, and (3) for the purpose of guiding important professional decisions. The many complexities, challenges, and criticism are reviewed in Nelson and Hayes (1986) as well as other sources. Model mixes, such as adding traditional personality or behavior rating techniques to direct assessments of problem situations, which also may be described as a professional eclecticism, are not panaceas; they may lead to heightened sense of professional confidence at the same time that error rates may be increased, decision processes may be obfuscated, and time and resources may be depleted. The dilemmas of model mixes or nonsystematic eclectism have been examined often (Haynes & Uchigakiuchi, 1993; Williams & Thompson, 1993). In summary, to achieve confidence, the study of technical adequacy of assessment for specific professional practices is needed; this would be the technical adequacy of problem solving in natural contexts (Barnett et al., 2000; Macmann et al., 1996).

What Can Be Said That Is Helpful? An aspect of professional judgment is to determine the correspondence between personality constructs (or syndromes, traits) that appear to apply to the client and those that have an empirical foundation. Based on the characteristics of one’s client, logical generalizations can be made from researched interventions with children having similar characteristics. Many personality variables suggested by idiographic tech-

reyn2-1.qxd

6/6/2003

8:59 AM

Page 25

1. Personality Assessment Research

niques are widely researched (e.g., anxiety, depression, and aggression), and logical comparisons can be made between the idiographic client-based understanding of the construct, or how the client sees him- or herself, and its researched dimensions, the most likely nomothetic link for professional practice but one that does not necessarily meet the criterion of helpfulness. In other words, the client leads the way with regard to personality description (Mischel, 1973), but helpfulness is judged by outcomes. The “criterion” with respect to the adequacy of plans is that of resulting changes in behaviors. Intervention design typically involves a process of sequential decisions (Bandura, 1969) in contrast to diagnostic or classification decisions: “By retaining flexibility in the selection, sequencing, and timing of objectives, the treatment program remains highly sensitive to feedback from resultant changes. . . . Successful treatment . . . requires the selection and attainment of a variety of specific objectives rather than single omnibus outcome” (pp. 103–104). Time-series methods associated with single-case experimental designs can be used flexibly to analyze the effects of plans with a wide range of problem behaviors, interventions, and even “philosophies” (Barlow, Hayes, & Nelson, 1984). Replications are critical to the process (see Kazdin, Kratochwill, & VandenBos, 1986). Hayes and colleagues (1986) discuss specific research strategies that enable the analysis of the effects of assessment information on intervention decisions. Consequential Validity The major theme from our analysis is that psychometric issues are best examined by focusing primarily on the effectiveness of context-specific decisions that are made during problem-solving processes, and with an ultimate regard to the consequences of these decisions (Messick, 1995). Validity is a unitary concept that is studied within a specific decision context. The validity of an assessment procedure is ultimately judged by whether it accomplishes some clear purpose, and whether consequences for those assessed are positive. He argues that there is a sequence of decisions that needs be made

25

about whether any metric should be used to help make a specified decision, a sequence that begins with demonstration that a construct to be assessed is meaningful. The characteristic of a meaningful, interpretable construct organizes and subsumes the various types of validities that have been discussed in the literature Once construct meaning is demonstrated within Messick’s model, evidence supporting specific use for measuring the construct is needed. This is the point at which traditional psychometrics have typically stopped. However, Messick strongly advocates that there are values and consequences associated with assessment that are the final arbiters of whether assessment data should be used to make some decision. Thus, a construct that may be measured with meaningfulness, and useful for some purpose, may not have ultimate validity if consequences of use are detrimental.

CONCLUSION: NEEDED RESEARCH FOR PROFESSIONAL PRACTICES Long sought after, a major goal of personality research has been to meet the criteria of helpfulness: The hallmark has been the identification of traits, with the hope of identifying strong trait × treatment interactions. The difficulties are numerous: (1) limited consensus on the classification and measurement of traits; (2) limited consensus on intervention design; (3) treatment integrity issues; (4) experimental design issues, including the development of adequately sized groups for relatively rare syndromes, adequate control groups, and random assignment to groups; and (5) unique moderating effects for individuals and subgroups even within defined taxonomic groups or classifications. This has been a worthy goal, but it has been constrained by another highly significant problem: the inability to use personality appraisal techniques at appropriate levels of professional confidence. Furthermore, assessment practices need to encompass methods of assessing the potential impact of life events that alter developmental trajectories, including those that are accidental. New research is needed to address the criteria of confidence and helpfulness when

reyn2-1.qxd

6/6/2003

26

8:59 AM

Page 26

I. GENERAL ISSUES

methods are suggested for personality and social appraisals. This would the psychometry of professional practices based on problem solving.

REFERENCES Achenbach, T. M. (1985). Assessment and taxonomy of child and adolescent psychopathology. Beverly Hills, CA: Sage. Achenbach, T. M., & Edelbrock, C. S. (1978). The classification of child psychopathology: A review and analysis of empirical methods. Psychological Bulletin, 85, 1275–1301. Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington: University of Vermont, Department of Psychiatry. Achenbach, T. M., & McConaughy, S. H. (1997). Empirically based assessment of child and adolescent psychopathology (2nd ed.). Thousand Oaks, CA: Sage. Allport, G. W. (1937). Personality: A psychological interpretation. New York: Wiley. Allport, G. W., & Odbert, H. S. (1936). Trait-names: A psycholexical study. Psychological Monographs, 47(1, Whole No. 211). American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Authors. Anastasi, A. (1982). Psychological testing (5th ed.). New York: Macmillan. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice-Hall. Baer, D. M., Wolf M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97. Bakeman, R., & Gottman, J. M. (1986). Observing interaction: An introduction to sequential analysis. New York: Cambridge University Press. Bandura, A. (1969). Principles of behavior modification. New York: Holt, Rinehart & Winston. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. Barlow, D. H., Hayes, S. C., & Nelson, R. O. (1984). The scientist–practitioner: Research and accountability in clinical and educational settings. New York: Pergamon Press. Barnett, D. W. (1988). Professional judgment: A critical appraisal. School Psychology Review, 17, 656–670. Barnett, D. W., Lentz, F. E., Jr., & Macmann, G. M. (2000). Psychometric qualities of professional practice. In E. S. Shapiro & T. R. Kratochwill (Eds.), Behavioral assessment in schools: Theory, research, and clinical foundations (2nd ed., pp. 355–386). New York: Guilford Press. Barnett, D. W., & Macmann, G. (1992a). Decision reliability and validity: Contributions and limitations

of alternative assessment strategies. Journal of Special Education, 25, 431–452. Barnett, D. W., & Macmann, G. (1992b). Discrepancy score analysis: Accuracy in analysis misdirected. School Psychology Review, 21, 494–508. Barnett, D. W., & Zucker, K. B. (1990). The personal and social assessment of children: Current status and professional practice issues. Boston: Allyn & Bacon. Barrios, B., & Hartmann, D. P. (1986). The contributions of traditional assessment: Concepts, issues, and methodologies. In R. O. Nelson & S. C. Hayes (Eds.), Conceptual foundations of behavioral assessment (pp. 81–110). New York: Guilford Press. Block, J. H., & Block, J. (1980). The role of ego-control and ego-resiliency in the organization of behavior. In W. A. Collins (Ed.), Development of cognition, affect, and social relations (pp. 39–101). Hillsdale, NJ: Erlbaum. Burisch, M. (1984). Approaches to personality inventory construction: A comparison of merits. American Psychologist, 39, 214–227. Cairns, R. B., & Green, J. A. (1979). How to assess personality and social patterns: Observations or ratings? In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 209–226). Hillsdale, NJ: Erlbaum. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. Campbell, D. T., & O’Connell, E. J. (1982). Methods as diluting trait relationships rather than adding irrelevant systematic variance. In D. Brinberg & L. Kidder (Eds.), Forms of validity in research (pp. 93–111). San Francisco: Jossey-Bass. Cattell, R. B. (1982). The inheritance of personality and ability: Research methods and findings. New York: Academic Press. Cone, J. D. (1981). Psychometric considerations. In M. Hersen & A. S. Bellack (Eds.), Behavioral assessment: A practical handbook (pp. 38–68). New York: Pergamon. Cone, J. D. (1986). Idiographic, nomothetic, and related perspectives in behavioral assessment. In R. O. Nelson & S. C. Hayes (Eds.), Conceptual foundations of behavioral assessment (pp. 111–128). New York: Guilford Press. Costa, P. T., & McCrea, R. R. (1986). Personality stability and its implications for clinical psychology. Clinical Psychology Review, 6, 407–423. Costa, P. T. & Widiger, T. A. (1994). Personality disorders and the five-factor model of personality. Washington, DC: American Psychological Association. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–434. Cronbach, L. J. (1984). Essentials of psychological testing (4th ed.). New York: Harper & Row. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions (2nd ed.). Urbana: University of Illinois Press. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.

reyn2-1.qxd

6/6/2003

8:59 AM

Page 27

1. Personality Assessment Research Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. Crowley, S. L., & Fan, X. (1997). Structural equation modeling: Basic concepts and applications in personality assessment research. In J. A. Schinka & R. L. Greene (Eds.), Emerging issues and methods in personality assessment (pp. 285–308). Mahwah, NJ: Erlbaum. Dielman, T., & Barton, K. (1983). Child personality structure and development: Multivariate theory and research. New York: Praeger. Dudek, F. J. (1979). The continuing misinterpretation of the standard error of measurement. Psychological Bulletin, 86, 335–337. Edelbrock, C. (1983). Problems and issues in using rating scales to assess child personality and psychopathology. School Psychology Review, 12, 293–299. Emmerich, W. (1966, March). Stability and change in early personality development. Young Children, 233–243. Epstein, S. (1984). The stability of behavior across time and situations. In R. A. Zucker, J., Aronoff, & A. I. Rabin (Eds.), Personality and the prediction of behavior (pp. 209–268). New York: Wiley. Fiske, D. W. (1949). Consistency of the factorial structures of personality ratings from different sources. Journal of Abnormal and Social Psychology, 44, 329–344. Fiske, D. W. (1982). Convergent-discriminant validation in measurements and research strategies. In D. Brinberg & L. Kidder (Eds.), Forms of validity in research (pp. 77–92). San Francisco: Jossey-Bass. Friman, P. C., Hayes, S. C., & Wilson, K. G. (1998). Why behavior analysts should study emotion: The example of anxiety. Journal of Applied Behavior Analysis, 31, 137–156. Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981). Measurement theory for the behavioral sciences. San Francisco: Freeman. Goldberg , L. R. (1995). What the hell took so long? Donald Fiske and the big-five factor structure. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A Festscrift honoring Donald W. Fiske (pp. 29–43). Hillsdale, NJ: Erlbaum. Gresham, F. M. (1984). Behavioral interviews in school psychology: Issues in psychometric adequacy and research. School Psychology Review, 13, 17–25. Hansen, D. J., Tisdelle, D. A., & O’Dell, S. L. (1985). Audio recorded and directly observed parent–child interactions: A comparison of observational methods. Behavioral Assessment, 7, 389–399. Hayes, S. C., Nelson, R. O., & Jarrett, H. B. (1986). Evaluating the quality of behavioral assessment. In R. O. Nelson & S. C. Hayes (Eds.), Conceptual foundations of behavioral assessment (pp. 461– 503). New York: Guilford Press. Haynes, S. N. (1986). The design of intervention programs. In R. O. Nelson & S. C. Hayes (Eds.), Conceptual foundations of behavioral assessment (pp. 385–429). New York: Guilford Press. Haynes, S. N., & Uchigakiuchi, P. (1993). Incorporating personality trait measures in behavioral assess-

27

ment: Nuts in a fruitcake or raisins in a mai tai? Behavior Modification, 17, 72–92. Hibbs, E. D., & Jensen, P. S. (Eds.). (1996). Psychosocial treatments for child and adolescent disorders: Empirically based strategies for clinical practice. Washington, DC: American Psychological Association. Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Hillsdale, NJ: Erlbaum. Jones, R. R., Reid, J. B., & Patterson, G. R. (1975). Naturalistic observations in clinical assessment. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3, pp. 42–95). San Francisco: JosseyBass. Judd, C. M., Jessor, R., & Donovan, J. E. (1986). Structural equation models and personality research. Journal of Personality, 54, 149–198. Kanfer, F. H., & Grimm, L. G. (1977). Behavioral analysis: Selecting target behaviors in the interview. Behavior Modification, 1, 7–28. Karon, B. P. (1968). Problems of validities. In A. I. Rabin (Ed.), Projective techniques in personality assessment (pp. 85–111). New York: Springer. Kaufman, A. (1979). Intelligent testing with the WISCR. New York: Wiley. Kazdin, A. E. (1977). Assessing the clinical or applied significance of behavior change through social validation. Behavior Modification, 1, 427–452. Kazdin, A. E. (1985a). Alternative approaches to the diagnosis of childhood disorders. In P. H. Bornstein & A. E. Kazdin (Eds.), Handbook of clinical behavior therapy with children (pp. 3–43). Homewood, IL: Dorsey Press. Kazdin, A. E. (1985b). Selection of target behaviors: The relationship of treatment focus to clinical dysfunction. Behavioral Assessment, 7, 33–47. Kazdin, A. E., Kratochwill, T. R., & VandenBos, G. H. (1986). Beyond clinical trials: Generalizing from research to practice. Professional Psychology: Research and Practice, 17, 391–398. Kelly, G. A. (1955). The psychology of personal constructs (Vols. 1 & 2). New York: Basic Books. Kenny, D. A. (1995). The Multitrait–Multimethod Matrix: Design, analysis, and conceptual issues. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A Festscrift honoring Donald W. Fiske (pp. 111–124). Hillsdale, NJ: Erlbaum. Komaki, J. L. (1998). When performance improvement is the goal: A new set of criteria for criteria. Journal of Applied Behavior Analysis, 31, 263–280. Lewis, M. (1997). Altering fate: Why the past does not predict the future. New York: Guilford Press. Linehan, N. M. (1980). Content validity: Its relevance to behavioral assessment. Behavioral Assessment, 2, 147–159. Livingston, S. A. (1977). Psychometric techniques for criterion-referenced testing and behavioral assessment. In J. D. Cone & R. P. Hawkins (Eds.), Behavioral assessment: New directions in clinical psychology (pp. 308–329). New York: Brunner/Mazel. Loevinger, J. (1987). Paradigms of personality. San Francisco: Freeman.

reyn2-1.qxd

28

6/6/2003

8:59 AM

Page 28

I. GENERAL ISSUES

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum. Lord, F. M., & Novick, M. (1968). Statistical theory of mental test scores. Reading, MA: Addison-Wesley. Lumsden, J. (1976). Test theory. Annual Review of Psychology, 27, 251–280. Macmann, G. M., & Barnett, D. W. (1984). An analysis of the construct validity of two measures of adaptive behavior. Journal of Psychoeducational Assessment, 2, 239–247. Macmann, G. M., & Barnett, D. W. (1985). Discrepancy score analysis: A computer simulation of classification stability. Journal of Psychoeducational Assessment, 4, 363–375. Macmann, G. M., & Barnett, D. W. (1994). Structural analysis of correlated factors: Lessons from the Verbal–Performance dichotomy of the Wechsler Scales. School Psychology Quarterly, 9, 161–198. Macmann, G. M., & Barnett, D. W. (1997). Myth of the master detective: reliability of interpretations for Kaufman’s “Intelligent Testing” approach to the WISC-III. School Psychology Quarterly, 12, 197–234. Macmann, G. M., & Barnett, D. W. (1999). Diagnostic decision making in school psychology: Understanding and coping with uncertainty. In C. R. Reynolds & T. B. Gutkin (Eds.), Handbook of school psychology (3rd ed., pp. 519–548). New York: Wiley. Macmann, G. M., Barnett, D. W., Allen, S. J., Bramlett, R. K., Hall, J. D., & Ehrhardt, K. E. (1996). Problem solving and intervention design: Guidelines for technical adequacy. School Psychology Quarterly, 11, 137–148. Macmann, G. M., Barnett, D. W., Sharpe, M., Lombard, T. J., & Belton-Kocher, E. (1989). On the actuarial classification of children: Fundamental studies of classification agreement. Journal of Special Education, 23, 127–149. Magnusson, D., & Endler, N. S. (1977). Personality at the crossroads: Current issues in interactional psychology. Hillsdale, NJ: Erlbaum. Mahoney, M. J. (1980). Psychotherapy and the structure of personal revolutions. In M. J. Mahoney (Ed.), Psychotherapy process: Current issues and future directions (pp. 157–180). New York: Plenum Press. McFall, R. M., & McDonel, E. C. (1986). The continuing search for units of analysis: Beyond persons, situations, and their interactions. In R. O. Nelson & S. C. Hayes (Eds.), Conceptual foundations of behavioral assessment (pp. 201–241). New York: Guilford Press. Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–216. Meichenbaum, D., & Gilmore, J. B. (1984). The nature of unconscious processes: A cognitive-behavioral perspective. In K. S. Bowers & D. Meichenbaum (Eds.), The unconscious reconsidered (pp. 272–298). New York: Wiley. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35, 1012–1027.

Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: MacMillan. Messick, S. (1995). Validity of psychological assessment: Validity of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749. Millsap, R. E. (1995). The statistical analysis of method effects in Multitrait-Multimethod data: A review. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A Festscrift honoring Donald W. Fiske (pp. 93–109). Hillsdale, NJ: Erlbaum. Mischel, W. (1968). Personality and assessment. New York: Wiley. Mischel, W. (1973). Toward a cognitive social learning reconceptualization of personality. Psychological Review, 80, 252–283. Mischel, W. (1984a). Convergences and challenges in the search for consistency. American Psychologist, 39, 351–364. Mischel, W. (1984b). On the predictability of behavior and the structure of personality. In R. A. Zucker, J. Aronoff, & A. I. Rabin (Eds.), Personality and the prediction of behavior (pp. 269–305). New York: Academic Press. Mischel, W., & Peake, P. K. (1982). Beyond deja vu in the search for cross-situational consistency. Psychological Review, 89, 730–755. Moss, H. A., & Sussman, E. J. (1980). Longitudinal study of personality development. In O. G. Brim, Jr. & J. Kagan (Eds.), Constancy and change in human development (pp. 530–595). Cambridge, MA: Harvard University Press. Nelson, R. O., & Hayes, S. C. (Eds.). (1986). Conceptual foundations of behavioral assessment. New York: Guilford Press. Norman, W. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66, 574–583. Nunnally, J. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill. Peterson, D. R. (1968). The clinical study of social behavior. New York: Appleton-Century-Crofts. Plomin, R., DeFries, J. C., McClearn, G. E., & Rutter, M. (1997). Behavioral genetics (3rd ed.). New York: Freeman. Poth, R. L., & Barnett, D. W. (1988). Establishing the limits of interpretive confidence: A validity study of two preschool developmental scales. School Psychology Review, 17, 322–330. Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior assessment system for children. Circle Pines, MN: American Guidance Service. Ronka, C. S., & Barnett, D. W. (1986). A comparison of adaptive behavior ratings: Revised Vineland and AAMD-ABS-SE. Special Services in the Schools, 2, 87–96. Runyon, W. M. (1983). Idiographic goals and methods in the study of lives. Journal of Personality, 51, 413–437. Rushton, J. P., Brainerd, C. J., & Pressley, M. (1983). Behavioral development and construct validity: The

reyn2-1.qxd

6/6/2003

8:59 AM

Page 29

1. Personality Assessment Research principle of aggregation. Psychological Bulletin, 94, 39–53. Salvia, J., & Ysseldyke, J. E. (1998). Assessment (7th ed). Boston: Houghton Mifflin. Schinka, J. A., & Greene, R. L. (1997). Emerging issues and methods in personality assessment. Mahwah, NJ: Erlbaum. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books. Schulte, A. C., & Borich, G. D. (1985). Using confidence intervals appropriately. Paper presented at the annual meeting of the American Psychological Association, Los Angeles. Schwartz, I. S., & Baer, D. M. (1991). Social validity assessment: Is current practice state of the art? Journal of Applied Behavior Analysis, 24, 189–204. Sechrest, L. (1963). Incremental validity: A recommendation. Educational and Psychological Measurement, 23, 153–158. Sechrest, L. (1984). Reliability and validity. In A. S. Bellack & M. Hersen (Eds.), Research methods in clinical psychology (pp. 24–54). New York: Pergamon Press. Seligman , M. E. P., & Csikszentmihalyi, M. (2000). Positive psychology. American Psychologist, 55, 1–14. Sells, S. B., & Murphy, D. (1984). Factor theories of personality. In N. S. Endler & J. M. Hunt (Eds.), Personality and the behavior disorders (2nd ed., Vol. 1, pp. 39–72). New York: Wiley. Stagner, R. (1948). Psychology of personality (2nd ed.). New York: McGraw-Hill. Stagner, R. (1984). Trait psychology. In N. S. Endler & J. M. Hunt (Eds.), Personality and the behavior disorders (2nd ed., Vol. 1, pp. 3–38). New York: Wiley. Stevens, J. (1996). Applied multivariate statistics for the social sciences (3rd ed.). Hillsdale, NJ: Erlbaum. Thorndike, R. L. (1963). The concepts of over- and underachievement. New York: Teachers College, Columbia University. Thorndike, R. L. (1967). Reliability. In D. N. Jackson

29

& S. Messick (Eds.), Problems in human assessment (pp. 217–240). New York: McGraw-Hill. Thorndike, R. L. (1982). Applied psychometrics. Boston: Houghton Mifflin. Thorndike, R. L., & Hagen, E. (1961). Measurement and evaluation in psychology and education (2nd ed.). New York: Wiley. Tupes, E. C., & Christal, R. E. (1961). Recurrent personality factors based on trait ratings. (USAF-ASD Technical Report No. 61–97). Lackland Air Force Base, TX: U. S. Air Force. Tversky, A., & Kahneman, D. (1984). The framing of decisions and the psychology of choice. In G. Wright (Ed.), Behavioral decision making (pp. 25–41). New York: Plenum Press. Wainer, H., & Braun, H. I. (1988). Test validity. Hillsdale, NJ: Erlbaum. Wiggins, J. S. (1973). Personality and prediction: Principles of personality assessment. Reading, MA: Addison-Wesley. Willems, E. P. (1977). Steps toward an ecobehavioral technology. In A. Rogers-Warren & S. F. Warren (Eds.), Ecological perspectives in behavioral analysis (pp. 39–61). Baltimore: University Park Press. Williams, D. E., & Thompson, J. K. (1993). Issues confronting clinical behavioral therapy: Are we up to the challenge? Behavior Modification, 17, 4–7. Wirt, R. D., Lachar, D., Klinedinst, J. K., & Seat, P. D. (1984). Multidimensional description of child personality: A manual for the Personality Inventory for Children. Los Angeles, CA: Western Psychological Services. Wolf, M. M. (1978). Social validity: The case for subjective measurement, or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214. Wothke, W. (1995). Covariance components analysis of the multitrait-multimethod matrix. In P. E. Shrout & S. T. Fiske (Eds.), Personality research, methods, and theory: A Festscrift honoring Donald W. Fiske (pp. 125–144). Hillsdale, NJ: Erlbaum. Zsambok, C. E., & Klein, G. (Eds.). (1997). Naturalistic decision making. Mahwah, NJ: Erlbaum.

reyn2-2.qxd

6/6/2003

9:00 AM

Page 30

2 Procedural Issues Associated with the Behavioral Assessment of Children

CHRISTOPHER H. SKINNER JENNIFER T. FREELAND EDWARD S. SHAPIRO

A variety of reasons may account for the increased interest among educational and psychological professionals in behavioral assessment that has occurred during the last quarter of the 20th century (Shapiro & Kratochwill, 2000). Recently, legal changes related to inappropriate classroom behaviors have resulted in an increase in the need for training in behavioral assessment theory and practice. Specifically, the 1997 Amendments to the Individuals with Disabilities Education Act (IDEA) require that individual education plan (IEP) teams address behavioral challenges of students with disabilities. These changes include requirements for conducting functional behavioral assessments for the purpose of intervention development. Furthermore, schools are charged with using postintervention data to evaluate intervention effectiveness and alter interventions as needed (Nelson, Roberts, Rutherford, Mathur, & Aaroe, 1999; Yell & Shriner, 1997). Both the language (e.g., functional assessment) and the emphasis on linking assessment to intervention is causing increased attention to behavioral assessment procedures. The amendments to IDEA were preceded and influenced by advancements in behav-

ioral assessment research and procedures. Specifically, methodological advances in behavioral assessment procedures (e.g., experimental functional analysis and descriptive functional assessment) have been shown to be effective for producing data that lead practitioners and researchers to effective interventions (Iwata, Dorsey, Sifler, & Richman, 1982; Lalli, Browder, Mace, & Brown, 1993; McComas, Hoch, & Mace, 2000; O’Neill et al., 1997). Additional reasons for an increase interest in behavioral assessment procedures include a dissatisfaction with indirect measures, frustration by consumers with the failure of traditional models of assessment to lead to more effective interventions, a lack of accountability at the individual student level associated with the use of indirect measures, and a desire for more flexible assessment procedures that can serve a variety of purposes (Reschly & Ysseldyke, 1995; Shapiro, 1987; Telzrow, 1999). A broader more inclusive conceptual framework of behavioral assessment procedures that include more traditional assessment methodologies also may have increased the number of professionals, researchers, and educators inter30

reyn2-2.qxd

6/6/2003

9:00 AM

Page 31

2. Procedural Issues

ested in behavioral assessment (Cone, 1978; Nelson & Hayes, 1979, 1986). Regardless of the reasons for the increased interest in behavioral assessment procedures, psychologists, educators, counselors, social workers, and others who work with children are likely to be expected to be able to understand and apply behavioral assessment procedures (Barnett et al., 1999; Hendrickson, Gable, Conroy, Fox, & Smith, 1999). In the next section, we briefly describe characteristics of behavioral assessment procedures that may serve to differentiate them from other more traditional assessment procedures. More traditional assessment procedures such as interviews, checklists, and rating scales have been incorporated into a behavioral framework and applied to the assessment and treatment of children (see Shapiro & Kratochwill, 2000). Because other chapters in this series cover many of these procedures, this chapter focuses on direct behavioral assessment procedures (i.e., direct observation in natural and analogue environments) and issues related to the application of these procedures across behaviors and settings.

ASSUMPTIONS REGARDING CAUSES OF BEHAVIOR Hartmann, Roper, and Bradford (1979) provided a comprehensive overview that contrasts behavioral and traditional assessment with respect to assumptions, implications, use of data, and other characteristics. Each of the differences can be traced to primary assumptions associated with causes of behavior. Under a behavioral model, behaviors are caused and/or maintained by current environmental conditions and past learning history. However, with more tradi-

31

tional models of psychology, behavior is seen as caused by intrapsychic or withinchild traits, conditions, or mediating variables. Under more traditional models of assessments, overt behavior is measured and used to infer these within-child variables assumed to cause these behaviors. Thus, a child’s inappropriate behavior may be caused by his passive–aggressiveness, attention-deficit disorder, or faulty cognitions. Because these conditions that are thought to cause behaviors are also seen as relatively stable, traditional models of assessment have a history of being used for identifying, classifying, or diagnosing problems and predicting future behavior (Hartmann et al., 1979). Under a behavioral model, behavior is not seen as a mere symptom of some other underlying (i.e., within-child) problem. Rather, behaviors are viewed as legitimate problems in and of themselves. However, the most important distinction may be that behavioral theory does not make large leaps of influence and attribute the cause of a child’s behavior to some underlying construct. Instead, current behaviors, which are directly observable are maintained by current environmental events (e.g., antecedent and consequent stimuli) that in many instances are also directly observable. Thus, under a behavioral model, both the behaviors of interest and the events thought to maintain behaviors can be assessed more directly, regardless of what procedures are used to measure the behavior and/or environmental events that may maintain the behaviors (Carr, 1993; Gresham, 1998; Skinner, Dittmer, & Howell, 2000). In addition to lending themselves to direct assessment, environmental variables that are thought to maintain behaviors under a behavioral model are often mutable. For this reason, behaviorists have been less likely to focus all their assessment efforts on identifying and measuring problems or problem behaviors. Instead, behavioral psychologists are equally if not more concerned with developing procedures for identifying and measuring environmental events that are thought to cause these behaviors (Carr, 1993; Gresham, 1998). Once variables that cause problem behaviors are identified, interventions designed to alter target behav-

reyn2-2.qxd

6/6/2003

9:00 AM

Page 32

32

I. GENERAL ISSUES

iors that are based on assessed causes of these behaviors can be developed (e.g., McComas et al., 2000; Myerson & Hale, 1984).

BEHAVIORAL ASSESSMENT PROCEDURES Shapiro and Browder (1990) present a continuum of behavioral assessment procedures from most direct to least direct. Indirect assessment procedures include self-report and informant report measures (e.g., checklists, rating scales, and interviews) that are covered in other chapters in the current series. Direct behavioral assessment procedures include direct observation in natural settings, direct observation in analogue settings, and self-monitoring. Additional direct observation procedures require peers, parents, teachers, or others who are a part of the child’s natural environment to observe and record data (Skinner, Rhymer, & McDaniel, 2000). With each of these procedures, behaviors can be observed and recorded as they occur. Furthermore, each of these direct assessment procedures can employ similar data collection procedures. The primary difference across procedures is who collects the data (independent observers, parents, teachers, peers, or target children themselves) and under what conditions (natural vs. analogue or artificial environmental conditions). Next, we describe general procedures for directly observing and recording behaviors. Then, we describe issues related to specific direct observation data collection and procedures.

RECORDING DIRECT OBSERVATION DATA A variety of procedures can be used to record direct observation data. These procedures have relative strengths and weakness related to (1) the goal of data collection; (2) the rates, topography, and duration of target behaviors; (3) the conditions under which data collection must occur; and (4) physical, temporal, and resource constraints related to data collection.

Narrative Recording Procedures Benefits of Narrative Recording Procedures The least structured form or narrative recording merely requires an observer to write narrative descriptions of behaviors and sometimes events surrounding those behaviors. Narrative recording procedures are often used to communicate general information about a child’s behavior (e.g., Johnny seemed tired today). Narrative recordings can also be used in the initial stages of problem solving to help (1) identify, validate, or confirm target or problem behaviors; (2) to form a general idea of problem behaviors rates, intensity, and topography; and (3) to begin to identify variables that may be serving to maintain target or problem behaviors (Skinner, Rhymer, & McDaniel, 2000). Although there are numerous narrative recording procedures and a variety of reasons to collect narrative recordings, daily communication logs, descriptive time sampling, and Antecedent–Behavior–Consequence (A–B–C) analysis provide a fairly broad overview of these procedures. Daily communication logs can allow one to record data that can be used to track a child’s general progress or to communicate with others who also work with a child. For example, a teacher working with students with autism may write a general description of a child’s school behavior to be sent home to the child’s parents. A parent of a child with social–emotional problems may be asked to make similar recordings for a child’s therapist (e.g., Johnny appears to be becoming less anxious). These narrative descriptions are useful because individuals can provide flexible and rich descriptions of a child’s behavior that may assist others who are working with that child. If specific behaviors are of concern (e.g., self-injurious behavior during school), narrative recordings can provide data about the general topography (e.g., scratching self vs. slapping self), intensity (hitting self hard), rate (hitting self infrequently), or duration of a behavior. These data can prove useful when developing more systematic observation procedures. For example, descriptions of the topography of a behavior can be used to help develop operational definitions. Data on behavior rates and durations are useful when one is developing interval

reyn2-2.qxd

6/6/2003

9:00 AM

Page 33

2. Procedural Issues

recording procedures. Furthermore, narrative recordings can provide a general indication of behavior variability and the conditions when the target behavior is likely to occur (hitting self only in the first hour of school). These data can be used to determine when observers may be more likely to have the opportunity to directly observe target behaviors. In addition, because data on variability may provide some initial information regarding conditions that may be serving to maintain target behaviors (e.g., is disruptive during independent seat-work time), these data could prove useful for developing intervention procedures. Narrative recordings are often used to keep records of specific low-rate behaviors or events. Following unusual or dangerous events, teachers, parents, or staff often use narrative recording when completing “incident reports.” When behaviors are clear and obvious and occur at low rates, incident reports can be converted to frequency counts of specific behaviors or events (e.g., number of times a child became physically aggressive). In this manner, narrative recordings can sometimes be used to determine whether students’ behaviors are increasing, decreasing, or remaining stable. Although narrative recording can sometimes be translated into empirical data for low-rate behaviors or events, narrative time sampling may allow one to collect data on more typical behaviors and conditions. Narrative time sampling requires observers to write a narrative recording of a student’s behavior at predetermined intervals. For example, every 2 minutes an observer could look at a child and then write what the child was doing at the time and record other environmental conditions that were present at that time. This type of momentary time sampling can provide an estimate of rates of specific behaviors (Shapiro & Skinner, 1990). A–B–C narrative recording requires observers to record narrative descriptions of both behaviors of interest and antecedent and consequent conditions that may be functionally related to those target behaviors. Typically, observers use the occurrence of target behaviors as a cue to write a description of the (1) target behavior; (2) general antecedent conditions (e.g., class completing independent seat-work assignment

33

involving punctuation) and specific antecedent events (e.g., a peer takes a students pencil), that precede target behaviors; and (3) consequent events or events that immediately followed the target behavior. Limitations of Narrative Recording Procedures Although narrative recording procedures can be extremely useful for identifying or verifying behaviors and environmental variables that may be maintaining those behaviors, there are several limitations associated with narrative recording procedures. By definition, narrative recordings involve nonsystematic data recording procedures. Although this method of recording may allow for rich, flexible data collection, narrative recordings often yield imprecise data. For example, an observer may spend some time with a family and write that Johnny disobeyed his parents often. Without a more precise definitions of the words “disobey” and “often” it is difficult to use these data to determine what was actually occurring during the observation period. In addition, the quality and quantity of narrative data are likely to be influenced by many variables, including the observer’s writing skills (e.g., writing speed and vocabulary) and his or her training in recognizing and recording environmental events that may be related to target behaviors (e.g., recognizing that shifting from one type of mathematics problem to another may occasion escape–avoidance behaviors). Unless behaviors or events of interest are extremely obtrusive or obvious, discreet, uncomplicated, and occur at low rates, it is often difficult, if not impossible, to verify data collected via narrative recordings (Shapiro & Skinner, 1990). A second major limitation of narrative recording procedures is related to the time required to write narrative descriptions. Because it takes considerable time to write narrative descriptions, observers cannot record all behaviors or events observed. Furthermore, because it takes time and attention to write narrative description, narrative recording procedures often require discontinuous observation (i.e., observers halt observation in order to record data). Discontinuous observation has several limitations. Because the observer is not observ-

reyn2-2.qxd

6/6/2003

9:00 AM

34

Page 34

I. GENERAL ISSUES

ing behaviors continuously (i.e., they pause to write out their narratives) observers may miss the opportunity to observe and record important antecedent or consequent events that could lead to effective interventions (Skinner, Dittmer, & Howell, 2000). For example, while busy recording target behaviors, observers may fail to observe an event that occurred after the target behavior and is reinforcing that target behavior. Fortunately, a variety of empirical recording procedures have been developed that can address many of the limitations associated with narrative recordings.

Empirical Recording Empirical recording procedures yield more precise measures of behavior than narrative recoding procedures. This precision can allow for the fine distinction necessary for measuring the effects of interventions over brief periods. In addition, empirical recording procedures yield data that can be independently verified. Verification is especially useful when (1) important decisions are being made (e.g., is the child’s self-injurious behavior getting worse or is it severe enough to warrant alternative placement), (2) primary observers may be susceptible to biases (e.g., the residential staff member who has recently had a physical altercation with the student is collecting data), or (3) when the behaviors being measured are difficult to observe and record. Empirical recording systems typically require observers to record tally marks in appropriate columns or interval blocks when target behaviors are observed. Because this data recording procedure is much more time efficient than writing a narrative description, empirical recording often allows observers to collect data in a more continuous manner and also record observations across a greater array of behaviors or events in a relatively brief period of time. These advantages may make it easier to precisely measure target behaviors. In addition, efficient recording may increase the probability of observers being able to observe and record important antecedent and consequent events that may be maintaining these behaviors.

Operational Definitions To record the presence of a target behavior, the behavior must first be operationally defined. Operational definitions are typically based on the topography or shape of the behavior. In some instances the intensity or duration of behaviors is also included in operational definitions. For example, “inappropriate voice” may include speaking too loud or too soft (intensity) and “passive– aggressive behavior” may include complying with mothers directions within 5 seconds of the direction being issued (duration). However, to avoid imprecise data collection, operational definitions rarely include inferred characteristics of the behavior such as intent or purpose of the behavior (Skinner, Dittmer, & Howell, 2000). Several procedures are useful for developing operational definitions of target behaviors. The typical procedure is to collect descriptive data from the referring agent (e.g., teacher or parent). Often referrals are accompanied by broad, vague descriptions of problem behaviors and interviews can be used to form more precise definitions of target behaviors (Bergan & Kratochwill, 1990). Including examples of behaviors that fit an operationally defined behavior and those that do not fit can help clarify operational definitions (see Saudargus, 1992). In other instances, it is easier to collect direct observation data when target behaviors are operationally defined to include an entire class of behaviors. For example, physical aggression against peers could include any instance of biting, hitting, or kicking a peer. However, it is difficult and often not meaningful to record each particular bite, hit, or kick. Instead, aggressive instances could be defined as beginning when a child engages in any of the previously mentioned behaviors and ending with the absence of those behaviors over a 2-minute interval. When target behaviors are unusual or idiosyncratic (e.g., hand flapping), using narrative recording procedures to develop a description of the behavior may help one develop operational definitions. This observation time may also help one obtain a clearer picture of the duration, rate, and continuity of the target behaviors which is often necessary when developing direct observation recording procedures. Finally, it is

reyn2-2.qxd

6/6/2003

9:00 AM

Page 35

2. Procedural Issues

not always necessary to construct a new operational definition for every referred problem. In many instances, it may be possible to find appropriate operational definitions in empirical intervention journal articles, behavioral psychology text, and structured codes.

Data Recording Procedures Developing direct observation systems also requires the development of data recording procedures. Successfully matching the data recording procedures to the target behavior can make data collection more reliable and enhance the social, educational, and clinical validity of the data collected. Event Recording Event recording requires an observer to record the number of times a behavior occurs during an observation session. Event recording is often used to collect data on discrete behaviors that have a clear beginning and end (e.g., leaving one’s seat). However, because every instance of the behavior is recorded, it can be difficult to collect data on behaviors that occur at high rates (e.g., pencil tapping). Event data can be reported as frequency counts (e.g., Johnny did not comply with his father’s directions on 20 occasions and Jane had six aggressive outburst). However, converting data to either rate or percentage data often provides more useful information. For example knowing that Johnny failed to comply with his father’s directions 10% of the time or 20 out of 200 requests provides much more meaningful data regarding Johnny’s compliance. Reporting frequency counts as rate data (e.g., six aggressive outbursts for the schoolweek vs. a schoolday) makes frequency counts more meaningful. Converting frequency counts to rate or percentage data allows one to compare data across observation periods when opportunities to engage in behaviors or interval lengths are unequal. Thus, these types of conversions are extremely useful when repeated measures data are used to (1) analyze behavior trends, (2) analyze behavior variability, and/or (3) compare a child’s behavior across conditions (e.g., baseline vs. treatment phases).

35

Duration Recording Whereas event recording is often used for discrete behaviors that occur at moderate or low rates, duration recording can be used to collect data on continuous behaviors or behaviors that occur at extremely high rates. It is possible for an observer to use a stopwatch to record the amount of time a child spends engaged in specific behaviors. However, it is often difficult to start and stop a stopwatch in a reliable manner. Furthermore, with continuous behaviors it can be difficult to reliably determine when a behavior begins and ends. For example, if on-task were defined as head oriented toward assignments or a speaker, an observer would be constantly starting and stopping the stop watch every time the student reoriented. Fortunately, time-sampling procedures allow observers to collect reliable duration estimates. There are three types of time sampling (i.e., momentary, whole interval, and partial interval) and each requires observers to record behaviors on an interval-by-interval basis. Although observers can use a clock, stopwatch, or wristwatch to mark intervals, using audiotapes to mark intervals may make it easier to observe and record student behavior. Marking intervals with audiotapes may also make it easier to synchronize recording intervals when direct observation data are being verified by another observer (e.g., when collecting interobserver agreement data). Momentary time sampling requires observers to record the presence of a target behavior if it is occurring at the moment the interval begins. When whole interval time sampling is used, observers record the presence of a target behavior only if it occurs for the entire interval. With partial interval time sampling, an interval is scored if the behavior occurs at any point during an interval. Momentary time sampling is often used when data collection systems are complex and require observers to record a variety of behaviors and events. Whole and partial interval time sampling often require more continuous observation (e.g., observe for entire intervals). Thus, these procedures are less useful when observers are attempting to collect data on multiple behaviors and events or across multiple people (e.g., several peers of a child and the child’s parents and siblings).

reyn2-2.qxd

36

6/6/2003

9:00 AM

Page 36

I. GENERAL ISSUES

Time-sampling methods involve sampling and therefore are susceptible to sampling error. Whole interval time sampling tends to underestimate behavior durations and partial interval time-sampling procedures overestimate behavior durations (Lentz, 1982; Powell, Martindale, Kulp, Martindale, & Bauman, 1977). The length of observation intervals can also affect time-sampling estimates. Longer intervals tend to amplify overestimates of partial interval time sampling and shorter intervals will tend to underestimate estimates provided by whole interval time sampling. When momentary time sampling is being used, shorter intervals would increase the size of the sample collected during an observations session, thus reducing sampling error of the duration estimates. Because time sampling provides estimates of durations, when reporting data observers should not indicate the percentage of time the students was engaged in target behaviors. For example, it would be inaccurate to report that Jim was engaged in self-injurious behavior 80% of the time. Instead one should report the length of the observation sessions (e.g., 10 minutes), the length of the intervals (e.g., 20 seconds), the type of time sampling recording being used (e.g., momentary time sampling), the number of intervals the behavior was recorded, and the number of intervals the behavior was not recorded (e.g., self-injurious behavior was recorded for 24 of 30 intervals) and the percentage of intervals the behavior was recorded (e.g., self-injurious behavior was recorded for 80% of the observed intervals).

Interval Recording, Sequential Events, and Direct Observation Systems Interval recording can be used to estimate the duration of time children are engaged in specific behaviors. Frequency data can also be recorded using intervals. This is especially useful when observers are collecting data on both target behaviors and variables thought to be maintaining those behaviors in natural environments. Recording events that occur during intervals provide a more precise record of the sequence of events. Thus, using interval recording observers can identify events that occurred immediately prior to and following target behaviors (see

Saudargus, 1992). To the extent that these events are serving to occasion and maintain target behaviors, this type of recording may provide data that lead to more effective interventions.

OBSERVERS AND ENVIRONMENTS Several issues are related to direct observation that can affect the quality and clinical utility of direct observation data. These issues are affected by who collects the data (e.g., independent observer, participant observer, peer observer, and/or self-observer) and under what conditions data are collected (i.e., natural environments vs. analogue environments).

Independent Observer Collecting Data in Naturalistic Settings Having an independent observer enter a child’s natural setting (e.g., home or school) and record the child’s behavior and environmental events surrounding those behaviors is the most direct and often considered the most desirable form of behavioral assessment (Cone, 1978; Hintze & Shapiro, 1995). However, there are several major concerns about using independent observers to collect data in a child’s natural environments. Perhaps the most important concern is that the process of the data collected will affect the child’s and others’ (e.g., parent, teacher, and peers) behavior. This process is known as reactivity. A variety of variables including (1) obtrusiveness of the observer, (2) perceived power or role of observer, and (3) what the child is told about the observer’s presence may affect reactivity (Johnson & Bolstad, 1973). However, it is not possible to predict or measure the impact that reactivity is having on behavior(s) during any particular observation session (Shapiro & Skinner, 1990). Therefore, when the goal is to collect assessment data that most accurately reflect naturally occurring conditions, observers should make efforts to reduce reactivity. First, efforts should be made to reduce the conspicuousness of the observer. Oneway mirrors are an excellent device that allow one to collect data in an inconspicious manner. However, often independent ob-

reyn2-2.qxd

6/6/2003

9:00 AM

Page 37

2. Procedural Issues

servers cannot conceal their presence in the child’s natural environment. When this is the case, children are likely to ask questions about the observer. In these instances, teachers, parents, or others who are supervising the children should not lie to the children, but they should avoid providing too much information. For example, informing a class of fourth-grade students that the observer is there to collect data on John because people are concerned about his inappropriate behavior could be both unethical and illegal but clearly is likely to cause reactivity. Not only is John’s behavior likely to be affected, John’s peers are likely to interact differently with John. Thus, it may be best to provide vague, general statements about why the observer is present. For example, children could be told that Mrs. Smith is here to observe and learn more about fourth-grade classrooms. Children often attempt to interact with independent observers (Johnson & Bolstad, 1973). Observers should not respond to students, as even quick social interactions may encourage children to continue initiating other interactions. These interactions with observers who would not be present if they were not collecting direct observation data are obvious examples of reactivity. Not all interactions are verbal. Children may engage in nonverbal behavior (e.g., writing on a piece of furniture) while they know that an independent observer is watching them in order to see if the observer reacts (e.g., tell their parents and take away their pen). If observers react, the child’s future behavior is likely to be strongly influenced by the presence or absence of the observer. Of course, there are exceptions to this rule. If there is a clear and present danger (e.g., a child is leaning far out an open window or the observer sees a child pointing a knife at another child) observers are legally and ethically obligated to react. Several other procedures can be used to reduce reactivity (Johnson & Bolstad, 1973; Kazdin, 1977; Skinner, Dittmer, & Howell, 2000). Observers should not orient themselves directly toward the target child or children. Staring directly at target children can only increase reactivity. Observers should sit in an area of the room where they are less likely to be noticed (e.g., the back of the classroom). When possible, observers

37

should attempt to enter the natural environment at unobtrusive times (e.g., enter a classroom before students do). To children, it can appear that observers are not busy. This may increase the probability of children approaching the observer. Therefore, observers who appear extremely busy (e.g., constantly writing or quietly shuffling materials) may be less likely to occasion reactivity during observation periods. Timing related to other assessment activities may also be related to reactivity. For example, psychologists often interview children as part of the assessment process. When the person who interviewed the specific child enters the classroom, that child is likely to behave differently (i.e., reactivity) because of the interview. In these instances, reactivity may be reduced by delaying the child interview until direct observation data are collected. Another solution would be to have one person conduct the interview and another collect the direct observation data. Although independent observers may occasion reactivity, this reactivity is likely to subside over time as children and others in the environment return to their typical behavior. Thus, initial observation sessions may yield less naturalistic data than subsequent sessions. Video cameras can also be used to record data that observers can record at a later date. As with the presence of an outside observer, the presence of a video camera is likely to occasion high levels of initial reactivity (e.g., mugging for the camera and “Hi Mom”) that tend to subside as children become acclimated to the equipment. There are several advantages to using independent observers to collect direct observation data. Because external observers do not have a history of interacting with the child, they may be less susceptible to observer biases that could cause inaccurate or inconsistent data recording. Learning to collect direct observation data in a reliable manner can be time-consuming. It may be most efficient to train a few external observers and use them as direct observation specialists to collect data across environments (e.g., across classroom or homes). One way to make this system even more efficient would be to train these observers to collect data using preestablished direct observation systems (for examples of class-

reyn2-2.qxd

6/6/2003

9:00 AM

Page 38

38

I. GENERAL ISSUES

room observation systems, see Alessi & Kaye, 1983; Saudargus, 1992; Shapiro, 1996). These systems have been developed to allow observers to collect data on multiple behaviors, events, and conditions that may provide useful information for making decisions across problems and concerns. After becoming fluent with collecting data using these codes, observers can then make alterations in the system on a case-by-case basis (e.g., add a specific operational definition and recording procedure for a child’s idiosyncratic behavior).

Teacher, Parent, or Other Internal Observer in Natural Settings When target behaviors occur at low rates and/or unpredictable times it may not be practical for external observers to record direct observation data. For example, a child may have a tantrum about once every week where he throws and damages objects in his environment. Because this behavior occurs infrequently, it is difficult for an independent observer to schedule times when they can observe this behavior. In these instances, parents, teachers, or other adults who are part of the child’s natural environment can be used to record direct observation data. When working with teachers, parents, or others who are working or supervising children in their natural environments, it may be important to construct relatively simple recording systems that require little time to record data. For example, a teacher could keep a daily frequency count of a student’s compliant and noncompliant behaviors by moving a penny from one pocket to another every time a student complied and moving a dime every time a student did not comply. When interval recording systems are being used, manipulating interval length can also make data collection easier. For example, it is much easier for a parent to record whether a child wet the bed during an entire night than it is to record whether a child wet the bed during a specific hour interval. Altering operational definitions can also make collecting data easier. For example, rather than measuring a student’s in-seat behavior by determining if one or both buttocks are in contact with the chair seat, one could record whether or not any part of the student’s body is in contact with this desk or

chair. In addition, care should be taken with procedures used to mark intervals. Using natural occurring intervals may reduce disruptions (e.g., intervals that last from waking to lunch, lunch to dinner, and dinner to bedtime or intervals that last an entire class period during schooldays). Furthermore, some procedures for marking intervals are likely to be less obtrusive and disruptive. For example, using a wristwatch beep as opposed to a cooking timer may allow a teacher to mark intervals in a manner that is less likely to disrupt typical classroom behaviors and routines. When using parents, teachers, or other adults who are part of a child’s natural environment to record observations, the process of data collection could have a great impact on how they interact with the target child. For example, suppose a parent is collecting data on a child’s compliant and noncompliant behavior by moving pennies and dimes from one pocket to another. While collecting these data, the parent may come to realize that the child is rarely complying and reduce the number of requests or demands made on the child. This reduction in the number of demands may increase or decrease the child’s compliance rate. Thus, the reactivity occasioned by parents or other internal observers collecting data can alter the child’s natural environment and reduce the clinical utility of the data collection procedures. Internal observers who have a history of interacting with a child may also be more susceptible to observer drift. For example, a parent who has listened to her child scream all day may be more (frustration) or less (habituation) likely to score a child’s request as inappropriate verbal behaviors. Finally, parents, teachers, or others who work with a child may have a stake in the decisions being made based on that data. When this is the case, people who are making the decisions may be less likely to trust that data (e.g., a judge making a custody decision) when it is collected by a stakeholder (e.g., the father).

Peer Observation and Recording Children can be used to collect direct observation data on the behavior of classmates, siblings, or peers. The reactivity associated

reyn2-2.qxd

6/6/2003

9:00 AM

Page 39

2. Procedural Issues

with peer monitoring may occasion both desired and undesired behaviors, dependent on the behavior(s) being monitored. Within educational environments, tutoring programs often require peers to observe and record classmates academic behaviors (e.g., Greenwood et al., 1987). Peer monitoring is an efficient procedure that can allow for immediate evaluation of academic responses. This immediate evaluation can cue immediate error correction procedures that prevent students from practicing inaccurate responding. In addition, immediate evaluation can serve as immediate reinforcement for accurate responding (McLaughlin & Skinner, 1996). Besides enhancing tutee’s academic performance, the process of observing, recording, and evaluating peers’ responses has been shown to improve the tutor’s or monitor’s academic skills (e.g., Skinner, Shapiro, Turco, Cole, & Brown, 1992). Although there may be a tendency to have older, more mature students, or students with stronger academic skills, serve as monitors or tutors, researchers have trained younger students, same-age peers, and students with disabilities to observe, record, and evaluate their peers’ academic responses (CardenSmith & Fowler, 1984; McCurdy & Shapiro, 1992). Serving as a peer monitor or tutor may also enhance a student’s selfimage and academic esteem (Henington & Skinner, 1998; Stern, Fowler, & Kohler, 1988). Children can also observe, record, and report peers’ incidental social behaviors. In many environments, incidental antisocial behaviors are punished. In some instances, children learn to avoid punishment by not performing those behaviors. In other instances, children learn to avoid being punished by avoiding being caught or observed performing these behaviors. For example, it would be unusual for some children to forcefully take a toy from a peer when a parent or teacher was obviously watching. Thus, in some instances peers may be the only people who observe a child’s inappropriate behaviors. When these behaviors are serious (e.g., bringing a gun to school or torturing an animal), it is essential that children who observe these behaviors also report these behaviors. Often children learn to monitor and report their peers’ incidental antisocial behav-

39

ior (i.e., tattle) without any specific or programmed instruction (Skinner, Cashwell, & Skinner, 2000). Therefore, it should be relatively simple to train and encourage children to observe and report peers’ inappropriate behaviors. All methods of recording data described earlier could be used by students to record their peers’ behavior. However, peers may have difficulty recording behaviors that occur at high rates or accurately recording behaviors using complex observation systems. In addition, having children write narrative reports can be time-consuming and the quality of these data may be compromised depending on students’ writing (e.g., speed and vocabulary) and observation skills. Thus, in some instances, it may be best to train children to report peers incidental inappropriate behavior (e.g., tattle) and have adults record these behaviors (Henington & Skinner, 1998). One procedure used by an elementary school teacher was to have children describe their peers’ behavior into a tape recorder. There are several limitations associated with using peers to collect data on antisocial behaviors (Henington & Skinner, 1998; Skinner, Cashwell, & Skinner, 2000). Observing peer behavior can distract children’s attention from their own behavior or their teacher’s, parent’s, or supervisor’s instructions or directions. Peers may intentionally or unintentionally provide inaccurate reports of peers’ behavior. Intentionally inaccurate reports can be used as a form of aggression to get other children in trouble. Such reports can also be used to shift the blame for a behavior to another peer. Peers may threaten their classmates to prevent them from reporting behaviors or physically assault them in retaliation for reporting those behaviors. Requiring children to monitor inappropriate behaviors may teach children to focus on their peers’ inappropriate behaviors to the exclusion of their peers’ appropriate behaviors. This may reduce children’s awareness of their peers’ incidental prosocial behaviors and cause children to form negative impressions of their peers. Finally, it would be inappropriate to have peers collect data on the inappropriate behavior of a particular child, as this process would encourage peers to view this child as being deviant or bad and may increase the probability of

reyn2-2.qxd

6/6/2003

9:00 AM

40

Page 40

I. GENERAL ISSUES

children socially rejecting their peer (Cashwell, Skinner, Dunn, & Lewis, 1998; Ervin, Miller, & Friman, 1996). Although there are some serious negative side effects associated with using children to collect data on peer’s incidental inappropriate behaviors, having children observe, record, and report peers incidental prosocial behaviors could have several positive side effects. Researchers have attempted to use peer observation and peer reporting to improve the social interactions and social status of socially rejected children and/or children who displayed high rates of inappropriate behaviors. Results have shown that having groups of children publicly report peers’ incidental prosocial behaviors at daily group meetings can improve the student’s social status, increase interactions, and increase cooperative interactions among students (Bowers, McGinnis, Ervin, & Friman, 1999; Ervin, Johnston, & Friman, 1998; Ervin et al., 1996; Jones, Young, & Friman, 2000; Robinson, 1998). Although there are positive side effects associated with having children observe and report peers’ incidental prosocial and academic behaviors, these side effects represent reactivity. It is not possible to separate behavioral changes caused by reactivity versus behavioral changes caused by planned (e.g., interventions) or unplanned changes in the student’s immediate or temporally distant environments. Thus, even when peer monitors collect accurate and reliable data, interpretation of these data must be tempered with the possibility that these data do not reflect natural environmental conditions. Furthermore, the process of peer monitoring can have a significant impact on the behavior of target children and peer monitors. This impact can be significant enough to cause broad changes within the child’s natural environment that may alter how others (teachers and parents) interact with the child. Because this reactivity may be unpredictable, the use of peer monitoring requires careful monitoring for reactivity across individuals in the environment and all decisions based on these data must be tempered with the understanding that reactivity may have altered naturally occurring behaviors.

Self-Monitoring

Self-monitoring requires children to observe and record their own behavior. There are several advantages to using self-monitoring over other direct observation procedures (Cole, Marder, & McCann, 2000). Selfmonitoring is an efficient procedure for collecting data when other data collectors are unavailable or too busy. In addition, selfmonitoring may be the only procedure that allows one to collect data that are difficult for others to observe, such as (1) cognitive or emotional behaviors (e.g., fear or cognitive steps taken to solve a mathematics problems), (2) behaviors that occur at low rates, (3) behaviors that occur at unpredictable times, and (4) behaviors that occur in settings that do not lend themselves to direct observation by others. All previously described procedures for observing and recording data can be employed when self-monitoring is used. However, as with peer monitoring, several variables should be taken into account when structuring self-monitoring procedures. With young children and children with writing skills deficits, narrative recording may prove cumbersome and yield data that are difficult to interpret. It may also be difficult for children to observe and record several behaviors or events simultaneously (Nelson, 1977). Because children may find it timeconsuming and difficult to consistently make subtle distinctions in behaviors, operational definitions should also be both clear and described in simple terms. In some instances, self-monitoring procedures can disrupt children who are engaged in desirable behaviors. For example, if a student is having problems maintaining her on-task behavior, interrupting that student at random intervals and requiring her to record whether she was reading silently could make it more difficult for this student to remain on task and finish assignments (Skinner & Smith, 1992). As with all other direct observation procedures, reactivity and accuracy are a concern when children self-monitor. The quantity and quality of training can have an impact on the accuracy of self-monitoring data (e.g., Shapiro, McGonigle, & Ollendick, 1980). When children are told that someone else will also be monitoring and recording their behavior (e.g., checking their work or collecting interobserver agree-

reyn2-2.qxd

6/6/2003

9:00 AM

Page 41

2. Procedural Issues

ment data), they may be more likely to selfrecord accurately (Santogrossi, 1974). Reinforcement for accurate self-monitoring also enhances accuracy (Fixsen, Phillips, & Wolf, 1972; Lloyd & Hilliard, 1989). The valence of the behavior being monitored may also enhance self-monitoring accuracy. Researchers have shown that children may be more likely to accurately observe and record their own appropriate, as opposed to inappropriate, behaviors (Nelson, Hay, Devany, & Koslow-Green, 1980; Nelson, Hay, Hay, & Carsten, 1977). The process of observing and recording one’s own behavior often causes reactivity. Fortunately, the direction of the changes bought about via self-monitoring are generally in the desired direction. Thus, when children are trained to observe and record inappropriate behaviors, those behaviors tend to decrease. When they are trained to observe and record their appropriate behaviors, those behaviors tend to increase. A variety of other variables can affect the degree of reactivity that occurs when children observe and record their own behavior, including (1) training in self-monitoring, (2) obtrusiveness of self-recording devices, (3) timing of self-monitoring, and (4) accuracy of self-recording (Glynn & Thomas, 1974; Nelson et al., 1980). The reactivity caused by self-monitoring tends to make this a popular intervention. Not only is it efficient but self-monitoring is seen as moving children away from external control and encouraging students to become more actively involved in programs designed to maintain or alter their own behaviors (Kern, Marder, Boyajian, Elliot, & McElhattan, 1997). However, as with all forms of reactivity, it is difficult to predict the degree of behavior change that occurs in reaction to self-monitoring. For example, some studies have shown that accurate selfrecording results in higher levels of reactivity (Shapiro, 1984). Other studies have shown that accurate recording is not necessary for reactivity to occur. In fact, some students who where trained to self-record but failed to ever record data showed changes in their behavior (Hayes & Cavoir, 1977). Because the presence or absence of reactivity cannot be reliably predicted or measured, assessment data collected via self-monitoring must always be interpreted

41

with an understanding that these data may be affected by reactivity (Shapiro & Skinner, 1990).

Direct Observation in Analogue Settings For a variety of reasons it is sometimes difficult to assess specific behaviors in natural environments. For example, some specific social skills or social behaviors occur at low rates or unpredictable times (e.g., the opportunity to help or congratulate a peer or the opportunity to avoid a confrontation with an aggressive student). Escape- or avoidance-motivated behaviors can be difficult to observe in natural settings because children rarely come into even remote contact with the feared stimuli. Sometimes the setting in which behaviors of interest occur make it difficult to collect direct observation data. For example, it may be difficult to observe child–parent interactions in home environments. In these instances, analogue assessment procedure may prove useful (Hintze, Stoner, & Bull, 2000). Behavioral avoidance tests (BATs) have often been used to measure responses to feared or anxiety-producing stimuli (e.g., Van Hasselt, Hersen, Bellack, Rosenblum, & Lamparski, 1979). In some instances, children can be bought into proximity or direct contact with the specific stimuli and observers can record direct observation data (e.g., how close the child came to the dog and how long the child remained in the room with the dog). In addition, children can report their level of fear or anxiety while in these situations (Bellack, Kay, & Murrill, 1989). Sometimes, children are prompted to perform gradually more and more fear-provoking responses and observers can record how far the child progressed through the hierarchy (e.g., got within three feet of the dog, got within one foot of the dog, and petted the dog). Observers can also record the number or levels of prompts needed for the child to complete each step. For example, prompts could be less intrusive (e.g., a verbal prompt instructing the child to pet the dog), moderately intrusive (e.g., modeling petting the dog), or highly intrusive (e.g., use a hand-over-hand procedure to physically guide the child’s hand over the dog’s coat). Because it is difficult to unobtrusively col-

reyn2-2.qxd

42

6/6/2003

9:00 AM

Page 42

I. GENERAL ISSUES

lect naturalistic observation data on parent–child interactions, analogue conditions are often useful for assessing these interactions. Forehand and McMahon (1981) developed an analogue procedure where children and parents took turns playing a game where each made up their own rules. During the parent game session, children received a series of commands and child (e.g., compliance) and parents responses were observed and recorded. Barkley (1997) developed similar direct observation procedures to collect data on child–parent interactions within an analogue setting (e.g., clinic). However, Barkley’s procedure did not use a game format. Rather, parents were merely instructed to provide a series of scripted commands. With some role-play measures children are provided descriptions of scenarios and are asked to imagine themselves in these situations and to respond as they would if they were actually in the described situation. Observers then use direct observation procedures to record responses of the children. One of the earlier role play measures was the Behavioral Assertiveness Test for Children (BAT-C) developed by Bornstein, Bellack, and Hersen (1997). With this measure, as children responded to described situations, observers recorded duration of eye contact, intensity of speech, requests for new behavior, and overall assertiveness. Elliott and Gresham (1991) developed social skills assessment scenarios for cooperation, assertion, responsibility, empathy, and selfcontrol. After the situation is described, students provide information about the scenario verbally (e.g., define skill being prompted) and then are asked to demonstrate the skill (e.g., asking for help). A variety of other analogue assessment procedures have been developed (Camp & Bash, 1981; Goldstein, 1999; Spivak, Platt, & Shure, 1976). With some procedures children make overt responses to contrived stimuli. In other situations, students make overt responses to described situations or scenarios. With other analogue assessment procedures contrived situations may be presented and students provide verbal or written descriptions on how they may or should have responded (e.g., Goldstein, 1999). Finally, in some instances both the situation and the response are described (e.g., Spivak

et al., 1976).

Experimental Functional Analysis Procedures Experimental functional analysis procedures also employ analogue assessment conditions as well as direct observation and empirical recording of target behaviors. These procedures differ from previously described analogue assessment procedures in that they are not designed to identify or confirm problems. Rather, single-subject experimental design methodology is used to attempt to determine the function of problem behaviors after they have been identified. Experimental functional analysis procedures are based on operant behavioral psychology. Under this model, behaviors are maintained through positive or negative reinforcement. Furthermore, these positive and negative reinforcers can be idiosyncratic. Thus, two children may present similar behavior problems. These two children may engage in topographically similar behavior (e.g., both child repeat what others say), at similar rates, during similar general environmental conditions (e.g., during school only). However, one child may engage in the behavior in order to receive attention (positive reinforcement) and the other may engage in the behavior to avoid working on a specific task (negative reinforcement or escape/avoidance behavior). By determining the function of the behavior, experimental functional analysis procedures not only suggest classes of interventions that may be effective but may also prevent one from developing and implementing interventions that may exacerbate behavior problems. For example, ignoring a child who is tantrumming is likely to strengthen the behavior if its function is to escape or avoid attention. With experimental functional analysis procedures, students are placed in analogue environments or conditions for brief periods. These conditions are designed to simulate a child’s natural environment. However, these conditions are much more tightly controlled and only one variable at a time is intentionally altered. These experimental procedures allow for comparisons of target behavior levels across conditions in order to determine the variables that may be maintaining target behaviors in the child’s natur-

reyn2-2.qxd

6/6/2003

9:00 AM

Page 43

2. Procedural Issues

al environment. Typically, children are exposed to conditions that test positive and negative reinforcement hypotheses. In addition, children may be exposed to a control condition. Under the positive reinforcement conditions, the child is placed in an environment and allowed to engage in preferred behavior. Reinforcement is delivered following the occurrence of the target behavior. Typically reinforcers tested include attention (e.g., parent or teacher approach and interact with the child only after the child engages in target behaviors) and access to tangibles (e.g., students are given preferred toys for a brief period only after engaging in target behaviors). After a brief period, reinforcers are removed (e.g., parent takes preferred item or teacher turns away from child) and the condition continues. Under the negative reinforcement condition, the child is given a task or demand and that task or demand is removed only when the child engages in the target behavior. After a specified amount of time (usually a brief period, e.g., 15–30 seconds) the child is once again presented with the task or demand. Often a control condition is implemented during which students are given access to preferred items or activities and attention, but delivery or removal of these items, activities, and attention is never delivered contingent upon the child’s target behavior. This condition is designed to determine if performing the target behavior is reinforcing in and of itself (e.g., self-stimulation or self-reinforcement). Typically, children are exposed to the various conditions during brief sessions (e.g., 10–20 minutes) where only one condition is tested. After a brief break, other conditions follow. Session times are equivalent and conditions are presented in random or counterbalanced order. Although frequency counts are often used, observers could use any of the empirical data collection procedures to record levels of target behaviors for each session. The number of exposures to conditions required may vary (see Cooper, Wacker, Sasso, Reimers, & Donn, 1990, for examples of brief experimental functional analysis procedures and Iwata, Vollmer, & Zarcone, 1990, for examples of longer procedures). However, the goal is to collect data until

43

clear levels and trends occur within and across conditions. When this occurs, comparing target behavior levels across conditions can indicate the function or functions of the target behavior. Once the function of a behavior is known, treatments can be designed and implemented within the child’s natural environment that are based on assessed function(s). Experimental functional analysis procedures have been used to identify the function(s) of a variety of behaviors (e.g., aggression, loud vocalizations, noncompliance, self-injurious behavior, wandering, and echolalia) across settings and children (Crawford, Brockel, Schauss, & Miltenberger, 1992; Derby et al., 1992; Iwata et al., 1982; Mace, Webb, Sharkey, Mattson, & Rosen, 1988; Sasso et al., 1992; Townsend, 2000). It is beyond the scope of this chapter to review all the possible variations across experimental conditions, observations and recording procedures, and experimental design elements (see McComas et al., 2000, for a review of these procedures). Furthermore, although functional analysis procedures may indicate some general treatment strategies (e.g., extinction and differential reinforcement of alternative behaviors) because treatment selection, development, and implementation are likely to be influenced by other variables, including available resources, expertise of teacher or parent, and time constraints, it is not feasible to review all possible treatment alternatives suggested by different functional analysis outcomes.

Consideration When Using Analogue Assessment Procedures Analogue assessment procedures allow one to collect direct observation data that may be difficult to collect in natural environments. However, there are several limitations associated with experimental functional analysis and all other analogue assessment procedures. First, exposing children to analogue conditions can cause distress in children (Townsend, 2000). During experimental functional analysis procedures, conditions are established that result in inappropriate behaviors being reinforced. In some situations it may not be appropriate to occasion or reinforce dangerous or disruptive behav-

reyn2-2.qxd

44

6/6/2003

9:00 AM

Page 44

I. GENERAL ISSUES

iors (Iwata et al., 1990; McComas et al., 2000). In addition, experimental functional analysis procedures may not be useful when problem behaviors occur at low rates (e.g., temper tantrums). Perhaps the most serious concern with analogue assessment procedures is that the artificial conditions may not be sufficiently representative of the child’s natural environment. With many analogue assessment procedures, children are placed in novel, unfamiliar situations and often assessment procedures are conducted by someone unfamiliar with the child (e.g., a behavior analyst or school psychologist as opposed to the child’s teacher or parent). Thus, children’s behavior under these conditions may not represent their behavior in their natural environment (Lennox & Miltenberger, 1989; Sasso et al., 1992). Running experimental functional analysis procedures in children’s natural environments may enhance the continuity between analogue and natural conditions (Sasso & Reimers, 1988; Watson, Ray, Sterling, & Logan, 1999). With experimental functional analysis procedures, children are exposed to multiple analogue conditions. It can be difficult to match multiple conditions with natural environments. For example, one could fail to identify preferred stimuli to be used as reinforcers or demand conditions that are appropriate for testing the escape/avoidance function. Even when stimuli from the natural environment are accurately identified, contingent delivery or removal of these stimuli may operate differently on target behaviors across natural and analogue conditions (Iwata et al., 1990). Reinforcement rates and immediacy of reinforcement during experimental functional analysis procedures may not represent those in natural environments. Both these variables can have an impact on behavior. Finally, experimental functional analysis procedures employ highly sensitive measures that may be more susceptible to confounding variables (e.g., hunger, nervousness, habituation, and fatigue). When analogue conditions ask children to imagine themselves in a specific situation, the ability of children to respond to described scenarios is likely to be highly variable both (1) within conditions, across children, and (2) within children, across

conditions. Regardless, the quality of the child’s internal behavior (imagining) cannot be measured and is likely to affect the child’s response. Similarly, when children are asked to describe their responses, they may not provide responses that reflect how they would actually respond. Instead, demand characteristics associated with assessment conditions may cause children to respond how they think they should respond or how they think the person assessing them would want them to respond. These limitations suggest that any hypothesis formed via analogue assessment procedures may need to be confirmed by collecting data on the children’s behavior within their natural environments.

SUMMARY AND CONCLUSIONS The direct nature of the behavioral assessment procedures reviewed is often appealing to those who are uncomfortable with inferring within-subject problems based on overt behaviors and making inferences regarding the cause of those problems. Even though the assessment procedures described and analyzed here are more direct than traditional procedures, these procedures can still yield data that do not accurately reflect a child’s behavior in his or her natural environment. Therefore, in most instances it is useful to confirm data collected via all the procedures described previously. This confirmatory data can be collected using other direct assessment procedures. For example, analogue assessment data can be confirmed by collecting direct observation data in a child’s natural environment, interviewing a teacher, parent, or peer (e.g., Bergan & Kratochwill, 1990) or with checklist and rating scales (e.g., Achenbach, 1991). Data may also be confirmed by using multiple observers (see House, House, & Campbell, 1981, for procedures and formula for confirmed direct observation data). Confirming data before interventions are implemented may enhance the confidence one has in conclusions that have been based on those data. However, the ultimate purpose of collecting data is to provide information that will lead to more effective interventions or education programs. By indicating the strengths and limitations as-

reyn2-2.qxd

6/6/2003

9:00 AM

Page 45

2. Procedural Issues

sociated with direct behavioral assessment procedures, it is hoped that readers will be more likely to develop or choose assessment procedures that will yield data that accurately represent problem behaviors and also allow for the identification of variables that may be maintaining those target behaviors. However, because there are limitations associated with all direct assessment procedures, it is essential that assessment processes do not halt after problems have been identified or interventions have been developed based on hypothesized functions of behavior. Rather, one must continue to assess to determine if the interventions that were developed based on these assessment data were effective in bringing about socially significant changes in the target behavior(s) within the child’s natural environment (Barnett et al., 2000).

REFERENCES Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington: University of Vermont, Department of Psychology. Alessi, G., & Kaye, J. H. (1983). Behavior assessment for school psychologists. Kent, OH: National Association of School Psychologists. Barkley, R. A. (1997). Defiant children: A clinician’s manual for assessment and parent training (2nd ed.). New York: Guilford Press. Barnett, D. W., Daley, E. J., Hampshire, E. M., Hines, N. R., Maples, K. A., Ostron, J. K., & Van Buren, A. E. (1999). Meeting performance-based training demands: Accountability in an intervention based practicum. School Psychology Quarterly, 14, 357–379. Bellack, L., Kay, S. R., & Murrill, L. M. (1989). The Dysphorimeter: An objective analogue for the assessment of depression, anxiety, pair and other dysphoric states. American Journal of Psychotherapy, 43, 260–268. Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation in applied settings. New York: Plenum Press. Bornstein, M. R., Bellack A. S., & Hersen, M. (1977). Social skills training for unassertive children: A multiple-baseline analysis. Journal of Applied Behavior Analysis, 10, 183–195. Bowers, F. E., McGinnis, C., Ervin, R. A., & Friman, P. C. (1999). Merging research and practice: The example of positive peer reporting applied to social rejection. Education and Treatment of Children, 22, 218–226. Camp, B. W., & Bash, M. A. (1981). Think aloud. Champaign, IL: Research Press. Carded-Smith, L. K., & Fowler, S. A. (1984). Positive peer pressure: The effects of peer monitoring on chil-

45

dren’s disruptive behavior. Journal of Applied Behavior Analysis, 17, 213–227. Carr, E. G. (1993). Behavior analysis is not ultimately about behavior. The Behavior Analyst, 16, 47–49. Cashwell, C. S, Skinner, C. H., Dunn, M., & Lewis, J. (1998). Group reward programs: A humanistic approach. Humanistic Education and Development, 37, 47–53. Cole, C. L, Marder, T., & McCann, L. (2000). Selfmonitoring. In E. S. Shapiro & T. R. Kratochwill (Eds.), Conducting school-based assessment of child and adolescent behavior (pp. 121–149). New York: Guilford Press. Cone, J. D. (1978). The behavioral assessment grid (BAG): A conceptual framework and a taxonomy. Behavior Therapy, 9, 882–888. Cooper, L., Wacker, D., Sasso, G., Reimers, T., & Donn, L. (1990). Using parents as therapists to evaluate appropriate behavior of their children: Application to a tertiary diagnostic clinic. Journal of Applied Behavior Analysis, 23, 285–296. Crawford, J., Brockel, B., Schauss, S., & Miltenberger, R. G. (1992). A comparison of methods for the functional assessment of stereotypic behavior. Journal of the Association for Persons with Severe Handicaps, 17, 77–86. Derby, M., Wacker, D., Sasso, G., Steege, M., Northup, J., Cigrand, K., & Asmus, J. (1992). Brief functional assessment techniques to evaluate aberrant behavior in an outpatient setting: A summary of 79 cases. Journal of Applied Behavior Analysis, 25, 713–721. Elliott, S.N., & Gresham, F. M. (1991). Social skills intervention guide: Practical strategies for social skills training. Circle Pines, MN: American Guidance Services. Ervin, R. A., Johnston, E. S., & Friman, P. C. (1998). Positive peer reporting to improve the social interactions of a socially rejected girl. Proven practice: Prevention and remediation solutions for school problems, 1, 17–21. Ervin, R. A., Miller, P. M., & Friman, P. C. (1996) Feed the hungry bee: Using positive peer reports to improve the social interactions and acceptance of a socially rejected girl in a residential placement. Journal of Applied Behavior Analysis, 29, 251–253. Fixsen, D. L., Phillips, E. L., & Wolf, M. M. (1972). Achievement Place: The reliability of self-reporting and peer-tutoring and their effects on behavior. Journal of Applied Behavior Analysis, 5, 19–30. Forehand, R. L., & McMahon, R. J. (1981). Helping the noncompliant child: A clinician’s guide to parent training. New York: Guilford Press. Glynn, E. L., & Thomas, J. D. (1974). Effects of cueing on self-control of classroom behavior. Journal of Applied Behavior Analysis, 7, 299–306. Goldstein, A. P. (1999). The prepare curriculum (rev ed.). Champaign, IL: Research Press. Greenwood, C. R., Dinwiddie, G., Terry, B., Wade, L., Stanely, S. O., Thibadeau, S., & Delquadri, J. C. (1987). Teacher- versus peer-mediated instruction: An eco-behavioral analysis of achievement outcomes. Journal of Applied Behavior Analysis, 17, 521–538.

reyn2-2.qxd

46

6/6/2003

9:00 AM

Page 46

I. GENERAL ISSUES

Gresham, F. M. (1998). Designs for evaluating behavior change: Conceptual principles of single-case methodology. In T. S. Watson & F. M. Gresham (Eds.), Handbook of child behavior therapy (pp. 23–40). New York: Plenum Press. Hartmann, D. P., Roper, B. L., & Bradford, C. C. (1979). Some relationships between behavioral and traditional assessment. Journal of Behavioral Assessment, 1, 3–21. Hayes, S. C., & Cavoir, N. (1977). Multiple tracking and the reactivity of self-monitoring: I. Negative behaviors. Behavior Therapy, 8, 819–831. Hendrickson, J. M., Gable, R. A., Conroy, M. A., Fox, J., & Smith, C. (1999). Behavior problems in schools: Ways to encourage functional behavior assessment (FBA) of discipline-evoking behavior of students with emotional and/or behavioral disorders (EBD). Education and Treatment of Children, 22, 280–290. Henington, C., & Skinner, C. H. (1998). Peer monitoring. In K. Toppings & S. Ely (Eds.), Peer assisted learning (pp. 237–253). Hillsdale, NJ: Erlbaum. Hintze, J. M., & Shapiro, E. S. (1995). Best practices in systematic observation of classroom behavior. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology-III (pp. 651–660). Washington, DC: National Association of School Psychologists. Hintze, J. M., Stoner, G., & Bull, M. H. (2000). Analogue assessment: Emotional/behavior problems. In E. S. Shapiro & T. R. Kratochwill (Eds.) Conducting school-based assessment of child and adolescent behavior (pp. 55–77). New York: Guilford Press. House, A. E., House, B. J., & Campbell, M. B. (1981). Measures of interobserver agreement: Calculation formulas and distribution effects. Journal of Behavioral Assessment, 3, 37–58. Iwata, B. A., Dorsey, M. F., Sifler, K. J., & Richman, G. S. (1982). toward a functional analysis of self-injury. Analysis and Intervention in Developmental Disabilities, 3, 1–20. Iwata, B. A., Vollmer, T. R., & Zarcone, J. R. (1990). The experimental (functional) analysis of behavior disorders: Methodology, applications, and limitations., In A. C. Repp & N. N. Sigh (Eds.), Perspective on the use of nonaversive and aversive interventions for persons with developmental disabilities (pp. 301–330). Sycamore, IL: Sycamore. Johnson, S. M., & Bolstad, O. D. (1973). Methodological issues in naturalistic observation: Some problems and solutions. In L. A. Hamerlynch, L. E. Handy, & E. J. Marsh (Eds.), Behavior change: Methodology, concepts, and practices (pp. 7–68). Champaign, IL: Research Press. Jones, K. M., Young, M. M., & Friman, P. C. (2000). Increasing peer praise of socially rejected delinquent youth: Effects on cooperation and acceptance. School Psychology Quarterly, 15, 30–39. Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABC’s of reliability. Journal of Applied Behavior Analysis, 10, 141–150. Kern, L., Marder, T. J., Boyajian, A. E., Elliot, C. M., & McElhattan, D. (1997). Augmenting the independence of self-management procedures by teaching self-initiation across settings and activities. School

Psychology Quarterly, 12, 23–32. Lalli, J. S., Browder, D. M., Mace, F. C., & Brown, K. (1993). Teacher use of descriptive analysis data to implement interventions to decrease students’ maladaptive behavior. Journal of Applied Behavior Analysis, 10, 141–150. Lennox, O. I., & Miltenberger, R. G. (1989). Conducting a function assessment of problem behavior in applied settings. Journal of the Association for Persons with Severe Handicaps, 14, 304–311. Lentz, F. E. (1982). An empirical examination of the utility of partial interval and momentary time sampling as a measure of behavior. Unpublished doctoral dissertation, The University of Tennessee, Knoxville. Lloyd, M. E., & Hilliard, A. M. (1989). Accuracy of self-recording as a function of repeated experience with different self-control contingencies. Child and Family Behavior Therapy, 11, 1–14. Mace, F. C., Webb, M. E., Sharkey, R. W., Mattson, D. M., & Rosen, H. S. (1988). Functional analysis and treatment of bizarre speech. Journal of Behavior Therapy and Experimental Psychology, 19, 714–721. McComas, J. J., Hoch, H., & Mace, F. C. (2000). Functional analysis. In E. S. Shapiro & T. R. Kratochwill (Eds.), Conducting school-based assessment of child and adolescent behavior (pp. 78–101). New York: Guilford Press. McCurdy, B. L., & Shapiro, E. S. (1992). A comparison of teacher-, peer-, and self-monitoring with curriculum-based measurement in reading among students with learning disabilities. Journal of Special Education, 26, 162–180. McLaughlin, T. F., & Skinner, C. H. (1996). Improving academic performance through self-management: cover, copy, and compare. Intervention in School and Clinic, 32, 113–118. Myerson, J., & Hale, S. (1984). Practical implications of the matching law. Journal of Applied Behavior Analysis, 17, 367–380. Nelson, R. O. (1977). Methodological issues in assessment via self-monitoring. In J. D. Cone & R. P. Hawkins (Eds.), Behavioral assessment: New directions in clinical psychology (pp. 217–240). New York: Brunner/Mazel. Nelson, R. O., Hay, L. R., Devany, J., & KoslowGreen, L. (1980). The reactivity of children’s selfmonitoring: Three experiments. Child Behavior Therapy, 2, 1–24. Nelson, R. O., Hay, L. R., Hay, W. M., & Carsten, C. B. (1977). The reactivity and accuracy of teachers’ self-monitoring of positive and negative classroom verbalizations. Behavior Therapy, 8, 972–985. Nelson, R. O., & Hayes, S. C. (1979). Some current dimensions of behavioral assessment. Behavioral Assessment, 1, 1–16. Nelson, R. O., & Hayes, S. C. (Eds.). (1986). Conceptual foundations of behavioral assessment. New York: Guilford Press. Nelson, J. R., Roberts, M. L., & Rutherford, R. B., Mathur, S. R., & Aaroe, L. A. (1999). A statewide survey of special education administrators and school psychologists regarding functional behavioral

reyn2-2.qxd

6/6/2003

9:00 AM

Page 47

2. Procedural Issues assessment. Education and Treatment of Children, 22, 267–279. O’Neill, R. E., Horner, R. H., Albin, R. W., Sprague, J. R., Storey, K., & Newton, J. S. (1997). Functional assessment of problem behavior: A practical assessment guide (2nd ed.). Pacific Grove, CA: Brooks/Cole. Powell, J., Martindale, B., Kulp, S., Martindale, A., & Bauman, R. (1977). Taking a closer look: time sampling and measurement error. Journal of Applied Behavior Analysis, 10, 325–332. Reschly, D. J., & Ysseldyke, J. E. (1995). School psychology paradigm shifts. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology-III (pp. 17–31). Washington, DC: National Association of School Psychologists. Robinson, S. L. (1998). Effects of positive statements made by peers on peer interactions and social status of children in a residential treatment setting. Unpublished doctoral dissertation, Mississippi State University, Starkville. Santogrossi, D. A. (1974, October). Self-reinforcement and external monitoring of performance on an academic task. Paper presented at the fifth annual conference on Applied Behavior Analysis in Education, Kansas City, KS. Sasso, G. M., & Reimers, T. M. (1988). Assessing the functional properties of behavior: Implications and applications for the classroom. Focus on Autistic Behavior, 3, 1–15. Sasso, G. M., Reimers, T. M., Cooper, L. J., Wacker, D., Berg, W., Steege, M., Kelly, L., & Allaire, A. (1992). Use of descriptive and experimental analyses to identify the functional properties of aberrant behavior in school settings. Journal of Applied Behavior Analysis, 25, 809–821. Saudargus, R. A. (1992). State-Event Classroom Observation system (SECOS). Knoxville: University of Tennessee, Department of Psychology. Shapiro, E. S. (1984). Self-monitoring. In T. H. Ollendick & M. Hersen (Eds.), Child behavior assessment: Principals and procedures (pp. 148–165). New York: Pergamon Press. Shapiro, E. S. (1987). Behavioral assessment in school psychology. Hillsdale, NJ: Erlbaum. Shapiro, E. S. (1996). Academic skills problems: Direct assessment and intervention (2nd ed). New York: Guilford Press. Shapiro, E. S., & Browder, D. M. (1990). Behavioral assessment: Applications for persons with mental retardation. In J. Matson (Ed.), Handbook of behavior modification with persons with mental retardation (2nd. ed., pp. 93–122). New York: Plenum Press. Shapiro, E. S., & Kratochwill, T. R. (2000). Introduction: Conducting a multidimensional behavioral assessment. In E. S. Shapiro & T. R. Kratochwill

47

(Eds.), Conducting school-based assessments of child and adolescent behavior (pp.1–20). New York: Guilford Press. Shapiro, E. S., McGonigle, J. J., & Ollendick, T. H. (1980). An analysis of self-assessment and self-reinforcement in a self-managed token economy with mentally retarded children. Applied Research in Mental Retardation, 1, 227–240. Shapiro, E. S., & Skinner, C. H. (1990). Best practices in observational/ecological assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology-II (pp. 17–31). Washington, DC: National Association of School Psychologists. Skinner, C. H., Cashwell, T. H., & Skinner, A. L. (2000). Increasing tootling: The effects of a peer monitored group contingency program on students’ reports of peers’ prosocial behaviors. Psychology in the Schools, 37, 263–270. Skinner, C. H., Dittmer, K. I., & Howell, L. A. (2000). Direct observation in school settings: Theoretical issues. In E. S. Shapiro & T. R. Kratochwill (Eds.), Behavioral assessment in schools: Theory research and practice (2nd ed., pp. 19–45). New York: Guilford Press. Skinner, C. H., Rhymer, K. N., & McDaniel, E. C. (2000). Naturalistic direct observation in educational settings. In E. S. Shapiro & T. R. Kratochwill (Eds.), Conducting school-based assessments of child and adolescent behavior (pp. 21–54). New York: Guilford Press. Skinner, C. H., Shapiro, E. S., Turco, T. L., Cole, C. L., & Brown, D. K. (1992). A comparison of self- and peer delivered immediate corrective feedback on multiplication performance. Journal of School Psychology, 30, 101–116. Skinner, C. H., & Smith, E. S. (1992). Issues surrounding the use of self-management interventions for increasing academic performance. School Psychology Review, 21, 202–210. Spivak, G., Platt, J., & Shure, M. B. (1976). The problem-solving approach to adjustment. San Francisco: Jossey-Bass. Stern, G. W., Fowler, S. A., & Kohler, F. W. (1988). A comparison of two intervention roles: Peer monitor and point earner. Journal of Applied Behavior Analysis, 21, 103–109. Telzrow, C. F. (1999). IDEA amendments of 1997: Promise or pitfall for special education reform? Journal of School Psychology, 37, 7–28. Townsend, B. J. K. (2000). A functional analysis of the echolalic behavior of three children with autism in a residential school setting. Unpublished doctoral dissertation, Mississippi State University, Starkville. Van Hasselt, V. B., Hersen, M., Bellack, A. S., Rosenblum, N. D., & Lamparski, D. (1979). Tripartite assessment of the effects of systematic desensitization in a multi-phobic child: An experimental analysis. Journal of Behavior Therapy and Experimental Psychiatry, 10, 51–55. Watson, T. S., Ray, K. R., Sterling, H. E., & Logan, P. (1999). Teacher-implemented functional analysis and treatment: A method of linking assessment to intervention. School Psychology Review, 28, 292–302.

reyn2-2.qxd

6/6/2003

9:00 AM

Page 48

This page intentionally left blank

reyn2-3.qxd

6/6/2003

9:02 AM

Page 49

PART II

PROJECTIVE METHODS

reyn2-3.qxd

6/6/2003

9:02 AM

Page 50

This page intentionally left blank

reyn2-3.qxd

6/6/2003

9:02 AM

Page 51

3 The Projective Hypothesis and the Development of Projective Techniques for Children

LOUIS A. CHANDLER

It should be stated at the outset that this chapter is concerned with the use of projective techniques with children. A number of unique characteristics make children different from adults as experimental subjects and as participants in the projective experience. Too often, researchers and writers have ignored this important variable when generalizing about the use of projective techniques. It is within that context that the statements made here, both critical and supportive, must be evaluated.

to find application in the emerging practice of psychology. These early efforts developed the methods and techniques that would be employed in the study of individual differences. From these laboratory tasks, the line extended through such pioneering psychometric instruments as the Woodworth Personal Data Sheet of 1919 and the Bernreuter Personality Inventory of 1931. These instruments, based on face or content validity, contained items thought to represent the traits or dimensions under study. The items were arranged in a test format, allowing a subject’s response to be scored and quantified. Quantification allowed one subject’s scores to be compared to another’s. It was thus possible to examine similarities and differences in test performance. It was also possible to compare the individual’s score against some group average or norm. But while psychometric techniques continued to develop along one track, a radically new approach was being proposed by the 1930s. This involved methods that had their theoretical foundations in psychoanalysis— at that time, a growing influence on psychiatry and psychology. These new methods were exemplified by the technique intro-

HISTORICAL OVERVIEW In order to understand the current status of projective techniques, a brief historical overview is helpful. The history of personality assessment prior to the 1930s followed a fairly direct evolutionary path (see Figure 3.1). Its roots can be traced to the experiments of Wundt, Galton, and Cattel around the turn of the century. Indeed, Sattler (1982) has called the period from 1890 to 1905 the “laboratory period” in mental measurement, since it was during that time that tests were devised for measuring individual responses—tests that were gradually 51

reyn2-3.qxd

52

6/6/2003

9:02 AM

Page 52

II. PROJECTIVE METHODS

FIGURE 3.1. The historical development of personality assessment methods.

duced by Hermann Rorschach as a means of personality appraisal. In Rorschach’s (1921) technique, a subject is presented with a set of 10 inkblots and asked to provide associations and interpretations. No attempt is made to structure or limit the response; rather, the subject is encouraged to generate his or her own unique response pattern. L. K. Frank (1939) is generally credited with coining the term “projective technique” for those methods that employ a relatively ambiguous stimulus to which the subject is asked to give meaning. The Rorschach technique, with its emphasis on the idiographic features of each individual’s response pattern, stood in contrast to the psychometric focus of the comparative or nomothetic aspects. This was to become a fundamental point of difference between the two approaches. Rorschach’s technique was introduced in America by Samuel Beck in the 1930s. Also during that decade, Henry Murray and his colleague Christiana Morgan, while engaged in the study of personality, conceived of the idea of using stories generated by subjects to a standard set of relatively ambiguous pictures. The resulting technique, the Thematic Apperception Test (TAT; Murray, 1943) has subsequently found use in clinical practice.

At about the same time, there arose a growing interest in the use of creative drawings in psychological assessment. The clinical use of drawings can be traced back to Florence Goodenough (1926), who first proposed assessing children’s intellectual maturity from their drawings of a person. Goodenough noted variations in the drawings that could not be accounted for by age, and proposed that factors such as anxiety might influence drawings. But it was Karen Machover (1949) who was to emphasize the psychodynamic aspects of the drawing of a person in terms of the child’s self-image. She suggested that a qualitative assessment might be done using drawings that could be examined for emotional indicators. These major projective techniques were to spawn a number of others during the 1930s and 1940s. Such approaches to the study of personality, focusing as they did on individual responses, made few attempts to compare group responses. Unlike psychometric tests, these techniques did not easily lend themselves to quantification and statistical treatment. The different approaches to the study of personality have led to misunderstanding and controversy. As a result, within the last few decades projective techniques have come in for considerable criticism from

reyn2-3.qxd

6/6/2003

9:02 AM

Page 53

3. The Projective Hypothesis and Development of Projective Techniques

those who advocate psychometric, or, more recently, behavioral approaches to assessment. One result has been a recent decline in the number of published articles dealing with projective techniques (O’Leary & Johnson, 1979; Polyson, Norris, & Ott, 1985). Yet, while research interest appears to have waned, projective techniques continue to be popular among practicing psychologists (Lubin, Larsen, & Matarazzo, 1984; Piotrowski, 1984), particularly among those who conduct personality assessments with children (Goh & Fuller, 1983). It is worth examining these techniques more closely to see what it is that makes them so compelling in that context.

THE PROJECTIVE HYPOTHESIS AND ITS USE WITH CHILDREN The Projective Hypothesis Two fundamental questions need to be addressed in a discussion of projective techniques: What is the nature of the mechanism of projection, and how does it work? The projective hypothesis is based on the tendency of human beings to view and interpret their world in terms of their own experience. This anthropomorphic quality influences all our perceptions to a greater or lesser degree, and we can never totally escape it. For the same reason, every human production, from the most mundane to the most creative, reflects some aspect of the self. It is this tendency to interpret our world and to express ourselves in our productions that led Murray (1943) to explain the ability of TAT stories to reveal the personality as “the tendency of people to interpret an ambiguous human situation in conformity with their past experiences and present wants, and the tendency of those who write stories to do likewise” (p. 1). Although there is some agreement on the fundamental nature of projection, there are differences of opinion as to how the mechanism operates. Part of the problem lies in the fact that the term “projection” has two meanings. “Projection,” in common usage, means to cast forward. In this sense, projection implies a direct extension of psychological characteristics onto the outer world. But “projection” also has a specific meaning within psychoanalytic theory. Freud

53

(1936) used the term to refer to the process that occurs when the ego, faced with unacceptable wishes or ideas, thrusts them out onto the external world as a means of defense. In projection the individual attributes his or her own thoughts and actions to someone else. Thus, if one’s own faults or feelings are unacceptable to the ego, they may be seen as belonging to someone else; in the process, the material may become distorted or remain partially repressed. From such a perspective, projective material would not be seen as direct representations of aspects of the personality—certainly not with the sort of one-to-one correspondence that the first meaning of “projection” implies. Hammer (1958) reflects this point of view when he writes that “distortions enter into the process of projection to the extent to which the projection has a defensive function . . . qualities are ascribed to the object the presence of which the subject denies in himself” (pp. 53–54). This dual function of projection—as a psychological tendency to extend inner attributes to the outer world, and as a defense—creates problems in interpreting projective material. For example, if a child’s story contains frequent references to violence, are we to interpret that as an actual reflection of an aggressive personality, or perhaps of a personality wherein aggression is latent and unlikely to find behavioral expression? After several decades of research with projective techniques, the crucial issue of the correspondence between fantasies and behavior remains poorly understood (Klinger, 1971; McNeil, 1976; Megargee & Cook, 1967; O’Leary & Johnson, 1979).

The Use of Projective Techniques with Children In spite of these interpretive difficulties, projective techniques continue to be widely used, especially with children. Some of the reasons for this popularity are found both in the nature of childhood and in the nature of projective techniques. There are two sides to the coin. First, we must examine the developmental features of children that recommend them as ideal candidates for projective techniques; then we must examine the features of projective techniques that make them particularly useful with children.

reyn2-3.qxd

54

6/6/2003

9:02 AM

Page 54

II. PROJECTIVE METHODS

As they struggle with the developmental task of gradually constructing a world view in consonance with adult reality, children’s thinking processes undergo considerable change. Piaget has reported in great detail the changes in thinking that occur in the developing child. One such process involves externalization. For Piaget, “externalization” is a process by which we attribute to things in the external world the products of our own mental activity. According to Elkind (1976), this attribution is immediate and unconscious. Elkind makes the point that this process differs from the psychoanalytic notion of “projection” in that it is a normal process common to everyone (thus, it is closer to “projection” as commonly used). While externalization is a relevant facet of a child’s thinking in terms of the projective experience, it is especially useful when combined with another childlike quality— egocentrism. “Egocentrism” refers to the young child’s inability to put himself or herself in another’s position and to adopt the other’s point of view. Unlike the egocentric adult who can take another’s point of view but chooses not to, the child is incapable of the perceptual shift. Kessler (1966) has pointed to the similarity between childlike egocentrism and the projection of the paranoid patient. Like that of the paranoid patient, the child’s narcissism is overwhelming. And like the paranoid individual, “[the child] interprets the universe as revolving around him, either to hurt him or help him, and he personifies all happenings in terms of his own projected wishes and fears” (p. 33). These two tendencies—toward externalization (with its immediate and unconscious aspects) and egocentrism (with its personification)— mean that children, who tend to rely heavily on both, are naturally good candidates for projective techniques. The systematic study of childhood has also pointed to other relevant factors. We know, for example, that langauge acquisition is a gradual process, and that many crucial events occur in the life of the child before full mastery of speech and language. Even when they have acquired some language, children remain unable to express their feelings directly in words, and often resort to the more primitive means of motoric

expression. We find a similar phenomenon when, as adults, we are confronted with one of those rare experiences where words fail us. In attempting to describe such situations, we find ourselves groping for analogies or metaphors to express what seems beyond logical narrative (Kenniston, 1965). Even when speech and language are fairly well developed, verbal expression does not remain the sole or even the preferred way in which the child communicates. The difficulties found in conducting interviews with children illustrate the complexity of the problem. Ginsburg and Opper (1979) detail how, in the course of the interview, the child is asked to perform several tasks. He or she must interpret the examiner’s question, make a special effort to comprehend the crucial aspects of the question, and express his or her response in words. It is not surprising that children, especially young children, may prefer to express themselves by projective techniques, which employ visual stimuli and/ or motor or visual–motor response modes, as opposed to techniques that rely more on language. Along with those language and associated cognitive limitations that make children poor subjects for interviews, a number of conscious factors intervene to influence their responses. Social desirability, motivation, and degree of cooperation all may influence the interviewing process. Unlike the self-report measures of personality assessment used with adults, projective techniques do not depend on higher levels of language development and conceptual understanding, and they require only minimal cooperation. The latter point may make them especially attractive, since children seldom find themselves being assessed as a result of a selfreferral. One further aspect of children’s suitability for projective techniques needs to be mentioned. This has to do with the observation that children are in transition from prelogical to logical thought, from a magical to a realistic world view. Freud (1911/1959) explicated the developmental shift in terms of primary-process thinking versus secondary-process thinking. Primaryprocess thinking operates in accordance with the pleasure principle; secondaryprocess thinking in accordance with the reality principle.

reyn2-3.qxd

6/6/2003

9:02 AM

Page 55

3. The Projective Hypothesis and Development of Projective Techniques

Primary process thinking can be seen in dreams, where anything is possible, often in defiance of the laws of logic. It is also present to some extent in symptom formations of neurotic patients, and, of course, it dominates the thought processes of the young child. Primary-process thinking operates at the level of symbols. It is primitive and preverbal. Gradually, more of a child’s thinking processes come to be dominated by the reality principle, although lapses and regression are common, especially in times of emotional distress. Using magical thinking in the context of the adult real world leads to inevitable misunderstandings, frustrations, and conflict, often to the detriment of the child. This difference in thinking can also play a key role in children’s emotional problems. The difference may be observed, for example, when a child, in appraising some event or situation, perceives the threat of harm or loss. If this perception results from magical thinking, it may well happen that an observing adult will see the child simply as being unrealistic (this is literally true, from the adult’s point of view). The dark at the top of the stairs provides a case in point: Adult assurances, and even reality testing (turning on the light), may not be enough to eliminate the child’s fears. To understand a child’s emotional status, a method of assessment is needed that allows us access to this inner world of childhood. Projective techniques hold the promise of being such a method.

VALIDITY Several writers have stated that projective techniques are not tests, and should not therefore be subjected to the constraints normally applied to psychometric devices (Frank, 1939; Holt, 1978; Schwartz & Lazar, 1979). Yet fundamental validity and reliability issues cannot be avoided. Certain basic questions may legitimately be asked of any assessment method: Does it do what it purports to do, and does it do so with consistency? The validity of projective techniques rests, in the first instance, on the projective hypothesis. Thus initial validity is based on the assumption that one’s productions re-

55

flect qualities of the personality, although it may be argued that these techniques, specifically designed to elicit externalization, are at best only partially successful (Anastasi, 1982). The relative proportion of externalization manifested may vary among individuals, and even within the same individual if he or she is administered the same technique on two different occasions (Lindzay & Herman, 1955). This calls into question whether projective techniques are capable of accurately identifying relatively stable personality traits, or are more sensitive to situational variables. If the purpose of using projective techniques is to obtain some global picture of the individual’s personality (Anastasi, 1982), then the issue of whether they can identify enduring personality traits becomes a critical one. If. on the other hand, the purpose is to estimate the individual’s current motivational and emotional status (Obrzut & Cummings, 1983), perhaps in terms of certain specified factors (Chandler, 1985), then the issue becomes less critical. These and other problems associated with projective techniques beg the question as to the purpose of the assessment. If the validity question is “Does this method do what we want it to do?,” then we must be clear as to what it is we wish these techniques to do in the context of the personality assessment. The question as to what constitutes an appropriate outcome—in this case, what criteria the technique’s performance can be measured against—is ultimately a question of criterion-related validity. Beginning in the 1950s, a number of validation studies were performed on projective techniques (e.g., Little & Shneidman, 1959). Most of these studies were concerned with some aspect of criterion-related validity—that is, discriminative and/or predictive validity. Typically, such studies tested how well the techniques could discriminate between groups or predict membership in various psychiatric diagnostic categories. The results were generally negative; projective techniques were found to do poorly in discriminating between groups (GittlemanKlein, 1978) and to do little better than chance in predicting diagnostic group membership (Chambers & Hamlin, 1957). This poor performance led some critics to conclude that the techniques were not valid.

reyn2-3.qxd

6/6/2003

56

9:02 AM

Page 56

II. PROJECTIVE METHODS

However, another interpretation is that these experiments were ill conceived because the hypothesis that they were designed to test was inappropriate. It has been argued that the purpose of projective techniques is not to determine psychiatric group membership, but to increase understanding (Exner, 1983; Holt, 1970; Knoff, 1983) or to provide information useful in developing interventions (Batsche & Peterson, 1983). Wiggins (1980) finds the distinction proposed by Prelinger and Zimet (1964) to be an useful one. Those writers distinguish between assessment for” and “assessment of.” The former implies a predictive aspect of assessment; the latter implies the “understanding” that characterizes psychoanalytic investigations. Wiggins points out that the validation of each approach is established by recourse to different sources of evidence. For predicting diagnostic categories, such methods as those measuring the number of correct decisions are appropriate. For psychoanalytic investigations, interpretations are evaluated within the context of internal consistency among multiple sources of evidence. Until there is some agreement as to what constitutes an appropriate outcome when projective techniques are used in clinical assessment, validity problems will remain, and projective techniques will continue to be poorly understood.

RELIABILITY Although validity continues to be a concern with projective techniques, an issue of equal if not greater importance is reliability, since an assessment system that is unreliable must be invalid (McDermott, 1980; Spitzer & Fleis, 1974). “Reliability” is a psychometric concept that can be approached in several ways. There are unique considerations associated with various types of reliability when applied to projective techniques. One index of reliability is agreement between two administrations of the same test with the same subject. But test–retest reliability poses problems with projective techniques, since if the interval between administrations is short, recall may affect the results; if the interval is long, the changes found may reflect

actual changes that have occurred in the personality over time (Lanyon & Goodstein, 1982). Then, too, if projective techniques assess current emotional and motivational variables, test–retest reliability may be an inappropriate method to use (Obrzut & Cummings, 1983). Still another index of consistency is splithalf reliability, normally obtained by computing the correlation between scores on comparable halves of a test. But here, too, there are problems when applied to projective techniques. It can be argued that the cards used in the Rorschach technique, for example, are not comparable, so that this measure of internal consistency is not applicable (Anastasi, 1982). Although splithalf reliability may be of limited value with projective techniques, Wagner and his colleagues have recently proposed a new technique for optimizing internal reliability estimates of the Rorschach that takes into account split-half distributional anomalies. They have also suggested that their method may be applied to other projective techniques (Wagner, Alexander, Roos, & Adair, 1986). One commonly used test of reliability for projective techniques is interjudge agreement, although there are difficulties with this as well. For an adequate test of interjudge agreement, some standardized scoring procedure is necessary. Over the years there have been a number of attempts to develop scoring procedures for projective techniques; Hughes (1983), in her review of the literature, identified nearly 40 methods that had been proposed for analyzing and interpreting thematic apperception techniques. These efforts have met with varying degrees of success. (Because there are different issues associated with the various techniques, these are discussed in the section below devoted to the major techniques.) In general, the effort has been to develop relatively objective, quantitative scoring systems that can supplant or supplement the traditional methods of analysis and interpretation. Here, too, there are obstacles that make the task of developing a scoring system difficult. A lack of standardized procedures for administering some techniques, along with variations between different examiners, may influence the results. The stimulus material may not be standardized,

reyn2-3.qxd

6/6/2003

9:02 AM

Page 57

3. The Projective Hypothesis and Development of Projective Techniques

as in the case of the TAT, where examiners may select which cards to present. The varying lengths of response, and the representativeness of the scoring constructs throughout the stimulus material, cause further difficulties (Lanyon & Goodstein, 1982). Finally, there remains some degree of subjective judgment—if not in the scoring, then in the interpretation of the response. Because of these difficulties and the problem of controlling for the relevant variables in reliability studies, it is not surprising to find a wide range reported among the reliability data on projective techniques. For example, Magnusson (1960) has reported interjudge reliability on Children’s Apperception Test (CAT) protocols as ranging from .17 to .72.

MAJOR PROJECTIVE TECHNIQUES USED WITH CHILDREN Although a considerable number of projective techniques have been developed since the 1930s, most have been variations on a few major types. Of these, three major techniques have been widely used with children: creative drawings, thematic apperception techniques, and the Rorschach technique (Goh & Fuller, 1983; Prout, 1983).

The Projective Aspects of Drawings Creative drawings, like other forms of artistic production, have long been seen as expressing some aspect of the artist’s personality. Yet the systematic use of drawings in clinical assessment is a modern development. As pointed out above, drawings were initially proposed as a means of estimating the level of children’s mental development (Goodenough, 1926; Harris, 1963), but from the beginning writers have observed that drawings are influenced by emotional factors (Buck, 1948; Hammer, 1958; Machover, 1949). One of the first to call attention to the projective aspects of drawings was Karen Machover. Machover’s task, like Goodenough’s, was to ask the child to draw a person (the Draw-A-Person technique). A number of other drawing techniques have since been developed, including the House–Tree–Person (H-T-P; Buck, 1948), Human Figure Draw-

57

ing (Machover, 1949), Draw-A-Family (Hulse, 1951), and Kinetic Family Drawing (K-F-D; Burns & Kaufman, 1970) techniques. In analyzing drawings, the formal aspects as well as the content are considered. The formal aspects of drawings include such elements as the size and placement of items; the quality of lines and the amount of pressure used; erasures, distortions, and omissions; and undue emphasis on or unusual treatment of details. In general, the approach is to interpret content elements of the drawings by referring to associated personality features. Such associations are derived from clinical experience and/or research studies that have hypothesized or demonstrated a relationship between the drawings’ elements and various aspects of personality. For example, Buck (1971) suggests that high placement of a figure on the page may be interpreted as a high level of aspiration. Hammer (1958) maintains that a fence drawn around a house suggests defensiveness on the part of the individual. Various checklists of such associated features have been compiled to aid in interpreting drawings (Jolles, 1971; Ogden, 1977; Urban, 1963). There is general agreement that while drawings may be helpful as aids in understanding children, they are best used in combination with other assessment instruments, and are never recommended as the sole method of personality assessment (Klopfer & Taulbee, 1976; Koppitz, 1983). One frequently used drawing method is Buck’s (1948) H-T-P technique, wherein the subjects is asked to draw a house, a tree, and a person, in that order. It is hypothesized that the house drawing elicits associations concerning home life and the family situation, whereas the tree and person drawings reflect aspects of self-concept and self-image (Hammer, 1985). According to Hammer (1985), the tree represents the more unconscious aspects of the selfconcept; the person represents those that are more conscious. There are differing opinions as to what the person drawing actually represents. Hammer (1985) maintains that three types of person drawings are possible: selfportrait, self-ideal, or a representation of a significant other. Koppitz (1968), comment-

reyn2-3.qxd

58

6/6/2003

9:02 AM

Page 58

II. PROJECTIVE METHODS

ing on children’s person drawings, has hypothesized that they represent the selfportrait and reflect the self-concepts, attitudes, and concerns of the moment. Yet she goes on to say that some children do not identify with the person they draw, but may depict a person who concerns them most at the time they make the drawing. DiLeo (1973) also speculates on the meaning of the person when drawn by a child. He points out that often the figure the child tends to draw is an adult, a significant figure in his or her life. DiLeo goes on to make the interesting observation that a child who is relatively well adjusted is more interested in the external world; hence the drawing is more a concept of humankind with the self incorporated. In such a case, the product is more likely to be a significant adult. On the other hand, the child who is beset by worries and anxiety is more likely to be self-absorbed and tends to produce a self-image. Koppitz (1968, 1984) has proposed analyzing children’s drawings of the human figure in a six-step procedure. This includes the examination of the drawing for the presence of what she has termed “emotional indicators.” These clinical signs are unusual features that have been found to occur significantly more often in the drawings of children with serious emotional problems. Koppitz originally identified these signs by examining the drawings of more than 1,800 school children. The criteria for inclusion of a sign as an emotional indicator were as follows: The sign was not present in more than 6% of the drawings of normal children; it differentiated between children with and without emotional problems; and it did not increase in frequency solely as a function of age. Using these criteria, Koppitz was able to identify 30 emotional indicators. He warns that these signs can only be interpreted as tendencies and must be viewed in the context of a more comprehensive assessment. Another popular drawing technique requires the child to “draw a picture of your family.” This Draw-A-Family technique (Hulse, 1951) results in a sort of family portrait, which is then analyzed by reference to the relative size, proximity, and placement of figures, and the significant additions, omissions, and elaborations of various family members.

A more recent variation on this technique is the K-F-D, in which the child is asked to “draw a picture of everyone in your family doing something” (Burns & Kaufman, 1970). The K-F-D is designed to assess a child’s self-concept and perception of interpersonal relations. Family dynamics are suggested by the interaction among figures, and drawings are examined in terms of action, style, and symbols. A number of interesting scoring systems have been proposed for the K-F-D (Cummings, 1981; Knoff & Prout, 1985). For example, a promising system developed by Mostroff and Lazarus (1983) consists of 20 variables on which the drawing may be rated. These researchers report interjudge agreement of 86 to 100 percent with a mean agreement of 97. Their data has also led them to conclude that the K-F-D probably measures state rather than trait characteristics. Other interpersonal and situational dynamics may be explored by such techniques as school drawings (Prout & Phillips, 1974). And recently, Knoff and Prout (1985) have suggested integrating the K-F-D and school drawings into a single approach. The use of children’s drawings in assessment raises validity issues similar to those found with other projective techniques. Falk (1981) has examined the validity question by analyzing the research on one popular technique, the Draw-A-Person. Falk asks why an intuitively obvious way of understanding children has not been documented as such. He concludes that there are three reasons, all having to do with the approach of research studies that purport to examine validity. These are (1) a poor understanding of the theory of projection (and hence unrealistic outcomes for such studies); (2) the relatively small number of studies using children as subjects, and generalizations that take place from adult studies; and (3) poorly conceived psychiatric categories, which make predictive studies difficult. Finally, Falk (1981) suggests some reasons for using drawing techniques with children. They tend to communicate indirectly, often by giving clues about what they think and feel. They often cannot express their feelings in words, especially their fears. Lastly, drawings are seen as a more “natural” activity for children than for adults. For

reyn2-3.qxd

6/6/2003

9:02 AM

Page 59

3. The Projective Hypothesis and Development of Projective Techniques

these reasons drawings seem useful with children, and, if used appropriately, may help in furthering the understanding of children.

Thematic Apperception Techniques Undoubtedly, the thematic projective technique that has had the most profound effect on applied psychology has been Henry Murray’s TAT. The TAT was originally developed in the 1930s as a means of applying Murray’s “need–press” theory of personality (Dana, 1985). Murray conceived of personality theory as a theory of motivation. He proposed a set of motives as antecendents of personality traits and types; these motives could be assessed by analyzing the stories told by subjects viewing a set of pictures. After some experimentation, Murray and his colleagues came up with 31 plates—30 black-andwhite illustrations and 1 blank card—which were to comprise the standard TAT set. Certain cards were designated for use with males, females, boys, and girls. The illustrations were meant to be rather ambiguous and to represent a wide variety of life situations: some common and everyday, others unusual and bizarre. Murray (1938) advocated that the stories be examined by using a need–press analysis—that is, by interpreting them in terms of the needs of the hero or heroine of the story and the environmental forces or presses to which he or she was subjected. Murray suggested that the interpretation of TAT stories should include consideration of unusually high or unusually low frequencies of motives and feelings in the set of stories. With this method, scores could be tallied, and a rank order of needs and presses could be established. However, Murray’s scheme proved cumbersome, requiring many hours of examiners’ time, so that it was not practical for clinical work (Bellak, 1971). (At one point, Murray, 1943, had identified some 40 needs and “thirty or more presses that could be applied to each story.) As the popularity of the TAT grew, various schemes were proposed for analyzing, scoring, and interpreting the TAT protocol. Murstein (1963) has reviewed a number of these in some detail. In a more recent review, Hughes (1983) has identified 39 such

59

methods, which she has classified as “structured psychodynamic,” “unstructured psychodynamic,” “response productivity,” and “formal characteristics” methods. One of the more promising approaches was proposed by Cox and Sargent (1950), whose scoring method was designed as a research tool for analyzing the normative aspects of selected TAT cards. Cox and Sargent proposed that the cards be analyzed in terms of six broad categories: needs, threats, feelings, heroes, action, and outcomes. A number of possible variables were to be considered within each category. With the exception of the action category, interjudge reliability ranged from .67 to 1.00, prompting the researchers to conclude that a scoring system for thematic material based on such “adaptive” aspects of the stories could be reliably employed. In more recent years, the scoring systems have tended to focus on a few specific factors, moving away from broad general categories. As Thomas and Dudek (1985) have pointed out, the TAT scoring systems that have been the most successful have been those tailored for a specific purpose. Their own system was designed to examine interpersonal affect. Other scoring systems of this type include the well-known work of McClelland and his colleagues on achievement motivation (McClelland, Atkinson, Clark, & Lowell, 1953), and recent attempts by Kaliopska (1982) to assess empathy and by McAdams (1980) to assess intimacy motivation, all using the TAT. Another recent development is a scoring system for the TAT and similar thematic techniques that places the responses in the context of the child or adolescent’s experience of life stress. This system, called the Need–Threat Analysis (NTA), was developed as part of a stress assessment system for children designed specifically as a way of providing information on sources of perceived stress (Chandler, 1985; Chandler, Shermis, & Lempert, 1989). The NTA uses two categories derived from Murray’s concepts through Cox and Sargent (1950); needs and threats. In this context, “stress” is defined as emotional tension arising from unmet needs and/or environmental threats, so that the purpose of the projective technique within the assessment is to identify those underlying needs and threats that are

reyn2-3.qxd

60

6/6/2003

9:02 AM

Page 60

II. PROJECTIVE METHODS

likely to make a particular event or situation especially meaningful, and hence stressful to an individual child. Preliminary findings suggested that forcing motivations into a dichotomy of needs or threats was unrealistic, since motives generally contain some proportion of each. Therefore, selected needs and threats from the Cox and Sargent (1950) system were paired to provide five need–threat combinations, or “binaries”: Independence– Domination, Affiliation–Rejection, Security –Insecurity, Achievement–Failure, and Aggression–Punishment. Each story is checked for the presence of each binary, and a tally is kept. The relative percentage of the occurrence of each binary throughout the set of cards is then calculated and the binaries are rank-ordered, so that the more important ones can be identified. Two studies were conducted using the CAT, an instrument similar to the TAT (Hughes, 1983; Lempert, 1986). Both studies used graduate students as judges. After a brief orientation, they were asked to score a number of CAT protocols. Interjudge reliability correlations of .68 and .69 were found in the Hughes and Lempert studies, respectively. In addition, the percent of agreement with respect to the dominant binary ranged from 80 to 100. Intrajudge reliability correlation over a 1-month interval ranged from .76 to .85. While various researchers were at work designing scoring systems for the TAT, another track of development was being followed by those who were developing TATlike instruments. For example, Symonds (1948) published a picture-story test with TAT-like cards that were specifically designed to be relevant to adolescents. Symonds, like Murray (1948), urged that the interpretation of TAT stories take into account the frequency of themes occurring over the entire set of stories. He maintained that from this frequency “one may logically assume that the theme has special significance for the subject” (p. 2). Even as Symonds found the TAT lacking in relevance for the adolescent population, so Bellak and Bellak (1949) found it to be of questionable usefulness with children, particularly younger children. These authors, in response to the need for an instrument more suited for the expression of the needs, emo-

tions, and drives of children, developed the CAT. Like the TAT, this technique consists of a set of pictures about which the child is asked to tell a story. The authors recommend its use with children between the ages of 3 and 10 years. In constructing the stimulus cards, the authors chose situations that they felt reflected critical issues of childhood, such as sibling rivalry, oral problems, attitudes toward parents, Oedipal concerns, and aggression. In this way the development of the CAT differed, at least in degree, from that of the TAT, where the attempt was to make the situation depicted relatively ambiguous. The degree of ambiguity of the stimulus in projective tests varies along a continuum from the abstract forms of the Rorschach through the relatively ambiguous situations of the TAT to the more blatant situations of the CAT, and finally to specifically oriented techniques such as the Blacky Pictures (Blum, 1950), which are explicitly designed to elicit comments on situations of psychoanalytic relevance (e.g., the Oedipal situation, castration anxiety, etc.). Another important difference between the CAT and the TAT is the use of animal figures in the pictures, based on the assumption that children would identify more readily with animals. In discussing analysis and interpretation, the authors of the CAT suggest that 10 aspects of the protocol be examined: main theme, hero, needs of the hero, concept of the environment, parental figures, significant conflicts, nature of anxieties, main defenses, severity of superego, and integration of ego. The authors resist providing a quantitative method of scoring, justifying their position by insisting that when one is using projective techniques, the focus should be on semantics and not statistics. Recently, there have been several attempts to develop new thematic techniques that might be shown to be more valid and reliable. The Tasks of Emotional Development (TED) is a technique developed to provide a more objective method that could include normative data on the emotional development of children (Cohen & Weil, 1975). The purpose of the TED is to assess the degree to which developmental tasks are met and mastered at various age levels. The test consists of a set of 13 photographs, each of

reyn2-3.qxd

6/6/2003

9:02 AM

Page 61

3. The Projective Hypothesis and Development of Projective Techniques

which depicts a different task of emotional development. The stories children produce in response to the photographs are scored using a set of objective rating scales that have five dimensions (perception, outcome, affect, motivation, and spontaneity), each of which permits the assessment of certain aspects of the mastery process. Pollak, Cohen, and Weil (1981) have provided a review of research applications of the TED. Still another recent development is the Roberts Apperception Test for Children (RATC; McArthur & Roberts, 1982). The RATC consists of a set of 27 stimulus cards (of which only 16 are administered to a given child). These cards depict common situations, conflicts, and stresses in children’s lives. They are intended to measure areas of interpersonal interaction and are scored in terms of adaptive scales (eight), clinical scales (five), and other indicators (three). In the recommended scoring procedure, each story is scored separately on all the rating categories by checking whether the category applies to that story. The authors also recommend that both qualitative and quantitative aspects be taken into account when interpreting stories. Vane (1981), in a review of the TAT, draws some conclusions relevant to thematic apperception techniques in general. He notes that the TAT’s popularity as a clinical instrument remains high in spite of criticisms of its validity. He then goes on to urge the development of more standardized approaches to the TAT, because “research experience suggests that the TAT method elicits material regarding attitudes and motivation that is not adequately elicited by other methods” (p. 319).

The Rorschach Technique The Rorschach technique consists of a set of 10 symmetrical inkblots (5 achromatic, 5 chromatic) printed on white cards, to which the respondent is asked to give associations. After all 10 plates have been presented, the examiner goes back over each response with the respondent to determine the precise area of the blot that elicited the response. The response is then scored (minimally) in terms of four categories: Location, Determinants, Content, and Originality. Location refers to the part of the blot that elicited the re-

61

sponse; Determinants to the features of the blot that contributed to the formation of the percept; Content to the object represented in the response; and Originality to the uniqueness of the response. In some scoring systems, additional scoring categories are provided. Once scoring has been completed, an interpretation is made in terms of personality characteristics that are purported to be associated with certain scores or configurations of scores when placed in the context of the whole protocol. There is an extensive literature on the Rorschach; much of it is devoted to validity issues, and much of that has used adult populations as subjects. Some of these validity issues have already been considered in the discussion of other projective techniques, although some are unique to the Rorschach (Exner, 1974; Lanyon & Goodstein, 1982). Although the preponderance of studies have used adults as subjects, the Rorschach has been used with children and adolescents in clinical practice. Some believe that the Rorschach can be used effectively with children as young as 5 years of age (Erdberg, 1985); others maintain that it can be used with even younger preschoolers, although extensive modifications may be necessary (Ames, Metraux, Rodell, & Walker, 1974; Halpern, 1971). An important concern in the Rorschach literature on children and adolescents has to do with the developmental aspects of the Rorschach protocols. In general, the effort has been to focus more closely on development and the deviations in development than on the more crystallized traits and symptomatic conditions normally associated with adult psychopathology. There have been a few attempts to document developmental changes as reflected in Rorschach protocols by developing norms for children and adolescents (Ames et al., 1974; Ames, Metraux, & Walker, 1971; Hertz, 1961). Adoption and use of such norms depend upon which one of the several scoring methods the examiner follows. And this lack of an agreed-upon scoring system has caused problems for Rorschach use over the years. As the Rorschach gained acceptance in this country, five major approaches evolved for scoring and interpreting the technique. During the 1960s, Exner (1969) began a study of each of the extant systems to devel-

reyn2-3.qxd

62

6/6/2003

9:02 AM

Page 62

II. PROJECTIVE METHODS

op his Comprehensive System, in which he hoped to combine the best elements of each. Exner’s goal was to develop a data-based system that could be standardized. By 1974 he and his colleagues had completed more than 150 studies, establishing the data base and supporting empirical evidence that he presented in his first volume on the Comprehensive System (Exner, 1974). A later volume was devoted to assessing children and adolescents (Exner & Weiner, 1982). Exner believes it is an error to refer to the Rorschach as a “projective technique,” because it is much more than that (Exner, 1983). He conceives of the Rorschach in two ways: as a perceptual–cognitive task, and as a stimulus to fantasy. The dual nature of the technique means that in interpretation different information is used in each context. When the Rorschach is seen as a perceptual–cognitive task, the respondent must impose structure and organization on an ambiguous stimulus. From an analysis of this task, certain structural aspects of the personality may be inferred, such as cognitive styles, tendencies, and traits. These are seen as aspects of the personality that are relatively stable across time and situations, as well as ones that are more nearly representative of actual behavior (i.e., the approach the respondent uses in dealing with life experiences). Exner maintains that inferences based on structural responses can be made with some certainty. When the Rorschach is seen as a stimulus to elicit fantasy, it provides information on personality dynamics. Information is provided on such aspects as needs, attitudes, and conflicts. Here the focus is on the content of the protocol—what the respondent says, not how he or she says it. Whereas perceptual–cognitive interpretations are representative of behavior, fantasy interpretations are seen as symbolic. Because there are a number of possible ways of understanding how fantasied material may be employed, inferences drawn from this aspect must remain more speculative. These two levels of interpretation point to an important distinction, and one with implications not only for how much confidence the examiner can put in the interpretations of various aspects of the protocol, but also for the design and execution of re-

search on the validity of the Rorschach (Exner & Weiner, 1982). As for using his system with children, Exner argues that “Rorschach behavior means what it means regardless of the age of the subject” (Exner & Weiner, 1982, p. 14). The age of the subject becomes important, however, in interpreting the findings. Here a knowledge of developmental psychology and developmental psychopathology is essential. For example, a certain constellation of scores may indicate that the respondent lacks emotional control. This finding can only be interpreted by knowing what constitutes appropriate emotional control for a child of a certain age. Exner provides normative data for children and adolescents, as well as extensive evidence on reliability of various aspects of the scoring system. In general, the reliability studies have taken a temporal-consistency approach using a test–retest design. This allows the examiner to view the relative stability of various scores and constellations over time. Exner (1983) reports good results, with most reliabilities above .70 and some exceeding .90. He also interprets those few scores that are very unstable over both short and long periods of time as of questionable significance, but as clearly correlated with external factors such as situational stress. Clearly, Exner’s work represents a major contribution to the development of the Rorschach. With his data on the construct validity and temporal stability of Rorschach variables, he has provided a solid empirical foundation upon which to build. Anastasi (1982) has summarized Exner’s efforts thus: A major contribution of Exner’s work is the provision of a uniform Rorschach system that permits comparability among the research findings of different investigators. The availability of this system, together with the research completed thus far, has injected new life into the Rorschach as a potential psychometric instrument. (p. 569)

CONCLUSIONS Projective techniques have been used for more than 50 years, and for most of that time they have been the subject of controversy. Critics have questioned the validity,

reyn2-3.qxd

6/6/2003

9:02 AM

Page 63

3. The Projective Hypothesis and Development of Projective Techniques

the value, and even the ethics of using them. Although a number of issues are involved in the controversy, it seems that the more fundamental questions have to do with the place of projective techniques within the personality assessment, and, ultimately, the purpose of that assessment. The confusion arises from differing opinions as to what kind of information is expected and what kind of information the selected technique can best provide. If projective techniques are seen as capable of contributing toward a global personality assessment by providing information on traits, characteristics, motivation, tendencies, intelligence, coping style, needs, temperament, defenses, fears and anxieties, ego strength, conflicts, adjustment, and psychopathological deviations, then the goal may be too ambitious, in which case projective techniques are bound to be disappointing. Advocates who claim such sweeping goals must assume the burden of proof. It may be because of such ambitious goals that projective techniques were originally oversold. If this is the case, then what we are seeing now is a necessary correction as more modest goals are being proposed, refined, and validated. Similarly, if the goal is to provide data relevant to a dichotomous decision on membership in a psychiatric category—a category that may itself be of questionable validity—then projective techniques will fail to achieve that goal with any consistency, a finding amply demonstrated. On the other hand, if the goal is to contribute toward understanding a child by providing specific information for the psychological assessment, if that information is made relevant by some theoretical context, and if that information is of the type that is best provided by the selected technique, then the probability of success is considerably enhanced. After decades of experience, it is becoming increasingly clear that each of the major techniques best provides different types of information when used with children (see Table 3.1). A final word should be said about the nature of the inferences drawn from projective techniques. As Exner and Weiner (1982) have pointed out, some projective technique data are more speculative than others. The challenge for researchers in the years ahead

63

TABLE 3.1. Major Projective Techniques as Information Sources Major technique

Best used to provide information on:

Creative drawings

Self-image Interpersonal relations Overall level of adjustment

Thematic apperception techniques

Current concerns Motivations, needs, and threats Perception of significant others

Rorschach technique

Perceptual–cognitive aspects, cognitive style, behavioral tendencies Psychodynamic aspects

will be to sort out which conclusions will be the best justified, and which will have to remain speculative. The results of such research should enable the clinician to know how much confidence can be placed in children’s responses to projective techniques, so that the techniques can be judiciously and appropriately used with children.

REFERENCES Ames, L. B., Metraux, B., Rodell, J. L., & Walker, R. (1974). Child Rorschach responses (rev. ed.). New York: Brunner/Mazel. Ames, L. B., Metraux, R., & Walker, R. (1971). Adolescent Rorschach responses: Developmental trends from ten to sixteen years. New York: Brunner/Mazel. Anastasi, A. (1982). Psychological testing (5th ed.). New York: Macmillan. Batsche, G. M., & Peterson, D. W. (1983). School psychology and projective assessment: A growing incompatibility. School Psychology Review, 12, 440– 445. Bellak, L. (1971). The TAT and CAT in clinical use (2nd ed.). New York: Grune & Stratton. Bellak, L., & Bellak, S. (1949). A manual for the Children’s Apperception Test. Larchmont, NY: CPS. Blum, C. S. (1950). The Blacky Pictures: A manual of instructions. New York: Psychological Corporation. Buck, J. N. (1948). The H-T-P technique: A qualitative and quantitative scoring manual. Journal of Clinical Psychology, 4, 317–396. Buck, J. N. (1971). The House–Tree–Person manual supplement. Los Angeles, CA: Western Psychological Services.

reyn2-3.qxd

64

6/6/2003

9:02 AM

Page 64

II. PROJECTIVE METHODS

Burns, R. C., & Kaufman, S. H. (1970). Kinetic Family Drawing (K-F-D). New York: Brunner/Mazel. Chambers, G. S., & Hamlin, R. W. (1957). The validity of judgments based on “blind” Rorschach records. Journal of Consulting Psychology, 21, 105–109. Chandler, L. A. (1985). Assessing stress in children. New York: Praeger. Chandler, L. A., Shermis, M. D., & Lempert, M. E. (1989). The need–threat analysis: A scoring system for the children’s apperception test. Psychology in the Schools, 26, 47–52. Cohen, H., & Weil, G. R. (1975). Tasks of Emotional Development. Brookline, MA: TED Associates, Cox, B., & Sargent, H. (1950). Brief reports: TAT responses of emotionally disturbed and emotionally stable children: Clinical judgments versus normative data. Journal of Projective Techniques, 14, 60–74. Cummings, J. A. (1981, August). An evaluation of objective scoring systems for Kinetic Family Drawings. Paper presented at the meeting of the American Psychological Association, Los Angeles, CA. Dana, R. H. (1985). Thematic Apperception Test. In C. S. Newmark (Ed.), Major psychological assessment instruments (pp. 89–135). Boston: Allyn & Bacon. DiLeo, J. H. (1973). Children’s drawings as diagnostic aids. New York: Brunner/Mazel. Elkind, D. (1976). Child development and education. New York: Oxford University Press. Erdberg, P. (1985). The Rorschach. In C. S. Newmark (Ed.), Major psychological assessment instruments (pp. 65–89). Boston: Allyn & Bacon. Exner, J. E. (1969). The Rorschach systems. New York: Grune & Stratton. Exner, J. E. (1974). The Rorschach: A comprehensive system. Vol. 1. New York: Wiley. Exner, J. E. (1983). The Rorschach: A history and description of the comprehensive system. School Psychology Review, 12, 407–413. Exner, J. E., & Weiner, I. B. (1982). The Rorschach: A comprehensive system. Vol. 3. Assessment of children and adolescents. New York: Wiley. Falk, J. D. (1981). Understanding children’s art: An analysis of the literature. Journal of Personality Assessment, 45, 465–472. Frank, L. K. (1939). Projective methods for the study of personality. Journal of Psychology, 8, 389–413. Freud, S. (1936). The problem of anxiety (J. Strachey, Trans.). New York: Norton. Freud, S. (1959). Formulations regarding the two principles in mental functioning. In The collected papers of Sigmund Freud (Vol. 4, pp. 13–22). New York: Basic Books. (Original work published 1911) Ginsberg, H., & Opper, S. (1979). Piaget’s theory of intellectual development (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall. Gittleman-Klein, R. (1978). Validity of projective tests for psychodiagnosis in children. In R. L. Spitzer & D. F. Klein (Eds.), Critical issues in psychiatric diagnosis. New York: Raven Press. Goh, D. S., & Fuller, G. B. (1983). Current practices in the assessment of personality and behavior by school psychologists. School Psychology Review, 12, 240–243.

Goodenough, F. L. (1926). Measurement of intelligence by drawings. New York: Harcourt, Brace & World. Halpern, F. A. (1971). The Rorschach test with children. In A. I. Rabin & M. R. Haworth (Eds.), Projective techniques with children. New York: Grune & Stratton. Hammer, E. F. (1958). The clinical application of projective drawings. Springfield, IL: Charles C Thomas. Hammer, E. F. (1985). The House–Tree–Person Test. In C. S. Newmark (Ed.), Major psychological assessment instruments (pp. 135–164). Boston: Allyn & Bacon. Harris, D. B. (1963). Children’s drawings as measures of intellectual maturity. New York: Harcourt, Brace & World. Hertz, M. R. (1961). Frequency tables for scoring Rorschach responses. Cleveland, OH: Case Western Reserve University. Holt, R. R. (1970). Yet another look at clinical and statistical prediction. American Psychologist, 25, 337–349. Holt, R. R. (1978). Methods in clinical psychology (Vol. 1). New York: Plenum Press. Hughes, C. A. (1983). Reliability of the need–threat scoring system for projective material. Unpublished master’s thesis, University of Pittsburgh. Hulse, W. C. (1951). The emotionally disturbed child draws his family. Quarterly Journal of Child Behavior, 3, 152–174. Jolles, I. (1971). A catalogue for the qualitative Interpretation of the House–Tree–Person (H-T-P). Los Angeles, CA: Western Psychological Services. Kaliopska, M. (1982). Empathy as measured by Rorschach and TAT. British Journal of Projective Psychology and Personality Study, 27, 5–11. Kenniston, K. (1965). The uncommitted. New York: Dell. Kessler, J. W. (1966). Psychopathology of childhood. Englewood Cliffs, NJ: Prentice-Hall. Klinger, E. (1971). Structure and functions of fantasy. New York: Wiley. Klopfer, W., & Taulbee, E. (1976). Projective tests. Annual Review of Psychology, 27, 543–569. Knoff, H. M. (1983). Personality assessment in the schools: Issues and procedures for school psychologists. School Psychology Review, 12, 391–398. Knoff, H. M., & Prout, H. T. (1985). The Kinetic Family System: A review and integration of the Kinetic Family and School Drawing Techniques. Psychology in the Schools, 22, 50–59. Koppitz, E. M. (1968). Psychological evaluation of children’s Human Figure Drawings. New York: Grune & Stratton. Koppitz, E. M. (1983). Projective drawings with children and adolescents. School Psychology Review, 12, 421–427. Koppitz, E. M. (1984). Psychological evaluation of Human Figure Drawings by middle school pupils. New York: Grune & Stratton. Lanyon, R. I., & Goodstein, L. D. (1982). Personality assessment (2nd ed.). New York: Wiley. Lempert, M. E. (1986). The need–threat scoring sys-

reyn2-3.qxd

6/6/2003

9:02 AM

Page 65

3. The Projective Hypothesis and Development of Projective Techniques tem: A study of reliability and agreement. Unpublished manuscript, University of Pittsburgh. Lindzay, G., & Herman, P. S. (1955). Thematic Apperception Test: A note on reliability and situational validity. Journal of Projective Techniques, 19, 36–42. Little, K. B., & Shneidman, E. S. (1959). Congruencies among interpretations of psychological tests and anamnestic data. Psychological Monographs, 73 (No. 476). Lubin, B., Larsen, R. M., & Matarazzo, J. D. (1984). Patterns of psychological test usage in the United States: 1935–1982. American Psychologist, 39, 451–454. Machover, K. (1949). Personality projection in the drawings of a human figure. Springfield, IL: Charles C Thomas. Magnusson, D. (1960). Some personality tests applied on identical twins. Scandinavian Journal of Psychology, 1, 55–61. McAdams, D. P. (1980). A thematic coding system for the intimacy motive. Journal of Research in Personality, 14, 413–432. McArthur, D. S., & Roberts, G. E. (1982). Roberts Apperception Test for Children: A manual. Los Angeles, CA: Western Psychological Services. McClelland, D. C., Atkinson, J. W., Clark, R. A., & Lowell, E. L. (1953). The achievement motive. New York: Appleton-Century-Crofts. McDermott, P. A. (1980). A systems–actuarial method for the differential diagnosis of handicapped children. Journal of Special Education, 14, 7–22. McNeil, E, (1976). The psychology of being human. San Francisco: Canfield. Megargee, E., & Cook, P. E. (1967). The relationship of TAT and inkblot aggression content scales with each other and with criteria of overt aggression in juvenile delinquents. Journal of Projective Techniques and Personality Assessment, 31, 48–60. Morstoff, D. L., & Lazarus, P. J. (1983). The Kinetic Family Drawing: The reliability of an objective scoring system. Psychology in the Schools, 20, 16–20. Murray, H. (1938). Explorations in personality. New York: Oxford University Press. Murray, H. (1943). Thematic Apperception Test manual. Cambridge, MA: Harvard University Press. Murstein, B. I. (1963). Theory and research in projective techniques. New York: Wiley. Obrzut, J. E., & Cummings, J. A. (1983). The projective approach to personality assessment: An analysis of thematic picture techniques. School Psychology Review, 12, 414–420. Ogden, D. P. (1977). Psychodiagnosis and personality assessment: A handbook (2nd ed.). Los Angeles, CA: Western Psychological Services.

65

O’Leary, K. D., & Johnson, B. (1979). Psychological assessment. In H. Quay & J. S. Werry (Eds.), Psychopathological disorders of childhood (2nd ed.). New York: Wiley. Piotrowski, C. (1984). The status of projective techniques: Or, “wishing won’t make it go away.” Journal of Clinical Psychology, 40, 1495–1502. Pollak, J., Cohen, H., & Weil, G. (1981). The Tasks of Emotional Development: A survey of research applications. Psychology: A Quarterly Journal of Human Behavior, 18, 2–11. Polyson, J., Norris, D., & Ott, E. (1985). The recent decline in TAT research. Professional Psychology: Research and Practice, 16, 26–28. Prelinger, E., & Zimet, C. N. (1964). An ego-psychological approach to character assessment. New York: Free Press. Prout, H. T. (1983). School psychologists and social– emotional assessment techniques: Patterns in training and use. School Psychology Review, 12, 377–383. Prout, H. T., & Phillips, P. D. (1974). A clinical note: The Kinetic School Drawing. Psychology in the Schools, 11, 303–306. Rorschach, H. (1921). Psychodiagnostik. Bern: Bircher. Sattler, J. M. (1982). Assessment of children’s intelligence and special abilities. Boston: Allyn & Bacon. Schwartz, F., & Lazar, A. (1979). The scientific status of the Rorschach. Journal of Personality Assessment, 43, 3–11. Spitzer, R. L., & Fleis, J. L. (1974). A re-analysis of the reliability of psychiatric diagnosis. British Journal of Psychiatry, 125, 341–347. Symonds, P. (1948). A manual for the Symonds Picture–Story Test. New York: Columbia University Press. Thomas, A. D., & Dudek, S. Z. (1985). Interpersonal affect in TAT responses: A scoring system. Journal of Personality Assessment, 49, 30–37. Urban, W. (1963). The Draw-A-Person catalogue for interpretive analysis. Los Angeles, CA: Western Psychological Services. Vane, J. R. (1981). The TAT: A review. Clinical Psychology Review, 1, 319–336. Wagner, E. E., Alexander, R. A., Roos, G., & Adair, H. (1986). Optimum split-half reliabilities for the Rorschach: Projective techniques are more reliable than we think. Journal of Personality Assessment, 50, 107–112. Wiggins, J. S. (1980). Personality and prediction: Principles of personality assessment. Reading, MA: Addison-Wesley.

reyn2-4.qxd

6/6/2003

9:03 AM

Page 66

4 Projective Storytelling Techniques

JAMES L. DUPREE FRANCES PREVATT

Despite decades of criticism regarding their poor psychometric properties, projective storytelling methods continue to be among the most popular and frequently used assessment techniques (Anastasi & Urbina, 1997; Piotrowski, Keller, & Ogawa, 1993). Beginning with Murray’s introduction of the Thematic Apperception Technique (TAT) in the mid-1930s, more than two dozen similar methods have been developed in the intervening 65 years. This chapter reviews a total of 14 projective storytelling techniques that are used with children and adolescents. These techniques, all similar to the original TAT, have been attempts to expand the adult method for use with children, various ethnic groups, specific age groups, certain disorders; to improve psychometrics; or simply to “modernize” the original drawings. Because the different techniques generally have some grounding in the “projective hypothesis,” this theory is discussed first.

feelings are transferred or attributed to external events or individuals. However, more recently, projection in relation to thematic storytelling techniques has taken on a less stringent definition. According to Rabin (1986), projection is a common everyday occurrence, a part of the normal thought process by which all individuals tend to “externalize” as they interpret the environment and respond to it. Rabin goes on to state: In a sense, people are projecting all the time; when perceiving and responding to the environment they are expressing their personal needs, motivations, and unique characteristics. When a person faces a particular stimulus or situation she responds . . . in her own particular manner. (p. 5)

Thus, in this light, projection is seen as less of an unconscious defensive maneuver and more of a normal cognitive operation. Projective responses are viewed differently from responses given on more structured tasks, in that projection allows the child to pull from a wide range of experiences and fantasy material while being relatively unaware of the manner in which the responses will be interpreted.

THE PROJECTIVE HYPOTHESIS A concept from early Freudian psychoanalytic theory, “projection” is a defense mechanism in which anxiety-producing thoughts or 66

reyn2-4.qxd

6/6/2003

9:03 AM

Page 67

4. Projective Storytelling Techniques

In TAT-type techniques, the child is asked to make up a story and is assured that there is no right or wrong answer. Depending on the ambiguity of the picture (ranging from a blank card to a boy huddled on the floor with a gun-like object beside him), any number of stories are possible. Although the latter picture pulls more for a story with a sad or aggressive theme than does the blank card, it nonetheless remains up to the subject to interpret the picture. That interpretation will necessarily borrow on events, feelings, and important needs in the subject’s own life. Just as a young child is unlikely to tell a story involving a kibbutz in Israel if he or she has never personally or indirectly known of such a thing, the projective hypothesis assumes that a child is unlikely to recite numerous stories involving being rejected or unloved if that experience is not in some way important to that child. Therefore, we come to a definition of “projection” as an apperceptive process by which the child integrates a stimulus (the picture) with both past experiences and current psychological concerns (Kagan, 1960). The ability to indulge in make-believe or fantasy, rather than to give a factual description of this picture, will vary across children. Of course, the child’s fantasy should be bound by the reality constraints inherent in the picture. A child who gives only a factual description may be manifesting some resistance or inhibition, whereas a child who reports themes consistently unrelated to the scene portrayed in the picture may be reacting with fewer controls to his or her own internal needs or anxieties. It is believed that when both this and other aspects of the child’s stories are examined, storytelling techniques lead to an assessment of defensiveness, ego strength, assets and liabilities, coping styles, interpersonal relationships, and problem solving, as well as basic organizing principles of the subject’s personality (Rabin, 1960). Although there is general agreement that projection is the cornerstone of most thematic storytelling techniques, there has been constant criticism that techniques based on this theory are psychometrically untenable. Numerous reviews have stressed the lack of empirical support for the reliability or validity of storytelling techniques in particular and projective assessments in general (Anastasi

67

& Urbina, 1997; Cleveland, 1976; Dean, 1984; Holmen & Docter, 1972; Lanyon, 1984; Munter, 1975). Defenders of projective techniques generally resort to one of two claims: (1) that negative research findings were often due to methodologically flawed studies, and (2) that projectives should be evaluated differently from more objective assessment techniques. Blatt (1978) characterizes the former view, charging that many negative findings failed to consider the configural approach needed to interpret projective test data, studied variables in isolation (e.g, the relationship between TAT stories and aggression rather than TAT stories and overall personality), or used imprecise diagnostic classifications (e.g., attempted to use the TAT to distinguish conduct-disordered from hyperactive subjects, but used questionable criteria to diagnose the two groups). The second viewpoint continues to flourish, as stated by Schwartz and Lazar: Although projectives may have poor psychometrics by some standards, interpretation deals with the possible meanings of a response, not probabilistic status. Over an extensive projective battery, the clinician is more interested in common themes or consistencies generated from responses than in psychometric properties. (cited in Obrzut & Cummings, 1983, p. 189)

Specifically directed at the TAT are the claims that although it does not fulfill the requirements of a psychometric instrument, it is indeed a method of describing personality that provides rich idiographic data (Aronow, Reznikoff, & Rauchway, 1979; Dana, 1985; Goldman, Stein, & Guerry, 1983). Surprisingly, although projective storytelling techniques consistently receive poor empirical support, they continue to enjoy widespread use in clinical practice and applied settings (Piotrowski, 1983). In a 1997 survey by Rossini & Moretti (cited in Cramer, 1999), the TAT was found to be one of the two most frequently used projective techniques. Wade and Baker (1977) attribute the continued employment of the TAT and other projectives to the fact that assessment continues to be a critical need, and that as yet there are few practical alternatives to projective tests. In addition, they stress that clinicians give greater weight to

reyn2-4.qxd

6/6/2003

9:03 AM

Page 68

68

II. PROJECTIVE METHODS

personal clinical experience than to experimental evidence.

FACTORS RELEVANT TO EVALUATING CHILDREN Before moving on to a discussion of specific storytelling techniques, we first review several general factors relevant when evaluating children. Of course, it is important to consider a child’s age, developmental status, and verbal ability when interpreting thematic stories. In using any type of age norms for test variables, one must realize that the norms mask a good deal of unevenness in the profiles of individual children (Rabin, 1960). Thus, a child’s actual performance is likely to spread over several age levels, reflecting the lability of his or her ego development and cognitive capabilities. For example, a 4- or 5-year-old is just beginning to exercise control over impulses; thus impulsivity and poor judgment would be expected in a child by this age, and would not necessarily be indicative of emotional problems (Obrzut & Cummings, 1983). It becomes quite difficult, when working with young children, to distinguish between aberrant and merely immature responses (Altman, 1960). One cannot expect to see patterns that reflect general coping styles until at least age 6 or 7. The length of thematic stories tends to increase with age, with girls telling longer stories than boys from age 6 up until early adolescence (Kagan, 1960). Kagan also contends that the fantasy productions of younger children contain more distortions and omissions, while 9- to 10year-olds tend to give more concrete, descriptive stories. Thus, before assessing any young child, one should have an idea of normative development for children of different ages. In the following sections, we review several storytelling techniques, with emphasis on specific usage, scoring systems, and psychometric qualifications. We have selected the 14 techniques to review based on present usage, availability, cultural focus, historical value, and new developments in technology. Our choices were partially guided by Tests in Print (Murphy, Impara, & Plake, 1999) and Psychological Assessment in the Schools (Impara & Murphy, 1994),

both from the Buros Institute of Mental Measurements, as well as by catalogs of publishers and computerized searches. A number of variations on basic storytelling techniques are also discussed at the end of the chapter, but in less detail. Although our coverage is not exhaustive, we hope that the techniques selected for inclusion in this chapter will provide the reader with a solid historical base, as well as coverage of some of the more recent developments in the field.

THE THEMATIC APPERCEPTION TEST The TAT was the first widely used thematic technique, with most other methods being modifications of this test. Developed by Henry Murray (1938, 1943), the TAT was based on psychoanalytic theory and was originally intended to hasten the process of therapy. It was expected that through projection, the therapy client would reveal personality characteristics that might otherwise take months to uncover. In particular, the TAT was presumed to reveal the individual’s dominant drives, emotions, traits, and conflicts by identifying significant needs, presses, and themes (Murray, 1943). As such, the TAT was intended to explore personality dynamics rather than to provide a differential diagnosis. After a number of revisions, Murray’s TAT contained 31 pictures: 11 suitable for both sexes, 7 for girls and women, 7 for boys and men, and 1 blank card for both men and women. Administration occurred over two sessions a week apart, with 10 cards used each session in a particular order. The cards were purposefully varied in content, in order to elicit fantasies concerning most areas of importance in the subject’s life. They were structured enough to allow easy storytelling, yet ambiguous enough to allow for a variety of possible stories (Karon, 1981). Murray(1943) gave the following instructions: Tell me what has led up to the event shown in the picture, describe what is happening at the moment, what the characters are feeling and thinking, and then give the outcome. Since you have 50 minutes for 10 pictures, you can devote 5 minutes to each story. (p. 3)

reyn2-4.qxd

6/6/2003

9:03 AM

Page 69

69

4. Projective Storytelling Techniques

Murray allowed for a reminder of the instructions after the first card. After this, nothing was to be said except to inform the subject of the time limits, to give occasional praise, and to give brief prompts such as “What led up to this situation?” if crucial details were omitted. Although Murray intended that 20 cards be used, the TAT as currently administered is more likely to include only 10 cards. There is no general consensus as to which cards should be administered. Some argue that each examiner should become familiar with a certain set of 10 cards and use those cards exclusively. Others argue for specific cards, depending on the client’s particular referral issue. Table 4.1 presents a summary of cards suggested for particular age groups and psychological concerns. Some investigations have focused on administration factors affecting story production. Dana (1985) reports that cards of medium ambiguity are most useful in eliciting relevant personality data, at least for a college student population. Obrzut and Boliek (1986) raise the issue of whether

clarifying questions should be asked, and they conclude that the more questions asked, the less projective the material solicited will be. Finally, there is the issue of how stories should be recorded. Dana (1986) contends that when the examiner leaves the room and asks the subject to record his or her own stories, there is a tendency for more negative stories to be produced. However, when the examiner remained in the room, Baty and Dreger (1975) found no differences in tape-recorded, subject-written, or examiner-written stories. It should be noted that both these studies involved college students. Most examiners find that for younger age groups, it is often infeasible to have the child write his or her own stories. The TAT has continually been criticized for not having a scoring system that is both usable and sufficiently inclusive to be clinically relevant. Although a number of systems have been developed, they either tend to be too lengthy and cumbersome to be practical or lack any sound psychometric basis (Prevatt, 1999). One difficulty with re-

TABLE 4.1. Suggested Cards for TAT Administration Cards

Suggested use

Source

1, 2, 3BM, 4, 6BM, 7BM, 11, 12M, 13MF

Standard administration: males

Bellak (1971)

1, 2, 3BM, 4, 6GF, 7GF, 9, 11, 13MF

Standard administration: females

Bellak (1971)

1, 6BM, 7GF, 8BM, 12M, 13MF, 14, 17BM

Standard administration

Cramer (1999)

1, 3BM, 3GF, 4, 6, 7, 10, 11, 12M, 13MF, 14, 16, 20

Standard administration

Karon (1981)

1, 2, 5, 7GF, 12F, 12M, 15, 17BM, 18BM, 18GF

Standard administration: adolescents

Rabin & Haworth (1960)

1, 8BM, 14, 17BM

7- to 11-year-olds: achievement and status goals

Obrzut & Boliek (1986)

3BM, 8BM, 12M, 14, 17BM

7- to 11-year-olds: aggression

Obrzut & Boliek (1986)

3BM, 7GF, 13B, 14

7- to 11-year-olds: concern for parental nurturance and rejection

Obrzut & Boliek (1986)

1, 3BM, 7GF, 14

7- to 11-year-olds: parental punishment and attitudes toward parents

Obrzut & Boliek (1986)

7GF, 18GF, 3GF, 8GF

5- to 10-year-olds

Gerver (1946)

1, 3, 6, 7, 12M, 14, 16

Suicidal ideation

Karon (1981)

12M, 12F

Subject’s reaction to therapy/therapist

Karon (1981)

reyn2-4.qxd

6/6/2003

9:03 AM

Page 70

70

II. PROJECTIVE METHODS

search on the TAT is that so many scoring systems are used, it is difficult to determine whether study findings are the result of method variance or test variance. Table 4.2 summarizes a sample of TAT scoring systems. This table is not exhaustive, yet it does indicate the difficulty in attempting to

TABLE 4.2. TAT Scoring Systems by Author Morgan & Murray (1935): see text. Bellak (1947): content analysis of interpersonal relationships, themes, heroes, needs, drives, the environment, significant conflicts, anxieties, defenses, punishments, and ego integration. McClelland, Atkinson, Clark, & Lowell (1953): measures need for achievement. Fine (1955): 31 feelings categorized as positive, negative, or other. Davids & Rosenblatt (1958): measures hostility and aggression. Hafner & Kaplan (1960): measures hostility and aggression. Aaron (1967): measures depression. Newman, Newman, & Sells (1974): scaling procedure to evaluate psychological deviance, including verbal fluency, conceptual maturity, dysphoric mood, emotionality, and narrative fluency. Westen, Lohr, Silk, Kerber, & Goodrich (1985): see text. Taylor & Franzen (1986): internalized Object Relations Scale. Thompson (1986): Affect Maturity scales. McGrew & Teglasi (1990): includes characteristics of the story structure (perceptual organization and internal logic), verbalizations (personalization, expressions of inadequacy, bizarre comments), and formal characteristics of story content (judgment, social conventionality). Cramer (1991): Defense Mechanism Inventory that measures three defense mechanisms: denial, projection, and identification. Locraft & Teglasi (1997): five dimensions of empathy—comfort with feelings, source of positive feelings, source of negative feelings, mutuality in relationships, and inner cohesion. Woodrich & Thull (1997): scores based on physical aggression, supernatural powers (heroes), naming of specific popular characters, and quantities (mentions numbers).

evaluate the TAT apart from its myriad systems. The original system (Morgan & Murray, 1935) is discussed in more depth, as is a more recent system (Westen, Lohr, Silk, Kerber, & Goodrich, 1985) that has been used fairly extensively in the past decade. The first systematic approach was developed by Morgan and Murray (1935). They suggested that after the stories had been related, the examiner should inquire generally as to the source of the stories. A content analysis was then conducted, with each event analyzed for “needs” (forces emanating from the hero) and “presses” (forces emanating from the environment). Next, each story was analyzed for the following: 1. Characteristics of the hero. 2. Motives, trends, needs, and feelings of the hero, as measured by their intensity, duration, frequency, and importance to the plot. These included such variables as achievement, aggression, dominance, and nurturance. 3. Forces in the hero’s environment, such as traits of others, their effect on the hero, aggression by others, dominance, seduction, rejection, or physical danger. 4. Outcomes to each story. 5. Themes, defined as the interaction of the hero’s needs and presses combined into a pattern that is played out in the person’s story. For example, the need for nurturance and the press of parental rejection might be portrayed in themes centering on children being left behind by parents. 6. Interests and sentiments, such as the positive or negative value of the older woman (mother figure), father figure, or same-sex figure. The Object Relations and Social Cognition Scales (SCORS; Westen et al., 1985) is based on the premise that mental representations of one’s self and others develop throughout childhood, changing with each new relationship. These mental schemas endure and guide interpersonal behaviors throughout the lifespan (Conklin & Westen, 1998). The TAT cards are felt be an excellent source for accessing these interpersonal schemes that guide thoughts, feelings, and behaviors. Westen emphasizes that this system elicits implicit motives (similar to unconscious), as opposed to attributed mo-

reyn2-4.qxd

6/6/2003

9:03 AM

Page 71

4. Projective Storytelling Techniques

tives (similar to conscious), explaining the oft-cited lack of correlation between results obtained from thematic stories and various self-report measures of personality. The 1985 SCORS assessed four dimensions: Complexity of Representations of People, Affect Tone of Relationship Paradigms, Emotional Investment in relationships and Moral Standards, and Understanding Social Causality. With the exception of Affect Tone, each is thought to follow a developmental progression. The latest version of the SCORS splits the Emotional Investment scale into Emotional Investment in Values and Moral Standards and Emotional Investment in Relationships. A sixth scale is Dominant Interpersonal Concerns. The SCORS includes a detailed scoring manual. Several empirical investigations have documented the reliability and validity of the SCORS. As cited in Ackerman, Clemence, Weatherill, and Hilsenroth (1999), these include Hibbard, Hilsenroth, Hibbard, and Nash (1995); Ornduff, Freedenfeld, Kelsey, and Critelli (1994); Ornduff and Kelsey (1996); Porcerelli, Hill, and Dauphin (1995); Stricker and Healey (1990); and Westen (1990, 1991). Specifically, the SCORS has been used to differentiate among adult patients with various personality disorders (Ackerman et al., 1999) and physically abused versus nonabused children (Freedenfeld, Ornduff, & Kelsey, 1995; Ornduff & Kelsey, 1996). Whether using a structured scoring system or merely scoring the TAT informally, many clinicians include an analysis of themes present in the stories. Karon (1981) lists common TAT themes, including relations with parents, relations between mother and father, heterosexual relationships, suicidal ideation, tenderness and affection, and loneliness. In 1971, Bellak included such themes as autonomy versus compliance, depression, aggression, sexual conflicts, childhood issues, and ambitions. Also used in more informal scoring systems are some general scoring principles. Dana (1985) discusses the principal of distance, stating that less acceptable drives are more likely in stories containing characters who differ from the client in some way, such as age, sex, race, or social status. Dana further suggests that the examiner consider how well the client is able to adhere to di-

71

rections. For example, if the client is instructed to give an ending to every story but does so on only 8 of 10 cards, the examiner should determine whether cards eliciting certain themes have been left unfinished. Karon (1981) suggests that bright people and healthy people tend to give longer stories. Therefore, disturbing material in a brief protocol should be given more credence than disturbing material embedded in a longer, more elaborate story. Karon also suggests that the less the content “fits” the card, the more meaningful it will be. For example, a story about a man killing someone is more suggestive of pathology or anxiety if the picture depicts a woman and a girl sitting on a sofa. Finally, Karon believes that a subject’s behaviors are predicted by behaviors of characters in the story, that a subject’s verbalizations are predicted by characters who say things, and that a subject’s thoughts are predicted by characters who think things. For example, a story about a man who considers suicide is more likely to be predictive of thinking about suicide than of actually attempting suicide. This last-mentioned scoring principle attempts to resolve a common criticism of the TAT—namely, that story content is not necessarily related to a client’s actual behaviors. Many investigators have suggested that a need or personality characteristic exhibited on the TAT may never be acted on in real life; rather, it exists only in the client’s fantasies or unconscious (Dana, 1986; Lanyon & Goodstein, 1982; Vane, 1981). Santostefano (1970) contends that a need expressed in a TAT story may be manifested in the client’s later verbalizations, thoughts, or behaviors, but that there is no way of determining which relationship will actually exist. Thus, there seems to be concurrence that a client who produces a number of agressive stories may actually be agressive, say aggressive things, or merely think aggressive thoughts. Overall, the literature on the TAT leads to a rather curious conclusion: Despite the fact that there appears to be no consistently used, psychometrically sound scoring system for it, the TAT continues to enjoy widespread usage in academic training programs, internships, and clinical practice. Even the administration of cards and specific instructions is flexible, and is generally left to the discretion of the individual examiner. Some general

reyn2-4.qxd

6/6/2003

9:03 AM

Page 72

72

II. PROJECTIVE METHODS

scoring principles are often adhered to, such as examining the cards for consistent needs, presses, and themes; however, interpretation almost certainly depends more on the clinical skills of the examiner than on the psychometric properties of the test.

THE CHILDREN’S APPERCEPTION TEST The Children’s Apperception Test (CAT) was designed by Bellak and Bellak (1949) as a downward extension of the TAT suitable for children ages 3–10. The CAT consists of 10 pictures of animal characters. An alternate form, the CAT-H, contains human figures. A new version, the CAT-S (Bellak & Bellak, 1991), is designed for young or retarded children and depicts less common themes, such as parental separation, physical disability, and mother’s pregnancy. The original CAT pictures were designed from preconceived ideas about problems, situations, and roles that would be especially relevant to children. Animals were used for three reasons: (1) Most animals with which children are familiar are small and below children in “pecking order”; (2) animals are not as threatening as humans, and thus it is thought to be easier for children to ascribe unacceptable traits or emotions to them (e.g., jealousy and aggression); and (3) it is easier to draw animal figures of an ambiguous sex and age (Bellak & Adelman, 1960). Administration of the CAT is similar to that of the TAT: Children are asked to make up stories for each card with a beginning, middle, and end, and to tell what the characters might be thinkng or feeling. All 10 cards are used, with a set order maintained. As with most tests given to children, it is important to establish good rapport before initiating the CAT. The CAT pictures were designed to elicit responses to typical childhood concerns, such as sibling rivalry, relations with parents, aggression, fear of being lonely at night, toilet behavior, feeding problems, and problems of growth. Interpretation is similar to that for the TAT; Bellak and Bellak (1961; see also Bellak & Abrams, 1997) analyzed the stories for the following: 1. Main theme; 2. Main hero and self-image;

3. Main needs and drives of the hero; 4. Concept of the environment; 5. The way figures are seen and reacted to (e.g., supportive and competitive); 6. Conflicts; 7. Nature of child’s anxieties; 8. Defenses used; 9. Adequacy of the child’s superego as manifested by the punishment received for wrongdoings; and 10. Integration of the ego as manifested by the compromise between drives and demands of the ego. The majority of research on the CAT is several decades old. For example, Passman and Lautman (1982) examined factors affecting responsiveness to the CAT during administration. Haworth (1986) reviewed the research on the CAT over the past 20 years. Almost none of these studies dealt with the psychometric properties of the CAT. Most investigations had to do with aspects of the testing situation, such as effects of child anxiety on responses, the role of the examiner, coping style of children when taking the CAT, sex differences, and differences in the CAT and CAT-H. Three of the studies reported by Haworth involved attempts to differentiate diagnostic groups by means of the CAT: stutterers from nonstutterers, emotionally disturbed from normal children, and psychotics from nonpsychotics. Differences on some scoring criteria were found in all three studies; however, different scoring systems were used in each of the studies. Given the dearth of recent research validating a structured scoring system for the CAT, it does not appear that the CAT meets common criteria for a psychological test. This is supported by recent reviewers, who conclude that it is best used as a clinical tool (Kroon, Goudena, & Rispens, 1998). Knoff (1998) is more adamant, stating that the CAT “is an historical anachronism” (p. 233).

THE ROBERTS APPERCEPTION TEST FOR CHILDREN The Roberts Apperception Test for Children (RATC; McArthur & Roberts, 1982) is a storytelling technique designed for children

reyn2-4.qxd

6/6/2003

9:03 AM

Page 73

4. Projective Storytelling Techniques

and adolescents ages 6–15. The RATC is intended as a projective test with a standardized scoring system. The RATC consists of 27 cards, 11 of which have parallel forms for males and females. Sixteen cards are administered during a testing, which takes about 20 to 30 minutes. The examiner queries the subject on the first two cards only, asking what happened in the story, what happened before, what the characters are feeling and talking about, and how the story ends. The following scores may be obtained from the RATC: 1. Adaptive scales: Reliance on Others, Support Others, Support Child, Limit Setting, Problem Identification, Resolution (three types). 2. Clinical scales: Anxiety, Aggression, Depression, Rejection, Unresolved. 3. Critical indicators: Atypical Response, Maladaptive Outcome, Refusal. 4. Supplementary measures: Ego Functioning, Aggression, Levels of Projection. The RATC was reviewed for many years as having great promise (Worchel, 1987). Early reviews lauded the comprehensive test manual with its detailed administration, scoring system, interpretation guide, and case studies. The pictures, designed to be more modern than those in the TAT and CAT, depict scenes intended to elicit common concerns. For example, specific cards portray family confrontation, family conference, parental depression, parental limit setting, parental affection, physical aggression, sibling rivalry, school issues, peer and racial interactions, and observation of nudity. A new handbook (Roberts, 1994) provides greatly expanded and more explicit scoring procedures, has added a chapter on ethnic and cultural considerations, and gives numerous case studies. In addition, “in the interest of providing ethnically appropriate testing materials for black children, a parallel series of stimulus cards depicting black characters was developed” (p. 143). Unfortunately, the RATC has not lived up to its promise. The majority of research regarding the RATC involves unpublished doctoral dissertations. Studies using the RATC as a measure of depression or anxi-

73

ety failed to establish the discriminant validity of the test (Joiner, 1996; Joiner, Schmidt, & Barnett, 1996; Worchel, Rae, Olsen, & Crowley, 1992). The original 200-person standardization sample has been criticized as being too small and overly select, resulting in nonrepresentative norms (Finch & Belter, 1993; Kamphaus & Frick, 1996). Bell and Nagle (1999) evaluated the standardization sample by comparing it with a nonclinical sample. Their sample differed from normative sample on six of eight adaptive scales and three of five clinical scales. Some investigators have successfully used the RATC to discriminate sexually abused from nonabused children; however, they have used their own scoring systems (Friedrich & Share, 1997; Smith, 1992). The new manual gives no new information about the standardization sample, no normative data, and no technical information on validity or reliability. We can only conclude that RATC, at the present time, has not fulfilled its promise as a psychometrically validated test.

THE MAKE-A-PICTURE STORY TEST The Make-a-Picture Story Test (MAPS; Shneidman, 1952, 1960) was designed to provide more choices for respondents and examiners than those provided by the TAT. The test consists of 22 background cards and 67 figure cards to be superimposed over the backgrounds. Most of the background cards are easily identifiable scenes, such as a living room, bridge, or street. A few background cards have less structure, such as a “dream” card. Of the 67 figures, most are human, varying in gender, age, race, pose, and clothing. Two animal and six fictitious figures (e.g., Superman and a pirate) are included. The test was designed for both adults and children and can be used with children as young as 6 years old. Test time may vary from 45 minutes to 3 hours, depending on the purpose of the evaluation and the abilities of the child. Usually 10 backgrounds are used, with the examiner selecting the first 8 and the child selecting the last 2 from the remaining cards. A suggested alternative is to let the child look over all the material and pick his

reyn2-4.qxd

74

6/6/2003

9:03 AM

Page 74

II. PROJECTIVE METHODS

or her own backgrounds (Koppitz, 1982). Upon presentation of each background, the child selects one or more of the 67 figures and, after placing them on the background, tells a story. After the story is finished, the child is asked to provide a title for the story. Although general prompting is acceptable, more detailed inquiry is discouraged in diagnostic usage. The examiner records the responses verbatim, including the title and incidental remarks, and also indicates the choice and placement of the figures on a location sheet provided with the test. The test is psychodynamically based, although examiners may also gain information regarding cognitive distortions and interpersonal relationships. Story analysis may be accomplished in a manner similar to that for the TAT, such as noting the main character, needs, motivations, affect, and conflicts. No objective scoring system is typically used with children. The manual addresses diagnostic usage only, although the author considers therapeutic and research applications feasible. The original observations used in the development of the test were based on responses by adults, and many representative responses are presented in the manual. Only a few child responses are presented, although elsewhere Shneidman (1960) provides a case illustration with a 13-year-old Mexican American girl. For some hyperactive children, the MAPS may be an alternative to the more passive TAT format (Obrzut & Boliek, 1986), although the administration may need to be more controlled to avoid having the large number of materials strewn about (Goldman et al., 1983). The MAPS test has been considered useful in studying identification patterns in retarded and nonretarded children. Ward (1973) found that nonretarded boys projected themselves more readily into the stories, more often chose boys as main characters, and more often identified the main characters as themselves. Schneidman admits to the diffculties in establishing reliability and validity for the MAPS, and Jensen (1965) does not recommend the use of the MAPS for this very reason. However, Schneidman (1999) continues to be strongly supportive of both the MAPS and the TAT as being able to uncover the “secret wishes and unconscious fantasies” of respondents if used in “psychdy-

namically skilled hands.” He goes so far as to state that the thematic projective test is “no less than a magic set of optics without which psychologists have only partial psychological vision” (p. 97). In a study that compared MAPS stories by adjusted children to those made by children from special education classes, Heuvelman and Graybill (1990) demonstrated some indication of acceptable validity. Their research also provides some norms for types of characters chosen by children and for choice of background. Regardless of the psychometric properties, researchers may find the MAPS materials useful, as one or more scenes with preselected characters can be easily constructed to create a protective stimulus that is custom-made for the purpose of a study. Although infrequently used in the United States today compared to a few decades ago, the MAPS continues to be popular in some foreign countries, most likely because the unstructured material is free from language, and some cultural, barriers (Piotrowski et al., 1993). Finally, it should be noted that in the 1990s only three studies were found in a computer-based search of U.S. journals that used the MAPS in research with children.

THE TELL-ME-A-STORY TEST The Tell-Me-A-Story Test (TEMAS) was developed out of the need for a culturally sensitive thematic projective test (Costantino & Malgady, 1999; Costantino, Malgady, & Rogler, 1985). Most of the work on the test to date has been done under the auspices of the Hispanic Research Center of Fordham University and the Sunset Park Mental Health Center of Lutheran Medical Center in Brooklyn New York. A manual for the TEMAS (Costantino, Malgady, & Rogler, 1988) has been published. The test has two parallel sets, one for Hispanics and black children depicting minority characters and one for white children. An additional set using Asian American characters has been completed and is undergoing preliminary validation and other pilot studies (Costantino & Malgady, 2000). Each set of pictures consists of 23 chromatic drawings with contemporary inner-city settings. Scene contents include home and family,

reyn2-4.qxd

6/6/2003

9:03 AM

Page 75

4. Projective Storytelling Techniques

peer interactions, street experiences, school, and fantasy or daydreaming. Generally the cards present psychological dilemmas that are familiar to urban children and require some resolution. For example, one card draws on the conflict of whether to obey parents or play with friends. In another card, the main character must decide between putting money in a piggy bank and buying ice cream. Scenes may juxtapose antithetical material on an individual card, and part may be primarily positive (a group helping to repair a bicycle) and part primarily negative (children fighting). Twelve cards are used with both sexes, and 11 are sex-specific. Full administration of the 23 cards, which is conducted similarly to the TAT, may take 2 hours; a short form of 9 cards, taking about 45 minutes, is available. Standardization and other studies have been done on children from kindergarten through sixth grade. Normative data, for both the full form and the short form, are available in the manual on Hispanic, black, and white children up to age 14. The Hispanic norms are divided into Puerto Rican and non-Puerto Rican; the latter category includes Dominican Republic, South American, and Mexican American respondents. The theoretical basis of the TEMAS is less psychodynamic than that of most other thematic apperception tests; it is based on the developmental and personality theories of Piaget, Kohlberg, McClelland, Atkinson, and Bandura. The intent is not to draw out the unconscious as much as the inner verbal and imaginal processes (Costantino et al., 1985). The developers have attempted to present situations that will evoke one or more of the following nine categories of ego functioning (Costantino, 1982). The categories are scored on a 4-point scale, ranging from “very maladaptive” to “mature and responsible.” 앫 앫 앫 앫 앫 앫 앫 앫 앫

Interpersonal Relations Aggression Anxiety/Depression Achievement Delay of Gratification Self-Concept of Competence Sexual Identity Moral Judgment Reality Testing

75

In addition to being scored on the nine personality variables, the responses are scored on seven affective functions (e.g., happy, sad, and angry) and 18 cognitive functions (e.g., total time, number of words per story, and omissions). Face validity for the pull of each card to the nine categories is high (Malgady, 1982). Test–retest reliability is reported as low to moderate by the authors, but Lang (1992) describes it as “alarmingly low” and says that the TEMAS stories may be more situational than an assessment of underlying personality. Interrater reliability in scoring the TEMAS is reported as moderate to high, depending on the personality category being tested; there is also moderate support for concurrent validity and ability to predict treatment outcome (Malgady, Costantino, & Rogler, 1984). There is support for the ability of the TEMAS to discriminate between clinical and nonclinical samples of minority children (Costantino et al., 1985). An attempt to distinguish children with and without attention-deficit/hyperactivity disorder (ADHD) using the TEMAS suggested that the ADHD children used more omissions (Constatino, Colon-Malgady, Malgady, & Perez, 1991); however, more research is needed to evaluate whether any apperception technique would adequately discriminate this disorder. Furthermore, the authors point out that the nonminority TEMAS profile “affords gross differentiation of pathology from nonpathology, but thus far there has been little evidence of refined, differential diagnostic capability” (Constatino, Malgady, Colon-Malgady, & Bailey, 1992). Hispanic children tend to be more responsive, as measured by verbal output, in the TEMAS than on the TAT (Costantino, Malgady, & Vasquez, 1981). Interestingly, in comparing the output of Hispanic, black, and white elementary school children to the minority form, the nonminority form, and the TAT, researchers found that Hispanics and blacks were more verbally fluent on the TEMAS but not significantly more fluent on the ethnic form than on the nonminority form. The authors suggest that the richness of the TEMAS may be due as much to the themes and settings as to the ethnicity of characters (Costantino & Malgady, 1983). However, Cramer (1996) points out that

reyn2-4.qxd

76

6/6/2003

9:03 AM

Page 76

II. PROJECTIVE METHODS

larger number of characters in the TEMAS may have contributed to longer stories, and Telgasi (1993) considers the tasks and administrative instructions of the TAT and TEMAS to be different enough that they fail to warrant simple comparison. In many of the studies conducted in the development of the TEMAS, the children were tested by a bilingual examiner in their dominant language. Many of the children were more likely to respond in Spanish to the TEMAS than to the TAT (Costantino et al., 1981). We do not know whether a nonSpanish-speaking psychologist who is examining a bilingual Hispanic will obtain the same results in a nondominant language. As for generalization, most of subjects reported in the published supportive research came from inner-city settings and lower-socioeconomic-status families. In our review of the TEMAS in the first edition of this book, we expressed concern that the standardization and research populations of children tested did not include Hispanic children from the western United States or from rural settings. Since then, the authors have reported (Costantino & Malgady, 2000) unpublished research that compared the responses on the TEMAS and RATC of 40 12-year-old Mexican American and Anglo American children in Los Angeles. They concluded that the TEMAS is appropriate for West Coast Hispanics. They also reported comparisons of TEMAS responses made by 46 nonclinical elementary school children in Buenos, Argentina, with 44 similar children from Lima, Peru. Generally cognitive functions differed, with notably higher omissions with the Peruvians. The Argentineans expressed more happy affect, but there were no differences between groups on the nine personality functions. The authors apparently are struggling with an issue that most researchers with Hispanic populations must contend. The question is whether national subgroups can be reported in aggregate or whether separate norms and other results for research purposes need to be reported to capture differing social and economic characteristics as well as differing degrees of enculturation (see Weaver & Martinez, 2000). The TEMAS continues to be a promising projective tool for the assessment of urban schoolchildren, particularly those of ethnic minority. In their extensive review for use of

the TEMAS by school psychologists, Flanagan and Giuseppe (1999) consider the TEMAS an “improvement over other projective techniques in use with ethnic minority children” (p. 28), but the standardization sample is limited geographically and research about the TEMAS is primarily conducted by the authors. Lang (1992) recommends caution when using the TEMAS due to limited psychometric properties. Finally, Bellak and Abrams (1997) considered the TEMAS one of the better cultural adaptations of an apperceptive test, but young children of any background may best be given the CAT because the animal forms are “basically independent of cultural specificity” (p. 4). They further suggest that older chhildren of a differing culture may be unsuited for both the CAT and CAT-H; they “should be given a more culturally relevant thematic test, such as the Hispanic American TEMAS (p. 421). We encourage further research on the TEMAS to include a larger number of examinees in studying Hispanics from western states; a comparison of urban and rural Hispanic children; continued inclusion of African American, Asian American, and nonminority children; and a body of research conducted by investigators not associated with the founding institutions.

THEMES CONCERNING BLACKS Themes Concerning Blacks (TCB; Williams, 1972) is a thematic apperception test consisting of 20 black-and-white drawings, 10 of which include children or adolescents. Although the intended age range for the test is kindergarten through adulthood, most of the support data have been obtained from child populations, and most research studies have been conducted with children as subjects. Administration is similar to that for the TAT, with the additional availability of open-ended story questions to assist the more hesitant child. The TCB has been compared with the TAT; results suggest that black children are more attracted to the TCB, more likely to give a wider range of affect, and more likely to give stories with a positive tone and outcome (Dlepu & Kimbrough, 1982; Triplett & Brunson, 1982; Weaver, 1981). The user should understand that the content of the

reyn2-4.qxd

6/6/2003

9:03 AM

Page 77

4. Projective Storytelling Techniques

TCB is dfferent from that of the TAT. Weaver (1981) points out that the type of scenes may have influenced responses in that, compared to the TAT, the TCB contains more positive situations, includes more children of the same economic class and age as the respondents, is more culturally up-to-date and relevant, and offers more scenes of maternal nurturance. Because of the wider range of emotional responses on the TCB and the tendency of blacks to give sad or remorseful stories to the TAT, Williams, Williams, and Williams (1981) give little credence to pathological responses given by blacks on the TAT. They consider that the TCB provides a more accurate assessment, and they have more confidence in assessing true pathology when abnormal responses are given to the TCB, because the chance for false positives is less. Daum (1985) warns that the inner-city themes in the TCB may not be relevant to rural or middle-class blacks and calls for an update of the manual if more recent and broader data on reliability, validity, and norms are presently available. Although no one argues that cultural and experiential differences exist between the inner-city black respondent and the mainstream white respondent, Ness (1985) points out that there is no theoretical basis for a personality difference based on racial lines and calls for more experimental research if the TCB is to be applied as an assessment tool. We found no studies comparing white and black responses to the TCB. The psychologist who is assessing urban black children may find the TCB more helpful in evoking a wider range of themes and feelings than the TAT, and more helpful in some clinical decisions, but the user should bear in mind that the test construction as reported has psychometric shortcomings. Little published research has been conducted on the TCB since 1990. One study (White, Olivieira, Strube, & Meertens, 1995) did compare responses of black adults in the United States, Holland, and Surinam. They found 7 of the 20 cards to be specific to African Americans and used only the remaining 13 cards. Racespecific content was conspicuously absent and feeling tone was moderately sad. The authors call for restandardizing the norms due to historical effects and to social and

77

political effects specific to certain nationalities. Costantino and Malgady (2000) consider the TCB a culturally sensitive instrument but point out that it is not commercially distributed and that more research is needed regarding its reliablity and validity. Finally, it may be impossible to develop an instrument that covers such a wide age range of kindergarten to adult. Themes and life issues change with human development, regardless of race, and separate child and adult versions would be helpful if the TCB is going to continue to be a useful tool in the assessmenmt of either.

CHILDREN’S APPERCEPTIVE STORY-TELLING TEST One of the most recent story projective tests was developed by Mary Frances Schneider of the Adler School of Professional Psychology in Chicago titled the Children’s Apperceptive Story-Telling Test (CAST; Schneider, 1989). The test was designed in the tradition of the TAT and CAT but with the intent to provide and establish superior validity and reliability, improved norming, objective scoring, and a clear theoretical base (Schneider & Perney, 1990). The test is based on Adlerian theory that includes the childhood life tasks of family, peers, and school. Scenes are intended to present a situation that calls for some resolution but are of a neutral pull, allowing either positive or negative responses to each card. The final test consists of 17 cards, with two parallel sets of 14 cards for boys and girls and 3 cards to be used with both genders. The target ages are 6 to 13. The cards are chromatic and have a moderate ambiguity that emphasizes the Adlerian themes related to social functioning (Aronow, 1995). Administration is similar to the TAT and certain prompting is allowed as discussed in the manual. An attempt was made to ensure that the pictures were sensitive to race with the inclusion of culturally diverse figures and a cultural diversity in the normative sample. Schneider (personal communication, August 3, 2000) described the characters as having ambiguous racial features that have allowed children to view the characters as members of their own racial or ethinic category.

reyn2-4.qxd

78

6/6/2003

9:03 AM

Page 78

II. PROJECTIVE METHODS

A definitive scoring system is presented and a profile in T-scores addresses four main factors: adaptive, nonadaptive, immature, and uninvested. There is an emphasis on quality of thinking in the child’s ability to problem-solve. An adapative response creates a solution within a story. A nonadaptive response identifies the problem but offers no clear solution other than blame or attributing the problem to others. Immature and univested responses may include magical thinking, confusion about the scene, or refusal to report a story. In more detail, Schneider and Perney (1990) state that scoring consists of four adaptive thematic scales (Instrumentality, Interpersonal Cooperation, Affiliation, and Positive Affect), five nonadaptive thematic scales (Inadequacy, Alienation, Interpersonal Conflict, Limits, and Negative Affect), and six problem scoring scales (Positive Preoperational, Positive Operational, Refusal, Unresolved, Negative Preoperational, and Negative Operational). Administration of the test takes between 20 and 45 minutes and scoring another 30 minutes and a further 30 minutes for interpretation. Aronow (1995) opines that many clinical or school psychologists “may be unwilling to invest such time in an apperceptive technique . . . taking up more of the examiner’s time than the typical intelligence test” (p. 181). The test was standardized on a nationally representative sample of 876 children with an addition of 322 children identified as “behavioral disordered” (Schneider & Perney, 1990). In a review, Wiese (1995) considered the CAST to have a strong stability, due to five measures of reliability, and to have validity established as to content, construct, and criterion. It should be noted that despite the successful efforts to establish the psychometric properties lacking in so many other apperception tests, no research was found that expanded the utility or usability of the test in the past decade. Schneider (personal communication, August 3, 2000) told us that norming and restandardization are needed because the test is 10 years old as of this writing. She is unable to address those issues at this time and has directed the test to be taken off the market effective December 2000. She reported that any future work with the CAST may also include a simplified scoring system as

well as minor changes to the drawings to allow for more projection and more neutral pull.

ADOLESCENT APPERCEPTION CARDS The Adolescent Apperception Cards (AAC; Silverton, 1993) were designed primarily to assist in the clinical assessment of individual adolescents. There are two sets, each containing 11 stimulus cards. One set uses white children and one uses black children. Scenes are the same in both versions. Four of the cards have separate versions for males and females; the rest are appropriate for either gender. The scenes selected were intended to address such adolescent issues as interactions with parents, siblings, and peers, as well as abuse, neglect, depression, drug use, and domestic violence. The achromatic drawings are fine, pencil-shaded, and likely not to become as dated as other apperception cards. The facial expressions and interactions such as touch are ambiguous and should allow a client to produce themes with a wide range of content and affect. The author suggests that presentation of the cards be conducted in a “friendly, warm, and accepting manner” and considers it more of an interview than an administration of a formal test. Instructions to the client are similar to the TAT, requesting present, past, and future content as well as thoughts and feelings. Questioning and prompts are encouraged when appropriate. Total presentation should take about 1 hour. Silverton (1993) warns more than once in the manual that evaluation and interpretation of the stories should only be conducted by a skilled clinician trained in projective techniques. Several areas are described as important for consideration, including setting, themes, episodes within the story, resolution, affect, relationships, and general verbal method of presentation. The manual consists of five pages that provide a general overview, administrative details, interpretative suggestions, and other diagnostic considerations. No norms or other data of any kind are presented and the author seems to intend that the cards be used as a structured interview rather than an assessment tool that requires scoring and normative comparisons. A computerized

reyn2-4.qxd

6/6/2003

9:03 AM

Page 79

4. Projective Storytelling Techniques

search of the psychological literature found no mention of using the cards in research or in published clinical reports. A person working with adolescents may find the use of the cards, especially in the early diagnostic or assessment stages of therapy, helpful in avoiding the question–answer or crossexamination format that some teens find threatening or insensitive. The cards may help in self-disclosure, in avoiding resistance, and in establishing trust and rapport. Our only concern is that the drawings may be overshaded, giving a somewhat gloomy impression, and that none of the cards clearly depicts a teacher or classroom setting.

THE INTERNET: THE SELF-ADMINISTERED GLOBAL APPERCEPTION SCALES AND THE MULTIPLE-PERSON-ADMINISTERED CHILD TEST In an extensive search of the Internet we found two apperception tests that are intended to be used by clinicians for diagnostic and treatment decisions, as well as tools in general psychological assessment. One test is for adults and one is in the process of being developed for children. Both are the inspiration of Yanon Volcani of Psychological Testing Services International of LaJolla, California, and can be found at www. psytesting.com. A brief history of the development of the tests may be found at www.victoriapoint.com/tale_of_sagas.htm. The introduction to the adult test identifies it as the first apperception test to be administered, scored, and normed through the Internet. The test is called the Self-Administered Global Apperception Scales (the SAGAS). The author has mentioned that some more mature adolescents are attracted to the adult version more than the child version. The SAGAS presents 15 photographs intended to elicit a variety of psychological themes with variations related to age, race, and gender. Scoring is accomplished though a content analysis program. A sample report that would be sent to the clinician may be seen on the web. An individual’s responses are compared to others and reported in percentiles related to seven variables: Hostility, Social Alienation/Personal Disorganization,

79

Cognitive Impairment, Depression, Hope, and Health. A summary of clinical considerations is included. The full test also includes some self-report measures in which the client describes or rates personality traits or feelings seen as self-descriptive. The selfascribed characteristics may be used for research and validation purposes. The final report is e-mailed to the clinician within 24 hours for a $25 fee. The author clearly warns users that this test should be only one part of a “comprehensive integration of multiple data sources.” More pertinent to this book is the development by the author (Volcani, 2000a) of an apperception test intended for children ages 4 through 18 called the Parent Administered Child Test (PACT) or Multi-person Administered Child Test (M-PACT). The PACT is the title being used for parents during the norming process. The term “MPACT” will be used as the final name for the test, with the emphasis being on permitting multiple test administrators, including clinicians, parents, teachers, nurses, and others. According to Volcani (personal communication, July 11, 2000), the M-PACT is being developed to provide the profession with an instrument that can be administered more efficiently than the RATC or the TEMAS or many of the other tests that can take 1 to 3 hours to administer and sometimes even longer to score. The theoretical background is said to be based on established developmental principles. The PACT consists of seven achromatic drawings chosen to represent the “basic themes in the life of a child.” The seven progress chronologically from a wakingup-in-a-bedroom scene to encountering parental figures, going to school, arriving at school, a peer interaction play scene, a dining scene, and bedtime. The figures of the child superimposed on the scenes are first selected for a boy or girl version; then a selection is made for the characteristics of African American, Asian, Caucasian, Hispanic, or physically challenged (wheelchair). Volcani (personal communication, July 11, 2000) points out that the face of the targeted child is typically turned away to allow for maximum projection. Administration is a TAT-like format, requesting that the child make up a story to the picture that includes what the child is

reyn2-4.qxd

80

6/6/2003

9:03 AM

Page 80

II. PROJECTIVE METHODS

doing, thinking, and feeling, as well as what is going to happen next in the story. In the PACT, the parent and child sit in front of the computer and view the seven pictures. The parent may type in the responses to the pictures or older children may type in their own. At this writing parents are clearly notified that participation is to help in establishing norms. At the end of the seven pictures is a short questionnaire that asks for demographic data, a return e-mail address, and 15 items related to the child’s emotions and behaviors. The scoring system is in the process of being developed and is described by Volcani (personal communication, July 11, 2000) as an “amalgamation of the scoring system for the RATC and Children’s Apperceptive Story-Telling Test that is being developed into a computer program.” An extensive informed consent agreement precedes any participation. The final report to clinicians is expected to include 10 to 15 scales that are both adaptive (such as reliance on others and prosocial behavior) and clinical (such as depression and anxiety). The scales scores will be reported in percentiles, similar to the SAGAS. We applaud what appears to be a continually increasing set of norms. Each time a child responds, that response is included in an ongoing, additive normative data base. Variables will include not only age, gender, and race but a variety of subpopulations such as abused children or children who have experienced parental divorce. The drawings are simple and should not become easily dated, and a variety of children (e.g., gender and race) are available, but the scenes do represent middle America. For example, the “going to school” card has the child passing by a woods rather than a warehouse district. Also, the SAGAS are already attracting responses from international sources such as the West Indies and France. We expect the M-PACT will do the same. We wonder how a child in India might respond to the various scenes. The brevity of a 10- to 15-minute administration is welcomed, as is the lack of facial expressions. We would like to see future research that addresses whether the role of the test administrator (e.g., parent, clinician, and teacher) affects fantasy output. We are concerned that despite the ethical guidelines ex-

pressed by the author, the test may be misused or misunderstood by persons untrained in assessment in general and in projective tests in particular. Finally, we share the author’s concern of establishing and maintaining enough electronic security to ensure confidentiality in both sending stories and receiving reports. A wide variety of tests related to psychological and social issues are available to the average web browser, most of them the kinds of questionnaires available through popular journals. Controversy exists as to whether it is ethical, and in some cases legal, to provide psychological services on line. It will be interesting to see if a serious assessment tool can survive and become established in a professional and ethical manner on the Internet.

THE PICTURE-STORY TEST The Picture-Story Test was developed from a study conducted by Percival Symonds (1949) on adolescent imagination and fantasy. Influenced by the TAT, Symonds established his own set of 42 cards, which included adolescent characters and themes that might elicit adolescent concerns. The drawings were done in black-and-white crayon. His report of that study in Adolescent Fantasy (Symonds, 1949) included norms, thematic data, and an exhaustive analysis of the responses of 20 male and 20 female high school students in New York City. This book includes a good review of the literature, discussion of apperception techniques, and more than 100 sample stories. It has historical significance for the interested reader. From the original drawings, 20 cards were chosen (two sets of 10 each) as the final test (Symonds, 1948). The cards are not segregated for gender. In administration, the child is invited to imagine being a story writer for a magazine that would use the picture as an illustration. The examiner stresses that the study can be silly or fantastic and that it will be confidential. Card content includes family and peer interaction, adult interactions, street scenes, and a hint at school themes with the presence of books in a few of the cards. Facial expressions, however, appear grim or morose, and the shadowy nature of the cards is

reyn2-4.qxd

6/6/2003

9:03 AM

Page 81

4. Projective Storytelling Techniques

likely to evoke stories of gloom, depression, or aggression. Clothing is dated, with some of the adolescent characters in suit and tie, and the characters would probably appear more like college students than teenagers to the modern adolescent. Because of limited normative data and limited reliability and validity studies, the test is not recommended for current use. No research was reported in the published psychological literature since the first edition of this handbook. However, this test may prove to be a productive historical background for anyone attempting to create an up-to-date apperception test for adolescents.

THE TASKS OF EMOTIONAL DEVELOPMENT TEST The Tasks of Emotional Development Test (TED; Cohen & Weil, 1975a, 1975b) consists of 12 photographs for children ages 6–11 and 13 photographs for adolescents ages 12–18. Separate sets are presented to males and females, with an additional card eliciting heterosexual adjustment themes for adolescents. The four sets have nearly identical content, varying only by age group and gender. Photos are somewhat outdated and mainly depict white, middle-class persons from the early 1960s. The TAT-style administration includes an explantion of what is meant by “feelings.” Each card represents one of 13 developmental tasks purported by the authors to be common across child personality development. Tasks include separation issues, socialization with peers, and the establishment of positive attitides toward school and parents. The theorectical background for the test is based on the ego-oriented, psychoanalytic thinking of Erik Erikson (1950) and George Gardner (1959); the cards, however, seem amenable to any orientation. The tasks depicted are clear, structured, and concerned with everyday events such as watching television or joining a play group. Kohlbert (cited in Cohen & Weil, 1975a) applauds the pictures as allowing the children to project their needs and motives without getting sidetracked in the cognitive complexity of ambiguous stimuli. The authors claim that the lack of ambiguity is

81

more apt to bring out responses pertinent to the real lives of children and their actual, rather than fantasized, emotions. They add that responses may be shorter than those given to the TAT. The test is intended by the authors to be scored on five dimensions: Perception (the ability to accurately see what the story is supposed to be about), Outcome (the level of maturity implied by the developmental “solution” to the presented task), Affect (an indicator of the appropriateness of the feelings as related to the outcome; a successful outcome should have positive feelings and an unsuccessful outcome should have negative feelings), Motivation (the maturity of reasoning associated with the outcome), and Spontaneity (related to the comfort and completeness with which a respondent tells the story). Norms are given for boys and girls from a wide variety of socioeconomic backgrounds who were either “normal” or referred to a psychological clinic. Unfortunately, norms are given for only 6 of the 13 developmental tasks. No adolescents were included in the norming sample, and the norm statistics are not broken down by age. Levitt (1975) considers the TED to be a poorly developed test with questionable validity. The validity of the TED has also been questioned (Ammons & Ammons, 1972), and the rating scale has been described as “tedious” and “cumbersome” in terms of the test’s potential use by dynamically oriented clinicians (Wise, 1975). The authors report good interrator reliability for scoring but have not fully established reliabiltiy for the test itself (Cohen & Weil, 1975a). Gotts (1974) is enthusiastic about its use with all children and claims to have received good clinical material from children with mental ages of 5 and above. Since 1970, the TED has been used in research to study abused children (Kinard, 1980, 1982), obese adolescents (Karpowitz & Zeis, 1975), battered adult women (Koslof, 1984), and adults with mental retardation (Wagner, 1991). More recent use of the TED in published research has not been found and the test appears to being losing what little popularity it had. A survey of some research applications can be found in Pollack, Cohen, and Weil (1981, 1982). The strengths of the TED are in its theoretical base, in the lack of picture ambiguity

reyn2-4.qxd

82

6/6/2003

9:03 AM

Page 82

II. PROJECTIVE METHODS

for some examiners, and in an objective scoring system. The weaknesses of the test may be in the lack of data to support validity or test reliability, in photos that are outdated and middle class, and in limited norms on only half the test. Anyone intending to use this test for research or for extended clinical use is encouraged to regard the book by Cohen and Weil (1975a) as an extension of the manual. This book provides seven full cases and the use of the TED with other tests. Many sample stories are given, demonstrating a variety of scoring issues, but again only for the first six tasks.

THE MICHIGAN PICTURE TEST—REVISED The Michigan Picture Test—Revised (MPTR; Hutt, 1980, 1986) was first developed through the Michigan State Department of Mental Health in 1953. A primary goal was to develop an assessment technique to differentiate adjusted from maladjusted children that would offer quantified approaches for clinicans rather than simple reliance on clinical intuition and skill of interpretation. This purpose is laudable, and the efforts are extensive, but the final product as presented in this revised edition is of limited success. The present test consists of a total of 15 cards: 7 general cards, 4 cards for girls, and 4 cards for boys. A “full series” administration consists of 11 cards; a briefer “core series” version uses only four of the general cards when the purpose of the test is for general screening. The content of the four core cards is as follows: (1) a breakfast scene, (2) children playing checkers, (3), a lightning scene without human characters, and (4) a blank card. Administration is similar to that for the TAT: requesting a story that includes what is happening and how it will turn out. Although the stories are not timed, inordinate reaction or story length is to be noted. Electronic recording is encouraged. Suggested ages are 8 to 14. Four indices are presented in the manual that purport to discriminate between adjusted and maladjusted children. A “Tension Index” is based on the frequency of four needs expressed. A “Tense” variable relates to the percentage of past- versus present-tense verbs used in the stories. A “Direction of

Forces” score tabulates and weighs references to action in the story, noting whether the main character is acting or being acted upon or whether there is an absence of action. The “Combined Maladjustment Score” is a checklist of the number of times scores on the first three variables are at or above the critical scores or norms provided in the manual. The responsible test user will note many of the following observations before deciding on whether to use the MPT-R. The standardization sample remains that of children in the early 1950s, with a limited description of population demographics. The “maladjusted” children were identified as such because they either scored in the lower third of a teacher rating scale or were undergoing treatment at child guidance clinics; 90% of the clinic group were boys. The samples were third-, fifth-, seventh-, and ninth-grade children. The manual has grouped the seventh- and ninth-grade data, without any statistical or theoretical explanation that we could find. Despite the sufficient number of subjects (n = 700) used in developing the test over a series of studies, some of the tables from which one must assess maladjustment have a limited number of subjects per group. The norms for discrimination are based on the four “core” cards for only two of the scorable indices. Two additional cards must be presented to assess the Direction of Forces Index. We found no clearly demonstrated reason for using the four cards as the core series. The use of the critical scores on the Tension Index can give false positives at a hit rate worse than chance (55%). The Tense variable only discriminated between the fifth graders and seventh/ninth graders in the standardization sample, and research (Hartwell, Hutt, Andrew, &, Walton, 1951) casts doubt as to how well it does at the fifth-grade level. The Combined Maladjustment Score has no statistical support, as admitted by the author, other than correlational data between the other three variables. Bauserman (1985) questions whether the MPT-R has demonstrated any “clear or significant” gain over the TAT and CAT. Bischoff (1985) complains that the cards are dated, that ethnic variability is lacking, that good clinical judgments in assessing responses to these cards may be more accu-

reyn2-4.qxd

6/6/2003

9:03 AM

Page 83

4. Projective Storytelling Techniques

rate than using the limited data in the manual, and that the revisions in the MPF-R appear more “cosmetic” than substantial. In a data-based search of the literature in the past decade, no mention of the MPT-R was noted. The test user who assumes that he or she can objectively discriminate between maladjusted and adjusted children with only four cards may be greatly overestimating the assessment qualities of the MPT-R. However, anyone attempting to replicate the praiseworthy goal of the authors might do well to review the history of the development of this test.

THE SCHOOL APPERCEPTION METHOD The School Apperception Method (SAM; Solomon & Starr, 1968) was developed to provide a projective thematic test to assess a child’s emotional and academic adjustment to school. Prior to the introduction of the SAM, tests for children failed to adequately address the milieu in which children spend most of their time and in which children may gain or lose in the areas of self-concept, self-esteem, and social relations outside the home. The 22 cards are clear black-and-white drawings, with degrees of shading reminiscent of water colors. Backgrounds are vague and nonintrusive; the characters (dress and hair) are depicted in such a way that the test is not as likely to appear dated as are photographs. A variety of school-oriented situations are presented. There are 12 main or standard cards depicting children, often with adult figures (teacher, principal), in active situations that seem to call for some kind of resolution. For example, in one card a boy is reading while two pupils off to the side appear to be whispering about him. In another card, one boy is holding back another while the teacher talks to a third boy. In addition to the 12 main cards, 10 additional cards are provided, 5 of which include black students or black adults. The additional cards may be substituted when the examinee is black or when the school is racially integrated. The remaining 5 cards depict situations that are less common but may need to be explored, depending on the purpose of the testing.

83

The intended main characters are children in the middle grades of elementary school and are predominantly boys. The test can be used with children in elementary or junior high school. Administration is similar to that for the TAT, and inquiry is encouraged following the primary response if the test is used for assessment purposes. Analysis may be accomplished through several approaches, but the authors also provide nine categories suggested for assessment: 앫 Formal Qualities (general manner and length of stories) 앫 Attitudes toward Teachers and Other Authorities 앫 Attitudes toward Schoolmates 앫 Attitudes toward Academic Activity 앫 Aggression 앫 Frustration 앫 Anxiety and Defense Mechanisms 앫 Home and School 앫 Punishment Sample responses are included in the manual, but without norms. The authors suggest that the SAM responses may be used to communicate more easily with school personnel in explaining a child’s problems and in supporting recommendations. The SAM should be used with other assessment measures, interviews, and observations and not alone as a basis for decision making. An elementary school child may have quite different sets of feelings, attitudes, and self-concepts, depending on whether the child is considering the home or school environment. Many authors suggest using at least some SAM cards in a full child assessment to make up for the lack of schooloriented cards in other projective tests (Koppitz, 1982; Peterson, Kroeker, & Torshen, 1976). Rarely has the SAM been used in research and no mention of it has been made in a literature review of the past decade.

VARIATIONS ON THEMATIC APPERCEPTION TECHNIQUES Full story responses are sometimes difficult to obtain from younger children. The psychologist who wishes to use projective tech-

reyn2-4.qxd

84

6/6/2003

9:03 AM

Page 84

II. PROJECTIVE METHODS

niques with children 5–8 years old has an alternative with story completion methods. In such methods, a brief story or scene is presented and the child is asked either to complete the story or to respond to a question at the end of the story. Norms, reliability studies, and validity studies are generally unavailable, and the tester must rely on clinical skills alone in evaluating the responses. These methods take about 10–20 minutes, and three are briefly described herein. The Despert Fables (see Despert, 1946) were originally developed by Louisa Duss of Switzerland. Ten short stories, some with animal characters, are intended to elicit psychoanalytic themes. For example, in one story a mother, a father, and a baby bird are asleep in their nest. A wind knocks the nest to the ground. The two parents fly to separate trees, and the child is asked what the little bird is going to do. Fine (1948) expanded the number of stories to 20 and categorized them according to such variables as dependency, hostility, identification, sibling rivalry, Oedipal issues, and fears. The Madeleine Thomas Completion Stories (Mills, 1953) are 15 quite brief scenarios that create suspenseful situations and are followed by a direct question. For example, in one story the child is simply asked to tell what a mother will do when she finds two brothers fighting. Mills (1953) introduced these stories (originally written in French; Helmut Wursten, translator) to the American psychological community. He offers a way to organize a clinical analysis, and lists recurring themes common to particular stories. As part of a larger study in the 1950s, Elizabeth Koppitz (1982) developed what she calls the Munsterburg Incomplete Stories. There are 12 incomplete stories, to be varied by gender-identifiable names (e.g., “Sammy [Sandra] wakes . . .”), each setting up a scene that the child is asked to finish. The author suggests an analysis that includes positive and negative feelings or attitudes of the child toward self and others, positive and negative attitudes of others toward the child, coping mechanisms, and the quality of language. These three methods are typically found within the archives of testing labs of universities and clinics. Other thematic apperception methods have been designed to explore particular ar-

eas of development or concern rather than to assess general personality. The Adolescent Separation Anxiety Test (Hansburg, 1980) was developed, starting in the mid–1960s, to assess the intensity of feelings and patterns of responses to experiences of separation, individuation, and attachment in children and adolescents. It is not, strictly speaking, a thematic test. The test consists of 12 black-and-white drawings of mild (going to camp) to strong (death of parent) separation scenarios. Two parallel forms are available, differing only in the genders of the children in the pictures. For each picture, the child is asked to choose one or more of 17 written statements as to how the child in the picture is feeling. The statements are intended to reflect reactions such as withdrawal, anxiety, somatization, adaptation, or denial. Norms are presented for ages 11–14, although the intended use is for ages 11–18. Use of this test, in its present stage of development, for diagnostic or child placement decisions is not recommended by some reviewers (Bailey-Richardson, 1985; Hartsough, 1985). The Blacky Pictures (Blum, 1950) are a series of cartoon drawings of dogs, wherein Blacky is the main character, appearing at times with a mother, a father, and a sibling (Tippy). Administration is comparable to that for the TAT. The content is specifically designed to elicit themes directly related to the psychoanalytic theory of psychosexual development such as oral eroticism, anal sadism, and castration anxiety. These cards are generally out of favor because of advances in formulations of personality development. For a recent and comprehensive review we refer interested readers to Sappenfield (1994). Some methods are apperceptive, in that they ask for a projective response to visual stimuli, but are not truly thematic, in that only brief responses are required or responses are based on the selection of one or more provided responses. An example of the former is the Rosenzweig Picture-Frustration Test (Rosenzweig, 1977, 1978a, 1978b). Much of the work with this test has been done with the adult form, but a child’s form is also available. The child is presented with 24 comic-strip-like scenes, with one person saying something in the “balloon” that might cause frustration to the other

reyn2-4.qxd

6/6/2003

9:03 AM

Page 85

4. Projective Storytelling Techniques

character in the picture. The child is to write as quickly as possible a response in an empty “balloon” that sits over the other character. The test may be read to younger children. Rosenzweig (1988) continues to update norms and Graybill (1993; Graybill & Heuvelman, 1993) continues to assess validity. The reader who intends to explore issues of hostilty and aggression with children may find this test worthy of consideration. Two interesting variations of apperception methods simply show a hand or pair of hands; both have child forms. The Hand Test (Wagner, 1983) is used to explore such areas as aggression and withdrawal. The child is presented with a series of drawings of hands in different positions and is asked to describe what the hands might be doing. Because of its simplicity and quick administration it continues to be popular as a measure in research with chilren and in the schools. (See Sivec & Hilsenroth, 1994, for an excellent overview that focuses on the use of the Hand Test within the school system.) A similar test is the Paired Hands Test (Zucker & Barnett, 1977; Zucker & Jordan, 1968), which is a series of drawings showing a black hand and a white hand interacting. It is intended to be suited to explore attitudes toward, or expectations of, others. It is beyond the scope of this chapter to mention the abounding variations available in the literature. Many techniques developed in Great Britain, especially through the Tavistock Institute, are not well known in this country and can be reviewed in Projective Techniques (Semeonoff, 1976).

CONCLUSIONS When one is reviewing the numerous thematic storytelling techniques currently available, several issues become apparent. Foremost is the ongoing debate as to whether storytelling techniques should be evaluated as psychometric instruments or as a means of exploring personality dynamics. One should bear in mind that the TAT was originally developed as an aid to therapy, with no intention that it be used to compare individuals or lead to a diagnosis. Continuing in this tradition, many therapy-oriented

85

clinicians and practitioners defend storytelling techniques as emphasizing the uniqueness of individuals. Although the test is still used to compare individuals, clinicians do so by means of their clinical intuition, which is often based on their particular experience with a wide range of individuals and personality types. Although this usage can often be quite accurate and helpful, and even appears wonderously convincing to students learning the technique, it unfortunately cannot be taught in any systematic fashion. In addition, the TAT is now used almost exclusively for purposes of assessment. Thus, it is only fair to assume that current evaluations of the TAT and similar instruments will focus on psychometric properties of the tests. Of the psychometric problems mentioned in this chapter, two seem to stand out. First, most storytelling techniques have normative information that is at worst nonexistent and at best based on inadequate samples. Second, in terms of validity, the major problem appears to be with content interpretations: Do events portrayed in stories relate to the subject’s actual behaviors, repressed personality characteristics, or mere fantasies? These and similar difficulties must be addressed before storytelling techniques can be considered psychometric instruments. As should be evident from our review, several older techniques are still enjoying widespread popularity. Any number of reviews concur that many instruments are psychometrically untenable, should not be used for placement decisions or diagnosis, and should never be used except as part of an assessment battery. Alternately, there has been renewed interest in research on the TAT. Several scoring systems have undergone empirical evaluations in the last decade, with the SCORS producing some promising results. An additional highlight of this review is the observation that some older tests and all the more newly developed tests are making attempts to respond to the needs of children from diverse cultural and ethnic backgrounds. The stimulus cards and the standardizations both try to address a more inclusive range of children. Although we praise that endeavor, we are also concerned that the evaluation of these techniques are typically conducted by the authors of the tests. Almost no independent

reyn2-4.qxd

6/6/2003

86

9:03 AM

Page 86

II. PROJECTIVE METHODS

researchers seem motivated to assess their psychometric properties, and rarely are they used in research other than by persons connected in some way to the authors. Finally, we are curious to see what influence both computer-assisted administration and scoring and the international use of the Internet may have on projective storytelling techniques. We wonder if this assessment tool is giving way to more objective, behaviorally oriented approaches or whether there will be some renovation or reinvention that will help apperception techniques meet the psychometric demands of the psychologists and educators of the 21st century.

REFERENCES Aaron, N. S. (1967). Some personality differences between asthmatics, allergic and normal children. Journal of Clinical Psychology, 23, 336–340. Ackerman, S. J., Clemence, A. J., Weatherill, R., & Hilsenroth, M. J. (1999). Use of the TAT in the assessment of DSM-IV cluster B personality disorders. Journal of Personality Assessment, 73, 422–448. Altman, C. H. (1960). Projective techniques in the clinical setting. In A. I. Rabin & M. Haworth (Eds.), Projective techniques with children (pp. 329–349). New York: Grune & Stratton. Ammons, C. H., & Ammons, R. C. (1972). Review of the Tasks of Emotional Development Test. Psychological Reports, 31, 679. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th. ed.). Upper Saddle River, NJ: Prentice-Hall. Aronow, E. (1995). Review of the Children’s Apperceptive Story-Telling Test. In J. C. Conoley & J. C. Impara (Eds.), The twelfth mental measurements yearbook (pp. 180–181). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Aronow, E., Reznikoff, M., & Rauchway, A. (1979). Some old and new directions in Rorschach testing. Journal of Personality Assessment, 43(3), 227–234. Bailey-Richardson, B. (1985). Review of Separation Anxiety Test. In J. V. Mitchell, Jr. (Ed.), The ninth mental measurements yearbook (pp. 53–54). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Baty, M. A., & Dreger, R. M. (1975). A comparison of three methods of record TAT protocols. Journal of Clinical Psychology, 31, 348. Bauserman. D. N. (1985). Review of the Michigan Picture Test—Revised. In J. V. Mitchell, Jr. (Ed.), The ninth mental measurements yearbook (pp. 970–971). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Bell, N. L., & Nagle, R. J. (1999). Interpretive issues with the Roberts Apperception Test for Children: Limitations of the standardization group. Psychology in the Schools, 36, 277–283. Bellak, L. (1947). A guide to the interpretation of the

Thematic Apperception Test. New York: Psychological Corporation. Bellak, L. (1971). The TAT and CAT in clinical use. New York: Grune & Stratton. Bellak, L., & Abrams, D. M. (1997). The Thematic Apperception Test, the Children’s Apperception Test, and the Senior Apperception Technique in clinical use (6th ed.). Needham Heights, MA: Allyn & Bacon. Bellak, L., & Adelman, C. (1960). The Children’s Apperception Test. In A. I. Rabin & M. Haworth (Eds.), Projective techniques with children (pp. 62–94). New York: Grune & Stratton. Bellak, L., & Bellak, S. S. (1949). The Children’s Apperception Test. New York: CPS. Bellak, L., & Bellak, S. S. (1961). Children’s Apperception Test (C.A.T.) manual (4th ed.). Larchmont, NY: CPS. Bellak, L., & Bellak, S. S. (1991). Supplement to the Children’s Apperception Test (CAT-S) (rev. ed.). Larchmont, NY: CPS. Bischoff, B. G. (1985). Review of the Michigan Picture Test. In J. V. Mitchell, Jr. (Ed.), The ninth mental measurements yearbook (pp. 971–972). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Blatt, S. J. (1978). Review of E. Aronow and M. Reznikoff, Rorschach content interpretation. Contemporary Psychology, 23, 251–253. Blum, G. S. (1950). The Blacky Pictures: Manual of Instructions. New York: Psychological Corporation. Cleveland, S. E. (1976). Reflections on the rise and fall of psychodiagnosis. Professional Psychology, 7, 309–318. Cohen, H., & Weil, G. R. (1975a). Tasks of Emotional Development: A projective test for children and adolescents. Brookline, MA: Tasks of Emotional Development Associates. Cohen, H., & Weil, G. R. (1975b). Tasks of Emotional Development test manual. Brookline. MA: Tasks of Emotional Development Associates. Conklin, A., & Westin, D. (2001). Thematic Apperception Test. In W. Dorfman & M. Hersen (Eds.), Understanding psychological assessment (pp. 107– 134). New York: Kluwer/Plenum Press. Costantino, G. (1982). TEMAS: A new technique for personality assessment and psychotherapy for Hispanic children. Research Bulletin, Hispanic Research Center, 5, 3–6. Costantino, G., Colon-Malgady, G., Malgady, R. G., & Perez, A. (1991). Assessment of attention deficit disorder using a thematic apperception technique. Journal of Personality Assessment, 57(1), 87–95. Costantino, G., & Malgady, R. G. (1983). Verbal fluency of Hispanic, black and white children on TAT and TEMAS, a new thematic apperception test. Hispanic Journal of Behavioral Sciences, 5, l99–206. Costantino, G., & Malgady, R. G. (1999). The TellMe-A-Story test: A multicultural offspring of the Thematic Apperception Test. In L. Gieser & M. I. Stein (Eds.), Evocative images: The thematic apperception test and the art of projection (pp. 191–206). Washington, DC: American Psychological Association.

reyn2-4.qxd

6/6/2003

9:03 AM

Page 87

4. Projective Storytelling Techniques Costantino, G., & Malgady, R. G. (2000). Multicultural and cross-cultural utility of the TEMAS (Tell-MeA-Story) test. In R. H. Dana (Ed.), Handbook of cross-cultural and multicultural personality assessment (pp. 481–513). Mahwah, NJ: Erlbaum. Costantino, G., Malgady, R. G., Colon-Malgady, G., & Bailey, J. (1992). Clinical utility of the TEMAS with nonminority children. Journal of Personality Assessment, 59(3), 433–438. Costantino, G., Malgady, R. G., & Rogler, L. H. (1985, August). Cross-cultural validation of TEMAS: a minority projective test. Paper presented at the annual meeting of the American Psychological Association, Los Angeles, CA. Costantino, G., Malgady, R. G., & Rogler, L. H. (1988). Tell-Me-A-Story (TEMAS): Manual. Los Angeles, CA: Western Psychological Services. Costantino, G., Malgady, R. G., & Vazquez, C. (1981). A comparison of the Murray–TAT and a new thematic apperception test for urban Hispanic children. Hispanic Journal of Behavioral Sciences, 3, 291–300. Cramer, P. (1991). The development of defense mechanisms: Theory, research, and assessment. New York: Singer. Cramer, P. (1996). Storytelling, narrative, and the Thematic Apperception Test. New York: Guilford Press. Cramer, P. (1999). Stories are telling. Society for Personality Assessment (SPA) Exchange, 8(2), 5–7. Dana, R. H. (1985). Thematic Apperception Test. In C. S. Newmark (Ed.), Major psychological assessment instruments (pp. 89–134). Boston: Allyn & Bacon. Dana, R. H. (1986). Thematic Apperception Test used with adolescents. In A. I. Rabin (Ed.), Projective techniques for adolescents and children (pp. 14–36). New York: Springer. Daum, J. M. (1985). Review of Themes Concerning Blacks. In J. V. Mitchell, Jr. (Ed.), The ninth mental measurements yearbook (p. 1617). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Davids, A., & Rosenblatt, D. (1958). Use of the TAT in assessment of alienation. Journal of Projective Techniques, 22, 145–152. Dean, R. S. (1984). Commentary on “Personality assessment in the schools: The special issue.” School Psychology Review, 13(l), 95–98. Despert, J. L. (1946). Psychosomatic study of fifty stuttering children. American Journal of Orthopsychiatry, 16, 100–113. Dlepu, O., & Kimbrough, C. (1982). Feeling-tone and card preferences of black elementary children for the TCB and TAT. Journal of Non-White Concerns, 10, 50–56. Erikson, E. H. (1950). Childhood and society. New York: Norton. Finch, A. J., & Belter, R. W. (1993). Projective techniques. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child and adolescent assessment (pp. 224–238). Boston: Allyn & Bacon. Fine, R. (1948). Use of the Despert Fables (revised form) in diagnostic work with children. Rorschach Research Exchange and Journal of Projective Techniques, 12, 106–118.

87

Fine, R. (1955). A scoring scheme for the TAT and other verbal projective techniques. Journal of Projective Techniques, 19, 306–309. Flanagan, R., & Giuseppe, R. D. (1999). Critical review of the TEMAS: A step within the development of thematic apperception instruments. Psychology in the Schools, 36(1), 21–30. Freedenfeld, R. N., Ornduff, S. R., & Kelsey, R. M. (1995). Object relations and physical abuse: A TAT Analysis. Journal of Personality Assessment, 64, 552–568. Friedrich, W. N., & Share, M. (1997). The Roberts Apperception Test for Children: An exploratory study of its used with sexually abused children. Journal of Child Sexual Abuse, 6, 83–91. Gardner, C. E. (1959). Psychiatric problems of adolescence. In S. Arieti (Ed.), American handbook of psychiatry (Vol. 1, pp. 870–892). New York: Basic Books. Gerver, J. M. (1946). Level of interpretation of children on the Thematic Apperception Test. Unpublished master’s thesis, Ohio State University, Columbus. Goldman, J., Stein, C., & Guerry, S. (1983). Psychological methods of child assessment. New York: Brunner/Mazel. Gotts, E. (1974). Reviews of school psychology materials. Journal of School Psychology, 12, 84–85. Graybill, D. (1993). A longitudinal study of changes in children’s thought content in response to frustration on the children’s Picture-Frustration Study. Journal of Personality Assessment, 61(3), 531–535. Graybill, D., & Heuvelman, L. R. (1993). Validity of the children’s Picture-Frustration Study: A socialcognitive perspective. Journal of Personality Assessment, 60(2), 379–389. Hafner, A. J., & Kaplan, A. M. (1960). Hostility content analysis of the Rorschach and TAT. Journal of Projective Techniques, 24, 137–143. Hansburg, H. C. (1980). Adolescent separation anxiety (2 vols.). Melbourne, FL: R. E. Krieger. Hartsough, C. S. (1985). Review of Adolescent Separation Anxiety Test. In J. V. Mitchell, Jr. (Ed.), The ninth mental measurements yearbook (pp. 54–55). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Hartwell, S. W., Hutt, M. L., Andrew, G., & Walton, R. E. (1951). The Michigan Picture Test: Diagnostic and therapeutic possibilities of a new projective test in child guidance. American Journal of Orthopsychiatry, 21, 124–137. Haworth, M. R. (1986). Children’s Apperception Test. In A. I. Rabin (Ed.), Projective techniques for adolescents and children (pp. 37–72). New York: Springer. Heuvelman, L. R., & Graybill, D. (1990). Assessment of children’s fantasies with the Make A Picture Story: Validity and norms. Journal of Personality Assessment, 55(3&4), 578–592. Hibbard, S., Hilsenroth, M. J., Hibbard, J. K., & Nash, M. R. (1995). A validity study of two projective representation measures. Psychological Assessment, 7, 332–339. Holmen, M. G., & Docter, R. (1972). Educational and

reyn2-4.qxd

88

6/6/2003

9:03 AM

Page 88

II. PROJECTIVE METHODS

psychological testing. New York: Russell Sage Foundation. Hutt, M. L. (1980). The Michigan Picture Test— Revised. New York: Grune & Stratton. Hutt, M. L. (1986). The Michigan Picture Test— Revised. In A. I. Rabin (Ed.), Projective techniques for adolescents and children (pp. 73–84). New York: Springer. Impara, J. C., & Murphy, L. L. (Eds.). Psychological assessment in the schools. Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Jensen, A. R. (1965). Review of the Make-A-Picture Story. In O. K. Buros (Ed.), The sixth mental measurements yearbook (pp. 468–470). Highland Park, NJ: Gryphon Press. Joiner, T. E. (1996). The relations of thematic and nonthematic childhood depression measures to defensiveness and gender. Journal of Abnormal Child Psychology, 24, 803–813. Joiner, T. E., Schmidt, K. L., & Barnett, J. (1996). Size, detail, and line heaviness in children’s drawings as correlates of emotional distress: (More) negative evidence. Journal of Personality Assessment, 67, 127–141. Kagan, J. (1960). Thematic apperceptive techniques with children. In A. Rabin & M. Haworth (Eds.), Projective techniques with children (pp. 105–129). New York: Grune & Stratton. Kamphaus, R. W., & Frick, P. J. (1996). Clinical assessment of child and adolescent personality and behavior. Boston: Allyn & Bacon. Karon, B. P. (1981). The Thematic Apperception Test. In A. I. Rabin (Ed.), Assessment with projective techniques (pp. 85–120). New York. Springer. Karpowitz, D. H., & Zeis, F. R. (1975). Personality and behavior differences of obese and nonobese adolescents. Journal of Consulting and Clinical Psychology, 43, 886–891. Kinard, E. M. (1980). Emotional development in physically abused children. American Journal of Orthopsychiatry, 50, 686–696. Kinard, E. M. (1982). Experiencing child abuse: Effects on emotional adjustment. American Journal of Orthopsychiatry, 52, 82–91. Knoff, H. (1998). Review of the CAT. In J. C. Impara & B. S. Plake (Eds.), The thirteenth mental measurements yearbook. Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Koppitz, E. M. (1982). Personality assessment in the schools. In C. Revnolds & T. Gutkin (Eds.), The handbook of school psychology (pp. 273–295). New York: Wiley. Koslof, K. E. (1984). The battered woman: A developmental perspective. Smith College Studies in Social Work, 54, 181–203. Kroon, N., Goudena, P. P., & Rispens, J. (1998). Thematic Apperception tests for child and adolescent assessment: A practitioner’s consumer guide. Journal of Psychoeducational Assessment, 16, 99–117. Lang, W. S. (1992). Review of the TEMAS. In J. J. Kramer & J. C. Conoley (Eds.), The eleventh mental measurements yearbook (pp. 925–926). Lincoln: Buros Institute of Mental Measurements, University of Nebraska.

Lanyon, R. I. (1984). Personality assessment. Annual Review of Psychology, 35, 667–701. Lanyon, R. I., & Goodstein, L. (1982). Personality assessment (2nd ed.). New York: Wiley. Levitt, E. E. (1975). A methodological review. Professional Psychology, 6, 101–102. Locraft, C., & Teglasi, H. (1997). Teacher rated empathic behaviors and children’s TAT stories. Journal of School Psychology, 35, 217–237. Malgady, R. C. (1982). Reliability and validity of TEMAS: A new thematic apperception test for urban ethnic minority children. Research Bulletin, Hispanic Research Center, 5, 10–13. Malgady, R. G., Costantino, C., & Rogler, L. H. (1984). Development of a thematic apperception test (TEMAS) for urban Hispanic children. Journal of Consulting and Clinical Psychology, 52, 986–996. McArthur, D. S., & Roberts, C. E. (1982). Roberts Apperception Test for Children: Manual. Los Angeles, CA: Western Psychological Services. McClelland, D. C., Atkinson, J. W., Clark, R. A., & Lowell, E. L. (1953). The achievement motive. New York: Appleton-Century-Crofts. McGrew, M. W., & Teglasi, H. (1990). Formal characteristics of Thematic Apperception Test stories as indices of emotional disturbance in children. Journal of Personality Assessment, 54, 639–655. Mills, E. S. (1953). The Madeline Thomas Completion Stories Test. Journal of Consulting Psychology, 17, 139–141. Morgan, C. D., & Murray, H. A. (1935). A method for investigating phantasies: The Thematic Apperception Test. Archives of Neurology and Psychiatry, 34, 289–306. Munter, P. 0. (1975). The medical model revisited: A humanistic reply. Journal of Personality Assessment, 39, 344. Murphy, L. L., Impara, J. C., & Plake, B. S. (Eds.). (1999). Tests in print: An index to tests, test reviews, and the literature on specific tests. Lincoln: Buros Institute of Mental Measurements, University if Nebraska. Murray, H. A. (1938). Explorations of personality. New York: Oxford University Press. Murray, H. A. (1943). Thematic Apperception Test manual. Cambridge, MA: Harvard University Press. Ness, M. K. (1985). Review of Themes Concerning Blacks. In J. V. Mitchell, Jr. (Ed.), The ninth mental measurements yearbook (pp. 1618–1619). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Newman, R. S., Newman, J. F., & Sells. S. B. (1974). Language and adjustment scales for the Thematic Apperception Test for youths 12–17 years. Vital and Health Statistics (Series 2), 62, 1–84. Obrzut, J. E., & Boliek, C. A. (1986). Thematic approaches to personality assessment with children and adolescent. In H. Knoff (Ed.), The assessment of child and adolescent personality (pp. 173–198). New York: Guilford Press. Obrzut, J. E., & Cummings, J. A. (1983). The projective approach to personality assessments: An analysis of thematic picture techniques. School Psychology Review, 12, 414–420.

reyn2-4.qxd

6/6/2003

9:03 AM

Page 89

4. Projective Storytelling Techniques Ornduff, S. R., Freedenfeld, R. N., Kelsey, R. M., & Critelli, J. W. (1994). Object relations of sexually abused female subjects: A TAT analysis. Journal of Personality Assessment, 63, 223–238. Ornduff, S. R., & Kelsey, R. M. (1996). Object relations of sexually and physically abused female children: A TAT analysis. Journal of Personality Assessment, 66, 91–105. Passman, R. H., & Lautman, L. A. (1982). Fathers’, mothers’, and security blankets’ effects on the responsiveness of young children during projective testing. Journal of Consulting and Clinical Psychology, 50, 310–312. Peterson, R. A., Kroeker, L., & Torshen, K. (1976). Predicting clinical judgment for a primary grade apperception battery. Journal of Personality Assessment, 40, 378–382. Piotrowski, C. (1983). The status of projective techniques: Or, “wishing won’t make it go away” (Report No. CG-107-178). Pensacola: University of West Florida. (ERIC Document Reproduction Service No. ED239 134) Piotrowski, C., Keller, J. W., & Ogawa, T. (1993). Projective techniques: An international perspective. Psychological Reports, 72, 179–182. Pollack, J., Cohen, H., & Weil, G. (1981). The Tasks of Emotional Development Test: A survey of research applications. Psychology: A Quarterly Journal of Human Behavior, 18(4), 2–11. Pollack, J., Cohen, H., & Weil, G. (1982). The Tasks of Emotional Development Test: A survey of research applications. Psychology: A Quarterly Journal of Human Behavior, 19(4), 42–44. Porcerelli, J. H., Hill, K. A., & Dauphin, V. B. (1995). Need-gratifying object relations and psychpathology. Bulletin of the Menninger Clinic, 59, 99–104. Prevatt, F. F. (1999). Personality assessment in the schools. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (pp. 434–451). New York: Wiley. Rabin, A. I. (1960). Projective methods and projection in children. In A. I. Rabin & M. Haworth (Eds.), Projective techniques with children (pp. 2–11). New York: Grune & Stratton. Rabin, A. I. (1986). Concerning projective techniques. In A. I. Rabin (Ed.), Projective techniques for adolescents and children (pp. 3–11). New York: Springer. Rabin, A. I., & Haworth, M. (Eds.). (1960). Projective techniques with children. New York: Grune & Stratton. Roberts, G. E. (1994). Interpretive handbook for the Roberts Apperception Test for Children. Los Angeles, CA: Western Psychological Services. Ronan, G. F., Senn, J., Date, A., Maurer, L., House, K., Carroll, J., & VanHorn, R. (1996). Personal problem-solving scoring of TAT responses: Knowngroups validation. Journal of Personality Assessment, 67, 641–653. Rosenzweig, S. (1977). Manual for the children’s form of the Rosenzweig Picture-Frustration [P-F] Study. St. Louis, MO: Rana House. Rosenzweig, S. (1978a). Aggressive behavior and the Rosenzweig Picture-Frustration Study. New York: Praeger.

89

Rosenzweig, S. (1978b). Rosenzweig Picture-Frustration Study (P-FS). Odessa, FL: Psychological Assessment Resources. Rosenzweig, S. (1988). Revised norms for the children’s form of the Rozenzweig Picture-Frustration (P-F) Study, with updated P-F reference list. Journal of Clinical Child Psychology, 17(4), 326–328. Santostefano, S. (1970). Assessment of motives in children. Psychological Reports, 26, 639–649. Sappenfield, B. R. (1994). Review of the Blacky Pictures. In J. C. Impara & L. L. Murphy, Psychological assessment in the schools (pp. 313–319). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Semeonoff, B. (1976). Projective techniques. New York: Wiley. Schneider, M. F. (1989). The Children’s Apperceptive Story-Telling Test. Austin, TX: Pro-Ed. Schneider, M. F., & Perney, J. (1990). Development of the Children’s Apperceptive Story-Telling Test. Psychological Assessment, 2(2), 179–185. Shneidman, E. S. (1952). Manual for the Make-A-Picture Story (MAPS), with 1988 addendum. Los Angeles, CA: Western Psychological Services. Shneidman, E. S. (1960). The MAPS Test with children. In A. I. Rabin & M. R. Hayworth (Eds.), Projective techniques with children (pp. 139–148). New York: Grune & Stratton. Shneidman, E. S. (1999). The thematic apperception test: A paradise of psychodynamics. In L. Gieser & M. I. Stein (Eds.), Evocative images: The thematic apperception test and the art of projection (pp. 87–97). Washington, DC: American Psychological Association. Silverton, L. (1993). Adolescent Apperception Cards: Manual. Los Angeles, CA: Western Psychological Services. Sivec, H. J., & Hilsenroth, M. J. (1994). The use of the Hand Test with children and adolescents: A review. School Psychology Review, 23(3), 526–545. Smith, V. R. (1992). Sexually abused children’s performance on the Robert’s Apperception Test for Children (Doctoral dissertation, California School of Professional Psychology, 1991). Dissertation Abstracts International, 52, 5549B. Solomon, I. L., & Starr, B. D. (1968). School Apperception Method: SAM. New York: Springer. Stricker, G., & Healey, B. (1990). Projective assessment of object relations: A review of the empirical literature. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 219–230. Symonds, P. M. (1948). Picture-Story Test: Manual and set of 20 pictures. New York: Columbia University Press. Symonds, P. M. (1949). Adolescent fantasy: An investigation of the picture-story method of personality study. New York: Columbia University Press. Taylor, B., & Franzen, S. (1986). The internalized object relations scale. Unpublished manuscript, University of Chicago. Teglasi, H. (1993). Clinical use of story telling: Emphasizing the T.A.T. with children and adolescents. Needham Heights, MA: Allyn & Bacon. Thompson, A. E. (1986). An object relational theory of

reyn2-4.qxd

90

6/6/2003

9:03 AM

Page 90

II. PROJECTIVE METHODS

affect maturity: Applications to the thematic apperception test. In M. Kissen (Ed.), Assessing object relations phenomena (pp. 207–224). Madison, CT: International Universities Press. Triplett, S., & Brunson, P. (1982). TCB and TAT response characteristics in black males and females: A replication. Journal of Non-White Concerns, 10, 73–77. Vane, J. R. (1981). The Thematic Apperception Test: A review. Clinical Psychology Review, 1, 319–336. Volcani, Y. (2000a). Psychological Testing Services International [Online]. Available: www.psytesting.com Volcani, Y. (2000b). The tale of SAGAS: Bringing apperception tests into the cyber world [Online]. Available: www.victoriapoint.com/tale_of_sagas.htm Wade, T. C., & Baker, T. B. (1977). Opinions and use of psychological tests: A survey of clinical psychologists. American Psychologists, 32, 874–882. Wagner, E. E. (1983). The Hand Test manual: Revised 1983. Los Angeles, CA: Western Psychological Services. Wagner, P. A. (1991). Developmentally based personality assessment of adults with mental retardation. Mental Retardation, 29(2), 87–92. Ward, J. (1973). Self identification in the Make-A-Picture Story, protocols of a group of retarded boys. American Journal of Mental Deficiency, 77, 469–471. Weaver, C. N., & Martinez, Z. L. (2000). Should Hispanic persons from different countries of origin be combined into one sample for analysis? Psychological Reports, 86, 335–364 Weaver, V. (1981). Racial attribution, story length, and feeling-tone of young black males to the TCB and TAT. Journal of Non-White Concerns, 10, 31–43. Westen, D. (1990). Towards a revised theory of borderline object relations: Contributions of empirical research. International Journal of Psycho-Analysis, 71, 661–693. Westen, D. (1991). Clinical assessment of object relations using the TAT. Journal of Personality Assessment, 56, 56–74.

Westen, D., Lohr, N., Silk, K., Kerber, K., & Goodrich, S. (1985). Measuring Object relations and social cognitions using the TAT: Scoring manual. Ann Arbor: University of Michigan, Department of Psychology. White, A. M., Olivieira, D. F., Strube, M. J., & Meertens, R. H. (1995). The Themes Concerning Blacks (TCB) projective technique as a measure of racial identity: An exploratory cross-cultural study. Journal of Black Psychology, 21(2), 104–123. Wiese, M. J. (1995). Review of Children’s Apperceptive Story-Telling Test. In J. C. Conoley & J. C. Impara (Eds.), The twelfth mental measurements yearbook (pp. 181–183). Lincoln: Buros Institute of Mental Measurements, University of Nebraska. Williams, R. L. (1972). Themes Concerning Blacks. St. Louis, MO: R. L. Williams. Williams, Y., Williams, R. L., & Williams, A. L. (1981). Suicidal themes in the TCB: Two case studies. Journal of Non-White Concerns, 10, 23–29. Wise, A. J. (l975). Clinical review. Professional Psychology, 6, 102–103. Wodrich, D. L., & Thull, L. M. (1997). Childhood Tourette’s syndrome and the Thematic Apperception Test: Is there a recognizable pattern? Perceptual and Motor Skills, 85, 635–641. Worchel, F. (1987). The Roberts Apperception Test for Children. In C. B. Reynolds & L. Mann (Eds.), The encyclopedia of special education (p. 1367). New York: Wiley. Worchel, F. F., Rae, W. A., Olson, T. K., & Crowley, S. L. (1992). Selective responsiveness of chronically ill children to assessments of depression. Journal of Personality Assessment, 59, 605–615. Zucker, K. B., & Barnett, O. W. (1977). The Paired Hands Test manual. Dallas, TX: McCarron-Dial Systems. Zucker, K. B., & Jordan, D. C. (1968). The Paired Hands Test: A technique for measuring friendliness. Journal of Projective Techniques and Personality Assessment, 32, 522–529.

reyn2-5.qxd

6/6/2003

9:05 AM

Page 91

5 Evaluation of Projective Drawings

HOWARD M. KNOFF

In the context of modern psychology and personality assessment, projective drawings have had a significant impact and history. Originally used as tests of creativity and intellectual maturity, projective drawings have been used as personality assessment techniques since the 1920s and continue to be used frequently in the field (Hutton, Dubes, & Muir, 1992; Kennedy, Faust, Willis, & Piotrowski, 1994; Stinnett, Havey, & Oehler-Stinnett, 1994; Watkins, Campbell, Nieberding, & Hallmark, 1995), most appropriately within multisetting, multisource, multi-instrument personality assessment designs (Knoff, 2002). Significantly, projective drawing approaches and adaptations continue to be developed, refined, and introduced to the field (e.g., Hammer, 1997). Therefore, it is important to understand their historical and theoretical development, approaches toward their psychometric and clinical evaluation, and their use in the context of a comprehensive personality assessment battery. This chapter addresses these three major components which are necessary to evaluate projective drawings, while recommending “best practice” approaches toward their interpretation and use by psy-

chologists and other mental health professionals.

HISTORICAL DEVELOPMENT AND THEORETICAL CONTEXTS Historical Review of Projective Drawings As noted previously, drawings were first used to assess creativity and intellectual maturity. In fact, the use of drawings to evaluate children’s intellectual functioning continues to the present. Historically, Goodenough (1926, 1931) first developed and standardized an approach to evaluate children’s intelligence through their drawings of a man (i.e., the Goodenough Draw-A-Man Test). By crediting children at different chronological ages with points for the characteristics or details included in their drawings and, separately, for their drawings’ qualitative maturity, Goodenough was able to quantitatively assess their intellectual functioning and derive individual intelligence estimates. This technique was updated by Harris (1963) whose Goodenough–Harris scoring system provided a more recent standardization and 91

reyn2-5.qxd

92

6/6/2003

9:05 AM

Page 92

II. PROJECTIVE METHODS

standard scores that were related to children’s intellectual maturity. In the Goodenough–Harris scoring system, children’s drawings of a man and/or a woman are given points for the presence of specific characteristics (e.g., parts of the body, clothes, and accessories), for the quality of the drawing (e.g., appropriate and well-drawn lines and angles within the figure and good proportionality between different body parts like between head and trunk), and for the integration of figures into whole, recognizable persons. These points are entered as raw scores into norm tables which are separated by the gender of the figure drawn (i.e., male vs. female), the gender of the child completing the drawing, and the chronological age of the child. The resulting standard scores represent an evaluation of the child’s intellectual maturity (as opposed to his or her “intelligence”), which Harris (1963) defines as the “ability to form concepts of increasingly abstract character” (p. 5). Clearly, the Draw-A-Man technique assesses only one type of intellectual maturity, that which is operationalized by a visual–motor task requiring the cognitive, experiential, and environmental recognition of people’s physical characteristics and the ability to represent them pictorially during a test/demand situation. Other techniques or tests operationalize intellectual maturity differently; thus, an understanding of the Draw-A-Man versus these other techniques is critical to sound assessment and interpretation. A separate qualitative component of the Goodenough–Harris approach involves evaluating a child’s drawing against 12 criterion drawings that represent increasing levels of drawing sophistication and intellectual maturity. This more global assessment component integrates a developmental perspective into the evaluation. This perspective recognizes that children’s drawing maturity (and therefore intellectual maturity) changes over time, and that it must be evaluated periodically if a child is to be accurately assessed. Without this developmental perspective, some drawings might be interpreted projectively when they actually represent good developmental approximations of the child’s current drawing maturity and/or chronological age. It would be a grave error to interpret the lack of a draw-

ing’s detail as some manifestation of an “apparent” emotional trauma or dysfunction when the absence of that detail is developmentally appropriate and expected. Although the Goodenough–Harris DrawA-Man scoring system has been used for many years, it has also been criticized for its outdated scoring standards, norms and standardization, and approach toward deriving standard scores. In 1988, Naglieri revised the Goodenough–Harris test, publishing a Draw-A-Person assessment tool with scoring systems and norms developed to evaluate a child’s intellectual ability (the Quantitative Scoring System [QSS]) or emotional or behavioral disorder status (1991; the Screening Procedure for Emotional Disturbance [SPED]). Both of the scoring systems use norms, based on a standardization sample of 2,622 5- through 17-year-old children matched to the 1980 census data, who were analyzed by gender, age, and culture/race. For the Draw A Person: QSS (DAP:QSS), a quantitative scoring system evaluates students’ Man, Woman, and Self drawings using 14 scoring criteria that rate the presence of specific body parts, the detail of the body parts, the proportions of the body parts, and other elaborations for bonus credit. As a nonverbal measure of intellectual ability, the DAP:QSS has scoring criteria that are somewhat more defined than the Goodenough–Harris system, and the manual reports good test–retest, interrater, and internal reliabilities and factor analyses that suggest strong correlations with g. Standard scores from matched pairs of males and females, blacks and whites, and Hispanics and non-Hispanics did not significantly differ, although a low scoring ceiling limits its utility for children over the age of 11. Since its publication in 1988, few articles (six were found) have used or investigated the psychometric properties of the DAP:QSS—most written by one or more of the tool’s authors. Reviews suggest that the psychometric quality of the instrument still needs to be demonstrated and that beyond the restandardization, improvement over the Goodenough–Harris test has been unremarkable. While the use of drawings to assess children’s emotional status is addressed later in this chapter, Naglieri (1991) used basically the same standardization sample as the

reyn2-5.qxd

6/6/2003

9:05 AM

Page 93

5. Evaluation of Projective Drawings

DAP:QSS to generate norms for the Draw A Person: SPED (DAP:SPED). Here, scoring criteria were developed by using a sample of 2,260 students and working through a series of scoring criteria descriptors until appropriate levels of scoring reliability were attained. Once applied to the larger sample, little variation in scores by age was found, but gender differences were found. T-score norms, therefore, were published by gender and by the age clusters of 6 to 8, 9 to 12, and 13 to 17 years old. While relatively high reliability and some evidence of discriminant validity is noted in the manual, the use of the DAP:SPED was noted in only three articles—two by one of the tool’s authors. Given this, and concerns about its assumptions regarding the nature of emotional disturbance and its functional contribution as a screening device, it appears that the DAP:SPED should only be used as a research (and not a diagnostic) tool at the present time. Besides the Draw-A-Man or Draw-APerson tests, the Bender Gestalt Test (BGT) also has a significant history as a technique evaluating children’s intellectual development and/or visual–motor integration. Developed by Bender (1938), the test consists of nine geometric drawings which are presented individually to children in a standard order, with directions to reproduce them on paper with pencil as accurately as possible. The Koppitz scoring system for the BGT (1963, 1975) is still probably best known and most widely used for intellectual and visual–motor assessment; its use in studies correlating BGT performance with neurological and cognitive development, the presence of brain injury, math and reading skill, and other academic areas continues with variable levels of diagnostic or discriminant success. Still an extremely popular technique with psychologists (Hutton et al., 1992; Kennedy et al., 1994; Stinnett et al., 1994), the BGT remains a part of most assessment batteries. There continues, however, to be significant question as to its utility beyond its basic visual–motor integration assessment, a fact discussed herein relative to personality assessment. The use of drawings to evaluate children’s personality characteristics and functioning has a history that parallels their use in the assessment of intelligence, intellectual matu-

93

rity, and visual–motor and other development. In fact, with the publication of Goodenough’s Draw-A-Man Test, a number of individuals noted the impact of personality variables, characteristics, and influences on the clinical drawing process. This interest in the projective use of drawings culminated in the late 1940s with a number of significant works, one related to the House–Tree–Person (H-T-P) technique (Buck, 1948) and one related to Human Figure Drawings (HFDs) (Machover, 1949). The H-T-P technique involves having children complete separate drawings of a house, a tree, and a person with opportunities to ask them questions that clarify and expand the apparent information in each picture after each has been drawn. Although Buck (1948) published his H-T-P monograph to expand the use of drawings as intellectual assessment measures, he also provided a host of personality-related hypotheses concerning the global and specific characteristics of the H-T-P figures and their relationship to a child’s personality dynamics. Since that seminal work, Hammer (1958, 1980), Burns (1987), and Buck (1970) again published major works that summarized and expanded on the clinical use of the H-T-P in evaluating children’s personality dynamics and dimensions. Machover’s (1949) work on the projective uses of HFDs, which could involve any drawing of a person for the purposes of personality assessment (including the Draw-AMan or Draw-A-Person), was based on her work with emotionally disturbed adolescents and adult psychotics. Expanded by Hammer (1958), Koppitz (1968, 1984), Schildkraut, Shenker, and Sonnenblick (1972), and others, HFDs now have an extensive projective literature on both child to adult drawings, yielding information on their relationships to self-concept, personality style and orientation, sexual and other development, and inter- and intrapersonal conflicts (Handler, 1985). Koppitz’s (1968) work is especially significant in that she analyzed the HFDs of boys and girls ages 5 through 11 and 12 to identify developmentally expected and unusual items and characteristics. As noted previously, this developmental perspective helps the psychologist to differentially analyze HFDs; this insures that drawings characteristic of a child’s chronological or maturational age are not

reyn2-5.qxd

94

6/6/2003

9:05 AM

Page 94

II. PROJECTIVE METHODS

interpreted projectively, thereby resulting in inappropriate interpretations and improper clinical conclusions and recommendations. Another strand of projective drawing history relates to drawings that involve more complex or interactive subject matter—that is, drawings that go beyond simply “Draw me a house,” “Draw me a tree,” or “Draw me a person.” Initially, these more complex drawings involved asking children to draw specific events or groups of people, most notably their families (Hulse, 1951, 1952). While these “nonkinetic” drawings were helpful in discerning children’s perceptions of their families, they also were often portrait-like, with noninteracting figures and elements. Burns and Kaufman (1970, 1972), therefore, saw the need for and utility of demanding that the figures in their complex projective drawings be actively engaged. Their Kinetic Family Drawing (K-F-D) technique asked children to “draw a picture of everyone in your family, including you, doing something.” According to the authors, these directions resulted in significantly more projective data for interpretation—data that extended their social–emotional hypotheses beyond simple child-focused, intrapersonal concerns to complex, ecologically focused, interpersonal, multisetting, and multidimensional concerns and issues. After the K-F-D, a number of other books and monographs describing additional kinetic drawing approaches were published. Prout and Phillips (1974) and Sarbaugh (1983) developed separate variations of Kinetic School Drawing (K-S-D) techniques asking children to “draw a school picture. . . . Put yourself, your teacher, and a friend or two in the picture. . . . Make everyone doing something.” These projective drawing techniques tap children’s school-based problems, issues, expectations, and/or concerns, while also evaluating their relationships with peers and adult models. However, the K-S-D was not able to differentiate meaningfully between school-based issues versus home issues that were being manifested in school. In a projective sense, this need was addressed by integrating the K-F-D and K-SD techniques into a combined system, the Kinetic Drawing System (Knoff & Prout, 1985) which was developed to facilitate this differential analysis while also updating and

reviewing the literature specific to both techniques in one place. Although the major works and trends in projective drawing have been described earlier, the discussion is far from exhaustive. New books and chapters on projective drawings continue to be published periodically and “new” techniques or approaches (e.g., the Kinetic Drawing—School: First Memory Technique) emerge as well. For example, Burns (1990) introduced an approach involving family-centered circle drawings (F-C-C-D) to help clinicians to understand how individuals “see” themselves in relation to their “inner parents.” And, a more recent book on the projective use of drawings (Hammer, 1997) provides information on a chromatic H-T-P test, projective mother-and-child drawings, a Draw-AGroup test, a Kinetic H-T-P Test, and the Draw-A-Person-In-The-Rain. Clearly, the history of projective drawings will continue, but the context and theoretical interpretations of projective drawings will change as the Zeitgeist of psychology changes. This fact is exemplified next, in a discussion of the possible theoretical perspectives of projective drawings, and later in a discussion of best-practice approaches for their use in comprehensive personality assessment.

Theoretical Perspectives of Projective Drawings Among the many psychological trends and movements that have occurred during the modern history of projective drawing techniques (i.e., from the 1920s to the present), four important psychological perspectives or orientations are particularly relevant: the psychodynamic, the cognitive-developmental, the behavioral, and the cognitive-behavioral orientations. These four orientations are discussed in this section, with emphasis on the way in which projective drawings are interpreted in each one. The Psychodynamic Orientation The psychodynamic orientation has its roots in Freudian psychoanalysis, where projective drawings are seen as symbolic representations of a child’s world or perception of reality. In this context, the paper on which any figure is drawn corresponds to

reyn2-5.qxd

6/6/2003

9:05 AM

Page 95

5. Evaluation of Projective Drawings

the child’s environment, figures represent significant people in the child’s life (past, present, or future) or significant role figures (father or authority figures, sibling figures, perceived self figures), and objects may be symbolic of specific conflicts, attitudes toward or between figures, or behavioral tendencies (e.g., spiders are said to suggest conflicts with dominant or threatening mothers, and monsters are thought to indicate specific power issues related to male or authority figures). The drawing process, according to this orientation, involves the presentation of an ambiguous task. For example, the child is told to draw a house but is not told what type of house, how to hold the paper, how many elaborations to put onto the house, or whether to put people or objects in or around the house. The child therefore must “project” his or her feelings, attitudes, strivings, and/or perspectives into the drawing process, thereby allowing the examiner to see a particular facet of his or her personality and/or personality functioning. At times, children draw distorted figures or scenes in their projective drawings. For example, given the K-F-D instructions, a child who comes from a physically abusive home may draw a close-knit family unit with everyone participating together in a joyous birthday celebration. Psychodynamically, this scene may be analyzed as a distortion of reality; the child, it is believed, is using this defense mechanism to deny his or her current reality in order to deal with a tortuous, anxiety-provoking situation. At other times, children may draw their perceptions or interpretations of how their lives exist, despite the fact that these perceptions do not quite mesh with actual situations or events. Psychodynamically, these drawings may not be considered distorted; rather, they may be interpreted as depictions of the children’s actual (albeit inaccurate) interpretation of a life situation that has idiographically traumatized them and affected their social-emotional or psychological development. Because of the symbolic nature of the psychodynamic approach, many of the projective drawings have both global and specific aspects that are interpreted in fairly consistent ways. Following are brief psychodynamic interpretations of the H-T-P and Ki-

95

netic Drawing System techniques to exemplify this point. Psychodynamically, the house drawing is said to reflect a child’s perception of his or her home life and the quality of the relationships among family members. Although the house drawing may represent a child’s past, present, future, ideal, or fantasized home, it also provides information about his or her feelings about the home environment and his or her approach toward dealing with home- and family-related issues (e.g., to be defensive, aggressive, withdrawing, insecure, and regressive). Within the house, the door and windows reflect a child’s openness to direct contact with the environment or individuals in the environment; windows, in addition, may indicate whether a child is controlled by family or home issues or is excluded because of those same issues. Roofs relate to a child’s need for and type of fantasy, whereas walls provide a measure of the child’s ego strength and stability. Tree drawings are said to reflect the unconscious, underlying personality dynamics of a child, along with indications as to the child’s ability to adjust to intrapersonal, interpersonal, and environmental crises, events, and issues. The tree drawing is considered symbolic of life and growth. It may suggest traumas that the child has experienced, deep-seated self-concepts and selfattitudes, and resources that are available to the child and whether or not they will be used. With respect to specific components, the tree’s roots generally reflect the child’s reality testing or need to hold onto reality, the trunk relates to the child’s ego strength and feelings of psychological control and power, the branches suggest the child’s ability to derive satisfaction from the environment and ability to interact interpersonally in appropriate and socially successful ways, and the crown symbolizes the child’s interand intra-active style—his or her use of fantasy, emotional lability, reactions to environmental pressure, and ability to deal with reality. Person drawings can usually be categorized as drawings of the self, the ideal self, and/or significant others. Self-drawings tend to reflect children’s feelings or self-concepts at the time of the drawing; these self-concepts may range from body image issues to issues related to a child’s interpersonal, in-

reyn2-5.qxd

96

6/6/2003

9:05 AM

Page 96

II. PROJECTIVE METHODS

tellectual, behavioral, vocational, religious, extracurricular, and/or familial self. Drawings of the ideal self tend to reflect children’s aspirations, needs, and/or desires—whether realistic, fantasized, troubled, or irrational. These drawings can also specifically focus on the self-concept areas noted previously, and they can depict another person who in actuality symbolizes the child doing the drawing. Drawings of significant others often identify specific individuals (e.g., a sibling, peer, or adult) or individuals who symbolize specific roles or issues (e.g., a vice principal as an authority figure or a doctor as a helping figure) for the purpose of communicating some strong past, present, or expected negative or positive affect, experience, or interaction. Parent figures are the most typical drawings of significant others, and a child’s choice in drawing such a figure is not considered random; it is thought to indicate some significant conscious or unconscious psychological issue specific to the individual drawn, with which the child is currently dealing. Relative to other specific parts of the figure, the head of a person drawing is considered, psychodynamically, to provide hypotheses about the child’s intellect, fantasy activity, ability to control impulses and emotional lability, and ability to socially interact in appropriate ways. Facial features help to assess the potential quality of the child’s interactions with the environment, as well as the child’s overall affect and feelings toward self and/or others. Characteristics of the neck are considered symbolic of relations between the child’s cognitive, intellectual, and problem-solving capabilities and his or her ability to enact those capabilities so that emotional impulses are controlled. The body or trunk is often related to children’s basic drives, while arms, fingers, legs, and toes provide information on the qualitative nature of children’s social and interpersonal adjustments (e.g., friendly, constructive, hostile, and destructive), their interactive styles (e.g., rigid, flexible, relaxed, and stilted), and their openness or closedness to interpersonal relationships and dealing with significant social-emotional issues or developmental circumstances. Globally, the Kinetic Drawing System’s family (K-F-D) and school (K-S-D) drawings look at children’s perceptions of the

psychological warmth and support within their home and school environments, respectively, as well as potential themes and issues in those settings that may be interfering with the child’s ongoing social, emotional, personality, or other development. Also suggested within the drawings may be specific events that have distressed the child, and when and how the distress might have occurred and interfered with the ongoing development noted previously. Specifically, both the family and school drawings are evaluated for the (1) actions of and between drawn figures; (2) characteristics of specific figures; (3) position, distance between, and barriers between figures; (4) drawing style; and (5) psychodynamic or other symbols present in the drawings. All these areas have potential psychodynamic interpretations. For example, in K-F-D drawings, activities such as a father mowing a lawn or chopping wood may indicate some fears of castration of an overly dominant father/authority figure. If the father’s pants are excessive darkened, it may suggest the child’s conflict with some aspect of sexual development or impulse. In K-S-D, the presence of a prominent “X” shape within the drawing suggests the influence of a strong superego, which may be needed by the child to control specific aggressive or other inappropriate id impulses. The child’s use of heavy and overworked lines in drawing this “X” may suggest an excessive amount of anxiety specific to that need for superego involvement. Finally, the presence of light bulbs, lights, or electricity in either K-F-D or K-S-D may indicate that the child has such a great need for psychological warmth or love that all thoughts and activities are directed toward that need. This overview of some of the psychodynamic interpretations of various projective drawing techniques is necessarily brief and simplistic. While the psychodynamic interpretation of drawings dominates the literature historically, it is important to differentiate between the empirical versus clinical nature of the work cited. As discussed and reviewed later, many of the psychodynamic interpretations are based on clinical judgment and interpretation, not on research that has demonstrated an objective, empirical link to the conclusions drawn. This leaves the application of this orientation

reyn2-5.qxd

6/6/2003

9:05 AM

Page 97

5. Evaluation of Projective Drawings

and perspective to the clinician who needs to decide how to best use—or use at all— the information reviewed earlier.

The Cognitive-Developmental Orientation The cognitive-developmental orientation emphasizes that figure drawings are first and foremost reflections of children’s developmental and cognitive characteristics, and that they should be evaluated in this context before any projective or personality-related interpretations are hypothesized. Furthermore, the effects of the environment on the child’s cognitive development (e.g., the amount of sensory stimulation at home and the availability of preschool and play groups) should be analyzed as part of these characteristics, and again should be evaluated prior to projective considerations. All this is accomplished in five ways, by evaluating (1) the child’s developmental and health history, including specific prenatal, perinatal, and postnatal milestones and events; (2) the child’s cognitive and intellectual development, through IQ and other processing tests; (3) the child’s visual–motor ability, through measures developed for that purpose (e.g., the Developmental Test of Visual–Motor Ability—4); (4) relevant and critical components of the child’s socioeconomic status that interact with the cognitive, intellectual, visual–motor, and other developmental skills that are apparent in figure drawing tasks; and finally (5) the child’s visual–motor drawing skill as compared to same-age peers from some standardization or norm group. Theoretically, this orientation has been best conceptualized by Jean Piaget (Wadsworth, 1996), with his emphasis on cognitive development as a central and determining feature of most other areas of development. Historically, this orientation has been best operationalized by the writings of Goodenough (1926, 1931), Harris (1963), and Koppitz (1968). Piaget would say that the child’s cognitive perspective of his or her world, and his or her schema of a “person” within that cognitive perspective, should significantly determine (1) how a child being given a Draw-A-Person Test will complete a requested drawing, (2) what details will be in that drawing, and (3) to what degree it compares with drawings of same-age

97

peers. Although the schema may be influenced by social-emotional experiences, it is first and foremost a cognitive structure that is based on an interaction of intelligence and experience. That is, children’s drawings of a “person” will most reflect their cognitive conceptualizations of those physical or other characteristics possessed by their prototypical person. If they have not assimilated that all people have feet, their drawings will lack that characteristic. Koppitz (1968), taking a more normative perspective of cognitive and visual–motor development, summarized her analyses of children’s expected and unusual HFD characteristics in 1968. Looking separately at boys’ and girls’ drawings from ages 5 through 11 and 12, Koppitz defined an expected drawing characteristic as one present in 86–100% of the drawings for a specific sex at a specific age, and an unusual characteristic as one present in fewer than 15% of the drawings. According to these criteria, a typical 5-year-old boy would be expected to draw a head, eyes, nose, mouth, body, and legs on his HFD. A typical 11- or 12-yearold girl’s drawing would contain these same six characteristics plus two-dimensional arms and legs (with the arms attached at the shoulder and positioned downward), feet, hair, a neck, and at least two articles of clothing. Exceptional HFD characteristics for 5-year-old boys would include knees, elbows, two lips, nostrils, arms drawn at the shoulder, two-dimensional feet, five fingers, pupils in the eyes, at least four articles of clothing, a figure drawn in profile, and a figure drawn in appropriate body proportions. For 11- or 12-year-old girls, only the presence of knees would be considered exceptional. According to these norms and this cognitive-developmental perspective, characteristics of drawings that are not expected, given a child’s chronological and/or maturational age, may not be accurately interpreted projectively when they are missing from a drawing. Thus, this perspective must be considered prior to projective interpretations to prevent the overinterpretation and inappropriate interpretation of projective drawings. Naturally, the presence of these expected and exceptional characteristics differ across the 5–12 age span and for boys versus girls. While Naglieri’s (1988) DAP:

reyn2-5.qxd

98

6/6/2003

9:05 AM

Page 98

II. PROJECTIVE METHODS

QSS norms may provide the most recent “baseline” for figure drawings using this perspective, updated norms, using a reputable national stratification and sampling process, would be most welcome at this time. To summarize, the cognitive-developmental orientation suggests that a child’s intellectual ability and experiential development must be considered before projective drawings can be interpreted in personality-related ways. Therefore, information about a child’s IQ and his or her home environment and developmental opportunities is critical as an evaluative baseline. This baseline facilitates an analysis of a child’s cognitive × personality style, abilities, and status, such that projective interpretations are as accurate as possible. For example, if a 12-year-old girl who has grown up in a low-socioeconomicstatus background with little sensory and environmental stimulation is asked to draw a person, the absence of feet, two-dimensional arms and legs, and hair may suggest one or more of the following: 1. The child simply has not attended appropriately to environmental stimuli and does not yet cognitively perceive (or have the schema of) people as having those characteristics. 2. The child is delayed, compared to sameage peers, because she has not had the necessary and sufficient environmental and/or sensory experiences that result in those features’ being cognitively included in the concept of “person.” 3. The child has not had the necessary and sufficient environment and/or sensory experiences that result in those features’ being motorically reproduced in person drawings—whether or not the child has an accurate cognitive perspective of a “person.” 4. The child does not have the intellectual ability to benefit from any experiential learning, and the missing features result from this deficiency. 5. The child does not want to reproduce the person completely, because of culturally or socioeconomically related poor motivation or a situational fear of the testing situation. 6. The child has some social-emotional difficulty (as hypothesized from the projec-

tive literature) related to the absence of feet, two-dimensional arms and legs, and hair. These are just hypotheses that need to be validated more objectively; doing this, however, is further discussed in a later section. The Behavioral Orientation Within this orientation, the projective drawing task represents an ambiguous task, completed in a formal assessment situation, with an examiner who may or may not be familiar, under conditions that are inherent both to the child (e.g., defensiveness, anxiety, and resistance to authority) and to the environment (e.g., a comfortable room that is quiet and with no chance of interruptions). Thus, the projective task allows an examiner to sample a child’s behavior under specific task, situational, and environmental conditions and to make innumerable behavioral observations and analyses across a number of dimensions. With enough appropriate and representative behavioral samples during projective testing, the examiner may hypothesize about and then determine how the child reacts, for example, in similar ambiguous situations or under similar evaluative conditions. And with enough appropriate and representative multimethod, multisetting, and multisource behavioral samples, the examiner may formulate and validate hypotheses about the child’s general behavioral style and functioning (i.e., personality trait patterns) and the child’s differential style and functioning under more specific ecological circumstances (i.e., personality state patterns). In essence, behavioral observations during projective test administration are no different from behavioral observations during any other testing situation. What differ, as noted previously, are the task demands, the reactions and responses of the child to the task demands, the examiner’s reactions and responses to the task demands, and the complex behavioral effects that occur because of child–test–examiner interactions. Possible behavioral observations during projective testing, then, include the child’s (1) behavior or physical reactions during the test performance or inquiry process; (2) speech and language; (3) attitude and behavior toward the examiner; (4) reaction to the examiner’s

reyn2-5.qxd

6/6/2003

9:05 AM

Page 99

5. Evaluation of Projective Drawings

style, questions, and comments; (5) reaction to the test situation and demands; (6) problem-solving and behavioral or work style in completing the task demands; and (7) comments to him- or herself as a reflection of self-concept and self-confidence. Many of these observation areas are also used to assess the child’s general mental status (Sattler, 1990). Among the additional mental status areas available for observation during projective testing and not yet mentioned are the child’s (1) content of thought, (2) sensory and motor functioning, (3) intellectual and cognitive processing functioning, and (4) insight and judgment. All these observation areas focus on overt behavioral functioning, interactions within interdependent facets of the behavioral ecology, and representative samples of behavior that can be used to predict future behavior under similar or more generalized circumstances. The Cognitive-Behavioral Orientation The cognitive-behavioral orientation represents a theoretical extension of the behavioral orientation that includes children’s cognitions as part of their behavior. In this context, projective drawing techniques provide information that permits hypotheses about children’s thoughts, beliefs, expectations, self-statements, aspirations, attributions, needs, and perceptions. This information is collected either passively or actively. The passive approach uses the “stimulus pull” that is inherent within some requested drawings. Stimulus pull occurs when the request for a specific drawing results in a high probability of certain responses or certain cognitive themes. The active approach uses the “Inquiry” process, where the clinician asks a set of standard or self-generated questions to clarify a child’s overt or covert cognitive processes in choosing the content, style, or interactions depicted within a requested drawing. Cognitive-behavioral research has shown that cognitions can cause, encourage, support, reinforce, change, and/or influence behavior. Thus, the identification of consistent and significant cognitive patterns, within children’s drawings, may help to predict their behavioral actions or reactions to certain situations, a critical component of any personality evaluation. Unfortunately, projective drawings pro-

99

vide a fairly unsystematic and disorganized assessment of a child’s cognitions. Unlike the situation in a structured or semistructured interview, the psychologist is dependent on the social-emotional themes and cognitions that a child reveals, actively or passively, in his or her drawings. The requested drawings do not systematically survey a broad range of possible problem areas, such that the assessment process can then focus on and fully analyze those areas that are identified. Furthermore, because of the ambiguity inherent in each drawing technique, a child can reveal information about any of the possible cognitive areas listed previously, and then from any number of perspectives. For example, a child may reveal cognitions that are related (1) to past, present, or future issues or events; (2) to a specific person or to myriad people or roles; (3) to a specific setting or to multiple settings or circumstances; and/or (4) to specific social-emotional processes or to more generalized social-emotional or personality processes. Although it is assumed that children generally will reveal their most predominant and/or troubling cognitions within the themes present in their projective drawings, this assumption must be tested with every case. The psychologist’s job, then, is to analyze any cognitive hypothesis across all the personality assessment data in the comprehensive battery, and to systematically expand on and validate the most relevant cognitive information through additional interviews, observations, and/or objective processes. Potential problem areas that are not assessed should be addressed in other parts of the assessment battery. As noted and despite the possible weaknesses of projective drawings within this orientation, one of the variables that does provide some cognitive organization to the various drawing techniques is the “stimulus pull” of each. Stimulus pull is the phenomenon that explains why certain projective drawings tend to elicit certain themes, cognitions, or types of responses. For example, the request to draw a house tends to elicit a drawing or a story about the drawing that reveals a child’s remembrances, beliefs, or expectations related to a house he or she has experienced, either in real life or in fantasy. The house may be a past house (e.g., where the child was last happy), a present house

reyn2-5.qxd

6/6/2003

100

9:05 AM

Page 100

II. PROJECTIVE METHODS

(e.g., depicting the current conflicts between family members), or a future house (e.g., alluding to the child’s expectation that his or her future home also will contain significant conflictual relationships). But, according to this orientation, the drawing will represent some cognition that has been stimulated by the task demand. Similarly, the Person drawing in H-T-P tends to uncover children’s cognitions about themselves or significant others; the K-F-D obviously provides the children’s perceptions of their real or idealized family life; and the K-S-D elicits reflections about academics, the school environment and interactions, or the schooling process. But, unless these cognitions are explicitly apparent in a child’s drawings, an Inquiry often is needed to determine not just their presence but their meaning. Relative to the Inquiry process, a number of authors (e.g., Hammer, 1997) have suggested structured or semistructured “interview” formats to help children to clarify, for the clinician, the figures, actions, and interactions within their drawings. On a less formal level, any clinician can follow a more unstructured interview format, simply asking a child the relevant questions about a drawing, and then about the responses to these questions, such that a broader clinical picture of the child’s cognitions and state of mind emerges. Ultimately, although they are unsystematic, projective drawings may help to sample some important areas and environments of children’s lives within the cognitive-behavioral perspective, and an understanding of a drawing’s potential stimulus pull, guided by an effective Inquiry, may facilitate the assessment process. However, an Inquiry is no better or worse than any other interview format, it often is needed with projective drawings to clarify aspects of a child’s drawings that are more implicit than explicit, and the validity of the child’s cognitions—whether obtained from the drawing or from the Inquiry—needs to be determined through more objective approaches regardless of the clinician’s “certainty” that they exist and are accurate.

Summary It is important to reemphasize that projective drawings can be interpreted within any

combination of the four theoretical orientations described previously. The tenets and beliefs of the theory, then, drive the interpretation of the projective drawings, rather than vice versa. In practice, all four orientations may be useful for a comprehensive analysis. The cognitive-developmental orientation may provide information on the impact of cognitive, visual–motor, and developmental variables and experiences on the projective drawing process, and thus the degree to which the drawing can be interpreted from a social-emotional perspective. The psychodynamic orientation can offer hypotheses as to the inner dynamics and conflicts that may be represented within the projective drawings—hypotheses that should be objectively evaluated before their acceptance. The cognitive-behavioral orientation can provide insight into the child’s self-statements, beliefs, expectations, and other perceptions, so that the psychologist then can explore how these cognitions translate into behavior and social-emotional functioning for the child. Finally, the behavioral orientation analyzes the drawing process and the child’s interactions within the behavioral ecology, making predictions when appropriate as to how the child might respond to similar situations in his or her day-to-day functioning. Analyses of projective drawings, regardless of the theoretical orientation used, generate hypotheses about personality and social-emotional functioning. Many of these hypotheses are reported in the professional literature. The next section reviews a number of these studies, analyzing their methodological strengths and weaknesses and commenting on the utility of their results and implications. Implicit in these analyses, however, are dimensions that involve (1) the explicitness of projective drawing data, (2) the validity of the information depicted, and (3) level of interpretive inference of the clinician. Specifically, in contrast with behavioral observations of a child’s disorder or impaired interpersonal functioning, projective drawings reflect implicit levels of dysfunction at best. In the same context, the projective information portrayed in a drawing may range from a child’s perceived reality to an externally and objectively demonstrated reality. And, finally, given the presence of more objective and empirically

reyn2-5.qxd

6/6/2003

9:05 AM

Page 101

5. Evaluation of Projective Drawings

derived personality assessment tools, projective drawings require far more clinician inference and interpretation—a weakness that, in itself, may suggest a minor or nonexistent role in the assessment process.

A PSYCHOMETRIC AND CLINICAL EVALUATION OF SELECTED STUDIES A psychometric and clinical analysis of the hundreds of projective drawing studies available in the literature necessarily involves an evaluation of each study. This is primarily because of the vast differences in psychometric and experimental quality among these studies, and because these studies provide such widely differing methodological details that one cannot assume that results are clinically accurate and socially valid without additional validation and replication. Despite this need for individual evaluations, few critical reviews of the literature are available for projective drawings. More common are (1) collections of projective characteristics and interpretations based on the work of individual clinicians, who have investigated self-selected populations and truncated samples that often are nonrandomly distributed, and/or (2) reviews that fail to integrate projective drawings into a multitrait, multisetting, multimethod assessment approach (Campbell & Fiske, 1959), which would demonstrate their ability to generate integrated hypotheses rather than isolated, relative meaningless results. The lack of a comprehensive literature review leaves the clinician without an objective and useful measure of the current status of projective research, and the field without a clear direction as to what future directions are most necessary and most relevant. Relative to the psychometric qualities of projective drawings, most of the recent research has investigated the ability of projective drawings to identify and/or discriminate various specific clinical groups of children and adolescents—sexually abused children (e.g., West, 1998), mentally ill chemical abusers (Taylor, Kymissis, & Pressman, 1998), and mood- and anxietydisordered children (Tharinger & Stark, 1990). In addition, whereas West (1998) conducted a meta-analysis of studies assess-

101

ing the efficacy of projective techniques in discriminating child sexual abuse and Handler and Habenicht (1994) reviewed the psychometric literature on the Kinetic Family Drawing Technique, few, if any, recent studies have specifically evaluated the reliability of projective drawings. Indeed, most of these studies have simply reported reliability data within the context of their broader research agenda. More historically then, relative to reliability, Cummings (1986) evaluated a dozen studies between 1968 and 1981 with respect to interjudge reliability and reported reliabilities ranging from .75 to .97, with the critical determinants of good interjudge agreement being (1) the specificity of the items, characteristics, or drawing styles to be scored and (2) the training and supervision of the various judges to insure similar scoring criteria and conditions. With respect to test–retest stability, Cummings critically reviewed nine studies with retest intervals from 1 day to 3 months and stability coefficients ranging from .69 to .91. With stability generally decreasing as the test–retest interval increased, the importance of simultaneously evaluating psychometric quality and clinical utility becomes apparent. Simplistically, this decreased stability could suggest that projective drawing techniques are unreliable over time, or it may be accurately reflecting a child’s change in social-emotional status. Or, the decreased stability may be due to a scoring system that attends more to structural changes in a child’s initial versus later drawings (e.g., the presence of excessive shading in the first drawing and lack of shading in the second drawing) than to drawing characteristics that reflect changes in a child’s clinical status (e.g., the presence of anxiety). Regardless, it appears that with the right scoring criteria and evaluative approaches, acceptable levels of interrater reliability are possible with projective drawings (see, e.g., Handler & Habenicht, 1994, for a review with the K-F-D technique). With respect to test–retest reliability, the dilemma of evaluating the psychometric nature of projective drawings with their clinical meaning and validity becomes apparent. And, in the absence of (many) composite scale approach(es) to scoring projective drawings, calculations of their internal reliability are

reyn2-5.qxd

6/6/2003

102

9:05 AM

Page 102

II. PROJECTIVE METHODS

virtually impossible. These limitations, relative to determining a projective drawing technique’s reliability, explain why these approaches should be used only for the generation of social, emotional, or behavioral hypotheses and not to make clinical diagnoses or conclusion. Indeed, consistent with the multitrait–multimethod approach, hypothesis generation may be the most defensible and helpful contribution of projective drawings. The validation of projective drawing techniques has been extremely difficult to document, and an extremely variable research literature exists in this area. Three types of validity are commonly found in this literature: concurrent validity, construct validity, and incremental validity. There are few studies of concurrent validity in the projective drawing literature. These studies typically correlate specific diagnostic indicators in projective drawings with more objective scales or indices that already are validated with respect to these indicators. Conceptually, this is an acceptable method of validation. Pragmatically, however, its success is based on the integrity of the methodological procedures used and the ability of the researchers to choose samples that have some level of generalizability to specific and important clinical groups or to more universal populations. More concurrent validity research with projective drawings is needed before definitive conclusions can be made as to their utility. Although the issue of interpretation accuracy is important with projective drawings, this issue is equally critical in the interpretation of many socalled objective personality assessment tools. Studies of construct validity with projective drawing techniques have often focused on the “known-groups” method, which uses a validated anchor scale to separate a large sample of children into clinically differentiated subgroups. These subgroups then complete a projective drawing battery, which is analyzed for its ability to approximate the same clinical separations. A variation of this approach investigates the discriminant validity of projective drawings—that is, their ability to discriminate between samples that either represent substantively different clinical groups or involve a clinical group that is compared with

a control or nonclinical group. This type of construct validity has been used in numerous studies of projective drawing techniques with equivocal or nonsignificant results. For example, Hibbard and Hartman (1990) compared the presence of Koppitz’s Emotional Indicators and categories in HFDs completed by alleged sexual abuse victims between the ages of 5 and 8 years and comparison (presumably nonabused) children. The results indicated that although more anxiety indicators were present in the sexually abused children’s drawings, no significant differences in specific indicators or categories between the two groups was evident. Extending this research to include studies in which effect sizes could be calculated, West (1998) conducted a meta-analysis of 12 studies that used projective techniques (among them the Draw-A-Man, H-T-P, HFD, and K-F-D) to discriminate sexually abused versus nonabused children. While the results indicated that projective tests, in fact, did discriminate abused versus nonabused students, the specific contribution of projective drawings in this result could not be determined. The lack of discriminant validity for projective drawings again was found by Tharinger and Stark (1990), who administered the DAP and K-F-D to 52 fourth through seventh graders with mood, anxiety, or mood and anxiety disorders and 13 nonclinical control students. Using the Koppitz scoring system for the DAPs and an adapted Reynolds scoring system for the KF-Ds, the results indicated that none of the 30 DAP or 37 K-F-D emotional indicators differentiated any of the three clinical groups from the normal controls. Finally, these results were generally supported by Handler and Habenicht (1994), whose review of the K-F-D literature found numerous studies in which the K-F-D did not consistently discriminate between different groups of children, including those who were well-adjusted versus those with specific clinical difficulties. Overall, construct or discriminant validity is dependent on the scales or procedures used to initially separate a sample into clinical subgroups; the methodological characteristics and reliability of the projective drawing administration and scoring, respectively; and the statistical analyses used to as-

reyn2-5.qxd

6/6/2003

9:05 AM

Page 103

5. Evaluation of Projective Drawings

sess the diagnostic “hit rates” and false-negative and false-positive decisions. Despite numerous studies failing to demonstrate the discriminant validity of projective drawing tests, Handler and Habenicht’s (1994) recommendation that future studies use a holistic, integrated scoring approach rather than a specific, indicator-by-indicator approach is notable. In the end, more validity research using this integrated approach is needed before conclusive statements can be made. The third type of validity, incremental validity, has focused on projective drawings’ ability to facilitate more accurate diagnostic and clinical judgments when used in conjunction with other assessment tools and approaches in a stepwise decision-making process. To date, few studies have demonstrated this type of validity with projective drawings, and some studies have reported that various clinicians were unable to discriminate clinically unremarkable versus clinically identified populations beyond the chance level with projective drawings (Cummings, 1986). Despite the absence of more recent incremental validity research, Hammer (1997) still defends their use and minimizes the results of the previous research, noting (1) that projective drawings provide hypotheses that are more descriptive than diagnostic, especially when many clinical groups share similar psychological characteristics and behavioral patterns; and (2) that projective drawings should not be used independently to discriminate clinical groups but as part of an integrated assessment battery involving numerous objective and other diagnostic approaches. Thus, the psychometric debate with respect to projective drawings continues. Some researchers decry their lack of psychometric integrity, whereas others argue that projective drawings should not and cannot be fairly evaluated in a traditional statistical and psychometric manner. This argument aside, the hypothesis-generating use of projective drawings and their ability to be interpreted within various psychological orientations remains both viable and defensible. Moreover, as noted previously, the generation of hypotheses tied to specific drawing characteristics is discouraged in favor of a more integrated, composite approach to projective drawing analysis and interpreta-

103

tion. This latter approach, however, still must be validated in the future research. To assist in the overall analysis and evaluation of the individual studies that make up the research with projective drawings, a review of many of the studies most often cited in the projective drawing literature are shown in Appendices 5.1 and 5.2 in this chapter. These tables analyze the predominant interpretations of projective drawing techniques and the BGT according to their experimental designs, their statistical and methodological characteristics, their dependent variables and results, and their generalizability to other clinical situations in the field. The first part of each of these appendices reviews the research prior to 1990, while the second part of each reviews the research after 1990. Although exhaustive descriptions and evaluations of the studies in Appendices 5.1 and 5.2 are beyond the scope of this chapter, some descriptive analyses of these studies are possible as one way to analyze the state of projective research to date. Within the projective drawing studies described in the first part of Appendix 5.1 (i.e., prior to 1990), 50% used ex post facto designs, 37% used descriptive “designs,” 9% used experimental designs, and 4% used case-study designs. Sixty-eight percent of the studies did no systematic type of sampling, whereas 22% used some type of randomized sampling and 10% used some type of matched sampling. Seventy-two percent of the studies did not use control groups, whereas 28% did; 56% analyzed their data using nonparametric statistics, whereas the other 44% used some type of parametric analysis. Finally, in an external analysis, 20% of the studies were deemed to have good generalizability to other samples, 51% of the studies were considered to have fair generalizability, and 29% of the studies were felt to have limited or no generalizability. Of the 12 projective drawing studies reviewed since 1990, 75% used ex post facto designs, 33% used some type of control or comparison group with some type of systematic sampling (no study used random sampling), 75% calculated interrater reliability (one calculated internal reliability), 83% analyzed their data using parametric statistics, and only one study was considered to have good generalizability. Notably, of the newer studies, one study did a litera-

reyn2-5.qxd

6/6/2003

104

9:05 AM

Page 104

II. PROJECTIVE METHODS

ture review of the K-F-D research (Handler & Habenicht, 1994) and one completed a meta-analysis of projective tests with sexually and nonsexually abused children (West, 1998). Of the pre–1990 BGT studies described in Appendix 5.2, 53% used ex post facto designs, 44% used descriptive “designs,” and 3% used experimental designs, and no casestudy designs were reviewed. Sixty-three percent of the studies did no systematic type of sampling, whereas 22% used some type of matched sampling and 15% used some type of randomized sampling. Fifty-three percent of the studies used control groups, whereas the other 47% did not; 62% analyzed their data using parametric statistics, whereas the other 38% used some type of nonparametric analysis. Finally, in the same type of external analysis, 22% of the studies were deemed to have good generalizability to other samples, 51% of the studies were considered to have fair generalizability, and 27% of the studies were felt to have limited or no generalizability. Significantly, only two BGT studies were found after 1990, one with subjects ranging from 15 to 65 years old (Maloney & Wagner, 1991) and one with male adolescents in a residential treatment center (Nyfield & Patalano, 1998). Neither of these studies had good generalizability to broader populations. It is important to note the significant number of descriptive “designs” used in both the projective drawing and BGT studies, even though they were outnumbered by the ex post facto designs. Even more notable is the dearth of experimental studies, a gap that points to the need for far more sophisticated research in the future—research that should especially target the validity and clinical utility of projective drawings. Furthermore, the studies’ general lack of appropriate sampling and control groups and their dependence on the less robust nonparametric statistical analyses suggest that research on projective drawings has yet to fully evaluate their real potential and their actual impact. Finally, the relatively small percentage of studies considered to have good generalizability indicates that much of the projective drawing research cannot be used for differential diagnosis, or even for validation of specific clinical characteristics within a referred individual or an identified

group. This reinforces the use of projective drawings as hypothesis-generating tools rather than hypothesis-validating tools. This use and perspective are further explicated later. Two final points, however, remain. First, it is interesting to note that the BGT studies (Appendix 5.2) used significantly more control groups and parametric statistical analyses than did the projective drawing studies, although this may be changing given the post-1990 studies (Appendix 5.1). Although this may be due to the BGT’s more explicit and investigated scoring systems, which have been standardized and normed around the country, the real reasons are not otherwise apparent. Second, few if any of the studies reviewed used multitrait–multimethod designs. This is a significant flaw that again, in summary, points out the relative weakness of current projective drawing research. For the future, the areas of potential and needed research are both apparent and unlimited. Projective drawings cannot be evaluated on the basis of the present research; only after a great number of experimentally sound studies have been completed can these assessment tools and techniques be fairly critiqued.

INTEGRATING PROJECTIVE DRAWINGS INTO COMPREHENSIVE PERSONALITY ASSESSMENT When projective drawings are being integrated into a comprehensive personality assessment, the primary goals of such an evaluation need to be considered. Elsewhere (Knoff, 2002), in discussing the comprehensive personality assessment process and its goals, I have identified two critical conceptualizations of that process: (1) personality assessment is a hypothesis-testing, problemsolving process that works within a probability model; and (2) personality assessment uses a multitrait, multisetting, multimethod assessment approach, which is sensitive to the ecological and reciprocally determined nature of personality and behavior. Personality assessment, in its most basic form, exists to functionally evaluate (e.g., O’Neill et al., 1997) children who have been referred for some behavioral, affective, intrapersonal, and/or interpersonal difficulty that threatens some critical domain or facet of

reyn2-5.qxd

6/6/2003

9:05 AM

Page 105

5. Evaluation of Projective Drawings

their development (e.g., social-emotional, academic, and cognitive). The process involves systematic problem solving as it attempts, first, to identify and functionally analyze the primary referred (and other relevant) problems from a multifaceted, ecological perspective, and second, to intervene in the problem with appropriate and effective interventions. Evaluation of these identification, analysis, and intervention components occurs throughout as a way to maintain both the integrity and the utility of the entire process. During problem analysis, the psychologist evaluates the referred problem and situation, using multitrait, multisetting, multimethod assessment approaches to generate hypotheses that best explain and facilitate an understanding of the significant variables that are causing, supporting, maintaining, and/or encouraging the problem. Once these hypotheses are identified and refined, the analysis process continues at a higher level as the hypotheses are tested, using the most objective, empirically proven methods, for their validity and reliability across people, settings, time, and ecological circumstances. Confirmed hypotheses that explain why a referred problem occurs (e.g., the child exhibits anxiety because he was once left home alone during a power loss) then links directly to interventions (e.g., relaxation therapy and/or systematic desensitization), while rejected hypotheses necessitate continued problem analysis. Given their more clinical nature (both in the research and in actual practice), and their focus on children’s covert processes, projective drawings probably are best used to generate hypotheses about a referral situation rather than to validate those hypotheses. Indeed, at times, projective drawings may be unnecessary in the assessment battery, because other tools or techniques can identify and test the relevant hypotheses more validly and effectively. Within the personality assessment process, problem analysis is deemed successful when a functional understanding of a child’s referred problem is reached and interventions to address the precipitating problem conditions or events are identified. In most cases, and in the face of their limited clinical validity, this suggests a limited role for projective drawings in this process.

105

After the referred problem has been analyzed to the extent that its primary determinants or contingencies are clear and the necessary interventions are apparent, the process of psychological change can begin. Although the link between assessment and intervention is logical and methodologically necessary, the presence of unassessed and unknown intervening variables can decrease the predictive validity of intervention success. Thus, personality assessment in general and the development of effective interventions in particular work within the context of probability. That is, given thorough and well-conceived analyses, we expect that our explanations of referred problems have the highest probability of accuracy, and that our interventions based on these analyses have the highest probability of success. At times, however, intervening variables (e.g., a teacher’s covert lack of commitment to the intervention) are missed, and the efficiency and efficacy of the process are diminished. In most cases, the intervening and disruptive variables will be identified through formative evaluations of the ineffective intervention program. In other cases, the intervening variable may be the intervention program itself, and the decrease in program effectiveness may indicate that some facet of the original referred problem has been solved and that a different problem, unaffected by the intervention, has taken its place. In both cases, the intervention must be reconceptualized, and the assessment-to-intervention link must be forged anew. At this juncture, problem analysis and intervention may focus only on observable and measurable behavior. Once again, beyond suggesting additional explanative hypotheses, projective drawings may be of little assistance at this level. To summarize, personality assessment is a process, not a product. The real product, the primary goal of the process, is the treatment or resolution of referred behavioral or social-emotional problems so that a child’s normal development and positive mental health can continue. Thus, it is not enough to describe or understand a child’s socialemotional problems; we must move from problem analysis to intervention by using this understanding. On a different level, given the aforementioned goals of personality assessment, it should be apparent that not

reyn2-5.qxd

6/6/2003

106

9:05 AM

Page 106

II. PROJECTIVE METHODS

all personality assessment evaluations with a referred child necessitate the use of projective tests, or, more specifically, projective drawings. In fact, psychologists should use only those personality assessment tools or techniques that are needed (1) to fully and validly identify and analyze the significant variables that are causing, supporting, maintaining, and/or encouraging the problem; and (2) to fully and validly identify those interventions that will effectively and efficiently resolve these significant variables and the referred problem. In most cases, these necessary and sufficient personality assessment tools and techniques will involve those that are behaviorally based, objective, standardized, and/or well researched. Projective drawings, then, may be used when a referred problem situation is particularly complex, when an in-depth evaluation of a child’s intrapersonal and cognitive-behavioral status is needed, and/or when behavioral and objective assessments have not provided an understanding of the problem situation that makes intervention success highly probable. Projective drawings can be important to the personality assessment process. They should be used strategically, however, and not randomly; they should be used to maximize problem understanding and treatment effectiveness, not to cloud the issues with unnecessary redundancies or irrelevancies. On a more pragmatic level, when used in the personality assessment battery, projective drawing techniques do provide potentially unique information and samples of behavior that are unavailable through other approaches, be they behavioral, objective, or anecdotal. From a cognitive-behavioral perspective, projective drawings use administration approaches and test-related stimuli that allow children to introduce their own significant, egocentric beliefs, attitudes, expectations, or attributions into the assessment session. These cognitions generally are not elicited by the more standardized techniques, which are often so structured and so specific in the personality domains they are intended to evaluate that they either suppress children’s more personal self-reflections and intricacies or assess in ways that appear not to relate to a child’s actual or perceived difficulties. Projective drawings are among a handful of techniques that do

not fully structure the assessment interaction, providing instead an opportunity for children to communicate, in their own way, the specific issues that are troubling them. In a sense, then, projective drawing techniques may be most similar to unstructured or semistructured clinical interviews, except that the psychologist initially asks the child to draw (or, in a sense, to discuss visually) some event or facet of his or her life. Later, during the Inquiry process, the child can describe or discuss the drawing and, with effective clinician rapport and interview skills, may reveal important personality- or behaviorally related issues. To expand on this idea briefly, the clinical interview can be just as “projective” as the projective drawings. On one hand, some interview questions are more direct, and the interpretation of a child’s “direct” responses may appear to be self-evident and objective. However, the child still can decide whether or not to answer these direct questions honestly or completely. Thus, like the projective drawings, interview responses may only generate hypotheses that need additional, external validation. On the other hand, many interview questions are purposely global and ambiguous, in the hope that they will trigger some important memory, emotional reaction, or relevant response by the child, and/or so that the child has the opportunity to respond with personal issues that are of real and current concern. Again, this is similar to the projective drawings, except that the drawings use a visual modality as compared to the interview’s verbal modality. To continue, and as noted, the success of the clinical interview in eliciting information of importance often depends on the rapport and trust between the child and the interviewer, and the ability of the interviewer to ask the right questions at the right time. The success of projective drawings is dependent on the same factors. Finally, and somewhat parenthetically, it is interesting to note that clinical interviews suffer the same potential problems with respect to reliability and validity as do projective drawings, yet they seem to be more often included within the personality assessment battery because the psychologist somehow “has more control” over the interview. I would suggest that this logic is faulty, but not in order to recommend that the clinical

reyn2-5.qxd

6/6/2003

9:05 AM

Page 107

5. Evaluation of Projective Drawings

interview be purged from our assessment procedures. Clearly, the clinical interview is an important, almost irreplaceable, procedure. Projective drawings, in certain cases, may be just as important; they cannot be categorically dismissed from the assessment battery, and their use should be at least considered for referrals that require face-to-face testing with the referred child. Although space considerations preclude a description of every projective drawing technique—its administration and scoring procedures, its interpretive approaches, and its integration into the comprehensive personality assessment battery—a few generalizations are important. First, projective drawings can be analyzed from both structural and content perspectives. Specific structural characteristics of a drawing (e.g., actions of and between drawn figures; characteristics of specific figures; the position, distance between, and barriers between figures; the drawing style; and symbols or objects present in the drawing) are interpreted depending on the psychological orientation being used for analysis. These interpretations are evaluated both within drawings and across drawings, such that recurrent themes and issues are identified. These recurrent themes, and not the random and isolated ones, are integrated into the broader diagnostic picture of the referred child, which includes all other assessments and pieces of data, all multisetting evaluations, and all multisource observations. These themes are what generate the projective hypotheses for further study and validation. Overall, however, given the research reviewed earlier, there appears to be little, if any, consistent empirical support for a structural approach to projective drawing analysis and interpretation. Those structural indicators that have some interpretive support are generally idiosyncratic to the studies in which they are cited, and they do not, figuratively, account for enough variance in explaining a child’s difficulties to warrant a structural analysis for all other indicators. For those indicators that have little interpretive support, while additional research can proceed, it should focus on clusters or composites of indicators that can contribute to a functional analysis of a child’s affect or behavior that will then link to effective intervention. At present, then, the structural

107

analysis of projective drawings is discouraged. The content analysis is related to the second important generalization: the importance of an Inquiry process during the projective drawing administration. Projective drawings are administered in two phases. The Performance phase involves providing the child with the materials needed to complete the drawings and with the actual directions needed to proceed. After the drawing is completed, the Inquiry phase begins, using a series of questions aimed at (1) clarifying the objects and actions in the actual drawing and (2) eliciting a broader description or story about the drawing, which expands into a semistructured interview focusing on the child’s underlying decisions and cognitions in choosing to draw what was completed. The child’s responses to the inquiry-phase questions are used in the content analysis. The content analysis has the same goal as the structural analysis: to generate hypotheses that contribute to a comprehensive understanding of the referred child. Although the source of the data differs from the structural analysis, the content analysis should also evaluate themes within and across projective drawings and integrate only recurrent themes in the broader assessment and interpretation process. In some ways, the content analysis can be conceptualized as similar to that done with the thematic techniques (e.g., as in the Thematic Apperception Test). That is, questions about the projective drawing can be put into story format, and important details about the characters in the pictures can be ascertained. This is not necessary; what is necessary is that the inquiry process be used to clarify and expand the information in the projective drawing. This process decreases the potential for misinterpretation and increases the potential for eliciting significant cognitions and issues that are inherent in the material of the projective drawing. The third important generalization relates to training and expertise with projective drawings. The administration, scoring, and interpretation of projective drawings take a great deal of training, practice, experience, and supervision. One does not simply read “the book” or take “the course” to become “expert” in projective drawing use. Instead,

reyn2-5.qxd

6/6/2003

108

9:05 AM

Page 108

II. PROJECTIVE METHODS

the effective utilization of projective drawings is an ongoing endeavor. It would be improper to require some specific amount of training or supervision before one could use projective drawings independently. In a sense, the only criterion of readiness is the same criterion on which projective drawing interpretation should be evaluated as valid: whether or not the projective drawings elicit valid hypotheses that contribute significantly to a greater understanding of the referred child and to a subsequently successful intervention program. If a psychologist is able to use projective drawings consistently in this way under supervision, then he or she should be considered ready to use them independently. All this, however, appears far too specific to projective drawing techniques. Clearly, if projective drawings are a potential part of the comprehensive personality assessment process, then psychologists need to be expert in the entire process before practicing it independently. Personality assessment is both a content- and a processoriented skill. It is a skill that develops over time and experience. It is a practice that requires ongoing supervision and continuing education to best serve referred children in the settings in which they live and interact.

CASE STUDY: INTEGRATING AND INTERPRETING PROJECTIVE DRAWINGS This section presents a case study that demonstrates the interpretation of projective drawings and their potential integration into a comprehensive personality assessment battery. A “stream-of-consciousness” format is used throughout this discussion to show the psychologist’s thinking processes as numerous personality assessment tools and techniques were integrated and a complex case was fully analyzed. The projective drawings in this case were analyzed primarily as data providing behavioral and social-emotional hypotheses that would need to be confirmed using more objective and direct means. However, they did sample areas of personality and cognitive-behavioral functioning that other assessment mechanisms might evaluate only tangentially or in different ways. Finally, a multitrait, multisetting, multimethod assessment approach was used in this case study. The number of multiple assessments,

however, was limited to only those needed for reliable and valid evaluation of the various traits, settings, and methods unique to the referred child and situation. This number will vary from case to case and from psychologist to psychologist; with every case, however, there is a point of diminishing returns where another assessment tool or interview adds nothing new to the data or the analysis of the referred problem.

Referral and Background David, at the time of referral and evaluation, was between 15 years, 4 months and 15 years, 8 months old. An athletically built white adolescent, David was placed during this assessment period in a full-time residential program for students with learning disabilities (LDs), many of whom had associated behavioral and social-emotional difficulties. Indeed, David has recently been kicked out of school after numerous fights with other students and a serious confrontation with his high school principal, who eventually suspended him. In an initial interview, David spent a good deal of time bragging about how tough his high school was and made allusions to his possible drug use. He complained that he was weak in spelling, English, and reading; that he could do better in math, but that he did not like to hand in assignments or study for exams; and that he did best in classes in which he could listen to the course material rather than take a lot of class notes. A developmental interview with David’s mother indicated that David was an unhappy baby, that he cried constantly during his first months of life, and that he never stayed in his playpen for a long period. The mother characterized him at 9 months as a “child who constantly defied authority—when he learned to walk, he would not stop grabbing things, even when he was told.” She noted that he walked comparatively early, and that “he always amazed everybody” with what he could do. For example, she noted that he would carry on a full conversation at a young age. Specific to their disciplinary styles, the mother noted that she and her husband never agreed on how best to discipline David. She would typically spank him, carry a specific punishment through for a period of

reyn2-5.qxd

6/6/2003

9:05 AM

Page 109

5. Evaluation of Projective Drawings

time, and then give David back the privileges lost. The father would usually threaten David but not carry through. Early on, however, the mother noted that David was quite facile at manipulating her and his father. This manipulation had continued to the present day, and the mother fully acknowledged that she was tired of dealing with David, that she was ineffective in controlling and structuring him, and that the residential placement was probably best for David and her. Academically, David did appear to present a classic LD pattern. His problems began in second grade, when he was tested for reading and associated problems and considered “dyslexic.” David’s mother recalled many frustrations in trying to help David to learn how to read, especially because a significant number of behavioral problems were already surfacing, at home and in school, as a result of David’s frustration with his academic failure. In school, David was exhibiting a lot of out-of-seat behavior, daydreaming, argumentativeness, and anger. He was eventually placed in an LD program for part of the year. At the same time, however, David also was considered lazy, and his parents would put him on restriction at home because he was not doing his work in school. After the special education placement, David’s LD teacher actually blamed David and his behavior for a miscarriage that she experienced during the school year. In fourth and fifth grades, David went on to a private school, where he was provided with special training and support as a student with LD. In fact, during one of the summer programs at that school, David’s mother recalled a “phenomenal” teacher who was particularly successful with him. It was during this summer that he made the most progress, academically and emotionally. David’s mother bemoaned the fact that he had never had as talented a teacher since that summer, and that David had never felt so positive about his academic accomplishments or future. Ultimately, David’s family relocated to another state and he returned to the public schools for his middle school and early high school years. During those latter years, David’s mother noted that she lost control of him, that he was running with a crowd of underachievers who were

109

involved with drugs and continually skipped school, and that the school administration was unresponsive to both educational and behavioral needs. David’s mother saw the current program at the residential LD institution as David’s “last chance”; she felt that his failure in that program would leave him as a dropout with no educational foundation to get an appropriate job in the work force. As for his family life, David had a younger sister, but she did not seem to have any relevance to his referral problem or history. More significantly, David’s parents experienced a number of marital problems as he was growing up. In fact, they divorced when David was 10 years old but were separated only 3 months; they eventually remarried only 9 months after the divorce. David’s mother noted that her husband’s problems with alcohol were a significant reason for the divorce. She also noted that David blamed himself for the divorce and that he was very “lost” during it. Nonetheless, no one in the family was involved in any psychological counseling or support during the time immediately before or after the divorce and remarriage. In fact, the only counseling the family ever experienced occurred for only 1½ months and centered around David’s LD problems when he was in third grade. David’s mother noted that the psychologist involved was negative and tried to blame her and her husband for many of the problems they were experiencing. The psychologist’s “blame” notwithstanding, it did appear that David’s parents had unrealistic expectations for his behavior even when he was 9 months old (e.g., describing his behavior as “defying authority”), and that their inability to discipline him consistently had left David with little external structure, few explicit behavioral and interpersonal expectations, and no real consequences for either positive or inappropriate behavior. David’s parents did not fully understand his LD, nor were they prepared to deal with its academic and social-emotional characteristics. Finally, all these situations had existed for such a long time that most of the familial behavioral patterns were ingrained and intractable— both for David’s parents with respect to parenting and for David with respect to his

reyn2-5.qxd

6/6/2003

9:05 AM

110

Page 110

II. PROJECTIVE METHODS

interactions both at home and in school. At the time of the interview, it appeared that a residential program was appropriate for David, but that both he and his parents needed specific counseling support to change their patterns of behaviors and expectations so that David could eventually and successfully be reintegrated into the family and community setting. The assessment goal, then, was to gain a full understanding of David’s intrapersonal, interpersonal, academic, social-emotional, and familial behavior and self-concept, so that appropriate and comprehensive therapeutic directions could be developed and implemented.

Initial Assessments The first part of this ongoing assessment process began when David was evaluated for entrance into the residential placement at 15 years, 4 months of age. At that time, a battery of intellectual (the Wechsler Intelligence Scale for Children—Revised [WISCR]), academic achievement (the Wide Range Achievement Test, the Detroit Tests of Learning Aptitude, the Gates–MacGinitie Reading Tests, the Gilmore Oral Reading Test, the Metropolitan Achievement Tests, the Schonell Spelling Lists), processing (the Illinois Test of Psycholinguistic Abilities, the Frostig Visual Perceptual Tests), and personality tests (the Personality Inventory for Children [PIC], the Millon Adolescent Personality Inventory [MAPI]) were given. On the WISC-R, David received a Verbal Scale IQ in the average to above-average range, a Performance Scale IQ in the average range, and a Full Scale IQ in the average range (100 ± 6). On the Verbal subtests, he demonstrated consistent verbal skills—conceptually, expressively, with respect to academic information, and in arithmetic. Significantly, David achieved a scaled score of 13 on the Comprehension subtest, indicating that he understood societal expectations and appropriate behavior, even though he might not choose or might be unable to behaviorally meet those expectations. On the Performance subtests, David did extremely well on the Picture Completion and Picture Arrangement subtests, less well and average on the Block Design and Object Assembly subtests, and extremely poor-

ly on the Coding subtest. It appeared that David’s strong verbal skills were helping him to “talk himself through” and score well on these first two subtests. Thus, David might be able to control his behavior and aggressive actions through cognitive, mediational processes or through a “thinkaloud” approach if motivated to do so. His low Coding subtest (scaled score, 4) and his low Digit Span subtest (scale score, 4) suggested some significant distractibility, probably associated with his LD, and/or some significant anxiety and emotional lability. Actually, both of these hypotheses were found to be accurate. David scored in the at-risk range on the Gordon Diagnostic System, an objective attention and vigilance task that assesses for attention-deficit disorders. And the personality tests and interview with David’s mother suggested that he had significant behavioral and emotional difficulties, especially when frustrated and stressed. In the academic achievement areas, David’s spelling achievement was measured at approximately the third-grade level, his oral reading at the fourth- to early fifthgrade level, and his math skills at the fifthto early sixth-grade level. Significantly, his oral reading comprehension was at grade level or above when he was able to decode the specific words in the reading passage. In the processing area, David’s auditory memory was almost age-appropriate when the stimulus material was concrete and relevant to him, and his ability to process and follow oral directions was excellent. But when the auditory material became abstract and/or novel, David had significantly more memory problems. David’s visual memory and visual processing abilities, in contrast, were well developed. In fact, the visual modality in general appeared to be David’s stronger processing area. When this psychometric information was compared with his intellectual test results, it appeared that David was still manifesting an LD pattern: specific difficulties in reading, auditory memory, auditory processing, and spelling, and a potential attentiondeficit disorder. As for his social-emotional status, David’s history of LD and its concomitant pattern of academic and social failure had clearly influenced his personality development, and in some cases his LD

reyn2-5.qxd

6/6/2003

9:05 AM

Page 111

5. Evaluation of Projective Drawings

might be influencing how he saw and interpreted circumstances and events in his past and present worlds. Although his socialemotional status might be evaluated separately, it could not be analyzed without considering his academic history and its influence both on his self-concept and on the behaviors and expectations of his parents. This was not an attempt to explain all of David’s behavior as a part of his LD; rather, it was an attempt to recognize the complexity of his behavior and the need for a complex analysis of his difficulties and a multifaceted intervention program. David’s social-emotional status initially was evaluated with the PIC, completed by his mother, and the MAPI, completed by himself. David’s personality profile on the PIC (see Figure 5.1) indicated severely delinquent and hyperactive tendencies that were apparent to such a degree that significant antisocial, out-of-control, and irrational behavior could be expected. In addition, David was seen as an adolescent (1) who lacked appropriate interpersonal and social skills and the ability to deal with conflictproducing situations in the environment; (2) who had significantly withdrawn from his environment and who might exhibit periods of depression along with agitated, actingout behavior; (3) who felt a great deal of anxiety and distrust in his world; and (4) who continued to have significant intellectual and academic achievement problems in the midst of all the social-emotional turmoil. All but three of the PIC’s clinical scales reached significance, suggesting a disturbed, emotionally labile individual. Two of those nonsignificant scales related to David’s overall development and his lack of somatic concerns. The other scale was the Family Relations scale, a scale that assesses the relationship between family and parental variables and the child’s psychological difficulties. This result suggests (1) that many of David’s problems were thought to be specific to him, rather than related to family problems or issues; (2) that many of David’s problems influenced characteristics that did exist within the family unit; and (3) that David’s absence from the home because of his placement in the residential program was having a positive effect on the atmosphere and intrafamily relationships at home.

111

The MAPI (see Figure 5.2) reinforced many of the results from the PIC. On this scale, David was described as both fearful and apprehensive about his relationships with others, and as resentful and critical of those who had not supported him in the past. His profile reflected a person who was sensitive as to how others evaluated him as a person and predicted that David would tend to withdraw and distance himself from close involvements to avoid further rejections or disappointments. The MAPI further indicated that David revealed feelings of low self-worth, expressing a minimal degree of self-acceptance and a great deal of difficulty with academic achievement and success. The family problems and David’s perceived rejection by his parents were also evident in the profile, as were his indifference to others and his lack of empathy for others and their welfare. In total, the MAPI results predicted that David would have limited success in therapy and that he would probably withhold his innermost feelings, primarily because he would not believe that anyone could really care about him in the end.

Continuing Assessments After the initial assessments, David was accepted by the residential program and placed in an individualized LD curriculum and a dorm with his own room. Over the next 4 months, he was behaviorally monitored and discussed at monthly meetings of the clinical staff. During the first of these staff meetings, David was described as hyperactive, sneaky, and nontrusting, and as continually breaking the no-smoking rules. It was noted that he did serve consequences when they were applied, but that he had put his head through a wall of his dorm room, had made additional holes there with some metal rods, liked to stare people down, and was making no progress in his academic program. At the second meeting, this pattern was described as continuing, with additional concerns about his manipulative behavior and his constant challenging of both school and residential staff. By the third meeting, David’s belligerent attitude, lack of respect for adults, inability to work independently, and inability to be trusted during off-campus activities resulted in a call for

reyn2-5.qxd

6/6/2003

112

9:05 AM

Page 112

II. PROJECTIVE METHODS

FIGURE 5.1. David’s Personality Assessment Profile on the PIC.

reyn2-5.qxd

6/6/2003

9:06 AM

Page 113

5. Evaluation of Projective Drawings

113

FIGURE 5.2. David’s personality assessment profile on the MAPI.

additional diagnostic assessments to determine whether he was appropriate for the program or not. At this point, a more projectively oriented battery was completed to supplement the data already collected during the intake process. This battery consisted of the Rorschach, some incompletesentence blanks, the H-T-P, the Kinetic Drawing System (the K-F-D and K-S-D), and four sessions of diagnostic counseling and clinical interviews. David’s Rorschach evidenced a significantly high number of total responses; significantly high percentages of large blot area responses (D%) and Animal responses (A%); and significantly low percentages of small blot area responses (Dd%), high-qual-

ity Form responses (F+%), Animal Movement responses (FM), and Human responses (H%) as compared to same-age norms from Ogdon (1977, 1982). In addition, his responses qualitatively focused on aggressive activities, monsters, and “scary” scenes fairly often. The interpretation of the Rorschach suggested an above-average intellectual potential, anxiety, a suppression of emotions and spontaneity as a way to protect against rejection, social isolation, and an absence of empathy for others. The analysis also indicated that David would probably be a poor therapy candidate because of his inability to trust others and to overcome his feelings of vulnerability and cautiousness.

reyn2-5.qxd

6/6/2003

114

9:06 AM

Page 114

II. PROJECTIVE METHODS

David’s incomplete-sentence blank responses revealed the following issues: his desire to smoke on the grounds of the campus, his dislike for authority figures who told him what to do, his poor peer interpersonal relationships, his dislike for school and the residential school in particular, and his secret admiration for his father and his desire to graduate from high school to please his father. Some of the more pertinent sentences were as follows (the stems are in roman, David’s responses in italics): People suck—they are sometimes not the nicest things in the world. In the lower grades I was always a very bad student. Other kids annoy me. The future holds a lot for me—working with my father, marriage, owning a car. I am best when I’m good. I hate school. I wish I were at home right now. My father is cool. I secretly admire my father. My greatest worry is that I graduate from high school. After the incomplete-sentence blanks, David was asked what he would do with three wishes. He wanted (1) “to be with Sharon [his girlfriend] whenever I want,” (2) “to graduate from high school and go to college for a civil engineering degree,” and (3) to be

FIGURE 5.3. House drawing (on the H-T-P) from David’s comprehensive personality assessment battery.

“rich off of life—and I don’t mean just have money.” David’s H-T-P drawings are shown in Figures 5.3, 5.4, and 5.5, and his Kinetic Drawing System drawings in Figures 5.6 and 5.7. As each drawing was completed, a series of inquiry questions was asked to describe and clarify the objects and actions within the drawing. (These are indicated by [Q]: but are not given in full.) David’s verbatim responses to the questions were as follows: House: [Q]: “This is the house I lived in for around 2 to 3 years when I was about 7 to 10 years old.” [Q]: “This is a real house, a sketch of my old house in Tennessee.” [Q]: “It is 7 years old.” [Q]: “Me, my parents, and my sister live in the house.” [Q]: “The house is in with a group of houses.”

FIGURE 5.4. Tree drawing (on the H-T-P) from David’s comprehensive personality assessment battery.

reyn2-5.qxd

6/6/2003

9:06 AM

Page 115

5. Evaluation of Projective Drawings

FIGURE 5.5. Person drawing (on the H-T-P) from David’s comprehensive personality assessment battery.

[Q]: “It’s thinking about running into the woods and going up to this huge cave—like an airplane hanger.” Tree: [Q]: “It’s a treehouse in Tennessee— the same place as my house drawing.” [Q]: “It’s a huge oak tree—4 feet in diameter.” [Q]: “Me and my friends built the treehouse. I have good memories of this treehouse. It was in the backyard of my house on about 1 1/2 acres of land.” Person: [Q]: “This is just something that I drew—it’s a nothing, just a body—a ‘he–she–it.’” [Q]: “It’s anywhere from 5 to 90 years old.” [Q]: “It lives alone—in the middle of the Alps. It’s a hermit—it doesn’t like society—it doesn’t get along with crowds.” [Q]: “In the Alps, it hunts, fishes—the usual.” [Q]: “It’s thinking, ‘Where’s my clothes? I’m freezing.’” [Q]: “It’s feeling cold—and lonesome. Even hermits have to show some feeling. It has a dog—but it’s been dead for six years.” K-F-D: “This is a happy picture—every-

115

one is having a good time. It is a picture of us in a river near our house in Florida. We go water skiing all the time. There is my father driving, my mother sitting behind him watching my sister ski, me sitting, and my sister skiing and saying ‘Wheeee!!’ My father is saying, ‘David, get me another beer.’ I’m saying, ‘Why do you want another beer? You just had one.’” K-S-D: “This is my classroom with kids who are just doing their homework. There’s the teacher. She thinks she can control everything—she thinks she’s the dominant power. But that’s not true—we just follow her for fear of being suspended. I respect her ’cause she’s the teacher—you should respect adults.” From a content perspective, a number of social-emotional and behavioral hypotheses were generated as hypotheses that would need later, objective validation. Integrating the qualitative/content information for each projective drawing resulted in the following analyses: House: It was interesting that David chose to draw a house that he had lived in for 2–3 years in Tennessee, when he was about 7–10 years of age. Structurally, the house looked like a typical house, except that it was hard to tell whether the driveway led to a door or a garage (one of these was missing, in any case). Chronologically, this was the period when David’s family was all together, although the parents were having marital problems that led to their divorce when David was 10. Immediately after the divorce, David and his family had to move from this house in Tennessee to another state. Thus, the fairly typical structure of the house might suggest that David was secure with the nuclear family together. His comment about what the house was thinking (“It’s thinking about running into the woods and going up to this huge cave—like an airplane hanger”) might suggest that his

FIGURE 5.6. K-F-D from David’s comprehensive personality assessment battery.

reyn2-5.qxd

6/6/2003

116

9:06 AM

Page 116

II. PROJECTIVE METHODS

FIGURE 5.7. K-S-D from David’s comprehensive personality assessment battery.

parents’ fighting made him want to escape from the house. It also might suggest that he wanted to escape blame for his parents’ fighting, given his mother’s interview comment that David blamed himself for his parents’ divorce. Tree: The tree drawing helped the examiner to continue the analysis of David’s “Tennessee” years, because he chose to draw a tree that was in the backyard of his house in Tennessee. The tree had a treehouse that was built by David and his friends, and David had good memories about this treehouse. The tree appeared to be strong, perhaps indicating that David used his peers as a support group while his parents were leading up to their divorce. However, there was some shading, a shaky baseline grounding the tree, and a barrenlooking branch, collectively indicating that this peer group did not provide David sufficient support to compensate him fully (in a social-emotional sense) for the turmoil at home. In addition, these characteristics may suggest that David might have perceived this peer support as being stronger than in fact it was. The generalized barrenness of the tree also might reflect David’s current emotional coldness, his social and interpersonal aloofness, and his lack of empathy toward others. This coldness might be accentuating his memories of Tennessee as “the last time I was really happy,” thereby allowing him to ignore his current difficulties through fantasy and to blame his current situation on his parents’ divorce and his move from his home in Tennessee. Person: David’s person drawing provided

insight into how he was really feeling and into his current self-concept. First of all, the figure was “an it.” David did not feel as if he had an identity at the present time. He had conflict all around him—with his family, at the residential school, with his peer group, in the classroom. Furthermore, he had lost all sense of control over his own life: Everyone was making decisions for him, and he was caught in a downward spiral in which his attempts to reassert his control were causing the adults in his life to make more restrictive and more controlling decisions. Thus, David wanted to be all alone (“a hermit”), but this was a forced isolation. The hermit (David) did not like “society”—that is, having all the adults forcing him to conform to their societal rules and expectations—and he did not like having all the adults (both his parents and those at the residential school) working together to apply consistent expectations and consequences (the hermit “doesn’t get along with crowds”). It was a forced isolation also in the sense that David described the hermit as “lonesome—even hermits have to show some feeling.” His final comment about having a dog that had been dead for 6 years was very sad. It suggested that David really wanted to interact and share his feelings with others and that he wanted to be accepted, but the overall tone of the drawing indicated that this was a deep, underlying aspiration that would not soon be fulfilled. Finally, it was interesting that 6 years before drawing this picture, David was living in Tennessee and his parents were approaching their divorce. Perhaps, David had never emotionally recovered from that traumatic event. K-F-D: David’s family drawing included all the members of his family, but the figures were incomplete and poorly formed and could not be differentiated from one another. This suggested that David felt a significant amount of conflict among the members of his family—conflict that he focused around his father. Indeed, the potential issue of and conflict around David’s father’s alcohol abuse was readily apparent: David had the father in the drawing asking for another beer and himself rejecting the request, saying, “You just had one.” Significantly, the mother in the drawing was depicted as somewhat passive, and as an individual

reyn2-5.qxd

6/6/2003

9:06 AM

Page 117

5. Evaluation of Projective Drawings

who often got in between David and his father to act as a buffer or intermediary. Finally, David’s sister appeared to be the psychologically safest and healthiest one in the family—she was allowed to water ski behind the boat, free from the conflict within the boat, and was able to have fun. It was interesting that David described the picture as a happy picture and as one taking place in Florida. No longer was David drawing pictures about himself or his family in Tennessee. He was now focused on his more recent experiences in Florida, and he still perceived his family as his father, mother, and sister despite his being placed in the residential program. The description of the picture as happy, despite the subtle indications of conflict and confrontation, might suggest that David perceived any family interaction that had only limited conflict as “happy.” This, then, might provide a baseline as to the amount of interpersonal conflict actually in the home setting, and might suggest that David had not recently experienced a truly supportive family environment, at least while living in Florida. On a more positive note, David did include himself within the family unit. He had returned from “the Alps,” although his drawing did reflect a sense of isolation within the family unit. It might be that despite its faults, David felt most supported within his family, and that his perceptions of rejection and isolation were related more to peers, other adults, and school and community settings. K-S-D: David’s reaction to and need for control in the school setting were clearly apparent from his school drawing. His choice of the bird’s-eye view, looking omnisciently over the classroom, with the teacher’s head drawn more heavily than those of the students, suggested his need to be even more powerful than the teacher and to be more of an “overseer” or authority. And the fact that he did not identify himself in the picture almost suggested that the teacher could not force him to be at one of the desks, the way the other students were depicted. David verbally expressed this power theme and need when he stated that the teacher thought she could control everything in the classroom and that, from his perspective, she really could not. He then identified the powers that might control him most: the

117

fear of being suspended (although he did not appear to fear that in reality), and the more moralistic power or reason that students should respect teachers and adults because they are adults. (Once again, this was a power that did not seem to curtail David’s resistance to authority figures.) One interesting question not answered by the school drawing was the reason why David seemed to harbor these power issues. On the one hand, it might be that David’s acting out and controlling behavior countered his feelings of powerlessness with respect to his LD and his inability to achieve academically in school. That is, he needed to feel in control of something, and his negative behavior and ability to anger and frustrate others were the things he could best control. Or, on the other hand, it might be that David had always exhibited this negative pattern of behavior, and that the behavior had affected his academic failure and his poor peer interrelationships and acceptance. Regardless of the answer, it was clear that the issues of power and conflict were well embedded in David’s perceptions of school and family, and that these perceptions had to be considered in any intervention program that might hope to address David’s social-emotional, behavioral, and academic needs.

A Final Integration Objective tests such as the PIC and the MAPI, and behaviorally oriented measures such as behavior rating scales and behavioral observations, are clearly important to the personality assessment process; they provide reliable and valid samples of the student’s functioning intrapersonally, interpersonally, ecologically, and across multiple settings. From a diagnostic perspective, David was correctly placed in the residential setting, according to these objective and behavioral assessments. In addition, the psychoeducational assessments relative to his intellectual and academic functioning were critical in developing an appropriate educational program with reasonable demands and expectations. From a therapeutic perspective, however, the addition of the projective tests, and the drawing tests in particular, provided a more in-depth cognitive-behavioral assessment

reyn2-5.qxd

6/6/2003

118

9:06 AM

Page 118

II. PROJECTIVE METHODS

that contributed to a fuller understanding of David’s attitudes, attributions, expectations, and perceptions. Who would have expected David to draw a figure in the Alps feeling lonesome and reflecting that it didn’t like society and didn’t get along with crowds, and what personality assessment measure could have elicited a feeling that descriptive and intense other than the projective drawings? Who would have predicted that David would have addressed his need for power and control so explicitly in response to the K-S-D inquiry? Clearly, no one. But these cognitions were what completed the diagnostic picture within a multitrait, multisetting, multimethod assessment approach. Furthermore, these cognitive beliefs and perceptions might be most instrumental in creating an intervention program that addressed both David’s cognitions and his behaviors. Integrating the projective drawings with all the other assessments done with and for David resulted in a comprehensive personality picture. David had a long history of personality and adjustment difficulties, beginning with an interaction between his own behavior and his parents’ perceptions of him being a child who “defied authority” at 9 months, and extending through a diagnosis of hyperactivity and numerous failure experiences at home and at school; parents with inconsistent disciplinary styles; a parental divorce and subsequent remarriage; a suspension from school; and a move to a residential school because of his behavior and active resistance. Qualitatively, David had serious feelings of negative selfconcept, deep feelings of inadequacy and vulnerability, and a great need to be accepted by others. Behaviorally, however, he had poor impulse control, and he lacked the social and conflict resolution skills to interact appropriately with peers. Thus, he was caught in a vicious cycle: He was rejected continually by peers and adults; he withdrew from them in anger and self-protection; he increasingly resented his rejection and isolation; and then he tried to interact again, only motivated by revenge, anger, and a self-fulfilling prophesy for another social failure. This vicious cycle had played itself out continually over the past several years, such that the behavioral pattern was now occurring many times per day, and

David’s behavior was more and more out of control. David now presented himself as someone with serious conduct disorder and delinquency problems and behaviors, and as one with little internal self-control or trust in anyone who could help him to assert that self-control. David was not a good therapy prospect at the time, and unfortunately his parents were not either: They did not want to acknowledge their part in the broad, ecological problem, or their need to change their own behavioral styles and cognitive beliefs so that David could be fully reintegrated into the family system. Therapeutically, the picture of David emerging from this assessment was quite distressing. It was, however, accurate. Within 2 months of the final diagnostic assessments, David’s behavior was so far out of control that he was asked to leave the residential school. At last contact, David’s parents were looking for a residential psychiatric setting for David, still disavowing their part and responsibility in the presenting problems and the needed comprehensive therapy and intervention process.

SUMMARY This chapter has attempted to provide a comprehensive picture of projective drawing approaches: their historical and theoretical development, their psychometric and clinical strengths and weaknesses, and their use in the context of a comprehensive personality assessment battery. A case study has also been presented to exemplify the use of projective drawings in the comprehensive personality assessment process and their potential contributions to that process. There is no doubt that projective drawing techniques— indeed, projective techniques as a whole— continue to be controversial and questioned (Knoff, 1991). However, there is also no doubt that they continue to be discussed in graduate training, employed in the field, and integrated into clinical practice. Relative to the latter use, projective drawings need more sophisticated research attention and evaluation. However, they should be evaluated as part of the entire personality assessment process, not in an isolated and out-of-context way, and their use should be strategic and well considered. Projective drawings are

reyn2-5.qxd

6/6/2003

9:06 AM

Page 119

5. Evaluation of Projective Drawings

not needed in most assessments of a referred child, but when they are used, they should be used correctly and with an eye toward an accurate understanding of the referred child and an appropriate and effective intervention and service delivery program. This is the bottom line for all personality assessment: not that we complete our tests with just a better description or understanding of a child but that we implement intervention programs that have a positive, lasting impact on the children, families, and systems who are referred to us as needing emotional, behavioral, and other support.

ACKNOWLEDGMENTS I gratefully acknowledge the assistance of Dr. William Carlyon in helping to review the projective drawing literature, and for his precision and dedication in helping to develop the appendices. I also am grateful for the assistance of Carrie Finch and Amanda Denecke in helping to update the research in projective drawings for the revision of this chapter for the second edition of this volume.

REFERENCES Abell, S. C., Horkheimer, R., & Nguyen, S. E. (1998). Intellectual evaluations of adolescents via human figure drawings: An empirical comparison of two methods. Journal of Clinical Psychology, 54, 811–815. Abell, S. C., Von Briesen, P. D., & Watz, L. S. (1996). Intellectual evaluations of children using human figure drawings: An empirical investigation of two methods. Journal of Clinical Psychology, 52, 67–74. Adler, P. T. (1970). Evaluation of the figure drawing technique: Reliability, factorial structure, and diagnostic usefulness. Journal of Consulting and Clinical Psychology, 35, 52–57. Aikman, K. G., Belter, R. W., & Finch, A. J. (1992). Human figure drawings: Validity in assessing intellectual level and academic achievement. Journal of Clinical Psychology, 48, 114–120. Albee, G. S., & Hamlin, R. M. (1949). An investigation of the reliability and validity of judgments of adjustment inferred from drawings. Journal of Clinical Psychology, 5, 389–392. Anderson, B., & Rallis, K. (1981). Relationship between Bender errors, emotional indicators and performance on Bender recall. Perceptual and Motor Skills, 53, 497–498. Baroff, G. S. (1957). Bender–Gestalt visuo-motor function in mental deficiency. American Journal of Mental Deficiency, 61, 753–760. Beck, H. S. (1959). A comparison of convulsive organics, non-conclusive organics, and non-organic public

119

school children. American Journal of Mental Deficiency, 63, 866–875. Bender, L. (1938). A visual motor gestalt test and its clinical use (American Orthopsychiatric Association Research Monograph No. 3). New York: American Orthopsychiatric Association. Bensberg, G. (1952). Performance of brain-injured and familial mental defectives on the Bender–Gestalt Test. Journal of Consulting Psychology, 16, 61–64. Berman, S., & Laffal, J. (1953). Body type and figure drawing. Journal of Clinical Psychology, 9, 368–370. Bieliauskas, V. J. (1960). Sexual identification in children’s drawings of human figure. Journal of Clinical Psychology, 16, 42–44. Billingslea, F. Y. (1948). The Bender–Gestalt: An objective scoring method and validating data. Journal of Clinical Psychology, 4, 1–27. Blaha, J., Fawaz, N., & Wallbrown, F. (1979). Information processing components of Koppitz errors on the BVMGT. Journal of Clinical Psychology, 35, 784–790. Bodwin, R. F., & Bruck, M. (1960). The adaptation and validation of the Draw-A-Person test as a measure of self concept. Journal of Clinical Psychology, 16, 427–429. Bradfield, R. H. (1964). The predictive validity of children’s drawings. California Journal of Educational Research, 15, 166–174. Breen, M. J., & Butler, L. (1983). Applicability of Bender–Gestalt emotional indicators for emotionally disturbed and nonreferred students. Psychological Reports, 52, 569–570. Britain, S. D. (1970). Effect of manipulation of children’s affect on their family-drawings. Journal of Projective Techniques and Personality Assessment, 34, 234–237. Buck, J. N. (1948). The H-T-P technique, a qualitative and quantitative method. Journal of Clinical Psychology, 4, 317–396. Buck, J. N. (1970). The House–Tree–Person technique: Revised manual. Los Angeles, CA: Western Psychological Services. Burns, R. C. (1987). Kinetic House–Tree–Person Drawings (K-H-T-P): An interpretive manual. New York: Brunner/Mazel. Burns, R. C. (1990). A guide to family-centered circle drawings (F-C-C-D) with symbol probes and visual free association. New York: Brunner/Mazel. Burns, R. C., & Kaufman, S. H. (1970). Kinetic Family Drawings (K-F-D): An introduction to understanding children through kinetic drawings. New York: Brunner/Mazel. Burns, R. C., & Kaufman, S. H. (1972). Actions, styles, and symbols in Kinetic Family Drawings (K-F-D). New York: Brunner/Mazel. Burton, A., & Sjoberg, B. (1964). The diagnostic validity of human figure drawings in schizophrenia. Journal of Psychology, 57, 3–18. Byrd, E. (1956). The clinical validity of the Bender– Gestalt Test with children: A developmental comparison of children in need of psychotherapy and children judged well-adjusted. Journal of Projective Techniques, 20, 127–136. Campbell, D. T., & Fiske, D. W. (1959). Convergent

reyn2-5.qxd

6/6/2003

120

9:06 AM

Page 120

II. PROJECTIVE METHODS

and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56, 81–105. Carlson, K., Quinlan, D., Tucker, G., & Harrow, M. (1973). Body disturbance and sexual elaboration factors in figure drawings of schizophrenic patients. Journal of Personality Assessment, 37, 56–63. Cassell, R. H., Johnson, A. P., & Burns, W. H. (1958). Examiner, ego defense, and the H-T-P test. Journal of Clinical Psychology, 14, 157–160. Chase, J. M. (1941). A study of the drawings of a male figure made by schizophrenic patients and normal subjects. Character and Personality, 9, 208–217. Chorost, S., Spivack, G., & Levine, M. (1959). Bender–Gestalt rotations and EEG abnormalities in children. Journal of Consulting Psychology, 23, 559. Craddick, R. A. (1963). The self-image in the Draw-APerson Test and self-portrait drawings. Journal of Projective Techniques, 27, 288–291. Craddick, R. A. (1964). Size of drawings-of-a-person as a function of simulating “psychosis.” Perceptual and Motor Skills, 18, 308. Craddick, R. A., & Leipold, W. D. (1968). Note on the height of Draw-A-Person figures by male alcoholics. Journal of Projective Techniques and Personality Assessment, 323, 486. Cummings, J. A. (1986). Projective drawings. In H. M. Knoff (Ed.), The assessment of child and adolescent personality (pp. 199–244). New York: Guilford Press. Curnutt, R. H. (1953). The use of the Bender–Gestalt with an alcoholic and non-alcoholic population. Journal of Clinical Psychology, 9, 287–290. Datta, L., & Drake, A. (1968). Examiner sex and sexual differentiation in preschool children’s figure drawings. Journal of Projective Techniques and Personality Assessment, 32, 397–399. Delatte, J. G. (1985). Significance of femininity in human figure drawings of girls. Psychological Reports, 56, 165–166. Delatte, J. G., & Hendrickson, N. J. (1982). Human figure drawing size as a measure of self-esteem. Journal of Personality Assessment, 46, 603–606. DeMartino, M. F. (1954). Human figure drawings by mentally retarded males. Journal of Clinical Psychology, 10, 241–244. Dunleavy, R. A., Hansen, J. L., & Szasz, C. W. (1981). Early kindergarten identification of academically not-ready children by use of human figure drawing developmental score. Psychology in the Schools, 18, 35–38. Exner, J. E. (1962). A comparison of the human figure drawings of psychoneurotics, character disturbances, normals, and subjects experiencing experimentally-induced fear. Journal of Projective Techniques, 26, 392–397. Fabian, A. A. (1945). Vertical rotation in visual–motor performance: Its relationship to reading reversals. Journal of Educational Psychology, 36, 129–154. Fellows, R., & Cerbus, G. (1969). HTP and DCT indicators of sexual identification in children. Journal of Projective Techniques and Personality Assessment, 3, 376–379. Fiedler, F. E., & Siegel, S. M. (1949). The Free Drawing Test as a predictor of nonimprovement in psy-

chotherapy. Journal of Clinical Psychology, 5, 386–389. Fisher, G. M. (1961). Nudity in human figure drawing. Journal of Clinical Psychology, 17, 307–308. Fisher, G. (1968). Human figure drawing indices of sexual maladjustment in male felons. Journal of Projective Techniques and Personality Assessment, 32, 81. Fuller, J. B., & Chagnon, T. (1962). Factors influencing rotation in the Bender–Gestalt performance of children. Journal of Projective Techniques, 26, 36–46. Gavales, D., & Millon, T. (1960). Comparison of reproduction and recall size deviations in the Bender–Gestalt as measures of anxiety. Journal of Clinical Psychology, 16, 278–280. Gobetz, W. (1953). A quantification, standardization and validation of the Bender–Gestalt Test on norm and neurotic adults. Psychological Monographs, 67(356). Goldberg, F. H. (1956–1957). The performance of schizophrenic, retarded, and normal children on the Bender–Gestalt Test. American Journal of Mental Deficiency, 61, 548–555. Goldfried, M. R., & Ingling, J. (1964). The connotative and symbolic meaning of the Bender–Gestalt. Journal of Projective Techniques, 28, 185–191. Goldstein, A. P., & Rawn, M. L. (1957). The validity of interpretive signs of aggression in the drawing of the human figure. Journal of Clinical Psychology, 13, 169–171. Goldstein, H. S., & Faterson, H. F. (1969). Shading as an index of anxiety in figure drawings. Journal of Projective Techniques and Personality Assessment, 33, 454–456. Goodenough, F. L. (1926). Measurement of intelligence by drawings. New York: Harcourt, Brace & World. Goodenough, F. L. (1931). Children’s drawings. In C. Murchison (Ed.), A handbook of child psychology (pp. 480–514). Worcester, MA: University Press. Goodman, M., & Kotkov, B. (1953). Prediction of trait ranks from Draw-A-Person measurements of obese and non-obese women. Journal of Clinical Psychology, 9, 365–367. Goodstein, L. D., Spielberger, C. D., Williams, J. E., & Dahlstrom W. G. (1955). The effects of serial position and design difficulty on recall of the Bender– Gestalt Test designs. Journal of Consulting Psychology, 19, 230–234. Graham, S. R. (1956). A study of reliability in human figure drawings. Journal of Projective Techniques, 20, 385–386. Granick, S., & Smith, L. I. (1953). Sex sequence in the Draw-A-Person Test and its relation to the MMPI Masculinity-Femininity scale. Journal of Consulting Psychology, 17, 71–73. Gravitz, M. A. (1966). Normal adult differentiation patterns on the figure drawing test. Journal of Projective Techniques and Personality Assessment, 30, 471–473. Gravitz, M. A. (1967). Marital status and figure drawing choice in normal adults. Journal of Projective Techniques and Personality Assessment, 31, 86–87.

reyn2-5.qxd

6/6/2003

9:06 AM

Page 121

5. Evaluation of Projective Drawings Gravitz, M. A. (1968). The height of normal adult figure drawings. Journal of Clinical Psychology, 24, 75. Gravitz, M. A. (1969a). Direction of psychosexual interest and figure drawing choice. Journal of Clinical Psychology, 25, 311. Gravitz, M. A. (1969b). Marital status and figure drawing choice in normal older Americans. Journal of Social Psychology, 77, 143–144. Gravitz, M. A. (1971). Nudity and amount of clothing in the figure drawings of normal adults. Journal of Genetic Psychology, 118, 141–145. Gray, D. M., & Pepitone, A. (1964). Effect of self-esteem on drawings of the human figure. Journal of Consulting Psychology, 28, 452–455. Green, R., Fuller, M., & Rutley, B. (1972). It-scale for children and Draw-A-Person test: 30 feminine vs. 25 masculine boys. Journal of Personality Assessment, 36, 349–352. Griffith, A. V., & Peyman, D. A. R. (1959). Eye–ear emphasis in the DAP as indicating ideas of reference. Journal of Consulting Psychology, 23, 560. Griffith, R. M., & Taylor, V. H. (1960). Incidence of Bender–Gestalt figure rotations. Journal of Consulting Psychology, 24, 189–190. Griffith, R. M., & Taylor, V. H. (1961). Bender– Gestalt figure rotations: A stimulus factor. Journal of Consulting Psychology, 25, 89–90. Groth-Marnat, G., & Roberts, L. (1998). Human Figure Drawings and House Tree Person drawings as indicators of self-esteem: A quantitative approach. Journal of Clinical Psychology, 54, 219–222. Guertin, W. (1952). A factor analysis of the Bender– Gestalt tests of mental patients. Journal of Clinical Psychology, 8, 362–367. Guertin, W. (1954a). A factor analysis of curvilinear distortions on the Bender–Gestalt. Journal of Clinical Psychology, 10, 12–17. Guertin, W. (1954b). A transposed analysis of the Bender–Gestalts of brain disease cases. Journal of Clinical Psychology, 10, 366–369. Guertin, W. (1954c). A transposed factor analysis of schizophrenic performance on the Bender–Gestalt. Journal of Clinical Psychology, 10, 225–228. Guertin, W. (1955). A transposed analysis of the Bender–Gestalts of paranoid schizophrenics. Journal of Clinical Psychology, 11, 73–76. Hackbarth, S. G., Murphy, H. D., & McQuary, J. P. (1991). Identifying sexually abused children by using kinetic family drawings. Elementary School Guidance and Counseling, 255–260. Hain, J. D. (1964). The Bender–Gestalt Test: A scoring method for identifying brain damage. Journal of Consulting Psychology, 28, 34–40. Halpin, V. (1955). Rotation errors made by brain injured and familial children on two visual motor tests. American Journal of Mental Deficiency, 59, 485–489. Hamilton, L. (1984). Human figure drawings as measures of self-concept development in bilingual children. Journal of Instructional Psychology, 11, 28–36. Hammer, E. F. (1953a). An investigation of sexual symbolism: A study of H-T-P’s of eugenically steril-

121

ized subjects. Journal of Projective Techniques, 17, 401–413. Hammer, E. F. (1953b). Negro and white children’s personality adjustment as revealed by a comparison of their drawings (H-T-P). Journal of Clinical Psychology, 9, 7–10. Hammer, E. F. (1954a). A comparison of H-T-P’s of rapists and pedophiles. Journal of Projective Techniques, 18, 346–354. Hammer, E. F. (1954b). An experimental study of symbolism on the Bender–Gestalt. Journal of Projective Techniques, 18, 335–345. Hammer, E. F. (Ed.). (1958). The clinical application of projective drawings. Springfield, IL: Charles C Thomas. Hammer, E. F. (1969). DAP: Back against the wall? Journal of Consulting and Clinical Psychology, 33, 151–156. Hammer, E. F. (1980). The clinical application of projective drawings. Springfield, IL: Charles C Thomas. Hammer, E. F. (Ed.). (1997). Advances in projective drawing interpretation. Springfield, IL: Charles C Thomas. Handler, L. (1985). The clinical use of the Draw-APerson Test (DAP). In C. S. Newmark (Ed.), Major psychological assessment instruments (pp. 165– 216). Boston: Allyn & Bacon. Handler, L., & Habenicht, D. (1994). The Kinetic Family Drawing Technique: A review of the literature. Journal of Personality Assessment, 62, 440– 464. Handler, L., & Reyher, J. (1964). The effects of stress on the Draw-a-Person Test. Journal of Consulting Psychology, 28, 259–264. Handler, L., & Reyher, J. (1966). Relationship between GSR and anxiety in projective drawings. Journal of Consulting Psychology, 30, 60–67. Hanvik, L. J., & Andersen, A. L. (1950). The effect of focal brain lesions on recall and on the production of rotations in the Bender–Gestalt Test. Journal of Consulting Psychology, 14, 197–198. Harris, D. B. (1963). Children’s drawings as a measure of intellectual maturity. New York: Harcourt, Brace & World. Heberlein, M., & Marcuse, F. L. (1963). Personality variables in the DAP. Journal of Consulting Psychology, 27, 461. Helgert, L. D. (1985). A graphic analysis Bender– Gestalt Test. Journal of Clinical Psychology, 41, 505–511. Hellkamp, D. T., & Hogan, M. E. (1985). Differentiation of organics from functional psychiatric patients across various IQ ranges using the Bender–Gestalt and Hutt Scoring System. Journal of Clinical Psychology, 41, 259–264. Hibbard, R. A., & Hartman, G. L. (1990). Emotional indicators in human figure drawings of sexually victimized and nonabused children. Journal of Clinical Psychology, 46, 211–219. Hiler, E. W., & Nesvig, D. (1965). An evaluation of criteria used by clinicians to infer pathology from figure drawings. Journal of Consulting Psychology, 29, 520–529. Hinkle, J. A. (1983). Comparison of reproduction on

reyn2-5.qxd

6/6/2003

122

9:06 AM

Page 122

II. PROJECTIVE METHODS

the Bender–Gestalt and Memory-for-Design for delinquents and non-delinquents. Perceptual and Motor Skills, 57, 1070. Holmes, C. B., & Stephens, C. L. (1984). Consistency of edging on the Bender–Gestalt, Memory-forDesign, and Draw-a-Person Test. Journal of Psychology, 117, 269–271. Holtz, R., Moran, P.W., & Brannigan, G. G. (1986). Social schemas in the Kinetic Family Drawings of young adults. Journal of Social Psychology, 126, 689–690. Holzberg, J. D., & Wexler, M. (1950). The validity of human form drawings as a measure of personality deviation. Journal of Projective Techniques, 14, 343–361. Hoyt, T. E., & Baron, M. R. (1959). Anxiety indices in same-sex drawings of psychiatric patients with high and low MAS scores. Journal of Consulting Psychology, 23, 448–452. Hulse, W. C. (1951). The emotionally disturbed child draws his family. Quarterly Journal of Child Behavior, 3, 152–174. Hulse, W. C. (1952). Child conflict expressed through family drawings. Quarterly Journal of Child Behavior, 16, 66–79. Hutt, M. L., & Monheit, S. (1985). Effectiveness of the Hutt Adaptation of the Bender–Gestalt Test configuration scale in differentiating emotionally disturbed adolescents. Psychological Reports, 56, 439–443. Hutton, J. B., Dubes, R., & Muir, S. (1992). Assessment practices of school psychologists: Ten years later. School Psychology Review, 21, 271–284. Jensen, K. W. (1985). Sex of the administrator as a variable affecting draw-a-person. Perceptual and Motor Skills, 60, 72–74. Jernigan, A. J. (1967). Rotation style on the Bender– Gestalt Test. Journal of Clinical Psychology, 23, 176–179. Johnson, J. H. (1971). Upper left hand placement of human figure drawings as an indicator of anxiety. Journal of Personality Assessment, 35, 336–337. Johnson, J. H. (1973). Bender–Gestalt constriction as an indicator of depression in psychotic patients. Journal of Personality Assessment, 37, 53–55. Jolles, I. (1952a). A study on the validity of some hypotheses for the qualitative interpretation of the HT-P for children of elementary school age: I. Sexual identification. Journal of Clinical Psychology, 8, 113–118. Jolles, I. (1952b). A study of the validity of some hypotheses for the qualitative interpretation of the HT-P for children of elementary school age: II. The “phallic tree” as an indicator of psychosexual conflict. Journal of Clinical Psychology, 8, 245–255. Jolles, I., & Beck, H. S. (1953). A study of the validity of some hypotheses for the qualitative interpretation of the H-T-P for children of elementary school age: IV. Vertical placement. Journal of Clinical Psychology, 9, 164–167. Jordan, S. (1970). Projective drawings in a cerebellar disorder due to chicken pox encephalitis. Journal of Projective Techniques and Personality Assessment, 34, 256–258. Judson, A. J., & MacCasland, B. (1960). A note on the

influence of the season on tree drawings. Journal of Clinical Psychology, 16, 171–173. Kamano, D. K. (1960). An investigation on the meaning of human figure drawing. Journal of Clinical Psychology, 16, 429–430. Kennedy, M. L., Faust, D., Willis, W. G., & Piotrowski, C. (1994). Social-emotional assessment practices in school psychology. Journal of Psychoeducational Assessment, 12, 228–240. Knoff, H. M. (1991). The utility of human figure drawings in personality and intellectual assessment: Why ask why? School Psychology Quarterly, 8, 191–196. Knoff, H. M. (2002). Best practices in personality assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology: III (pp. 1281–1302). Washington, DC: National Association of School Psychologists. Knoff, H. M., & Prout, H. T. (1985). The Kinetic Drawing System: Family and school. Los Angeles, CA: Western Psychological Services. Koppitz, E. M. (1958). The Bender–Gestalt Test and learning disturbances in young children. Journal of Clinical Psychology, 14, 292–295. Koppitz, E. M. (1960). The Bender–Gestalt Test for children: A nonnative study. Journal of Clinical Psychology, 16, 432–435. Koppitz, E. M. (1962). Diagnosing brain damage in young children with the Bender–Gestalt Test. Journal of Consulting Psychology, 26, 541–545. Koppitz, E. M. (1963). The Bender–Gestalt Test for young children. New York: Grune & Stratton. Koppitz, E. M. (1966a). Emotional indicators on human figure drawings of children: A validation study. Journal of Clinical Psychology, 22, 313–315. Koppitz, E. M. (1966b). Emotional indicators on human figure drawings and school achievement of first and second graders. Journal of Clinical Psychology, 22, 481–483. Koppitz, E. M. (1966c). Emotional indicators on human figure drawings of shy and aggressive children. Journal of Clinical Psychology, 22, 466–469. Koppitz, E. M. (1968). Psychological evaluation of children’s human figure drawings. New York: Grune & Stratton. Koppitz, E. M. (1984). Psychological evaluation of human figure drawings by middle school pupils. Orlando, FL: Grune & Stratton. Kotkov, V., & Goodman, M. (1953). Draw-A-Person tests of obese women. Journal of Clinical Psychology, 9, 362–364. Kurtzberg, R., Cavior, N., & Lipton, D. (1966). Sex drawn first and sex drawn larger by opiate addict and non-addict inmates on the Draw-A-Person Test. Journal of Projective Techniques and Personality Assessment, 30, 55–58. Lachmann, F. M. (1960). Perceptual–motor development in children retarded in reading ability. Journal of Consulting Psychology, 24, 427–431. Laird, J. (1962). A comparison of male normals, psychiatric patients and alcoholics for sex drawn first. Journal of Clinical Psychology, 18, 302. Lakin, M. (1956). Certain formal characteristics of human figure drawings by institutionalized aged and

reyn2-5.qxd

6/6/2003

9:06 AM

Page 123

5. Evaluation of Projective Drawings by normal children. Journal of Consulting Psychology, 20, 471–474. Lehner, G. F., & Gunderson, E. K. (1948). Height relationships in DAP test. Journal of Personality, 17, 199–209. Lewinsohn, P. M. (1964). Relationship between height of figure drawings and depression in psychiatric patients. Journal of Consulting Psychology, 28, 380–381. Lyons, J. (1955). The scar on the H-T-P tree. Journal of Clinical Psychology, 11, 267–270. Mabry, M. (1964). Serial projective drawings in a patient with a malignant brain tumor. Journal of Projective Techniques, 28, 206–209. Machover, K. (1949). Personality projection in the drawing of the human figure. Springfield, IL: Charles C Thomas. Maloney, P., & Wagner, E. E. (1991). Predicting normal age-related changes with intelligence, projective, and perceptual–motor test variables. Perceptual and Motor Skills, 71, 1225–1226. Marzolf, S. S., & Kirchner, J. H. (1970). Characteristics of House–Tree–Person drawings by college men and women. Journal of Projective Techniques and Personality Assessment, 34, 138–145. Marzolf, S. S., & Kirchner, J. H. (1972). House–Tree– Person drawings and personality traits. Journal of Personality Assessment, 36, 148–165. McCormick, T. T., & Brannigan, G. G. (1984). Bender Gestalt signs as indicants of anxiety, withdrawal, and acting-out behavior in adolescents. Journal of Psychology, 118, 71–74. McHugh, A. (1963). Sexual identification, size, and associations in children’s figure drawings. Journal of Clinical Psychology, 19, 381–382. McHugh, A. (1966). Children’s figure drawings in neurotic and conduct disturbances. Journal of Clinical Psychology, 22, 219–221. McPhee, J. P., & Wegner, K. W. (1976). Kinetic Family Drawing styles and emotionally disturbed childhood behavior. Journal of Personality Assessment, 40, 487–491. Melikian, L. H., & Wahab, A. Z. (1969). First-drawn picture: A cross-culture investigation of the DAP. Journal of Projective Techniques and Personality Assessment, 33, 539–541. Meyer, B. C., Brown, F., & Levine, A. (1955). Observations on the House–Tree–Person drawing test before and after surgery. Psychosomatic Medicine, 17, 428–454. Michal-Smith, H. (1953). The identification of pathological cerebral function through the H-T-P technique. Journal of Clinical Psychology, 9, 293–295. Modell, A. H. (1951). Changes in human figure drawing by patients who recover from regressed states. American Journal of Orthopsychiatry, 21, 584–596. Modell, A. H., & Potter, H. W. (1949). Human figure drawing of patients with arterial hypertension, peptic ulcer, and bronchial asthma. Psychosomatic Medicine, 11, 282–292. Mogar, R. E. (1962). Anxiety indices in human figure drawings. Journal of Consulting Psychology, 26, 101. Moll, R. P. (1962). Further evidence of seasonal influ-

123

ences on tree drawings. Journal of Clinical Psychology, 18, 109. Moore, C. L., & Zarske, J. A. (1984). Comparison of Native American Navajo Bender–Gestalt performance with Koppitz and SOMPA norms. Psychology in the Schools, 21, 148–153. Mosher, D. L., & Smith, J. P. (1965). The usefulness of two scoring systems for the Bender–Gestalt Test for identifying brain damage. Journal of Consulting Psychology, 29, 530–536. Mostkolf, D. L., & Lazarus, P. J. (1983). The Kinetic Family Drawing: The reliability of an objective scoring system. Psychology in the Schools, 20, 16–20. Naglieri, J. A. (1988). Draw-A-Person: A quantitative scoring system. San Antonio, TX: Psychological Corporation. Naglieri, J. A. (1991). Draw-A-Person: Screening procedure for emotional. San Antonio, TX: Psychological Corporation. Nathan, S. (1973). Body image in chronically obese children as reflected in figure drawings. Journal of Personality Assessment, 37, 456–463. Neale, M. D., & McKay, M. F. (1985a). Predicting early school achievement in reading and handwriting using major “error” categories from the Bender–Gestalt Test for young children. Perceptual and Motor Skills, 60, 647–654. Neale, M. D., & McKay, M. F. (1985b). Scoring the Bender–Gestalt Test using the Koppitz Developmental System: Interrater reliability, item difficulty, and scoring implications. Perceptual and Motor Skills, 60, 627–636. Nyfield, B., & Patalano, F. (1998). Effects of serial position on Bender–Gestalt errors using Koppitz’s criteria. Psychological Reports, 83, 1227–1247. Oas, P. (1984). Validity of the Draw-A-Person and Bender–Gestalt Test as measures of impulsivity with adolescents. Journal of Consulting and Clinical Psychology, 52, 1011–1019. Ogdon, D. P. (1977). Psychodiagnostics and personality assessment: A handbook (2nd ed.). Los Angeles, CA: Western Psychological Services. Ogdon, D. P. (1982). Handbook of psychological signs, symptoms, and syndromes. Los Angeles, CA: Western Psychological Services. O’Neill, R. E., Horner, R. H., Albin, R. W., Sprague, J. R., Storey, K., & Newton, J. S. (1997). Functional assessment and program development for problem behavior: A practical handbook. Pacific Grove, CA: Brooks/Cole. Ottenbacher, K., Haley, D., Abbott, C., & Watson, P. J. (1984). Human figure drawing ability and vestibular processing dysfunction in learning disabled children. Journal of Clinical Psychology, 40, 1084–1088. Paine, P., Alves, E., & Tubino, P. (1985). Size of human figure drawing and Goodenough–Harris scores of pediatric-oncology patients: A pilot study. Perceptual and Motor Skills, 60, 911–914. Peek, R. M. (1953). Directionality of lines in the Bender–Gestalt Test. Journal of Consulting Psychology, 17, 213–216. Pollitt, E., Hirsch, S., & Money, J. (1964). Priapism, impotence and human figure drawing. Journal of Nervous and Mental Disease, 139, 161–168.

reyn2-5.qxd

6/6/2003

124

9:06 AM

Page 124

II. PROJECTIVE METHODS

Prout, H. T., & Celmer, D. S. (1984). A validity study of the Kinetic School Drawing technique. Psychology in the Schools, 21, 176–180. Prout, H. T., & Phillips, P. D. (1974). A clinical note: The Kinetic School Drawing. Psychology in the Schools, 11, 303–306. Quast, W. (1961). The Bender–Gestalt: A clinical study of children’s records. Journal of Consulting Psychology, 25, 405–408. Reznikoff, M., & Tomblen, D. (1956). The use of human figure drawings in the diagnosis of organic pathology. Journal of Consulting Psychology, 20, 467–470. Robins, C. E., Blatt, S. J., & Ford, R. Q. (1992). Changes in human figure drawings during intensive treatment. Journal of Personality Assessment, 57, 477–497. Rosen, A., & Boe, E. E. (1968). Frequency of nude figure drawings. Journal of Projective Techniques and Personality Assessment, 32, 483–485. Saracho, O. N. (1984). The Goodenough–Harris drawing test as a measure of field-dependence/independence. Perceptual and Motor Skills, 59, 887–892. Sarbaugh, M. E. (1983). Kinetic Drawing-School (KSD) technique. Illinois School Psychologists Association Monograph Series, 1, 1–70. Sattler, J. M. (1990). Assessment of children (3rd ed.). San Diego, CA: Sattler Associates. Schildkraut, M., Shenker, I., & Sonnenblick, M. (1972). Human figure drawings in adolescence. New York: Brunner/Mazel. Schubert, D. S. P. (1969). Decrease of rated adjustment on repeat DAP tests apparently due to lower motivation. Journal of Projective Techniques and Personality Assessment, 33, 34. Schulberg, H., & Tolor, A. (1962). The “meaning” of the Bender–Gestalt Test designs to psychiatric patients. Journal of Projective Techniques, 26, 455–461. Short-DeGraff, M. A., Slansky, L., & Diamond, K. E. (1989). Validity of preschoolers’ self-drawings as an index of Human Figure Drawing performance. Occupational Therapy Journal of Research, 9, 305– 315. Sobel, H., & Sobel, W. (1976). Discriminating adolescent male delinquents through the use of Kinetic Family Drawings. Journal of Personality Assessment, 40, 91–94. Stawar, T. L., & Stawar, D. E. (1989). Kinetic Family Drawings and MMPI diagnostic indicators in adolescent psychiatric inpatient. Psychological Reports, 65, 143–146. Stinnett, T. A., Havey, J. M., & Oehler-Stinnett, J. (1994). Current test usage by practicing school psychologists: A national survey. Journal of Psychoeducational Assessment, 12, 331–350. Storv, R. I. (1960). The revised Bender–Gestalt and male alcoholics. Journal of Projective Techniques, 24, 186–193. Strumpfer, D. J. W. (1963). The relation of Draw-APerson test variables to age and chronicity in psychotic groups. Journal of Clinical Psychology, 19, 208–211. Suczek, R. F., & Klopfer, W. G. (1952). Interpretation

of the Bender–Gestalt Test: The associative value of the figures. American Journal of Orthopsychiatry, 22, 62–75. Taylor, R. L., Kaufman, D., & Partanio, A. (1984). The Koppitz developmental scoring system for the Bender–Gestalt: Is it developmental? Psychology in the Schools, 21, 425–428. Taylor, S. A., Kymissis, P., & Pressman, M. (1998). Prospective kinetic family drawing and adolescent mentally ill chemical abusers. Arts in Psychotherapy, 25, 115–124. Tharinger, D. J., & Stark, K. D. (1990). A qualitative versus quantitative approach to evaluating the Draw-A-Person and Kinetic Family Drawing: A study of mood- and anxiety-disorder children. Psychological Assessment, 2, 365–375. Thomas, A. D. (1984). Bender scores and the horse as a distinct item on Object Assembly on the WISC. Perceptual and Motor Skills, 59, 103–106. Tolor, A. (1957). Structural properties of Bender– Gestalt Test associations. Journal of Clinical Psychology, 13, 176–178. Tolor, A. (1958). Further studies on the Bender– Gestalt Test and the Digit-span test as measures of recall. Journal of Clinical Psychology, 14, 14–18. Tolor, A. (1960). The “meaning” of the Bender– Gestalt Test designs: A study in the use of the semantic differential. Journal of Projective Techniques, 24, 433–438. Trahan, D., & Stricklin, A. (1979). Bender–Gestalt emotional indicators and acting-out behavior in young children. Journal of Personality Assessment, 43, 365–375. Vane, J., & Eisen, V. (1962). The Goodenough DrawA-Man Test and signs of maladjustment in kindergarten children. Journal of Clinical Psychology, 18, 276–279. Vroegh, K. (1970). Lack of sex-role differentiation in preschoolers’ figure drawings. Journal of Projective Techniques and Personality Assessment, 34, 38–40. Wadsworth, B. J. (1996). Piaget’s theory of cognitive and affective development: Foundations of constructivism. White Plains, NY: Longman. Wagner, E. E., & Murray, A. Y. (1969). Bender– Gestalts of organic children: Accuracy of clinical judgment. Journal of Projective Techniques and Personality Assessment, 33, 240–242. Wallbrown, F. H., & Fremont, T. (1980). The stability of Koppitz scores on the Bender–Gestalt for reading disabled children. Psychology in the Schools, 17, 181–184. Watkins, C. E., Campbell, V. L., Nieberding, R., & Hallmark, R. (1995). Contemporary practice of psychological assessment by clinical psychologists. Professional Practice: Research and Practice, 26, 54–60. Weider, A., & Noller. P. (1950). Objective studies of children’s drawings of human figures: I. Sex awareness and socio-economic level. Journal of Clinical Psychology, 6, 319–325. Weider, A., & Noller, P. (1953). Objective studies of children’s drawings of human figures: II. Sex, age, intelligence. Journal of Clinical Psychology, 9, 20–23. West, M. M. (1998). Meta-analysis of studies assessing

reyn2-5.qxd

6/6/2003

9:06 AM

Page 125

5. Evaluation of Projective Drawings the efficacy of projective techniques in discriminating child sexual abuse. Child Abuse and Neglect, 22, 1151–1166. Wiener, G. (1966). The Bender–Gestalt Test as a predictor of minimal neurologic deficit in children eight to ten years of age. Journal of Nervous and Mental Disease, 143, 275–280. Wildman, R. W. (1963). The relationship between knee and arm joints on human figure drawings and paranoid trends. Journal of Clinical Psychology, 19, 460–461. Wisotsky, M. (1959). A note on the order of figure drawings among incarcerated alcoholics. Journal of Clinical Psychology, 15, 65. Woods, W. A., & Cook, W. E. (1954). Proficiency in drawing and placement of hands in drawings of the human figure. Journal of Consulting Psychology, 18, 119–121. Worden, M. (1985). A case study comparison of the

125

Draw-A-Person and Kinetic Family Drawing. Journal of Personality Assessment, 49, 427–433. Wright, D., & DeMers, S. T. (1982). Comparison of the relationship between two measures of visual– motor coordination and academic achievement. Psychology in the Schools, 19, 473–477. Wu, Y., Rogers, B. J., & Searight, H. R. (1991). The reliability of the draw-a-house task under different instructional conditions. Psychology—A Quarterly Journal of Human Behavior, 28, 33–38. Wysocki, B. A., & Whitney, E. (1965). Body image of crippled children as seen in Draw-A-Person test behavior. Perceptual and Motor Skills, 21, 499–304. Yates, A., Beutler, L. E., & Crago, M. (1985). Drawings by child victims of incest. Child Abuse and Neglect, 9, 183–189. Zolik, E. S. (1958). A comparison of the Bender–Gestalt reproductions of delinquents and non-delinquents. Journal of Clinical Psychology, 14, 24–26.

reyn2-5.qxd

APPENDIX 5.1. A Methodological Summary and Analysis of Projective Interpretations for Selected Drawing Techniques

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

Studies Prior to 1990 N

Psychiatric patients of varying diagnoses, 18–65 yr old

Subjected 32 scoring criteria to factor analysis

Yielded one large factor (Maturity of Body Image Concept) and three small factors; diagnostic categories were not differentiated by factors

—

—

P

2

Albee & Hamlin (1949)

EPF

10

N

Individuals with normal to severe psychological problems

HFDs rated by 15 clinicians in pairwise comparisons; clinicians were asked to pick betteradjusted individual from each pair

Ratings for clinicians split into two groups correlated .955 (reliability); rank-order correlation between rank by case records and rank by ratings on HFD = .624 (validity)

—

—

P NP

0

Beck (1959)

EPF

843

Y

805 normal children, 5–6 yr old; 25 organic MH children, 6–19 yr old; 13 nonorganic MH children

House drawing of H-T-P: recognizable or not, bizarre appearance, details present/or absent

Signif. diff. between MH children and normals in recognizabilities and omissions; organic, nonorganic MH children were not diff.; 5- and 6-yr-old normals signif. diff. on omissions

—

M?

NP

1

Berman & Laffal (1953)

EPF

88

N

Neuropsychiatric male patients

Subjects body-typed (endomorphic, mesomorphic, ectomorphic) and compared on whether selfdrawing matched actual body type or not

Correlations between drawing and body types were signif.—patients did tend to draw themselves as they were

Body type rating r = .73

—

P

0

Bieliauskas (1960)

EPF

1,000

N

Normal school children, 5–14 yr old; 50 male and 50 female for each age

Tested hypothesis that there would be no signif. diff. between boys and girls at various age levels in drawing figure as male or female

Males tended to draw males, females drew females; tendency increased with age

—

—

NP

2

Page 126

216

9:06 AM

126

Adler (1970) DESC

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Children 10–17 yr old; 30 male and 30 female

Compared DAP scores High correlation between based on 13 characteristics ratings and scales thought to be associated with self-concept to clinical ratings of self-concept based on interviews

—

—

P

0

Bradfield (1964)

EPF

50

Y

Chronic schizophrenic women

Percentage of total height represented by the head

Signif. diff. (p < .01)

—

—

P NP

1

Bradfield (1964)

EPF

85

Y

Children identified by teachers as acting out, withdrawn, overachievers, underachievers, normals (n = 17 in each group)

Compared groups on size, location, and degree of implied movement in DAPs

No signif. diff.

—

—

P NP

0

Britain (1970)

EPF

64

Y

Middle-class nursery school children

Four groups compared on family drawings: control, control free play, play therapy, play session designed to reduce self-esteem

Signif. diff. among groups on such things as sequence of self figure, colors, area of family figures, elaboration; predictable by emotional–evocative level induced

90% agreement

R

P

2

Burton & Sjoberg (1964)

EPF

98

Y

49 schizophrenic females, 49 normal females

Judges sorted random HFDs into two stacks to determine hit rate and identify their criteria for selection; also tested within method, anatomical distortions, and a checklist of 67 items

Trained clinicians were able to discriminate between schizophrenics and normals, but no clear agreement on characteristics used to distinguish

Interjudge r = .79

—

NP

1

Carlson, Quinlan, Tucker, & Harrow (1973)

DESC

59

N

Psychiatric patients; 28 schizophrenics, 11 personality-disordered, 8 neurotic, 13 mixed

Factor analysis of 14 specific features of HFD along with global ratings of sophistication and artistic skill; correlated results with other measures of body image disturbance and psychopathology

Two factors, Body Disturbance (BD) and Sexual Elaboration (SE); BD signif. correlated with artistic skill and sophistication; SE correlated with pathological thinking

Interrater r = .79

—

P

1

6/6/2003

N

reyn2-5.qxd

60

Page 127

(continues)

9:06 AM

127

Bodwin & DESC Bruck (1960)

reyn2-5.qxd

APPENDIX 5.1. Continued

Results

Scoring reliability

Sampling Stats

Generalizabilitya

EXP

130

N

White applicants for employment

Compared H-T-Ps of appli- Same comparisons statisticants where examiner recally but not practically mained in room with those signif. where examiner was not present during drawing

r > .90 on 16 pairs

—

P

1

Chase (1941) EPF

150

Y

50 schizophrenic, 50 hebephrenics, 50 normal male adults

Compared on Goodenough Schizophrenics had signif. drawings lower scores than normals; other variables, such as age, level of education, duration of psychosis, and mental age did not differentiate

—

—

P

1

9:06 AM

Cassel, Johnson, & Burns (1958)

Variables

46 (23 pairs)

N

Fifth-grade students and college sophomores

Compared DAP and selfportrait on size, no. of same-sex drawings, position on page. no. of subjects’ correct pairing of pictures by judges

Signif. relationship of size and grades (p < .05); more females drew opposite sex on DAP (ANOVAs); most maintained same position on two drawings

95% agreement

—

P NP

0

Craddick (1964)

EXP

86

N

College males

Subjects asked to draw a person, draw themselves, and draw a person the way someone who is “crazy” would

“Crazy” drawings signif. larger than other two, suggesting expansiveness as a perceived characteristic of craziness

NA

—

P

0

Craddick & Leipold (1968)

DESC

200

N

Male alcoholics

Compared size of male vs. female figure drawing

Male drawings signif. smaller (t = 3.4, p < .01)

NA

NA

P

1

Datta & DESC Drake (1968)

939

N

Head Start children: 487 males, 452 females aged 3 yr to 6 yr, 11 mo

Sex differentiation of DAP figure

Girls drew more sexdifferentiated DAP, but dependent on sex of examiner

49 out of 50 agreements

R?

NP

2

Delatte (1985)

38

NA

38 females, 16–18 yr old

Compared femininity ratings of HFDs with selfesteem on Rosenberg SelfEsteem Scale

Small but signif. correlation (.31) between femininity of HFD figures and selfesteem

Interjudge (on 38 drawings) r = .97

—

P

0

Page 128

EPF

128

Craddick (1963)

DESC

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Explored relationship between size (height, width, area) of HFD and Rosenberg Self-Esteem Scale scores

DeMartino (1954)

DESC

100

N

MH males, mean age 19.5 yrs

DeMartino (1954)

EPF

74

N

Dunleavy, DESC Hansen, & Szasz (1981)

141

Exner (1962) EXP

80

No signif. correlations for females; signif. correlation between self-esteem and HFD width and area for males (only .35 for width); some restriction of range (mostly high scores) in selfesteem scores

NA

—

P

0

DAPs analyzed on 39 char- Most drew male first; more acteristics and which sex than 75% had mouth drawn first open, front view, feet/ shoes, large head, nose, arms, generally poor proportions, standing; fewer than 25% had mouth closed, teeth, arms perpendicular to body

—

—

NP

1

37 homosexual MHs; 37 straight MHs

DAPs analyzed as above

Homosexuals had signif. more eyelashes and high heels

—

—

NP

0

NA

Kindergarten students in six randomly selected schools representing three SES levels

Tested Koppitz scores of HFD in predicting school readiness of Kindergarten children (measured by Metropolitan Readiness Test)

42% of “nonready” children identified; 10% false positives, so HFD was a reasonable predictor

—

R?

P

2

Y

Psychoneurotics, character-disturbed, normals, and group experiencing induced fear conditions (n = 20 in each group); attributes not clearly defined

Compared groups on line pressure, sketchiness, shading movement, profiles, buttons, feet, holding object, using bottom edge of page as baseline

Character-disturbed group used lighter pressure, more sketchiness, more shading than the rest; psychoneurotics used more unbroken lines. Altogether, 6 of 10 variables differentiated pathological groups from each other and other groups; nothing really conclusive, though

—

—

P

0

Page 129

Male and female high school seniors

9:06 AM

NA

6/6/2003

76

reyn2-5.qxd

129

Delatte & DESC Hendrickson (1982)

(continues

reyn2-5.qxd

Authors

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

278

N

Enrollment of parochial elementary grade school

Compared drawing and Drawing Completion Test (DCT); six variables of H-T-P between males and females

DCT scores not signif. diff. for males vs. females up to age 12; signif. diff. between males and females on sex of figure drawn first on HIP; strongest correlations at ages 11, 12, and 13

Only DCT reported; r = .89

—

P

1

Fiedler & EPF Siegel (1949)

46

N

15 “improved” psychotherapy clients, 19 “unimproved” clients; all male veterans

Used characteristics of Free Drawing Test to predict success or nonsuccess of therapy

Unimproved patients received signif. lower (poorer) scores on criteria for formation of head

92% agreement

—

NA

0

Page 130

EPF

9:06 AM

Fellows & Cerbus (1969)

Type Number of of Control design subjects (Y/N) Population sampled

6/6/2003

APPENDIX 5.1. Continued

130

Fisher (1961)

DESC

1,154

N

White male adolescents jailed for delinquent behavior

Degree of nudity in figure drawings

Subjects drawing female first had signif. more nudity; low numbers of drawings contained nudity overall

—

NA

NP

1

Fisher (1968)

DESC

1,000

N

Male felons

Nudity in DAP, sex of figure drawn first

Drew male figure first less often than reported for normals in literature; twice as many adolescents indicated some nudity compared to adults

—

—

NP

2

A. P. Goldstein & Rawn (1957)

EXP

39

Y

Male and female attendants in state mental hospital

Analyzed pre- and postHFDs for control and experimental groups for increased aggression level by announcing they must work longer hours for same pay

No diff. in line pressure (no. of carbons imprinted) or figure size; other qualitative diff. were signif. (symbolic representation of aggression)

—

—

NP

0

reyn2-5.qxd

23

Y

Normal males

Shading differences between subjects’ HFDs following high- and lowstress films

Signif. more shading following high-stress films

Interrater r = .90

—

NP

0

Goodman & Kotkov (1953)

DESC

8

Y

Obese vs. nonobese women (n = 4 in each group)

Rank-order correlations between judges’ ordering of drawing and ordering based on scoring criteria

Signif. no. of high correlations, but also many nonexistent correlations

NA

R

NP

1

9:06 AM

Graham (1956)

EXP

23

N

Graduate students: 12 males, 11 female

Compared subjects’ initial figure drawings and their drawings following lectures emphasizing negative characteristics inferred from certain aspects of figure

Very few changes from first to second drawing

—

—

P

0

Granick & EPF Smith (1953)

571

N

Male and female undergraduates

Compared sex drawn first on DAP to masculinity– femininity scale of MMPI

Most drew own sex first, but males signif. more than females; no relation between MMPI scores and sex drawn first

—

—

P

1

Gravitz (1966)

DESC

2,000

N

Normal adults; 1,088 males, 912 females

Drawing of same-sex, opposite-sex, or undifferentiated-sex figures

76% drew same sex, 21% opposite, 3% undiff.; more than twice as many women drew opposite sex

NA

R Percentages only

2

Gravitz (1967)

DESC

800

N

Normal adults, 20–30 yr old; half single, half married

Sex of drawing

No differences between married people and singles on whether drawing was same-sex

NA

R

NP

2

Gravitz (1968)

DESC

200

N

100 male and 100 female job applicants

General characteristics of HFDs compared

Males, 85% same-sex, 15% opposite-sex; females, 67% same-sex; no diff. in mean heights of figures

NA

—

P

1

(continues)

Page 131

EXP

6/6/2003

131

H. S. Goldstein & Faterson (1969)

reyn2-5.qxd

APPENDIX 5.1. Continued

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

EPF

200

N

Male and female adult job applicants divided into four groups: males with high Masculinity scores on the MMPI; males with low scores; females with high scores; females with low scores

Compared groups on drawing of same-sex figures

High-masculinity males, 85% signif.; lowmasculinity males, 78%; both female groups were identical

NA

—

NP

1

Gravitz (1969b)

EPF

469

N

328 men, 141 women; 40–60 yrs old

Compared HFDs of married people and singles for percentage of same-sex drawings

Males drew same sex signif. more than females, but no diff. between male or female married people or singles

NA

—

NP

1

Gravitz (1971)

EPF

1,000

N

500 normal male adults, 500 normal female adults; 20–50 yrs old, job applicants

Compared on drawing of opposite or same sex, degree of nudity

No diff. on fully clothed same-sex figures; males drew more fully clothed females; males and females no diff. on opposite- and same-sex nude figures

By joint agreement

R

P NP

2

Gray & Pepitone (1964)

EXP

88

Y

College students: 25 high self-esteem (HSE), 25 low self-esteem (LSE), 38 controls

HSE took personality battery and received a report stating that they made unusually favorable scores; LSE got opposite report; controls didn’t get report before doing HFD; groups compared on figure size, placement on page, emotional tone, activity level

LSE group drew smaller figures, but not signif.; LSE had signif. more isolated and smaller figures; HSE more similar to controls

Interjudge agreement 88–94%

R

P

1

Green, Fuller, & Rutley (1972)

EPF

55

Y

30 “feminine” boys, 4–10 yrs old (judged to like dressing in girls’ clothes); 25 control normals

Compared groups on sex drawn first in DAP; subjects matched on age, sex of children in family, and marital status and SES of parents

Feminine boys drew girl first (57%); controls drew boy first (76%)

NA

M

NP

1

9:06 AM Page 132

132

Gravitz (1969a)

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

18 male mental patients selected on overemphasis of eyes and ears on DAP; 58 controls

Compared eye–ear emphasis to whether clinically judged “ideas of reference” were present

Signif. more of eye–ear group had ideas of reference

—

—

NP

0

DESC

55

NA

Male and female kindergarten through third-grade children in bilingual education programs (three groups; n’s = 16, 17, and 22)

Compared HFD sample protocols across grades for presence of indicators of self-concept

ANOVAs revealed signif. diff. in growth of selfconcept during kindergarten and remained stable through third grade

3 judges’ agreement by discussion

R

P

1

Hammer (1953a)

EPF

40

Y

Normal controls and eugenically sterilized males (n = 20 in each group)

Looked at H-T-Ps before and after operation (controls had other types of surgery)

Signif. diffs. suggested more genital symbolism and feelings of castration in sterilized group

—

—

NP

0

Hammer (1953b)

EPF

400

N

148 black children and 252 white children in first through eighth grades

Clinical judgments of H-T-P on 6-point scale from 0 (well adjusted) to 6 (psychotic); whites and blacks compared

Blacks got higher overall adjustment scores; whites and blacks got closer together as age increased

Among three judges, r = .90

—

P

1

Hammer (1954a)

EPF

64

N

All sex offenders: 31 adult rapists, 33 pedophiles

Compared H-T-Ps: age of tree, dead or alive, age of people, male or female figures

Pedophiles drew signif. younger trees; pedophiles drew signif. older females

NA

—

P

0

Handler & Reyher (1964)

EXP

57

Y

21 nonstressed control undergraduates; 36 stressed undergraduates

Compared DAPs for Hoyt– Signif. diff. Baron scoring of anxiety indices; stressed subjects were hooked up to a polygraph in a small, dimly lighted room with experimenter looking over their shoulders

Percentage of agreement between two judges ranged from 67% to 100%

—

NP

1

Handler & Reyher (1966)

DESC

96

N

Male college students

Compared drawings of male, female, and automobile on GSR and Hoyt–Baron scoring indicators of anxiety

67%–97% agreement

R

P NP

1

Auto yielded lowest measures of anxiety; female yielded highest measure of anxiety; low but signif. correlations between GSR and 10 of 23 graphic indicators

(continues)

Page 133

Hamilton (1984)

9:06 AM

Y

6/6/2003

76

reyn2-5.qxd

EPF

133

Griffith & Peyman (1959)

reyn2-5.qxd

APPENDIX 5.1. Continued

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

N

Four groups of college females: those drawing same-sex figures on two DAPs; opposite-sex figures on two DAPs; oppositethen same-sex DAP figures; same- then opposite-sex DAP figures

Compared groups on Aggression, Heterosexuality, and Abasement scales of Edwards Personal Preference Schedule

Greater need for aggression in females who drew males

NA

R

P

1

Hiler & Nesvig (1965)

EPF

60

Y

30 adolescent psychiatric patients; 30 normal adolescents

Compared on criteria scores of DAP by six psychologists and eight nonpsychologists to determine valid criteria for differentiation

Valid criteria for pathology were bizarre, distorted, incomplete, and transparent characteristics; criteria for normality were happy expression and no pathological characteristics present; nonpsychologists discriminated just as well as psychologists

—

R

NA

1

Holtz, Moran, & Brannigan (1986)

EPF

146

N

American college students; 50 male, 96 female

K-F-D: measured length, activity, strength of figures; which parent figure was closest to self figure

Signif. number of females — drew self closer to mother figure; signif. size diff. for females: father biggest, next mother, then self; only men signif. diff. between figures for strength/activity: self highest, then father, then mother

—

P

0

Holzberg & Wexler (1950)

EPF

108

Y

Control group, 78 female student nurses; experimental group, 38 schizophrenic women

Comparisons between groups on checklist of 174 drawing variables

Signif. diffs. on 27 variables

Scored only items on which judges agreed

M?

NP

1

Hoyt & EPF Baron (1959)

112

N

Female psychiatric patients

Subjects divided into two groups according to high or low Manifest Anxiety Scale (MAS)

Placement and size of drawing signif. related to MAS score but not eight other indicators

Mean absolute difference between

—

NP

1

9:06 AM

160

Page 134

134

Heberlein & EPF Marcuse (1963)

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

175

N

College students randomly assigned into mixed male and female groups

DAP administered by either male or female examiner; groups compared by sex on sex of first drawing

No diff. for male or female subjects according to sex of examiner when DAP administered in mixed-sex groups

NA

R

NP

2

Johnson (1971)

DESC

103

N

College students

Compared on IPAT Anxiety scale scores and anxiety indicator of upper lefthand placement of DAP

Signif. relationship: more upper left-hand placement as IPAT score higher

—

—

NP

1

Jolles (1952a)

DESC

8,500

N

Children in Illinois public schools, 5–12 yrs old

Drawing of same-sex person (H-T-P)

Younger children drew opposite sex more than older; girls drew opposite sex more often than boys

NA

R?

NP

2

Page 135

Jolles (1952b)

DESC

2,701

N

Same as Jolles (1952a)

Phallic-looking trees on H-T-P

More common in younger children; became more psychosexually signif. with age; more common in females

—

R?

NP

2

Jolles & Beck (1953)

DESC

2,083

N

Same as Jolles (1952a)

Horizontal placement of drawing as indicator of intellectual control over affect

Supported Buck’s hypothesis that psychological center is to the left of geometric center; normal range varied with age

NA

R?

P

2

Jordan (1970)

CS

1

N

Child 9 yrs old with cerebellar disorder

Analyzed HFD and Bender– Case study: Drawings Gestalt Test drawing “floating in space”

NA

NA

NA

0

Judson & DESC MacCasland (1960)

240

N

Psychology patients (mixed diagnoses)

Foliage present or absent on trees of H-T-P drawings drawn over the four seasons of the year

Signif. more bare trees drawn in winter by females but not males

Only 1 disagreement in 240 protocols

NA

NP

1

Kamano (1960)

45

N

Institutionalized schizophrenic women

HFDs rated by subjects as to whether they were most like: ideal self, actual self, least-liked self; these compared to selfratings on semantic differential scales

Signif. more rated drawing most like actual self; supported idea that HFD is a perception of drawer’s self

NA

NA

P

1

DESC

6/6/2003

EXP

9:06 AM

reyn2-5.qxd

135

Jensen (1985)

(continues)

reyn2-5.qxd

APPENDIX 5.1. Continued

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

161

N

100 “good students,” 61 “poor students”; first and second grade (by Metropolitan Achievement Test)

Analyzed HFDs on 30 Koppitz emotional indicators (EIs)

Five EIs were signif. more often present in poorstudent group, including poor integration of parts, slanting

—

—

NP

1

Koppitz (1966b)

EPF

152

Y

76 children from guidance clinic; 76 normal school children

Compared HFDs on 30 EIs

11 EIs signif. more frequent in clinic group

95% agreement on 25 protocols

M

NP

1

Koppitz (1966c)

EPF

62

N

Guidance clinic children; 31 identified as aggressive, 31 identified as shy

Compared HFDs on 30 EIs

Asymmetry of limbs, presence of teeth, long arms, big hands were present signif. more often in aggressive group; hands cut off, no mouth were more frequent in shy group

—

M

NP

1

Kotkov & Goodman (1953)

EPF

55 (pilot); 56 (experiment)

N

61 obese women, 40 normal-weight women; all matched on age, education, IQ, marital status, and employment status

Compared groups on 43 scoring items on the DAP

32 signif. diff. measures in pilot group led to seven combined signs that differentiated obese from normal women’s drawings; primarily related to obese using more of page

—

M

NP

2

Kurtzberg, Cavior, & Lipton (1966)

EPF

125

Y

“Normal” inmates; inmates addicted to opiates

DAP; female drawn first, female larger than male

Signif. more addicts drew female first; addicts drew females signif. larger than males

NA

—

NP

1

Laird (1962) EPF

303

N

132 male introductory psychology students; 100 male alcoholics, mean age 45 yrs, mean education 11th grade; 71 male psychiatric patients, mean age 42 yrs, mean education 9th grade

Compared HFDs on sex of figure drawn first

Percentage who drew same-sex figure first: normals, 94.7%; psychiatrics, 84.5%; alcoholics, 81.0%; (latter two groups signif. diff. from normals but not from each other)

NA

—

NP

1

9:06 AM

EPF

Page 136

136

Koppitz (1966a)

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

25 third-grade children; 24 elderly residents of a home for aged

Compared groups on area used, figure height, and centeredness of DAP as indicators of self-concept and body image

Supported hypothesis: Drawings by aged more constricted, shorter, and less centered

—

—

NP

0

Lehner & Gunderson (1948)

DESC

421

N

College students; 229 males and 192 females

Subjects’ age compared to age assigned to DAP

Men assigned slightly older ages to male and female DAPs as own age became greater; for females, the function was curvilinear—age of drawn figure increased with actual age until actual age of 35, then decreased

NA

— Percentage only

1

Lewinsohn (1964)

EPF

100

N

Four groups of psychiatric patients rated as depressed male or female, or nondepressed male or female, by physicians (n = 25 in each group)

Compared groups on height of DAPs

Depressed patients had signif. shorter figures

NA

—

P

1

Lyons (1955) EPF

50

N

Last 50 people the author tested at work

On H-T-Ps, asked subjects to imagine the tree had been struck by lightning and place an “X” where this might have occurred; compared height of mark on tree to relative age at time when “worst thing that ever happened to you” occurred and “best thing”

Signif. correlation between “scar” height and age when “worst thing” occurred; not signif. for “best thing”

NA

—

NP

0

Mabry (1964)

CS

1

N

Patient with malignant brain tumor

DAP

Case study

NA

NA

NA

0

Marzolf & Kirchner (1970)

DESC

850

N

College students

List of 73 H-T-P characteristics: compared males and females, and diff. between first and second drawing

29 items were signif. diff. across sex

Median and agreement r = 91.8

—

NP

1

6/6/2003

N

reyn2-5.qxd

49

9:06 AM Page 137

137

Lakin (1956) EPF

(continues)

reyn2-5.qxd

APPENDIX 5.1. Continued

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

760

N

College males and females

Analyzed presence or absence of 108 drawing characteristics and compared to 16PF scores

Some signif. but low correlations; 347 signif. comparisons out of 3,672 correlations (17 16PF traits × 108 drawing characteristics × sex)

Interjudge agreement > 90%

R

P

2

McHugh (1963)

DESC

626

N

Male and female students, first through sixth grades

Analyzed whether same sex was drawn first and size of drawings

Overall tendency to draw same sex first; females drew female figure larger than males drew male figure

NA

—

NP

1

McHugh (1966)

EPF

108

N

Four groups (r = 27 Groups compared on 23 each): two diagnosed with variables on HFD adjustment reaction of childhood (neurotic traits or conduct-disturbed); two diagnosed with adjustment reaction of adolescence (neurotic traits or conduct-disturbed); both sets matched on age and IQ

Children with neurotic traits drew first figure significantly shorter and both figures farther from bottom of page; neurotic boys drew opposite sex first more frequently than conduct-disordered children

—

—

NP

1

McPhee & Wegner (1976)

EPF

264

Y

102 ED children (male and female), 162 normals; no ages reported

Compared groups by sex on stylization of K-F-Ds

No sex diff.; signif. diff. between ED and normals on stylization

Five judges; r = 66–1.00

—

P

0

Melikian & Wahab (1969)

DESC

162

N

137 male and 25 female Moslem, African-born college students

Percentage drawing samesex figure first on DAP

Women started signif. more often than males on opposite sex (females, 48%; males, 18%); similar to literature with American samples

NA

—

NP

0

Meyer, Brown, & Levine (1955)

CS

22

N

People undergoing a number of surgical procedures

Case study comparisons on pre- and postoperative HIPs

Various conclusions; many postoperative changes in drawings noted

NA

NA

NA

1

9:06 AM

DESC

Page 138

138

Marzolf & Kirchner (1972)

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

reyn2-5.qxd

50

Y

25 individuals with normal H-T-P comparisons on six EEG; 25 with abnormal variables from Buck’s EEG scoring system

Only line quality differentiated

—

—

NP

0

Modell (1951)

DESC

28

Y

28 mental patients: 13 recovered, 8 unimproved, 7 uncertain

Subjective scoring of HFD on “body image maturation” and “sexual maturation” in patients returning from a regressed state

An illustrative case study is presented in full; recovered group showed diff. in body image maturation and sexual maturation

Tested by ␹2 for signif. diff.

—

NA

0

Modell & Potter (1949)

CS

32

N

Medical patients with hypertension, peptic ulcers, or bronchial asthma

Compared features of HFDs for diff. types of patients; qualitative descriptions and case study provided for each type presented

Made conclusions about various personality characteristics

NA

NA

NA

1

Mogar (1962)

DESC

123

N

Male psychiatric patients

Looked at relationship between Manifest Anxiety (MA) scores and Hoyt– Baron anxiety indicators on DAP, and same anxiety indicators with Rorschach Content Text (RCT) scores

No results with MA, thus supporting previous research; several RCT variables had signif. correlation with DAP anxiety indicators

Interrater r = .84–1.00

—

NP

1

Moll (1962)

DESC

269

N

Normal college students

Foilage or not on H-T-P trees; drawings done by subjects during all four seasons

Signif. no. of bare trees in fall and winter drawings

Agreement reached by two judges in all but three cases

NA

NP

1

Mostkoff & Lazarus (1983)

DESC

50

NA

25 male, 25 female students qualifying for Title I; second–fifth grade

Determined interrater and test–retest reliability of an objective scoring system for K-F-D

Signif. agreement on nine variables: self in picture, evasions, arm extensions, elevated figures, rotated figures, omission of body parts (self, other), barriers, drawings on back of page (out of 20 variables)

Interrater agreement 97%

—

NP

1

9:06 AM

EPF

6/6/2003

(continues)

Page 139

139

MichalSmith (1953)

reyn2-5.qxd

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

Y

36 chronically obese children, 36 controls; 7, 10, and 13 yrs old, matched on sex, IQ, SES

Compared on Goodenough–Harris scoring of HFD looking for diff. in detail and sex differentiaton

Obese sample signif. more global and undifferentiated than controls; related to selfimage/body image

—

M

P

1

Oas (1984)

EPF

214

Y

100 adolescent psychiatric patients, 114 “normal” adolescents from regular and special education classes

Tested Bender–Gestalt Test and HFD ability to discriminate between impulsive and nonimpulsive adolescents (Matching Familiar Figures Test, Behavioral Checklist, Impulsive Behavior Checklist used to discriminate groups)

Discriminant-function analyses of Bender– Gestalt and HFD variables indicated high relation; HFD variables slightly better at discriminating impulsives from nonimpulsives in each group; 79% of school sample correctly classified; 93% of hospitalized sample correctly classified

—

M

P

2

Otterbacher, DESC Haley, Abbott, & Watson (1984)

40

NA

Male and female LD students, 59–146 months old

Investigated relation between HFD and postrotary nystagmus

All variables (age, nystagmus, IQ, sex) yielded signif.; age and nystagmus were as predictive of HFD performance without IQ and sex added into equation; nystagmus was still predictive with age partialed out

Interrater r = .94

—

P

1

Paine, Alves, & Tubino (1985)

24

Y

12 pediatric oncology patients, 12 general pediatric surgery patients and students

Compared groups on Koppitz HFDs according to size of drawing

Cancer patients’ drawings signif. smaller than those of surgery patient/student group; implied increased anxiety

NA

M

P

0

EPF

Page 140

72

9:06 AM

EPF

140

Nathan (1973)

6/6/2003

APPENDIX 5.1. Continued

Prout & Celmer (1984)

DESC

100

NA

Resnikoff & EPF Tomblen (1956)

75

N

Rosen & Boe (1968)

DESC

98

N

Saracho (1984)

DESC

480

NA

Schubert (1969)

DESC

22

N

Males with priapism (prolonged erection)

Case studies used Bender– Gestalt Test and DAP

Concluded that a “blind” investigator would find no clear signs in drawings to distinguish this group

NA

NA

NA

1

Normal male and female fifth-grade students

Examined relation between indicators of emotional conflict and negative affect on K-S-D and academic achievement

Modest but signif. correlations between achievement (SRA achievement test) and 6 of 10 variables, 26% of variance in achievement accounted for by K-S-D variables in stepwise multiple regression

—

—

P

1

25 organic brain-impaired, Compared HFDs on 17 25 schizophrenic, 25 indicators neurotic individuals

Organics signif. diff. from other two groups on five to six indicators: weak synthesis, parts misplaced, shrunken appendages, etc.

Two scores, 91% agreement

—

P NP

1

Male college students enrolled in weight-lifting class

64% completely nude, 48% with penis; discussed as very unusual finding

NA

— Percentage only

0

Random samples: 240 first Assessed interjudge, testgraders, 240 third graders retest, and split-half reliability; tested relationship between Goodenough DAP and Children’s Embedded Figures Test and Articulation of Body Concept Scale (measures of field dependence and independence)

High reliabilities found: Embedded Figures, .90s; body concept, .50s; Goodenough, 80s; high correlation with articulation of Body Concept Scale (.90s); low correlation with Embedded Figures Test; concluded DAP was good measure of field dependence–independence

Interjudge r = .91 for DAP

R

P

2

Army enlisted men

Revealed signif. linear trend toward poorer quality; indicated a motivation deficit that must be considered in studies where more than one administration is involved

Interrater r = .80s– .90s

—

P

0

Nudity in DAP

Compared three administrations of DAP (male and female) on a DAP quality scale (drawings once a week)

(continues)

Page 141

N

9:06 AM

141

7

6/6/2003

CS

reyn2-5.qxd

Pollitt, Hirsch, & Money (1964)

reyn2-5.qxd

Authors

Results

15

N

8 boys, 7 girls; 47–57 mo old

DAP using Ayres and Reid guidelines and estimated IQ scores (from WPPSI)

Draw-A-Man and DrawA-Woman scores signif. correlated; Ayres and Reid scores signif. related to longer DAPT scoring system; IQ did not signif. correlate to any drawings

Sobel & EPF Sobel (1976)

40

Y

20 institutionalized male delinquents, 14–17 yrs old; 20 normals from a public school, 15–17 yrs old

Compared K-F-Ds on 16 traits

Stawar & Stawar (1989)

EPF

52

N

Adolescent psychiatric inpatients between 13 and 19 yr old; 54% female, 90% white, 10% black

Strumpfer (1983)

EPF

81

N

Vane & EPF Eisen (1962)

662

N

Interrater rs ranged from .93 to .97

Sampling Stats

Generalizabilitya

R?

NP

0

Only 3 of 16 traits showed signif. diff. between groups; questionable ability to diagnose delinquency

R

NP

1

Participants grouped into overlapping diagnostic groups of disruptive, depressive, anxiety, and thought disorders; KFDs scored across 24 variables; select MMPI scales used (D, Pt, Pd, Sc)

MMPI D and Sc scales signif. — correlated with diagnostic groups; no signif. results with any KFD indicators

—

NP

1

Psychotic inpatients; 45 male, 36 female

DAP compared by age and length of time since diagnosis; DAP variables included quality, height, sex differentiation

Most variables showed signif. negative correlations with chronicity (length of illness); the longer subjects had been sick, the poorer the performance

Interrater r = .85

—

P

1

Kindergarten children divided into three groups by teacher adjustment ratings of good, fair, or poor behavior on a 9-item scale

Compared Goodenough Poor group showed signif. Draw-A-Man on four signs more signs compared to and combinations other two groups

NA

R?

NP

2

(continued)

Page 142

142

EPF

Scoring reliability

9:06 AM

ShortDeGraff, Slansky, & Diamond (1989)

Variables

6/6/2003

APPENDIX 5.1. Continued Type Number of of Control design subjects (Y/N) Population sampled

N

Preschoolers rated most masculine and most feminine, and least masculine and least feminine, by teachers

Sex of DAP

No diff. based on degree of gender; signif. diff. on percentage drawing same sex by biological sex (males, 76%; females, 58%); only 25% of figures sexdifferentiated at all

—

—

NP

1

Weider & Noller (1950)

EPF

438

N

210 boys, 228 girls; 7–12 yrs old

Compared HFD: Subject sex × sex of first figure; subject sex × larger figure drawn; location on page × age; subject sex × IQ × full face or profile

Girls drew own sex more often; girls drew own sex larger more often; younger children placed drawing closer to upper left quadrant more often

NA

R?

NP

2

Weider & Noller (1953)

EPF

153

N

Children 8–11 yrs old

For HFD: Divided subjects into upper, middle, lower SES; compared no. of characteristics drawn, size of figures, same or opposite sex drawn first; overt drawing compared to covert things learned from interview but not in drawing

Girls drew own sex first more than boys; boys drew own sex larger; boys increased no. of characteristics drawn as SES increased, especially in same-sex figure; girls did too, but no signif. diff. depending on sex of figure; responses in interview did not differ from drawing

—

R?

NP

2

Wildman (1963)

EPF

60

N

Patients from psychiatric department of state hospital divided in two groups according to whether or not their knee/arm joints were present in H-T-Ps (n = 30 in each group)

Judges rated patient records on degree of paranoid pathology (little, moderate, high); compared across jointed and nonjointed groups

More than twice as many patients who had drawn joints were rated highly paranoid compared to no joint patients; more than twice as many who didn’t draw joints were rated low compared to joint patients

On ratings of paranoia, interrater r = .80

—

NP

0

Wisotsky (1959)

DESC

490

N

Black and white male incarcerated alcoholics

Compared percentage who drew male figure first between whole group and normative group from literature; also between blacks and whites

Whole group drew male figure first signif. more often than normals; no diff. between blacks and whites in sample

NA

R

NP

2

6/6/2003

151

reyn2-5.qxd

EPF

Page 143

(continues)

9:06 AM

143

Vroegh (1970)

reyn2-5.qxd

APPENDIX 5.1. Continued

Woods & DESC Cook (1954)

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

N

Eighth-grade students

Looked at relationship between detailing in drawing of hands in HFD and level of proficiency in drawing

Found signif. relationship; questioned use of detailing in hands as a measure of personality variables

Reported with ␹2 significance only

—

NP

0

9:06 AM

138

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

CS

1

N

11-yr-old male; recovered from viral encephalitis

Administered DAP and KFD pre/post (4 mos) relative to therapy for oppositional behavior

Author used results to indicate usefulness of tools in tracking therapeutic changes

—

NA

NA

0

Wysocki & Whitney (1965)

EPF

100

Y

50 crippled school children (due to polio, cerebral palsy, clubfoot, dislocated hips), 50 noncrippled school children; all 6–11 yrs old

Machover DAPs compared on 15 variables

Eight variables showed signif. diff.—large head, opposite sex, large figure, placement, shading, pressure, paper rotation, and an area of insult

—

—

P

1

Yates, Beutler, & Crago (1985)

EPF

34

Y

17 females referred to psychiatric clinic as victims of incest; 17 referred for other reasons

Developed clinical rating scale of indicators of potentially disturbed functioning while comparing groups

Only two signif. diff. (dimensions of impulse control and quality of repressive defenses); signif. diff. in variability of hypersexualization

Combined ratings of two raters

M

P

1

Abell, Von Briesen, & Watz (1996)

EPF

199

N

Children, ages 5–15, whose drawings were obtained from the archives of a University Psychological Services Center

Evaluated the Goodenough– Harris and Koppitz HFD scoring systems’ ability to assess children’s intellectual abilities via the WISC-R or Stanford–Binet

Both drawing systems signif. correlated with IQ tests; the Goodenough–Harris had a signif. higher correlation with WISC-R Performance Scale than Koppitz

Interrater r: Koppitz: .93; G-H: .88

Archive

P

1

Abell, EPF Horkheimer, & Nguyen (1998)

200

N

Males, ages 14–15, whose drawings were obtained from the archives of a University Psychological

Raw scores on the Goodenough–Harris and H-T-P were converted to standard scores and compared to WISC-R IQ

Goodenough–Harris scores signif. lower than Verbal, Performance, Full Scale IQs; G-H and H-T-P about equally correlated with IQ;

Interrater r: H-T-P: .91; G-H: .88

Archive

P

1

Studies from 1990 to 2000

Page 144

144

Worden (1985)

G-H and H-T-P scores signif. correlated with each other

EPF

216

N

Child and adolescent psychiatric inpatients (ages 6–28 yr)

Assessed HFDs with IQ, academic achievement, and the BGT

Sign. but low correlations between Goodenough– Harris HFD scoring and IQ/achievement; 58% of Ss were misclassified when HFD used to predict IQ; BGT did not improve accuracy of IQ prediction

NA

—

P

0

GrothMarnat & Roberts (1998)

EPF

40

N

9 male and 31 female undergraduates, ages 18–47

Assessed the concurrent validity of HFDs and H-T-P as measures of self-esteem with the Coopersmith and Tennessee self concept scales

Neither HFD or H-T-P scores were signif. indicators of either measure of selfesteem; age, gender, and artistic ability accounted for 21–22% of variance in HFD/H-T-P scores

Interrater rs of .88

—

P

1

Evaluated KFDs using “Like to Live in Family” rating procedure to investigate discriminant validity

Comparison scored higher on scoring, indicating more positive adjustment as did comparison mothers; mothers of sexually abused children scored higher than their children; no signif. diff. between comparison mothers and children

M

P

1

Reviewed research relative to reliability, normative, validity outcomes as well as cultural norms and implications for personality assessment

Interrater reliability good to See results excellent in all studies; test– retest reliability not as good; KFD generally failed to discriminate between groups of people and very few signif. diff. found in drawings of well-adjusted and children with pathology; authors recommend more research using composites of indicators rather than single scoring variables

NA

NA

2

EPF

60 children, 60 mothers

Y

30 sexually abused children (25 girls, 5 boys); 30 unidentified comparison school children

Handler & Habenicht (1994)

Lit. Review

NA

NA

Primarily children and adolescents

Internal rs ranging from .50 to .76 Kendall’s W indicated high interrater agreement between raters

(continues)

Page 145

Hackbarth, Murphy, & McQuary (1991)

9:06 AM

145

Aikman, Belter, & Finch (1992)

6/6/2003

scores

reyn2-5.qxd

Services Center

reyn2-5.qxd 6/6/2003

Authors

EPF

134

Y

94 females and 40 males, ages 5–8 years

146

65 children were alleged sexual abuse victims from a clinic or therapeutic program; 64 were comparison children from general pediatric clinics

Variables

Results

Study compared HFDs via Koppitz’s Emotional indicators to determine discriminant validity

No sign. diff. in the presence of individual indicators were observed; abused children’s drawings appeared to show more anxiety than the comparison children

Scoring reliability

Sampling Stats

Test–retest ranged from 84% to 100% (M = 94%) on indicators from first to second drawing

Generalizabilitya

M

P

0

Robins, Blatt, & Ford (1992)

EPF

32

N

14 females and 18 males, 16 to 29 yrs old, who were hospitalized in a private, intensive, inpatient treatment center for serious disturbances

Study compared HFDs and Robins Balance-Tilt Scale after participants were hospitalized for 6 weeks and then 15 months after

Goodenough–Harris scoring Interrater of the HFDs from Time 1 to r = .95 Time 2 were highly correlated, independent of the gender of the figure drawn; data indicated a main effect of time with later drawings more fully articulated and differentiated

—

P

0

Taylor, Kymissis, & Pressman (1998)

CS

3

N

2 males, 1 female, 16–18 yrs old attending a day hospital for adolescents with psychiatric and substance abuse problems

Analyzed KFD and Prospective KFD drawings (the latter of participants’ families in 10 yrs)

Authors report that additional information on the thoughts and feelings of the adolescents was gained using the drawings

—

—

0

—

Page 146

Hibbard & Hartman (1990)

Type Number of of Control design subjects (Y/N) Population sampled

9:06 AM

APPENDIX 5.1. Continued

reyn2-5.qxd

Y

Clinical group: 41 girls, 11 boys (86% white) in grades 4–7 with mood, anxiety, or combined mood/anxiety disorders

Compared children’s DAPs via Koppitz system on 30 emotional indicators and KFDs via Reynold’s system on 37 indicators; MANOVA used to Control group: 13 children evaluate scoring system’s from same schools ability to discriminate presenting with no between the broad DSM-III symptomatology diagnostic groups

Individual emotional indicators for both tests failed to differentiate children with internalizing disorders vs. normal controls; when indicators summed, no sign. diff. among the four groups

Interrater r for DAP = .92; for KFD = .80

Comparison group

P

1

Meta12 analysis studies

Y

Meta-analysis of 12 studies, published between 1986 and 1996, assessing the efficacy of projective techniques to discriminate between sexually and nonsexually abused children

Used studies where effect sizes could be calculated; projective reviewed included Rorschach, Hand Test, TAT, KFD, HFD, H-T-P, others

Results indicated that projective tests could discriminate between children who were sexually abused and those who were not abused sexually

Study by study

Study by study

P

1

EPF

N

30 male, 97 female college students

Assessed one week test–retest reliability of “House” drawing of the H-T-P with half of the participants asked to draw a different house during the second drawing

Using Buck (1948) scoring system, no sign. diff. between two groups on 21 scoring indicators; sameness scores reached over 50% agreement for 18 indicators and 75% for 5 indicators

See Results

—

P

0

127

Page 147

Wu, Rogers, & Searight (1991)

9:06 AM

West (1998)

52

6/6/2003

EPF

147

Tharinger & Startk (1990)

Note. EPF, ex post facto design; DESC, descriptive “design”; EXP, experimental design; CS, case study; LD, learning-disabled; ED, emotionally disturbed; MH, mentally handicapped; SES, socioeconomic status; HFD, Human Figure Drawing; DAP, Draw-A-Person Test; H-T-P, House-Tree-Person Test; K-F-D, Kinetic Family Drawing; K-S-D, Kinetic School Drawing; MMPI, Minnesota Multiphasic Personality Inventory; GSR, Galvanic skin response; IPAT, Institute for Personality and Ability Testing; 16PF, Sixteen Personality Factor Test; SRA, Science Research Associates; NA, not available; ANOVA, analysis of variance; (?), results questioned; R, random sample; M, matched sample; R?, possible random sample; M?, possible matched sample; P, parametric statistics used; NP, nonparametric statistics used. a Generalizability rated on a scale of 2 (good), 1 (fair), or 0 (limited or none).

reyn2-5.qxd

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

100

NA

School children referred for ED or behavioral problems in metropolitan school district

Explored relationship between BGT recall scores and no. of emotional indicators

No signif. correlation found; children with signif. number of emotional indicators did not recall fewer BGT figures

—

—

NP

1

Baroff (1957)

EPF

76

N

Twins institutionalized as mental defectives (endogenous)

Compared results on Pascal–Suttell scoring with results from previous research

Similarities suggested feasibility of establishing clinical norms for endogenous retardates

—

—

P

0

Bensberg (1952)

EPF

322

N

161 organic brain damage MH, 161 familial etiology MH; matched on MA and CA, mean MA about 6 yrs, 6 mos

Bender scoring of BGT

Familial group signif. more accurate in repro duction; reversals, parts repeated, and lines instead of dots signif. more frequent in brain-injured

—

M

P NP

2

Blaha, Fawaz, & Wallbrown (1979)

DESC

74

N

Male and female black first graders

Examined relationship among BGT errors, BGT time, DAP scores, DAP time, Matching Familiar Figures errors & time, and Slosson IQ

16% of variance in BGT performance due to preprocessing and central processing with general IQ partialed out; conceptual tempo accounted for little variance

—

—

P

1

Billingslea (1948)

EPF

150

Y

100 male psychoneurotic patients, 50 normal males (all soldiers)

Compared groups on 38 factors in scoring and interdrawing reliability

Equivocal results

NA

R?

NP

0

Breen & Butler (1983)

EPF

59

Y

30 students diagnosed ED; 29 normal controls (ages 7–11 yr)

Compared groups on 12 Koppitz emotional indicators on BGT

No signif. diff.

—

—

P

1

Page 148

148

Anderson & DESC Rallis (1981)

9:06 AM

Studies Prior to 1990

6/6/2003

APPENDIX 5.2. A Methodological Summary and Analysis of Projective Interpretations for the Bender–Gestalt Test (BGT)

reyn2-5.qxd

Y

200 children in need of psychotherapy; 200 children judged well adjusted; 8–16 yr old

15 interpretation factors developed by Hutt on BGT groups compared by age level

About half of factors signif. at each age level; pattern differed for these factors

—

M?

NP

1

Chorost, Spivack, & Levine (1959)

EPF

68

N

Children under 18 who had EEG and a scorable BGT

Compared BGT rotations to whether EEG was normal or abnormal

Correct prediction of abnormal EEG by rotations in 65% of cases; not a great increase over other clinical signs

—

—

NA

1

Curnutt (1953)

EPF

50

Y

25 males in Alcoholics Anonymous (AA); 25 control males matched on age, SES, occupation, education

Compared BGT scored using Pascal–Suttell method (scored by author, who knew which group each case was in)

Signif. diff. between mean scores; signif. indicators included rotations, counting dots; higher scores (poorer performance) in AA group

—

M

P

0

Fabian (1945)

DESC

692

Y

106 children with a variety of psychological problems who were all “retarded” in reading; 586 normal school-age children

Comparisons, descriptions of BGT rotations and reversals

Established developmental nature of rotation of horizontally oriented figures to the vertical; persistent “verticalization” concluded to be a clinical sign of problems

—

—

NA

1

Fuller & Chagnon (1982)

EPF

270

Y

Normals, ED, schizophrenic (n = 90 in each group)

Rotation of BGT drawings

Signif. diff. between each pairwise group comparison; also diff. depending on original orientation of figure, or figure–ground

—

—

P

2

Gavales & Millon (1960)

EXP

80

Y

40 undergrads with high Taylor Manifest Anxiety (TMA) scores; 40 undergrads with the lowest TMA scores (out of 195)

Subjects divided into experimental (anxietyinducing tasks) and control groups; sizes of initial and recall BGT figures compared

Signif. greater diff. in size in high-TMA inducedanxiety group

—

R

P

1

(continued)

Page 149

400

9:06 AM

DESC

6/6/2003

149

Byrd (1956)

reyn2-5.qxd

APPENDIX 5.2. Continued

Results

Scoring reliability

Sampling Stats

Generalizabilitya

393 118

Y

108 neurotic, 285 normal male adult veterans (validation study) 64 neurotic, 54 normal adult male nonveterans (cross-validation study)

Two BGTs from each subject; neurotics and controls compared using objective scoring system with 82 general categories and 312 specific signs

Result was a final scoring system of 30 signs that discriminated consistently between groups on both test and retest; test-retest reliabilities in the high .60s did not corroborate Hutt’s “neurotic syndrome”; BGT recommended as a screening device and supplement to other instruments, not for elaborate interpretation

—

—

NP

1

Goldberg EPF (1956–1957)

45

Y

White male children: 15 schizophrenic, 15 MH, 15 normals; ages 11–16

Looking for diff. among groups on BGT according to Pascal–Suttell scoring

Signif. diff. between normals and other two groups; concluded that objective scoring in conjunction with qualitative analysis effective for clinical diagnosis

—

M

P

0

Goldfried & Ingling (1964)

DESC

80

N

College undergrads; 40 male, 40 female

Rated BGT drawings on descriptive semantic differential scales (e.g., kind, strong, fast)

Generally negative findings; no “universal” symbolic meanings among individuals

NA

—

NP

1

Goodstein, Spielberger, Williams, & Dalstrom (1955)

EXP

114

N

54 male and 60 female undergrads

Exp. 1—effect of serial position on BGT design recall; Exp. 2—effect of where more difficult designs placed in presentation order on recall

Designs differed signif. on ease of recall; recall related to both difficulty and serial position; no diff. between males and females

86.3%– 91.8% agreement

—

P

1

1,000

N

Matched neuropsychiatric patients; mean age 35.5 yrs

Compared no. of BGT rotations to diagnostic category

Categories with signif. more rotations were chronic brain syndrome and mental deficiency

—

NA

NA

2

Griffith & DESC Taylor (1960)

Page 150

150

EPF

9:06 AM

Gobetz (1953)

Variables

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Compared no. of BGT rotations for regular admission and paper presented lengthwise

Signif. fewer rotations when blank paper presented lengthwise instead of vertically

—

—

NP

1

Guertin (1952)

DESC

100

N

Male and female mental patients: organic braininjured, nonpsychotic, and schizophrenic

41 BGT scoring characteristics from Billingslea method factor-analyzed

Six factors: propensity for curvilinear movement, poor reality contact, careless execution, constriction, poor spatial contiguity, unidentified

—

—

P

0

Guertin (1954a)

DESC

100

N

Male and female schizophrenics and nonschizophrenics

46 BGT scoring variables factor-analyzed

Six factors: unstable closure, curvilinear distortion, propensity of curvilinearity, fragmentation, irreg. propensity of curvilinear movement, experimental dependence; suggested using clusters of variables in scoring rather than single variables

—

—

P

0

Guertin (1954b)

DESC

27

N

Organic brain-diseased males

100 items as above factoranalyzed

—

—

P

0

Guertin (1954c)

DESC

37

N

Male schizophrenics

BGT scored on 100 items, factor-analyzed

Three factors: curvilinear distortions, spatial disability, construction and feelings of inadequacy Four types of schizophrenic performance factored out

—

—

P

0

Guertin (1955)

DESC

30

N

Male schizophrenics

100 items as above factoranalyzed

Four types of schizophrenic performance factored out

—

—

P

0

Hain (1964)

EPF

101

Y

Patients in neuropsychiatric ward; 20 brain-damaged, 38 psychiatric, and 20 controls not diagnosed in either category

Compared groups on a scoring system designed to differentiate brain damaged group from others

High scores did differentiate, but low scores did not predict lack of brain damage

—

—

NP

1

9:06 AM

56 neuropsychiatric patients; 157 control neuropsychiatric patients

6/6/2003

Y

reyn2-5.qxd

213

(continues)

Page 151

151

Griffith & EPF Taylor (1961)

reyn2-5.qxd

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

6/6/2003

APPENDIX 5.2. Continued

EPF

30

N

15 brain-damaged children; 15 familial retardates; ages 7–13 yrs matched on CA, MA, IQ

Compared no. of rotations on BGT

No signif. diff.

96% agreement

M

P NP

1

9:06 AM

Hammer (1954b)

EPF

40

Y

20 males undergoing sterilization; 20 males undergoing other surgeries

Numerous variables selected for each drawing indicative of castration anxiety before and after surgery

Many variables showed signif. diffs.

—

—

NP

1

Page 152

152

Halpin (1955)

Hanvik & Andersen (1950)

EPF

44

Y

20 patients with lesions in dominant hemisphere; 24 patients with lesions in nondominant hemisphere; control group of patients with low back pain

Compared groups on no. of BGT figures recalled from memory and no. of rotations

Dominant and nondominant groups not signif. different, but signif. more rotations than control group

—

—

P

0

Helgert (1985)

DESC

120

Y

60 severely ED children; 60 “normals”—referred but not placed

Compared traditional and digital computer methods of scoring BGT

Interrater reliability for computer, .99+; interrater reliability for traditional, .70s; small but signif. correlations between methods; computer method accurately placed 61.7% of subjects; traditional method accurately placed 70% of subjects

(?)

—

P

1

Hellkamp & Hogan (1985)

EPF

180

Y

Psychiatric patients, organic and nonorganic (functional) etiology

Investigated ability of Hutt BGT scoring system to diff. organic from functional group as a function of IQ

Signif. correlation between IQ and BGT errors for either group; no acceptable percentages of accuracy scores for groups across IQ ranges

—

M

P

2

Y

30 institutionalized male delinquents, 30 nondelinquents

Compared BGT error scores; tested a cutoff criterion for discriminating delinq. from nondelinq. adolescents

Signif. diff. in error scores between groups; cutoff criterion yielded 35% false positives and 47% false negatives; indicated need for further research

—

M

P

1

Holmes & Stephens (1984)

DESC

76

NA

Male and female college students

Investigated amount of edging on BGT. Memory for Design, and DAP

Not a consistent factor; very little edging found; signif. Q indicated lack of consistency

NA

—

NP

1

Hutt & Monheit (1985)

EPF

180

Y

Carefully screened normal male students (13–16 yrs old) and children diagnosed ED

Compared on Hutt adaptation of BGT scoring

Signif. diff. in mean scores; the configuration score for disturbed adolescents signif. differentiated groups

Interrater

—

NP

1

Jernigan (1967)

EPF

779 206 247

N

Patients with central nervous system, psychiatric, physical, and unknown problems

Rotation of BGT figures compared on type of patient, education, direction of rotation

Three separate studies: Rotators were older, had lower IQ, had less education; 45% rotated in clockwise direction

—

—

NP P

2

Johnson (1973)

EPF

50

N

25 psychiatric patients with and 25 without constricted BGT figures, respectively

Constriction defined using less than half of page to complete figures; compared groups on MMPI Depression scores

t test yielded signif. diff.; constriction indicated depression, but low rate of occurrence detracted from its usefulness

NA

—

P

0

Koppitz (1958)

EPF

51

Y

Elem. students (first through fourth grades) divided into good and poor achievement groups (reading, writing, spelling)

Group comparison on 7 of 20 scoring categories that tested signif. on a first sample and cross-validated on second sample

Signif. discrimination in cross-validation group

93% agreement on 14 protocols

—

NP

1

Koppitz (1960)

DESC

1,055

N

School children; 5 to 10½ yrs old, kindergarten– fourth grade

Blind scoring using Koppitz approach

Normative table with mean scores SDs by age groups and grade

—

R?

NA

2

Koppitz (1962)

EPF

384

Y

103 brain-damaged elem. students; 5–10 yrs old; 281 normals

Compared groups on Bender scoring system

Signif. more “poor” BGT scores in brain-damaged group (p < .001) across ages

—

M

NP

1

6/6/2003

60

reyn2-5.qxd

EPF

Page 153

(continues)

9:06 AM

153

Hinkle (1983)

reyn2-5.qxd

Authors

EPF

Scoring reliability

Sampling Stats

Generalizabilitya

Results

40 reading-retarded elem. students; 40 behavior disordered normal readers, 40 controls (no problems); all normal IQ, matched on age

Compared groups on BGT scores (Pascal–Suttell)

Reading-disabled group tended to produce more distortions, but diff. not quite signif.

—

M

NP

2

Matched ED adolescents 12–17 yrs old from day treatment and residential centers

Examined ability of emotional indicators on BGT to discriminate behavioral indicators of Devereaux Adolescent Behavior Rating Scale

Correlations between BGT and Devereaux indicated some signif. correlations for individual signs; overall BGT withdrawal and anxiety signs did not signif. correlate; caution indicated in using BGT to draw implications about behavior

—

NA

P

1

Y

McCormick DESC & Brannigan (1984)

40

NA

Moore & Zarske (1984)

452

Y

150 LD, 189 educationally disadvantaged, 113 nonhandicapped Navajo children; ages 6–11 yrs

Compared normals to 1974 Koppitz and SOMPA norms for BGT; compared BGT scores across groups by age and sex

Nonhandicapped compared favorably with Koppits norms and SOMPA norms; nonhandicapped and LD groups showed signif. diff., with nonhandicapped having lower error rates across age and sex

—

—

P

1

262

Y

142 brain-damaged, hospitalized veterans (BD); 120 nonbrain-damaged controls (NBD)

Compared ability of two BGT scoring systems (Peek–Quast and Hain) in discriminating BD from NBD patients

Each discriminated statistically, but not at clinically useful level

On 30 protocols, interscorer r = .84–.86

—

P

1

154

120

DESC

Mosher & EPF Smith (1965)

Page 154

Variables

9:06 AM

Lachmann (1960)

Type Number of of Control design subjects (Y/N) Population sampled

6/6/2003

APPENDIX 5.2. Continued

reyn2-5.qxd

NA

Kindergarten students followed through fourth grade

Explored relationship between kindergarten BGT performance and later reading and writing performance (Neale analysis of reading abilities and developed handwriting analysis system)

BGT performance was predictive of overall school achievement; major error categories (e.g., circles for dots, distortion) more predictive than total errors; only predictive of one reading or writing variable (reading comprehension at end of first grade)

—

R

P

2

Neale & McKay (1985b)

DESC

200

NA

School children 5–7 yrs old

BGT (Koppitz) errors described; item difficulty analyzed

Reported percentage of each error type across figures; interjudge agreement across error categories in low .90s for total error scores (r’s ranged from .71 to .98); angulation errors were the most frequent

For three scores, r = .90s

R

P

2

Peek (1953)

EPF

150

Y

75 neuropsychiatric patients who had certain characteristics on BGT Figure 5; 75 randomly selected patients from same population

Compared groups on frequency of list of 40 external characteristics of behavior and personality and complaints

27 variables were signif. diff. between groups

—

—

NP

0

Quast (1961)

EPF

100

N

50 suspected braindamaged psychiatric patients, 50 ED psychiatric patients; all 10–12 yrs old

Compared groups on 17 attributes of BGT drawings

10 of 17 signif. differentiated groups; low intercorrelations between these 10 differentiating attributes

—

M?

NP

1

Schulberg & Tolor (1962)

DESC

106

N

15 neurotics; 41 functional psychotics; 15 acute organic psychotics; 45 with character disorder

Wanted to see what connotative meaning psychiatric patients attached to BGT drawings using semantic differential scales

Not much difference by patient type

NA

—

P

0

9:06 AM

195

6/6/2003

DESC

(continues)

Page 155

155

Neale & McKay (1985a)

reyn2-5.qxd

Variables

Results

Scoring reliability

Sampling Stats

Generalizabilitya

60

Y

Control, white male elem. and second. school teachers; exp., male inpatient alcoholics (n = 30 in each group)

Various characteristics of BGT drawings

Most characteristics showed signif. diff. including more counting of dots, nonintersections, “liquid” responses on Design 6

Interscorer r = .99

R

P NP

1

Suczek & Klopfer (1952)

DESC

48

N

Matched college students (male and female)

Subjects asked to free associate to BGT figures

Compilation of more frequent symbolic associations to each of the figures

Disagreement resolved by discussion

NA

NA

1

Taylor, Kaufman, & Partanio (1984)

DESC

652

NA

Statewide school children, ages 5–11 yrs

Subjected elements of Koppitz scoring of BGT to multiple regression by age

Variance associated with age for total sample was only 35% (for 9- to 11-yr olds, 3%); concluded BGT scores not valid developmental indicator beyond age 8

—

R

P

2

Thomas (1984)

DESC

66

NA

Children with low scores Assessed relationship beon horse puzzle of WISC-R tween ratio of horse score Object Assembly and other three Object Assembly puzzles and BGT errors

BGT errors positively correlated to “horse ratio” (.35)

—

—

P

0

Tolor (1957) DESC

50

N

Matched male and female neuropsychiatric patients (Air Force pop.)

Tried to establish associational stimulus value of the BGT figures; found that some produced signif. better associative responses than others

—

NA

NP

0

Had patients match Rorschach blots with BGT design that best represented or stood for each blot; rated them on type of response, or quality

Page 156

156

Story (1960) EPF

9:06 AM

Authors

6/6/2003

APPENDIX 5.2. Continued Type Number of of Control design subjects (Y/N) Population sampled

reyn2-5.qxd

Compared groups on ability to reproduce BGT designs from memory after standard testing

Some signif. diff. but not of clinical value

NA

M?

NP

0

Tolor (1960) DESC

68

N

College undergrads; 41 males, 27 females

Ratings of BGT designs on descriptive semantic differential scales

79 out of 180 ␹2 comparisons of item ratings between pairs of figures were signif.; connotative meanings for each were different; no sex differences

NA

R

NP

2

Trahah & Stricklin (1979)

EPF

93

N

Male and female students, kindergarten– fourth grade

Investigated relation between 15 proposed indicators (emotional) on BGT and acting-out behavior in class as rated by teachers

No signif. correlations

Determined by ␹2 (?)

R?

P NP

2

Wagner & Murray (1969)

EPF

50

Y

25 children with organic brain damage; 25 normal controls

Clinicians (MDs, PhDs, MAs) made diagnoses (brain-damaged or not) based on BGT and design reproduction test

No signif. diff. between professional groups’ high degree of correct diagnoses

—

—

NP

1

Wallbrown & Fremont (1980)

DESC

84

NA

Matched reading-disabled children referred from one school district

Examined test–retest reliability of Koppitz BGT scoring (12- to 24-day interval)

Pearson correlation was .83 for total error scores

Judges resolved differences by discussion

NA

P

1

Wiener (1966)

EPF

822

N

822 children from a Johns Hopkins study on prematurity, ages 8–10 yrs; matched sample (race, sex, SES)

Compared BGT scores to neurological data to see whether BGT could predict minimal brain damage for premature and full-term groups

Some variables were signif.: ability to copy angles and curves, tendency to produce distortions

For 50 protocols, interrater r = .92

M

P

2

Page 157

Three groups (n = 18 each): Character-behaviordisordered, schizophrenic, organic brain-injured; all matched on IQ and age

9:06 AM

N

6/6/2003

54

157

Tolor (1958) EPF

(continues)

reyn2-5.qxd

APPENDIX 5.2. Continued

DESC

Sampling Stats

Generalizabilitya

Results

55 boys, 31 girls; ages 6–11 yrs; referred for learning or adjustment problems

Compared VMI and Koppitz scoring of BGT on ability to predict academic achievement (WRAT)

Partialing out variance assoc. with IQ yielded signif. correlations between VMI and WRAT Spelling and BGT and WRAT Reading and Spelling; absolute magnitude was very small; added little to prediction beyond measure of general ability (WISC-R)

—

—

P

1

Scores signif. diff. (delinquents higher) on all but Design 8; each item also further analyzed for specific characteristics

—

M?

P

1

—

P

0

P

1

NA

Zolik (1968) EPF

86

Y

Two groups: delinquents and nondelinquents matched on age and IQ (roughly)

Compared on BGT scored by Pascal–Suttell method

Maloney & Wagner (1991)

EPF

240

N

Subjects (ages 15–65 yr) separated into five age intervals

Assessed each subject with the BGT, WAIS, Hand Test, Rorschach

Variables correlated with age-related changes; most of signif. variance attributed to intellectual factors

NP

Nyfield & Patalano (1998)

EPF

46

N

Male adolescents in a residential treatment center for behavioral disorders

Investigated BGT errors that were valid or invalid indicators of organicity hypothesizing that these errors were separate and distinct from Koppitz’s Developmental scoring errors; also collected WISC-III Block Design, Connors Parent Rating Scale, Bender Recall data

Confirmed major hypothesis; also found that BGT errors of organicity had signif. correlations with other like measures (Block Design, Bender Recall); BGT emotional indicators did not sign. correlate with Connors, nor did BGT organicity indicators

Interrater rs M: for BGT Ss split ranged into 2 from .88 groups to .94

Page 158

158

86

9:06 AM

Wright & DeMers (1982)

Scoring reliability

Variables

6/6/2003

Authors

Type Number of of Control design subjects (Y/N) Population sampled

Studies from 1990 to 2000

Note. EPF, ex post facto design; DESC, descriptive “design”; EXP, experimental design; LD, learning-disabled; ED, emotionally disturbed; MH, mentally handicapped; MA, mental age; CA, chronological age; DAP, Draw-A-Person Test; SES, socioeconomic status; MMPI, Minnesota Multiphasic Personality Inventory; SOMPA, System of Multicultural Pluralistic Assessment; WISC-R, Wechsler Intelligence Scale for Children—Revised; VMI, Test of Visual Motor Integration; WRAT, Wide Range Achievement Test. a Generalizability rated on a scale of 2 (good), 1 (fair), or 0 (limited to or none).

reyn2-6.qxd

6/6/2003

9:07 AM

Page 159

6 The Sentence Completion as a Tool for Assessing Emotional Disturbance

RUTH ADLOF HAAK

The sentence completion technique grew out of one of the earlier approaches to psychological assessment, the word association technique. Using partial sentence stems to stimulate the verbal production of a client rather than single words, A. F. Payne (1928) devised the first sentence completion instrument. He hoped to overcome the limitations of simple word association, which were primarily the cultural and structural biases being experienced with that procedure. Although these biases no doubt affect the sentence completion technique also, the sentence completion is far more capable of providing a proper degree of set for the responses to be elicited than is word association or any other projective procedure. This makes it practical as an assessment instrument for investigators who wish to study personal attitudes or adjustment related to specific situations.

its use as a measurement of adjustment in certain settings, including school settings, to measuring the effects of conditions upon performance, effects of interventions upon mental health, and the lasting effects of trauma over time. By midcentury, the sentence completion had become a popular measurement approach in countries across the globe, including Japan, China, Germany, Denmark, India, Taiwan, France, and others (see, e.g., Agesen, Brun, & Skovgaard, 1964; Derichs, 1977). Clearly its most popular international use, including that of the United States, has appeared to be the assessment of attitudes and degree of adjustment of persons to specific situations, conditions, and settings (see, e.g., studies of Costin & Eiserer, 1949; Irving, 1967; Sanford, Adkins, Miller, & Cobb, 1943; Wilson, 1949). Since 1976, there have been 864 studies reported in the psychological literature using the sentence completion. These studies cannot be reviewed here as that is not the purpose of this chapter. Nevertheless, it may be useful to know at least how the sentence completion has been most recently employed. For this purpose, the author briefly

BRIEF HISTORY AND DISCUSSION OF THE TECHNIQUE The sentence completion has had many uses since Payne’s initial creation, ranging from 159

reyn2-6.qxd

6/6/2003

9:07 AM

Page 160

160

II. PROJECTIVE METHODS

analyzes the functions of the last 100 studies involving the sentence completion that have been reported for this past decade (the 1990s) in Psychinfo (American Psychological Association, 1967–2000). (See Table 6.1.) It is rather a surprising trend to see that the sentence completion has become an important research instrument in the burgeoning fields of neuropsychology and linguistics. This, of course, exceeds its past history as primarily an instrument measuring personal adjustment. Even in its use as a measure of personal functioning, the sentence completion is now often used as an outcome measure. Those familiar with the sentence completion will realize the difficulties in this use—the lack of reliability and validity for many sentence completion measures, the highly clinical nature of establishing clear outcome targets expected on the sentence completion, and so forth. And yet, apparently due to its extreme degree of face validity (it is hard to interpret “I . . . hate school” as anything but rejection of the school experience), the sentence completion seems to be making itself useful more and more in a wide range of studies. The topics under “personal functioning” that most studies of that type appear to be measuring by use of the sentence completion are illustrated in Table 6.2. Only three studies in this whole decade appear to be centered directly on clinical analysis itself in the use of the sentence completion in the assessment of clients. This does not indicate that the sentence completion has become solely a research instru-

TABLE 6.1. Uses of the Sentence Completion Reported in the Last 100 Studies for 1990–2000 Studies of the sentence completion technique itself

12%

Neuropsychological studies using the sentence completion

14%

Studies of language using the sentence completion

18%

Studies of personal functioning Exploratory Correlational Outcome

56% 17% 25% 14%

TABLE 6.2. Personality Topics Investigated 1990–2000 Using 56 Sentence Completion Studies Exploratory Personal development by age Personal development of drug users Personal development of Ss with eating disorders Stages of ego development Ego development of twins reared apart Life satisfaction by the elderly Mother–child relationships Motives of psychology majors Styles of interaction of nurse therapists Patterns of maturity by age Correlational Social identity to personal history Trust to factors in adolescence Body image to dieting Ego development to a number of factors Fear of war to mental health Goal setting to personality traits Abnormal eating to family relations Conflict level to stages of adolescence Time perception in women to social roles Cognitive complexity to functioning as principals Reference patterns to depression Internal self representations to reality Irrational beliefs to impression management Outcome studies Effects of facilitated communication in autism Effects of incest on ego development Effects of special education on gifted maladjusted Effects of language programs on Romany children Effects of posttraumatic stress on meaninglessness Effect on values of large group awareness training Effects of psychodrama on depressed subjects Effects of using trained fifth graders as mentors Effects of instruction upon measures of ego

ment (in spite of the surprising degree to which this is happening). Clinical psychologists are “out there,” evaluating clients on a daily basis, writing reports, and meeting in conferences; they seldom have or take the time to report their activities to scholarly journals. It is a safe guess that the clinical use of the sentence completion still far outnumbers the research uses of it; neverthe-

reyn2-6.qxd

6/6/2003

9:07 AM

Page 161

6. Sentence Completion as a Tool

less, it must be noted that the sentence completion is being accepted more and more as a reliable measure of outcomes in spite of its lack of standardization. A number of forms of the sentence completion technique are in use today. In many studies, the form is not reported and is obviously one invented for the occasion. A particularly impressive standardized sentence completion instrument has appeared during this time, Loevinger’s (1998) University of Washington Sentence Completion, which is involved in about a dozen of the studies reported in Table 6.2. Its scientific properties—reliability, validity, interpretation—have been studied far more than usual. It is centered on the concept of ego development. Another, the Hart Sentence Completion Test (Hart, 1986), also has more than the usual degree of psychometric sophistication. Hart describes the measure as an “effective social-emotional screening instrument” but notes that it is robust enough to predict behavior and to suggest appropriate educational intervention strategies. Other sentence completions are available today, of course, some for specific purposes. Those interested in efforts that have been made to standardize the sentence completion responses may be interested in reviewing an earlier version of this chapter (Haak, 1990). It is as much the case now as a decade ago that the sentence completion technique could profit from a much higher degree of study than it has received in both standardization and clinical interpretation. Nevertheless, the procedure remains so face valid, flexible, and useful, its results so disclosing, that both clinicians and researchers continue to use it extensively. Is the sentence completion primarily just a form of structured interview? Some suggest that this is the case (Dean, 1984; Hart, Kehle, & Davies, 1964; Knoff, 1963). Nevertheless, the sentence completion covers a broader range of subjects faster than the interview, and with more uniformity, especially when time and comprehensiveness are important. Sentence completion remains a highly useful clinical instrument 70 years after it was invented. It may not always be a good thermometer, but it is nearly always a useful x-ray.

161

SELECTING A SENTENCE COMPLETION INSTRUMENT Components of a Satisfactory Sentence Completion for Assessing Emotional Status Although the sentence completion format has seldom been subjected to the desirable reliability and validity studies, there are ways to judge a more dependable and useful measure to be used in establishing the emotional status of a subject. A satisfactory sentence completion for this purpose will usually contain the following features: 1. The sentence completion form needs to elicit responses that can contribute to a general judgment about mental health versus mental disturbance. It also needs to elicit responses that tap the major categories of mental disturbance, such as anxiety and depression. There could hardly be a case of potential emotional disturbance where these judgments would not be necessary. 2. The sentence completion form needs to yield information of a unique or personal nature. It should provide evidence of specific concerns and allow for the discovery of the mental organization and motivational organization of the individual client. This imposition of structure upon structure is part of the richness of the sentence completion’s possibilities. 3. The sentence completion form needs to be biased toward the population and the questions regarding that population, which the examiner needs to assess. The instrument is not a “fishing trip” for pathology. It needs to be geared to the population and the context in question. 4. The sentence completion needs to provide a sufficient number of the types of stems in question so that some degree of confidence can be put in their interpretation. For example, to have only one stem geared to achievement in a school population would be pointless, for the response would be undependable. Six stems about achievement scattered throughout the protocol, on the other hand, might provide much more reliable information. As in all psychological testing, one response cannot be viewed as reliable, although a single response may at times be highly pathognomonic and meaningful.

reyn2-6.qxd

6/6/2003

162

9:07 AM

Page 162

II. PROJECTIVE METHODS

5. The sentence completion should be appropriate for the age served. Adolescents are “put off” by childish language. Younger children cannot “hang on” to a long stem (neither can children with reduced mental ability or severely depressed children).

The Haak Sentence Completion Forms Haak developed a sentence completion instrument (in its first version) in conjunction with the then Learning Disabilities Center of the University of Texas at Austin in 1973. The final version of this instrument is the one that is used for discussion and illustration in this chapter. Two levels of the current forms exist: the “elementary,” for children approximately below the eighth grade, and the “secondary,” for students above that level. These titles are rough guidelines, however, as mental ability and maturity can affect which form the examiner chooses to use. As far as downward extension goes, most normal 5-year-olds can respond to the elementary form and many younger children can do so. The original version of the Haak Sentence Completion was based partly on the earlier work of the late Robert Peck, who provided some of the original stems and the critical insight to organize the stems into content areas to improve reliability of interpretation. The first version of the Haaks was also influenced by the late Dr. Fern Williams, first director of the Learning Disabilities Center. It was she who requested the writer’s help to develop a relatively short procedure for screening the emotional problems of children brought to the Center for learning problems. It was she who also contributed some of the categories of the protocol, particularly the “Openness to Help” category; and it was she who first named the instrument the “Haak.” Like other sentence completion forms, this instrument has never been adequately studied, though three doctoral dissertations have addressed it (Baggerly, 1999; Baker, 1988; Wells, 2000). The Haaks are not claimed to be model forms of the sentence completion technique, although they do meet the requirements discussed earlier for an adequate sentence completion instrument. They are widely used in the Texas area and elsewhere (Haak, 1996, 2000). (Psychologists interested in using the

Haaks may obtain permission by writing the author. See Appendix 6.1.) Even the most useful sentence completion can be invalidated by its delivery. If the instrument is read to children, it should be read in the most neutral manner and tone possible. One should read the stems as quickly as the child can respond to them without, of course, creating a frantic pace. There is one exception to this habit: When a child shows by his or her behavior that he or she is thinking about a response before giving it, one should exercise restraint and simply wait the child out. Such responses are usually telling. Sometimes a child waits so long to give a response, and works it over in his or her head so long, that he or she loses the stem. The stem may be repeated, but if the child chooses to avoid it, the examiner should move on (discreetly marking such a response for his or her own benefit in interpretation later). It is best not stop and discuss responses. Some query of responses may be carried on when the protocol is finished, but even this is likely to create suspicion and distrust if there are remaining instruments to complete. The child who is being examined will take his or her cues from the examiner; therefore, a calm, pleasant, but neutral administration is best. The more such an attitude can be maintained, the more confiding the child is likely to be. If the examiner behaves nervously about any items on the sentence completion, the child will notice. This is also true if the examiner reacts to the child’s responses. An effective examiner should be as nonreactive to “I like . . . axe murderers” as to “I like . . . chocolate cake.”

USING THE SENTENCE COMPLETION AS PART OF A TEST BATTERY The sentence completion should normally be used as part of a comprehensive battery of tests if the emotional status of a subject is being evaluated. Hypotheses generated by the sentence completion can be explored for support or nonsupport in the other, more objective data (and vice versa). The assumption is that a rounded battery of tests will generate data that can be integrated into a meaningful, organized picture of the client’s

reyn2-6.qxd

6/6/2003

9:07 AM

Page 163

6. Sentence Completion as a Tool

functioning. This picture, of course, must invariably rest on some theoretical base. The sentence completion is particularly suited for use with a theoretical orientation to the assessment, which depends on general systems theory. In systems theory, the client—the schoolage child—constitutes a small system composed of important subsystems: physical, intellectual, emotional, temperamental, and attitudinal, at the least. These subsystems interact in ways that support or interfere with each other to produce the behavior of the child in question. A child, or anyone else, whose major subsystems function together to support normal daily functioning, growth, and development can be said to be healthy. When a child is healthy, however, that judgment infers more than just that the child him- or herself is doing well, and this is because human beings are open systems, affected by their surroundings. The child operates within a larger context of systems—home, school, and community and, with increasing age, national and global systems. It is difficult for even a healthy child to function normally if he or she must live within a dysfunctional home, school, or neighborhood system. The effects of such negative factors are felt by the child. It may be the case, of course, that at times the supportive systems around the child are functionally adequate for most normal children, but the child in question has serious problems due to the imbalance within his or her own system caused by disabilities (inadequate subsystems). Disabilities distort the smooth growth and development of the child, and that fact makes demands on the environment for compensatory interventions and treatments to improve or restore the inadequate subsystem(s). In the best scenario, these demands are met and the child is restored to normal functioning. However, the more common case occurs when disabilities are found to exist in both the child and the environment. When disabilities exist in both the child and his or her environment, in its mildest form this is certainly understandable—it is wearing on parents and teachers to make the daily compensations that must be made for a handicapped child. Such compensations are often expensive in terms of both

163

time and money. In its most virulent form, the disabilities in the environment may actually cause or definitely exacerbate the disabilities in the child. This is the complicated situation that the assessor most often faces when conducting an emotional evaluation. One must understand how the dysfunctional aspects of a child’s system itself interact with possible causes or contributors in systems at large and then make appropriate recommendations for remediation, alleviation, or compensation of these dysfunctions wherever they exist. Such is the role of the comprehensive assessment; and it is rather a daunting task. When an examiner is first given a referral, he or she must consider how to approach the examination task. Some highly obvious disabilities such as physical, visual, or auditory problems have their own specialized pathways to be followed in assessment. However, a child referred for emotional problems is not so uniformly approached because one is sure of neither the problems nor their sources. Personally, I prefer to approach such an examination by gathering and considering the data in a most objective to least objective track—standardized intellectual and achievement data first, health history, developmental history, history of previous treatments (if any), referral information, parent and teacher information and checklists next, and, finally, the personal examination of the child—checklists, sentence completion, drawings, and projective stories. (Here we speak of nonstandardized clinical data originating with the subject as “subjective” and other data as “objective,” recognizing that much of the “objective” data [i.e., parent and teacher information] are hardly objective at all in the true sense of the word.) When beginning with the more objective data, a framework—intellectual abilities, physical information, history of treatment, and opinions of teachers and parents—is created to which the dimensions of the more subjective data can be related. Sometimes the objective explains the subjective, and sometimes vice versa. A comprehensive examination for emotional disturbance should result in a tightly woven assessment in which issues of major concern are supported by data from multiple sources in the assessment.

reyn2-6.qxd

6/6/2003

164

9:07 AM

Page 164

II. PROJECTIVE METHODS

School is the work of the child; the intellectual abilities of the child are the tools he or she has to do the work. That is why it is critical to have a full battery of results from the Wechsler Intelligence Scale for Children—III (Wechsler, 1991), the Stanford– Binet Intelligence Scale IV (Thorndike, Hagen, & Sattler, 1986), or the Kaufman Assessment Battery for Children (Kaufman & Kaufman, 1983). Also, as important as the verbal IQ and performance IQ are, an analysis of the individual subscores of the test is equally informative. Anyone not comfortable with analyzing these subscores can find a useful assist from several measures of organization of these subscores—for example, the Individual Ability Profile (Dean, 1983)—which casts these subscores into more neuropsychologically meaningful and useful dimensions. History is, of course, critical. The developmental accomplishments (talking, walking, etc.) are little standardized tests within themselves. Is this a child who met these tests successfully? The history tells us about past illnesses, accidents, and accomplishments. The history tells us about the child’s environment. Nowadays, for example, one wishes to understand how many adjustments the child has been called on to make (e.g., in relating to father or mother figures in the home). What are the stresses with which the child has had to cope, and how are they reflected in the child’s more subjective productions? In gathering parent and teacher information, it is common today to use, in addition to history questionnaires, some form of behavioral checklists. The checklists with perhaps the most acceptable reliability and validity are the Achenbach Child Behavior Checklists (Achenbach, 1991), although others have now been developed that meet reliability and validity standards. Nevertheless, it should be remembered that however much behavior scales may have been standardized, they are not necessarily reliable or valid in the individual case. The instrument may be reliable in the general population, but the person filling it out may not be. This is not altogether an unfortunate thing. One learns methods of evaluating the evaluators—a process that provides highly useful information about the major individuals with whom the child interacts. For example,

when teacher reports and parent reports agree, it is seldom the case that either one of them is inaccurate in reporting behavior. When they do not agree, there are several possibilities. One possibility is that the child behaves differently in the two settings represented by the two reporters. Some assessors do not give full credence to this possibility. It is likely that the school is the problem when this happens. Many children escape from school to a happier, more relaxed home life; few children escape from a disturbed home into school and function well there. Some children do function differently in different settings, and when they do, this ability argues against a diagnosis of generalized emotional disturbance. There are several other possibilities when teacher and parent behavioral reports disagree. Either of these reporters may be reflecting a present high level of concern about the child in the reporter’s setting. In this case, the concerned reporter will elevate most scores. The profile of the child based on such data will be elevated across the board, violating the factor structure that provided the profile. This is not liable to be a realistic report of the child, but it is an indication of the reporter’s level of concern, either with the child or from other sources of stress. Reporters may also have definite feelings about the child’s eventual diagnosis, which they know is forthcoming. If they personally believe the child is disturbed, one way to ensure this diagnosis is to elevate scores. The reverse, too, is also true: the underreporter, whose scores create a flat profile, may be expressing his or her idea about whether a diagnosis of disturbance is appropriate. Or, he or she may be registering a hostile attitude toward the examination itself (“There is nothing wrong with my family. . . . Leave us alone . . . etc.”) When either the teacher or the parent reports in a manner that generates a differentiated profile of results, and the other reporter produces either a flat profile or a generally elevated profile of scores, it is usually the case that the reporter who produced the differentiated profile is more accurate. A behavior observation in the classroom can also be a valuable piece of information. Some teachers are truly expert in performing these observations, providing a concrete

reyn2-6.qxd

6/6/2003

9:07 AM

Page 165

6. Sentence Completion as a Tool

sample of a child’s behavior that answers many of the questions raised by the checklist reports. The most valid procedure for obtaining behavioral observations is to use one teacher who regularly makes these observations. If this is not possible, teachers can perhaps do observations in each other’s classes. It is nearly impossible for the head teacher in a classroom both to teach and to do a truly detailed, concrete behavioral observation at the same time. The examiner, of course, may be required to make these observations. The most common objection to this procedure is that the teacher may protest the resulting observation, saying that the child did not behave as he or she usually does. This may be true, as the examiner is a complete stranger to the classroom and may have a generally dampening effect on all behavior in the room while he or she is there. Unless the examiner is specifically invited to see the behavior of a certain student (this is not unusual with a teacher who is self-confident and concerned about a specific student), using a single teacher or corps of teachers trained to make concrete observations appears to produce the most reliable data. Once all the objective data (discussed earlier) have been gathered and examined, one is prepared to go ahead with the more “emotional” part of the exam. One has a set: One knows the capacities of the child, what has happened historically to the child, the environmental impact on the child, and something of the views of that child as seen by the parent and the teacher. It is time to go behind the curtain and see what effect all these variables have had on the child, for, after all, emotions are reactions. It is time to know what the emotions are reacting to. The child will tell us that.

USING THE SENTENCE COMPLETION TO ASSESS EMOTIONAL DISTURBANCE The focus of the remainder of this chapter is on the clinical interpretation of the sentence completion technique itself in evaluating school-age children for the condition of emotional disturbance. The intention here is to offer assistance to the user in analyzing the rich data usually obtained with this instrument toward that end. The sentence

165

completion is a particularly useful subjective instrument because it casts a wide net in a short period for issues of immediate concern to the assessor: relationships, work attitudes, view of self, ambitions, sources of emotional support or pain, the child’s own view of the educational difficulties, and others. The sentence completion is a semiprojective instrument: It allows the projection of the child to come into play, but it focuses on reality issues important to the assessment. Its face validity makes it more believable than more subjective approaches to professional persons who are not clinical psychologists. A complete emotional evaluation usually includes, however, the more subjective and projective instruments, particularly drawings and stories made up in response to stimuli such as pictures. As the examiner moves from more objective to less objective data, one loses the comforts of standardization and normalization but gains in richness of data. This is, of course, where clinical expertise becomes necessary.

Establishing “Emotional Disturbance” To establish the simple-appearing matter of whether or not the child is “emotionally disturbed” is the goal that must be kept in mind in working through all the child’s productions. This task is analogous to requiring a physician to certify someone as sick or well. Every professional person who deals in clinical settings knows that such arbitrary dichotomies are usually false. Some persons may be clearly sick and others clearly well, but most will fall into some degree of what is often called “the worried well,” neither free of concerns nor totally dysfunctional— and this applies also to children (who were not officially capable of being classified as depressed until 1970!). In addition to the fact that most functioning persons have some degree of identifiable problem dynamics, there are further difficulties that appear with children. Children are always going through predictable changes and stresses, if only those associated with stages of development. Each of these stages presents its known challenges, which are probably the result of temporary imbalances in physical and psychological growth and maturity. In Western culture, these seemingly

reyn2-6.qxd

6/6/2003

166

9:07 AM

Page 166

II. PROJECTIVE METHODS

inherent problems associated with aging have been variously conceptualized as “childhood stages” (Ilg & Ames, 1960; Spock, 1946), “psychosocial stages” (Freud, 1949), developmental tasks (Havighurst, 1953), and “psychosocial crises” (Erickson, 1959), all resting on some fundamental stages of neurological growth and development (Reinis & Goldman, 1980), cognitive accomplishments (Piaget & Inhelder, 1958), and the simpler matter of “on time” developmental milestones (age of sitting, standing, walking, talking, etc.). No psychological examiner should attempt to evaluate children without a sound knowledge of all these developmental relationships. Another complication of assessing children is that they are far less stable and predictable than adults. Assessors may have fairly constant notions of what capabilities to expect in a 30-year-old adult with an eighth-grade education with a Verbal IQ of 80 and a Performance IQ of 105. What to expect in a child with similar scores is far less clear for a number of reasons: (1) all test scores are less stable in children; (2) children’s performances are more affected by environmental conditions than those of adults; (3) children have not formed stable work habits; (4) children’s performances on verbal measures are affected by their state of verbal development; and (5) children’s performances are affected by the degree of pressure or structure they feel from significant adults, especially on items they feel to be revealing or intrusive. Finally, assessing “emotional disturbance” as a global condition of a child involves to a great extent, as we discussed previously, a judgment about the general adequacy of the environment in which the child finds him- or herself. Every practicing school psychologist has had to label a child “emotionally disturbed” when this disturbance was clearly the result and only the result of observable environmental deficiencies. There is rarely an analogous problem in adult psychology or psychiatry. Not all emotionally disturbed children fit this description, but a troubling number do.

Systems of Emotional Classification There are two major systems by which children today are classified as “emotionally

disturbed”: (1) the criteria of Public Law 94-142 (since partially revised as IDEA, but these criteria have not changed) and (2) the criteria set out in the fourth edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994). The federal criteria (Individuals with Disabilities Education Act [IDEA]) must be met for the child to be classified as “seriously emotionally disturbed” and receive appropriate services in the public school. The federal criteria are based on the work of Eli Bower (1969) and are as follows: (i) The term (“emotional disturbance”) means a condition exhibiting one or more of the following characteristics over a long period of time and to a marked degree, which adversely affects educational performance: (A) An inability to learn which cannot be explained by intellectual, sensory or health factors (B) An inability to build or maintain satisfactory interpersonal relationships with peers and teachers (C) Inappropriate types of behavior or feelings under normal circumstances (D) A general pervasive mood of unhappiness or depression; or (E) A tendency to develop physical symptoms or fears associated with personal or school problems. (ii) The term includes children who are schizophrenic. The term does not include children who are socially maladjusted, unless it is determined that they are also seriously emotionally disturbed. (Education for All Handicapped Children Act, 1975, §300.5)

The matter of what constitutes “social maladjustment” has never been satisfactorily determined, although it has been the center of much discussion. Also, recent legal and governmental opinion leads us to believe that abnormal feelings (the affection component) must be part of the reason for the diagnosis of “emotional disturbance,” not merely abnormal behavior per se. Apparently, the condition of emotional disturbance was never intended to cover problem behavior without the affective component. In short, for a child to receive many services today, especially services provided by special education in the public schools, some professional must finally make a judgment that a child is emotionally disturbed

reyn2-6.qxd

6/6/2003

9:07 AM

Page 167

6. Sentence Completion as a Tool

(the “sick” or “well” question). The reliability problem inherent in using such a label across the whole of the United States by variously trained personnel using various systems of judgment simply boggles the mind. Unlike other categories of disability, especially the physical ones which have clear-cut and measurably criteria, the disability of emotional disturbance remains a matter for professional judgment alone. The only available “reliability” for this situation at present is to ensure that such judgments are made by fully trained and experienced professional personnel without conflicts of interest, using a range of instruments which provide as much information as possible about the condition at issue. The professional who assesses a child must also determine at what point on the continuum of emotional disturbance he or she is to declare that a condition has reached the stage defined by federal law as “serious emotional disturbance.” After the matters of having the condition for a “long time” and having the condition to a considerable “degree” have been met, that judgment (a sufficient degree of emotional disturbance in the school setting) is supposed to be made on the basis of “educational need.” The effects of the proposed emotional disturbance on the child’s ability to learn, either academic material or the normal level of interactional (behavioral) skills, must be clear and serious. Often that point on the continuum is hard to determine and subject to argument. Many schools want to consider only “academic” need, yet the rule clearly states that retardation in “developmental” skill constitutes an educational need. And what of the bright child whose performance is significantly below expectation for his or her ability level but still average or better when compared to the whole population? “Educational need” is not always easy to determine. But it must be the final hurdle an assessor clears before judging a child to be emotionally disturbed.

INTERPRETING THE SENTENCE COMPLETION If it is the case that the sentence completion can make a major contribution to the deci-

167

sion of whether or not a child is emotionally disturbed, it is also the case that a critical element in acquiring the interpretive knowledge necessary to be comfortable with the sentence completion is the matter of knowing what to expect in the child’s response. “Norming” the sentence completion responses is inextricably meshed with the reliability and validity questions and knowledge of developmental stages noted earlier. The highly experienced examiner carries such norms in his or her head, but that is not helpful to other examiners, especially those who are just beginning their careers. In one attempt to generate some such “norms,” Peck and Haak (1973) studied the responses of 600 normal children ages 10 and 14 to the sentence completion and found some surprising results. For example, 75% of the children would actively problem-solve if they saw others doing it successfully in school, but only 50% would ask for appropriate help. Also, only half would attempt to resolve a problem with a friend or comply with an (adult) request, and only 33% would try to do something active about a class bully or try to deal with parental anger. In short, coping skills in normal preadolescents were not remarkably positive. It would seem from such results that one should be hesitant to overvalue such deficits in an individual child being evaluated for pathology. All types of norms for children need to be understood by the assessor, including subcultural norms. But particularly important are developmental norms and language developmental norms. For example, it is fairly typical of a 5-year-old to state that his or her favorite school activity is “playing”; this response given by an 8-year-old is a red flag for developmental delay. Unfortunately, the ability to make many of such interpretations comes from experience. The sentence completion contains both behavioral and affective information seen from the viewpoint of the child. The behavioral reports of the child can be checked against the “reality” of the behavioral reports of the assessor and parents, teachers, and students. The affective report can be checked against the “reality” of the parents’, teachers’, and students’ perception of how they think the child feels (an area in which great discrepancies often exist). Some

reyn2-6.qxd

6/6/2003

9:07 AM

168

Page 168

II. PROJECTIVE METHODS

of the comparative judgments that can usually be made with assistance from the sentence completion data are as follows.

The “Rule-Outs” Intellectual Difficulties A clinician experienced with the sentence completion can often predict a child’s IQ score within a few points from the sentence completion data. This can be done primarily on the basis of fullness or poverty of language usage, articulateness, concreteness of language, complexity of language, perseveration in language, imagination, general level of expressiveness, empathy, openness versus defensiveness, expressed social intelligence, and qualifications of responses. In particular, qualification of responses almost never occurs without good intellectual potential (“I am generally nice but sometimes I am not”). Certainly no emotional evaluation should be conducted without hard, recent data regarding intellectual potential and achievement status. But the sentence completion can act as a functional check on other intellectual data, especially in the sense of indicating more potential than the child is presently able to demonstrate on testing. (Beware, however, of overstating this case; a great deal of damage is done by careless clinicians who lead parents to believe their child is “much brighter than the test scores indicate.” Such statements sometimes lead to years of parental pressure on a child who is not having his or her needs met.) It is also not enough to have simple intellectual “test scores” without considering which test was used, the subscores involved, and the pattern of subscores reported by the test. (We have discussed this issue previously.) A child with a great deal of within-child variance in important mental abilities, as represented by the subscores on the aforementioned tests, may be suffering from learning disabilities, other neuropsychological dysfunctions, or a mismatch between the abilities of the child and the curriculum he or she is required to master. Such a child can experience much stress and discomfort with school. It is also common to find children who have masked intellectual difficulties for

years behind a front of exceptional social competence. This particular adaptation often breaks down at the high school level, when the student’s social competence can no longer compensate for the increasing degree of difficulty of educational expectations. The charming little girl who has watered the pot plants all the way from first grade to high school is suddenly engaging in acting-out behaviors. Be suspicious that she has real difficulties in meeting new and higher expectations. Will these academic stresses be reflected in the child’s sentence completions? Almost always. The child complains that school and certain subjects are hard; the teacher does not try to help him or her; there is no use to study; the tests are tricky; peers are rude; the smart kids get to do everything. The self-concept often reflects feelings of stupidity and hopelessness. Increased alienation from peers and school is reflected in many stems. Depression may be expressed in some stems. A child experiencing intellectual difficulties may also be experiencing emotional disturbance, of course; however, if the emotional symptoms being displayed can be directly traced to the intellectual deficits and the stress that the deficits place on the child, then it is improper to label the child “emotionally disturbed.” In such a case, the emotional symptoms are secondary effects of the primary diagnosis, and the child requires other academic remedies than emotional remedies, though emotional support for a time may be helpful indeed. Attention-Deficit Disorder (AttentionDeficit/Hyperactivity Disorder) Symptoms of attention-deficit disorder (attention-deficit/hyperactivity disorder; ADD/ ADHD) can be easily elicited by the sentence completion. In fact, the child with one of these disorders will commonly and openly complain about his or her own attentional problems (“I have a hard time paying attention”; “I often think about other things”). The child will also often complain that others shout at him or her, yell too much, lose their tempers at him or her, are too impatient, become exasperated, and are otherwise frustrated with the child (they are!). The child often seeks isolation, free-

reyn2-6.qxd

6/6/2003

9:07 AM

Page 169

6. Sentence Completion as a Tool

dom, a high degree of personal movement (the life of a bird, motorcycles, high-speed cars) (i.e., being alone, in control, and away from the “bugging” of others). The child with ADD/ADHD typically demonstrates a “thin-skinned” attitude and feels that other people are always picking on him or her. The child often feels acutely unfairly treated. No one understands the child. He or she nonetheless displays a paucity of empathy and understanding of others and sees nothing from any viewpoint other than his or her own. The empathetic and reflective processes are usually markedly deficient. This does not mean that the child does not give some positive social responses; indeed, this is often the case. But positive social attitudes are positive in the conceptual sense only and often do not translate into concrete social skills. (Again, beware: We speak of the typical case. Because of other compensating factors, an individual child with ADD/ADHD may be quite socially competent.) The child with ADD/ADHD often projects few plans, including plans of action. This of course may relate to deficient frontal lobe development. This lack of planning is often a critical factor. Responses to the stems that refer to future plans are often vague, undifferentiated, and obviously not previously considered. And, there is another matter connected with this paucity of projection: The child does not appear to feel “responsible” for impulsive actions that he or she did not mentally consider in the first place. (Indeed, the history of common law agrees with this concept of premeditation and responsibility.) Such children complain bitterly about other people’s reactions to their impulsive misbehavior. And the other people rail (constantly!) about the child’s lack of “responsibility.” Between the two points of view rests a “moral” battleground. These issues are often reflected in the sentence completions and in the referral originally received. (“My father thinks I am irresponsible.” “My teacher thinks I am stupid.” “I wish teachers would leave me alone.” “My friends always expect me to get in trouble first.” “Adults blame me for everything.” “I am always blamed.”) As previously stated, social intelligence is often not lacking in the sentence completion data of an ADD/ADHD child. Such a

169

child will express positive social attitudes that adults will scarcely believe (and often do not). Rather than being immune to the effects of others, such children seem hypersensitive to these effects. They are often in acute states of affective starvation, with perhaps the mother the only compensating factor in the picture (“My mother loves me anyway”). Their behavior, of course, is what puts them in this position. Attentional deficit leads to emotional double binds for children, and many of their difficulties are expressed in the sentence completion data. How does one differentiate whether the symptoms displayed in the sentence completions of a child with possible ADD/ADHD are attentionally or emotionally based, particularly when the child has not been previously diagnosed as having an attention deficit? This is a difficult and ongoing question even for research. The current answer appears to be that when almost all the troublesome sentence completion responses coalesce into what is known of the common secondary symptoms associated with the deficit (examples have been given previously), one can clearly feel comfortable in assuming that ADD/ADHD rather than something else is the basic cause of the child’s disturbing behaviors. At that point, when one has clustered the emotionally loaded sentence completion responses of the child with true ADD/ADHD, few such responses will be left out of the picture painted previously. Of course, a child can be both attentionally disordered and emotionally disturbed. In that case, the examiner will see the usual coalescence of secondary symptoms associated with ADD/ADHD, but there will also be a large number of negative, emotionally loaded responses left over that do not neatly fit this picture and do fit another symptom picture, such as depression. (One caution needs to be observed in this regard, however, as a mild level of depression is common in ADD/ADHD children.) Nevertheless, strong levels of emotional conditions, including depression, should not be viewed as deriving solely from attentional deficits. It is poor practice to classify such a child as ADD/ADHD and ignore the second or third symptom picture because one has already gotten the child “qualified.” Such a child

reyn2-6.qxd

6/6/2003

170

9:07 AM

Page 170

II. PROJECTIVE METHODS

will perhaps receive services for one condition and fail to thrive because other disabilities have been ignored. Stress One of the major rule-out decisions facing the clinician who must decide the emotional status of a child is a decision regarding the contribution of stress to the child’s symptom picture. Sometimes this decision is difficult. The clinician needs to consider the number and intensities of stressors upon the child, their recency, and their duration. Important is the child’s reaction(s) to the stressor(s). A child for whom multiple and strong stresses in the immediate past are found cannot safely be fudged to be emotionally disturbed unless the emotional disturbance preceded the period of the stressors or is in itself extreme. The “rodeo rule” is particularly apt here: One must evaluate the rider by first evaluating the horse. Evaluating the “horse” (in this case the stressors, of course) is not always simple. Persons causing the stresses will often be tempted to underplay their role in upsetting the child. Parents looking for an excuse for their child’s condition that does not point back to any kind of family variable will overevaluate minor “outside” stressors that any healthy child should be able to handle. Teachers may exaggerate home conditions for the same reason. Yet, difficult as it may be, it is usually productive to try to get an accurate picture of the “horse” before proceeding to evaluate the strength of the rider. (A discussion of how to judge the accuracy of parental and teacher data appeared earlier.) And, of course, life being what it is, the causes of the child’s stresses may be beyond the ability of any human being to change (death of a parent, terminal illness in the family, etc.). Obviously, this latter event is blatant to anyone and does not require a complicated assessment to determine that a severe stress exists. Some typical responses of children under heavy stress include the following: the child will report physical problems, often more than anyone else has reported for him or her (adults are surprised to learn the child says he experiences daily stomachaches in the morning, for example). The physical conditions of headache and stomachache may be

reported as occurring together. In this case— and in any other appearing potentially serious—the clinician is justified in asking the parents to obtain a physical examination of the child. Although stress and anxiety can cause co-occurrence of symptoms, so can subclinical seizures, migraine headaches, and other serious conditions. Although it is the case that most often the child will be found to have no physical problems, this judgment can only be made by a person trained to do so and should not be assumed. The content of the stressed child’s sentence completions can reflect a low level of energy (“I am tired most of the time”; “I cannot get my lessons”). There is seldom the focused hostility of the acting-out child, for the student is often blaming him- or herself for any problems at this stage. There may be consequent confusion and sparse, somewhat noninformative and defensive responses. But, in any case, the most important feature of the sentence completion responses in the case of an overstressed child who is not emotionally disturbed is that usually his or her responses are “better” than one would have expected from knowledge of the stressors (and often from knowledge of overt behavior). This is important. There will be some preserved positive social attitudes, even though spotty; preserved though damaged belief in oneself (“The one thing people need to understand about me is that I am really trying . . .”; “I am really a good person”); articulated though not necessarily followed-out coping strategies; some belief that things will get better; and some hope for the future with scraps, at least, of future plans. The examiner’s reaction to such a child is often something like, “This is a pretty good kid!” The examiner finds him- or herself “rooting” for the child. These reactions in the examiner are an important source of data. That one wishes to rescue such a child is telling—for this is a child who usually needs to be “rescued.” Sad to say, when multiple and/or strong stressors persist over time, they will almost inevitably erode the functional capacities of the stressed child. Finally, that sad state is reached in which, even though all the causes of the child disturbance can be traced directly to the environment, the child is actually emotionally disturbed. His or her functional capacity is scarred as a consequence of the

reyn2-6.qxd

6/6/2003

9:07 AM

Page 171

6. Sentence Completion as a Tool

damaging conditions. To call the child emotionally disturbed—a term many still see as pejorative—seems unfair to the child. To fail to call the child emotionally disturbed, which he or she has finally become, may be unrealistic and, more important, may deny the child critical services. And, of course, it is dishonest. Most often this diagnostic call in strong cases of stress will be resolved by the presence of depression, which clearly qualifies the child for services. Fortunately, when these cases are discovered earlier and at milder levels, interventions can be made to circumvent the further effects of the unreasonable stressors. A proper report that outlines the problems and suggests practical interventions is due the child, his or her teachers, and his or her family. Just because the child does not “qualify” for services in special education does not mean that proper services should not be sought in the regular education programs of the school and in the community. Specifying a time to reevaluate the condition of the child (if only informally) can also be helpful, as it serves as a surveillance function or goad for others to seek proper services. The assessor becomes the child’s informal advocate in such cases. Of course, we have not considered the case in which the child’s stresses are emanating from activities that are against the law. Clearly the examiner is duty-bound to report such events to the proper authorities. In that case, it is often best to inform the school’s major administrator of what has been found and the fact it will be reported to the proper authorities. School at times may be the child’s best protector—sadly, at times his or her only protector.

2.

3. 4.

5.

6.

7.

The “Rule-Ins” Depression

8.

A depressed child almost never completely “escapes” the sentence completion. Evidence for depression may be found in the following instances, at the least:

9.

1. Outright references to depression, flatly stating that the child is depressed and sad. (It is rather remarkable how a child can often say this and have it totally ignored.) Other language the child

10.

171

may use to indicate depression includes tiredness, hopelessness, noninterest in the daily affairs of school or play, a desire for isolation, and a need to escape the school situation. Outright reference to self-destruction. These references can also be vague, such as responding to questions about future plans with an indication the child will no longer be around. Responses that confuse anger and sadness (“When I am angry, I cry”; and “When I am sad, I am mad”). A response to either of the following stems that indicates sadness: “I am nearly always . . .” “I am never . . .” The denial form of this particular stem often allows the child to express a depression he or she otherwise defends against. Quite a number of expressions of anger combined with a number of expressions of positive social and moral attitudes (depressed persons are more apt to be socialized than not). A self-concept almost altogether negative except, at times, for the person’s own view of him- or herself (i.e., he or she may report that others see him or her negatively, but he or she holds on to a positive though perhaps defensive self view). Homicidal ideation. This is not rare and in recent years, with the advance of violent video games, movies, and TV, appears to be more easily expressed and less socially inhabited than previously. (This means, also, that its expression is harder to judge.) Such ideation can be accompanied by concrete and vivid images. Anger may be more obvious in some younger children, who lack the controls to subvert their depression into adult type, self-depreciative forms of this emotion. The expression of a conscious level of nonaffection and nonsupport from parental or adult figures, often with a yearning for this support. Premature sexual and romantic ideas. A young child thinking about getting married, for example, has usually given up on obtaining any immediate source of affection. A general paucity of expressiveness or high degrees of defensiveness in com-

reyn2-6.qxd

6/6/2003

172

9:07 AM

Page 172

II. PROJECTIVE METHODS

pleting the sentence completion stems; protests over having to complete the instrument or obvious upset in doing so; demands to stop the examination; obvious and building agitation; anger directed toward the examiner (“My biggest problem is you”). These behaviors in completing the instrument are an indication of the low tolerance of depression. 11. Intolerance of noise and stimulation (“Everyone yells”). 12. A strong desire for activities combining escape, aloneness, and self-sufficiency, sometimes but not always involving the element of speed (“Nothing feels as good as being alone.” “I would make a good pilot.” “If I were a king, I would go skiing on a high magic mountain— the tallest in the world.” “I often daydream about life on a little island.”). Of course, the quantity and intensity of such expressions as the aforementioned will determine to what degree one suspects depression in a child. But the sentence completion is a productive place to look for depression, for its partial structuring of the responses in the sentences stems is an aid to expression in the low-energy, depressed youngster. Anxiety The presence of strong anxiety in a child may exhibit itself in the sentence completion as follows: 1. A large number of statements expressing fear and apprehension. One needs to be careful in evaluating fear, however, for some fears are known to be associated with certain developmental states. Also, certain fears are related to real-life concerns, and in such cases, these statements are more indicative of stress than of an emotional disturbance of anxiety. Children also have fears blown out of proportion by television, even well-meaning TV ads. For example, it is common now to find children who fear that their parents are eminently going to die because they smoke cigarettes. The number and intensity of fears can be helpful in evaluating whether fears are indicative of a generalized anxiety state.

2. Unexpected expressions of anxiety in response to otherwise innocuous stems. When a child responds anxiously to a stem designed to elicit anxiety, that is one thing; when the same child gives an anxious response to a stem probing for other issues, that is another. (A couple of examples: “If I had all the books in the world, I would stack them up around me and hide inside” or . . . use them to throw at burglars.”) 3. A number of statements that express vulnerability. Anxious students do not feel safe. One indication of vulnerability does not constitute anxiety; however, several normally do. 4. Premature responding to an item (an instance of overreaction). Anxious students often cannot wait for the examiner to finish the sentence stem (if it is being read aloud) or do not read the whole stem carefully themselves. It is valuable for the examiner to notice the content of a stem that draws this type of overreaction. What develops with such a sentence completion may be logical in thought but simply incorrect in grammatical structure (“I often think that she is leaving”). 5. Overproduction of responses. The overproduction of the anxious (including the compulsive) student is mere verbal running on and not the same as the long, complex, and, more important, qualified responses of the articulate, thoughtful student. The anxious student may repeat him- or herself over and over; go off on a pointless tangent in a linear, compulsive manner; or simply run on and on, unable to put a stop to his or her verbalization. This may happen particularly in response to a stem elicited to probe for anxiety. Such behavior is indicative, of course, of anxiety. 6. Lack of defensiveness. Most children are open in their responses to the sentence completion, and a few terse sentences are not necessarily significant. A few stems simply may not interest the child or allow for little development. The following example—“When my father gives me a lot of work, he gives me a lot of work”—may be quite meaningful in relation to other stems suggesting hostility or problems with authority, but if no such responses exist elsewhere, the single, terse response should not be overevaluated. Nevertheless, the protocols of anxious children are usually void of

reyn2-6.qxd

6/6/2003

9:07 AM

Page 173

6. Sentence Completion as a Tool

defensiveness and replete with overresponding because control over their responses (or anything else) is beyond their power. 7. Worry. Anxious children indicate in many responses that they are worried, even in response to stems often seen as pleasant. The psychologist working with an anxious child will not lack for reports of agitated behavior in the child. But the sentence completion information will help the examiner to determine the sources of this agitation and the extent of it. The absence of many secondary symptoms of ADD/ADHD, the ruling out of major stresses, the ruling out of physical problems, the presence in the sentence completion of many responses typical of anxiety (as discussed previously)—these leave a picture of a child experiencing an emotional anxiety state. It is also common to find an anxiety state in a child suffering from other emotional problems, especially depression. Thought Disturbance Thought disturbance is certainly an indication of disturbance, and a rather uncommon, severe one in a child at that. Children may truncate their responses, summarize their responses, respond with partial data, suppress their responses, convert their responses, and do all manner of things to try to avoid fully expressing how they actually feel or what they think. These maneuvers are generally unsuccessful if the testing instrument is of sufficient length. But thought disturbance is generally frankly expressed: the thought content is just outright disturbed. And it is usually expressed in a manner that is at odds with the context of the stimulation. Indications of a thought disturbance produced by the sentence completion include the expression of illogical ideas and the expressions of these and other ideas in grammatically abnormal, fractionated sentence structure. One highly illogical thought does not constitute thought disturbance, but it should not be ignored either. One such red flag may alert the examiner to look for thought disturbance at a more subtle level in other expressions. Thought disturbance also has an intrusive quality. This can be overt, in the interruption of an otherwise appropriate response; or it can be inferred from a response incongruent with the sen-

173

tence stems (“My father wanted I was a chicken in the Easter play.” “If I were a king, I would buy a whole truckload of toys. I would invite all the children. And I would murder the queen. We would all get together in the park and have a good time.” “In my family, I am he cane to see us.”). Thought disturbance may be differentiated from the premature responding of the anxious student. Even though the anxious student’s response may not be grammatically smooth, the responses will usually be in the same ballpark as the stem. In thought disturbance, the response may have no obvious relation to the stem (though it may have an illogical personal relationship, which at times the examiner can trace). The use of highly convoluted grammar itself is often a sign of thought disturbance. Grammatical improprieties are rare in children, even uneducated ones. The grammatical mistakes of uneducated children are fairly well known and mean nothing psychologically (“I ain’t finished”; “He done it”). But actual fracturing of the basic grammatical structure should not be excused on the basis of lack of education. It may indicate brain damage, severe language dysfunction, neuropsychological conditions, psychosis, or a combination of any of them. The danger in assessing thought disturbance is the tendency of the examiner not to “make anything of it” with a response that makes no sense. There is the temptation to ignore a sentence or two like this (“Oh well, kids say a lot of nonsense things”). Although one is certainly not going to arrive at a diagnosis of thought disturbance upon the basis of a couple of responses, the opposite danger—to disregard that possibility because one cannot easily explain such sentences—is an equally dangerous behavior. The child in whom the possibility of thought disturbance is raised should have follow-up assessments, including the basic matter of whether the child’s auditory processes are intact. Often such a child can be quite disturbed but sedate and withdrawn, and no one appreciates the depth of the child’s suffering. It is such a child who may be “caught” by the sentence completion if the child’s responses are taken with appropriate seriousness. Such a child seriously needs help. In a rare case, a child who concocts a system of nonsense responses may not be

reyn2-6.qxd

6/6/2003

174

9:07 AM

Page 174

II. PROJECTIVE METHODS

thought disturbed but under some intense psychological pressure. The child is frightened to be giving responses to the threatening items in the sentence completion but sees no way out of it. One such protocol, for example, was given by a child involved in a bitter divorce and custody suit. He was being pummeled by both sides for his loyalty. In responding to the sentence completion, the child substituted animal figures for family members, actions for other actions, and so forth. For example, one of his responses was “I love to dance with a horse.” The examiner counseled with the boy and obtained a second, more straightforward protocol. The two could be laid side to side to see the inventive system the child had employed. Obviously, no thought-disturbed child could have done this, a task requiring superior intelligence, imagination, quick invention, and good short-term memory. Defensiveness must always be considered when a child gives nonsense responses to sentence stems. Even more, the reasons for the defensiveness need to be determined. But the fractionated, illogical responses of thought disturbance derive from, though probably include, more than the need to be defensive. Other “Rule-Ins” Depression, anxiety, and thought disturbance are major contributors to a final diagnosis of emotional disturbance. If the child is being assessed in school for possible services, the evidence for these conditions will still need to be subsumed under one of the categories of the definition of “emotional disturbance” (see previous discussion for a definition of “emotional disturbance” under federal law). In this system, depression has a category of its own, but anxiety, thought disturbance, and other extreme psychological conditions do not so easily fit under one of the remaining categories of that definition, namely, nonperformance for which no other cause can be found; inability to relate to teachers or peers; abnormal behavior under normal conditions; and psychosomatic behaviors and fears, plus, of course, schizophrenia—and specifically not “social maladjustment (see previous federal definition for exact wording for these categories). In my experience, a state of high anxiety

or extreme anger most often qualifies as abnormal behavior under normal conditions. Thought disturbance, fairly rare, may be the cause of any qualifying condition or may not qualify a student at all depending on the investigation of the causes for this condition. This leaves “the inability to relate to teachers or peers” and “psychosomatic behavior and fears” as two conditions we have not discussed. The reason for this lack of explication is that the evidence for these two categories is usually so overwhelmingly obvious, both in the sentence completion and in other instruments obtained. Data for the relational category usually come in with the referral, loud and clear, and are often the very basis of others’ concerns. It must be remembered that experienced teachers who are concerned about a child’s relationships have a great deal of normative data on which they make this judgment—usually more than psychologists. It is sometimes the case that the complaint is based on a teacher–pupil mismatch, but not often. The sentence completion is particularly helpful with assessing the quality of the child’s relationship. Blatantly expressed hostility in stems that measure relationships often characterize the child about whom others complain. Rarer but still indicating a severe relational problem is the totally withdrawn child who expresses this by short, unknowing types of responses. Such a child cannot relate how he or she feels about others or how they feel about him or her. Such a child often responds with “I don’t know.” The selfconcept is an empty shell. This behavior in the extreme may be a component of autism or Asperger’s syndrome, but it will more often be a consequence of emotional neglect or abuse. The child has simply not had ordinary relational opportunities. In other cases, when a child is referred for constant acting-out behavior, the assessor may find that the child has much more motivation to get along with people than others suspect; his or her motives are much more positive than people will believe. Absent are sentence completions that indicate strong negative feelings or ideas about others. Critical is that the sentence stems that often elicit emotional responses in the child with disturbed relationships simply are responded to with other or milder content. Such a child may have disturbances in his or her behavior

reyn2-6.qxd

6/6/2003

9:07 AM

Page 175

6. Sentence Completion as a Tool

but is not emotionally disturbed. The usual requirement for such a child is concrete assistance in acquiring skills needed to get along with peers and adults, sometimes a change in conditions and often short-term counseling. The teacher should be encouraged to explicate the concrete behaviors that are expected and not to assume the child knows them. Such a child is an ideal candidate for mentoring because he or she is often the product of an unstructured, fairly chaotic environment. One to one assistance is perfect for such a child. The psychosomatic category is also quite obvious. The child usually checks off a list of physical problems, if asked. He or she typically openly complains in the sentence completion of physical symptoms or fears. Often adults working with the child have not taken these reports seriously, or the child may exhibit these symptoms so much that everyone is thoroughly tired of it. Parents may resent the suggestion that there is something physically wrong with their child that they have not attended to. Nevertheless, an assessor cannot dismiss reported physical symptoms. In most cases, real physical disorders will indeed be ruled out. But in some cases, overlooked and sometimes serious problems will be discovered. Headaches and stomachaches together, for example, seldom denote nothing, as we have previously discussed. Only a physical examination can answer physical questions, and one needs to be done to complete a responsible assessment that contains physical complaints, if these problems have not already been addressed and ruled out. When a physical basis for the problems is not established, there is then the possibility that the child qualifies as emotionally disturbed. Usually this diagnosis has been made simpler by the discovery of a strong anxiety state in the child.

A Negative but Unclear Signal: Defensiveness Whereas the presence of ADD/ADHD and stress are rule-outs (ADD/ADHD has its own category, and stress is not yet disturbance) and anxiety, depression, and thought disturbance, as well as severe relationship difficulties and psychosomatic behaviors, are usually rule-ins for “emotional distur-

175

bance,” the presence of a high degree of defensiveness in the sentence completion protocols of children is a less clear and discrete signal. For the most part, the sentence completion protocols of children are generally nondefensive. Where there is the appearance of a high degree of defensiveness (e.g., stress), further examination needs to be pursued. Defensiveness will manifest itself as sparse, short, nonengaged, mundane kinds of responses. The child will be hesitant, both in overt and in verbal behavior, to engage in the task. His or her responses will be as unrevealing as possible. Sometimes the sentence completion format, however, brings forth a rich production of responses in an otherwise defensive student. A student who is remarkably hostile in behavior toward others, including psychological examiners, may flow forth in the sentence completion with the most blatant statements: exact plans for homicide, hatred of individuals, retaliatory schemes, injuries, fantasies, and bitter feelings. This degree of revelation can also occur in the positive sense: A student who appears defensive in his or her interactions with other people will reveal a high level of conceptual social intelligence, plans for changing the world to a better place, personal dreams and plans for the future, and even a detached perspective on those with whom he or she is having difficulties. The results of the sentence completion are hard to predict, and this is one of the features that makes it valuable (and interesting!). But there are students who react defensively to the sentence completion procedure itself. After all, the sentence completion format is straightforward, not disguised. Students know exactly what is wanted. (That is why they often take the opportunity to tell the examiner of their need for help—hoping something will come of their remarks.) But no matter how defensively the student reacts to the sentence completion, he or she cannot get around telling the examiner something. Not telling is also telling with the sentence completion. An analysis of the items to which the student is responding defensively can be instructive; further investigation into the daily life conditions of the student is often thereby warranted (why do they feel it so necessary to hide information on the defended sub-

reyn2-6.qxd

6/6/2003

176

9:07 AM

Page 176

II. PROJECTIVE METHODS

ject?). And, again, as with all conditions, the apparent “defensiveness” may rest on other than purely psychological conditions: lowered verbal ability, inferior intellectual ability, depression, shyness, lack of trust of the examiner, and fear of what will be done with the test results. Also important but often overlooked is the fact that some subcultures do not approve of a person’s discussing intimate and family matters with strangers, and such children can be made extremely uncomfortable by the sentence completion. There are also the cases in which children have been warned by their parents, who may have given reluctant permission for the testing to be done, that they are not to discuss certain matters away from home. These children are so busy trying to remember all the categories they are not to discuss, they simplify matters by attempting not to disclose anything. It needs to be remembered that children have few rights. An adult brought into a strange room by a strange person who asks all kinds of personal questions and gives one all kinds of tests without so much as the person’s own permission: How would an adult react to such a situation? Yet this is precisely what happens to children. It is up to the examiner to be straightforward, emotionally neutral, and pleasantly businesslike. It helps as far as possible to treat the child as an equal. The examiner should tell the child plainly that he or she is a person who works with children having problems in school. He or she is there to see whether together they can figure out why the child is having problems. The child certainly knows he or she is having trouble in school—it is no surprise. Inviting the child into a process that may alleviate that awful fact usually sets a positive tone. It is useful to let children express their own views of the problem (views that often turn out to be fairly accurate!), and if they ask for an opportunity to find out what the tests told about them, the examiner should be sure to set up such a session and follow through. Finally, if after all of the examiner’s best efforts are made to obtain a meaningful sentence completion protocol and the results of that effort are sparse, he or she may switch to a more projective approach. It is sometimes the case that a child who produces little content on the sentence completion will produce effusive projective stories. Such

children are frightened by the straightforwardness of the sentence completion and consider that imagination, as they perceive what is involved in the projectives, is less readable. For others, the unstructured nature of projectives means that they will guard even more against providing any revealing information.

Individuality: Beyond the Rules The sentence completion, as we have discussed previously, can help to rule in or rule out a diagnosis of emotional disturbance. But it can do more. The sentence completion can provide data that will help to reflect the individual structure of the child. If people did not show some uniformity in their behavior, we could never predict them or develop classification systems of disturbances and illnesses, but it is also the case that we cannot completely predict the behavior of anyone. Each individual is uniquely made up of far more factors than we can measure and of qualities we fail to measure. The behavior of a person at any moment is a most complicated equation, made up of countless multiplicands for which we have no measure. Although a psychological assessment can certainly not take in all possible factors, the sentence completion can help to make the picture we generate of the student more accurate and more useful by noticing at least some of the following elements in the data. Individual Qualities The individuality of the client will emerge in the sentence completions in the form of humor, imaginativeness, insight, tolerance, forgiveness, ambition, drive, social intelligence, talent, and many other qualities. Clearly, we cannot measure every student for all these and other positive and productive factors, although a number of checklist-type instruments do set out models of psychological functioning that attempt to measure the components variables in that model. Some of these instruments can be useful, particularly if the instrument is selected to measure the student for what is already somewhat obvious as a personality propensity. In opposition to the effort to try to measure everything, however, is simply the matter of noticing what is plainly there. Like cream,

reyn2-6.qxd

6/6/2003

9:07 AM

Page 177

6. Sentence Completion as a Tool

strong talents and personality factors will rise to the top (though often no more than as potentials). The key is for the alert assessor not to ignore signs, not to consider them “blips” on the screen while hunting for emotional monsters. In the sentence completion, children often display realistic, surprisingly appropriate plans for their own future that take into account their particular abilities. Some children divulge needs and dreams they are too shy to have articulated anywhere else. Often these dreams could be pursued if children could be reinforced in their plans or appropriate support could be obtained. If a child’s strong desire is to learn to play the violin, for example, a violin can probably be found somewhere. Not all class clowns are acting out of pathology; there are some true comedians in the group, children with acute social perception. Why are they not in the drama club? And the boy with disabilities whose lifetime dream is to play football: Can he be a student assistant to the team (probably so, if the examiner approaches the coach and works it out). The individual characteristics of the student are potent elements that should be consciously used in any successful treatment plan. Motivation derives from these elements. Idiosyncratic Reactions For all kinds of idiosyncratic reasons, the meaning of one person’s experience may be different from that of the majority of people in the culture or subculture. For example, a sentence completion such as “People don’t know that I secretly enjoy missing parties” is not the standard teenage response. But it explains a lot in a “hurried child,” one whom the parents have overbooked to the point of having no free time at all. Even parties do not rank as high with this student as just having a free hour to herself. Religious or philosophical teachings can also cause a student’s responses to differ from the typical cultural responses, and this can mislead the examiner. Consider the following examples: “When I do something good in school, . . . I am embarrassed.” “It is wrong . . . to compete with others.” If this is an able or even talented child who simply refuses to perform as well as he or she can, the difficulties may be originating from

177

teachings in the family that posit that it is wrong to compete or show pride in oneself. Oversensitivity is another personal idiosyncratic way of responding to sentence stems. This is observed in sensitivity to stems that do not appear to affect other children in the same way. This style is often generalized to many stems. There is much evidence today that this type of oversensitivity or shyness has a physiological basis, and such children may need help but do not need to be psychologically overinterpreted. Subcultural behaviors can also be misinterpreted. One teacher became upset and referred a child for psychological evaluation because he wrote a story about his grandmother swinging a chicken round and round and drew a picture of the headless chicken jumping in the grass, blood going everywhere. Perhaps the action of the grandmother was disturbing to the child and he was working out his concern in this manner, but the action of the grandmother was nothing but common behavior in the farming subculture in which the child lived—wringing a chicken’s neck as the first step in preparing it for dinner. It certainly said nothing about the level of hostility in the child (or the grandmother). Many sources contribute to making some of our personal characteristics different from the norm of the culture in which we live. Sentence stems that appear potentially pathological can rest on nothing more than misunderstanding. Furthermore, the source of some of our idiosyncracies will never be known. Then again, who really cares how a child came to love that awful butterscotch pudding served in the cafeteria? What is important is that the child will work 2 hours on math, which he hates, in order to have a bowl of the stuff on Friday. Like individual qualities, idiosyncratic responses can guide us toward or away from certain variables that can strongly affect our treatment plans. Personal Decisions and Strategies Many persons have arrived at decisions and strategies as to how they will handle certain instances or classes of stimulation. Many of these decisions and strategies are hardly unique (e.g., that is how we can understand and predict some common behaviors). For example, most people have decided to stop

reyn2-6.qxd

6/6/2003

178

9:07 AM

Page 178

II. PROJECTIVE METHODS

at a red light. But often the decisions and strategies of a person are unique to the person, originate in childhood, and comprise the stuff of which therapy is made. Many personal decisions and strategies can be apprehended from the sentence completion responses, particularly in comparing one response to another or to a cluster of other responses. For example, the stem “When my father gives me lots of work . . .” should be examined in relation to the stem, “When my teacher corrects me . . .”; a common pattern of compliance or resistance may emerge to demands of authority. (That is why the Haak sentence completion has categories of responses in which there are several stems.) But these responses that divulge decisions and strategies arrived at by the person do not always appear with the stem designed to elicit them. For example, one young child who was quite deprived of affection produced this response: “If I had all the books in the world, I would fill up my room to the ceiling and then I would have to sleep with my mother.” Almost any situation this child viewed was related back to his abject emotional need and was converted into strategies for meeting this need. Decisions of the client can be seen anywhere in the sentence completion, but one especially telling example is the stem. “It is no use to . . .” If the response is “act silly” or some such inane and moralistic reply, it is not of the same level of meaning as the response “think things will get better.” This latter response implies that a decision has been reached (and one that cannot wait a long time for someone to change, in this case). When decisions and strategies have been reached in a person’s mental makeup, it implies that the person has encountered a serious problem, usually over and over again, and has made a conscious decision about how to handle it. Some of the decisions are productive and some are of no real consequence, but others may be maladaptive or even dangerous. The boy who wished to fill up his room with books in order to sleep with his mother is going to be unable to handle the crushes that come with adolescence; he is going to place so much importance on the affection he finds in a girlfriend that he will in no way be able to let such a teenage crush just break up naturally. Some

of the decisions and strategies that the child divulges in the sentence completion must be made part of his treatment plan: those that work for him or her need to be reinforced as compensatory strengths; those that are pathological or productive of present and future discord need proper therapy to be eliminated. Almost nothing could be more important in a treatment plan than the “long look forward.” Language Translations Most people have an individualistic and unique way of handling some of the language. When these personal expressions become too bizarre, we become concerned about thought disturbance, for instance. But because language is the codification of experiences and all persons’ experiences differ, language will always be somewhat personal and will provide a window into the person’s unique experience. The ability to express this experience is not, however, uniform. A great deal of verbal ability is required for a person to be able to fully articulate his or her experience and feelings. Most persons can do this to some degree, but with many the examiner must become a translator in order to understand the significance of the child’s codifying of his or her experiences. The simplest translation is the individual word itself. “When I am afraid, I hiccup.” “When we have a math test, I hiccup.” The personal meaning of “hiccup,” and probably the actual behavior, is “am anxious.” The examiner understands that math tests provoke a high level of anxiety in this student, although he may not have mentioned math anxiety elsewhere. Common reactions to diverse experiences is a translator. A teenage boy responds, “When I am mad, I drive my car too fast.” He also divulges in one sentence that “at Christmas time we never get much. My dad always drives us around town to see the lights after we have had our tree, but he drives so fast he scares me.” The translation in this case is that the way the father handles depression has been picked up by the son. Emotions themselves can be translated— often a sign in itself of emotional disturbance. In a normally functioning adult, emotions are differentiated and stand alone; this is not altogether achieved by a child (or

reyn2-6.qxd

6/6/2003

9:07 AM

Page 179

6. Sentence Completion as a Tool

many adults). But even with children, major emotions should not be translated to any marked degree. The most common translation for children is the crossover of sadness and anger: “When I am sad, I am mad.” Unhealthy translations also occur in how the emotions are handled. Far better is the somewhat defensive “When I am sad, I am sad” then “When I am sad, I am stupid.” In translations there is a common denominator, often of which the student is unaware. “I am childish and do not care enough about my grades.” “My mother thinks I am childish.” One can suspect a conflict with the mother over this child’s school performance even though such conflict is not otherwise stated. Common denominators often tie a number of responses together and provide for the examiner a major dimension of the assessment analysis. Consider these examples: “I cannot work when I am tired,” “When my father yells at me, I feel tired,” “When a dog comes toward me, I feel tired,” “I could be happy if I wasn’t so tired.” All of these child’s responses seem reduced to “being tired,” which raises questions about depression and anxiety, a hypothesis that can be checked out other places in the data as well. Many children have formed certain identifications that they do not openly express. “My father was in the army.” “When I grow up, I want to join the army”—this from a child apparently having all kinds of trouble with his distraught father. The last thing on earth the father would suspect is that the child has an identification with him. Identifications, whether conscious or unrealized, are some of the more powerful psychological variables that exist. They can be unearthed by careful attention to translations in the sentence completion—the detective work in its analysis. We don’t always “know” what a behavior indicates from our catalogue of psychological stereotypes. Sometimes we have to figure it out.

THE PSYCHOLOGICAL REPORT AND TREATMENT PLAN The culmination of all assessment activities should be a usable psychological report and treatment plan. Simply put, when people have referred a troubled student for assess-

179

ment they expect to find out “what is wrong” with the student and what to do about it. They deserve a report that draws a comprehensive picture of major dimensions of the student’s functioning, both productive and nonproductive, and a succinct, reasonable, clear, and even creative treatment plan tied to those major dimensions of functioning. Additionally, if an assessment is done for possible school services, the student must be considered in the context of education and school. The Haak sentence completion is an instrument specifically created to evaluate a person within a certain context (its most common use, as we learned earlier). It reveals how major dimensions of the child’s functioning are reflected in school behavior. Because some school systems like to follow a standardized report format for the report, and because the federal rules governing special education set out certain topics in the report that must be addressed, local or federal requirements for the report may be less than appropriate in addressing the psychological organization of the child from an examiner’s point of view. Nevertheless, the analysis of the psychological examination can still be worked out and reported succinctly as a major component of the report, and an organized treatment plan can be outlined, as below. Too many psychological assessments are mere laundry lists, citing item after item and interpretation after interpretation with little effort devoted to the creation of organization or internal consistency. The sentence completion if misused can contribute to such an undesirable outcome. The examiner may use the sentence completion as a kind of “bank,” picking and choosing certain sentences to quote in support of hypotheses generated elsewhere. This is an inappropriate use of the sentence completion. It is like quoting the Bible, selecting words and sentences out of context (one such report was received by an attorney of the writer’s acquaintance who threw it on the desk exclaiming in frustration, “And what am I supposed to do with this?”). A useful assessment should usually produce from two to four or five major dimensions of concern in the child’s functioning, as previously stated, both negative and positive, and explain how these concerns play

reyn2-6.qxd

6/6/2003

180

9:07 AM

Page 180

II. PROJECTIVE METHODS

out in the daily life of the child. Persons working with the child should understand clearly after reading the psychological report what the dimensions of major concern are and how individual actions of the student, particularly the ones about which they are most concerned, are related to these major dimensions of behavior. This involves a lot of old-fashioned work with a pencil, clustering behaviors and drawing lines. It calls on everything the examiner has been taught and, in truth, on everything the examiner has ever seen or heard filtered through the concepts of psychology. The good (i.e., useful) psychological report is like bread: its smooth consistency is the result of much kneading and working (with perhaps a little bit of time to let some of the issues “rise” in the mind of the examiner, if that is not taking the analogy too far). The treatment plan should address the needs of the student that derive from his or her major dimensions of behavior and how these needs are to be met. This is often a place for creativity. The federal rules specifically prohibit dishing out only those treatments an institution has in its cupboard. Obviously, if specialized services are needed, as, for example, physical therapy, that need will be indicated with minimal language in the psychologist’s report. The physical therapist will write the detailed, professional report for that area. But the needs that have no standard, available service are those that call on the creativity of the assessor (and the ones that are often disregarded for want of a ready fix). Let us suppose that a child is diagnosed who has much verbal skill, a strong motivation to help others, and a desire to be a leader, but he is pushy, loud, and awkward and completely alienates his peers. What should we do? We must try to reduce the need to its barebones dimensions: The child needs to be taught how to be a social leader. Now remedies suggest themselves: bibliotherapy; a mentor who is a community leader; short-term counseling. What about getting the highly popular, civicminded junior class president to spend some sessions with this student through the peer-tutoring program? If anybody knows how to be a peer leader, clearly he or she has figured it out. Solutions come more creatively when the bare essentials of what is

needed are stated in plain English (not within some conceptual system). It is surprising how often creative solutions can be found. The whole psychological assessment process is a venture in creativity. The artist gathers and evaluates his or her materials, forms a plan, and decides what he or she needs to do to bring the plan to fruition. So does the psychological assessor: He or she measures the relevant properties of the child, decides on a treatment plan (with input from important others), and enumerates what actions or resources are needed to bring the plan to fruition. The outcome of this plan is intended to be a happier, freer, more productive child. In such a laudable enterprise, the sentence completion technique can play a useful part.

REFERENCES Achenbach, T. M. (1991). Child Behavior Checklist. Burlington: University of Vermont, Department of Psychiatry. Agesen, N., Brun, B., & Skovgaard, B. (1964). Rotter’s Sentence Completion Test. Nordisk Psykologi (Danish), 36, 188–200. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. American Psychological Association. (1967–2000). PsycINFO [A computerized database of psychological abstracts]. (Available at any major library) Baggerly, T. N. (1999). Adjustment of kindergarten children through play sessions facilitated by fifth grade students trained in child-centered play therapy procedures and skills. Dissertation Abstractions International, 60(6-A), 1918. Baker, D. (1988). Establishing inter-rater reliability and criterion-related validity for the Haak Sentence Completion. Unpublished doctoral dissertation, Texas A&M University. Costin, F., & Eiserer, P. E. (1949). Students’ attitudes toward school life as revealed by a sentence completion test. American Psychologist, 4, 289. Bower, E. (1969). Early identification of emotionally handicapped children in school. Springfield, IL: Charles C Thomas. Dean, R. S. (1983). Individual Ability Profile. Odessa, FL: Psychological Assessment Resources. Dean, R. S. (1984). Commentary on “Personality assessment in the schools: The special issue.” School Psychology Review, 13, 95–98. Derichs, G. (1977). Sentence completions as intake instruments. Praxis der Kinderpsychologie und Kinderpsychiatrie (German), 26, 142–149. Erickson, E. H. (1959). Identity and the life cycle. Psychological Issues, 1, 1–171.

reyn2-6.qxd

6/6/2003

9:07 AM

Page 181

6. Sentence Completion as a Tool Haak, R. A. (1990). Using the sentence completion to assess emotional disturbance. In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Personality, behavior, and context (pp. 147–167). New York: Guilford Press. Haak, R. A. (1996). Haak Sentence Completion (Elementary) (rev.). Unpublished instrument. Haak, R. A. (2000). Haak Sentence Completion (Secondary) (rev.). Unpublished instrument. Hart, D. H. (1986). The sentence completion technique. In H. Knoff (Ed.), The assessment of child and adolescent personality (pp. 245–272). New York: Guilford Press. Hart, D. H., Kehle, T. J., & Davies, M. V. (1963). Effectiveness of sentence completion techniques: A review of the Hart Sentence Completion for Children. School Psychology Review, 12(4), 423–434. Havighurst, R. J. (1953). Human development and education. New York: Longmans, Green. Ilg, F. L., & Ames, L. B. (1960). Child behavior. New York: Dell. Irving, F. S. (1967). Sentence completion responses and scholastic success or failure. Journal of Counseling Psychology, 14, 269–271. Kaufman, A. L., & Kaufman, N. L. (1983). Kaufman Assessment Battery for Children (K-ABC). Odessa, FL: Psychological Assessment Resources. Knoff, H. M. (1963). School based personality assessment. School Psychology Review, 12, 391–393. Loevinger, J. (Ed.). (1998). Technical foundations for measuring ego development: The Washington Uni-

181

versity Sentence Completion Test. Mahwah, NJ: Erlbaum. Payne, A. F. (1928). Sentence completions. New York: New York Guidance Clinic. Peck, R. F., & Haak, R. A. (1973). Coping skills of early adolescents. Unpublished study. Piaget, J., & Inhelder, B. (1958). The growth of logical thinking from childhood to adolescence. New York: Basic Books. Reinis, S., & Goldman, J. M. (1980). The development of the brain: Biological and functional perspectives. Springfield, IL: Charles C Thomas. Sanford, R. N., Adkins, M. M., Miller, R. B., & Cobb, E. A. (1943). Physique, personality and scholarship. Monographs of the Society for Research in Child Development, 8(1, Series No. 34). Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). The Stanford–Binet Intelligence Scale IV. Itasca, IL: Riverside. Wechsler, O. (1991). Wechsler Intelligence Scale for Children, Third Edition. San Antonio, TX: Psychological Corporation. Wells, B. (2000). The use of the Haak Sentence Completion Measure and the Child Behavior Checklist/ Teacher Report Form by school psychologists in the identification of students with serious emotional disturbance. Unpublished doctoral dissertation, University of Texas. Wilson, L. (1949). The use of the sentence completion test in differentiating between well-adjusted and maladjusted secondary school pupils. Journal of Consulting Psychology, 13, 400–402.

reyn2-7.qxd

6/6/2003

9:08 AM

Page 182

7 Using the Rorschach with Children and Adolescents: The Exner Comprehensive System

JERRY C. ALLEN JILLAYNE HOLLIFIELD

The Rorschach test has one of the most controversial histories of any instrument in the era of psychological testing. Although the controversy is still far from over, the Rorschach is continuing to gain popularity among psychologists as the instrument of choice when their purpose is to understand and describe the psychological features of an individual. This renewed popularity is evidenced in the increasing numbers of training institutions offering graduate courses in the Rorschach; in the number of internship sites requiring students to have training and skill in using the Rorschach; and in the increasing frequency with which the Rorschach is being used in clinics, hospitals, and schools. This renewed popularity is due in large measure to the continuing efforts of John E. Exner, Jr. and his associates at the Rorschach Research Foundation. The Exner Comprehensive System is the most widely used system of administering, scoring, and interpreting the Rorschach, according to data provided by Piotrowski (1996). This chapter presents a brief introduction to the Rorschach, using the procedures and norms established in Exner’s Comprehensive System (Exner, 1974, 1978, 1986, 1991, 1993; Exner & Weiner, 1982, 1994) and a

discussion of the use of the test with special populations of children and adolescents. A thorough knowledge of the aforementioned references is essential if one is to use the Rorschach properly and wisely, as are intensive study and training in administration, scoring, and interpretation.

THE EXNER COMPREHENSIVE SYSTEM Three factors that contributed to the Rorschach controversy, according to Exner and Martin (1983), were “(1) confusion and disagreement concerning the theoretical underpinnings of the Rorschach, (2) the lack of a single, consistent administration, scoring and interpretation procedure, and (3) the oversimplified classification of the Rorschach as a projective technique” (p. 407). Exner and his associates have expended much effort in addressing the second and third critical factors. Although projection may be present in an individual’s verbalizations about what the inkblots might be, Exner defines the Rorschach as a problem-solving (perceptual–cognitive) task, similar to the way Rorschach himself considered it to be a 182

reyn2-7.qxd

6/6/2003

9:08 AM

Page 183

7. Using the Rorschach with Children and Adolescents

“form interpretation test” (Exner & Martin, 1983, p. 407). If projection is present, the examiner may wish to use that material as supplemental to the formal, psychometrically based data obtained from the Structural Summary of the Comprehensive System. Thus, the Rorschach may now be viewed as a test, administered under standard procedures and scored according to established criteria. These scores (or codes) may then be compared with the scores obtained by an appropriate reference group. The Comprehensive System originated from Exner’s attempt to develop a single, consistent procedure of administration, scoring, and interpretation of the Rorschach test, based on empirically defensible data. The history of this endeavor can be found in Exner and Martin (1983). The outcome resulted in the development of the system (Exner, 1974, 1978, 1986, 1991, 1993; Exner & Weiner, 1982, 1994). The development is continuing today through efforts of the Rorschach Research Foundation and other professionals’ research contributions.

ADMINISTRATION Two skills are of prime importance to the Rorschach examiner who chooses to work with children and adolescents. The first is a thorough understanding of childhood developmental processes; the second is a high level of knowledge and expertise with the instrument itself and the Comprehensive System. These skills are essential to obtain complete and accurate data during the administration phase, which in turn permit appropriate scoring and interpretation. Because of the differences in verbal skill levels inherent in the various developmental levels of childhood, and because Rorschach data collection is almost entirely dependent on verbal exchange, the administration process can be quite challenging. Nevertheless, the individual child’s verbal skills are always uniquely revealing.

Preparing the Child Because it is so important to be particularly careful in collecting Rorschach data from children, it is advisable for the examiner to spend adequate time with the child prior to

183

administration to put the child at ease. Children generally have some idea as to why they are being tested, but often they do not know how the information gained in the assessment will be used. It is not uncommon for them to develop negative forms of resistance, and consequently, to perform less than optimally. Thus, it is wise for the examiner to take as much time as possible with the child, explaining the nature and purpose of the test and indicating how the results will be used, as well as answering any questions the child or the parents may have. This should be done in as honest and straightforward a manner as the situation permits, and without unnecessary elaboration. Children value honesty and will generally respond with minimal resistance to an examiner who has taken adequate preparation time before the procedure begins.

The Response Process It is vital that the examiner understand the nature of the Rorschach response process— what is involved in the formation of a response and its ultimate delivery. Exner and his colleagues have studied in detail the processes an individual goes through in making a response from the moment the person is asked “What might this be?” and is handed the inkblot until the first response is delivered. Exner (1986) summarizes the operations as follows: Phase I:

1. Visual input and encoding of the stimulus and its parts. 2. Classification of the stimulus and/or its parts, and a rank ordering of the many potential responses that are created. Phase II: 3. Discarding potential answers that have low rankings. 4. Discarding other potential answers through censorship. Phase III: 5. Selection of some of the remaining responses because of traits or styles. (p. 51) The crucial elements in assuring the cooperation of the child and a valid Rorschach protocol are, as in the case of all assessment procedures, the time and effort the examiner expends in putting the child at ease prior to the examination proper.

reyn2-7.qxd

6/6/2003

184

9:08 AM

Page 184

II. PROJECTIVE METHODS

Exner and Weiner (1982) stress that the response process is a complex interaction among at least three interrelated variables: (1) the set of the subject toward the test and the testing situation, (2) the evaluation of the subject regarding which of several perceived responses is most appropriate or “correct” in light of the set, and (3) the impact of the composite of response tendencies or styles plus the ongoing psychological operations of the subject. (p. 19)

Thus, the necessity of preparing the child adequately to yield the most appropriate test set is evident and will usually produce a valid record even when some amount of resistance to the task remains. The next variable in the response process of the child involves his or her evaluation of which of all the perceived responses is most appropriate to the established test set. Children, as well as adults, see more potential responses than they are willing to verbalize. In fact, when a sample of children was encouraged to give as many responses to a card as they could in 60 seconds, they averaged slightly more than 94 total responses for the test (Exner, Armbruster, & Mittman, 1978). Nevertheless, most children, regardless of socioeconomic status or environmental setting, restrict the number of responses they deliver. This restriction is related to the evaluation process and involves both rankordering and censorship operations (Exner, 1986). That is, on the basis of the established test set, the child selects from among numerous potential responses those that seem most appropriate or correct and censors those that do not. Not surprisingly, Leura and Exner (1978) found that examinees do not restrict as much and, in fact, give significantly greater numbers of responses to examiners who are known to them. The largest increases are seen in Human Movement responses (M), Color responses (FC, CF, and/or C), and Blends, with significant increases in Whole (W) and Uncommon Detail (Dd) locations. Nevertheless, the proportions of both M and C responses to total number of responses remains unaffected because of the increased number of responses generated overall. Leura and Exner found the greatest difference in proportion occurring in the gray–black Shading responses. Persons

whose examiners were known to them gave about 11% Shading responses, whereas the control group gave about 15%. Consequently, examiners who test children they know well should be alert to the possibility that these children’s profiles may reflect some or all of the features just described. Other studies related to response productivity have yielded mixed results with regard to the potential influence of sex of the examiner (i.e., Greenberg & Gordon, 1983; Tuma & McCraw, 1975). Thus, examiners may wish to consider the potential for gender effects in Rorschach response productivity. Another factor that may influence both the quantity and quality of children’s responses is their need for reinforcement. Children are far more concerned than adults about giving the correct responses. They are motivated by needs to please the examiner, to perform well, to do the socially acceptable thing, and to avoid potentially unpleasant outcomes. Reassuring a child that there are no right or wrong answers is not always sufficient to provide the reinforcement the child needs. Even adults have been shown to be strongly influenced by the perceived social acceptability of their responses. Exner and Leura (1976) demonstrated that adults would report that certain sexual and aggressive responses were “easy to see” when they had received prior instructions that these responses were frequently given by successful businessmen. A second group of adults indicated that these same sexual and aggressive responses were “difficult to see’ when prior instructions implied that they were produced by seriously disturbed persons. Children, too, want to do the right thing and to gain approval for their performances or, at the very least, to avoid any undesirable consequences. The inherent lack of structure in the Rorschach is problematic for the child. Depending on the stage of development, or in some cases on developmental lag, children are to one degree or another concrete in their thinking and prefer situations in which there are rules or guidelines. Ambiguity is troublesome. As mentioned earlier, to attempt encouragement by telling the child that there are no right or wrong answers does not remove the necessity of dealing with an ambiguous situation. Exner and Weiner (1982, 1994) point out that, in fact,

reyn2-7.qxd

6/6/2003

9:08 AM

Page 185

7. Using the Rorschach with Children and Adolescents

the only “right” answer is that it is an inkblot. By coaxing the child to report what it might be, or by encouraging the reluctant child with “Most children see more than one thing,” the examiner engages the child’s preferred problem-solving operations and coping mechanisms. Therefore, although the test itself causes the surfacing of test sets, it also requires that the child use his or her unique psychological response style in the perceptual, cognitive, and affective spheres. These will be apparent in the responses generated by the child, regardless of any residual resistance. Children’s psychological response styles tend to be relatively consistent, although far less so than those of adults. Their response styles tend to become increasingly stable over the developmental years and, like those of adults, are susceptible to situational variables. Stability of a psychological response style is determined by its utility in reducing stress for the child. Consequently, change in the cognitive, perceptual, and affective operations of the child is to be expected, and the younger the child at the time of initial testing, the more the psychological response style will change with time (Ames, Metraux, Rodell, & Walker, 1974). The child’s psychological response style is highly unique and individualistic and determines which of the many responses that could be articulated will be selected in the child’s problem-solving operations. For example, in giving the popular response to Card V, the child could report “a butterfly” (Form or F); “a black butterfly” (Form/ Achromatic Color), a response in which form is the primary determinant but achromatic color is also used; “a butterfly flapping its wings” (Animal Movement or FM); or “a soft, velvety-looking butterfly” (Form/ Texture or FT), a response in which form is the primary determinant but texture is also used. It can also be seen in whether or not the child prefers to respond to the blot as a whole (W) or to common (D) or unusual detail (Dd) areas. It can also be seen in whether or not the child attempts to organize details of the blot in the response. Furthermore, the child’s psychological response style is apparent in the use of pairs, reflections, or special scores or contents, or in the failure to use these. Because children’s psychological response styles are at best only

185

relatively consistent, it is of utmost importance that the examiner know which of the Rorschach test variables are relatively consistent and which are highly vulnerable to change over time. A study by Exner, Rosenthal, and Thomas (1980) underscores the lack of consistency in many of the child’s psychological operations. In this study, two groups of children, initially ages 6 and 9, were examined at specified intervals over time. The only variables with high test–retest correlations over a 2-year period were X+%(perceptual accuracy for the total record), frequency of Active Movement, and frequency of Popular (P) responses. All other variables showed great disparity over the same interval. Moreover, there were substantial differences between the two groups of children. It is highly unlikely that a child’s current protocol will predict accurately his or her future psychological response style. Children change over time; as noted earlier, the younger the child at the time of initial testing, the more change is to be expected over even relatively short intervals. Older children and adolescents exhibit less change, and their records are apt to be more stable over intervals. In particular, X+% is consistent in the early years, perhaps even prior to school entry (Exner & Weiner, 1982), whereas Human Movement (M) production is not consistent until midadolescence. Thus, the preferred problem-solving or coping style tends to fluctuate during the child’s first 10–12 years, and a dominant style is not formed until somewhere between 14 and 18 years of age. Thus, it is apparent that knowledge of both the instrument and the developmental process are essential to competent Rorschach administration with children and adolescents. The examiner must spend adequate preparation time with the child to ease tensions and minimize resistance, so that a clear picture of the psychological operations of each child may be obtained. The examiner must be aware that children evaluate and censor potential responses on the basis of how they expect their test data to be used, and also on the basis of what sorts of responses they believe are “correct” or will please the examiner. Examiners must also be alert to the potential effects of testing children who are well known to them

reyn2-7.qxd

6/6/2003

9:08 AM

Page 186

186

II. PROJECTIVE METHODS

and to possible gender effects in Rorschach response productivity. Finally, the examiner must be aware of the high degree of inconsistency in children’s Rorschach records over time and the considerable potential for change in this age group.

Administering the Rorschach The actual formal administration of the Rorschach test is a relatively straightforward task; nevertheless, certain basic guidelines must be followed. The examiner must adhere to the standard administration practices in order to insure accurate scoring and interpretation. Practice is required so that the examiner can avoid inadvertently establishing unwanted test sets or reinforcing any particular type of response. The preferred seating is side by side. Seating can be altered to accommodate very young children or those with special needs (e.g., sitting on the floor), as long as standard administration is maintained. The formal administration of the Rorschach consists of two phases: the “Administration Proper” and the “Inquiry.” In the Administration Proper phase (often termed the “Free Association” phase), after appropriate preparation of the child for the test, the examiner hands each of the inkblots in sequence to the child, who is asked, “What might this be?” Responses are recorded verbatim; therefore, the examiner needs to develop some type of shorthand method for recording responses. After responses to all 10 cards have been obtained, the Inquiry phase begins. The purpose of the Inquiry is to allow the examiner to obtain all information needed to score the response appropriately (see Exner, 1986, pp. 70–78, for details on conducting the Inquiry). As normally the Rorschach is just one of a battery of instruments given to the child, its description should be brief and similar to that given for other procedures. For example, the examiner may say, “And then we will be doing the inkblot test. Maybe you’ve heard of it.” If the child’s parents are present, the explanation should be delivered in a manner that insures understanding for both the parents and the child. Subsequent questions should be answered directly and honestly. During the administration proper, there is

no need to change the standard instructions, except in the case of very young children or children with language difficulties, for whom even more simplified instructions may be in order. The examiner should refer to the cards as inkblots; that is what they are. Typical instructions are as follows: “I am going to show you some inkblots and I would like you to tell me what this might be.” Children tend to ask a lot of questions, and it is best to answer them in as direct and straightforward manner as possible. “Where did you get these?” “I bought them.” “Do you think it looks like that?” “Right now I’m interested in what you think it looks like.” “Did I get that one right?” “If that’s the way it looks to you.” “Do other kids say things like that?” “People say all sorts of things.” “What kind of kids do you show these pictures to?” “All sorts of kids.” Some questions require lengthy explanations or are complex and should be deferred until the end of the test: “That’s a pretty complicated question. Let’s wait until we finish and then I’ll explain. I’ll write it down so I won’t forget.” Instructions for the Inquiry should be equally straightforward and direct. “We are going to go through the cards again now, and I will read to you what you said. I want you to show me where on the blot you saw what you saw, and then tell me why it looks like that to you. I want to see it just the way you did.” The Inquiry is the most difficult part of the administration and much practice will be necessary before one becomes proficient at it. Generally, if the examiner has given clear instructions at the outset of the Inquiry, scoring can proceed without difficulty; remember, the purpose of the Inquiry is to allow the examiner to obtain all the necessary information so that the response can be scored accurately. Some children require more encouragement and questioning even when adequate instructions have been given. “Yes, I know it looks like that to you but help me see it the way you do. Remember, I need to see it too just like you do.” “What in the blot makes it look like that to you?” “Draw a line around the ______ with your finger.” The examiner should not become directive, however. If the attempt at clarification is not productive, examiner and child should move to the next

reyn2-7.qxd

6/6/2003

9:08 AM

Page 187

7. Using the Rorschach with Children and Adolescents

response. It is important to avoid establishing test sets.

Some Typical Administration Problems Occasionally, a child is extremely resistant and defensive. In this case, the examiner must decide whether to proceed with the test or to postpone it until a later date. Reviewing two areas of the administration process may be helpful in making this decision. First, have sufficient time and effort been spent in establishing rapport with the child? If not, it is possible that the problem can be remedied within the session. Second, and only in extreme cases, the examiner should consider the possibility of altering the administrative process to accommodate such difficulties as hyperactivity, unusual anxiety, and the like. Ames and colleagues (1974) and Halpern (1953), for example, advocate the use of the Inquiry immediately after each card, particularly with preschool and easily distracted children. However, Exner and Weiner (1982, 1994) caution that this procedure should be used only as a last resort, as Leura and Exner (1977) have found a resulting elevation in Common Detail (D) and Form (F) responses, especially on cards VI and X, when immediate Inquiry is used. Another problem frequently encountered by the examiner is the brief record. Young children tend to give relatively short records, and it is not uncommon for 5- and 6-year-olds to give 15 or fewer responses. Moreover, children generally give more pure F responses than do adolescents or adults, although they usually give enough responses with a sufficient variety of determinants to be interpretively useful. When this is not the case, the examiner should consider several possibilities, including intellectual or neurological difficulties. When defensiveness or resistance is present, the child is often able to control his or her affect sufficiently to yield a record that is not interpretively useful. The examiner then needs to assess whether sufficient time was spent in establishing rapport with the child or whether the use of other procedures might be more productive. When none of these possibilities is plausible, however, the presence of a brief and barren record signals the question of a psychological style in which the child copes

187

by oversimplifying the stimulus field (Exner & Weiner, 1982, 1993). That is, the child may be ignoring important environmental cues and oversimplifying complexity to keep his or her world manageable, but at the price of almost inevitable conflict. Occasionally, the examiner will find that a child who has been somewhat resistant or defensive in the Free Association phase will respond more freely in the Inquiry phase. Klopfer (1956) and Klopfer, Ainsworth, Klopfer, and Holt (1954) have indicated the importance of noting behavioral differences in the child’s performance on the two portions of the test as providing valuable information to the interpretation. Exner and Weiner (1982) suggest that by the time the Free Association has been completed, and with the additional structure of the Inquiry, the child may have become more relaxed. Thus, it is advisable to record any additional responses generated in the Inquiry, while not including these as scored responses, to supplement the interpretation. In some instances, when the scorable record is extremely brief and barren (less than 14 responses), it may be advisable to retest the child. Studies by Hulsart (1979) and Exner (1980) have shown that interpretively useful records may be obtained after brief intervals. A viable and recommended procedure is simply to say at the completion of the Free Association: “Good. Now that you have the idea of how to do these, let’s go through them again. Most children see more than one thing, and you may too, now that you know how to do these.” This procedure is often all that is necessary to yield an interpretively useful record.

NORMATIVE DATA All the major Rorschach methods currently in use in the United States include some mention of administrative procedures and interpretation guidelines for use with children. However, only two scoring systems provide normative data obtained from stratified samples of children at various ages and exclusively designed to facilitate interpretation of children’s records (Ames et al., 1974; Ames, Metraux, & Walker, 1971; Exner, 1986; Exner & Weiner, 1982). Ames and her colleagues have based their work

reyn2-7.qxd

6/6/2003

188

9:08 AM

Page 188

II. PROJECTIVE METHODS

on the scoring system of Dr. Marguerite Hertz. The Exner Comprehensive System provides norms for children ages 5 through 16, as well as adult norms, thus giving added flexibility to the examiner who works with both children and adults. The child and adolescent norms are based on a sample of 1,580 nonpatient children who were recruited through schools and other organizations. Sample selection was partially stratified for socioeconomic and geographic status (Exner, 1986, p. 255). Normative tables provide the following information for 70 of the scoring variables at each year level: mean, standard deviation, mode, minimum and maximum range, frequency, skewness and kurtosis. These data, both for children and adolescents, are helpful to the examiner in serving as baseline data. Although the availability of normative data specifically designed for use with children is a substantial aid to interpretation, norms, like all other standards for normal performance, must be used intelligently. Digressions from the norms may be signs of uniqueness or ideographic ways of looking at the world and may be in no way abnormal. Conversely, a child can produce a normal record and still lack or be deficient in certain adaptive skills. The normative data itself may underrepresent certain groups or be insensitive to developmental differences. Finally, because many of the Rorschach scores occur infrequently, they are not normally distributed, and rigid application of the normative data is inappropriate. In these instances, the examiner should rely more heavily on either the mode or the frequency with which the responses is likely to occur for the age of the child in question.

SCORING A full explanation of the Comprehensive System’s scoring procedures is beyond the scope of this chapter; the interested or unfamiliar reader might consult the book describing the system (Exner, 1986). A summary of the scoring variables is given in Exner and Martin (1983, pp. 410–411). The Comprehensive System currently includes approximately 90 possible scores (or codes) for the Rorschach responses. There

are seven major categories into which the scores are placed: 1. Location (Where on the blot is the response seen?); 2. Determinants (What features in the blot contributed to the formation of the percept?); 3. Form Quality (How “good a fit” is the percept to the area of the blot to which the response was offered?); 4. Content (What is the content of the response?); 5. Popularity (Is the response one that is commonly given?); 6. Organizational Activity (To what degree does the response organize the blot elements?); and 7. Special Scores (Does the response include unusual verbal material or unusual characteristics?). Each response a child makes to the inkblots is scored according to specified criteria. These scores are then expressed as percentages, frequencies, or ratios on the “Structural Summary” and are compared to the normative data for purposes of interpretation. One of the strengths of the Comprehensive System is its continuing research program investigating the psychometric qualitites of the scores remaining on the Structural Summary. Exner (1986) reported that more than 30 reliability studies of temporal consistency had been completed at the Rorschach Research Foundation by 1983. Time intervals between test and retest have varied from a few days to 3 years. Exner and Martin (1983) report that in the longest test–retest study, using 100 nonpatient subjects, reliability coefficients for 20 critical variables were uniformly high for that amount of elapsed time. For 10 variables, the coefficients were between .80 and .90; for 7, between .73 and .80; and only 3 fell at or below .70. Short-term reliability coefficients are even higher. Many studies have also been conducted investigating interscorer reliability for each of the variables. Exner (1986) reports that a standard of 90% agreement among scorers or a .25 intercorrelation among scorers was used as a standard (p. 89). The reader should be aware of ongoing criticism related to interscorer reliability in the Comprehensive System as well as other

reyn2-7.qxd

6/6/2003

9:08 AM

Page 189

7. Using the Rorschach with Children and Adolescents

reliability and validity issues (see Exner, 1996; Wood, Nezworski, & Stejskal, 1996a, 1996b).

INTERPRETATION OF CHILDREN’S RECORDS General Considerations Skilled interpretation demands a thorough understanding of the developmental process, together with a complete comprehension of what the normative data can be expected to provide. Rorschach data have essentially the same meaning at all ages; however, the particular frequency, percentage, or weight of a response varies in interpretive significance with age. Nevertheless, the examiner must not apply the normative data concretely but, instead, must be sensitive to the unique qualities of the individual scores. Moreover, examiners must be familiar with psychopathology and with personality theory. This background is essential to making determinations regarding the prognosis and the form of treatment or remediation most likely to elicit a positive response from the child. Children’s Rorschach scores are not consistent over time; thus, the examiner must think in terms of probabilities rather than predictions, except in the case of severe pathology. Exner, Thomas, and Mason (1985) have demonstrated that only the X+% is relatively stable from ages 8 through 16. In the same study, the majority of Rorschach variables were inconsistent until approximately ages 14–16, and even then the ability to modulate affect, the extent of self-focus, the effort expended in organizing, and some of the factors involved in cognitive slippage remained unstable. Among younger children, the EB (Erlebnistypus or preferred response style) generally reflects an extratensive style. When this persists beyond age 12, it is highly likely to become a stable feature. On the other hand, the establishment of an introversive style appears to occur somewhat earlier and the child who demonstrates an introversive style at age 8 may well maintain it (Exner, Leura, Wylie, Armbruster, & Thomas, 1980). Similarly, the es (Experienced Stimulation) of young children frequently is greater

189

than the EA (Experience Actual); however, when the EA does become greater than the es, it tends to remain so in subsequent protocols. Depending on the age of the child, EA > es can be either a positive or a negative finding. This shift is expected during the developmental process and generally signals psychological maturity. A premature shift, however, could portend rigidity and inflexibility (Exner, Leura, et al., 1980). The es tends to be relatively stable, but because it consists of some highly unstable elements (Inanimate Movement and Diffuse Shading), it warrants the examiner’s careful attention. When any of the unstable elements of the es are present in a child’s record, particularly if the element appears in greater than the expected number, the stability of the es and the direction of the EA : es are both suspect. The examiner must determine whether the scores are truly representative of the child or whether they reflect temporary situational phenomena. The interpretation of the EA : es ratio, therefore, must take several factors into account. First, the age of the child must be considered in conjunction with the appropriate normative data to establish the expected directionality of the ratio and range of values for the component scores. The examiner should note the magnitude of the difference between the EA and es values, as well as the frequencies and kinds of variables that comprise the es, with particular attention to those that are transitory and situational. The presence of Texture (T) in a child’s record is vital (Leura & Exner, 1976; Pierce, 1978). Elevations in T can signal personal loss, whereas the absence of T is highly uncommon and may indicate either a tendency toward defensiveness and guardedness in interpersonal relationships or the absence of a need for or expectation of interpersonal closeness. The examiner must integrate the child’s history and background information with the data to ensure meaningful and useful interpretation. Another aspect of Rorschach interpretation where findings are unique to children is the FC : CF + C ratio. Young children do not (and are not expected to) modulate their expressions of affect, resulting in a ratio that is weighted on the right. As the developmental process progresses, the capacity for control increases, and a gradual shift

reyn2-7.qxd

6/6/2003

190

9:08 AM

Page 190

II. PROJECTIVE METHODS

from right to left generally occurs. The composition of the scores on the right side of the ratio, however, undergoes a rather dramatic change at about age 6, when the occurrence of pure C and Color-Naming (CN) responses diminishes markedly. Thus, although a FC : CF + C ratio weighted to the right is not uncommon at age 12, the majority of responses on the right side should be CF. When the capacity for affective modulation develops in the child, it tends to persist and is reflected in a shift in the weight of the ratio, from right to left. Once again, the age of the child, the magnitude of the difference between the values for the right and left sides of the ratio, and the integration of the child’s history and background are vital to accurate interpretation. An FC weighting in a child of 12 years of age or less is often premature and may signal overcontrol. Moreover for certain youngsters, an FC weighting even at age 16 may be problematic (Exner, Leura, et al., 1980). Another common occurrence in the records of children and adolescents is the Personalized Response (PER). This tends to be frequent in the records of very young children but disappears gradually through the course of development. The appearance of one such response in an adolescent record is not unusual; however, a high frequency of PER responses suggests defensiveness and a need to protect the selfimage. Therefore, the examiner should note the frequency of these responses relative to the age of the child, as well as the overall quality of the child’s PER responses. Some responses are distinctly more defensive than others and yield substantial indications of the child’s unique vulnerabilities. The W : M ratio will also show some characteristic variability in the records of children. Commonly, the left side of the ratio will be markedly higher—particularly in the very young child, where grandiosity and limitless aspirations prevail. During the course of normal development the ratio is expected to decrease gradually until it approaches adult levels in late adolescence. The examiner should be alert for a W : M ratio that is too constrained in relation to the age of the child. Gordon and Tegtmeyer (1982) substantiate the use of the Egocentricity Index [3r + (2/R)] as a measure of self-focusing as op-

posed to self-esteem in children. However, the authors indicate that possibly these results were an artifact produced by limiting the number of responses to two per card. These same authors (Tegtmeyer & Gordon, 1983) found that the children who gave a high number of space (S) responses also gave significantly more W and total Shading responses, and more blends and a greater number of Content categories, than children who gave no more than one S response. They concluded that relatively high frequencies of S responses appear to be related to cognitive complexity and active mastery in children. In a somewhat different approach to the question of cognitive complexity, Smith (1981) found a positive relationship between children’s cognitive-developmental stage and the number and percentage of W responses. In addition, the level of cognitive functioning and W production was found to vary with the complexity of the stimulus. That is, children who had achieved the developmental stage of formal operations showed a marked increase in the number and complexity of W responses to the broken blots, compared to children in the concrete stage.

The Rorschach and Children from Disadvantaged Backgrounds Some special considerations are in order when interpreting the records of children from low-socioeconomic-status (low-SES) backgrounds. Exner and Weiner (1982) have delineated specific findings for sex, age, and SES. Generally speaking, there appear to be some differences in the records of children from low-SES backgrounds, but these differences are not consistent across all the samples. The authors caution that although the Structural Summary may not yield differences among the various SES groups in terms of location, determinant, or content category, substantial variation may be found in the verbal material or verbal expression of these children that is not reflected in their scores. Although a child’s cognitive and perceptual operations certainly influence the content of his or her verbal material, interpretations of the verbal material should be done with extreme caution (Exner & Weiner, 1982).

reyn2-7.qxd

6/6/2003

9:08 AM

Page 191

7. Using the Rorschach with Children and Adolescents

A study of inner-city children (Krall et al., 1983) found a lower F accuracy level, a lower percentage of W responses, and a higher frequency of D responses among these children when compared to both the Ames and colleagues (1974) and the Exner and Weiner (1982) normative data. The authors also found, however, that the children in their study were similar to those in both data bases with regard to response rate and development of the P response. Although caution is always warranted in content interpretation, a study of the aggressive content in the Rorschach records of inner-city children (Crain & Smoke, 1981) yielded information that may be of use to the prudent examiner. Children in the control group gave aggressive content related to interpersonal interactions and equal power, such as fighting, whereas children in the clinical group gave aggressive responses characterized by victimization and feelings of being overwhelmed, such as devouring monsters. Other studies involving content analysis of children’s Rorschach records, although not limited to children from lower-SES backgrounds, have produced findings that may be used to augment the basic interpretation. Gordon and Tegtmeyer (1983) found that oral-dependent responses in the records of children do not have the same interpretive significance that they do in the records of adults. Rather, these responses in the records of children tend to be associated with various internalizing behaviors, such as withdrawal, somatic complaints, and obsessive and compulsive behaviors.

The Rorschach and Children with Learning Disabilities The Rorschach test has been shown to be sensitive to many of the variables that affect children’s ability to learn. For example, Ames and Walker (1964) demonstrated that children with cognitive flexibility, as indicated on Rorschach protocols collected in their kindergarten years, were better readers in the fifth grade. These children were found to be more open to information in the environment and more able to use these cues in their learning. Conversely, Smock (1958) showed that when anxiety is artificially induced in children, it can promote

191

rigidity and premature closure on cognitive tasks. Subsequently, Smock and Holt (1962) found significant negative correlations between perceptual rigidity and IQ, school achievement, and curiosity. Although children with learning problems do not necessarily demonstrate diminished intelligence or curiosity, they frequently show marked perceptual difficulties and other impediments to school achievement. In addition to documented learning disabilities, learning problems can include limited intellectual functioning and various types of neurological impairment. When a child has demonstrated limited intellectual functioning, the Rorschach may or may not be useful. Other measures may better address the education and training needs of these youngsters. On the other hand, the Rorschach can occasionally be useful, particularly with the higher functioning of these children, in detecting personal assets that can be used to the best advantage in educational interventions. Certain Rorschach features are common to this group of children; they include low number of total responses, predominance of pure F, few or no M responses, few or no chromatic C responses with poor integration, few or no Synthesized responses, low F+% and X+%, narrow range of Contents, low number of P responses, and a higher than usual occurrence of CN responses (Exner & Weiner, 1982). When a child has a neurological impairment and/or attention-deficit disorder, the Rorschach can be a useful addition to the assessment battery because of the complex emotional and behavioral features attendant upon these difficulties (see Bartell & Solanto, 1995). Testing can reveal the child’s preferred coping mechanisms, level and availability of controls, ability to direct cognitions, interpersonal attributes, and personality assets, all of which will be helpful in determining the most appropriate methods of interventions and education (Exner & Weiner, 1982). Certain specific features of the Rorschach records of children with neurological impairment have been noted. One of these is a lower frequency of M responses. Gordon and Oshman (1981) suggest that this is related to the inability to delay in hyperactive children. Champion, Johnson, McCreary, and Dough-

reyn2-7.qxd

6/6/2003

9:08 AM

Page 192

192

II. PROJECTIVE METHODS

tie (1984) propose that the lower-than-average Egocentricity Index frequently found in these children’s records is related to the social stigma accompanying their learning problems. In addition, these authors attribute the lower X+% to problems with perceptual accuracy. Scott (1985) indicates that the data in the “Four Square” can be extremely useful in formulating interventions. Williams and Miles (1985) found that dyslexic children (some from the United States and some from the United Kingdom) in their sample gave Rorschach records that were very similar to one another’s but unlike those of other clinical populations or a matched control group. Specifically, the dyslexic children gave fewer responses, a high percentage of F responses, and a limited range of Determinants. Also, they rarely turned the cards. Although the generalizability of these findings is limited by the small number of youngsters in the sample, the authors noted that there were essentially no differences between the responses of the children from the United States and those from the United Kingdom, regardless of age or type of school attended. A study of two groups of children with learning disabilities (Champion et al., 1984) revealed that the response patterns of children with learning disabilities were distinctly different from Exner and Weiner’s (1982) normative data for nonpatients, behavior problems, or withdrawn children. Compared to nonpatients, children with learning disabilities had lower F+% and X+%, lower Egocentricity Indices, higher Lambdas, more Dd and S, and more Shading responses. In the group of 8-year-old children with learning disabilities, the mean number of H percepts was at least one standard deviation below the mean of the normative sample. This group also gave more C⬘, V, and Y, and fewer T than nonnpatients of the same age. Children with dyslexia in the 11-yearold group gave more of all types of Shading responses, and their mean Lambda was at least one standard deviation above the mean for the 11-year-old norm group.

The Rorschach and Children with Depression When depression is suspected in a child, critical information can be obtained from a

thorough and dependable history and from the behavioral observations of the parents and teachers. Adults, however, are not always alert to the signs of childhood depression or are preoccupied with personal concerns and fail to attend sufficiently. Parents engrossed in personal concerns may overlook the child’s cues. The teacher may misinterpret the child’s “perfect” behavior. Consequently, from time to time, the child’s Rorschach data will show signs of depression even when there is no report of substantiating evidence. In adults, the two most common features found in the records of depressives are a low Egocentricity Index and elevation in the right side of the eb (Experience Base), or a sum of the shading responses that is greater than FM + m. Occasionally, the Egocentricity Index will be elevated rather than depressed. This is the exception rather than the rule, but in either case, there is a problem in self-focusing and evaluation of the self. When the depression is reactive rather than chronic, however, the Egocentricity Index frequently falls in the normal range (Exner, 1978, 1986). Adult depressives also frequently give more V (Vista) and FD (Form-Dimension) responses than are normally expected. The Rorschach records of children who are depressed show features similar to those of adults, although it is unusual to find both a depressed Egocentricity Index and a higher right-sided eb in the record of a child. When this occurs, it is also likely that there will be an elevation in Morbid Content (MOR) responses (Exner & McCoy, 1981). Children who are depressed, whether chronic or situational, tend to give more MOR responses than do those who are not. Nevertheless, MOR does appear in the records of children who are not depressed at all. Thus, the features commonly associated with childhood depression are more than one V or FD response, the presence of even one Color–Shading blend, a low Egocentricity Index, eb elevated on the right, and an unusually high frequency of MOR content responses (Exner & Weiner, 1982). Exner and Weiner (1982, 1994) state that when four of these five are present, the child is experiencing either distress or, more likely, depression. The examiner must then decide whether the condition is reactive or chronic. Both V

reyn2-7.qxd

6/6/2003

9:08 AM

Page 193

7. Using the Rorschach with Children and Adolescents

responses and the Egocentricity Index are not particularly responsive to situational effects and take a long time to form. Thus, if the child’s record contains both V responses and a low Egocentricity Index, the condition is probably chronic. Conversely, if the child’s record contains m and Y responses with no elevation in V or depression of the Egocentricity Index, the condition is more likely to be situational or reactive. A more complex circumstance exists when an earlier situational stressor in the life of the child has not been effectively resolved. In these instances, both V responses and a low Egocentricity Index can appear together with m and Y in the child’s record, even when there is no evidence of enduring problems. There are several other Rorschach indices that may attend depression in children. The Affective Ratio may be low because of the child’s tendency to withdraw and avoid emotional situations. The frequency of S responses may be elevated. A low Lambda indicates that the child has become too involved in complexity, which actually aggravates the depression. An elevated W : M ratio indicates that the child cannot or will not economize in using resources; this is confirmed by an elevated Zf. If only the Zf is elevated, it is important for the examiner to note how much of the elevation is accounted for by W and how much consists of more complex responses. Finally, when the child’s X+% is below what is normally expected, the examiner should ascertain whether this is a function of unique and ideographic responses (Xu%) or actual distortion (X–%). The number of P responses in the child’s record can be useful in determining the significance of a low X+% (Exner & Weiner, 1982). Obviously, when a child is depressed, the question is not only whether the experience is situational or chronic but also how acutely the distress is felt by the child. More precisely, what is the likelihood that the child will act on this experienced distress and attempt or succeed in committing suicide? Unfortunately, there are no infallible guidelines. Exner (1978) has constructed a Suicide Constellation for Adults and attempted to construct one for children. However, the Rorschach Workshops (1987, p. 6) have issued a severe caution concerning the

193

Children’s Suicide Constellation. A crossvalidation study was conducted using a total sample of 51 cases, 36 of whom attempted and 15 of whom completed suicide. The results did not cross-validate, and Exner concluded that the “constellation is psychometrically worthless” and was not to be used. Rather, the Adult Suicide Constellation possesses a “true positive” hit rate of approximately 67% for children ages 15 and 16. The same is not true for children below this age.

The Rorschach and Childhood Schizophrenia The diagnosis of schizophrenia is, of course, never made on the basis of the Rorschach record alone. A thorough and reliable history and the behavioral observations of parents and teachers are invaluable supplements to information obtained in the assessment. Part of that information will be the examiner’s own observations of the child during the examination period. Thus, the examiner must be able to identify the salient behavioral features of schizophrenia. There are four major areas that pose difficulty for the individual with schizophrenia: cognition, perception, interpersonal relationships, and the ability to exert appropriate controls in each of these endeavors. The cognitions of the child or adolescent with schizophrenia are often marked by difficulties in maintaining clear and coherent associations, strained or deviant logic, and inappropriate levels of abstraction. Initially, and particularly in the younger child, these cognitive difficulties are extremely subtle, and the examiner may attribute their occurrence to inattentiveness, anxiety, and the like. However, the logic of the child with schizophrenia tends to be highly circumstantial, with frank dissociations or associations that are tenuous at best. For example, when asked his name, a boy with schizophrenia may respond, “My name is Bill, but my dog’s name is Charlie. I don’t know where he is, though.” Although this sort of thinking is common in young children, it usually disappears by age 7 (Exner & Weiner, 1982, 1994). Moreover, these children have marked deficiencies in their ability to abstract at appropriate levels. They are inclined to polarize, being either too concrete

reyn2-7.qxd

6/6/2003

194

9:08 AM

Page 194

II. PROJECTIVE METHODS

or too abstract. For example, in response to the comment, “Time is money,” the child may say, “No, it’s three o’clock.” At the other extreme, these youngsters become preoccupied with abstraction, such as “This bat looks more peaceful, young, like he’s got these big wings but doesn’t know what to do with them yet, just trying to experience.” Manifestations of these sorts of cognitive difficulties will appear in the language of the child during the Free Association and the Inquiry. Neither Ames and colleagues (1971, 1974) nor any of the other major Rorschach systematizers deal with schizophrenia across children’s age groups as succinctly as does the Comprehensive System (Exner, 1978, 1986, 1991, 1993; Exner et al., 1985; Exner & Weiner, 1982, 1994) . Therefore, Rorschach findings unique to the protocols of youngsters with schizophrenia will be expressed in the scoring terminology of the Exner method. In addition to evidence of cognitive disturbance in the child’s general manner of expression, the response language of the child will contain scorable features that range on a continuum from mild to severe. At the mild extreme are slips of the tongue and redundancies (DV) and Incongruous Combinations (Incom), such as “blue chickens.” Deviant Responses (DR), such as inappropriate or irrelevant phrases or circumstantial responses, and Fabulized Combinations (FABCOM), such as “two cats sitting on a rocket,” are in the midrange of severity. At the severe extreme of the continuum are the Contamination (CONTAM) and Inappropriate Logic (ALOG) responses. CONTAM responses represent the fusion of two percepts into one, as exemplified in the response, “I see a kitty and a bird. It’s a catbird.” ALOG responses, on the other hand, contain strained or circumstantial logic, such as “They must be in love because this heart is between them.” In terms of interpretive significance, the presence of one or even several of the milder responses is expected in a child’s record, but the presence of only one CONTAM is a serious indicator of pathology. Moreover, each of these types of scores has an inherent range of possible responses from mild to severe. For some of these special scores, Exner (1986) has developed a

system for differentiating levels of severity, with Level I scores indicating mild to moderate cognitive slippage and Level II scores suggesting more severe slippage. The child’s cognitive difficulties may include an inappropriate level of abstraction. On the Rorschach, this is often manifested in a preoccupation with abstraction and symbolism. It may take the form of an unusually high frequency of responses containing conventional symbolism, or it may emerge in responses containing highly personalized ideographic symbolism. In either case, the child’s overuse of or preoccupation with symbolism increases the possibility of a thought disturbance. It should be noted, however, that in the intelligent and otherwise healthy child, moderate use of symbolism and/or abstraction is probably a positive indicator. Youngsters with schizophrenia also demonstrate distorted perceptions that lead to poor judgment. They tend to misperceive both themselves and others. An adolescent girl with no training or skills will insist that she is a highly talented designer who simply has no outlet for her superior gifts, for example, and will refuse to consider employment opportunities appropriate to her background. Or, a casual remark, such as “It’s been nice to see you,” will be misconstrued as a serious romantic innuendo. On the Rorschach, distortions in perception are measured by the Form Quality, the F+%, and the X–% (Distorted Form). Scores below .70 for both the F+% and the X+% generally signal impaired perceptual functioning, and scores below .60 are always indicative of inaccuracy. In a brief Rorschach record, it is advisable to rely on the X+%, because the F+% may be representative of only a few responses. When the X+% is below .70, the X–% will indicate whether the low X+% is the result of idiosyncratic responding (as represented in Form Quality scores of “unusual”) or whether it is the result of frank distortion (as represented in “minus” Form Quality scores). Ideographic responding does not have the same prognostic implications as does seriously impaired perceptual accuracy. The examiner should scan the sequence of scores to determine whether clusters of “unusual” or “minus” responses occur in relation to particular determinants or con-

reyn2-7.qxd

6/6/2003

9:08 AM

Page 195

7. Using the Rorschach with Children and Adolescents

tents. When patterns occur, it is highly likely that the impaired perception is linked to specified precursors as opposed to overall inaccuracy, and interventions are far more apt to be successful. Another Rorschach indicator of perceptual accuracy is the P response, which measures the child’s ability to see things as others do, or to respond conventionally. Youngsters age 11 or older who deliver less than four P responses have an impaired ability to respond conventionally. Before age 11 this is not always true, and the examiner must use the normative data carefully to determine the significance of a low number of P responses in the individual child. Moreover, P responses occur most frequently to Cards I, III, V, and VIII; thus if the child gives only three or four P responses, they should occur on these cards. When the child does this, the concern regarding perceptual distortions and poor judgment is somewhat diminished. When the child delivers three or four P responses to cards other then these, the concern is magnified considerably. Because of disordered thinking and misperceptions of self and others, youngsters with schizophrenia have poor social skills and therefore great difficulty in forming and maintaining interpersonal relationships. Even when participating in group activities, these children tend to be distant and emotionally withdrawn. Often they are physically withdrawn as well. The child’s history and the behavioral observations of parents and teachers will provide vital information regarding patterns of interpersonal functioning, as well as onset and duration of any deviations from established patterns. Naturally, the assessment process itself provides an opportunity for the examiner to engage the child in various interactions directly. The primary Rorschach indicators of interpersonal difficulties are inferior M and H production. Although conservatism is warranted when interpreting the records of young children, the absence of M in a child’s record generally signals a serious withdrawal from interest in people. Moreover, M is expected to appear with good Form Quality. A child, even as young as age 5 who delivers an M response with “minus” Form Quality will tend to assign inaccurate and illusory meanings to social situations and to

195

have deficient social skills (Exner & Weiner, 1982, 1994). Other, less severe indicators of the child’s difficulty in dealing with people are M responses with fictional or mythological human or animal content or human or animal detail content. Because it is possible for the child to give responses with human content but without human movement, the presence of good H in the child’s record is a positive sign in the young child or in the child whose operations are not yet sufficiently complex to produce M. However, the absence of H or the presence of H in conjunction with M– is interpreted in essentially the same way as the absence of M or the presence of M– responses. An additional problem for the child with schizophrenia is the inability to exert appropriate controls in the cognitive, perceptual, and interpersonal spheres. These children are preoccupied with anxiety-producing thoughts, largely of sexual and aggressive content. Moreover, they are unable to integrate their thoughts and feelings properly, resulting in either blunted or unsuitable expressions of affect. These difficulties are expressed in Rorschach contents of cruelty, violent aggression and flagrant sexuality. Whereas the average child reports aggression in terms of “fighting” or some type of MOR, the child with schizophrenia delivers responses that reflect gore and brutality. Similarly, although sexual responses are rare in a child, they normally tend to represent developmental concerns when they do occur. Sexual responses occur more frequently in the records of children with schizophrenia and tend to reflect a disturbed preoccupation. These children’s inability to exert effective controls is often apparent in an FC : CF + C ratio that is weighted to the right. Throughout the developmental process, there is a gradual shift of the weighting from the right to the left side of the ratio. Nevertheless, the appearance of pure C in the record after age 8 warrants investigation and signals the possibility of episodes of unmodulated affect. Once the examiner has established substantial support for the presence of a thought disorder in a child, the question of an acute versus a chronic process remains. Generally speaking, when the frequency of scores on variables measuring cognitive disturbance,

reyn2-7.qxd

6/6/2003

9:08 AM

196

Page 196

II. PROJECTIVE METHODS

perceptual inaccuracy, interpersonal difficulties, and problems with control are relatively low and do not deviate excessively from the norms, the process is likely to be an acute and reactive one, probably in the early stages. In this context, the presence of increased numbers of INCOM, FABCOM, and ALOG responses may be indicative of the child’s attempt to manage and make sense of his or her world, in contrast to the apathy typical of the chronic process. Acute onset is usually related to a precipitating event or a situational precursor; the child is aware of his or her diminished capacities and is upset and anxious in this regard. These youngsters often spontaneously recognize the inadequacy of their responses. The hallmarks of schizophrenia are disordered thinking and perceptual inaccuracy. Although bizarre contents and peculiar language may augment these findings, they may not substitute for more concrete evidence of serious cognitive slippage and impaired perception. Moreover, Exner and colleagues (1985) demonstrated that many of the cognitive and perceptual operations unique to schizophrenia are highly resistant to change, even with therapeutic intervention and even over extended periods of time.

A LAST WORD Now, a word of caution. We are all familiar with the need to select and use instruments with current, up-to-date norms, but we are also familiar with problems that arise when old tests are modified and new norms are established (e.g., the original vs. revised versions of the Wechsler Intelligence Scale for Children, or the Stanford–Binet, Form L-M vs. the Fourth Edition). As Exner (1986) suggests, “It would be foolhardy to suggest that the work is finished. The Rorschach continues to pose many unanswered questions” (p. 4). Although the Comprehensive System is currently the most widely used approach to the Rorschach, controversy about the system and the Rorschach continues today, particularly around such issues as interrater reliability, reliability of administration and recording of responses, and questions of validity of certain scores and indices (i.e., Exner, 1996; Wood et al., 1996a, 1996b).

Research on the Exner Comprehensive System of the Rorschach is an ongoing process that results in modifications in scoring criteria, changes in norms for some scores, and refinement of definitions for some variables. The Rorschach Research Foundation (Rorschach Workshops) publishes periodic newsletters, updating for previous participants any changes that have been made in the system and keeping them abreast of ongoing research. However, not all individuals trained in the Exner system receive this information. The caution, then, is that all users of the system should keep current with the progress being made.

REFERENCES Ames, L. B., Metraux, R. W., Rodell, J. L., & Walker, R. N. (1974). Child Rorschach responses (rev. ed.). New York: Brunner/Mazel. Ames, L. B., Metraux, R. W., & Walker, R. N. (1971). Adolescent Rorschach responses. New York: Brunner/Mazel. Ames, L., & Walker, R. (1964). Prediction of later reading ability from kindergarten Rorschach and I.Q. scores. Journal of Educational Psychology, 55, 309–313. Bartell, S. S., & Solanto, N. V. (1995). Usefulness of the Rorschach Inkblot Test in assessment of attention deficit hyperactive disorders. Perceptual and Motor Skills, 80, 531–541. Champion, L., Johnson, P. J., McCreary, J. H., & Doughtie, E. B. (1984). Preliminary investigation into the Rorschach response patterns of children with documented learning disabilities. Journal of Clinical Psychology, 40, 329–333. Crain, W. C., & Smoke, L. (1981). Rorschach aggressive content in normal and problematic children. Journal of Personality Assessment, 45, 2–4. Exner, J. E. (1974). The Rorschach: A comprehensive system (Vol. 1). New York: Wiley. Exner, J. E. (1978). The Rorschach: A comprehensive system: Vol. 2. Recent research and advanced interpretation. New York: Wiley. Exner, J. E. (1980). But it’s only an inkblot. Journal of Personality Assessment, 44, 562–577. Exner, J. E. (1986). The Rorschach: A comprehensive system. Vol. 1. Basic foundations (2nd ed.). New York: Wiley. Exner, J. E. (1991). The Rorschach: A comprehensive system. Vol 2: Interpretation (2nd ed.). New York: Wiley. Exner, J. E. (1993). The Rorschach: A comprehensive system. Vol. 1: Basic foundations (3rd ed.). New York: Wiley. Exner, J. E. (1996). A comment on “The comprehensive system for the Rorschach: A critical examination.” Psychological Science, 7(1), 11–13.

reyn2-7.qxd

6/6/2003

9:08 AM

Page 197

7. Using the Rorschach with Children and Adolescents Exner, J. E., Armbruster, G. L., & Mittman, B. L. (1978). The Rorschach response process. Journal of Personality Assessment, 42, 27–38. Exner, J. E., & Leura, A. V. (1976). Variations in the ranking of Rorschach responses as a function of situational set (Workshops Study No. 221). Unpublished manuscript, Rorschach Workshops. Exner, J. E., Leura, A. V., Wylie, J. R., Armbruster, G. L., & Thomas, E. A. (1980). A longitudinal Rorschach study with children (Workshop Study No. 207). Unpublished manuscript, Rorschach Workshops. Exner J. E., & Martin, L. S. (1983). The Rorschach: A history and description of the Comprehensive System. School Psychology Review, 12, 407–413. Exner, J. E., & McCoy, R. (1981). An experimental score for morbid content (MOR) (Workshop Study No. 269). Unpublished manuscript, Rorschach Workshops. Exner, J. E., Rosenthal, N., & Thomas, E. (1980). A Rorscha.ch study involving retesting several times at brief intervals (Workshop Study No. 270). Unpublished manuscript, Rorschach Workshops. Exner, J. E., Thomas, E. A., Mason, B. J. (1985). Children’s Rorschachs: Description and prediction. Journal of Personality Assessment, 49, 13–20. Exner, J. E., & Weiner, I. B. (1982). The Rorsahach: A comprehensive system: Vol. 3. Assessment of children and adolescents. New York: Wiley. Exner, J. E., & Weiner, I. B. (1994). The Rorschach: A comprehensive system: Vol. 3. Assessment of children and adolescents (2nd ed.). New York: Wiley. Gordon, M., & Oshman, H. (1981). Rorschach indices of children classified as hyperactive. Perceptual and Motor Skills, 52, 703–707. Gordon, M., & Tegtmeyer, P. F. (1982). The Egocentricity Index and self-esteem in children. Perceptual and Motor Skills, 55, 335–337. Gordon, M., & Tegtmeyer, P. F. (1983). Oral–dependent content in children’s Rorschach protocols. Perceptual and Motor Skills, 57(3, Pt. 2), 1163–1168. Greenberg, R., & Gordon, N. (1983). Examiner’s sex and children’s Rorschach productivity. Psychological Reports, 53, 335–337. Halpern, F. (1953). A clinical approach to children’s Rorschachs. New York: Grune & Stratton. Hulsart, B. (1979). The effects of a second chance instructional set on the Rorschach of emotionally disturbed and culturally deprived children. Unpublished doctoral dissertation, Long Island University. Klopfer, B. (1956). Developments in the Rorschach technique (Vol. 2). Yonkers, NY: World. Klopfer, B., Ainsworth, N. D., Klopfer, G., & Holt, R. (1954). Developments in the Rorschach technique (Vol. 1). Yonkers, NY: World. Krall, V., Sacks, H., Lazar, B., Rayson, R., Growe, G., Navar, L., & O’Connell, L. (1983). Rorschach

197

norms for inner city children. Journal of Personality Assessment, 47, 155–157. Leura, A. V., & Exner, J. E. (1976). Rorschach performance of children with a multiple foster home history (Workshop Study No. 220). Unpublished manuscript, Rorschach Workshops. Leura, A. V., & Exner, J. E. (1977). The effects of Inquiry after each card on the distribution of scores in the records of young children (Workshops Study No. 265). Unpublished manuscript, Rorschach Workshops. Leura, A. V., & Exner, J. E. (1978). Structural differences in the records of adolescents as a function of being tested by one’s own teacher (Workshops Study No. 247). Unpublished manuscript, Rorschach Workshops. Pierce, G. E. (1978). The absent parent and the Rorschach “T” response. In E. J. Hunter & D. S. Nice (Eds.), Children of military families (pp. 71–87). Washington, DC: U.S. Government Printing Office. Piotrowski, C. (1996). The status of Exner’ s Comprehensive System in contemporary research. Perceptual and Motor Skills, 82, 1341–1342. Rorschach Workshops. (1987). 1987 alumni newsletter. Asheville, NC: Author. Scott, R. S. (1985). Exner’s Four Square: Useful index in appraisal of LD? Perceptual and Motor Skills, 60, 525–526. Smith, N. N. (1981). The relationship between the Rorschach Whole responses and level of cognitive functioning. Journal of Personality Assessment, 45, 13–19. Smock, C. (1958). Perceptual rigidity and closure phenomenon as a function of manifest anxiety in children. Child Development, 29, 237–247. Smock, C., & Holt, R. (1962). Children’s reactions to novelty: An experiential study of curiosity motivation. Child Development, 33, 631–642. Tegtmeyer, P. F., & Gordon, N. (1983). Interpretation of White Space responses in children’s Rorschach protocols. Perceptual and Motor Skills, 57, 611–616. Tuma, J. A., & McCraw, R. K. (1975). Influences of examiner differences on Rorschach productivity in children. Journal of Personality Assessment, 39, 362–368. Williams, A. L., & Miles, T. R. (1985). Rorschach responses of dyslexic children. Annals of Dyslexia, 35, 51–66. Wood, J. N., Nezworski, M. T., & Stejskal, W. J. (1996a). The comprehensive system for the Rorschach: A critical examination. Psychological Science, 7(1), 3–10. Wood, J. N., Nezworski, N. T., & Stejskal, W. J. (1996b). Thinking critically about the comprehensive system for the Rorschach: A reply to Exner. Psychological Science, 7(1), 14–17.

reyn2-8.qxd

6/6/2003

9:10 AM

Page 198

8 Use of the Holtzman Inkblot Technique with Children

WAYNE H. HOLTZMAN JON D. SWARTZ

the Rorschach, which has only 10 inkblots in a single form, the HIT consists of two parallel forms, A and B, each of which contains 45 inkblots constituting the test series and 2 practice blots (X and Y) that are identical in both forms. The inkblots were drawn from a large pool of several thousand, many of which were created by an artist working with special papers and inks that produced brilliant colors and rich shading. Only about 1 blot in 50 survived initial screening by a group of judges who were familiar with the Rorschach. Selection of inkblots for the final version of the HIT was aimed at maximizing the reliability of these scores, as well as maximizing the discriminatory power of the final forms in differentiating superior normals from mental hospital patients in a series of standardization studies (Holtzman, Thorpe, Swartz, & Herron, 1961b). The two parallel forms were constructed by pairing blots on stimulus qualities as well as item characteristics from the scores, and then randomly assigning members of each pair to either Form A or Form B. The final order of presentation for the 45 inkblots in each form was arranged so that most of the “best” inkblots appear rather early in the series. In the major standardization program,

The Holtzman Inkblot Technique (HIT)1 was developed to overcome psychometric limitations in the Rorschach by constructing completely new sets of inkblots. Although much of the early research on the Rorschach was either irrelevant or poorly conceived, an impressive number of well-designed validity studies generally yielded negative results. The growing realization that the Rorschach had inherent psychometric weaknesses came to a head in a symposium on failures of the Rorschach that was sponsored by the Society for Projective Techniques (Zubin, 1954). The evidence to date seems to indicate that the HIT has answered most, if not all, of these criticisms of the Rorschach (Holtzman & Swartz, 1983). By the end of 1999, more than 800 publications using the HIT had appeared in the world literature (Swartz, Reinehr, & Holtzman, 1999). HIT research to date, therefore, generally verifies the initial promise of this newer inkblot technique (Sundberg, 1962).

DESCRIPTION OF THE TECHNIQUE The HIT attempts to capture the best qualities of both the projective and the psychometric approaches to the Rorschach. Unlike 198

reyn2-8.qxd

6/6/2003

9:10 AM

Page 199

8. Holtzman Inkblot Technique with Children

using printed versions of the original inkblots, nearly 2,000 individual protocols were collected on samples ranging from 5year-olds to mature adults and from chronic schizophrenic patients to mentally retarded individuals. Through the cooperation of psychologists in other settings across the country, 15 different, well-defined populations were sampled to provide the standardization data. Five of these samples were normal children—122 children age 5 from nursery schools in Austin, Texas; 60 children in second through sixth grades from a middle-class, private school in Austin; 72 fourth graders from Hamden, Connecticut; 197 seventh graders from four Texas communities other than Austin; and 72 adolescents in 11th grade from Chicago high schools. This last sample was given both the Rorschach and HIT in a comparative study of the two methods (Bock, Haggard, Holtzman, Beck, & Beck, 1963). Percentile norms subsequently were published for emotionally disturbed children and adolescents and for male juvenile delinquents (Hill, 1972). Additional normative data for four representative groups of children and adolescents seen in clinical practice have been published by Morgan (1968). The norms for emotionally disturbed children published by Hill are based on HIT protocols collected by Conners (1965) in clinical studies of 99 emotionally disturbed children and 114 neurotic adolescents. The norms for male juvenile delinquents are based on 75 cases collected by Megargee (1965) and replicated and extended by Mullen, Reinehr, and Swartz (1983). When taken together with the earlier sets of norms for mentally retarded and normal children, the percentile norms that are available for the HIT provide a rich source of helpful information to be used in psychodiagnosis and personality assessment. The standardization data also were used for a number of methodological studies, including investigations of scorer agreement, internal-consistency reliability, test–retest stabilities, and intergroup differences as a preliminary basis for differential diagnosis. The results of these studies, together with percentile norms and recommendations for use of the HIT in clinical assessment or for research, were published in 1961 (Holtzman et al., 1961b). While retaining the clin-

199

ical sensitivity of the Rorschach, the HIT yields 22 standardized variables that can be objectively defined, reliably scored, and efficiently handled by statistical methods. For the first time, it allows the clinician, the psychometrician, and the experimentalist to work with the same projective technique. Greater reliability and objectivity of scoring on the HIT are obtained because of the large number of inkblots and the fact that the subject is limited to one response per blot. A scoring guide (Holtzman, Thorpe, Swartz, & Herron, 1961a) further increases objectivity by making interscorer differences negligible on most of the major variables. The existence of two parallel forms of the same technique permits accurate retesting to evaluate change over time. Since the publication of the individual version of the HIT by the Psychological Corporation in 1961, hundreds of studies have been reported in the world literature (Swartz et al., 1999). Our own work has focused largely on the development of a group version (Herron, 1963; Holtzman, Moseley, Reinehr, & Abbott, 1963; Swartz & Holtzman, 1963; Swartz, Witzke, & Megargee, 1970); a quick group version (Swartz & Reinehr, 1983); a computer method for scoring and interpreting HIT variables (Gorham, 1967; Holtzman, 1975; Vincent, 1982); a new experimental version of the HIT (Holtzman, 1988; Swartz, 1992); and a major cross-cultural study of over 800 children in Mexico and the United States, involving repeated measures with alternate forms of the HIT 6 years in a row (Holtzman, Diaz-Guerrero, & Swartz, 1975). Some of the HIT findings from the crosscultural study that are particularly relevant in the assessment of children are presented later in this chapter.

TEST MATERIALS Standard materials for the HIT consist of two parallel series, Form A and Form B; the accompanying printed Record Forms and Summary Sheets; and the Guide for Administration and Scoring. The inkblots are printed on thin but tough white cardboard 5½ × 8½ inches in size. Cards X and Y contain practice blots that usually are not scored. These two cards appear at the be-

reyn2-8.qxd

6/6/2003

200

9:10 AM

Page 200

II. PROJECTIVE METHODS

ginning of both Forms A and B. Card X is a massive achromatic blot that looks like a bat or butterfly to most people. Few subjects reject this card, although some prefer to use a smaller area than the whole blot. Card Y is suggestive of a person’s torso to most subjects. Red spots of ink introduce the subject to color and often evoke responses such as “spots of blood,” either given alone or interpreted with the torso. Cards 1 and 2 in both Forms A and B are achromatic and sufficiently broken up to make a whole response difficult unless there is integration of detail, or unless the subject gives a vague concept or one in which the form of the concept fails to fit the form of the inkblot. Both cards have popular responses in smaller areas of the blots, helping to break up a response set to give only wholes. Card 3 is irregular in form and has a large red “sunburst” splotch overlaid on an amorphous black inkblot. It is difficult to give a form-definite, form-appropriate whole response to Card 3 because of the chaotic, unstructured nature of this inkblot. Card 4 is just the opposite, containing several finely detailed popular concepts that can be interrelated, together with color and shading that produces a vista-like effect. “A battle scene” or “a cowboy watching a sunset” is a typical response to Card 4A, and “a knight carrying a spear and shield” is a typical response to Card 4B. Cards 5A and 5B are asymmetrical, grayish-colored blots unlike any in the Rorschach. By penetrating the charcoal-like quality of these blots, one can distinguish a number of detailed objects. Together with several similar, rather wispy, amorphous, asymmetrical blots later in the series, these cards are difficult, particularly for the individual who is searching for definite concepts having good form or who wishes to use the entire blot. The remaining inkblots cover a wide range of stimulus variation, giving the individual ample opportunity to reveal certain aspects of his or her mental processes and personality by projecting thoughts onto otherwise meaningless inkblots. Twelve of the inkblots in Form A are black or gray; 2 are monochromatic; 11 are black with a bright color also present; and the remaining 20 are multicolored. Most of the blots have rich shading variations, which help to elicit tex-

ture responses. A similar distribution of color, shading, and form qualities is present in Form B.

ADMINISTRATION AND SCORING Standard procedures for administering the HIT have been developed so that published normative data may be used as aids to interpretation. Instructions to the subject have been designed to make the task as simple as possible while eliciting sufficient information to score major variables reliably. The instructions differ from those for the Rorschach in the following ways: (1) The examiner instructs the examinee to give only one response per card; (2) a brief inquiry is given immediately after each response; (3) permissible questions by the examiner during inquiry are limited both in number and in scope, and are asked rather routinely to avoid inadvertent verbal conditioning of certain determinants or content. Three kinds of questions are permissible as part of the brief inquiry in the standard administration. The actual wording used can vary a great deal, so that the inquiry becomes a natural part of the conversation between examiner and subject. Typical phrasing should be as follows: “Where in the blot do you see a ______________. “What is there about the blot that makes it look like a ___________? “Is there anything else you care to tell me about it?” After establishing rapport, the examiner picks up the cards one at a time, handing each one in an upright position to the subject. The instructions given to the subject should be informal and should stress the following points: (1) These inkblots were not made to look like anything in particular; (2) different people see different things in each inkblot; and (3) only one response for each card is desired. The examiner uses a Record Form for recording the responses and scoring. To facilitate the recording of the location of the response, schematic diagrams for the inkblots are included on the Record Form. As each response is given, the examiner outlines a specific area used. Adja-

reyn2-8.qxd

6/6/2003

9:10 AM

Page 201

8. Holtzman Inkblot Technique with Children

cent to the diagram is a blank space for recording the verbatim response or a shortened version of it. Usually the subject comprehends the nature of the task quickly, and the actual inquiry can be kept to a minimum. A skilled examiner, sensitive to subtle nuances in the examiner–subject interaction, can control the flow of conversation by stimulating a reticent individual and slowing down a verbose person. In spite of the many interesting variations in test administration that can be attempted, there is much to be said for adhering closely to the standard method of administration. This method has proved highly practical and yields objective, reliable scores on a number of important variables. Currently published normative data and statistical studies of value in the interpretation of the protocols assume close adherence to the standard method of administration. One exception to the standard method of administration has been successfully developed for young children, whose attention span is short and who therefore may get restless halfway through the testing session. As in the case of the 122 Austin 5-year-olds, the examiner may temporarily interrupt the testing session after 20–30 cards, returning to finish the task after interpolated activity of a relaxing kind. In testing young children, the task can be structured as a playful, game-like activity, heightening the children’s interest in attending to it. This technique has been used successfully for some children as young as 3 years of age, although no norms are available below the age of 5. In the course of standardization, 22 quantitative variables were developed to cover nearly all the important scoring categories and dimensions commonly employed with the Rorschach. Table 8.1 gives the names, abbreviations, brief definitions, and scoring weights for these variables. Summary scores for the individual variables are obtained by adding the weights for a given variable across the 45 inkblots in either Form A or Form B. Three of the variables routinely are “corrected” for the number of rejections in order to provide an estimate of what the total score would have been if the subject had given a response to each of the 45 inkblots. Scoring agreement is uniformly high when trained scorers are compared. Inter-

201

correlations between two independent scorers ranged from .89 to .995, with a median value of .98 in a sample of 40 protocols from a schizophrenic sample. When beginning scorers were compared, average scoring reliability for all variables yielded a median value of .86 in a large sample of normal adolescents. The more difficult variables to score, such as Pathognomic Verbalization and Form Appropriateness, require a greater degree of training. Qualified clinicians and research investigators, however, have had little difficulty achieving satisfactory scoring reliability.

RELIABILITY OF HIT SCORES FOR CHILDREN Both internal-consistency and parallelforms reliability based on repeated testing have been reported in great detail elsewhere (Holtzman et al., 1961b, 1975). Split-half reliabilities, determined by computing the correlations between scores based on oddnumbered and even-numbered blots, are generally high. The four scores with highest internal consistency (generally about .90), regardless of populations sampled (ranging from 5-year-old children to adults with schizophrenia), are Reaction Time, Rejection, Location, and Form Definiteness. Eight additional scores that have internal consistency reliability generally higher than .80 are Form Appropriateness, Color, Shading, Movement, Pathognomic Verbalization, Human, Animal, and Anatomy. Odd–even reliability coefficients for the symbolic content score—Anxiety, Hostility, Barrier, and Penetration—are only slightly lower on the average and are more variable. Four scores—Space, Sex, Abstract, and Balance—occur too infrequently for accurate estimates of reliability, leaving only Popular as a score with relatively unsatisfactory internal consistency. The most pertinent reliability measure for most clinical applications is the stability of an individual’s score across time. Unlike the Rorschach, with which spuriously high results are obtained because of retesting with the same inkblots, the HIT has truly parallel forms that provide conservative but realistic estimates of reliability of measurement over time. The best measure of such stability is

reyn2-8.qxd

6/6/2003

9:10 AM

Page 202

TABLE 8.1. Names, Abbreviations, Brief Definitions, and Scoring Weights for 22 HIT Variables Reaction Time (RT). The time in seconds from the presentation of the inkblot to the beginning of the primary response. Rejection (R). Score 1 when subject returns inkblot to examiner without giving a scorable response; otherwise, score 0. Location (L). Tendency to break down blot into smaller fragments. Score 0 for use of the whole blot, 1 for large area, and 2 for smaller area. Space (S). Score 1 for true figure–ground reversals; otherwise, score 0. Form Definiteness (FD). The definiteness of the form of the concept reported, regardless of the goodness of fit to the inkblot. A 5-point scale with 0 for very vague and 4 for highly specific. Form Appropriateness (FA). The goodness of fit of the form of the percept to the form of the inkblot. Score 0 for poor, 1 for fair, and 2 for good. Color (C). The apparent primacy of color (including black, gray, or white) as a response determinant. Score 0 for no use of color, 1 for use secondary to form (like Rorschach FC), 2 when used as primary determinant but some form present (like CF), and 3 when used as primary determinant with no form present (like C). Shading (Sh). The apparent primacy of shading as response determinant (texture, depth, or vista). Score 0 for no use of shading, 1 when used in secondary manner, and 2 when used as primary determinant with little or no form present. Movement (M). The energy level of movement or potential movement ascribed to the percept, regardless of content. Score 0 for none, 1 for static potential, 2 for casual, 3 for dynamic, and 4 for violent movement. Pathognomic Verbalization (V). Degree of autistic, bizarre thinking evident in the response as rated on a 5-point scale. Score 0 where no pathology is present. The nine categories of V and the range of scoring weights for each is as follows: Fabulation, 1; Fabulized Combination, 2, 3, 4; Queer Response, 1, 2, 3; Incoherence, 4; Autistic Logic, 1, 2, 3, 4; Contamination, 2, 3, 4; Self-Reference, 2, 3, 4; Deterioration Color, 2, 3, 4; Absurd Response, 3. Anatomy (At). Degree of “gut-like” quality in the content. Score 0 for none; 1 for bones, X-rays, or medical drawings; and 2 for visceral and crude anatomy. Sex (Sx). Degree of sexual quality in the content. Score 0 for no sexual reference; 1 for socially accepted sexual activity or expressions (buttocks, bust, kissing); and 2 for blatant sexual content (penis, vagina). Abstract (Ab). Degree of abstract quality in the content. Score 0 for none, 1 for abstract elements along with other elements having form, and 2 for purely abstract content (“ bright colors remind me of gaiety”). Anxiety (Ax). Signs of anxiety in the fantasy content as indicated by emotions and attitudes, expressive behavior, symbolism, or cultural stereotypes of fear. Score 0 for none, 1 for questionable or indirect signs, and 2 for overt or clear-cut evidence. Hostility (Hs). Signs of hostility in the fantasy content. Scored on a 4-point scale ranging from 0 for none to 3 for direct, violent, interpersonal destruction. Barrier (Br). Score 1 for reference to any protective covering, membrane, shell, or skin that might be symbolically related to the perception of body image boundaries; otherwise, score 0. Penetration (Pn). Score 1 for concept that might be symbolic of an individual’s feeling that his or her body exterior is of little protective value and can be easily penetrated; otherwise, score 0. Balance (B). Score 1 where there is overt concern for the symmetry–asymmetry feature of the inkblot; otherwise, score 0. Popular (P). Each form contains 25 inkblots in which one or more popular percepts occur. “Popular” in the standardization studies means that a percept had to occur at least 14% of the time among normal subjects. Score 1 for popular core concepts (or their precision alternatives) as listed in the scoring manual; otherwise, score 0. Integration (I). Score 1 for the organization of the two or more adequately perceived blot elements into a larger whole; otherwise, score 0. Human (H). Degree of human quality in the content of response. Score 0 for none; 1 for parts of humans, distortions, or cartoons; and 2 for whole human beings or elaborated human faces. Animal (A). Degree of animal quality in the content. Score 0 for none (including animal objects and microscopic life); 1 for animal parts, bugs, or insects; and 2 for whole animals.

202

reyn2-8.qxd

6/6/2003

9:10 AM

Page 203

8. Holtzman Inkblot Technique with Children

the intraclass correlation obtained in a Latin square design, where half of the subjects are randomly selected to receive Form A before B, while the other half receive B before A. The most extensive studies of this kind on college students yielded stability coefficients ranging from a low of .36 for Popular to a high of .82 for Location, with an interval of 1 week between tests. Similar studies on other normal subjects with time intervals between testing sessions ranging from 3 months to 5 years provide additional evidence of the stability of HIT scores across time. The most extensive data come from a major cross-cultural study of over 800 children in Mexico and the United States, involving repeated measures with alternate forms 6 years in a row (Holtzman et al., 1975). Beginning during the 1962–1963 school year in Austin, Texas, 133 first graders, 142 fourth graders, and 142 seventh graders were tested with the HIT as part of a large battery of perceptual, cognitive, and personality tests. Annual testing took place on the anniversary date of the initial testing until 6 years of repeated measurement had been completed. Table 8.2 presents the basic design of this study. A complete replication of the Austin longitudinal project was begun in Mexico City in 1964 under the direction of Rogelio Diaz-Guerrero and his associates. Table 8.3 presents split-half reliability coefficients for 17 HIT scores in the first year of testing for these large samples of children in Mexico and the United States. From these statistics, it is clear that scores on the HIT generally have high reliability for schoolchildren of all ages as well as for adults. The results for internal consistency of

203

HIT scores among preschool children are equally high, as evidenced by the results obtained for the 122 children age 5 who were tested in Austin nursery schools as part of the standardization sample. Split-half reliability coefficients ranged from .53 for Penetration to .97 for Reaction Time. The median or average reliability for all variables among the 5-year-olds was .86. Scores on the HIT, the Human Figure Drawing Test, and the Vocabulary and Block Design subtests of the Wechsler Intelligence Scale for Children (WISC) in the cross-cultural longitudinal project provide an unparalleled opportunity to examine the degree of test–retest stability of these measures over time intervals varying from 1 to 5 years and for schoolchildren of all ages. On a theoretical basis, one would anticipate that the magnitude of such correlations would fall somewhere in the middle ranges—say, from .40 to .80. Correlations much higher than this would indicate rather rigid, unchanging personality characteristics; correlations much lower would reveal instability sufficiently serious to call into question the enduring nature of the measured personality traits over time. Among young children, one would expect lower stability coefficients than among adolescents or adults, because personality and cognitive development proceed more rapidly at the younger ages. And, finally, on a theoretical basis, one would also expect that stability would gradually drop as the interval of time between testing increased from 1 to 5 years. The most stable of all HIT scores is Location. Table 8.4 presents the complete set of test–retest correlations for Location, to illustrate the power of this methodology for

TABLE 8.2. Overlapping Longitudinal Design for 6 Years of Repeated Testing Group

Initial agea

School grades covered

I II III

6.7 years 9.7 years 12.7 years

123456 456789 7 8 9 10 11 12

Note. From Holtzman, Diaz-Guerrero, and Swartz (1975). Copyright 1975 by the Hogg Foundation for Mental Health. Reprinted by permission. a The starting ages of 6 years, 8 months; 9 years, 8 months; and 12 years, 8 months were chosen when a pilot study revealed most children in the public schools of Texas reach these exact ages at some time during the school year. Actual testing took place within 30 days of the age specified.

reyn2-8.qxd

6/6/2003

9:10 AM

204

Page 204

II. PROJECTIVE METHODS

TABLE 8.3. Split-Half Reliability Coefficients for 17 HIT Variables

Variable Reaction Time Rejection Location Form Definiteness Form Appropriateness Color Shading Movement Pathognomic Verbalization Integration Human Animal Anatomy Anxiety Hostility Barrier Penetration

Mexico __________________________ 6 9 12 (147)a (141) (149) .94 .95 .94 .91 .90 .94 .61 .84 .86 .57 .84 .92 .83 .92 .95 .70 .52

.94 .87 .95 .88 .82 .84 .55 .83 .52 .69 .81 .83 .86 .73 .63 .57 .69

.96 .91 .95 .89 .77 .78 .58 .85 .87 .77 .79 .71 .75 .70 .66 .46 .63

United States _________________________ 6 9 12 (133) (142) (142) .92 .90 .95 .90 .86 .93 .74 .86 .90 .58 .82 .80 .91 .78 .88 .75 .81

.97 .93 .97 .80 .81 .83 .58 .87 .76 .85 .81 .73 .80 .68 .72 .51 .75

.97 .88 .95 .86 .67 .85 .78 .88 .79 .82 .83 .72 .69 .80 .78 .52 .63

Note. From Holtzman and Swartz (1975). Copyright 1975 by the Hogg Foundation for Mental Health. Reprinted by permission. a Numbers outside parentheses indicate age of each group; numbers within parentheses indicate the number of children in each group.

estimating stability across time. With the exception of the youngest children in the first year, the test–retest correlations for Location were high in both Mexico and the United States, ranging into the .80s for the older children even after several years of testing. It is interesting to note in Table 8.4 that even after an interval of 5 years, the stability of Location was still moderately high, averaging .46 for all six groups combined. It should be pointed out that the availability of parallel forms for the HIT means that an interval of 2 years takes place before the child responds again to the identical form. Two years is a sufficiently long time for memory of the initial responses to fade almost completely. The use of a staggered longitudinal design with overlapping groups, as noted in Table 8.2, also makes possible the isolation of any practice or adaptation effects regardless of the form used. A detailed analysis of the differences that can be attributed to culture, age, sex, and trial of repeating testing has been reported elsewhere (Holtzman et al., 1975). Only selected highlights of the findings are presented here.

In a major analysis of variance of Location scores, noticeable adaptation to repeated testing was indeed found over the 6-year period. Mexican children tended to use smaller areas of the inkblot than did American children. Similarly, children of all ages in both cultures tended to use smaller detail areas more often than whole inkblots as the test was repeated. The amount of adaptation was much greater for the Mexican children in the first 2 years of testing. Of all the variables analyzed, only Location showed this adaptation effect over a year of testing, and even then the stability of individual differences through time was unusually high. Close behind Location in stability were Reaction Time, Form Definiteness, Movement, and Human. These variables compared favorably with scores on Vocabulary and Block Design and with the Harris– Goodenough developmental score on Human Figure Drawing, with respect to stability over a long period of time. Of the 17 HIT scores sufficiently well distributed to permit the use of product-moment correlation coefficients, 6 had generally low stability coefficients ranging from insignificant values into the .40s and .50s, with an occasional value

reyn2-8.qxd

6/6/2003

9:10 AM

Page 205

205

8. Holtzman Inkblot Technique with Children

TABLE 8.4. HIT Location Test–Retest Correlations Years correlated 1&2 1&3 1&4 1&5 1&6 2&3 2&4 2&5 2&6 3&4 3&5 3&6 4&5 4&6 5&6

Mexico _______________________________ I II III .27 .20 .25 .24 .50 .58 .49 .42 .23 .64 .56 .44 .60 .62 .63

.57 .49 .49 .46 .26 .70 .73 .64 .57 .75 .70 .63 .77 .73 .76

.66 .60 .58 .56 .51 .71 .72 .67 .52 .76 .74 .55 .85 .68 .74

United States ______________________________ I II III .28 .27 .26 .26 .33 .50 .54 .46 .49 .68 .70 .64 .71 .68 .79

.72 .62 .59 .51 .56 .77 .68 .58 .62 .78 .80 .77 .76 .80 .86

.76 .70 .69 .67 .59 .84 .81 .76 .75 .82 .80 .75 .86 .86 .85

Note. Table covers 6 years of repeated testing; Groups I, II, and III started in Year 1 at ages 6.7, 9.7, and 12.7, respectively. From Holtzman, Diaz-Guerrero, and Swartz (1975). Copyright 1975 by the Hogg Foundation for Mental Health. Reprinted by permission.

into the .60s and .70s—Rejection, Form Appropriateness, Shading, Pathognomic Verbalization, Barrier, and Penetration. Four variables—Space, Sex, Abstract, and Balance—generally occurred too infrequently in samples of children to yield data amenable to treatment by correlation methods. Several generalizations concerning the stability of inkblot variables among the children and adolescents tested can be drawn from these findings: 1. Test–retest stability increases generally with an increase in the age of children. Older adolescents tend to have the highest degree of stability, while children of any age show higher test–retest stability in the later years of testing than in the initial years. 2. Test–retest stability dropped off in a regular fashion with increasing size of interval between tests. 3. Test–retest stability is generally slightly higher for American children than for Mexicans, regardless of age group. This cross-cultural difference is particularly marked for Integration, Hostility, and Barrier. 4. Most of the HIT scores show a sufficiently high degree of stability across time, ranging from .40 to .80, to justify

their use as predictors of later behavior. (A successful example of such a study over a 9-year period is reported later in this chapter.) At the same time, the test–retest correlations are not so high as to suggest any kind of fixed traits that remain relatively invariant as children grow older.

VALIDITY OF THE HIT FOR CHILDREN AND ADOLESCENTS Several hundred studies have been published bearing on the relationships between scores on HIT variables and independent measures of personality. Although most have been carried out with adults, many have dealt specifically with children or adolescents. Extensive reviews have appeared recently elsewhere (Holtzman, 1981, 1985; Holtzman & Swartz, 1983). A comprehensive annotated bibliography containing all known references to the HIT through 1999 (Swartz et al., 1999) contains abstracts of these articles. Only representative highlights of these many findings as they pertain to use of the HIT with children and adolescents are provided here. Because factor analyses of intercorrelations among the 22 HIT variables have indicated that, with few excep-

reyn2-8.qxd

6/6/2003

9:10 AM

Page 206

206

II. PROJECTIVE METHODS

tions, these scores tend to cluster into meaningful factors, the findings with respect to validity are arranged according to these more general dimensions.

Factor 1: Perceptual Maturity and Integrated Ideational Activity Factor 1 is defined by four variables: Movement, Integration, Human, and Barrier. High scores on these four variables taken together are indicative of well-organized ideational activity, good imaginative capacity, and welldifferentiated ego boundaries. All four variables increase with age among children and are significantly higher among college graduates than average adults, indicating a strong component of cognitive ability and creativity (Swartz, Reinehr, & Holtzman, 1983). Studies have shown repeatedly that these variables are indicative of creative potential. They show significant relationships with reading comprehension among children, even after general intelligence has been held constant (Laird, Laosa, & Swartz, 1973). The energy level of movement ascribed to the percept, regardless of content, has some other interesting correlates. A high score on Movement is associated with perceived empathy in counselors, whereas low Movement is associated with the reverse (Mueller & Abeles, 1964). The degree of eye contact and smiling among psychiatric patients when interviewed is also related to high Movement scores (Lefcourt, Telegdi, Willows, & Buckspan, 1972). Movement is correlated with the discharge or inhibition of cognitive energy, according to Covan (1976). Increased perception of movement in inkblots follows experimental inhibition of cognitive responses, while discharge of cognitive processes in a series of free-association tasks leads to a sharp decrease in reported perception of movement in inkblots. Studies of dream deprivation, whether induced by drugs (Lerner, 1966) or prevented by interrupting rapid eye movements (Feldstein, 1973), results in higher Movement scores. These findings support Rorschach’s views on the fundamental similarity between movement and dreams because of the centrality of kinesthetic experience in both; the results are also consistent with Heinz Werner’s sensory tonic theory of perception. Movement has a particularly strong cog-

nitive component among young children. In the first year of testing for 6-year-olds in the longitudinal study, the complete battery of tests for the WISC was given. Factor analyses of intercorrelations among the various cognitive tests were done with Movement included as an extra variable. Among the U.S. children, Movement was clearly a major part of the first factor defined by the Verbal subtests from the WISC. Movement did not show up heavily loaded on a similar factor for the Mexicans until the 9-year-olds were analyzed. Similar results were found for the 12-year-old Mexican children. No complete analyses could be performed for the U.S. 9- and 12-year-olds, because they had not been given the complete WISC test battery. Movement deals with that component of verbal ability characterized by a lively, active imagination and the ability to project outward from one’s fantasies. In this sense, it deals particularly with the expressive, imaginative aspects of verbal ability, rather than with factual information, word meanings, and analytic problem solving. Human content also has some special meaning worthy of note. As one would expect from projective theory, a high score on Human suggests high social interest, whereas lack of any Human content indicates the opposite (Fernald & Linden, 1966). One of the most interesting of the symbolic content scores is Barrier, developed by Fisher and Cleveland (1958). The score is given for references to any protective covering, membrane, shell, or skin that might be symbolically related to the perception of body image boundaries. High Barrier is indicative of strong ego identity, whereas low Barrier suggests diffusion. High Barrier is related to being influential and independent in group processes (Cleveland & Morton, 1962), adjusting well to physical disablement (Fisher, 1963), being able to tolerate pain (Nichols & Tursky, 1967), and having a positive evaluation of one’s own body (Conquest, 1963). These findings are consistent with others showing low Barrier related to juvenile delinquency (Megargee, 1965).

Factor 3: Psychopathology of Thought Factor 3 consists of Pathognomic Verbalization, Anxiety, and Hostility. Dealing with unbridled fantasies, affective expressivity,

reyn2-8.qxd

6/6/2003

9:10 AM

Page 207

207

8. Holtzman Inkblot Technique with Children

and loose imagination, these three variables, frequently also associated with Movement, constitute an important cluster of scores indicative of psychopathology. Among children, moderately high scores on these three variables may be a good sign rather than a bad one. In most factor analyses of inkblot variables among children, this factor often proves to be highly correlated with Factor 1, indicating creativity and imaginative capacity rather than psychopathology. Even among children, however, high scores on these variables are indicative of future emotional disturbance, as demonstrated by a long-term follow-up study of the 6-yearolds tested in Austin as part of the cross-cultural study. Nine years after the initial testing of the first graders in Austin, 46 of them (23 girls, 23 boys) were located within the Austin schools and rated on personal adjustment by school personnel (Currie, Holtzman, & Swartz, 1974). These 46 subjects constituted more than half the group (n = 89) that completed the 6-year period of testing and were all from families that had continued to live in Austin, Texas, over a 10-year span, thus representing a particularly stable sample in relation to current population mobili-

ty. Seven of the children were judged to have serious problems of adjustment; 7 were judged to have noticeable areas of maladjustment; 18 were judged to be generally well adjusted but with some problems in relating to others; and 14 were judged to be well adjusted. The mean scores on Pathognomic Verbalization, Anxiety, and Hostility from HIT protocols 6 years earlier for the seven most disturbed children were two standard deviations higher than the means for children judged to be well adjusted. Low Form Appropriateness was also associated with emotional disturbance. Interestingly enough, from the many tests in the original battery, the only other one that correlated significantly with later emotional disturbance in these children was the Koppitz Scale of emotional indicators in Human Figure Drawings (see Table 8.5). Pathognomic Verbalization is the best single indicator of psychopathology. Among adults, the bizarre perception and autistic logic underlying high scores on this variable are characteristic of either schizophrenia or extreme artistic license in responding to inkblots. Highly creative artists do tend to get much higher scores than do average individuals on this variable (Holtzman,

TABLE 8.5. Mean Test Scores from the WISC, the Human Figure Drawing, and the HIT by Four Levels of Adjustment and Correlations between Test Variables and the Adjustment Index Means by adjustment level A(3) (n = 14)

B(2) (n = 18)

C(1) (n = 7)

D(0) (n = 7)

Total (n = 46)

Total SD

r

111.8

112.6

116.3

99.5

110.8

10.9

.21

HFD Koppitz indicators Goodenough–Harris

0.6 18.8

0.9 16.8

1.4 18.4

2.0 14.7

1.0 17.4

1.1 5.1

–.44** .21

HIT Form Appropriateness (FA) Movement (M) Pathog. Verbalization (V) Integration (I) Human (H) Anatomy (At) Anxiety (Ax) Hostility (Hs) Penetration (Pn)

35.0 14.9 6.8 1.0 12.8 4.1 5.6 6.2 2.8

35.3. 21.4 6.8 1.3 16.9 4.8 6.8 6.7 3.6

35.7 17.5 6.3 1.6 20.7 6.4 4.9 5.7 3.3

25.6 23.3 18.4 0.6 16.7 7.7 16.0 19.3 3.0

33.8 19.2 8.5 1.2 16.2 5.3 7.5 8.3 3.2

9.2 15.5 8.3 1.1 9.7 6.3 7.4 9.1 2.6

.28* –.15 –.38** .10 –.19 –.19 –.37** –.39** –.03

Test variable WISC IQ

Note. From Currie, Holtzman, and Swartz (1974). Copyright 1974 by Pergamon Press. Reprinted by permission. *Correlation significant beyond .05 level. **Correlation significant beyond .01 level.

reyn2-8.qxd

6/6/2003

9:10 AM

Page 208

208

II. PROJECTIVE METHODS

Swartz, & Thorpe, 1971), but the quality of the response is noticeably different. Table 8.6 gives names, abbreviations, definitions, and scoring weights for the various categories of Pathognomic Verbalization. Normal individuals tend to give Fabulations with notable affectivity, mildly Fabulized Combinations of otherwise acceptable percepts, or even occasional Queer Responses that are often described in a playful manner. Schizophrenics, on the oth-

er hand, manifest a loss of distance between themselves and the inkblots, often giving severely Fabulized Combinations, Contaminations, Queer Responses, or special kinds of Autistic Logic that show faulty, fantastic reasoning as a justification for the response. Embellishing a response with highly personal meaning by Self-Reference is particularly characteristic of psychotic thinking when manifested repeatedly (Swartz, 1969). A predominance of Absurd Responses is char-

TABLE 8.6. Names, Abbreviations, Brief Definitions, and Scoring Weights for Nine Categories of Pathognomic Verbalization (V) Fabulation (FB). A response in which there is associative elaboration having notable affective components (16A “A kind monster . . . eyes, and he looks real sweet”). FB is scored 1 when present. Fabulized Combination (FC). An impossible, fantastic combination of otherwise fairly acceptable parts, based largely on a spatial rather than a logical relationship (14B “Some kind of get-together. Two caterpillars talking to each other in a sociable mood, don’t seem to mind having the sheep around”). FC is scored 2, 3, or 4, depending on the judged severity of pathology evident. Queer Response (QR). A response in which the subject employs peculiar or eccentric language and syntax in verbalizing the response (14A “The male part of the back ... like the muscles in the biceps formed in the V-shaped web”). QR is scored 1, 2, or 3, depending on judged severity. Incoherence (IC). A response in which there is a complete breakdown of rational control (14A “A dental of hell . . . under misunderstanding”). IC is scored 4 when present. Autistic Logic (AL). The presence of faulty, fantastic reasoning given by the subject as justification for the response (43A “A carbon copy of a person . . . because he’s lying down on carbon paper”). AL is scored 1, 2, 3, or 4, depending on judged severity. Contamination (CT). A response in which two conflicting interpretations are fused into one, or when the same area simultaneously stands for two or more interdependent but logically separate concepts (11B “That looks like a stone stain . . . looks like a heart . . . well, a stained heart”). CT is scored 2, 3, or 4, depending on judged severity. Self-Reference (SR). A response in which the subject draws himself or herself into the percept, giving the response a personal meaning (29B “A person’s face. Looks like my face . . . maybe I lost it”). SR is scored 2, 3, or 4, depending on judged severity. Deterioration Color (DC). Loose, fantastic color associations having bizarre content are given with an air of reality (4B “The yellow is a virulent disease, the yellow plague . . . the kiss of death”). DC is scored 2, 3, or 4, depending on judged severity. Absurd Response (AB). A response is categorized as absurd when a subject assigns a form-definite concept to an area of an inkblot in which by no stretch of the imagination can the form be conceived of as appropriate, and the response is not an abstract one (36A “The Empire State Building . . . whole card”). AB is scored 3 when present.

0 No pathology present

1 FB

Scale Value 2 3 FC QR AL CT SR DC AB

4 IC

Note. The schematic diagram above shows the range of scoring weights for each of the nine V categories and the relationship of the FB and QR categories to the FC and IC categories, respectively.

reyn2-8.qxd

6/6/2003

9:10 AM

Page 209

8. Holtzman Inkblot Technique with Children

acteristic of mentally retarded individuals, while a predominance of Deteriorated Color associations is indicative of severe disintegration. Among young children, moderately high scores on Pathognomic Verbalization may simply indicate immature thought processes coupled with uncontrolled fantasies and loose imagination, rather than serious psychopathology. Although few of the cases in either the Mexican or the U.S. sample received scores on this variable so high as to indicate serious psychopathology, the presence of some Pathognomic Verbalization among young children may indeed be taken as a good sign, provided that the qualitative nature of the disordered thinking reveals primarily Fabulized Combinations and invalid Integrations rather than bizarre perceptions. In an analysis of longitudinal data on the three groups of U.S. children from the Mexico–United States cross-cultural investigation discussed earlier, a highly significant curvilinear developmental trend was apparent for Pathognomic Verbalization (Swartz, 1969). Across the 11 years from 6 to 17, the lowest mean score on this variable occurred in 12-year-old children, with rising means both up and down the developmental order. When groups of children (n = 180)—6, 9, and 12 years of age, respectively—were matched for sex and total Pathognomic Verbalization score, using HIT data from initial testing sessions only, it was found that with increasing age there was a significant increase in the number of children giving Fabulation responses and significant decreases in the numbers of children giving Autistic Logic, Contamination, and Absurd Responses. The numbers of children giving Fabulized Combination, Queer Responses, and Deteriorated Color responses remained quite steady across the 6-year age span (with about half in all three age groups giving Fabulized Combination responses, more than half giving Queer Responses, and only about 8% giving Deteriorated Color responses). Almost none produced verbalizations falling into the Incoherence or SelfReference categories. The importance of Pathognomic Verbalization in psychodiagnosis can best be illustrated by an individual case drawn from our files of schoolchildren in the Austin longitu-

209

dinal study. While being tested with the HIT in the third year of repeated study, a young, apparently normal teenage boy shifted abruptly from normal responses one-third of the way through the test, thereafter giving responses heavily loaded with Pathognomic Verbalization. It is important to note that his responses to the first 12 inkblots and in the previous testing sessions over the past 2 years were generally normal, although under a high degree of self-control. Even in the remainder of this testing session, he continued to maintain an outward appearance of control, being polite, cooperative, and attentive. Although we were quite concerned about the sudden deterioration in the quality of his responses, nothing was said to his parents or authorities because of the research nature of the data collection. Some months later, we learned to our dismay that the boy had killed his father and was hospitalized for treatment as a patient with schizophrenia. The incipient psychosis was not apparent in his general behavior or on other tests, although it was clearly revealed by his Pathognomic Verbalization score on the HIT. More recently, Leichsenring (1991) successfully used HIT deviant verbalizations to classify acute schizophrenics, chronic schizophrenics, and borderline patients and, in another comparison, to differentiate between neurotic and normal individuals. Signs of anxiety or hostility in the fantasy content form the basis for the Anxiety or Hostility score. Moderate-level scores on both of these symbolic content scales are normal, particularly in young children, but very high scores should be interpreted as having likely clinical significance. Zero or low positive correlations can be expected between these two variables and anxiety or hostility scales in self-report inventories. The most important evidence of their validity comes from experimental studies. Subjects who rapidly acquire the conditioned eyelid response have higher Anxiety scores than do those who do not condition easily (Herron, 1965). Individuals with high Anxiety are less tolerant of pain (Nichols & Tursky, 1967). Individuals who show a marked increase in Hostility score after a frustrating situation are those who also show a predisposition to hostility as measured by Factor I of the Buss–Durkee Inven-

reyn2-8.qxd

6/6/2003

210

9:10 AM

Page 210

II. PROJECTIVE METHODS

tory (von Rosenstiel, 1973). Both Anxiety and Hostility scores are directly related to observed interpersonal distance characteristic of an individual in an experimental setting; the higher the inkblot scores, the greater the distance (Greenberg, Aronow, & Rauchway, 1977). These findings all are consistent with the theoretical conception of these symbolic content scores. In a series of studies, Kamen (1969, 1970, 1971) found several HIT variables related to State anxiety (but not Trait anxiety) as measured by the State–Trait Anxiety Inventory (STAI). A later study by Iacino and Cook (1974), however, found HIT Anxiety to be correlated positively with the STAI Trait scale. Auerbach and Edinger (1977) found that only HIT Barrier correlated significantly (negatively) with STAI Trait anxiety; no HIT variables were found to be related to STAI State anxiety. Mittenberg and Petersen (1984), using vasomotor biofeedback, supported the validity of the HIT as a measure of anxiety but failed to characterize this anxiety as either state or trait in nature. In a study of the aggression-reducing effects of d-amphetamine, Amery, Minichiello, and Brown (1984) studied 10 8- to 11year-old boys diagnosed with attention deficit disorder with hyperactivity in a double-blind placebo-counterbalanced trial of dextroamphetamine (DAM). It was predicted that aggressive behavior and aggressive/ impulsive attitudes and fantasies would be lower in the DAM phase than in the placebo phase. Findings in the predicted direction were significant on three of the five measures used, including HIT Hostility. Moreover, in a study of hostility employing 130 Caucasian, 109 African American, and 34 Hispanic forensic patients, Reinehr, Swartz, and Dudley (1984) found that ethnic group differences may exist on both objective and HIT measures of hostility. Although the meaning of Anxiety and Hostility scores on the HIT undoubtedly is complex, it seems clear that very high scores, even among children, have sufficient validity to justify clinical interpretation for individuals. While cautioning that the HIT and other projective techniques are not direct behavioral measurements but instead measure underlying processes based on an individual’s perception, Fehr (1983) sees the

HIT as superior to the Rorschach in measuring both anxiety and hostility.

Factor 2: Perceptual Sensitivity Factor 2 consists of Color, Shading, and Form Definiteness (reversed). The clustering together of these three variables is inevitable. As with scoring systems for the Rorschach, the greater the predominance of color or shading over form in a response, the higher the score. Among younger children, significant negative loadings on this factor also appear for Animal, suggesting that many children tend to use color and shading as a determinant only when they cannot find a familiar animal form. The positive pole on this factor indicates overreactivity to the stimulus determinants; the negative pole shows primary concern for form alone as a response determinant. Among normal subjects, a high Color score has been found to be related to impulsivity (Holtzman, 1950) and to increased expression of affect (Mayfield, 1968). In her clinical use of the HIT, Hill (1972) recommends paying attention to the quality of the Color responses, particularly those given to inkblots having a high stimulus strength for Color, in making interpretations about the lability of affect. There is little experimental evidence bearing on the validity of Shading or Form Definiteness, the other two variables that measure degree of perceptual sensitivity. Nor is there much information from correlational studies with other personality measures that would indicate the independent meaning of these variables for assessment purposes. To be sure, there is a consensus among Rorschach clinicians concerning the use of these scores for personality assessment, but the scientific evidence is too tenuous at this time to justify any confident interpretations, particularly among children.

Other Factors The remaining three factors are less important and vary somewhat in their patterning from one population to another. Location and Form Appropriateness generally appear as defining variables for Factor 4. A high score on Location results when an individual uses smaller areas of the inkblot while

reyn2-8.qxd

6/6/2003

9:10 AM

Page 211

8. Holtzman Inkblot Technique with Children

ignoring the rest. This perceptual style makes it easier to find percepts that have good form. The combination of low score on Location with high score on Form Appropriateness is less common and indicates a high level of perceptual maturity and organization, particularly when accompanied by high scores on Integration. Reaction Time and Rejection tend to be associated in a single factor, because both measure the extent of inhibition or outright perceptual inability. Both variables must be taken into account with other inkblot scores rather than interpreted alone. Three scores—Sex, Anatomy, and Penetration—deal with bodily preoccupation and are occasionally clustered together in one factor for this reason. Blatant Sex responses are relatively rare but are significant when they do appear, especially among children. Very high Anatomy scores are also quite unusual and significant. High Anatomy scores have been found to be closely associated with a high degree of somatic preoccupation among hospitalized patients, confirming the theoretical interpretation of Anatomy (Endicott & Jortner, 1967). Penetration frequently loads also on Factor 3, Psychopathology of Thought, suggesting that high scores on Penetration should be generally interpreted as pathological.

Use of Scores for Differential Diagnosis Closely related to the clinical validity of individual variables within the HIT is the use of patterns of scores for differential diagnosis. The original standardization data on 15 different populations are presented in percentile norms for eight major reference groups ranging from 5-year-olds to superior adults and including psychiatric patients as well as mentally retarded individuals. Chronic schizophrenics differ from normal reference groups on almost all of the standard HIT variables. Conners (1965) reported a number of highly significant differences between emotionally disturbed children seen in an outpatient clinic and normal controls of the same age and background. In Conners’s study, disturbed children got higher scores on Rejection and Anatomy and lower scores on all other variables except Pathognomic Verbalization, Sex, Abstract, Hostility, Penetra-

211

tion, and Balance. Using HIT factor scores, Conners found that neurotic children appeared to be more differentiated in response and more inhibited than did hyperkinetic children. At the same time, it should be noted that neurotic children received higher scores on Form Appropriateness and Location than did children with conduct disorders. Hill’s (1972) handbook on clinical application of the HIT has provided detailed suggestions on how best to interpret HIT scores, as well as the qualitative aspects of content with respect to cognitive functioning, affective functioning, and self-identity. In 1976, Aronow and Reznikoff compared Rorschach and HIT content scores and concluded that “the HIT is clearly the technique of choice for most research purposes” (p. 315). More recently, Pokhriyal and Ahmad (1988)—in a study of the HIT response patterns of acute schizophrenics, endogenous depressives, and normal subjects— found that HIT scores differentiated significantly between the two clinical groups and between the normal subjects and both of the psychotic groups.

CROSS-CULTURAL USE OF THE HIT Use of the HIT in research studies of differences in personality development among children of different cultures is particularly appropriate because of the more or less universal nature of the technique. The method has been translated into a number of languages and is relatively culture-free. The technique has been used successfully for both adults and children in cultures as widely varied as primitive groups in New Guinea, Aleutian Eskimos, peasant children in Latin America, and children from modern, industrialized societies. Our own work has concentrated on factors related to personality and cognitive development among children in Mexico and the United States (Holtzman et al., (1975). Of all the measures used in the cross-cultural longitudinal study, the HIT yielded the most striking differences between Mexican and U.S. children. One finding is of particular interest, because it sheds considerable light on the possible use of the HIT for measuring an important coping style.

reyn2-8.qxd

6/6/2003

212

9:10 AM

Page 212

II. PROJECTIVE METHODS

The most significant differences between Mexican and U.S. children, regardless of age, sex, or socioeconomic status, were found for seven HIT scores—Reaction Time, Pathognomic Verbalization, Location, Movement, Integration, Anxiety, and Hostility. Mexicans had a slower response time; showed less pathology, anxiety, hostility, or movement in their fantasy expressions; tended to use more small details within the blots for their responses; and showed lower ability to integrate the parts into a meaningful whole than did the American children. These differences tended to narrow with increasing age. Most of the differences between the Mexican and U.S. children on the HIT can be understood better in terms of coping style than of any other concept. The U.S. children produced faster Reaction Time, used larger portions of the inkblots in giving responses, gave more definite form to responses, and still were able to integrate more parts of the inkblots while doing so. In addition, they incorporated other stimulus properties of the inkblots, such as Color and Shading, into their responses more often than did the Mexican children, and they elaborated their responses by ascribing more movement to their percepts. In attempting to deal with all aspects of the inkblots in such an active manner, however, they failed more often than the Mexican children; that is, the Mexican children gave responses with better form and less often produced responses that showed deviant thinking and anxious and hostile content. In general, U.S. children tried to deal with the testing situation in much more active manner than did the Mexican children, even when unable to do so successfully. The work of Tamm (Haroz, 1967) in the American School in Mexico City allows for a deeper insight into the meaning of these cross-cultural findings with U.S. and Mexican schoolchildren. Tamm designed a study involving bilingual Mexican and U.S. children attending the same school. Thirty children in the first, fourth, and seventh grades were tested at 6 years, 8 months; 9 years, 8 months; and 12 years, 8 months of age, respectively, to provide precise parallels to the design employed in the larger cross-cultural study between Mexico and the United States. One-half of the children were native

Mexicans for whom Spanish was the primary language. These children generally came from upper-class Mexican families in which there was a strong desire on the part of the parents for their children to obtain a U.S.style education. The remainder of the children were Americans whose fathers were businessmen or government representatives in Mexico City. The U.S. families wanted their children to develop bilingual/bicultural skills and attitudes. The curriculum in the American School was taught half in English and half in Spanish. Tamm administered the HIT and all the subtests of the WISC to each of the 90 schoolchildren 2 years in a row. The children’s test performance was analyzed in a three-way analysis-of-variance design by culture, age group, and year of testing. Of the WISC subtests, only Digit Span proved significant across cultures, the Mexican children doing slightly better than the U.S. children. The usual developmental differences were clearly apparent in both groups. On the HIT, however, marked differences were found between the Mexican and U.S. children, differences that in every respect were essentially the same as the major differences found for HIT scores in the larger cross-cultural study. Mexican children used much more small detail and gave less Color, less Movement, less Pathognomic Verbalization, less Human content, less Anxiety, and less Hostility than did the U.S. children. The lack of any notable differences between the Mexican and U.S. children on the intelligence tests in Tamm’s study, regardless of the length of time the children had spent in the American School or the children’s ages, provides convincing evidence that the combination of home environment and schooling is important in the development of these mental abilities. At the same time, the dramatic differences in personality and perceptual style reflected in the HIT—differences identical to those obtained when U.S. children in Austin were compared with Mexican children in Mexico City—indicate that fundamental aspects of the U.S. and Mexican personality or “national character” remain intact, in spite of common schooling and other forces within the immediate environment of the children that would tend to produce convergence of the two cultures. The sociocultural premises un-

reyn2-8.qxd

6/6/2003

9:10 AM

Page 213

8. Holtzman Inkblot Technique with Children

derlying the U.S. and Mexican societies, and the basically different styles of coping with the challenges of life in the two cultures, provide a key to the interpretation of these results. U.S. children tend to be more actively independent and to struggle for a mastery of problems and challenges in their environment, whereas Mexican children are more passively obedient and adapt to stresses in the environment instead of trying to change them.

CONCLUSION The value of the HIT for clinical and research use with children depends on the reliability and validity of HIT scores for use in personality assessment and psychodiagnosis, as well as on the efficiency and ease of administration, scoring, and interpretation. Although the HIT is still a relatively young technique, the evidence to date seems to indicate that it has answered most if not all of the criticisms of the Rorschach. The availability of parallel forms and standardized variables, without sacrificing the qualitatively rich projective content of the Rorschach, provides clinicians and researchers alike with a powerful tool for the assessment of personality in children as well as adults, and explains the growing acceptance of the HIT throughout the world. Experts in assessment have been quick to point out its advantages over the Rorschach (Anastasi, 1982; Fehr, 1983; Kleinmuntz, 1982; Peterson, 1978). The HIT is more demanding of the clinician than the Rorschach; however, the time and effort involved in administering, scoring, and interpreting the HIT need not be any greater than for the Rorschach once the technique has been mastered. Those who have learned the method well have been enthusiastic about its value for both clinical and research purposes. Some find the HIT difficult because it has 45 inkblots rather than 10, or because it has only one response per card rather than as many as the child wishes to give. Yet these are the very features that produce superior psychometric qualities, rendering the HIT more suitable for rigorous scientific validity as well as for implementation by modern computer technology. As with any major test for the assessment of

213

personality, the final verdict on the HIT will be reached only after many years of experimental and clinical work with children and adults.

NOTE 1. Materials for the HIT can be obtained from the Psychological Corporation, 19500 Bulverde Road, San Antonio, TX 78259-3701. The Guide for Administration and Scoring is an offprint of the sections on administration and scoring from Holtzman and colleagues (1961b). Sets of 35 mm slides are used with the group method of administration. A research guide and annotated bibliography of the HIT (Swartz et al., 1999) and other monographs in addition to Holtzman and colleagues are also available.

REFERENCES Amery, B., Minichiello, M. D., & Brown, G. L. (1984). Aggression in hyperactive boys: Response to d-amphetamine. Journal of the American Academy of Child Psychiatry, 23, 291–294. Anastasi, A. (1982). Psychological testing (5th ed.). New York: Macmillan. Aronow, E., & Reznikoff, M. (1976). Rorschach content interpretation. New York: Grune & Stratton. Auerbach, S. M., & Edinger, J. D. (1977). The effects of surgery-induced stress on anxiety as measured by the Holtzman Inkblot Technique. Journal of Personality Assessment, 41, 19–24. Bock, D. R., Haggard, E. A., Holtzman, W. H., Beck, A. G., & Beck, S. J. (1963). A comprehensive psychometric study of the Rorschach and Holtzman inkblot techniques. Chapel Hill: University of North Carolina, Psychometric Laboratory. Buros, O. K. (Ed.). (1949). The third mental measurements yearbook. New Brunswick, NJ: Rutgers University Press. Cleveland, S. E., & Morton, R. B. (1962). Group behavior and body image: A follow-up study. Human Relations, 15, 77–85. Conners, C. K. (1965). Effects of brief psychotherapy, drugs, and type of disturbance on Holtzman Inkblot scores in children. Proceedings of the 73rd Annual Convention of the American Psychological Association, 1, 201–202. Conquest, R. A. (1963). An investigation of body image variables in patients with the diagnosis of schizophrenic reaction. Unpublished doctoral dissertation, Case Western Reserve University. Covan, F. L. (1976). The perception of movement in inkblots following cognitive inhibition. Unpublished doctoral dissertation, Yeshiva University. Currie, S. F., Holtzman, W. H., & Swartz, J. D. (1974). Early indicators of personality traits viewed

reyn2-8.qxd

6/6/2003

214

9:10 AM

Page 214

II. PROJECTIVE METHODS

retrospectively. Journal of School Psychology, 12, 51–59. Endicott, N. A., & Jortner, S. (1967). Correlates of somatic concern derived from psychological tests. Journal of Nervous and Mental Disease, 144, 133–138. Fehr, L. A. (1983). Introduction to personality. New York: Macmillan. Feldstein, S. (1973). REM deprivation: The effects of inkblot perception and fantasy processes. Unpublished doctoral dissertation, City University of New York. Fernald, P. S., & Linden, J. D. (1966). The human content response in the Holtzman Inkblot Technique. Journal of Projective Techniques and Personality Assessment, 30, 441–446. Fisher, S. (1963). A further appraisal of the body boundary concept. Journal of Consulting Psychology, 27, 62–74. Fisher, S., & Cleveland, S. E. (1958). Body image and personality. Princeton, NJ: Van Nostrand. Gorham, D. R. (1967). Computer use in psychological testing. In G. Gillespie (Ed.), Memorias del XIth Congreso, Interamericano de Psicologia (Vol. 9, pp. 1–7). Mexico City: Universidad Nacional Autonoma de Mexico. Greenberg, E., Aronow, E., & Rauchway, A. (1977). Inkblot content and interpersonal distance. Journal of Clinical Psychology, 33, 882–887. Haroz, M. M. (1967). El Holtzman Inkblot Test, el Wechsler Intelligence Scale para Children y otros tests en el estudio psicológico transcultural de niños de habla Española e Inglesa residentes en Mexico. Unpublished doctoral dissertation, Universidad Nacional Autonoma de Mexico, Mexico City. Herron, E. W. (1963). Psychometric characteristics of a thirty-item version of the group method of the Holtzman Inkblot Technique. Journal of Clinical Psychology, 19, 450–453. Herron, E. W. (1965). Personality factors associated with the acquisition of the conditioned eyelid response. Journal of Personality and Social Psychology, 2, 775–777. Hill, E. F. (1972). The Holtzman Inkblot Technique: A handbook for clinical application. San Francisco: Jossey-Bass. Holtzman, W. H. (1950). The Rorschach test in the assessment of the normal superior adult. Unpublished doctoral dissertation, Stanford University. Holtzman, W. H. (1975). New developments in the HIT. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3, pp. 243–274). San Francisco: Jossey-Bass. Holtzman, W. H. (1981). Holtzman Inkblot Technique (HIT). In A. I. Rabin (Ed.), Assessment with projective techniques: A concise introduction (pp. 47–83). New York: Springer. Holtzman, W. H. (1985). Clinical applications in personality assessment and psychodiagnosis. In J. J. Sanchez-Sosa (Ed.), Health and clinical psychology (pp. 3–19). Amsterdam: North Holland. Holtzman, W. H. (1986). The Holtzman Inkblot Technique with children and adolescents. In A. I. Rabin (Ed.), Projective techniques for adolescents and children (pp. 168–192). New York: Springer.

Holtzman, W. H. (1988). Beyond the Rorschach. Journal of Personality Assessment, 52, 578–609. Holtzman, W. H., Diaz-Guerrero, R., & Swartz, J. D. (1975). Personality development in two cultures: A cross-cultural longitudinal study of school children in Mexico and the United States. Austin: University of Texas Press. Holtzman, W. H., Moseley, E. C., Reinehr, R. C., & Abbott, E. (1963). Comparison of the group method and the standard individual version of the Holtzman Inkblot Technique. Journal of Clinical Psychology, 19, 441–449. Holtzman, W. H., & Swartz, J. D. (1983). The Holtzman Inkblot Technique: A review of 25 years of research. Zeitschrift fur Differentielle und Diagnostische Psychologie, 4, 241–259. Holtzman, W. H., Swartz, J. D., & Thorpe, J. S. (1971). Artists, architects, and engineers: Three contrasting modes of visual experience and their psychological correlates. Journal of Personality, 39, 432–449. Holtzman, W. H., Thorpe, J. S., Swartz, J. D., & Herron, E. W. (1961a). Administration and scoring guide. New York: Psychological Corporation. Holtzman, W. H., Thorpe, J. S., Swartz, J. D., & Herron, E. W. (1961b). Inkblot perception and personality: Holtzman Inkblot Technique. Austin: University of Texas Press. Iacino, L. W., & Cook, P. E. (1974). Threat of shock, state anxiety, and the HIT. Journal of Personality Assessment, 38, 450–458. Kamen, G. B. (1969). Effects of a stress-producing film on the test performance of adults. Journal of Projective Techniques and Personality Assessment, 33, 281–285. Kamen, G. B. (1970). The effects of a stress-producing film on the test performance of adults. Unpublished doctoral dissertation, University of Missouri. Kamen, G. B. (1971). A second look at the effects of stress-producing film on adult test performance. Journal of Clinical Psychology, 27, 465–467. Kleinmuntz, B. (1982). Personality and psychological assessment. New York: St. Martin’s Press. Laird, D. R., Laosa, L. M., & Swartz, J. D. (1973). Inkblot perception and reading achievement in children: A developmental analysis. British Journal of Projective Psychology and Personality Study, 18, 25–31. Lefcourt, H. M., Telegdi, M. S., Willows, D., & Buckspan, B. (1972). Eye contact and the human movement response. Journal of Social Psychology, 88, 303–304. Leichsenring, F. (1991). Discriminating schizophrenics from borderline patients: Study with the Holtzman Inkblot Technique. Psychopathology, 24, 225–231. Lerner, B. (1966). Rorschach movement and dreams: A validation study using drug-induced dream deprivation. Journal of Abnormal Psychology, 71, 75–86. Mayfield, D. G. (1968). Holtzman Inkblot Technique in acute experimental alcohol intoxication. Journal of Projective Techniques and Personality Assessment, 32, 491–494. Megargee, E. I. (1965). The relation between barrier

reyn2-8.qxd

6/6/2003

9:10 AM

Page 215

8. Holtzman Inkblot Technique with Children scores and aggressive behavior. Journal of Abnormal Psychology, 70, 307–311. Mittenberg, W., & Petersen, J. D. (1984). Validation of the Holtzman anxiety scale by vasomotor biofeedback. Journal of Personality Assessment, 48, 360–364. Morgan, A. B. (1968). Some age norms obtained for the Holtzman Inkblot Technique administered in a clinical setting. Journal of Projective Techniques and Personality Assessment, 32, 165–172. Mueller, W. J., & Abeles, N. (1964). The components of empathy and their relationship to the projection of human movement responses. Journal of Projective Techniques and Personality Assessment, 28, 322–330. Mullen, J. M., Reinehr, R. C., & Swartz, J. D. (1983). Holtzman Inkblot Technique scores of delinquent adolescents: A replication and extension. Journal of Personality Assessment, 47, 158–160. Nichols, D. C., & Tursky, B. (1967). Body image, anxiety, and tolerance for experimental pain. Psychosomatic Medicine, 29, 103–110. Peterson, R. A. (1978). Holtzman Inkblot Technique. In 0. K. Buros (Ed.), The eighth mental measurements yearbook (pp. 947–849). Highland Park, NJ: Gryphon Press. Pokhriyal, R., & Ahmad, H. (1988). Response patterns of acute schizophrenics and endogenous depressives on the Holtzman Inkblot Technique. Journal of Personality and Clinical Studies, 4, 205–207. Reinehr, R. C., Swartz, J. D., & Dudley, H. K. (1984). Ethnic differences in the measurement of hostility in forensic patients. Revista Interamericana de Psicologia, 18, 53–64. Sundberg, N. D. (1962). The Rorschach Americanized. Contemporary Psychology, 7, 250–252. Swartz, J. D. (1969). Pathognomic verbalizations in

215

normals, psychotics, and mental retardates. Unpublished doctoral dissertation, University of Texas. Swartz, J. D. (1992). The HIT and the HIT 25: Comments and clarifications. Journal of Personality Assessment, 58, 432–433. Swartz, J. D., & Holtzman, W. H. (1963). Group method of administration for the Holtzman Inkblot Technique. Journal of Clinical Psychology, 19, 433–441. Swartz, J. D., & Reinehr, R. C. (1983). A quick version of administration for the group Holtzman Inkblot Technique. Perceptual and Motor Skills, 56, 813–814. Swartz, J. D., Reinehr, R. C., & Holtzman, W. H. (1983). Personality development through the lifespan: Assessment by means of the Holtzman Inkblot Technique. In C. D. Spielberger & J. N. Butcher (Eds.), Advances in personality assessment (Vol. 3, pp. 35–51). Hillsdale, NJ: Erlbaum. Swartz, J. D., Reinehr, R. C., & Holtzman, W. H. (1999). Holtzman Inkblot Technique, Research Guide and Bibliography. Austin, TX: Hogg Foundation for Mental Health. Swartz, J. D., Witzke, D. B., & Megargee, E. I. (1970). Normative item statistics for the group form of the Holtzman Inkblot Technique. Perceptual and Motor Skills, 31, 319–329. Vincent, K. R. (1982). The fully automated Holtzman interpretation. In K. Herman & R. M. Samuels (Eds.), Computers: An extension of the clinician’s mind, a reference book (pp. 123–125). Norwood, NJ: Ablex. von Rosenstiel, L. (1973). Increase in hostility responses in the HIT after frustration. Journal of Personality Assessment, 37, 22–24. Zubin, J. (1954). Failures of the Rorschach technique. Journal of Projective Techniques, 18, 303–315.

reyn2-8.qxd

6/6/2003

9:10 AM

Page 216

This page intentionally left blank

reyn2-9.qxd

6/6/2003

9:13 AM

Page 217

PART III

INTERVIEWING AND OBSERVATIONS

reyn2-9.qxd

6/6/2003

9:13 AM

Page 218

This page intentionally left blank

reyn2-9.qxd

6/6/2003

9:13 AM

Page 219

9 Using the Clinical Interview to Assess Children’s Interpersonal Reasoning and Emotional Understanding

JANET A. WELSH KAREN L. BIERMAN

This chapter describes the purpose and techniques involved in open-ended clinical interviews with children designed to assess their interpersonal reasoning, emotional understanding, and thoughts and feelings about themselves and others. Although few studies have evaluated the reliability or validity of such interviews, they remain a core component of psychological assessment of children for a number of reasons (Greenspan, 1981; Rutter & Graham, 1968). First, although parents, teachers, and peers can provide valuable information regarding a child’s behavior and social adaptation, only children themselves are able to tell about their thoughts, feelings, and subjective experiences. Even young children can often successfully communicate thoughts and feelings that are unavailable through other sources; therefore, children provide unique information about their feelings and thoughts that may or may not correspond well with the perspectives of other informants. Second, research suggests that the quality of children’s socialemotional reasoning and their feelings about themselves and their relationships may have important implications for both case conceptualization (e.g., understanding the etiology or mechanisms maintaining a child’s difficul-

ties) and treatment (e.g., selecting developmentally appropriate intervention strategies and goals). Significant developmental changes occur in emotional understanding and interpersonal reasoning throughout early childhood, middle childhood, and adolescence. Delays or deficits in social cognition and emotional understanding, best understood relative to developmental norms, are often associated with significant social and behavioral difficulties (Harter, 1983). A solid understanding of normative developmental lines in these areas provides an important foundation for identifying and interpreting delays or deficits that may be contributing to clinical problems. Clinical interviewing is, of course, only one component of a comprehensive assessment of children’s social-emotional adaptation. Although child interviews are valuable because of the unique information they provide, they also have inherent limitations. There may be little overlap between children’s self-reports of their psychosocial adaptation and the perspectives of parents, teachers, or peers. For example, socially rejected and aggressive children often overestimate their popularity with peers (Hughes, Cavell, & Grossman, 1997), whereas de219

reyn2-9.qxd

6/6/2003

220

9:13 AM

Page 220

III. INTERVIEWING AND OBSERVATIONS

pressed children may underestimate their peer acceptance (Rudolph, Hammen, & Burge, 1995). For this reason, it would be inappropriate to draw conclusions about a child’s peer relations or other aspects of interpersonal adaptation based solely on selfreport. Children are more likely to report feelings of distress while parents focus more on behavioral problems; these could be regarded as distinct (but not contradictory) aspects of the same problem. In addition, it is important to remember that the clinical interview takes place in an artificial context and is often conducted by an unfamiliar adult. Hence, the characteristics of the child’s natural environment (e.g., home, school, and neighborhood) that may be contributing to his or her adjustment difficulties are missing, and the child’s behavior in the clinic setting is often not representative of the child’s behavior at home or school. Some children may react to the unfamiliar clinic setting with anxiety or excitability, or show more organized and compliant behavior in the structured and supportive clinical setting than they do in naturalistic home or school settings. Hence, comprehensive assessment should involve obtaining information from parents and teachers as well as children. When possible, it is also desirable to obtain peer reports and observe the child directly in the setting of concern. This chapter is organized into two major sections. First, we present an overview of normative developmental trends in social cognition, including children’s perceptions of themselves and others, children’s conceptualizations of friendship, and the development of emotional understanding. Then, we discuss general strategies for interviewing children, along with some specific suggestions for assessing particular aspects of social cognitive functioning. We also review the use of open-ended, semistructured, and standardized measures to assess various aspects of emotional understanding and interpersonal reasoning.

INTERPERSONAL REASONING AND EMOTIONAL UNDERSTANDING Interpersonal reasoning refers to the ideas, beliefs, and hypotheses children have about people around them, and about their own

social relationships and those of others. Emotional understanding involves the child’s ability to recognize and label his or her own emotions and those of others and to make causal interpretations of emotions based upon contextual cues. Both are associated with children’s social-emotional functioning. They are related to the goals that children select in their relationships with others, the strategies they use to achieve these goals, and their attributions and evaluations of their interpersonal experiences. Two aspects of interpersonal reasoning and emotional understanding warrant exploration in clinical interviews with children: (1) the developmental sophistication of the child’s reasoning, and (2) the affective valence of the child’s feelings about him- or herself and his or her relationships. There are a number of reasons for clinicians to be interested in assessing the developmental sophistication of a child’s interpersonal reasoning and emotional understanding. As mentioned earlier, developmental research suggests that children with deficits or delays in areas of social-emotional reasoning often experience difficulties with peers, poor school adjustment, and behavioral problems (Crick & Dodge, 1994). Conversely, children who are skilled at decoding the emotional states of others and who show more complex social perspective taking are rated by teachers and peers as more socially competent than those who are poor decoders and more egocentric in their social perceptions (Denham, 1986; Gamer, 1996; Manstead, 1995; Selman & Schultz, 1990; Spence, 1988). Second, the skills that children bring to their interpretations of relationships and interpersonal events can affect the ways in which they interpret and cope with specific life stressors such as parental divorce or transition to school. That is, children’s implicit personality theories and causal schemas affect the manner in which they interpret interpersonal events which, in turn, affects their reactions to those events and their subsequent attitudes and behaviors (Bierman & Furman, 1981; Harter & Marold, 1992).

Developmental Trends in Interpersonal Reasoning and Emotional Understanding Throughout development, children gradually accrue knowledge about themselves and

reyn2-9.qxd

6/6/2003

9:13 AM

Page 221

9. Using the Clinical Interview

other people, experience an ever-expanding array of social situations and relationships, and form hypotheses and expectations about their own behavior and that of others. In addition, particular developmental transition periods mark qualitative changes in children’s social cognitive capacities, including their ability to think about, organize, and integrate social information. As children’s cognitive complexity and flexibility increase, their conceptualizations of themselves and others, their ideas about friendship, and their emotional understanding become more sophisticated. Early Childhood Although toddlers and even infants have some rudimentary conceptualizations of themselves and others, clinical assessments of social-emotional functioning at this age rely primarily on observation and parent report rather than on the child’s ability to describe his or her thoughts and feelings. By age 4 or 5, however, many children have attained sufficient language proficiency to respond to verbal interview techniques, provided the interviewer structures the session in a developmentally appropriate manner. Preschool children typically conceptualize themselves and others in terms of concrete physical attributes and observable behaviors (Livelsley & Bromley, 1973; Watson & Fischer, 1980). For example, preschoolers accurately define themselves as boys or girls, and may describe themselves and others by size, hair color, and familiar behaviors, such as “drives a fire engine,” “takes care of sick people,” or “yells a lot.” As a result of their concrete basis and reliance on observable clues, the social conceptions of preschool children are typically rigid and inflexible. For example, preschool children may insist that mothers who fix cars will become fathers (Moore, Cooper, & Birckhard, 1977) or that girls who wear pants will become boys (Watson & Fischer, 1980). (This finding explains why many preschool children develop strong desires to dress in a particular way.) Preschoolers find it difficult to integrate multiple pieces of information simultaneously and cannot understand multifaceted personality characteristics or relationships. For example, young children tend to believe that people are “good” or “bad,”

221

and that good people cannot do bad actions, and vice versa (Gollin, 1958; Saltz & Medow, 1971). For example, when Saltz and Medow (1971) told preschool children that a good baseball player (one who caught well and hit many home runs) was a liar, these young children denied that he could still be a good baseball player, and even denied that he could catch well or hit home runs. These conceptual limitations make it difficult for young children to understand the complex motivations involved in actions such as divorce. They can also contribute to the difficulties many young children have in discussing or reconciling problem behaviors (“bad” behaviors) with a positive self-image (Harter, 1977). Young children’s conceptualizations of friendship are similarly straightforward and simplistic. Because the social interactions of preschoolers typically center around play, young children’s friends are those peers whom they see often and find fun to play with. Preschool friendships are less stable than those of older children, and more often marked by conflict and instrumental physical aggression (Hartup, 1983). By the age of 3, children begin to identify and describe basic emotions in themselves and others. Preschoolers can readily distinguish between happy and sad but may or may not be able to differentiate sadness from anger and fear. Understanding of others’ emotions lags somewhat behind understanding one’s own feelings, and young children rely heavily on facial expression and situational cues to infer the emotional state of another (Felleman, 1983; Shantz, 1983). For example, when asked to describe their emotions, preschool children often refer to particular situations or bodily reactions (e.g., happiness means “at a party” or “a smile”) (Carroll & Steward, 1984). The ability to self-regulate emotion and inhibit impulsivity becomes a critical task of socialemotional adaptation during the preschool years, and young children who fail to develop this capacity face a growing risk for peer rejection (Maszk, Eisenberg, & Guthrie, 1999). Because language development in general, and the development of affective vocabulary in particular, appears to play a central role in mediating social behavior, children with language delays often display impulsive and socially immature behavior

reyn2-9.qxd

6/6/2003

9:13 AM

222

Page 222

III. INTERVIEWING AND OBSERVATIONS

(Greenberg & Kusche, 1993). In addition, like their interpersonal concepts, preschool children’s emotional concepts are unidimensional. They typically deny that one can have two opposing feelings at the same time (e.g., can be mad at someone and still love him or her). This conceptual characteristic can contribute to emotional volatility for the young child and make it difficult for the child to come to terms with seemingly inconsistent feelings and actions (e.g., if daddy loved me, why did he leave?). Middle Childhood Throughout the grade school years, children’s conceptualizations of themselves and others become more complex and multidimensional. They are able to combine information in new ways, describing a series of actions, relations among ideas, and part–whole relations. Their descriptions of others move beyond the concrete, physical descriptions and begin to include more psychological attributes and relational terms (Barenboim, 1977). For example, increasingly children describe themselves with dispositional terms (nice, mean, friendly, shy, etc.), make comparative statements (e.g., he’s faster than Jimmy), and become more skilled at understanding others’ intentions and predicting the future behavior of others based on dispositional, rather than entirely situational, cues (Barenboim, 1977; Ruble, Newman, Rholes, & Altshuler, 1988; Shantz, 1983). With these new thinking skills, grade school children begin to emphasize the relational and normative functions of social roles, rather than focusing on discrete behaviors (Watson & Fischer, 1980). For example, they recognize that a person becomes a mother when she has children and that because she has children, she remains a mother, regardless of her particular behaviors (Moore et al., 1977). During middle childhood, children’s conceptualizations of friendship also become more sophisticated. Corresponding to a less contextually bound view of people’s behavior and characteristics, friendships in middle childhood are viewed as more stable and relational. Grade school children continue to play together, but their notions of what constitutes a “friend” begins to transcend play and includes expectations that the friend will

be loyal, trustworthy, helpful, and dependable (Bukowski & Hoza, 1989; Furman & Bierman, 1983). To make and keep friends, children need complex group entry and communication skills and must be able to follow rules in games and school activities (Bierman & Welsh, 1997). Children who continue to display aggressive behavior and who are immature and poorly regulated are increasingly viewed by their peers as annoying and are likely to experience peer rejection (Parker, Rubin, Price, & DeRosier, 1995). Grade school children also become increasingly sophisticated in their understanding of their own emotions and those of others. With regard to affective vocabulary, children move beyond the basic feelings of happy, sad, scared, and afraid and begin to conceptualize more complex emotions such as shame, guilt, pride, and jealousy (Borke, 1973). Grade schoolers become increasingly aware that emotions are private, and that they can deny or conceal their feelings whenever they choose (Caroll & Steward, 1984; Lewis, 2000; Manstead, 1995). Elementary school-age children begin to understand conflicting emotions, by considering that different emotions can occur sequentially, for example, that someone will first feel sad about moving away and then feel excited about going to a new town (Gnepp, 1983; Reichenbach & Master, 1983). Adolescence Finally, adolescents begin to consider abstract relationships among systems of logic and sets of concepts. This reasoning ability enables adolescents to generate and consider multiple solutions or perspectives for a given problem or situation; it makes possible sophisticated inductive logic and the consideration of a myriad of “what-if” hypothetical propositions about the world. Correspondingly, the descriptions of self and other given by adolescents are often well differentiated and well integrated, making use of abstract inferences to detect regularities in the diverse behaviors of self and other. Adolescence is a critical period in the development of self-concept. Adolescents increasingly define themselves through peer group affiliations and may use the behavior and value system of a particular peer group

reyn2-9.qxd

6/6/2003

9:13 AM

Page 223

9. Using the Clinical Interview

to shape their own identity. Consequently, adolescents may adopt styles of dress, musical taste, or activity preferences that identify them as members of a particular “crowd” (East, Lerner, Lerner, & Soni, 1992). The capacity for abstract and self-reflective thinking may have particular importance for the emotional experiences that accompany self-evaluation in adolescence. Because adolescents are more able to consider how they are viewed by various others, they may feel particularly self-conscious, as if others everywhere are interested in and judgmental of their thoughts, feelings, and actions (Damon, 1983). Correspondingly, they may increasingly experience feelings such as anxiety, guilt, and self-reproach, as they become cognitively able to consider and reconsider thoughts and feelings internally without needing an immediate external stimulus. Several investigators have posited that these advanced cognitive abilities may provide the foundation for the dramatic increases in internalizing disorders such as depression during adolescence (Cicchettei & SchneiderRosen, 1986). Adolescents’ ideas about friendship differ significantly from those of younger children. In adolescence, play is replaced with “hanging out,” and communication and intimate self-disclosure become central features of friendship, especially for girls (Hartup & Laursen, 1989; Laursen, 1993). In addition, adolescent conceptualizations of friendship are often related to group identity and affiliation, which in mm correspond with emerging ideas about personal identity and values (O’Brien & Bierman, 1988). Because of the increasing influence of peers and social comparison in adolescence, peer status may have an impact selfesteem and self-concept more significantly than in earlier years (O’Brien & Bierman, 1988). As one adolescent explained: The trendies are fashion-conscious, you know, but the hicks just wear jeans. You have to dress like they dress if you want to be accepted, and each group has its own dress code more or less. It’s important to be accepted because it give you a sense of pride and belonging. Otherwise, you just feel tossed away. (in O’Brien & Bierman, 1988, p. 1364)

By adolescence, young people are typically fairly sophisticated in their emotional un-

223

derstanding. In addition to possessing an extensive and well-differentiated affective vocabulary, adolescents recognize that facial and contextual cues are often inadequate to determine a person’s true feelings. Furthermore, the changes in cognitive capacities occurring in adolescence permit the recognition that people can, in fact, experience conflicting emotions simultaneously and deliberately manipulate their facial expressions in order to mask their feelings (Gnepp, 1983; Reichenbach & Masters, 1983). Implications for Adaptation and Functioning The ability to integrate information in person perceptions and reason flexibly about emotions and interpersonal relations thus develops gradually during childhood, with major transformations occurring around the ages of 4–7 (with the transition from preschool to middle childhood) and the ages of 11–14 (with the transition from middle childhood to adolescence). These changes have important implications for children’s social understanding and, correspondingly, for their social behavior. For example, the concrete and unidimensional social conceptions of the preschool child provide little basis for the prediction or explanation of others’ social behavior, making their social world somewhat unpredictable and “magical” and fostering a reliance on direct social feedback (e.g., adult praise or rebuke) to establish their standards of behavior and sense of self. By middle childhood, as children become more adept at inferring logical cause-andeffect sequences and making inferences about internal factors (motives and dispositions) and external factors (rules, standards, consequences) that influence their own and others’ behavior, they are more able to predict interpersonal events and to anticipate the behaviors and reasons of others (Kelvin, 1970). Self and other evaluations are based on their internalized set of conventional rules or standards of behavior and on social comparisons. Adolescents then begin to move beyond rule-based or conventional expectations, recognizing that the relevance or importance of particular standards may very depending on the circumstances and individuals involved. The capacity to consider

reyn2-9.qxd

6/6/2003

9:13 AM

Page 224

224

III. INTERVIEWING AND OBSERVATIONS

multiple concepts at multiple levels allows adolescents to consider and evaluate conflicting motivations and emotions in themselves and others and provides a basis for complex problem analysis and problem solving. At the same time, this capacity can support more extended social and selfanalysis and self-consciousness, which can increase vulnerability for depression and anxiety. Even if a child’s chronological age suggests that he or she should have the cognitive-developmental capacities to reason in a flexible and causal fashion about his or her behavior or emotional distress, the child may not actually be able to do so. Chronological age is a relatively poor marker of reasoning abilities in affectively charged domains, and children who have experienced a great deal of stress, conflict, or unpredictability in the social-emotional domain are particularly likely to show immature or variable reasoning. Regression in reasoning abilities in the face of stress is also not uncommon (Selman, 1980). The quality of a child’s reasoning about social-emotional issues can inform the selection of treatment goals and design of intervention approaches; hence, understanding the developmental level of the child’s social-emotional reasoning and interpersonal understanding is a central objective of the child clinical interview (see Bierman & Montiminy, 1993; Craighead, Meyers, Craighead, & McHale, 1982; Furman, 1980; Weisz, 1997; Weisz, Huey, & Weersing, 1998, for more discussion on the tailoring of intervention strategies to the developmental level of the child).

INTERVIEW TECHNIQUES Five general phases of the child–clinical interview are presented here: (1) preparing for the interview; (2) establishing rapport and getting acquainted: (3) the exploratory phase, in which semistructured prompts and open-ended questions may be used to elicit and examine social-emotional themes and issues of particular importance to the individual child; (4) a phase in which more specific and standardized self-rating instruments may be administered to enable comparison with age-graded norms; and (5) closure. Not all interviews incorporate all

five phases, and the content of each phase varies with the developmental level, verbal skills, and interpersonal responsivity of the particular child client. As stated earlier, a major advantage of child interviewing is that it provides the clinician with the flexibility to tailor his or her interviewing strategies to maximize the individual child’s ability to communicate. Hence, the specific techniques presented next are intended to serve as ideas and guides.

Preparing for the Interview Prior to the interview, it is essential that the interviewer obtain some background information about the child. At a minimum, this should include the child’s age and gender, the reason for the interview, and any special needs or developmental delays. In addition, it is ideal to have some preliminary assessment information from parents or teachers in the form of behavioral checklists or school reports, medical records, family background information, and reports from previous psychological evaluations (Eyeberg, Boggs, & Rodriguez, 1992). This information helps to orient the interviewer toward the needs of the particular child and family and facilitates planning and organization of the interview. In addition, it assists the interviewer in deciding how to present him- or herself to the child and family and to help address both parent and child expectations regarding the interview. When planning the physical space, it is particularly important to consider the age and developmental status of the child. The space should be comfortable and developmentally appropriate, with furniture appropriate to the child’s size, the room should be well-lit and inviting, and the space should be large enough to comfortably accommodate everyone who needs to be present for the interview. For young children, it is usually appropriate to have toys available, although careful consideration should be given to the type and quantity. Because the presence of many toys or highly compelling toys may be distracting to children, it may be desirable to present only a few items at a time and to keep materials stored in a cabinet or high shelf accessible only to the interviewer. Simple toys that foster communication, dramatic play, and interaction are best,

reyn2-9.qxd

6/6/2003

9:13 AM

Page 225

9. Using the Clinical Interview

including paper and crayons or a chalkboard, a doll house, a few dolls and cars, blocks, clay or Play-Doh, puppets, and toy telephones. Toys that tend to elicit stereotypical play (e.g., a punching ball and a baby bottle) are less useful, because they create a demand for a certain kind of play. For grade school children, toys are often less necessary. It may be appropriate to have drawing materials and quick, simple board games, such as Connect Four or Uno, that can be used as icebreakers. In addition, elementary school-age children may benefit from Play-Doh, craft sticks, or other manipulative materials to avoid restlessness or boredom during the information-gathering stage of the interview. As with preschool children, however, care should be taken to avoid highly involved games, crafts, or manipulative activities that are so absorbing that they draw children’s attention away from the interview. Adolescents will typically not require toys or games, but some may appreciate access to “knick-knacks” and other things that can be fidgeted with in the event of anxiety. Adolescents may be more adult-like in their ability to attend, complete written measures, and answer questions in an adult-like format, and may be offended by physical surroundings that appear “babyish.”

Establishing Rapport and Getting Acquainted Establishing rapport with the child and obtaining his or her cooperation is essential to the success of the interview. Preschool children are often reluctant to separate from caregivers and go with an unfamiliar adult, and they may need to be enticed or encouraged in the interview situation. If at all possible, children who express high levels of separation anxiety should be interviewed with a parent or familiar adult present. Often children warm up to the situation quickly and allow the caregiver to leave, particularly when the interviewer is supportive, positive, and engaging as a play partner. If the child cannot tolerate separation, it may be necessary to complete the interview with a caregiver present. With children of any age, the interview should begin with an explanation and statement of purpose. For young children, this

225

should be simple and straightforward, for example: “This is a place where moms and dads and kids come to talk with a helper like me. Sometimes they tell me they wish things could go better at home or at school, and I help them figure things out so that they can feel better.” The child should then be given time to explore the room and find a toy with which he or she would like to play. If the interview is focused around play, as it typically is for preschoolers, the interviewer should allow a few minutes of warm-up time prior to exploring social or emotional themes. The interviewer may learn about the child by observing the manner in which the child approaches the room and the play he or she initiates. That is, the organization and complexity of the child’s play and the themes that arise provide a basis for some initial hypotheses about the child’s thoughts and feelings, which can be followed up later in the interview (see Greenspan, 1981, for more information about clinical observation in play interviews). Most grade school children separate willingly from caregivers but may be more guarded than preschoolers and less likely to share sensitive personal information. It may be helpful to begin by asking the school-age child what his or her parents have said regarding the purpose of the interview, and clarify any misconceptions. In addition, it may be necessary with grade school children, and certainly with adolescents, to discuss issues of confidentiality in a frank and forthright manner. The degree of confidentiality will vary depending on the purpose of the interview, and children should never be promised a greater degree of confidentiality that can realistically be granted. Grade school children may benefit from a brief warm-up period that involves playing a game, coloring or doing a simple craft, or discussing nonthreatening topics such as the child’s pet or favorite sport prior to exploration of sensitive issues. Separation from caregivers is almost never an issue for adolescents, and most adolescents will not require toys or games to establish rapport or engage in the interview process. However, establishing rapport may be challenging, particularly if the youth views the interviewer as an adversary allied with parents or school personnel. Sullen,

reyn2-9.qxd

6/6/2003

226

9:13 AM

Page 226

III. INTERVIEWING AND OBSERVATIONS

hostile, avoidant, or resistant behavior on the part of adolescents can sometimes be overcome if the interviewer presents him- or herself as neutral and respectful. In addition, it may help if the interviewer conveys to the youth that his or her opinion and perspectives on issues are important and valued highly. Although adolescents may also benefit from a warm-up period where nonthreatening topics are discussed, this is usually less necessary than for younger children.

Exploring Social-Emotional Themes Obtaining Information from the Child: General Considerations The way in which the interviewer chooses to approach the information-gathering phase of the interview will vary considerably depending on the child’s age, developmental status, language ability, and attention span, as well as the specific goals of the interview. The younger the child, the greater the need for a flexible, free-flowing format that does not elicit resistance or behavioral control issues. A variety of approaches exist, including relatively nonstructured play interviews that are totally individualized and contain no standardized material, semistructured interviews that allow for some variation in the format and presentation of questions but contain some prespecified content, and highly structured interviews with specific administration and scoring procedures. Typically, when interviewing children, a combination of unstructured and semistructured methods is used. Regardless of the approach adopted, virtually all interviews rely to some extent on children’s willingness and ability to respond to questions. Therefore, interviewers should be familiar with developmental considerations involved in questioning children. The Significance of Language Development and Reading Ability Perhaps the most salient factor in children’s responsivity to questions is their level of language and literacy development. Questions addressed to children should contain age-appropriate vocabulary and syntax. For example, preschool children respond better to short, concrete questions with limited

complexity (e.g., “ Who are your friends at school?”) rather than more open-ended or abstract questions (e.g., “Tell me about your friends” or “What causes you trouble at school?”). Both language deficits and socioemotional factors may contribute to children’s failure to respond to questions. Young children often ignore questions than they cannot understand, or respond idiosyncratically (Crosby, 1976; Tyack & Ingram, 1977). Preschoolers may be particularly confused by when and why questions, which are more linguistically complex than who, what, and where. Even when they do respond appropriately, younger children’s answers to questions are typically shorter and less detailed than those of older children. Highly inhibited children who seem uncomfortable responding to even basic questions or children with communication deficits can be encouraged to respond nonverbally, through drawing or pointing to pictures (Bierman & Schwartz, 1986; Sattler, 1988). In addition, if children seem unable or unwilling to answer questions, the interviewer can refrain from questioning and engage instead in commentary and inductive statements regarding the child’s behavior and affect (Stone & Lemanek, 1990). Prior to about third grade, children are unable to respond reliably to written questionnaires due to limited reading vocabulary and comprehension (Stone .& Lemanek, 1990). When these measures are used with young children, they must read aloud, and the interviewer must first make sure that the child understands both the questions and the response format. Studies have revealed that children may respond capriciously to structured interview and questionnaire formats, even when the response required is a simple yes/no choice. They may answer “yes” to everything (Mischel, Zeiss, & Zeiss, 1974), or they may select the last choice presented (Crandall, Crandall, & Katovsky, 1965). Adaptations of Likerttype scales for young children have included the use of unhappy, neutral, or happy faces (to represent a 3-point scale) and a thermometer (to represent a continuous scale), but even when these modifications are used, the interviewer should begin with some sample items to make sure that the child understands the format before using it to obtain information. Even if a written question-

reyn2-9.qxd

6/6/2003

9:13 AM

Page 227

9. Using the Clinical Interview

naire or checklist seems age appropriate for a child, the interviewer should obtain some brief confirmation that the child’s reading and writing skills are adequate before administering the measure. The Use of Play Because structured interview formats are rarely appropriate for preschool children, the play interview is most commonly used with children this age. Therefore, play materials should be selected that maximize the likelihood of obtaining the types of information desired. In general, toys with high demand characteristics (e.g., punching bags) should be avoided, because they elicit specific behaviors that reveal little about the individual child. When exploring social-emotional themes, dolls and dollhouses or small action figures, paper and crayons, or manipulative materials such as blocks and PlayDoh may be the most facilitative. Sometimes the interviewer can join in the child’s play, creating scenarios and then asking the child to explain or interpret them. For example, if playing with dolls, the interviewer could suggest that one of the dolls is crying and ask the child what happened, what the doll is feeling, or what will happen next. By entering into the play, the interviewer may attempt to stimulate more expressive play by wondering aloud what a doll is saying or what events will happen next. The interviewer may even want to introduce events, such as a doll doing something bad and then having the child “guess” what bad thing was done or what will happen next, encouraging the child to show the consequences. Some children respond well to this type of play dialogue and are eager to play out family interactions and feelings; other children are not interested in story play or reject the interviewer’s bid for play entry. The Use of Affect Labels and Semistructured Tasks In general, children are able to recognize and identify concepts before they can spontaneously describe and discuss them, and they can demonstrate mastery of a concept on a nonverbal task before they can articulate their reasoning (see Shantz, 1975). Hence, designing tasks so that they require

227

nonverbal or less complex verbal responses may enable young children to communicate feelings that they might otherwise have trouble putting into words. One such strategy involved the use of affect labels. The basic strategy of the “affect label” technique is to use concrete aids to enable children to discuss their negative and positive feelings. Using simple drawings depicting happy, sad, angry (and, for older children, frightened) expressions, the interviewer can explore emotions. One method involves having the child label each picture and then indicate which one corresponds to his or her feelings in a particular situation (e.g., “How do you feel when your mom and dad fight?” or “Show me how you look when you’re at school”). The benefit of this method is that it does not require a verbal response by the child. A more demanding variation of this task is for the interviewer to point to each picture and ask the child, “What’s something that makes you feel this way?” When children offer spontaneous comments about their emotions, it may be possible for the interviewer to follow up with probing questions; however, careful attention should be paid to those topics that seem to make the child anxious or uncomfortable, and these should be explored in as nonthreatening way as possible. For example, if a child lists, “When my mom and dad fight” as something that makes him or her sad, the interviewer may ask follow-up questions such as these: “What about that makes you sad?” “Does it make you sad, or does it make you a little bit mad or scared too?” “What do you feel like doing when that happens?” “What do you think might happen when you hear them fight?” The interviewer should pay close attention to the child’s responsivity when conducting such follow-up probing, backing off if a child appears uncomfortable (verbally or behaviorally), and return to a less threatening mode of interview. Another example of the affect label technique of interviewing has been developed by Bene and Anthony (1957) in their Family Relations Test. Designed specifically to help children express their feelings about various family members, this task presents children with feeling statements (e.g., “I like to hug this person”), which they deliver to one of

reyn2-9.qxd

6/6/2003

228

9:13 AM

Page 228

III. INTERVIEWING AND OBSERVATIONS

the cardboard figures representing each member of the family. Statements include positive and negative feelings experienced toward and received from others. Using pictures to help children label their feelings provides young children with a fairly easy task and can provide older children with a fairly nonthreatening initial interview task. For older children who have greater verbal capabilities, pictures can also be used to elicit more elaborate responses, as in the picture–question interview techniques described next. The Assessment of Social Cognition Pictures and vignettes depicting hypothetical social situations can be helpful in the assessment of children’s social-cognitive processes. The simplest of these methods, and the one most appropriate for young children, involves showing children pictures of social situations and asking them to interpret the situation (e.g., “Uh-oh, I see two boys standing and looking at a broken toy. This one’s crying. What do you think happened?”), identify the emotions of the characters (e.g., “Look at this boy. How do you think he’s feeling?”), or predict what will happen next (e.g., “It looks like he’s pretty upset about his toy. What do you think he’ll do now?”). When using pictures or stories in this manner, it is most helpful for the interviewer to choose scenarios relevant to particular children’s issues. For example, if children are having difficulties at school, the interviewer may want to focus on academic or peer-related scenarios; for problems at home, pictures that focus on family relationships may be most helpful. With elementary school-age children and young adolescents, videos or vignettes depicting social situations can be used to assess multiple aspects of social cognition and interpersonal competence, including social goals and expectations, social knowledge, and strategies for dealing with interpersonal dilemmas conflict. When attempting to understand children’s social behavior and decisions regarding relationships, it is essential to assess their social goals. Children with maladaptive goals may choose interactive strategies that contribute to their victimization or rejection by peers or adults (Egan, Monson, & Perry, 1998; Hodges & Perry,

1999) For example, aggressive-rejected children frequently select goals involving dominance, revenge, or instrumental gain rather than friendship or acceptance, which may influence their choice of coercive (rather than prosocial) interactive strategies. To assess children’s social goals, the interviewer reads a story or shows a short video segment, then asks questions about what the protagonist was trying to accomplish and whether or not this actually occurred. For example, the interviewer can show the child a brief video in which someone wins a race by cheating and then ask what the character had hoped would happen, and whether or not it did happen. Alternatively, the interviewer can ask the child to describe what he or she would have done in that situation (Dodge & Coie, 1987; Dodge, Murphy, & Buschbaum, 1984). Vignettes can also be used to determine the types of strategies that children generate and endorse in particular social situations, and types of interpretations that guided the choice of strategies. To determine a child’s attributional style, the interviewer could show or tell a story involving a relevant social challenge (e.g., group entry, provocation, or conflict) and ask for the child’s interpretation of participants’ motives and whether the behavior was accidental or intentional. Similarly, if attempting to determine the extent, type, or appropriateness of interpersonal strategies in a child’s repertoire, the interviewer can ask the child to generate as many possible solutions to the problem as he or she can think of. If needed, the interviewer can frame the question in such a way as to make the social goal explicit (e.g., “Let’s say you really wanted them to play with you. Then what would you do?”). Finally, if interested in a child’s ability to evaluate or anticipate the effectiveness of a particular interpersonal choice, the interviewer could present the vignette, provide a strategy and an outcome, and then ask the child, “How did it work out for him (her)?”, or “What do you think will happen next time?” Research using these methods has found significant relationships among measures of social cognition and social dysfunction. Children with peer difficulties and impulsive, disruptive behavioral problems often

reyn2-9.qxd

6/6/2003

9:13 AM

Page 229

9. Using the Clinical Interview

have difficulty generating solutions to hypothetical social situations, are more likely to endorse aggressive strategies, are more likely to rate aggressive strategies as successful, and more frequently attribute hostile, negative intentions in ambiguous social situations (Dodge, 1993; Dodge, Petit, McClaskey, & Brown, 1986; Lochman, 1987). Studies examining differences in the clinical utility of stories versus videos have revealed that videos typically provide greater ambiguity than stories, because often the goals, thoughts, and feelings of the characters are implied rather than explicit. Therefore, stories may be easier for young children to comprehend, whereas age-appropriate videos may be more interesting or compelling for older children (Goldman, Stein, & Guerry, 1983). The Normative Beliefs About Aggression Scale (Huesman & Guerra, 1997) is a published measure designed to assess children’s social cognitions regarding aggression. Research using this scale revealed a link between children’s beliefs about aggression and patterns of aggressive behavior, especially for older children. Designed for use with elementary school students, it includes 20 questions, such as “Suppose a girl hits a boy. Do you think it’s wrong for the boy to hit her back?,” and “Is it wrong to insult people?” Children rate the questions on a 4point scale (perfectly OK, sort of OK, sort of wrong, really wrong). This scale can be used independently or in conjunction with hypothetical vignettes and stories to obtain a sense of a child’s beliefs and attitudes regarding interpersonal aggression. The Use of Drawing Stories Like picture–question techniques, children’s drawings can provide a concrete stimulus to help them focus on clinically relevant topics. For example, children may be asked to draw themselves and then to answer questions about their drawings, such as things they like or do not like to do, things they like or do not like about school or the family, things that get them into trouble, or things that make them happy, sad, and mad. After asking the child to draw him- or herself, for example, the interviewer can suggest, “What a nice drawing. Let’s do something special with it. I’m going to put some

229

numbers on the side here to help us tell all about this drawing. For the first list, I need to know something you like to do.” This strategy tasks the interviewer’s direct (often intimidating) focus off the child and puts it onto the task, and it gives the child a concrete response demand. The interviewer can maintain a positive and supportive attitude toward the child when using this technique, encouraging and praising the child for thinking of things while enabling the child to express his or her hopes, fears, and frustrations. Harter (1979) describes another drawing technique that she uses in her clinical interviews to help young children describe family interactions. First, she asks children to draw themselves and their family doing something together. She then draws a bubble above each figure to hold the words that each figure might be saying (as in a comic strip). Children are encouraged and helped to construct a story by suggesting things that family members might say to each other. They are asked to think of one thing at a time that a family member could say—a method that reduces the verbal and organizational skill demands made on the children. Although young children may find it difficult to describe the nature of family interactions on an abstract level, this strategy allows them to represent concretely their views of typical family interaction patterns. The Use of Open-Ended Interview Techniques Understanding and organizing coherent answers to open-ended questions requires a fairly high level of comprehensive and expressive abilities. However, interview questions can be structured to reduce verbal complexity and ambiguity by providing children with concrete references. For example, Yarrow (1960) suggests several ways in which open-ended questions may be made easier for children: (1) suggesting that other children feel that way (e.g., “One kid I know told me that he wished more kids like him. Do you ever feel that way?); (2) giving two alternatives (e.g., “Do you ever wish that your dad spent more time with you, or do you think he spends enough time with you?); (3) softening negative choices or phrasing questions to imply negatives (e.g., “What kinds of things do your brother and you fight

reyn2-9.qxd

6/6/2003

230

9:13 AM

Page 230

III. INTERVIEWING AND OBSERVATIONS

about?” rather than “Do you ever fight with your brother?”); and (4) combining questions about positive affects with those about negative feelings (e.g., “What’s one thing you mom does that you really like? What’s something she does that you don’t like very much?). Many of these questions can be extended with problems that provide concrete structure (e.g., “Tell me one thing that you don’t like about school,” or “When was the last time you felt that way?”). In addition to the content of the child’s responses, the clinician will want to attend to the child’s reasoning about his or her feelings and relationships. After noting the child’s spontaneous description of a feeling or an event, the interviewer may wish to explore and gradually clarify exactly how the child was reasoning, asking for elaboration and confirmation until the child’s logic becomes clear. For example, if a boy were to say that it makes him mad when his brother won’t share, the interviewer may ask, “What about what makes you angry?” It may be that the boy simply covets a particular toy, or it may turn out that the child’s perception of differential parental preference (his brother gets better toys than him) is the real issue. Young children often present spontaneous descriptions of interpersonal events, fantasies, or feelings that are vague or disorganized. If the clinician begins with the child’s statements, and then gradually helps the child with probes to expand and explain each piece of his or her story, the clinician may gain a better understanding of the child’s thought processes and may also help the child to clarify his or her understanding of the event in question. Another structured way to interview is to phrase questions as sentence completions. That is, the interviewer may tell the child that he or she would like to make a story together. The interviewer may begin with a prompt designed to explore an issue of significance for the child. For example, with a girl with learning disabilities, the interviewer may begin with a prompts such as the following: “One day a girl was at school and she was feeling . . . (to child:) what should we say she was feeling at school?” The child may respond, “Sad.” The interviewer may then continue, “She was feeling sad because . . . (to child:) What could we say she was sad about?” In this manner, the interviewer

can encourage the child to explain the various feelings she may herself experience (or wish to experience) at school. Similarly, an interviewer may wish to select some of the story beginnings provided in the Medeleine Thomas Stories, which are well suited for grade school children (Wursten, 1960). The themes that arise when a child completes these stories may provide the interviewer with insights concerning the child’s perceptions and feelings, which again can be examined or pursued further in the interview. In addition to these open-ended interview strategies, the clinician may wish to use more standardized self-report measures, particularly with grade school and older clients.

The Use of Self-Report Measures There are a number of published instruments available to assess various aspects of emotional understanding and interpersonal reasoning, including measures of global social-emotional functioning, self-perceptions and feeling of distress, and friendships and social connectedness. While a few of these are appropriate for use with young children, the majority of these measures were designed for use with grade school children and adolescents, who can attend well to the task and respond more reliably to multiplechoice and Likert-type scale formats. Because many of these measures provide a reference group and some information regarding their psychometric properties, they may be particularly useful if interviewers are attempting to understand aspects of a child’s interpersonal reasoning relative to similar-age peers. However, standardized assessments are minimally useful and can even produce misleading results if they are administered incorrectly or used inappropriately (e.g., with the wrong age group), so interviewers who opt to use these measures should take care to use them in the intended manner. Following are a number of published self-report measures. These measures do not represent an exhaustive list of instruments in their category but are meant to provide examples of conceptually relevant measures with good psychometric properties. Table 9.1 summarizes measures used to evaluate perceived competence and psycho-

reyn2-9.qxd

6/6/2003

9:13 AM

Page 231

9. Using the Clinical Interview

logical well-being. These include reports of subjective feelings of depression, anxiety, and loneliness as well as global and specific self-concept. Table 9.2 lists assessments designed to tap various aspects of relationship quality and social networks. These mea-

231

sures assess the functions of specific relationships in the life of a child or youth, including family members, friends, peers, and romantic partners. They also provide some measure of the size and density of a particular child’s social network.

TABLE 9.1. Assessment of Perceived Competence and Psychological Well-Being Instrument

Author(s)

Appropriate ages

Sample items

Revised Manifest Anxiety Scale

Reynolds & Richmond (1978)

6 and up

I worry a lot of the time. I am nervous. My hands feel sweaty.

Loneliness Scale

Asher, Hymel, & Renshaw (1984)

Forms for grade school and adolescents

Harter’s SelfPerception Profile for Children

Harter & Pike (1984)

Versions available for preschool through college age

Children’s Depression Inventory (CDI)

Kovacs (1992)

7 and up

It’s hard for me to make friends. I’m lonely. I’m well-liked by the kids in my class. Some kids feel that they are really good at school work, but other kids worry about whether they can do the work assigned to them. Some kids find it hard to make friends, but for other kids it’s pretty easy to make friends. I feel like crying every day. I am tired all the time. Nothing is fun at all.

TABLE 9.2. Assessment of Interpersonal Relations and Social Networks Instrument

Authors

Population

Sample Items

My Family and Friends

Reid, Landesman, Treder, & Jaccard (1989)

6–12 years

How angry or upset do you get with (specific person) even if you don’t show it? When you go to (specific person) for help with school things, how helpful is s/he?

Network of Relationships Inventory

Furman & 9 and up Buhrmester (1992)

How much free time do you spend with this person? How much do you share your secrets and private feelings with this person?

Friendship Qualities Scale

Bukowski, Hoza, & Boivin (1994)

10–14

My friend and I spend all our free time together. My friend would help me if I needed it.

Friendship Questionnaire

Bierman & McCauley (1987)

8–13

Is there someone who invites you over after school? (How often?) Is there someone who beats you up? (How often?)

10–18

When (person) is not around, I miss him/her. I know that whatever I tell (person) is kept secret between us.

16–20

My friends respect my feelings. My parents trust my judgment. I feel that no one understands me.

Intimate Sharabany (1994) Friendship Scale Inventory of Parent and Peer Attachment Scale

Armsden & Greenberg (1987)

reyn2-9.qxd

6/6/2003

232

9:13 AM

Page 232

III. INTERVIEWING AND OBSERVATIONS

Closure At the conclusion of the interview, children should be thanked for their participation and cooperation. If the exploration of emotionally laden themes has been upsetting or disorganizing for the child, it is best to give him or her a “regrouping” period of free play or light activity prior to ending the session. The interviewer may also use this as a time to reiterate support and positive regard for the child.

SUMMARY In summary, then, this chapter has presented interviewing strategies that may increase the effectiveness of open-ended clinical interviews with children and youth. The importance of developmental considerations has been stressed, both in selecting interview techniques and in interpreting children’s interview responses. Clinicians are encouraged to consider the supplemental use of structured self-report rating scales to explore areas of clinical relevance for specific children and to enable comparison with developmental norms. Although child interviews cannot replace parent and teacher interviews or behavioral observations as sources of information about children’s interpersonal reasoning and emotional understanding, they do provide a unique and valuable perspective and permit clinicians to gain insights into children’s current thoughts, perceptions and feelings. Moreover, child interviews can provide information useful for treatment planning. If individual child therapy is planned, the child interview can serve to establish initial rapport and to provide the clinician with information concerning the type of communication or expressive medium suited for a particular child.

REFERENCES Armsden, G. C., & Greenberg, M. T. (1987). The inventory of parent and peer attachment: Individual differences and their relationship to psychological well-being in adolescence. Journal of Youth and Adolescence, 16(5), 427–454. Asher, S. R., Hymel, S., & Renshaw, P. D. (1984). Loneliness in children. Children Development, 55, 1456–1464.

Barenboim, C. (1977). Developmental changes in the interpersonal cognitive system from middle childhood to adolescence. Child Development, 48, 1467–1474. Bene, E., & Anthony, J. (1957). Manual for the Family Relations Test. London: National Foundation for Educational Research in England and Wales. Bierman, K. L., & Furman, W. (1981). Effects of role assignment rationale on attitudes formed during peer tutoring. Journal of Educational Psychology, 73, 33–40. Bierman, K. L. &, McCauley, E. (1987). Children’s descriptions of their peer interactions: Useful information for clinical assessment. Journal of Clinical Child Psychology, 16, 9–18. Bierman, K. L., & Montminy, H. P. (1993). Developmental issues in social skills assessment and intervention with children and adolescents. Behavior Modification, 17, 229–254. Bierman, K. L., & Schwartz, L. A. (1986). Child clinical interviews: Approaches and developmental considerations. Journal of Child and Adolescent Psychotherapy, 3(4), 267–278. Bierman, K. L., & Welsh, J. A. (1997). Social relationship deficits. In E. J. Mash & L. G. Terdal (Eds.), Assessment of childhood disorders (3rd ed., (pp. 328–365). New York: Guilford Press. Borke, H. (1973). The development of empathy in Chinese and American children between three and six years of age: A cross-cultural study. Developmental Psychology, 9, 102–108. Bukowski, W., & Hoza, B. (1989). Popularity and friendship: Issues in theory, measurement and outcome. In T. J. Berndt & G. W. Ladd (Eds.), Peer relationships in child development (pp. 15–45). New York: Wiley. Bukowski, W., Hoza, B., & Boivin, M. (1994). Measuring friendship quality during pre and early adolescence: The development and psychometric properties of the Friendship Qualities Scale. Journal of Social and Personal Relationships, 11, 471–484. Carroll, J., & Steward, M. (1984). The role of cognitive development in children’s understanding of their own feelings. Child Development, 55, 1456–1492. Cicchetti, D., & Schneider-Rosen, K. (1986). An organizational approach to childhood depression. In M. Rutter, C. E. Izard, & P. B. Read (Eds.), Depression in young people: Developmental and clinical perspectives (pp. 71–134). New York: Guilford Press. Craighead, W. E., Meyers. A. W., Craighead, L. W., & McHale, S. M. (1982). Issues in cognitive-behavior therapy with children. In M. Rosenbaum, C. M. Franks, & Y. Jaffe (Eds.), Perspectives on behavior therapy in the eighties (pp. 234–245). New York: Springer. Crandall, V. C., Crandall, V. J., & Katovsky, W. (1965). A children’s social desirability questionnaire. Journal of Consulting Psychology, 29(1), 27–36. Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social information processing mechanisms in children’s social adjustment. Psychological Bulletin, 115, 74–101. Crosby, F. (1976). Early discourse agreement. Journal of Child Language, 3, 125–126.

reyn2-9.qxd

6/6/2003

9:13 AM

Page 233

9. Using the Clinical Interview Damon, W. (Ed.). (1983). Social and personality development: Infancy through adolescence. New York: Norton. Denham, S. A. (1986). Social cognition, prosocial behavior, and emotions in preschoolers: Contextual validation. Child Development, 51(1), 194–201. Dodge, K. A. (1993, March). Social information processing and peer rejection factors in the development of behavior problems in children. Paper presented at the biennial meeting of the Society for Research in Child Development, New Orleans. Dodge, K. A., & Coie, J. D. (1987). Social information processing factors in reactive and proactive aggression in children’s playgroups. Journal of Personality and Social Psychology, 53, 1146–1158. Dodge, K. A., Murphy, R. R., & Buschbaum, K. (1984). The assessment of intention cue detection skills in children: Implications for developmental psychopathology. Child Development, 55, 163–173. Dodge, K. A., Pettit, G. S., McClaskey, C. L., & Brown, M. M. (1986). Social competence in children. Monographs of the Society for Research in Child Development, 51(2, Serial No. 213). East, P. L., Lerner, R. M., Lerner, J. V., & Soni, R. T. (1992). Early adolescent-peer group fit, peer relations, and psychosocial competence: A short-term longitudinal study. Journal of Early Adolescence, 12(2), 132–152. Egan, S. K., Monson, T. C., & Perry, D. G. (1998). Social-cognitive influences on change in aggression over time. Developmental Psychology, 34(5), 996–1006. Eyeberg, S. M., Boggs, S. R., & Rodriguez, C. M. (1992). Relationships between maternal parenting stress and child disruptive behavior. Child and Family Behavior Therapy, 14(4), 1–9. Furman, W. (1980). Promoting social development: Developmental implications for treatment. In B. B. Lahey & A. E. Kazdin (Eds.), Advances in clinical child psychology (Vol. 3, pp. 1–40). New York: Plenum Press. Furman, W., & Bierman, K. L. (1983). Developmental changes in young children’s conceptions of friendship. Child Development, 54, 549–556. Furman, W., & Buhrmester, D. (1992). Age and sex differences in perceptions of networks of personal relationships. Child Development, 63, 103–115. Garner, P. W. (1996). The relations of emotional roletaking, affective/moral attributions, and emotional display rule knowledge to low income school-age children’s social competence. Journal of Applied Developmental Psychology, 17(1). 19–36. Gnepp, J. E. (1983). Children’s social sensitivity: Inferring emotions from conflicting cues. Developmental Psychology, 19(6), 805–814. Goldman, J., Stein, C. L., & Guerry, S. (1983). Psychological methods of child assessment. New York: Brunner/Mazel. Gollin, E. S. (1958). Organizational characteristics of social judgements: A developmental investigation. Journal of Personality, 26, 139–154. Greenberg, M. T., & Kusche, C. (1993). Promoting social and emotional development in deaf children: The PATHS project. Seattle: University of Washington Press.

233

Greenspan, S. I. (1981). The clinical interview of the child. New York: McGraw-Hill. Harter, S. (1977). A cognitive-developmental approach to children’s expression of conflicting feelings and a technique to facilitate such expression in play therapy. Journal of Consulting and Clinical Psychology, 45(3), 417–432. Harter, S. (1979). Play techniques for child therapy. Lecture presented at the University of Denver. Harter, S. (1983). Developmental perspectives on the self system. In E. M. Hetherington (Ed.), Handbook of child psychology: Vol. 4. Socialization, personality, and social development (4th ed.) (pp. 275–386). New York: Wiley. Harter, S., & Marold, D. B. (1992). The directionality of the link between self-esteem and affect: Beyond causal modeling. In D. Cicchetti & S. L. Toth (Eds.), Rochester symposium on developmental psychopathology: Vol. 5. The self and its disorders (pp. 333–369). Rochester, NY: University of Rochester Press. Harter, S., & Pike, R. (1984). The Pictorial Scale of Perceived Competence and Social Acceptance for young children. Child Development, 55, 1969– 1982. Hartup, W. W. (1983). The peer system. In E. M. Hetherington (Vol. Ed.). Handbook of child psychology Vol. 4. Socialization, personality, and social development (4th ed.) (pp. 103–196). New York: Wiley. Hartup, W. W., & Laursen, B. (1989, March). Contextual constraints and children’s friendship relations. Paper presented at the biennial meeting of the Society for Research in Child Development, Kansas City, MO. Hodges, E. V., & Perry, D. G. (1999). Personal and interpersonal antecedents and consequences of victimization by peers. Journal of Personality and Social Psychology, 76(4), 677–685. Huesman, L. R., & Guerra, N. G. (1997). Children’s normative beliefs about aggression and aggressive behavior. Journal of Personality and Social Psychology, 2, 408–419. Hughes, J. N., Cavell, T. A., & Grossman, P. A. (1997). A positive view of self: Risk or protection for aggressive children? Development and Psychopathology, 9(1), 75–94. Kelvin, P. (1970). The bases of social behvavior: An approach to terms of order and value. London: Holt, Rinehart & Winston. Kovacs, M. (1992). The Children’s Depression Inventory. North Tonowanda: Multi-Health Systems. Laursen, B. (1993). Conflict management among close peers. In B. Laursen (Ed.), Close friendships in adolescence (pp. 39–54). San Francisco: Jossey-Bass. Lewis, M. (2000). Understanding emotion. In M. Lewis & J. Haviland-Jones (Eds.), Handbook of emotions (2nd ed.) (pp. 253–322). New York: Wiley. Livesley, W. J., & Bromley, D. D. (1973). Person perception in childhood and adolescence. Chichester, UK: Wiley. Lochman, J. E. (1987). Self and peer perceptions and attributional biases of aggressive and nonaggressive

reyn2-9.qxd

6/6/2003

234

9:13 AM

Page 234

III. INTERVIEWING AND OBSERVATIONS

boys in dyadic interactions. Journal of Consulting and Clinical Psychology, 55, 404–410. Manstead, A. S. (1995). Children’s understanding of emotion. In J. A. Russell & J. M. Fernandez-Dols (Eds.), Everyday conceptions of emotion: An introduction to the psychology, anthropology, and linguistics of emotion. NATO Series D: Behavioural and social sciences. Vol. 81 (pp. 315–331). Dordrecht, Netherlands: Kluwer Academic. Maszk, P., Eisenberg, N., & Guthrie, I. K. (1999). Relations of children’s social status to their emotionality and regulation: A short-term longitudinal study. Merrill-Palmer Quarterly, 45(3), 468–492. Mischel, W., Zeiss, R., & Zeiss, A. (1974). Internal– external control and persistence: Validation and implications of the Stanford Preschool Internal-External Scale. Journal of Personality and Social Psychology, 29(2), 265–278. Moore, N. V., Cooper, R. G., & Birckhard, M. H. (1977, March). The child’s development of the concept of family. Paper presented at the meeting of the Society for Research in Child Development, New Orleans. O’Brien, S., & Bierman, K. L. (1988). Conceptions and perceived influence of peer groups: Interviews with preadolescents and adolescents. Child Development, 59, 1360–1365. Parker, J. G., Rubin, K. H., Price, J. M., & DeRosier, M. E. (1995). Peer relationships, child development and adjustment: A developmental psychopathology perspective. In D. Cicchetti & D. Cohen (Eds.), Developmental psychopathology: Vol. 2. Risk, disorder, and adaptation (pp. 96–161). New York: Wiley. Reichenbach, L., & Masters, J. C. (1983). Children’s use of expressive and contextual cues in judgments of emotion. Child Development, 54(4), 993–1004. Reid, M., Landesman, S., Treder, R., & Jaccard, J. (1989). “My Family and Friends”: Six to twelve year old children’s perceptions of social support. Child Development, 60, 896–910. Reynolds, C. R., & Richmond, B. O. (1978). What 1 think and feel: A revised measure of children’s manifest anxiety. Journal of Abnormal Child Psychology, 6(2), 271–280. Ruble, D. N, Newman, L. S., Rholes, W. S., & Altshuler, J. (1988). Children’s “naive psychology”: The use of behavioral and situational information for the prediction of behavior. Cognitive Development, 3(1), 89–112. Rudolph, K., Hammen, C., & Burge, D. (1995). Cognitive representations of self, family and peers in school-aged children: Links with social competence and sociometric status. Child Development, 66(5), 1385–1402. Rutter, M., & Graham, P. (1968). The reliability and validity of the psychiatric assessment of the child: I. Interview with the child. British Journal of Psychiatry, 114, 563–579.

Saltz, E., & Medow, M. L. (1971). Concept conservation in children: The dependence of belief systems on semantic representation. Child Development, 42, 1533–1542. Sattler, J. (1988). Assessment of children. San Diego, CA: Sattler. Selman, R. L. (1980). The growth of interpersonal understanding: Developmental and clinical analyses. New York: Academic Press. Selman, R. L., & Schultz, L. H. (1990). Making a friend in youth: Developmental theory and pair therapy. Chicago: University of Chicago Press. Shantz, C. U. (1975). The development of social cognition. In E. M. Hetherington (Ed.), Review of child development research (Vol. 5, pp. 257–324). Chicago: University of Chicago Press. Shantz, C. U. (1983). Social cognition. In J. Flavell & E. Markman (Eds.), Handbook of child psychology (Vol. 3, 495–555). New York: Wiley. Sharabany, R. (1994). Intimate Friendship Scale: Conceptual underpinnings, psychometric properties, and construct validity. Journal of Social and Personal Relationships, 11, 449–469. Spence, S. H. (1988). The role of social cognitive skills in the determination of children’s social competence. Behaviour Change, 5(1), 9–18. Stone, W. L. & Lemanek, K. L. (1990). Parental report of social behaviors in autistic preschoolers. Journal of Autism and Developmental Disorders, 20(4), 513–522. Tyack, D., & Ingram, D. (1977). Children’s production and comprehension of questions. Journal of Child Language, 4, 221 –224. Watson, M. W., & Fischer, K. W. (1980). Development of social roles in elicited and spontaneous behavior during the preschool years. Developmental Psychology, 16(5), 483–494. Weisz, J. R. (1997). Effects of interventions for child and adolescent psychological dysfunction: Relevance of context, developmental factors, and individual differences. In S. S. Luthar & J. A. Burack (Eds.), Developmental psychopathology: Perspectives on adjustment, risk and disorder (pp. 3–22). New York: Cambridge University Press. Weisz, J. R., Huey, S. J., & Weersing, V. R. (1998). Psychotherapy outcome research with children and adolescents: The state of the art. Advances in Clinical Child Psychology, 20, 49–91. Wursten, H. (1960). Story Completions: Madeleine Thomas Stories and similar methods. In A. I. Rubin & M. Haworth (Eds.), Projective techniques with children (pp. 192–209). New York: Grune & Stratton. Yarrow, L. H. (1960). Interviewing children. In P. H. Mussen (Ed.), Handbook of research methods in child development (pp. 561–602). New York: Wiley.

reyn2-10.qxd

6/6/2003

9:14 AM

Page 235

10 Structured Diagnostic Interviewing

BRYAN R. LONEY PAUL J. FRICK

Feighner Research Diagnostic Criteria (Feighner et al., 1972), which later became the National Institute of Mental Health (NIMH) Diagnostic Interview Schedule (DIS; Robins, Helzer, Croughan, & Ratcliff, 1981), and the Schedule for Affective Disorders and Schizophrenia (SADS; Endicott & Spitzer, 1978). Over the past two decades, structured interviews have moved from being strictly research instruments to being a part of many clinical assessments. In addition, several interview schedules have been developed for use with children and adolescents. The current review focuses on describing and comparing these child-oriented interviews.

HISTORY Clinical interviews have a prominent place in the history of psychological assessment. The face-to-face verbal dialogue between assessor and client is the prototypical format for most clinical enterprises. For most of this history, the most common type of clinical interview has been the unstructured interview. This interview format offers maximum flexibility to the clinician in developing questions based on the needs of the individual being assessed. However, this format relies heavily on the clinician’s theoretical orientation and expertise to determine what questions are asked and how the answers are to be interpreted. As a result, the flexibility of this format must be balanced by the unreliability often associated with such an unstructured symptom assessment (McClellan & Werry, 2000). The structured diagnostic interview was developed out of a need for more clear and consistent guidelines for assessing psychopathology. Structured interviews were originally developed for use with adults and were primarily used in research. Two of the better known interview schedules were the

COMMONALITIES ACROSS INTERVIEWS Table 10.1 provides a summary of the basic features of the most commonly used structured interviews for children and adolescents. Structured diagnostic interviews consist of a set of questions that the assessor asks the child or adolescent. There are explicit guidelines on how a child’s responses are to be scored. With a few exceptions not235

reyn2-10.qxd

6/6/2003

9:14 AM

Page 236

236

III. INTERVIEWING AND OBSERVATIONS

TABLE 10.1. Basic Features of Interview Schedules for Children Name

Primary references

Time

Age to Time administer frame

Informants

Degree of structure

Anxiety Diagnostic Interview for DSM-IV (ADIS)

Albano & Silverman (1996); Silverman & Albano (1996)

60–90 minutes

7–17 years

Present

Child and parent

Semistructured

Child and Adolescent Psychiatric Assessment (CAPA)

Angold & Costello (2000)

60–90 minutes

8–17 years

Present

Child and parent

Semistructured

Children’s Interview for Psychiatric Symptoms (ChIPS)

Weller, Weller, 30–60 Fristad, minutes Rooney, & Schecter (2000)

6–18 years

Present

Child and parent

Highly structured

Diagnostic Interview for Children and Adolescents (DICA)

Reich, 60–120 Herjanic, minutes Welner, & Gandhy (1982); Reich (2000)

6–18 years

Lifetime

Child and parent

Semistructured

Diagnostic Interview Schedule for Children (DISC-IV)

Shaffer et al. (1993, 2000)

90–120 minutes

6–17 years

Present (separate lifetime module)

Child and Highly parent structured (experimental teacher version available)

Dominic-R

Valla, Bergeron, & Smolla (2000)

15–25 minutes

6–11 years

Present

Child

Highly structured (pictorial interview)

Interview Schedule for Children and Adolescents (ISCA)

Kovacs (1985); Last (1987); Sherrill & Kovacs (2000)

120–150 minutes

8–17 years

Present and lifetime

Child and parent

Semistructured

Pictorial Instrument for Children and Adolescents (PICA-III-R)

Ernst, Cookus, & Moravec (2000)

40–60 minutes

6–16 years

Present

Child

Semistructured (pictorial interview)

Schedule for Affective Disorders and Schizophrenia for School-aged Children (K-SADS)

Ambrosini 90 (2000); minutes Ambrosini, Metz, Prabucki, & Lee (1989)

6–18 years

Present and Child and lifetime parent (contained in separate and combined interview formats)

Semistructured

Note. Time reflects estimated length of parent interview with the exception of the Dominic-R and PICA-III-R, which are exclusively child self-report. Data from Table 12.1 of Kamphaus and Frick (2002).

reyn2-10.qxd

6/6/2003

9:14 AM

Page 237

10. Structured Diagnostic Interviewing

ed later in the chapter, interview questions generally start with a stem question (e.g., Have you been involved in many physical fights?) followed by a series of follow-up or contingency questions to define relevant parameters such as frequency (e.g., How many fights have you been in the past year?), severity (e.g., Have you ever used a weapon in a fight?), duration (e.g., When was the first time you got in trouble for fighting?), and impairment (e.g., Has fighting caused problems for you at school, home, or with kids your age?). Due to the stem and followup format, the length of time it takes to administer a diagnostic interview depends heavily on the child being assessed. More symptomatic children require more interview time due to the additional follow-up questions. However, most interviews take between 60 and 90 minutes to administer. In addition to a commonality in format, most interviews have similar content. This content is typically based on one or more of the most recent versions of the Diagnostic and Statistical Manual of Mental Disorders, starting with the third edition, DSM-III (American Psychiatric Association, 1980), and continuing through its more recent revisions, DSM-IV (American Psychiatric Association, 1994) and DSM-IV-TR (American Psychiatric Association, 2000). A number of the structured interviews also allow for diagnoses based on the most recent version of the International Classification of Diseases, ICD-10 (World Health Organization, 1993) coding system (e.g., Diagnostic Interview Schedule for Children [DISC-IV] and the Child and Adolescent Psychiatric Assessment [CAPA]). The interviews contained in Table 10.1 are similar in their assessment of the most prevalent diagnostic categories in child and adolescent samples. All provide some coverage of anxiety, mood, and externalizing diagnostic categories. The majority also allow for an assessment of schizophrenia, substance use, elimination, and eating disorders. Tic disorders are covered exclusively by the CAPA, DISC-IV, and K-SADS, whereas the Anxiety Diagnostic Interview for DSM-IV (ADIS) contains unique screening questions for mental retardation, learning disorders, and somatoform disorders. The CAPA is unique in its detailed assessment of sleep disorder symptoms. Finally, the Dominic-R and Pictorial Instrument for

237

Children and Adolescents (PICA-III-R) are picture-based interviews developed for use with younger children (see Valla, Bergeron, & Smolla, 2000), and these interviews tend to be more limited in the breadth of content (e.g., no tic or elimination disorder items) than other interviews. Most interviews organize questions by diagnostic category. One notable exception is the Interview Schedule for Children and Adolescents (ISCA) which uses a symptomoriented interview format in which items are clustered by content (e.g., impaired concentration) and topic area (e.g., mental status) rather than specific diagnostic criteria. To promote multi-informant assessments, most interviews contain parallel forms to ask identical questions of both the child and parent. There are some exceptions to this parallel format. For example, the CAPA only includes attention-deficit/hyperactivity disorder (ADHD) items on the parent interview, given the questionable validity of the self-report for these symptoms. In addition, the pictorial interviews do not contain separate parent report formats. The DISC-IV is unique in the inclusion of an experimental teacher version (DISC-T) designed to tap the disruptive behavior and internalizing symptoms most likely to be observed in a school setting. The DISC-T was used in the DSM-IV field trials for the disruptive behavior disorders (Frick et al., 1994). The DISC was also one of the first interviews to experiment with a computer-assisted administration format. Currently, a number of the interview schedules have such formats including the DISC-IV, the Diagnostic Interview for Children and Adolescents (DICA), the CAPA, and the DominicR. The computer-assisted format was designed to enhance the reliability and ease of administration and data collection. Using this format, the examiner reads items from the computer screen and enters the patient’s responses. The computer quickly scores and stores responses, selects the appropriate follow-up questions, and skips out of diagnostic sections when failing to meet inclusionary criteria. There are now attempts to develop and test the usefulness of selfadministered computerized interviews (e.g., DICA; Reich, 2000). Most of the structured diagnostic interviews were designed to assess children and

reyn2-10.qxd

238

6/6/2003

9:14 AM

Page 238

III. INTERVIEWING AND OBSERVATIONS

adolescents between the ages of 8 and 17. Some interviews report applicability to younger children. However, there is some evidence that the reliability of children’s self-report on diagnostic interviews is low before age 9 (Edelbrock, Costello, Dulcan, Kalas, & Conover, 1985; Ernst, Cookus, & Moravec, 2000; Hodges & Zeman, 1993). To increase the usefulness of structured interviews for assessing younger children, several interviews have been developed that involve pictorial content to either replace or augment the typical question-and-answer format for structured interviews. One example of this approach is the Dominic-R developed by Valla and colleagues (Valla et al., 2000). The Dominic-R is designed to assess DSM-III-R criteria for anxiety disorders, mood disorders, and the disruptive behavior disorders in children ages 6–11. The interview involves pictures of a child named Dominic facing situations that are common in children’s daily life. The pictures are accompanied by written questions that are read by the examiner (e.g.,”Do you feel sad and depressed most of the time, like Dominic?”). The reliability of the Dominic-R was assessed in a sample of 340 community children ages 6–11 and it revealed reliability coefficients that were much improved over other structured interviews with young children. Specifically, test–retest reliability over 7–12 days for diagnoses from the DominicR ranged from a kappa of .44 to a kappa of .69 with most being above .60 (Valla, Bergeron, Bidaut-Russell, St-Georges, & Gaudet, 1997). Also, the diagnoses based on the Dominic-R were strongly associated with diagnoses made by experienced clinicians with kappa values ranging from .64 to .88 (Valla et al., 2000). Therefore, it appears that the combination of pictorial and verbal stimuli enhanced the reliability and validity of responses in young children.

MAJOR SOURCES OF VARIATION ACROSS INTERVIEW SCHEDULES Despite these many similarities, there are some important differences among structured interview formats that influence their suitability for specific testing situations. First, there are differences in the degree of structure required by the interview sched-

ule. All the interviews provide some degree of structure and give guidelines for standardized administration and scoring. However, the degree of structure varies across interviews. The distinction has been made between structured versus semistructured interviews to reflect these differences. The DISC-IV, ADIS, and Children’s Interview for Psychiatric Symptoms (ChIPS) are examples of highly structured interviews. These interviews, typically designed to be administered by trained lay examiners, require that questions are administered in a predetermined sequence with little room for interpretation of responses. For example, the manual for the administration of the DISC-2.3 includes following directions: The DISC-2.3 symptom questions are designed to be read exactly as written. There is very limited scope for independent questioning. DO NOT deviate from the prescribed question sequence. DO NOT make up your own questions because you think you have a better way of getting at the same information, or because you think the question is poorly worded (Fisher, Wicks, Shaffer, Piacentini, & Lapkin, 1992, p. 3)

In contrast, the ISCA, K-SADS, and CAPA are examples of semistructured interviews. These interviews provide greater flexibility in terms of wording and interpretation of a child or parent’s responses and generally require highly trained clinicians for administration. For example, the ISCA allows for the reordering of items and provides a systematic means for the integration of clinician impressions into the diagnostic summary. Similarly, the K-SADS and CAPA encourage the examiner to go beyond symptom prompts and ask as many questions as necessary to determine the presence or absence of relevant symptoms. Another major variation among the structured interviews is the time frame used to assess symptoms and diagnoses. All the interviews assess whether problems are currently evident. This is called a “present episode” frame of reference. Most interviews consider present episodes as being within the last 6 months, although in some instances the time frame may be as short as within the last 2 weeks (ISCA for emotional disorders) or as long as within the last year (DISC-IV for conduct disorder). Of note,

reyn2-10.qxd

6/6/2003

9:14 AM

Page 239

10. Structured Diagnostic Interviewing

the CAPA restricts the assessment of symptoms to the past 3 months due to concerns with the reliability of memory in children and adults over longer time intervals. Similarly, the Dominic-R does not obtain any parameter information such as onset and duration of symptoms because it was developed for use with very young children who may not be able to report on such parameters. Nevertheless, the major source of variation is whether or not interview schedules are limited to present episodes. A number of interviews restrict the focus of assessment to the present episode time frame (e.g., CAPA and ChIPS). However, an increasing number of interviews provide for the assessment of both present and “lifetime” diagnoses. For example, the DISC-IV provides a newly incorporated whole-life module assessing for whether or not a child has exhibited symptoms of diagnoses since age 5 but prior to the current year. Similarly, there are lifetime formats for both the ISCA and the K-SADS. The DICA is unique in its exclusive focus on lifetime diagnoses. A third source of variation within the interview schedules is in their answer format. For most interviews, the interview responses are categorically coded as either “yes” or “no.” This categorical format is consistent with the DSM diagnostic definitions in which symptoms are considered either present or absent. In contrast, the ADIS, CAPA, ISCA, and K-SADS have Likert-type scales that allow for symptom severity ratings. While this format makes it more difficult to translate responses into DSM diagnoses, it does not create an artificial

239

dichotomy between the presence or absence of a symptom and allows symptom scores to reflect gradations in severity.

EVALUATION OF STRUCTURED DIAGNOSTIC INTERVIEWS Advantages Structured interviews share with behavior rating scales the goal of obtaining a detailed description of a child’s emotions and behaviors from multiple informants. The logical question is: What advantage do the timeconsuming structured interviews offer in comparison to the more time-efficient behavior rating scales? Table 10.2 provides a summary of the strengths and weaknesses of structured diagnostic interviews. The primary advantage of structured interviews is their usefulness in obtaining important parameters of a child’s behavior that are not typically assessed by most behavior rating scales. Specifically, most interview schedules provide questions that assess the duration of a child’s behavioral difficulties and the age at which the problems began to emerge. This temporal information allows one to take a developmental perspective in understanding a case, a perspective that has proven to be crucial for assessing many forms of childhood psychopathology (e.g., Frick & Lahey, 1991; Lahey, Loeber, Quay, Frick, & Grimm, 1992). The temporal perspective offered by structured interviews also allows for the assessment of the contiguity and sequencing among different prob-

TABLE 10.2. Strengths and Weaknesses of Structured Diagnostic Interviews Strengths

Weaknesses

앫 More systematic, reliable, and comprehensive than the traditional (unstructured) clinical interview format 앫 Documentation of temporal information including onset, duration, and sequencing symptoms 앫 Strict adherence to current diagnostic manuals important for both clinical and research applications 앫 Assessment of symptom impairment 앫 Usefulness in training clinical assessors

앫 Time-consuming 앫 Overdependence on diagnostic criteria with weak empirical basis (e.g., ADHD impulsivity– hyperactivity symptoms for late adolescent interviewees) 앫 Lack of normative data 앫 Lack of teacher interview formata 앫 Potential for symptom attenuation effects both within and across administrations

a

The DISC-IV structured interview contains an experimental teacher format.

reyn2-10.qxd

240

6/6/2003

9:14 AM

Page 240

III. INTERVIEWING AND OBSERVATIONS

lem areas. For example, research has suggested that it is important in the assessment of childhood depression to determine whether periods of sadness occurred contiguously with other symptoms associated with depression, such as sleep disturbances, eating disturbances, or thoughts of death (Kazdin, 1988). Furthermore, how depressive symptoms are conceptualized and treated may be different if they precede the onset of behavioral difficulties as opposed to emerging only after a long-standing history of behavioral problems that has brought the child into conflict with many different persons and agencies (Frick, 1998). Another important parameter assessed by most structured interviews is the level of impairment associated with behaviors being reported. Most interviews have questions that assess the degree to which a child’s behavioral or emotional difficulties are affecting his or her functioning in major life arenas (e.g., at home, at school, and with peers). This type of information is critical because the level of impairment associated with any emotional or behavioral problems may be a better indicator of the need for treatment than symptom severity (Kamphaus & Frick, 2002). This point is further illustrated by the use of level of impairment as a critical part of defining a diagnostic threshold for most DSM disorders. Diagnostic interviews enhance the correspondence between assessment techniques and diagnostic criteria beyond the assessment of impairment. As mentioned previously, the content of most structured interviews were specifically designed and have been revised to correspond to the changing DSM system. This means that the usefulness of the interview is in part dependent on the usefulness of the diagnostic definitions that are being assessed. Some diagnostic categories have been poorly validated, and hence this dependency is not always beneficial to the assessment process. However, this tie between assessment and diagnosis could be advantageous for several reasons. First, it encourages revisions of the interviews to correspond with advances in our knowledge of the basic characteristics of child and adolescent psychological disorders, thereby increasing the integration of research and practice in diagnostic assessments (Frick, 2000). Second, it allows one to make a diag-

nosis based on strict adherence to diagnostic criteria. Due to theoretical, empirical, or practical (e.g., insurance reimbursement) reasons, many assessors attempt to make DSM diagnoses as a result of their assessments. Too often, diagnoses are made based on information (e.g., rating scales and projective tests) that does not directly assess the diagnostic criteria. As a result, the meaning of the resulting diagnosis is ambiguous. Finally, diagnostic interviews can be helpful in training clinical assessors. As assessors are developing their competence in interviewing, it is often helpful to have an explicit format from which to conduct the interview. It gives the assessor a good way to learn the basic characteristics of childhood emotional and behavioral disorders. After being trained in administration procedures and after conducting numerous interviews with actual clients, assessors often begin to internalize the diagnostic criteria for the most common disorders of childhood. This knowledge can then be applied in situations in which a structured interview may not be possible.

Disadvantages Two weaknesses in the use of diagnostic interviews have already been mentioned. These are the time-consuming nature of the interviews and the dependence of the interviews on the DSM criteria, which is a strength for assessing well-validated syndromes but a weakness for assessing disorders with a weak empirical basis. Also, diagnostic interviews are subject to the potential reporter biases that can influence any assessment system that relies on the perceptions of reporters to assess emotions and behavior. These include intentional or unintentional biases, imperfect knowledge of a child’s behavior, inaccuracies in perceptions due to an immature cognitive level, and differences in raters’ standards for judging the severity of behavior (Kamphaus & Frick, 2002). A weakness that is more specific to structured interviews is the difficulty in making norm-referenced interpretations based on information provided by structured interviews. Clinically significant levels of symptoms are often based on DSM criteria rather than based on a comparison with a representative normative sample. Therefore, the ap-

reyn2-10.qxd

6/6/2003

9:14 AM

Page 241

10. Structured Diagnostic Interviewing

propriateness of clinical elevations for a given age depends on the appropriateness of the diagnostic criteria for that age. This could be problematic for diagnoses whose symptomology may vary across distinct developmental periods. For example, Barkley (1997) has questioned the utility of applying the same diagnostic criteria for ADHD across the early-childhood and adolescent years. Specifically, he suggests that (1) ADHD symptoms typically focus on early manifestations of the disorder that may not be as relevant for adolescents or young adults and (2) the number of symptoms in children with ADHD, especially of impulsivity and hyperactivity, typically decreases in adolescence. Therefore, using the same symptom list and diagnostic threshold may not be appropriate across developmental periods. Another weakness of most diagnostic interviews is the failure to provide a format for obtaining information from a child’s teacher. This source of information is crucial in the clinical assessment of elementary school-age children (Loeber, Green, Lahey, & Stouthamer-Loeber, 1991). As a result, information from teachers must be obtained by some other method, thereby making it difficult to determine whether discrepancies between a teacher’s report and the report of others are due to real differences in a child’s classroom behavior or to differences in the assessment formats. Finally, structured interviews have limitations if repeated administrations are required. Several studies have documented that fewer symptoms are typically reported on diagnostic interviews in each subsequent administration of the interview (Jensen, Watanabe, & Richters, 1999; Piacentini et al., 1999). For example, Piacentini and colleagues (1999) found that when the DISCIV was readministered to the same sample of 245 parent–child pairs (ages 9–18) 12 days later, parent-reported symptoms dropped 42% and child reported symptoms dropped 58%. This problem is primarily an issue in attempts to establish the test–retest reliability for an interview or in longitudinal research assessing changes in diagnostic status over time. However, there are some applied contexts in which repeated administrations of a diagnostic assessment are also required, such as monitoring treatment progress. Such symptom attenuation across

241

administration can greatly affect interpretations in such contexts. More important, however, is that there is also evidence that the number of symptoms reported declines within an interview schedule, such that parents and children tend to report more symptoms for diagnoses assessed early in the interview, irrespective of which diagnoses are assessed first (Jensen et al., 1999). This type of symptom attenuation within an interview is of much greater concern because it clearly can influence the results from typical clinical uses of structured interviews. A number of hypotheses have been offered to explain this attenuation of symptoms across administrations and within the interview. These hypotheses include (1) the interviewee becoming sensitized to clinical issues that could result in a heightened threshold for symptom reporting, (2) a circumscribed focus of interviews on only the interval between assessment periods, (3) statistical regression to the mean, and (4) knowledge that denying stem questions will lead to fewer follow-up questions and, thus, shorten the duration of the interview (Piacentini et al., 1999). In an attempt to modify the administration of the DISC-2.3 to produce more stable symptom endorsements that are less susceptible to attenuation effects, Edelbrock, Crnic, and Bohnert (1999) (1) provided a more detailed introduction to the interview with an overview of areas to be covered, (2) provided definitions for key concepts used throughout the interview, and (3) employed a flexible order of assessing problems by allowing parents to select the order in which diagnostic modules were administered. With such administration modifications, these authors found little attenuation of reporting at either the symptom or diagnostic level both within the interview administration and across repeated administrations. These findings clearly suggest that alternative administration strategies need to be tested in an effort to reduce symptom attenuation. However, none of the standardized administration procedures that accompany structured interview schedules provide for this type of administration, and, therefore, the possible reduction in number of symptoms reported for disorders assessed later in the interviews, or in repeated administrations of the same interview, must be considered when interpreting the results for individual children and adolescents.

reyn2-10.qxd

242

6/6/2003

9:14 AM

Page 242

III. INTERVIEWING AND OBSERVATIONS

RECOMMENDATIONS FOR USE OF STRUCTURED INTERVIEWS Based on the strengths and weaknesses of structured interviews, several recommendations can be made regarding the appropriate use in the clinical assessment of children and adolescents. First, like any assessment technique, the diagnostic interview should never be used alone in a clinical evaluation. It should be one part of a comprehensive assessment battery. Information from diagnostic interviews should be supplemented by assessment techniques that provide better norm-referenced scores (e.g., behavior rating scales) and by assessment techniques that provide information on a child’s classroom functioning (e.g., behavioral observations in the school). In addition, the diagnoses derived from the diagnostic interviews should be viewed within the context of the overall assessment. A diagnosis can be viewed similarly to the way an elevation on a behavior rating scale is interpreted. Specifically, it is one piece of information that needs to be integrated with other sources of information to develop a good case formulation. Stated simply, “diagnoses” based solely on diagnostic interviews should not be considered final clinical diagnoses. Such final diagnoses should be based on an assessor’s integration of multiple sources of information. A child’s age is also an important consideration in the use of structured interviews. Generally the reliability for most interview schedules are low before the age of 9 for child self-report (Hodges & Zeman, 1993). It seems that the structured, face-to-face dialogue is not appropriate for assessing very young children. Although the parent versions of the interview are appropriate for young children, children’s self-report should be obtained by other methods. This recommendation may change in the near future as research has begun to address the use of alternative (i.e., briefer and more simplistic) wording and visual prompts in extending the utility of interview formats to younger age ranges. A final consideration in using structured interviews concerns when to administer diagnostic interviews in the assessment battery. There is no research to address the differential utility of interviews at different

points in an assessment battery. However, based on clinical experience in using diagnostic interviews, these assessments should not be the first assessment administered to parents. The structured format does not facilitate the development of rapport between the interviewer and parent, and some parents become frustrated in trying to fit their descriptions of their child’s behavior into the confines of the interview questions. Therefore, it is often helpful to precede diagnostic interviews with less structured questions that allow parents to express their concerns in their own words. For children and adolescents, the structured format may actually enhance rapport in some cases. Children often enter the assessment situation nervous because they are unsure about what is expected of them. The clear and explicit response format of diagnostic interviews makes the demands of the situation apparent for the child and thereby reduces his or her anxiety in many cases. Up to this point, we have tried to give an overview of structured interviews by comparing the various formats that are available for assessing children and adolescents, highlighting some of the major advantages and disadvantages of using interview schedules in a clinical assessment, and providing guidelines for appropriate use. The next section provides a more in-depth description of one particular interview schedule, the DISCIV, to more clearly illustrate the structure, administration, and interpretation of structured interviews. We chose the DISC-IV as an example of a structured interview because it is one of the most widely used interview schedules for children and adolescents and it has been one of the most systematically developed. However, the DISC-IV is one of the most structured interview schedules, and therefore it has all the advantages and disadvantages that accompany a high degree of structure.

FOCUS ON THE NIMH DIAGNOSTIC INTERVIEW SCHEDULE FOR CHILDREN Development The original version of the DISC (DISC-1; Costello, 1983; Costello, Edelbrock, Dulcan, Kalas, & Klaric, 1984) was designed

reyn2-10.qxd

6/6/2003

9:14 AM

Page 243

10. Structured Diagnostic Interviewing

to be a downward extension of the adultoriented DIS (Robins et al., 1981). The DISC-1 was developed as part of an initiative by the NIMH Division of Biometry and Epidemiology that focused on obtaining a greater understanding of the prevalence of childhood mental disorders (Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000). The DISC-1 was designed for use in epidemiological studies and was explicitly tied to the version of the DSM being used at the time (i.e., DSM-III; American Psychiatric Association, 1980). In 1985, Dr. David Shaffer at the New York State Psychiatric Institute and his colleagues undertook a revision of the interview to (1) improve its reliability for use with children and for use by lay interviewers and (2) provide diagnostic compatibility with the DSM-III-R (American Psychiatric Association, 1987) and anticipated DSM-IV and ICD-10 criteria (Fisher et al., 1992). Modifications to the DISC have been greatly informed by field-testing conducted as part of a large NIMH-funded multisite study titled the Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) study. The current version of the DISC, the DISC-IV (Shaffer et al., 2000) measures approximately 30 diagnoses of childhood and adolescence and is fully compatible with the DSM-IV (American Psychiatric Association, 1994) and ICD-10 (World Health Organization, 1993) classification systems.

Structure and Content The DISC-IV (Shaffer et al., 2000) contains 358 core questions and approximately 1,300 questions that are asked contingent upon a child’s responses to the core questions. There are two parallel versions of the DISC-IV, the youth version (DISC-Y) to be administered to children between the ages of 9 and 17 and the parent version (DISC-P) to be administered to the parents of children ages 6–17. There is also an experimental teacher version (DISC-T), which was developed for use in the DSM-IV field trials (Frick et al., 1994). The DISC-IV was designed to focus primarily on current psychological functioning. It assesses for symptoms occurring within two overlapping time intervals: the past 12 months and the past 4

243

weeks. The DISC-IV contains an optional whole-life module designed to measure symptoms occurring as early as age 5. An alternative present-state version has been developed targeting only the past 4-week time interval (see Shaffer et al., 2000). The DISC-IV is organized in “diagnostic modules.” There are six modules that comprise sets of related disorders. These include anxiety disorders, mood disorders, psychotic disorders, disruptive behavior disorders, substance use disorders, and miscellaneous disorders (i.e., anorexia/bulimia nervosa, elimination, tic, pica, and trichotillomania). For each diagnosis within these six modules, the DISC-IV is designed to obtain information about the presence of symptoms included in DSM criteria. If a certain threshold is met, usually below the DSM diagnostic threshold, the questions regarding the age of first onset, impairment, and past treatment are asked.

Administration The DISC-IV was designed to be administered by interviewers without clinical experience after approximately 2 to 6 days of training. Use of the computerized DISC-IV is accompanied by less stringent training requirements. Training includes (1) instruction on standard DISC-IV administration procedures, (2) viewing an actual administration of the DISC-IV, and (3) supervised practice in administration with a confederate in a controlled situation. At the beginning of the interview, the interviewer completes an introductory module consisting of several pieces of demographic information (e.g., age and sex of child) that are necessary to properly administer the interview. The interviewer also establishes a time line with the interviewee to assist in triggering recall for the onset and duration criteria contained in the interview. The time line establishes salient events (e.g., birthdays, vacations, start of the school year, and holidays) that occurred in the year preceding the interview. This anchor helps the child or parent remember the time frame for diagnostic questions. The verbal instructions given to the respondent are semistructured. That is, several topics that must be covered are outlined in the manual, but verbatim questions for

reyn2-10.qxd

6/6/2003

244

9:14 AM

Page 244

III. INTERVIEWING AND OBSERVATIONS

obtaining the information are not provided. The points include the following: 1. There are no right or wrong answers; the best answer is the one that tells the most about the child. 2. The informant should try to answer yes or no to each question. 3. The time frame is within the last year, unless otherwise specified. 4. Some of the questions on the form will be left out. 5. Some questions may be asked more than once. 6. It is possible to take breaks, if needed. Unlike the instructions, the administration of the actual DISC-IV questions is quite structured. The questions are designed to be read exactly as written and in the sequence prescribed. Interviewers are explicitly instructed not to make up their own questions or to ask for an example unless requested in the interview format. If a respondent does not understand the question, the interviewer should repeat the question, emphasizing the words that seemed to cause confusion. The interviewer is not allowed to interpret questions for the respondent (e.g., What do you mean by often? or Is one or two times considered frequent?). The interviewer is instructed to simply ask the respondent to interpret the question “whichever way he or she thinks is best.”

Reliability Psychometric information specific to the DISC-IV is minimal. This is the case with all DSM-IV compatible interview schedules leading to a heavy reliance on information gained from research using prior versions of the interview. Given the similar structure and content for all of the most recent versions of the DISC, it is likely that data from older versions (i.e., DISC-2.3) are likely to be relevant for evaluating the newer DISCIV. In addition, information on the test–retest reliability of the DISC needs to be interpreted in light of the problem common to most structured interviews that was discussed previously. Specifically, fewer symptoms are often reported on structured interviews in repeated administrations,

thereby reducing the test–retest correlations of scores. Within the context of these limitations, reliability data on the DISC-2.3 were reported in a series of articles by Shaffer and colleagues (Piacentini et al., 1993; SchwabStone et al., 1993; Shaffer et al., 1993). These authors tested the psychometric properties of the DISC-2.3 in a sample of 75 clinic-referred children ages 11–17. In 41 cases, the child and/or parent were reinterviewed 1 to 3 weeks later by a second interviewer. There were sufficient cases to calculate the test–retest reliability for five DSM-III-R diagnoses (ADHD, oppositional defiant disorder, conduct disorder, major depression, and separation anxiety disorder). On a diagnostic level, there was relatively high test–retest agreement for the parent interview (mean kappa = .62), the child interview (mean kappa = .55), and combined parent and child interview (mean kappa = .63). The one exception was the low reliability of the oppositional defiant disorder (ODD) diagnosis by child report (kappa = .16). The symptoms for this diagnosis also exhibited the lowest intraclass correlation (ICC) according to child report (ICC = .44) compared to mean ICC values of .83 (parent report), .62 (child report), and .73 (combine report). Most of the symptom clusters showed moderate to high internal consistency with the exception of conduct disorder (CD) symptoms (coefficient alpha of .56 parent report and .59 for child report). The low internal consistency of the CD symptoms is not surprising as only three symptoms are required for the diagnosis of CD and most of these symptoms tend to be relatively low base rate. Although these reliability data are based on an earlier version of the DISC, the DISC2.3, Shaffer and colleagues (2000) provide initial reliability estimates for the DISC-IV from a sample of 84 parents and 82 children (ages 9–17) selected from several outpatient psychiatric clinics. These data were derived from the computer-administered version of the DISC-IV, the C-DISC-IV. Interviews were conducted by lay interviewers with an average retest interval of 7 days. The preliminary findings are consistent with the results for the DISC-2.3 with kappa coefficients ranging from .43 (CD) to .96 (specific phobia) for the parent report and from .25

reyn2-10.qxd

6/6/2003

9:14 AM

Page 245

10. Structured Diagnostic Interviewing

(simple phobia) to .92 (major depressive episode) for the child report.

Validity Validity of diagnostic interviews is often assessed by comparing the results of structured interviews to diagnoses made by experienced clinicians. For example, Piacentini and colleagues (1993) reported moderate to strong agreement between the results of DISC interviews and clinician diagnoses when the parent DISC-2.3 was used (average kappa = .50) but low agreement based on the child DISC-2.3 (average kappa = .34). Combining the two interviews gave agreement estimates between those of either informant alone (average kappa = .41). These authors reported that most of the cases with disagreements between clinician diagnoses and the DISC-2.3 were cases that were close to the diagnostic threshold. For example, several disagreements emerged in which children had seven symptoms of ADHD (rather than the required eight symptoms in DSM-III-R criteria) and were not given the diagnosis according to the DISC-2.3 but were given the diagnosis of ADHD by the clinician. Friman and colleagues (2000) provided a unique test of the predictive validity of the DISC by comparing interview data to behavioral observations recorded in a residential treatment program. The researchers examined both convergent and discriminant validity across a lengthy time interval (i.e., 1 year). Validity data were obtained on 369 children (ages 9–17), who were administered a computerized version of the DISC-Y 2.3 upon enrollment in the residential program and at 1-year follow-up. Diagnoses of both ODD and CD were compared to daily observations of disruptive behavior that were coded by program staff and summed to form monthly behavior ratings of both oppositional and conduct problem behaviors. Youth meeting criteria for a DISC-2.3 diagnosis of ODD or CD upon enrollment exhibited significantly greater observed behavior difficulties upon program entry than did youth not meeting a diagnosis for either disorder. Furthermore, change in diagnostic status across the two assessment periods predicted changes in observed disruptive be-

245

havior across the same time interval. For example, youth who met criteria for an ODD/CD diagnosis at time 1 but not at time 2 were characterized by a downward pattern of observed antisocial behavior in the months separating the interviews. This is contrasted with youth whose observed antisocial behavior increased as they moved from no diagnosis at time 1 to diagnosis at time 2. As further evidence for the validity of the DISC interview, Edelbrock and Costello (1988) found strong associations between the diagnoses of attention deficit disorder, CD, and depression/dysthymia from the DISC-P and the Hyperactive, Delinquent, and Depressed scales of the Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983) in a sample of 270 clinic-referred children between the ages of 6 and 16. High rates of agreement were also found between the original DISC-P and the CBCL in another study of 40 psychiatric referrals and 40 pediatric referrals (Costello, Edelbrock, & Costello, 1985). In contrast, the relation between the CBCL and the child version of the DISC tended to be much lower. However, it is impossible to determine whether the low correlations with the DISC-C were due to differences in informants (parent-competed CBCL and child-respondent DISC) or to differences in the assessment instruments themselves.

CONCLUSIONS Structured diagnostic interviews have become an important part of many clinical assessments of children and adolescents. Like behavior rating scales, diagnostic interviews provide a reliable means of assessing a child’s emotional and behavioral functioning. In this chapter, we have attempted to highlight the advantages and disadvantages of using diagnostic interviews. Diagnostic interviews enhance clinical assessments by providing information on several important parameters related to a child’s symptom presentation. For example, there are questions pertaining to how long a child’s problems have been occurring, for determining the temporal sequencing of behaviors, and for estimating the degree of impairment associated with a

reyn2-10.qxd

6/6/2003

246

9:14 AM

Page 246

III. INTERVIEWING AND OBSERVATIONS

child’s emotional or behavioral difficulties. These parameters are often not assessed by other assessment modalities. In addition, diagnostic interviews are typically tied to the most recent revisions of the DSM. On the negative side, diagnostic interviews are often time intensive and typically do not provide any norm-referenced information on a child’s functioning above that which is accorded by DSM criteria. In addition, diagnostic interviews typically do not include a format for obtaining information from a child’s teacher and their reliability in obtaining self-report information for young children (below age 9) is somewhat questionable. Finally, the administration format of the structured interview typically leads to fewer symptoms being reported if an interview is repeated and to fewer symptoms being reported for symptoms later within a given interview. As a result of these weaknesses, diagnostic interviews are best used as part of a more comprehensive assessment battery. We have attempted to provide guidelines for their use in this capacity. We have also attempted to provide an overview of the most commonly used diagnostic interviews for children and adolescents, highlighting the major commonalities and differences across interviews. We concluded the chapter with a more detailed discussion of the DISC-IV as an example of one of the most well-established structured interviews used to assess children and adolescents.

REFERENCES Achenbach, T. M., & Edelbrock, C. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington: University of Vermont. Albano, A. M., & Silverman, W. K. (1996). Anxiety Disorders Interview Schedule for DSM-IV: Clinician manual. New York: Psychological Corporation. Ambrosini, P. J. (2000). Historical development and present status of the schedule for affective disorders and schizophrenia for school-age children (KSADS). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 39–48. Ambrosini, P. J., Metz, C., Prabucki, K., & Lee, J. (1989). Video tape reliability of the third revised edition of the K-SADS. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 723–728. American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author.

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorder, DSM-IVTR: Text revision. Washington, DC: Author. Angold, A., & Costello, J. (2000). The child and adolescent psychiatric assessment (CAPA). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 49–58. Barkley, R. A. (1997). ADHD and the nature of selfcontrol. New York: Guilford Press. Costello, A. J. (1983). The NIMH diagnostic interview schedule for children. Pittsburgh, PA: University of Pittsburgh. Costello, E. J., Edelbrock, C. S., & Costello, A. J. (1985). Validity of the NIMH Diagnostic Interview Schedule for Children: A comparison between psychiatric and pediatric referrals. Journal of Abnormal Child Psychology, 13, 579–595. Costello, A. J., Edelbrock, C. S., Dulcan, M. D., Kalas, R., & Klaric S. H. (1984). Report of the NIMH Diagnostic Interview Schedule for Children (DISC). Washington, DC: National Institute of Mental Health. Edelbrock, C., & Costello, A. J. (1988). Convergence between statistically derived behavior problem syndromes and child psychiatric diagnoses. Journal of Abnormal Child Psychology, 16, 219–231. Edelbrock, C., Costello, A. J., Dulcan, M. K., Kalas, R., & Conover, N. C. (1985). Age differences in the reliability of the psychiatric interview of the child. Child Development, 56, 265–275. Edelbrock, C., Crnic, K., & Bohnert, A. (1999). Interviewing as communication: An alternative way of administering the diagnostic interview schedule for children. Journal of Abnormal Child Psychology, 27, 447–453. Endicott, J., & Spitzer, R. L. (1978). A diagnostic interview: The Schedule for Affective Disorders and Schizophrenia. Archives of General Psychiatry, 35, 57–63. Ernst, M., Cookus, B. A., & Moravec, B. C. (2000). Pictorial instrument for children and adolescents (PICA-III-R). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 94–99. Feighner, J. P., Robins, E., Guze, S. B., Woodruff, R. A., Winokur, G., & Munoz, R. (1972). Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry, 26, 57–63. Fisher, P., Wicks, J., Shaffer, D., Piacentini, J., & Lapkin, J. (1992). NIMH Diagnostic Interview Schedule for Children: Users manual. New York: New York State Psychiatric Institute. Frick, P. J. (1998). Conduct disorders and severe antisocial behavior. New York: Plenum Press. Frick, P. J. (2000). Laboratory and performance-based measures of childhood disorders. Journal of Clinical Child Psychology, 29, 475–478. Frick, P. J., & Lahey, B. B. (1991). The nature and characteristics of attention-deficit hyperactivity disorder. School Psychology Review, 20, 163–173.

reyn2-10.qxd

6/6/2003

9:14 AM

Page 247

10. Structured Diagnostic Interviewing Frick, P. J., Lahey, B. B., Applegate, B., Kerdyck, L., Ollendick, T., Hynd, G. W., Garfinkel, B., Greenhill, L., Biederman, J., Barkley, R. A., McBurnett, K., Newcorn, J., & Waldman, I. (1994). DSM-IV field trials for the disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 33, 529–539. Friman, P. C., Handwerk, M. L., Smith, G. L., Larzelere, R. E., Lucas, C. P., & Shaffer, D. M. (2000). External validity of conduct and oppositional defiant disorders determined by the NIMH diagnostic interview schedule for children. Journal of Abnormal Child Psychology, 28, 277–286. Hodges, K., & Zeman, J. (1993). Interviewing. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child and adolescent assessment (pp. 65–81). Needham Heights, MA: Allyn & Bacon. Jensen, P. S., Watanabe, H. K., & Richters, J. E. (1999). Who’s up first? Testing for order effects in structured interviews using a counterbalanced experimental design. Journal of Abnormal Child Psychology, 27, 439–445. Kamphaus, R. W., & Frick, P. J. (2002). Clinical assessment of child and adolescent personality and behavior (2nd ed.). Boston: Allyn & Bacon. Kazdin, A. E. (1988). Childhood depression. In E. J. Mash & L. G. Terdal (Eds.), Behavioral assessment of childhood disorders (2nd ed., pp. 157–195). New York: Guilford Press. Kovacs, M. (1985). The Interview Schedule for Children (ISC). Psychopharmacology Bulletin, 21, 991–994. Lahey, B. B., Loeber, R., Quay, H. C., Frick, P. J., & Grimm, J. (1992). Oppositional defiant disorder and conduct disorders: Issues to be resolved for DSM-IV. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 539–546. Last, C. G. (1987). Developmental considerations. In C. G. Last & M. Hersen (Eds.), Issues in diagnostic research (pp. 201–216). New York: Plenum Press. Loeber, R., Green, S. M., Lahey, B. B., & StouthamerLoeber, M. (1991). Differences and similarities between children, mothers, and teachers as informants on disruptive child behavior. Journal of Abnormal Child Psychology, 19, 75–95. McClellan, J., & Werry, J. S. (2000). Introduction to special section: Research psychiatric diagnostic interviews for children and adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 19–27. Piacentini, J., Robper, M., Jensen, P., Lucas, C., Fisher, P., Bird, H., Bourdon, K., Schwab-Stone, M., RubioStipec, M., Davies, M., & Dulcan, M. (1999). Informant-based determinants of symptom attenuation in structured child psychiatric interviews. Journal of Abnormal Child Psychology, 27, 417–428. Piacentini, J., Shaffer, D., Fisher, P., Schwab-Stone, M., Davies, M., & Gioia, P. (1993). The Diagnostic Interview Schedule for Children—Revised version (DISC-R): III. Concurrent criterion validity. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 658–665.

247

Reich, W. (2000). Diagnostic interview for children and adolescents (DICA). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 59–66. Reich, W., Herjanic, B., Welner, Z., & Gandhy, P. R. (1982). Development of a structured psychiatric interview for children: Agreement in diagnosis comparing child and parent interviews. Journal of Abnormal Child Psychology, 10, 325–336. Robins, L. N., Helzer, J. E., Croughan, J., & Ratcliff, K. S. (1981). The National Institute of Mental Health Diagnostic Interview Schedule: Its history, characteristics, and validity. Archives of General Psychiatry, 38, 381–389. Schwab-Stone, M., Fisher, P., Piacentini, J., Shaffer, D., Davies, M., & Briggs, M. (1993). The Diagnostic Interview Schedule for Children—Revised version (DISC-R): II. Test–retest reliability. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 651–657. Shaffer, D., Fisher, P., Lucas, C. P., Dulcan, M. K., & Schwab-Stone, M. E. (2000). NIMH diagnostic interview schedule for children version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 28–38. Shaffer, D., Schwab-Stone, M., Fisher, P., Cohen, P., Piacentini, J., Davies, M., Conners, C. K., & Regier, D. (1993). The Diagnostic Interview Schedule for Children—Revised version (DISC-R): I. Preparation, field testing, interrater reliability, and acceptability. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 643–650. Sherrill, J. T., & Kovacs, M. (2000). Interview schedule for children and adolescents (ISCA). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 67–75. Silverman, W. K., & Albano, A. M. (1996). The Anxiety Disorders Interview Schedule for DSM-IV: Child version. New York: Psychological Corporation. Valla, J. P., Bergeron, L., Bidaut-Russell, M., StGeorges, M., & Gaudet, N. (1997). Reliability of the Dominic-R: A young child mental health questionnaire combining visual and auditory stimuli. Journal of Child Psychology and Psychiatry, 38, 717–724. Valla, J., Bergeron, L., & Smolla, N. (2000). The Dominic-R: A pictorial interview for 6- to 11-year-old children. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 85–93. Weller, E. B., Weller, R. A., Fristad, M. A., Rooney, M. T., & Schecter, J. (2000). Children’s interview for psychiatric syndromes (ChIPS). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 76–84. World Health Organization. (1993). The ICD-10 classification of mental and behavioural disorders: diagnostic criteria for research. Geneva: World Health Organization.

reyn2-11.qxd

6/6/2003

9:17 AM

Page 248

11 Direct Behavioral Observation for Classrooms

ANNE PIERCE WINSOR

This chapter provides an overview of the basic principles of direct observation for behavioral assessment purposes in the classroom, as well as a summary of the major advantages and limitations of its use. This overview discusses the important role that direct observation plays in addressing the recent requirement for functional behavior assessments of problematic student behavior. Also included is a brief review of three formal behavioral observation systems frequently chosen by schools for their ease of use and broad behavioral applicability. Observation of behavior is easily the oldest assessment method. As soon as mentor and apprentice established a learning interaction, a pattern of observation developed for assessment and feedback of learner performance. With time, direct observation of a child’s or adolescent’s overt behavior has become both an accepted and expected component in the psychoeducational assessment of youth. Classrooms provide a naturalistic environment for observation; observing behaviors displayed in the classroom give insight to student behavioral and academic adaptation to the schooling process. Observation is used frequently in schools for assessment purposes: for evaluation of teacher performance by administrators, for

evaluation of student academic skills and knowledge by teachers, and for applied contextual experience by preservice teachers. A frequent use of school-based observation is assessment of problematic student behaviors that interfere with the educational progress of a student, and in some cases, the progress of his or her classmates (Blakeslee, Sugai, & Gruba, 1994; Miller, Tansy, & Hughes, 1998). In fact, new legal regulations require schools to conduct functional behavioral assessments of problem behavior of which direct observation is a primary assessment component.

BEHAVIORAL OBSERVATIONS: ADVANTAGES AND LIMITATIONS Direct behavioral observation is often used interchangeably with the term “behavioral assessment” when, in fact, it is one important method among several methods (e.g., interviews and behavior rating scales) contributing to comprehensive behavioral assessment (Shapiro & Skinner, 1990). However, the validity of other assessment methods frequently is judged by the degree of correspondence with direct behavioral observations. Why are direct observations of 248

reyn2-11.qxd

6/6/2003

9:17 AM

Page 249

11. Direct Behavioral Observation for Classrooms

behavior so popular? First, observation makes intuitive sense; after all, it is a method of informal behavioral assessment frequently employed by people in daily social situations and, in a more formal sense, represents a cornerstone of scientific inquiry. Second, direct observations of behavior imply a purity of data; that is, information about behavior is reported objectively, “as is,” rather than filtered through the perceptions and biases of an informant, as in behavioral rating scales. Finally, direct observations of behavior allow for the assessment of environmental contingencies that contribute to the production, maintenance, and escalation of a student’s behavior, not unlike an “action shot” of the behavior in situ. Assessing behavior by observing it in a contextual framework like school can be informative in designing effective, practical behavioral interventions. Direct observation, like any assessment technique, has limitations. When done well by trained observers, direct observation can yield rich, quality information; however, it can be costly in labor, time, and money. Because of this strain on resources, direct observations are frequently excluded from behavioral assessment in schools and other contexts (Miller et al., 1998). In other cases, without rigorous and clear methodology, brief narratives are collected on student behavior; without an understanding for the potential introduction of biases into the collected data, these informal observations are often interpreted as if they were objective (Harris & Lahey, 1982). Also, high costs have deterred many formal observational systems from establishing reliability and validity of observational results in multiple samples; neither do they offer a normative sample that would allow comparison of a student’s results with those of a general sample (Kamphaus & Frick, 2002). Direct observations are additionally limited by reactivity, difficulty in observing an adequate sample of behaviors, and an in-

249

ability to observe internal events such as student thoughts and emotions. Reactivity refers to the phenomenon in which a subject will change his or her behavior when aware of being observed (Kazdin, 1981). The student’s behavior sample, as a result, may not be as representative or objective as desired for assessment purposes. Observers in classrooms should attempt measures that reduce student reactivity such as observing from inconspicuous locations and being present in the setting over multiple periods, allowing students to become used to their presence (Keller, 1986); however, reactivity remains a variable that potentially affects the results and, subsequently, the interpretation of observational data. Obtaining an adequate sample of behaviors is important in ensuring that the behaviors are both representative of a student’s usual school behavior and generalizable to other times and settings experienced by the student. To address this issue, observers should sample behavior under ecologically valid conditions (i.e., students engaged in regular school activities in school environments frequented by the student). In addition, observers should allow a large enough time frame, possibly over several observation sessions, to ensure opportunity for behaviors to occur. An overly short observational time frame risks missing the target behaviors or capturing nonrepresentative behavior because of variations in the environment (e.g., the class attended a field trip in the morning which has disrupted the daily schedule). Finally, behavioral observations focus on the assessment of overt behavior; direct observations do not allow for assessment of cognitive, emotional, or motivational contributors to student behaviors (Mash & Terdal, 1988). Research on child psychopathology supports the salience of these internal variables for assessment (Farrington, 1993) and treatment (Kendall & Braswell, 1985) of youth at risk for behavioral difficulties. Direct observations can provide important information to a comprehensive behavior assessment of student behavior. However, evaluators frequently ignore the limitations of direct observation, which should be weighed against its unique advantages. As the following section highlights, the best way to ensure representative results is to develop a methodologically sound observation system.

reyn2-11.qxd

6/6/2003

9:17 AM

250

Page 250

III. INTERVIEWING AND OBSERVATIONS

BEHAVIOR OBSERVATION SYSTEMS: BASIC PRINCIPLES Developing a system for direct observation in a school or any context involves several salient, interdependent decisions: what behaviors are to be observed; where will the observations take place; how the observations will be recorded and by whom.

Target Behaviors The first step in implementing an observation system involves the choosing and defining of target behaviors. Defining the behaviors to be observed will depend on the level of analysis needed (Barrios, 1993). Analysis can focus at the level of single, isolated behaviors (e.g., hitting other students and skipping class), at the level of behavior constellation or syndromes (e.g., aggressive behaviors and off-task behaviors), or at the level of interactions between two or more individuals (e.g., a teacher working with a group of students). Some observational systems are designed for the recording of social interactions such as how a child behaves in response to an adult’s behavior and how an adult responds to the child’s behavior (Gelfand & Hartmann, 1984). This level of analysis allows documentation of events/behaviors that elicit a student’s behavior (antecedents) and the responses (consequences) to the student’s behavior that serve to change or maintain it. The recording of antecedents and consequences of behavior is useful in determining the sequence of events in which the behavior is embedded and provides salient information on potential targets for intervention, which supports the overall goal of functional behavioral assessment. Once the focus of analysis has been chosen, the next step involves operationally defining the behavior(s) and, in some cases, the antecedents/consequences to be observed. These definitions should be specified in objective, clear terms to reduce the potential for misunderstanding and bias as well as to increase the reliability of the observations. For good reliability, observation systems must develop explicit definitions of each target behavior; without explicit definitions of behavior, an observer may employ a subjective or idiosyncratic definition, limiting the interpretations of the collected data

and compromising the primary advantage of direct observation—its objectivity. While behaviors such as “aggressive play,” “taking turns,” and “completes assignments” seem easily understood, decades of research have demonstrated that different observers do not necessarily code those behaviors in the same way (Kamphaus & Frick, 2002). For example, “completes assignments” might be more explicitly defined as “student sits in his assigned seat, attends to assignment, completes 100% of assignment by the end of current class period, and hands assignment to teacher before leaving the classroom,” thereby making the definition of “completion” clear to any observer. Typically, behavior in context is rich and complex; therefore, the number of target behaviors is often limited to a related class of behaviors (e.g., aggression manifested in hitting and yelling) or a particular interaction (e.g., teacher and student or student and cooperative group) because of the need for behaviors to be explicitly defined and the difficulty for an observer to focus on a large number of behaviors simultaneously.

Settings for School-Based Observations Once target behaviors are chosen and defined, the next step is determining the most appropriate location(s) to observe the target behaviors. Logically, one would choose settings in which the behaviors have been previously noted or seem most likely to occur. Students spend most of their time at school in classrooms so that teachers are often first to report behaviors that are disruptive or interfere with student learning. Therefore, the classroom provides a natural setting in which the targeted behaviors are most likely to be present. By contrast, clinic and laboratory settings often use analogue situations to simulate a particular context (“set the scene”), attempting to elicit target behaviors. In school settings, this is usually unnecessary, impractical, and lacking in ecological validity (Dunlap, Dunlap, Clarke, & Robbins, 1991). Anecdotal reports from teachers, school support staff, and students themselves can provide direction for setting selections. Schools have a surprising variety of subsettings: different classrooms with different teachers (employing different teaching and

reyn2-11.qxd

6/6/2003

9:17 AM

Page 251

11. Direct Behavioral Observation for Classrooms

management styles and expectations), cafeteria, school bus, playgrounds. Because many behaviors are situationally specific, student behavior often varies across activities, time, and place. Taking these factors into consideration, systematic observations should be made across situations to provide an adequate sample of student behavior as well as to determine which behaviors are specific to particular situations and which occur more generally (Merrell, 1994).

Collecting Observational Data The next step in designing a school-based behavioral observation system is to determine how to collect and code the target behaviors in the selected setting(s). Observational data can be recorded in a variety of ways; the choice of a recording procedure will depend on the characteristic duration and frequency of the target behaviors. Basic observational data collection methods are summarized into three general categories: event recording, duration recording, and time sampling. Event Recording Event recording involves documenting the number of times that the target behavior(s) occurs during preset time intervals or during an entire observation period. Observation periods can range from minutes to days, depending on the target behaviors and opportunities for observation. Easily implemented by minimally trained observers, event recording is the most frequently employed method of direct observation and is particularly useful for coding behaviors that occur briefly or for low-frequency behaviors that occur only once or twice during the observation session, such as hitting another student (Keller, 1986). Data collected are usually reported in terms of rate, where the total number of occurrences is divided by the amount of observation time (e.g., .25 verbal interruptions per minute). However, target behaviors must have a discrete beginning and end, such as raising a hand, hitting another student, or completing a math problem (Shapiro & Skinner, 1990). Target behaviors that are continuous, that persist over long duration, or have a high rate of occurrence, such as on- or off-task behaviors or social play, are difficult to code using

251

the event recording method because it is hard to distinguish the occurrence of one event from another. One way to maximize the utility of event recording is to record events sequentially. A useful and frequently employed technique uses the A–B–C procedure to organize the observations (Bergstrom & Crone, 2002; Sugai, 1996). This procedure involves a chart with three columns labeled, respectively, “antecedents,” “behaviors,” and “consequences.” The observer lists the target behaviors as they occur in the “behavior” column and then notes events and behaviors that preceded the target behaviors (antecedents) and those that followed (consequences). Though this procedure can be somewhat cumbersome, it provides ecologically sensitive data with potentially strong implications for intervention (Merrell, 1994). An example that illustrates its merits concerns a student who is reported as noncompliant by his teacher. The observer notes that the student engages in off-task behaviors such as talking to others, drawing, and out-of-seat wandering (target behaviors) each time the classroom teacher distributes seat-work assignments (antecedent), resulting in increased negative individual teacher attention such as nagging and threatening (consequences). Finally, the student is placed in time out (consequence). This behavioral pattern repeats throughout the day in all academic instruction. In this case, the student appears to be using behaviors that allow him to escape unpleasant tasks, and the use of time out as an intervention acts as the ultimate escape, increasing the probability that the noncompliant behaviors will continue. Teacher and student interviews reveal that this student does not read well independently, making individual assignments difficult. This finding suggests classroom accommodations of different, more appropriate antecedents and consequences, which, in turn, can be systematically observed during implementation. Duration Recording Duration recording involves recording the length of time from beginning to end of a response, focusing on duration rather than frequency of a behavioral occurrence. Data are reported in time units, usually minutes and

reyn2-11.qxd

6/6/2003

252

9:17 AM

Page 252

III. INTERVIEWING AND OBSERVATIONS

seconds. Just as target behaviors must be operationally defined, the starting and finishing points of the behavior should be determined to provide accurate duration data. The purpose for assessing school-related behaviors is often to inform interventions designed to decrease or increase the duration of particular behaviors such as tantrums or studying. Duration recording, though not difficult to perform, presents a practical limitation. By requiring a timing device and accurate timing, this method may result in poor execution when teachers are the data collectors. They report the process to be intrusive and disruptive (Kamphaus & Frick, 2002). For certain target behaviors, duration recording is combined with event recording to provide a richer, more comprehensive source of information. For example, an effective intervention to reduce tantrum behavior of a kindergarten student should demonstrate reductions in both frequency and duration. Event recording alone might be misleading should the student use tantrums less often but for longer periods. Time Sampling Time-sampling methods circumvent the need for continuous observation by dividing the observational period into separate, shorter timed intervals, and target behaviors are coded simply as being present or absent during each predetermined time interval. Data are reported as percentage of intervals in which the behavior occurred. Time-sampling methods allow for collection of accurate and reliable estimates of high-rate behaviors and avoids the difficulty of having to identify clear beginning and end points for responses. Shapiro and Skinner (1990) review three ways to conduct time-sampling techniques. In whole-interval recording, a behavior is recorded as “present” if it is evident throughout the entire time interval. This technique is most appropriate when assessing behaviors that are continuous, and the time intervals are short. For example, the observational session is divided into 15-second intervals, and the number of intervals for which the student remains in seat for the entire interval is recorded. In partial-interval recording, the observer scores “present” for an interval if the target behavior occurs

at any time during that interval and scores the interval as “absent” only when the target behavior is observed at no time during that interval. This technique is appropriate for low-rate behaviors as larger time intervals can be used. A teacher might divide the schoolday into 30-minute segments and record whether or not a student’s behavior, such as “noncompliance following a teacher request,” occurred at any time during a 30minute interval. In the momentary recording technique, the observer records the presence or absence of the target behavior only at the moment an interval ends, disregarding any occurrences of the behavior that ended before the final moment of the interval or after the interval is over. If group social engagement behaviors were targeted, the observer might divide a half-hour game time (the observation session) into 30-second intervals, recording whether the student was engaged in interactions with other students or not at the end of each interval.

Observers Many published observational systems, in an effort to establish control over the observational methodology, require rigorous training and monitoring of observers. Barrios (1993) suggests several steps for the training of observers, which include specialized education and assessment as well as supervised application experiences. As noted earlier, employing trained observers can be expensive. Ideally, observers should be knowledgeable in the methodology of behavior observation and experienced with the observation system to be used. In addition, observers should establish a high interrater agreement (reliability) on practice coding of target behaviors prior to the implementation of the observation system. Having people from the school environment conduct behavioral observations might seem a logical and practical choice; they would have knowledge of the context and scheduling and would seem less obtrusive. However, school personnel such as teachers, administrators, and support staff rarely have the training or time to conduct systematic behavior observations and may have enough prior experience with a student to compromise objectivity (Kamphaus & Frick, 2002; Wehby, Symons, & Hollo, 1997). Nelson,

reyn2-11.qxd

6/6/2003

9:17 AM

Page 253

11. Direct Behavioral Observation for Classrooms

Liaupsin, and Scott (2000) have designed an interactive, self-study training module and facilatator guide that addresses training of observers, so that certain school personnel could become trained observers. An alternate consideration for the role of observer is the use of self-monitoring, where an individual is trained to observe his or her own behavior. Although self-monitoring has been used primarily in clinical settings with adults, research suggests that students can accurately self-monitor if they are properly trained with a clear and simple observation system, reinforced for accuracy in behavior recording, and paired with an outside monitor to establish recording accuracy (Keller, 1986). Self-monitoring can be more costeffective and less intrusive than typical behavior observation; however, research documents that children often modify their behaviors as self-monitoring makes them aware of it (Keller, 1986) When used as part of a behavioral treatment program, beneficial change is the goal of self-monitoring. Unfortunately, behavioral changes occurring during self-monitoring observation limit the objectivity and, therefore, the usefulness, of self-monitored data for assessment purposes. Alberto and Troutman (1990) describe several school-based applications for self-monitoring.

OBSERVATION SYSTEMS In contrast to other assessment instruments/methods such as behavior rating scales, few direct observation systems have standardized procedures and are widely used in schools. In this chapter we present three popular, commercially available behavioral observation systems. Finally, although not specifically an observational system, functional behavioral assessment is discussed as an important model for student behavioral assessment, a model that relies on direct observational data.

Behavior Assessment System for Children—Student Observation System The Behavioral Assessment System for Children—Student Observation System (BASCSOS; Reynolds & Kamphaus, 1992) is a 15minute observational system designed for

253

use in the classroom setting. The BASC-SOS is part of a comprehensive behavior assessment system that includes parent and teacher behavior rating scales and a child self-report form, yielding a substantial amount of information on a student’s behavior. The BASCSOS defines 65 target behaviors, grouped into 13 categories: 9 categories of problem behaviors and 4 categories of positive/adaptive behaviors. For example, the category “Inappropriate Vocalization” (defined as disruptive vocal behaviors) lists four specific problematic classroom behavioral items: Teasing, Talking out, Crying, Laughing inappropriately. The BASC-SOS uses a momentary time-sampling method for recording data. A 15-minute observational period is broken into 30 intervals; at the conclusion of each 30-second interval, the student’s behavior is observed for 3 seconds and recorded. The observer uses a checklist to mark each category of behavior occurring within the 3second recording interval. Using the checklist, at the end of the observation session, the observer provides narrative information on the student’s behavior and the interactions between the student, teacher, and peers. It can be readministered over several observation sessions when time allows. The BASC-SOS is a simple, efficient instrument employing direct observation to document behaviors that are critical to the psychoeducational assessment of students. It can be used with minimal training and practice. However, there are psychometric considerations that limit the BASC-SOS’s contribution to assessment. First, there are no norms and, therefore, norm-referenced interpretation of scores is not possible. Second, there is limited information on validity and reliability. In addition, the BASC-SOS is not designed to assess the antecedents and consequences of student behavior, information that can provide a sequential or functional analysis of behavior and suggest potential targets for intervention, although the observer narrative can offer insight into possible antecedents and consequences (Kamphaus & Frick, 2002).

Child Behavior Checklist—Direct Observation Form The Child Behavior Checklist—Direct Observation Form (CBCL-DOF; Achenbach,

reyn2-11.qxd

6/6/2003

9:17 AM

254

Page 254

III. INTERVIEWING AND OBSERVATIONS

1986) is part of Achenbach’s CBCL system and, like the BASC-SOS, is designed to be interpreted in conjunction with parent and teacher behavior ratings and a child selfreport form. The CBCL is designed for direct observation of a child in a classroom or group setting for 10-minute periods. Guidelines recommend using the CBCL-DOF over three to six separate sessions to acquire a representative sample of a student’s behavior and then to average the scores across the sessions. The CBCL-DOF has three parts. First, the observer writes a narrative description of the student’s behavior throughout the 10-minute observation session (event recording), making note of the occurrence, duration, and intensity of specific behaviors. Second, at the end of each minute, student behavior is coded as on or off task for 5 seconds (time-sampling recording). Third, at the conclusion of the 10-minute observational period, the observer then rates the student on 96 behaviors using a 4point scale (0 = behavior not observed to 3 = definite occurrence with severe intensity or for greater than 3 minutes’ duration). With a high degree of item overlap with behaviors rated on the parent and teacher rating scales, the CBCL-DOF is well suited for multimodal assessment of child and adolescent behavior functioning (Kamphaus & Frick, 2002; Merrell, 1994). The CBCLDOF provides evidence that it can be used reliably by observers with minimal training and can differentiate between disturbed and normal children (McConaughy, Achenbach, & Gent, 1988). Like the BASC-SOS and other direct observation systems, the CBCLDOF offers no representative norms with which to compare scores. To address this issue, Achenbach (1986) suggests using the CBCL-DOF to observe two control students of the same age and sex as the student being assessed, with one control observed before the assessment observation and one observed subsequent to the assessment observation. As a frame of reference, the identified student’s scores can be compared to those of the control students, providing some limited normative reference.

Behavior Coding System The Behavior Coding System (BCS; Harris & Reid, 1981) uses an interval recording proce-

dure to assess patterns of coercive behavior and aggression in classrooms and playground settings. Using eight behavioral categories, the BCS is designed so that minimally trained school personnel can be effective observers. Harris and Reid (1981) report good interobserver agreement (93% in classroom settings; 86% in playground settings) as well as satisfactory consistency of behavior categories across settings: Therefore, the BCS may be useful in determining whether aggressive behaviors are situation specific or generalize across settings (Merrell, 1994).

Functional Behavioral Assessment An important educational issue related to the use of direct observation for assessment of classroom behavior concerns the recent reauthorization of the Individuals with Disabilities Education Act (IDEA) (Public Law [PL] 105-17; Individuals with Disabilities Education Act Amendments of 1997) in which the assessment, management, and educational accommodations for students with behavioral problems is addressed. More specifically, PL 105-17 mandates that functional behavioral assessment (FBA) be conducted for school children exhibiting behaviors that interfere with their own and classmates’ educational process. Using FBA, school personnel are directed to conduct preintervention assessments that establish the functional relationship between problem behavior and its suspected causes. While IDEA does not state specific guidelines, it proposes a team problem-solving approach based on a multimethod assessment process, which includes direct observation. “Functional behavioral assessment” is a term taken from the field of applied behavior analysis and refers to the collection and synthesis of multiple behavioral data to determine the function (cause) of problem behavior before interventions are developed (Miller et al., 1998). Typically, behavioral data include direct systematic observation of student behavior as well as indirect sources such as interviews and behavior rating scales. People who have firsthand experience with a student can provide valuable behavioral information, but interviews and rating scales alone do not constitute a functional assessment (Sugai, 1996; Vyse & Mulick, 1988). In some cases, a functional analysis

reyn2-11.qxd

6/6/2003

9:17 AM

Page 255

11. Direct Behavioral Observation for Classrooms

may be conducted by a behavior specialist or school psychologist whereby environmental events are strategically manipulated in order to observe associated changes in student behavior. FBA is a potentially powerful tool for comprehensive collection of behavioral data and directly linked to the design and implementation of appropriate interventions: yet, two reviews of intervention studies (Blakeslee et al, 1994; Haynes & O’Brien, 1990) found that less than 20% of the studies included FBA. The lack of application of FBA was attributed to the substantial resources needed to support the collection and integration of behavioral data for each subject. Although it is unclear how this finding translates into the practice of FBA in schools, the general pattern is for practice to lag behind research; presumably, PL 105-17’s inclusion of FBA in its requirements will stimulate both research and practice (Miller et al., 1998). For further reading, Bergstrom and Crone (2002) have designed and implemented a school-based model of ongoing FBA and functioned-based behavior support.

REFERENCES Achenbach, T. M. (1986). Child Behavior Checklist— Direct Observation Form (rev. ed.). Burlington: University of Vermont Press. Alberto, P. A., & Troutman, A. C. (1990). Applied behavior analysis for teachers (3rd ed.). Columbus, OH: Merrill. Barrios, B. A. (1993). Direct observation. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child and adolescent assessment (pp. 140–164). Boston: Allyn & Bacon. Bergstrom, M. K., & Crone, D. A. (2002, January). Implementing functional behavioral assessment and function-based behavior support in the typical school context. Paper presented at the meeting of the National Association of School Psychologists, Chicago. Blakeslee, T., Sugai, G., & Gruba, J. (1994). A review of functional assessment use in data-based intervention studies. Journal of Behavioral Education, 4, 397–413. Dunlap, G., Dunlap, L. K., Clarke, S., & Robbins, F. R. (1991). Functional assessment, curricular revision, and severe behavior problems. Journal of Applied Behavior Analysis, 24, 387–397. Farrington, D. P. (1993). Motivations for conduct disorder and delinquency. Development and Psychopathology, 5, 225–242. Gelfand, D. M., & Hartmann,D. P. (1984). Child behavior analysis and therapy (2nd ed.). New York: Pergamon Press.

255

Harris, A. M., & Reid, J. B. (1981). The consistency of a class of coercive child behaviors across school settings for individual subjects. Journal of Abnormal Child Psychology, 9, 219–227. Harris, F. C., & Lahey, B. B. (1982). Recording system bias in direct observational methodology: A review and analysis of factors causing inaccurate coding behavior. Clinical Psychology Review, 2, 539–556. Haynes, S. N., & O’Brien, W. H. (1990). Functional analysis in behavior therapy. Clinical Psychology Review, 10, 649–668. Kamphaus, R. W., & Frick, P. J. (2002). Clinical assessment of child and adloescent personality and behavior (2nd ed.). Boston: Allyn & Bacon. Kazdin, A. E. (1981). Behavioral observation. In M. Hersen & A. S. Bellack (Eds.), Behavioral assessment: A practical handbook (pp. 59–100) New York: Pergamon Press. Keller, H. R. (1986). Behavioral observation approaches to personality assessment. In H. M. Knoff (Ed.), The assessment of child and adolescent personality (pp. 353–390). New York: Guilford Press. Kendall, P. C., & Braswell, L. (1985). Cognitive-behavioral therapy for impulsive children. New York: Guilford Press. Mash, E. J., & Terdal, L. G. (1988). Behavioral assessment of child and family disturbance. In E. J. Mash & L. G. Terdal (Eds.), Behavioral assessment of childhood disorders (2nd ed, pp. 3–66). New York: Guilford Press. McConaughy, S. H., Achenbach,T. M., & Gent, C. L. (1988). Multiaxial empirically based assessment: Parent, teacher, observational, cognitive, and personality correlates of child behavior profile types for 6- to 11-year-old boys. Journal of Abnormal Child Psychology, 16, 485–509. Merrell, K. W. (1994). Assessment of behavioral, social, and emotional problems: Direct and objective methods for use with children and adolescents. New York: Longman. Miller, J. A., Tansy, M., & Hughes, T. L. (1998, November 18). Functional behavioral assessment: The link between problem behavior and effective intervention in schools. Current Issues in Education, 1(5) [Online]. Available: http://cie. ed. asu. edu/volume1/number5/. Nelson, C. M., Liaupsin, C. J., & Scott, T. M. (2000). Functional behavioral assessment: An interactive training module, user’s manual, and facilatator’s guide. Longmont, CO: SoprisWest. Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children (BASC). Circle Pines, MN: American Guidance Services. Shapiro, E. S., & Skinner, C. H. (1990). Principles of behavioral assessment. In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Personality, behavior, and context (pp. 343–363). New York: Guilford Press. Sugai, G. (1996). Providing effective behavior support to all students: Procedures and processes. SAIL, 11(1), 1–4. Vyse, S. A., & Mulick, J. A. (1988). Ecobehavioral assessment: Future directions in the planning and evaluation of behavioral interventions. In S. R. Schroed-

reyn2-11.qxd

6/6/2003

9:17 AM

Page 256

This page intentionally left blank

reyn2-12.qxd

6/6/2003

9:18 AM

Page 257

PART IV

SPECIFIC SYNDROMES AND SYMPTOMS

reyn2-12.qxd

6/6/2003

9:18 AM

Page 258

This page intentionally left blank

reyn2-12.qxd

6/6/2003

9:18 AM

Page 259

12 Assessment of Childhood Depression

MARGARET SEMRUD-CLIKEMAN LAURA BENNETT LAURA GULI

Childhood depression has been an area of controversy in the past. Previously many clinicians concluded that depression could not exist in children because children do not have the cognitive structures in place to appreciate challenges to their self-concepts. Children’s self-concepts were felt to be unstable and it was assumed that they had not formed sufficient ego structures to characterize a diagnosis of depression. Moreover, many psychodynamic theorists hypothesized that children feel sad only for short periods (Lyman & Hembree-Kigin, 1994). Newman and Garfinkel (1992) suggested that this conclusion is not based on clinical or empirical evidence but on the difficulty in fitting children into a theory of psychopathology. Lefkowitz and Burton (1978) further suggested that the sadness children feel is not suggestive of psychopathology but, rather, of normal developmental changes which resolve given sufficient time. Childhood depression is generally diagnosed using the same symptoms as those for adults. The fourth edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994) requires a relatively persistent and “depressed mood or loss of interest or plea-

sure in nearly all activities” must be present every day for at least two weeks in the past year (p. 320). Moreover, the child must show at least three to four additional symptoms, including appetite changes, sleeping problems, restlessness, agitation or lethargy, feelings of worthlessness or excessive guilt, problems with concentration, or frequent thoughts of death. In examining the empirical literature, Cantwell (1983) suggested four hypotheses concerning childhood depression: (1) childhood depression does not exist, (2) children show a unique presentation for depression, (3) depression in children may be masked by other symptoms such as attentiondeficit/hyperactivity disorder (ADHD), and (4) children and adults do not differ in symptoms. Of these hypotheses, the second and fourth have some empirical support (Newman & Garfinkel, 1992). The first hypothesis stems from the psychoanalytic theory that children do not have the superego development required to manifest depressive symptoms, including guilt and feelings of hopelessness (Rochlin, 1959). However, further discussion of this issue in the psychoanalytic literature suggested that children manifest symptoms that are equivalent 259

reyn2-12.qxd

260

6/6/2003

9:18 AM

Page 260

IV. SPECIFIC SYNDROMES AND SYMPTOMS

to depressive symptoms. These symptoms included oppositional defiant behavior, conduct problems, school performance difficulties, and somatic complaints (Nurcombe, 1994). Carlson and Cantwell (1980) used clinical interviews and found that depressed mood and problems with sleeping, eating, and low activity level were present in children with depression but were frequently expressed as irritable symptoms rather than sadness. This landmark study disputes the second hypothesis that Cantwell (1983) presented and provides some confirmation for the fourth hypothesis. DSM-IV (American Psychiatric Association, 1994) now allows irritable mood to be substituted for depression as a diagnostic criteria for depression. The hypothesis that children showed “masked depression” has not been supported in the literature. Hammen and Compas (1994) suggest that the basis for this hypothesis is that childhood depression is rarely seen by itself in children; rather, comorbidity of other disorders (i.e., anxiety and conduct problems) is frequently present and requires the practitioner to more fully evaluate the child’s symptoms and behaviors. Several studies have also found concordance between depression and anxiety, especially school refusal and separation anxiety, substance abuse, uncomplicated bereavement, and conduct disorder (Bernstein, 1991; deMesquita, & Gilliam, 1994; Gittelman-Klein & Klein, 1973; Kovacs, Feinberg, Crouse-Novack, Paulauskas, & Finkelstein, 1984; Puig-Antich & Rabinovich, 1986). Similarly, Nurcombe (1994) suggests that many emotionally disturbed children are polysymptomatic and frequently are in less than optimal living arrangements. Thus, the rating scale or interview used or the bias that is brought to the assessment may determine the resulting diagnosis. A 30-year study of childhood depression found that depressive symptoms are quite common in children and appear to be a good but nonspecific indicator of psychological distress and disturbance (Harrington, Rutter, & Fombonne, 1996). Moreover, this wide-ranging epidemiological study found that depression is quite heterogeneous in children and varies depending on the age of the child and severity of its ex-

pression. These authors concluded that some types of depression continue throughout life while others resolve with the developmental crisis passing.

PSYCHOSOCIAL ISSUES A number of investigators have suggested that in order to understand and appropriately diagnose depression, one must examine the psychosocial and developmental context in which it occurs (Cicchetti, Gaiban, & Barnett, 1991; Cicchetti & Schneider-Rosen, 1978; Sroufe & Rutter, 1984). In this view, experiences are a sum total of the child’s temperament, learned coping skills, biological heritage, and environmental experiences. Children who are unable to assimilate new experiences into their existing cognitive structures, or those who are inflexible in their ability to change with environmental demands, may well show depressive symptomatology. Others suggest that attachment is an important issue to understand in regard to depression (Cicchetti & Schneider-Rosen, 1978; Cummings & Cicchetti, 1990). A disruption in early attachment may well be linked to later depressive symptoms due to difficulty with interpersonal relationships (Kobak, Sudler, & Gamble, 1991; Kopp, 1989; Nurcombe, 1994).

DEVELOPMENTAL ISSUES IN THE ASSESSMENT OF CHILDHOOD DEPRESSION A study which compiled data from adult and child samples found that the symptoms of depressed mood, decreased concentration, sleep disturbance, and suicidal ideation were seen across the preschool to adult samples (Carlson & Kashani, 1988). There were changes however, in the presentation of depressive symptoms dependent on age. For example, a sad appearance, low self-esteem, somatization, and hallucinations decreased in frequency with age while sadness, hopelessness, lethargy, and delusions appeared to increase with age. The conclusion from this study was that the presentation of childhood depression appears to change with age. A developmental

reyn2-12.qxd

6/6/2003

9:18 AM

Page 261

12. Assessment of Childhood Depression

framework would appear to be most helpful for clinicians assessing depression at various ages. A full exposition of depressive symptoms at various ages is beyond the scope of this chapter. A brief description of current empirical knowledge is presented next.

Infancy and Early Childhood Documentation of affective disorders in early childhood is rare in the literature. Bemporad (1994) concluded from a review of studies of early childhood depression that it is a situation-bound reaction rather than a fixed, negative evaluation of the self and others. Few data exist on the occurrence of depression in early childhood because children younger than 7 are not typically included in large-scale community surveys of mental health (Hammen & Rudolph, 1996). Sadness in early childhood, and particularly infancy, is often associated with disruptions in the infant–caregiver bond (Trad, 1994; Zero to Three/National Center for Infants, Toddlers, and Families, 1994). If a particularly threatening event occurs, the infant’s fragile regulatory capacity may be disrupted. Because an infant’s experiences are limited, great significance may be placed on single events, causing a global interpretation of events, and negative events may be attributed to global loss of control. Repeated experiences of loss of control may result in disengagement from the environment as a type of coping strategy and may present as a state of withdrawal indicative of a depressive state (Trad, 1994). Dawson (1994) and colleagues (Dawson et al., 1999) studied depressed teenage mothers with 11- to 17-month-old infants. Slowing of brain waves, particularly in the left frontal region, was found in both mothers and infants. No such slowing was found in teenage mothers who were not depressed or in their babies. Dawson and colleagues found that infants of depressed mothers exhibited less brain activity in the left frontal region, a region associated with approachrelated emotions such as happiness, interest, and anger, relative to levels of brain activity in the right frontal region, an area associated with withdrawal-related emotions such as sadness and disgust. This pattern was demonstrated during interactions

261

with the mother, as well as with a familiar experimenter, suggesting that these responses may generalize to a variety of situations rather than being limited to interactions with a depressed mother. It could be hypothesized that the attachment in the infants with depressed mothers was less than optimal and that the situation resulted in brain activity changes consistent between mother and infant. Such changes at an early stage of development may well set the child up for later adjustment difficulties. In this manner, consistent with theories of Sroufe and Rutter (1984), early experiences interacting with biology may predispose a child to later psychological difficulties. In an assessment of childhood depression, an examination of the child’s living circumstances as well as an in-depth interview can shed light as to the underpinnings of the child’s distress as well as provide direction for intervention. It has been noted that infants of depressed mothers often “mirror” the behaviors of their depressed mothers (Field, 1984). Radke-Yarrow, Cummings, Kuczynski, and Chapman (1985) found that twice as many children of parents with major depression had insecure attachments as compared to those with clinically normal parents. It remains to be seen, however, how physiological and behavioral contributions influence each other in determining how infant depression is transmitted from parent to child (Dawson, 1994). Depressive symptoms in infants have also been attributed to early and extended stressful events, such as hospitalizations (Trad, 1994), abuse and neglect (Barnett, Manly, & Cicchetti, 1991), and grief from loss of parents through divorce or death (Zero to Three/National Center for Infants, Toddlers, and Families, 1994). The Zero to Three/National Center for Infants, Toddlers, and Families (1994) diagnostic manual is designed to provide classifications specifically for infants and toddlers, although there are few data on the reliability and validity of the instrument (Lyons-Ruth, Zeanah, & Benoit, 1996). The Zero to Three assessment tool for depression in infancy and early childhood requires the presence of patterns such as depressed or irritable mood, less interest and/or pleasure in developmentally appropriate activi-

reyn2-12.qxd

262

6/6/2003

9:18 AM

Page 262

IV. SPECIFIC SYNDROMES AND SYMPTOMS

ties, and withdrawal from social interactions or an increase in whining.

Preschoolers and Early-Elementary-Age Children The examination of depression in this population has lagged far behind studies of adolescent depression (Kashani, Allan, Beck, Bledsoe, & Reid, 1997). The lack of studies for this age group is understandable as children this age are limited in their ability to conceptualize and verbally express their feelings of distress (Kashani & Carlson, 1987). Studies of depression in preschoolers indicate that the incidence rate is rare (Stark, Sander, Yancy, Bronik, & Hoke, 2000). Kashani and Ray (1983) proposed that a disorder with such a low incidence rate in this age group would require extremely large samples to detect its presence. The existence of depression was found to be approximately 1% of children in regular preschools using teacher and parent measures, as well as child interviews (Kashani, Holcomb, & Orvaschel, 1986; Kashani et al., 1997). Notable contributions to preschool depressive symptoms include experiencing significantly stressful events (Kashani et al., 1986), being subjected to abuse and/or neglect (Kashani & Carlson, 1987), and having parents who are themselves depressed (Lyman & Hembree-Kigin, 1994). Due to the immaturity of cognitive and social development in young children, the expression of depression may be qualitatively different from that exhibited in older children (Bemporad, 1994). Preschoolers and early-elementary-age children with major depression often express somatic complaints and exhibit aggressive behaviors (Kashani et al., 1986, 1997; Kashani & Carlson, 1987) that decrease with age (Carlson & Kashani, 1988). Depressed children in this age range are unlikely to report their sadness and hopelessness but are likely to exhibit a depressed appearance (Hammen & Rudolph, 1996). Carlson and Kashani (1988) suggested that different symptoms have different meaning, depending on developmental stage, but that the basic phenomenology of depression is shared across age groups. Early-onset depression may represent the most

serious and chronic form of depression (Kovacs, 1996). Depression in early childhood can be particularly threatening due to the potentially damaging impairment that may occur during critical development periods (Hammen & Rudolph, 1996). There is a dearth of reliable and valid instruments available for assessing depression in preschool and early-elementary-age children (Kashani & Carlson, 1987; Lyman & Hembree-Kigin, 1994). Behavior rating scales from parents and teachers are often used in addition to or in lieu of child interviews (Lyman & Hembree-Kigin, 1994). Parents sometimes underreport their preschoolers’ depressive symptoms, possibly due to their fears that it might reflect poorly on their parenting skills (Kashani et al., 1986), or possibly because they do not fully understand the link between the behaviors they have observed and the diagnosis of depression in this age group. Researchers studying depression in preschool and early-elementary-age children have cited the limitations of not having reliable assessment tools (Kashani & Carlson, 1987) and have noted that questions remain as to whether accurate assessment of depressive disorders for this age group exist (Kashani et al., 1997; Puura, Tamminen, Almquist, & Kresanov, 1997). Difficulties remain in deciding how to weigh different sources of information (Kashani et al., 1997). A few instruments have been standardized for this age group. The Children’s Affective Rating Scale (CARS; McKnew, Cytryn, Efron, Gershon, & Bunney, 1979) is based on interviews with children and has been standardized for use with children as young as age 5. Other parent-report measures of psychopathology that index depression include the Personality Inventory for Children-2 (Wirt, Lachar, Seat, & Broen, 2001), which is standardized for use with children as young as 3, and the Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983), which has been standardized for children as young as 2.

Late Elementary Age and Adolescence The bulk of the assessment tools for diagnosis of depression are meant for this age group and are described in the following sections of this chapter. Emerging evidence

reyn2-12.qxd

6/6/2003

9:18 AM

Page 263

12. Assessment of Childhood Depression

indicates that early expression of childhood depression is likely a more severe form of depression and extends into adolescence and adulthood (Garland & Weiss, 1995). Depression in adolescence may vary depending on the situation and severity. Lewinsohn, Roberts, Seeley, and Rogde (1994) found that approximately 50% of adolescents diagnosed with depression had recovered following 8 weeks of treatment with another 25% responding with additional treatment. The remaining 25% had a more severe form of the disorder and a history of childhood depression. Garland and Weiss (1995) suggest that the expression of depression may be bimodal; that is, adolescent onset may be milder and quicker to recover whereas childhood onset is more severe and has a poorer treatment outcome. Depression is more common in adolescence and late childhood and somewhat easier to assess than at early ages. Findings of more externalizing behaviors and guilt in children compared to adolescents are an important developmental marker (Weiss, Weisz, Politano, & Carey, 1992). Furthermore, the recognition that adolescents are more likely to show affective symptoms and a concern about the future is important not only for assessment but also for appropriate treatment. Depressed mood has been found to be more predictive of later difficulties for females than for males (Kandel & Davies, 1986). Moreover, daughters of mothers who are depressed appear to have lower self-esteem at middle school and are diagnosed themselves as depressed by late adolescence (Miller, Warner, Wickramaratne, & Weissman, 1999). An increase in maternal criticism and less encouragement appears to be related to later expression of depression in these daughters (Garber, Braafladt, & Zeman, 1991). Symptoms shown by middle school children and adolescents include restlessness, flight to or from people, and problem behaviors, whereas those shown by children ages 6 to 12 include headache, abdominal pain and enuresis with sad affect, sleep problems, and irritability emerging over time (Kovacs & Paulausakas, 1984; Mullins, Siegel, & Hodges, 1985). Moreover, for both age ranges difficulty is present in tolerating routine and there is a need for

263

constant stimulation, possibly as an attempt to ward off unpleasant thoughts. Children in these age ranges have also been found to become overly involved with their pets and in turn to become increasingly isolated. Temper tantrums, running away, stealing, truancy, rebelliousness, and antisocial acts are frequently found in the histories of many in this age bracket. Adolescents ages 15–20 frequently engage in substance abuse, promiscuity, suicidal ideation, and alienation in conjunction with a diagnosis of depression. To put the construct of depression into a theoretical framework, a discussion of the foremost models for our understanding of depression is appropriate. It is important to note that many of these models have been adapted from adult data and their application to childhood depression is unknown at this time. The main models to be discussed are cognitive-behavioral, attributional style, and family.

PSYCHOLOGICAL MODELS OF DEPRESSION Cognitive and Behavioral Models The cognitive-behavioral model of depression attempts to combine the demonstrated effectiveness of behavior modification with the theories regarding the cognitive processes of the individual. It acknowledges the individual as playing an integral part in the determination of behavior, as well as the part that the environment plays in shaping that behavior. As a result, both cognitive and behavioral variables are seen as important to assess. Kendall and Braswell (1982) suggest that it is important not only to assess the child’s observed behavior but also to assess cognitive processes and their effect on behavior in both its maintenance and its occurrence. It would appear that this suggestion is particularly important in the assessment of depression in children, because much of depression is subjective and may not be readily observable.

Behavioral Models Behavioral models of depression suggest that depression is a result of inadequate or

reyn2-12.qxd

6/6/2003

9:18 AM

264

Page 264

IV. SPECIFIC SYNDROMES AND SYMPTOMS

insufficient reinforcers, which can be the result of lack of skill in obtaining reinforcers, lack of reinforcers in the environment for the individual, or inability to use reinforcers (Kovacs & Beck, 1977). Relatively little attention is paid to the cognitions of the depressed person in this model. According to Lewinsohn’s (1974) model, depressed persons explain their behavior to themselves following the behavior. The depressed persons are unable to identify or use contingent reinforcement in the environment, and as a result they experience low rates of responsecontingent positive reinforcement. Lewinsohn (1975) also hypothesized that social skills deficits may contribute to depression through the loss of social reinforcement, which in turn may disrupt interpersonal relationships. Therefore, depressive behaviors may lead to a greater tendency to avoid or withdraw from unpleasant situations, and when combined with potential rejection due to social difficulties, this may in turn maintain or heighten depressed affect.

Cognitive Models Beck’s (1967, 1976) theory emphasizes the role of cognitions in depression. He suggests that in depression, individuals structure their experience on the basis of cognitions that are often faulty and distorted. These cognitions in turn predispose the individual to misrepresent external events in such a way that loss and deprivation appear most evident in their interactions with the world. These distorted cognitions continue, despite independent or disconfirming evidence to the contrary. Beck further postulates the following triad of depressive symptoms based on the following faulty cognitions: 1. View of self. In cognitive schemas that relate to self-assessment, the self is pictured as unworthy or inadequate. 2. View of the world. The world is seen as making exorbitant demands and as being full of insurmountable obstacles. 3. View of the future. The person sees the difficulties continuing with no end in sight, which in turn engenders a hopeless attitude. These cognitions are validated in the selective attention of the depressed individual.

Hammen and Krantz (1976) found that the degree of improvement in depressive episodes over time was positively correlated with concurrent decreases in depressive distortions and more positive expectations about the future. Current cognitive theory assumes that maladaptive self-schema promotes errors in information processing, which, in turn, translates to the establishment of a negatively biased distortion in active information processing (Stark et al., 2000). Earlier studies have hypothesized that depressed children may experience a deficit in information processing (Schwartz, Friedman, Lindsay, & Narrol, 1982). More recent findings, however, indicate that depressed children suffer from distortions of information processing when evaluating situations related to the self, rather than a deficit in information processing (Kendall, Stark, & Adam, 1990). Taking a different perspective, Alloy and Abramson (1979) suggested that negatively biased information processing observed in depressed students could be a true reflection of reality rather than a distortion in processing, and that nondepressed students may be overly optimistic about their abilities and more motivated compared to their depressed peers. Similarly, Garber and colleagues (1991) found that depressed mothers exhibited reduced efficacy expectations compared to their peers when performing a collaborative task with their children, but their negative expectancies were actually found to be accurately representative of their poor strategic skills. Thus, prior expectation of success or failure may well shape the resulting interaction between parent and child.

Attribution Style and Learned Helplessness The ways in which individuals interpret causal relationships in evaluating positive and negative events has become an important component in the assessment of depressed individuals. The original learned helplessness model (Seligman, 1975) posited that individuals become helpless when they interpret events in their environment as being out of their control. Thus a helpless stance is developed which leads to a decrease in a person’s motivation and persistence. Once they believe they are incapable

reyn2-12.qxd

6/6/2003

9:18 AM

Page 265

12. Assessment of Childhood Depression

of influencing outcomes, they may withdraw or give up. Rehm’s (1977) self-control model postulated that the depressed individual shows deficits in self-monitoring, selfevaluation, self-attribution, and self-reinforcement behaviors. Attributions are seen as important motivators of behavior, and depressed individuals are seen as making more internal and stable attributions of failure and attributing their success to more external and unstable forces. Abramson, Seligman, and Teasdale (1978) proposed that there are three dimensions to consider when assessing attributional style: 1. Internal versus external. Responsibility for events may be placed on the self or may be externally placed. 2. Stable versus unstable. Causes may be seen as long lasting or temporary. 3. Global versus specific. Factors contributing to events may be interpreted as pervasive or as changing according to situational variables. It was proposed that individuals who possessed internal, stable, and global explanations for undesirable events would develop a pessimistic style, whereas those who attributed negative events to external, unstable, and specific sources would develop an optimistic style. In addition, attributing external, unstable, and specific attributions to positive events is considered to be characteristic of depressive cognitions. Seligman and colleagues (1984) confirmed this pattern in a study with children ages 8–13. Results indicated that children who attributed negative events to internal, stable, and global causes were more likely to report depressive symptoms than were their peers who attributed negative events to external, unstable, and specific causes. Similarly, Sacco and Graves (1984) found that when compared with their nondepressed peers, depressed children showed more negative selfevaluations and more negative attributional cognitive styles, and Kaslow, Brown, and Mee (1994) have suggested that depressed children may perceive a lack of control when important events of their lives are concerned. Studies have suggested that the ways that individuals interpret the causes of events remain fairly stable over time (Kaslow et al., 1994).

265

Recent research in the area of attributional styles and learned helplessness has focused on defining subtypes of depression and heterogeneity of cognitive patterns in depressed individuals. Abramson, Metalsky, and Alloy (1989) further refined the learned helplessness theory to propose a subtype of depression called hopelessness depression, which may result when someone who already possesses a depressogenic attributional style and pessimistic expectations is exposed to negative or stressful events. Asarnow and Bates (1988) found that children with remitting depression scored similarly to nondepressed children on measures of cognitive patterns and attributional styles. This finding suggests that there may be a difference in cognitive patterns between children with depression that resolves and those with a chronic disorder. Alloy, Hartlage, and Abramson (1988) have also suggested a need to differentiate between attributions for specific events and a pervasive attributional style. Similarly, Weisz, Sweeney, Proffitt, and Carr (1993) suggested distinguishing between subtypes of personal helplessness and universal helplessness forms of depression. They found that both perceived incompetence and perceived noncontingency were strongly related to children’s depression on Children’s Depression Inventory scores of elementary school children.

Family Theories As with any childhood disorder, depression in children cannot be comprehensively assessed without taking into consideration the effects of family environment and quality of interaction. The way parents model and coach emotion regulation strategies and contribute to the family environment in general helps to determine how children learn to deal with negative affect (Stark et al., 2000). Interesting descriptions of family characteristics of depressed children are beginning to take shape, including reports of giving affection contingent upon high standards of achievement (Cole & Kaslow, 1988; Cole & Rehm, 1986), descriptions of lower levels of family support (McCauley & Meyers, 1992), increased levels of family discord (Kaslow, Deering & Racusin, 1994), and engagement in fewer pleasant

reyn2-12.qxd

6/6/2003

266

9:18 AM

Page 266

IV. SPECIFIC SYNDROMES AND SYMPTOMS

activities than for families of nondepressed children (Stark et al., 2000). When Asarnow, Tompson, Hamilton, Goldstein, and Guthrie (1994) examined parents’ descriptions of their children, they found that parents of depressed children exhibited higher levels of criticism and emotional overinvolvement as compared to parents of nondepressed children. They hypothesized that excessive criticism may in fact reinforce a depressed child’s negative feelings, helping to maintain the depressive disorder. Puig-Antich and colleagues (1985) observed poorer communication, decreased warmth, and increased hostility in interactions of mothers and their clinically depressed 6–12-year-olds, compared to nondepressed psychiatric and normal control groups. They found the mother–child relationship in dyads of depressed children to be cooler and more distant. Similarly, Cole and Rehm (1986) found that parents of depressed children exhibited higher standards for their children and a rewarded them at lower rates compared to parents of nondepressed parents performing a collaborative family task. Interestingly, they also found that the children in these depressed dyads mirrored their mothers’ response patterns of withholding praise until high levels of achievement were reached when evaluating themselves. Parental depression also has a strong influence on parenting styles. Garber and colleagues (1991) found that the overall strategies of depressed mothers exhibited in a collaborative task with their children were significantly poorer than the strategies of dyads with nondepressed mothers. While each of these findings provide valuable insights about the environments of depressed children, it is important to remember the bidirectional nature of interpersonal relationships. It is therefore difficult to determine if certain environmental variables contribute to a child’s depression, if a child’s depression influences the quality of the environment, or more likely, if they are mutually influential across time.

Summary Thus, childhood depression is an evolving field and has changed dramatically since the 1970s when it was not believed to exist.

However, difficulties remain in our understanding of childhood depression, and, unfortunately, these problems have an impact on our ability to evaluate children with depression. The issue of comorbidity of depression is especially salient for our understanding of depression. Furthermore, the developmental progression or differences in expression of depression affect not only our ability to treat depression but also the initial difficulty with diagnosis. Emerging evidence indicates that depression may be present in early childhood, and that with this earlier expression of the symptoms there is an increase in the severity and duration of the disorder. Few studies have been conducted longitudinally to examine this issue in more detail. Methodological problems continue to plague our understanding of childhood depression and cloud our ability to evaluate children most appropriately. Moreover, models of depression are not often incorporated into the assessment of depression and many instruments are atheoretically based. The models that do exist are developed mainly from adult data and little work has been conducted to assist our understanding in how they translate into childhood. This chapter seeks to provide a brief overview of these issues with the main emphasis on the assessment tools available for an appropriate diagnosis of depression. A secondary goal is to provide recommendations for the clinical use of these instruments in mental health centers and schools.

ISSUES IN ASSESSMENT A multi-instrument, multi-informant method for assessing childhood depression provides the broad base of information required in developing a careful diagnosis. The optimum assessment needs to include information from parents, teachers, and the child. This information should include behavioral data (behavior rating scales, observations, etc.) as well as interviews and, as necessary, projective testing. The scope of this chapter does not include projective testing, and interested readers are referred to Chapter 7 (in this volume) on the Rorschach and the Thematic Apperception Test. Di Giuseppe (1981) suggests three aspects for assessment.

reyn2-12.qxd

6/6/2003

9:18 AM

Page 267

12. Assessment of Childhood Depression

First, the behaviors evidenced by the child need to be evaluated. This step requires an assessment of the frequency, duration, and severity of these behaviors. Moreover, it is important to identify antecedent as well as consequent events surrounding such behavior. Second, the child’s cognitions are assessed. It is important to evaluate not only what the child is thinking while the interviewer is working with him or her but also what was being thought during the occurrence of the behavior. Finally, an assessment of the child’s ability to problem-solve is important; this step is particularly important for the development of intervention strategies. In all parts of this assessment it is particularly important to assess the child’s developmental stage. As indicated in a previous section of this chapter, the expression of depression may differ depending on the age of the child. It is also important to assess who “owns” the problem. Most young children are referred for assessment by their parent or teachers—few self-refer. Frequently in my practice, when I ask a child why he or she is at my office, the response is because the parent wanted the child to come or that there are concerns about school. Over 25 years of practice I have had only one instance in which a child has said, “Because I am sad.” Thus, it is particularly important to use various methods for assessing childhood depression that include behavioral as well as cognitive variables. The assessment techniques reviewed in the following section include self-report scales, clinical interviews, peer nomination inventories, behavior rating scales, assessment of cognitions, cognitive style, and thought-sampling procedures. Before beginning a discussion of instruments and techniques, it is important to note the obvious but often neglected proposition that children are a special population and that results, conclusions, and measures appropriate to adults may not be appropriate for children. Moreover, the child needs to be an integral part of the assessment and an active participant. Assessment with children, and with adults for that matter, needs to be a problem-solving process in which working hypotheses are advanced and checked out. Moreover, as advanced in the first half of this chapter, a transactional approach that

267

takes into consideration environmental, personal, and biological influences on development is crucial for an ecological assessment of the child’s needs. Each of the following instruments, taken by themselves, is not sufficient to arrive at a diagnosis of depression in childhood and adolescence. Rather, the ability of the clinician to put all the data together into an integrated form is the most crucial task required of us.

ADDITIONAL CONSIDERATIONS WHEN ASSESSING CHILDREN The extent to which children are able or willing to report on their depressive symptoms varies greatly. Children who are depressed may show difficulties in paying attention and following directions on measures. Thus, procedures should be adapted to the child and their attentional abilities (Semrud-Clikeman & Hynd, 1991). In a study that examined the use of selfreport scales with children (Birleson, Hudson, Buchanan & Wolff, 1987), it was noted that the establishment of rapport was likely to increase the validity of the response, as children, particularly depressed children, may worry about revealing secrets and making mistakes. Furthermore, a child may show depression in one setting but not another; therefore, a clinician’s task is not only to assess the child but to assess the interaction between the child and adults in the child’s different settings. It is important to remember that the capacity of a child to understand an item or describe an experience is largely influenced by developmental and educational level. Younger and less cognitively developed children are less abstract and have more limited short-term memory spans, making longer questions difficult to answer. Current measures of depression ask subtle questions about symptom duration and intensity; children may not be able to describe their symptoms in such detail (Kazdin, 1990; Kazdin & Petti, 1982). Young children have shown more difficulty reporting depression and tend to underestimate it as well (Birleson, Hudson, Buchanan, & Wolff, 1987). An examination of the relationship between reliability and validity of child self-report to different developmental levels as to the

reyn2-12.qxd

268

6/6/2003

9:18 AM

Page 268

IV. SPECIFIC SYNDROMES AND SYMPTOMS

dimensions of time perspective, emotion differentiation, self and social awareness is needed (Hodges, McKnew, Burbach & Roebuck, 1987).

ple if only one source had been interviewed (Hodges, 1990).

Correspondence of Measures

Culture and ethnicity can affect the diagnostic utility of assessment measures in a variety of ways. Gibbs and Huang (1989) discuss the subtle differences in nonverbal communication, eye contact, affective expression, and value judgments that may influence a clinician’s psychosocial assessment of children from a different ethnic background. For example, affective expression in children is more animated or reserved in certain cultures; therefore, culturally appropriate norms of expressing affect should not be confused with lack of affect, depressed affect, or disrespect. Direct eye contact between children and adults is discouraged in some ethnic groups, so looking away should not always be interpreted as a sign of evasiveness. Because each ethnicity may place different value on self-esteem, interpersonal competence, emotion regulation, and the definition of mental illness itself, it is important for clinicians to familiarize themselves with these values when necessary, so that they do not confound a diagnosis of depression. Also, historical difficulties between ethnic groups may make it difficult for minority clients to trust a therapist from a different ethnic or socioeconomic background. Finally, low-income children may suffer from poor diet or home conditions, affecting energy level and sleep (Gibbs & Huang, 1989), which may be misread as symptoms of depression. Differences between ethnic backgrounds were examined by Ramirez (1989) specific to the assessment of Mexican American children. Ramirez states that the clinician should be aware that it is common for Mexican Americans to disclose personal information slowly. During assessment interviews, it is possible that parents may feel that it is impolite to disagree, because of a cultural emphasis on cooperation. Also, when addressing parents, the therapist should first address questions to the father, then to the mother, then to other adults, in respect of traditional family age and sex roles. A neutral opening statement that allows the family to choose its own spokesperson is also appropriate.

To what extent do measures of depression correspond with each other? Concordance among informants is often poor for an overall diagnosis of depression. Specifically, correspondence between child and parent/ teacher/peer report has been as low as .00–.30 (Kazdin, 1988). In another study, informants showed significant agreement on clusters of behavior symptoms for depression, although there was some differential patterns for self-reports between boys and girls (Epkins & Meyers, 1994). Children generally report fewer symptoms than do parents (Kazdin, 1988), with the exception of more frequent reporting of decreased energy and fatigue by both informants (Ivens & Rehm, 1988). Mothers have been shown to report more symptoms in children than fathers or children themselves. At the item level, interrater agreement is extremely high for certain symptoms and extremely low for others. In a study looking at measure items, results showed that self-report, peer report, and teacher report generally measure unrelated constructs. Second-order analysis, however, showed that the global construct of depression is being measured by items from all three instruments (Crowley, Worchel, & Ash, 1992). It may be the case that measures designed for different informants are predictors of different criteria. Child self-reports generally measure internal experience, whereas parent reports measure social behavior and affect-related experience. Peer–teacher reports measure popularity and academics (Kazdin, 1988). Although measures completed by different informants do not correlate well, each can still reliably predict different external criteria relevant to a diagnosis of depression. Therefore, despite discrepancies among measures, the importance of obtaining reports from various sources is required to adequately assess depression. One study, for example, points out that one-third of all diagnoses for depression would have been missed from the sam-

Issues of Culture and Ethnicity

reyn2-12.qxd

6/6/2003

9:18 AM

Page 269

12. Assessment of Childhood Depression

Possible multicultural variations in the assessment of depression using self-report scales are just beginning to be studied (Allen & Majidi-Ahi, 1989). Roberts, Chen, and Solovitz (1995) found few differences among Anglo, African, and Mexican American adolescents using the CBCL and a structured interview (Diagnostic Interview for Children 2.1). In studies using self-reported feelings of depression and suicidality, the Mexican American subjects were found to evidence a higher rate of depression and suicidal thoughts with the highest prevalence rates found for Mexican American females (Roberts, 1994; Roberts & Chen, 1995; Roberts, Chen, & Roberts, 1997; Roberts, Roberts, & Chen, 1997). Similarly, Mexican American adolescents have been found to be at higher risk for depression than Asian American adolescents, with females and those adolescents from lower socioeconomic groups scoring the highest (Roberts, Roberts, & Chen, 1997). Similarly, Takeuchi, Roberts, and Suzuki (1994) found that Hispanic youths scored higher on depressive symptomatology than did Japanese American adolescents, followed by Caucasians. Girls in all three ethnic groups were found to show more depressive symptoms than boys. Comparing boys across ethnicities, the Japanese American boys scored the highest, followed by the Hispanic group. Thus, a number of issues are important when assessing depression, including age, gender, and ethnic status. It may also be important to be aware of socioeconomic status given the emerging evidence that this variable may be a mediating influence on the expression of depression no matter what the ethnic background. It may well be that the expression of depression will vary between cultures as to what is accepted and what is not. For example, a young Asian American boy was referred for assessment because of seemingly unprovoked anger. When asked about his anger, he was unwilling to discuss these feelings or his behavior, although he admitted deep embarrassment about what he had done. When an Asian American therapist was brought into the school for a consultation, he reported that culturally it was not acceptable to show strong uncontrolled anger and that the boy was becoming quiet when first provoked. However, as

269

his anger built, he would become less and less able to maintain his behavioral control and the result would be an extreme overreaction to a minor situation. Assisting the understanding of the behavior within a cultural context helped in developing a plan to assist this young boy. It is not unlikely that similar cultural differences exist in the expression of depression.

SELF-REPORT SCALES In many aspects, self-report scales are the most frequently used instruments in hospitals, clinics, and schools. Kerr, Hoier, and Versi (1987) evaluated the empirical research assessing childhood depression and found that the most commonly used source of data was self-reports with an increase in the use of ratings provided by peers. Willcutt, Hartung, Lahey, Loney, and Pelham (1999) found that behavioral ratings add substantial information when used as a supplement to parent and teacher reports. The following section evaluates the most commonly used self-report scales for depression.

Children’s Depression Inventory The Children’s Depression Inventory (CDI; Kovacs, 1992) evolved from the Beck Depression Inventory (BDI). The CDI contains 27 multiple-choice items that cover an array of overt depressive symptoms, such as sadness, anhedonia, suicidal ideation, and sleep/appetite disturbance. Each CDI item contains three choices; each choice is rated on a 0–2 scale, with a rating of 2 being the most severe. The child rates the statement that best describes his or her feelings in the previous 2 weeks. The scale can be administered to children ages 7–17. Cutoff scores for various levels of severity have been developed. There are five subscales presented in the manual, including negative mood, interpersonal problems, ineffectiveness, anhedonia, and negative self-esteem. These subscales were reported in the manual to have been derived from the normative data. Several studies have found internal consistency in the .80s (Cole & Carpentieri, 1990; Crowley et al., 1992; Kovacs, 1981), moderate test–retest reliability (Nelson &

reyn2-12.qxd

270

6/6/2003

9:18 AM

Page 270

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Politano, 1990; Weiss et al., 1991) with variances in test–retest reliability ascribed to differences in time between assessments (Reynolds, 1994; Reynolds & Graves, 1988). Factor analysis has found two to three factors on the CDI. Hodges and Craighead (1990) found higher scores for depressed children on the scales measuring dysphoric mood, loss of personal and social interest, and low self-worth. A study contrasting clinical samples with a control sample found two factors in the clinical sample of psychiatric inpatients (depressive affect and oppositional behavior) and three factors for the control sample (depressive affect, oppositional behavior, and personal adjustment; Carey, Faulstich, Gresham, Ruggiero, & Enyart, 1987). Weiss and colleagues (1991) found that factors differed between children and adolescents, suggesting a developmental effect on this measure. Previous research has indicated that the CDI is more appropriate as a screening instrument than as a diagnostic tool. Carey and colleagues (1987) found that the CDI was able to discriminate between 70.4% and 71.6% in a control population of nonreferred children. It was unable to discriminate between depressed and the conduct-disordered clinical samples. Most studies indicate that the CDI is a good general measure of emotional distress but that it is unable to discriminate between conduct-disordered and affectively disordered groups (Nelson, Politano, Finch, Wendel, & Mayhall, 1987; Saylor, Finch, Baskin, et al., 1984; Saylor, Finch, Spirito, & Bennett, 1984). The CDI has been found to correlate poorly with parent and teacher ratings of depression (Doerfler, Felner, Rowlison, & Raley, 1988). The CDI has also been used for teacher rating of depression. Ines and Sacco (1992) compared the ratings of teachers and those of children using the CDI. There was moderate correspondence between the teacher and child ratings with the most concordance found for school-related behaviors. The lack of a manual for this instrument was a major drawback in the past; however, a manual has recently been published. Normative data are now available for the CDI. There are some difficulties with the manual. Reynolds (1994) points out that the data for children in grades 2–8 is provided while norms are provided for children up to age

17. Ethnicity is not fully reported in the manual. The main drawback to the data presented in the manual is that there is limited support for the clinical interpretation of the subscales. Low internal consistency reliabilities are reported as well as small numbers of items loading on each of the scales. Thus, although the CDI is the most commonly used instrument for diagnosing depression, empirical support for such use is mixed at best. Results vary depending on the type of groups used for contrast as well as the level of the cutoff scores (Reynolds, 1994). The most commonly recommended use of the CDI is for the screening of referred children for assessment and interpretation of elevated scales as signs of psychological distress, rather than to support a diagnosis of depression. Gender differences have also been found, with the CDI showing more accuracy with females than with males (Berard, Boermeester, Hartman, & Rust, 1997).

Children’s Depression Scale The Children’s Depression Scale (CDS) was developed by Lang and Tisher (1978). It is recommended for children ages 9–16 and has 66 items with 48 items focusing on depressive symptoms and reactions and 18 on positive experiences. The CDS also uses a somewhat different format from the typical paper-and-pencil rating scale. The child is asked to sort statements on cards into one of five boxes, ranging from “very wrong” to “very right.” There are alternative forms so that both children and adults can rate the behavior. The subscales themselves were drawn from the literature on depressive symptomatology. The subscales consist of symptoms including pleasure, guilt, affect, social difficulties, thoughts of sickness and death, and low self-esteem. Responses are tallied on a 1–5 scale, with low scores indicating the presence of depression. Factor analysis has found one general factor so the subscales are not individually reported (Kazdin, 1981). Psychometric characteristics of the test have been found to be adequate. Test–retest reliability has been found to be at .74 for a 1-week interval (Tisher, Lang-Takac, & Lang, 1992). Internal consistency reliability coefficients are high and range from .90 for total depression (Bath & Middleton, 1985;

reyn2-12.qxd

6/6/2003

9:18 AM

Page 271

12. Assessment of Childhood Depression

Knight, Hensley, & Waters, 1988) to .79 for positive affect (Kazdin, 1987). Correlations between the CDS and CDI have been found to be in the moderate range (r = .48 to .84; Kazdin, 1987; Knight et al., 1988; Rotundo & Hensley, 1985). The concordance between parent and child ratings is very poor (r = .04; Kazdin, 1987). The CDS has been found to discriminate between depressed and nondepressed children for the child form but not for the parent form (Fine, Moretti, Haley, & Marriage, 1984). The CDS shows promise for diagnosis as a self-report instrument. The parent form should not be used given the findings of low concordance. The main drawback to the CDS is the cumbersome administration and the limited empirical validation of this instrument (Fristad, Emery, & Beck, 1997).

Hopelessness Scale The Hopelessness Scale was developed by Kazdin, French, Unis, Esveldt-Dawson, and Sherick (1983). It was modeled after the scales developed for adults by Beck and is designed for children ages 8–13 years. The scale consists of 17 true–false items; it is scored so that the higher the score, the greater the assessed amount of hopelessness or negative view of the future. Acceptable internal consistency (r = .70) was found by the authors. Concurrent validity was assessed using performance on the CDI, the Bellevue Depression Inventory, and a depression checklist developed from Weinberg research criteria. All these measures were positively correlated with the Hopelessness Scale at a moderate level. Kazdin, Rodgers, and Colbus (1986) found that the Hopelessness Scale was positively correlated with depression and negatively correlated with selfesteem. The scale was also found to relate to diminished social behavior, especially when children with high hopelessness scores were compared with children with low hopelessness scores. Two factors were identified in the Kazdin and colleagues study; these were that the future would be negative and the child would not be able to alter this fact.

Depression Self-Rating Scale The Depression Self-Rating Scale (DSRS) was developed by Birleson (1981) on the

271

basis of a literature search for the most common depressive symptoms of childhood depression. The scale is designed for children ages 7–13. It consists of 18 items rated on a 0–2 scale. Birleson found test–retest reliability to be .80 and split-half reliability to be .86 with one major factor identified. He used a small sample of 20 children from a psychiatric residential school and 19 from a local school. Asarnow and Carlson (1985) modified the DSRS by including three items from the Hopelessness Scale and two items assessing the child’s capacity for empathy. Criterion validity of the DSRS was established with the finding that the previously diagnosed children with depression scored significantly higher on the DSRS than did the nondepressed children. A cutoff score of 17 was found to correctly classify 77% of the depressed children. The DSRS scores have been found to correlate significantly with scores on the CDI. It has been found to show good concurrent validity particularly with psychiatric inpatients (Asarnow & Carlson, 1985; Beck, Carlson, Russell, & Brownfield, 1987; Birleson et al., 1987; Ivarsson, Lidberg, & Gillberg, 1994). The DSRS has not been as carefully studied as the CDI, and its usefulness should be evaluated as more research is provided.

Reynolds Child Depression Scale The Reynolds Child Depression Scale (Reynolds, 1989) is designed for use with children ages 8–13 years. It consists of 30 items with 29 using a 4-point response format ranging from almost never to all the time. The final item is a set of faces ranging from sad to happy—the child places an × over the face most descriptive of his or her feelings that day. A strength of the Reynolds scale is the manual, which provides normative data for over 1,600 children of varying economic and ethnic backgrounds. Internal consistency coefficients reported in the manual are at .90 with test–retest reliability reported at .85 (Reynolds & Graves, 1989). Correlations between the Reynolds and the CDI range from .7 to .79 (Stark, Reynolds, & Kaslow, 1987) and with a structured clinical interview at .76 (Reynolds, 1989). The Reynolds has also been found to be sensitive to treatment outcome (Rawson & Tabb, 1993; Stark et al., 1987). There are few data

reyn2-12.qxd

272

6/6/2003

9:18 AM

Page 272

IV. SPECIFIC SYNDROMES AND SYMPTOMS

as to the ability of the Reynolds to discriminate depressed from nondepressed subjects. As such, it is unknown whether the measure is sensitive to general emotional distress or specific to depression. Further assessment of this issue is warranted.

The reliability is adequate for this instrument but there is little validity information. This measure is frequently used in conjunction with the structured interviews and there is a body of research as to its relationship to these interviews discussed later in this chapter.

Children’s Depression Rating Scale—Revised

Reynolds Adolescent Depression Scale

The Children’s Depression Rating Scale— Revised (CDRS-R) was developed by Poznanski, Cook, and Carroll (1979) to assess depression in children ages 6–12. It is a clinician-rated scale that provides a summed score from the items to indicate the severity of the depression. The scale was adapted from the Hamilton Depression Rating Scale for adults. There are 12 items that cover an array of behaviors, including communication, mood disturbance, physical complaints, and vegetative signs. The CDRS-R is completed by pooling information from parents, child, school, and medical personnel. The interview has a multiple-choice format with items scaled from 0 to 7 (“unable to rate” to “severe”) in the direction of increasing pathology. The summed total ranges between 18 and 118 with 40 being indicative of clinical depression and a score above 60 as severe depression. Agreement among judges is reported as high (r = .96) and acceptable correlations have also been reported between interviews and global clinical ratings of depression (Kovacs, 1981). Test–retest reliability has been reported as .81 with interrater reliability at .86 (Poznanski, 1984). Parent–child concordance has been found to be poor ranging from –.01 to .42 (Mokros, Poznanski, Grossman, & Freeman, 1987). Significant correlations between the CDRS-R and the CDI were found for females but not for males (Shain, Naylor, & Alessi, 1990). The specificity of the CDRS-R is questionable particularly in its ability to discriminate between depression and anxiety (Eason, Finch, Brasted, & Saylor, 1985). The CDRS-R has been found to be helpful in its ability to classify patients as acutely or chronically depressed and may be good for the prediction of rate of improvement (Shain et al., 1990). The advantages of the CDRS-R are that it is relatively easy to use and to administer.

The Reynolds Adolescent Depression Scale (RADS; Reynolds, 1986, 1987) is designed for use with adolescents ages 12–18 in grades 7–12. It consists of 30 items and uses a 4-point response format (“almost never” to “most of the time”). The RADS reflects DSM-III (American Psychiatric Association, 1980) criteria for major depression and dysthymia and requires approximately a thirdgrade reading level. The instrument was designed to evaluate the severity of depression (Reynolds, 1994). Psychometric properties of the RADS are generally good. Normative information from the manual includes data from more than 2,460 adolescents throughout the United States from an ethnically and geographically diverse sample. Scores range from 30 to 120 with a cutoff of 77 used to define significant depression. Reliability is reported to be high ranging from .91 to .96 (Schoenert-Reichl, 1994). Internal consistency reliability has been found to be good, ranging from .87 with a sample with mental retardation (Reynolds & Miller, 1985) and .88 with a sample of conduct-disordered adolescents (Nieminen & Matson, 1989). Test–retest reliability has also been found to be good and generally is in the high 70s and low 80s (Reynolds, 1986, 1987). Internal consistency has been found to be .91 at first assessment and .93 for a second assessment (Reynolds & Mazza, 1998). Validity is found to be adequate with the RADS, correlating strongly with the CDI (Brown, Overholser, Spirito, & Fritz, 1991; Kahn, Kehl, & Jenson, 1987). Higher scores have been found on the RADS in special education populations (Dalley, Bolocofsky, Alcorn, & Baker, 1992; Hagborg, 1992), children of alcoholic parents (Havey & Dodd, 1992), and suicidal adolescents (Brown et al., 1991; King, Raskin, Gdowski, Butkus, & Opipari, 1990). The RADS has also been

reyn2-12.qxd

6/6/2003

9:18 AM

Page 273

12. Assessment of Childhood Depression

found to discriminate from adolescents with major depression and normal controls (Shain et al., 1991). It has been found to be most sensitive for females compared to males (Shain et al., 1990). Thus, the RADS appears to have the strongest psychometric properties for the diagnosis of depression. The manual is helpful and clearly written. The RADS has also been found to be useful as an outcome measure for treatment efficacy (Hains, 1992). It has been recommended that the standardization sample needs to be expanded to include more than just Caucasian and African American midwesterners to be useful throughout the United States (Davis, 1990).

Other Self-Report Measures A number of additional self-report measures are either in press or have limited information as to psychometric properties. The Adolescent Psychopathology Scale (Reynolds, 1999) evaluates psychopathology in adolescents ages 13–19. It has 20 clinical disorder scales, including scales for major depression and dysthymia. The major depression scale consists of 29 items reflecting DSM-IV symptoms and the dysthymia scale consisting of 16 items. Preliminary reports of internal consistency are of .95 for the normal and clinical samples (Reynolds, 1993). Further study with this instrument is needed to determine its utility for the diagnosis of depression. The Automatic Thoughts Questionnaire for Children (ATQ-C; cited in Stark et al., in press) presents 30 depressive self-statements that the adolescent rates as to frequency of occurrence. There are four choices ranging from “not at all” to “all the time.” Psychometric data were not provided at this time. The instrument looks promising as it provides a measure of thoughts that are frequently seen in depression. The Self-Report Measure of Family Functioning—Children’s Version (SRMFF-C; Stark, Humphrey, Crook, & Lewis, 1990) is a measure including 65 items that assess family functioning and interaction patterns that contribute to the development of a depressive style of thinking. The SMRFF-C assesses characteristics such as family style, cohesion, conflict, sociability, and organization. It is organized on a 5-point scale rang-

273

ing from never true to always true. The SRMFF was developed by Bloom (1985) for adults through factor analysis of the most frequently used family functioning measures and revised by Stark and colleagues (1990) for children. There are 15 subscales included. Reliability has been found to be between .50 and .78 for the subscales with acceptable internal consistency. The validity of the SRMFF-C has not been studied, but the validity of the SRMFF has been reported to be good (Bloom & Naar, 1994). This instrument requires additional study to determine its usefulness in various populations.

TEACHER AND PARENT RATING SCALES Parent rating scales are generally omnibus scales that do not specifically identify depression but sample a wide variety of behaviors and symptoms. There are basically three main scales used in clinical practice as well as one scale particularly for teachers. Each of these is discussed with the understanding that these measures are not meant to diagnosis depression but, rather, to point to important symptoms for further investigation.

Personality Inventory for Children The Personality Inventory for Children— Second Edition (PIC-2; Wirt, Lachar, Seat, & Broan, 2001) is a revision of the PIC and is a parent-completed survey of his or her child’s behavior. There is a depression scale as well as an intellectual and achievement scale. The PIC can be used with children ages 5–19 and consists of a true–false format. The standard form has 275 questions while the brief form has 96 questions. The PIC-2 has 3 validity scales, 9 adjustment scales, and 21 subscales. Although there is no longer a depression scale, there are two scales of interest: psychological discomfort and social withdrawal. These scales measure fear, worry, depression, sleep disturbance, and social isolation. Test–retest reliability is good for this scale, as reported in the manual. Content validity has been described as satisfactory while construct validity is reportedly limited (Sattler, 2002). The PIC-2 has been found to correlate signifi-

reyn2-12.qxd

6/6/2003

9:18 AM

274

Page 274

IV. SPECIFIC SYNDROMES AND SYMPTOMS

cantly with other measures (Wirt et al., 2001).

Child Behavior Checklist The Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983) relies on parent or teacher report to provide information as to the child’s functioning. There are different forms for teacher and parent. The CBCL assesses social competencies as well as childhood problems for ages 2–16. Items are rated on a 3-point scale. Studies have found that scores on the social competency part of the CBCL reliably distinguished between suicidal and depressed children with the suicidal children withdrawing and the depressed children seeking out parent and teacher involvement (Cohen-Sandler, Berman, & King, 1982). The depressive items tap information that can be quite inferential in nature and may require the informant to make judgments based on the child’s behavior. For this reason the reliability of the depression scale is lower than desired. This scale also tends to focus on the internal life of the child rather than on environmental stressors or contributions to problematic behaviors. The teacher form of the CBCL is similar to the parent form and contains 85 out of the 118 items that are the same on both rating scales with an additional 9 with minor word changes. There are no validity scales . Psychometric properties are reported in the manual as adequate. Again, the teacher form is an omnibus measure and has been found to be less amenable to specifying a child’s difficulty than is the parent form (Clarizio, 1994). Thus, the CBCL may be better as a overall screening method rather than a diagnostic tool.

Teacher Rating Scales The Teacher Rating Scales (TRS) measure problem behaviors and adaptive skills in school settings. The TRS is available at three age levels: preschool (4–5), child (6–11), and adolescent (12–18). The behaviors described in each item of the form are rated on a 4-point scale ranging from “never” to “almost always.” The TRS generally takes 10–20 minutes to complete. The broad domains for clinical and adaptive assessment in the TRS include Externalizing Problems, Internalizing Problems, School Problems, and Adaptive Skills. The scales contain the same content at each age level, although the behavioral manifestations of child problems are adjusted according to developmental level. In addition to the composite scores for these domains, the TRS provides a score for the Behavioral Symptoms Index (BSI), which is a broad composite that assesses an overall level of problem behaviors in the child. National age norms (general, female, or male) and clinical norms are available for reference. Critical items may also be interpreted individually. The scale also includes an F (“fake bad”) index as a validity check to detect a negative teacher response set. The internal-consistency reliabilities of all age levels of the TRS in the general population sample are high, averaging above .80 for all three levels, and the reliability of the BSI ranges from .95 to .97. Internal-consistency reliabilities for the Clinical norm sample have median values ranging from .82 to .85. Test–retest reliability median values range from .82 to .91 for the three agegroups. Construct validity for the Depression scale is strongly supported as reported by the authors.

Behavior Assessment System for Children

Parent Rating Scales

The Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992) is a multimethod and multidimensional system for evaluating behaviors and self-perceptions of children ages 4–18 years. It measures several aspects of children’s behavior and personality in terms of both clinical and adaptive dimensions. The components that make up the BASC system can be used independently, or in any combination.

Similar to the TRS, the Parent Rating Scales (PRS) measure adaptive and problem behaviors, but in the home and community settings. It is scored on the same 4-point scale from “never” to “almost always,” is available at the same three age levels as the TRS, and takes approximately 10–20 minutes to complete. The same broad domains of assessment used in the TRS are present in the PRS, except for the domain of School

reyn2-12.qxd

6/6/2003

9:18 AM

Page 275

12. Assessment of Childhood Depression

Problems and the scales for Learning Problems and Study Skills. The same norm reference groups and validity index offered for the TRS are offered for the PRS. Internal-consistency reliability scores for the composites range from the middle .80s to the low .90s for all age levels in the general sample. The reliability scores for the BSI range from .88 to .94. Internal-consistency reliability scores for the clinical sample are generally higher than that of the general population sample (.84 to .94). Test–retest reliability for the PRS showed median values ranged from .70 to .88, with the adolescent group being the lowest. The construct validity of the Depression scale was strongly supported. Self-Report of Personality The Self-Report of Personality (SRP) consists of statements to which the child responds true or false and takes about 30 minutes to complete. The form is available at two age levels: child (8–11) and adolescent (12–18). Both forms have the same composite scores of School Maladjustment, Clinical Maladjustment, Personal Maladjustment, and the overall composite score, the Emotional Symptoms Index (ESI). The ESI includes both clinical and adaptive scales, unlike the BSI, which only includes problem items. Similar to the TRS and the PRS, national age norms and clinical norms can be referenced for interpretation. Validity check indexes include the F index, the L (“fake good”) index for the adolescent form, and the V index to determine if invalid responses may be attributed to failure to follow directions, poor reading skills, or questionable contact with reality. The internal-consistency reliability of the PRS averages about .80 for each gender at both age levels in the general-population sample. Composite score reliabilities range from the mid-.80s to the mid-.90s. Internalconsistency reliabilities for the Clinical norm sample are slightly higher than general-population sample reliabilities. Test–retest reliability scores have a median value for the scales at .76 for each age level, and in the low to middle .80s for the composites, with one exception. Construct validity for the Depression scale is supported as reported by the authors.

275

In addition to the TRS, PRS, and SRP, the BASC system also offers a Structured Developmental History (SDH) form, which serves to gather family history, and social and medical background information about the child, and a Student Observation System (SOS), used for recording the child’s classroom behaviors. The BASC components can be used separately or in combination to aid in the clinical diagnosis of child and adolescent disorders. The detection of problem behaviors as well as strengths and deficits in adaptive behaviors can assist the clinician in designing effective treatment plans.

Teacher Affect Rating Scale The Teacher Affect Rating Scale (TARS) was developed by Petti to evaluate depression through teacher report (cited by Cantwell, 1983). It consists of 26 items rated on a scale from 0 to 3 (“not at all” to “very much”) and is based on the child’s behavior in the past week. Items include information about the child’s work, time on task, concentration, mood, lability, and affect. It is hypothesized that the scale involves three factors: behavior, learning, and depression. This scale is still in the development stage and little information is available concerning its psychometric properties.

INTERVIEWS As discussed previously, a diagnosis of depression can be difficult to make if rating scales are the sole measure used. An overview of studies of adult depression found that rating scales are more likely reflect distress than depression (Coyne & Downey, 1991). Moreover, for children the main rating scale (CDI) has been found to be nonspecific and relatively insensitive for the diagnosis of depression (Hodges, 1994). In addition, results from the CDI and several scales similar to it (RADS, DSRS, CDRS, and Reynolds Child Depression Scale) may reflect the child’s inability or unwillingness to identify the key symptoms of depression and yield spuriously low results not indicative of depressive symptoms. Not only do clinical interviews allow for the evaluation of depressive symptoms, they also provide for the ability to assess disorders that are

reyn2-12.qxd

276

6/6/2003

9:18 AM

Page 276

IV. SPECIFIC SYNDROMES AND SYMPTOMS

found to be comorbid with depression. Moreover, several of the interviews allow for follow-up questions to further clarify the areas of concern. Interviews vary in terms of structured or semistructured status. Interviews that allow for additional questions and probes into the behaviors are considered semistructured. These interviews include the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS; Orvaschel & Puig-Anitch, 1987; Puig-Antich & Chambers, 1978), Interview Schedule for Children (ISC; Kovacs, 1984), and the Child Assessment Schedule (Hodges, 1990). Structured interviews allow for little variation in the questions and do not allow for further questioning. These interviews are designed to be administered by lay administrators, unlike the semistructured interviews that require trained professionals. The structured interviews include the Diagnostic Interview for Children and Adolescents (DICA; Herjanic, Herjanic, Brown, & Wheatt, 1975) and the Diagnostic Interview Schedule for Children (DISC; Costello, Edelbrock, Dulcan, Kalas, & Klaric, 1984). Each of these interviews has been modified to comply with the most recent version of DSM-IV. Moreover, each has parallel forms for parent interviews and the K-SADS allows for a teacher interview. One of the difficulties with the semistructured and structured interviews is that the concordance between informants can be very poor. As previously mentioned, several studies using interviews with parents, teachers, and the child have found less than optimum agreement. However, the consistency of the same informant across measures has been found to be adequate in several studies (Brunshaw & Szatmari, 1988; Jensen et al., 1996; Kazdin, 1994; Kazdin, French, & Unis, 1980), whereas others have found discrepancy between self-report measures and a structured interview (Pellegrino, Singh, & Carmanico, 1999). In an interesting twist on this issue, Bidaut-Russell, Reich, Cottler, and Robins (1995) evaluated 51 pairs of parents and adolescents who answered 12 questions from the DISC and were asked to guess how the other member of the pair would answer. The adolescents explained the discrepancy in terms of the parent forgetting or being unaware of the symptoms.

In contrast, the parent reported that the adolescent did not remember how he or she felt, lied, or did not recognize the symptoms or minimized the significance of the symptom. One of the ways to understand such poor concordance between parent and teacher interviews is that there may be valid differences between a child’s behavior at home and at school that can affect the expression of affective disorders. A landmark study by Leon, Kendall, and Garber (1980) found that depressed children exhibited different types of behavior problems depending on the environment. A sample of 138 children in third to sixth grade were diagnosed as depressed or nondepressed based on a number of rating scales (PIC, CDI, Conners Parent and Teacher Rating scales). The depressed children showed more conduct problems, anxiety, impulsivity, learning problems, psychosomatic difficulty, and perfectionism at home than did nondepressed children. In contrast, the depressed children were more inattentive and passive in school than they were at home. Behaviors were also found to vary depending on the age of the child, with younger children showing more conduct problems and older children showing more anxiety. Concordance among informants also appears to vary based on the disorder. Hodges, Gordon, and Lennon (1990) compared parent and child agreement and found that high concordance exists for externalizing behaviors with moderate agreement on affective symptoms. Parents tended to report more conduct problems and children tended to report more anxiety, somatic, and family problems. In a study evaluating the concordance between parent and child reports in a sample of nonreferred children, low concordance for affective difficulties was found and moderate concordance for externalizing behaviors (Thompson, Merritt, Keith, Murphy, & Johndrow, 1993). In addition, the concordance was found to be a function of age and gender, with poorer agreement for younger children and for males. Costello, Edelbrock, and Costello (1985) using a nonpsychiatric sample found that mild oppositional behavior was more frequently reported by parents and mild separation anxiety, fears, and dysthymia were more commonly reported by the child.

reyn2-12.qxd

6/6/2003

9:18 AM

Page 277

12. Assessment of Childhood Depression

Age may also have an effect on the reliability and validity of the interview. In a study comparing parent and child report, Edelbrock, Costello, Dulcan, Kalas, and Conover (1985) found that the reliability of the child’s report increases with age, being lowest for ages 6–9 and best for ages 10–23. In contrast, the parent’s report reliability decreased with the age of the child, possibly due to changes in parent perception and awareness of the child’s feelings and behavior. Schwab-Stone, Fallon, Briggs, and Crowther (1994) found that parent reliability was good using a structured clinical interview with ages 6–12, while the children being assessed reported fewer symptoms and were particularly unreliable in reporting duration, frequency, and onset of symptoms. The best reliability by report for these children was for affective disorders and ADHD with the poorest for oppositional defiant disorder. In addition, variables such as cognitive ability, age, and gender have been found to be related to the consistency of reports (Fallon & Schwab-Stone, 1994). Schawb-Stone (1995) suggests that children below the age of 12 have difficulty answering the questions on the DISC. Moreover, the clinician needs to be particularly sensitive to the child’s understanding of the questions as well as in his or her ability to gather such information through collecting and combining data from other informants. Multicultural issues in the use of structured interviews (and, for that matter, behavioral self-ratings) have been largely ignored in the literature. There are few studies evaluating the use of such interviews with populations that do not fit the typical Caucasian, middle-class scenario. Ezpetela, de la Osa, Domenech, Navarro, and Losilla (1997) used the Spanish adaptation of the DICA-R. Test–retest reliability was good and similar to findings with the English version. The parent report was the most stable with adolescent reports being less stable. Similar results were obtained by de la Osa, Ezpeleta, Domenech, and Navarro (1996) with a sample of Spanish school-age children and Bravos, Woodbury-Farina, Canino, and Rubio-Stipec (1993) with a sample of Puerto Rican children. A sample of children ages 9–27 from Puerto Rico was evaluated using the Spanish version of the DISC-2. Rubio-Stipec,

277

Canino, Shrout, and Dulcan (1994) found clinician diagnosis of depression to be more concordant with child report than parent report. Conversely, clinician report agreed more frequently with parent report of disruptive behaviors. Ribera, Canino, RubioStipec, and Bravo (1996) found that Spanish translations of the DISC-2 were reliable. Moreover, externalizing disorders showed higher reliability than did internalizing disorders. Reliability was also affected by the training of the examiner with psychiatrists having significantly higher reliability than lay interviewers. Roberts and colleagues (1995) compared the symptoms of DSM-III-R (American Psychiatric Association, 1987) among Anglo, African, and Mexican Americans ages 12–17. Similar symptoms were found across categories as well as prevalence. Similarly, Roberts, Solovitz, Chen, and Casat (1996) found that the DISC-2 to be reliable with male and female Anglo, African, and Hispanic American adolescents ages 12–17. Reliability was found to be somewhat higher for the African American sample than for Anglos or Hispanics. Males showed stronger reliability indices than did females and older adolescents showed better reliability than younger. Thus, it appears that several issues can affect the results of structured and semistructured interviews. Age, gender, informant, and possibly multicultural issues can affect the diagnostic utility. In addition, comparison of lay and professionally trained interviews may also affect the reliability of the results. There is strong empirical evidence that the age of the child reporting the symptoms and the agreement with the parent or teacher vary in reliability. The additional issues are not as well researched; however, the astute clinician will be sensitive to these variables with their assessment. In addition, for some of the symptoms of depression, a child who is forthcoming in his or her feelings may be the most appropriate informant. The clinical difficulty arises when a child is unable or unwilling to discuss uncomfortable feelings. In this case, interviews of parent and teacher as well as clinical observations would appear to be most important and informative. The five most commonly used interviews are discussed in the following section. These

reyn2-12.qxd

278

6/6/2003

9:18 AM

Page 278

IV. SPECIFIC SYNDROMES AND SYMPTOMS

interviews vary in format, training required for administration, and degree of structure. Each interview is discussed based on age range, reliability, validity, and suggestions for use. In addition the Children’s Depression Rating Scale—Revised (CDRS-R; Poznanski et al., 1979) is discussed. The CDRSR is frequently used in conjunction with a structured interview.

Schedule for Affective Disorders and Schizophrenia for School-Age Children The K-SADS is an adaptation of the interview for adults of the Schedule for Affective Disorders and Schizophrenia (SADS). It is intended for children and adolescents. There are two versions: K-SADS-P and KSADS-E. The K-SADS-P is appropriate for present episodes and the K-SADS-E for current and past episodes. The K-SADS allows for a diagnosis of major depression using DSM-IV criteria. It also allows for information to be gathered from parents, teachers, and the child. Age range for the K-SADS is 6–16 years of age. The K-SADS requires familiarity with DSM-IV criteria for major depression. It begins with an unstructured interview and proceeds to a semistructured format. The items begin with a definition of the symptoms and continue with probes designed to determine severity, duration, and frequency of difficulty. The items are clustered together by diagnosis. The K-SADS allows the examiner to reword questions and to query beyond the items if deemed appropriate. It is recommended that the parent be interviewed first and the child second. When discrepancies between reporters are found, the clinician is charged to use clinical judgment (Ambrosini, Metz, Prabucki, & Lee, 1988). Reliability over time has been found to be strongest for major depression, bipolar, generalized anxiety, conduct disorder, and oppositional defiant disorder with good reliability for posttraumatic stress disorder and ADHD (Kaufman, Birmaher, Brent, & Rao, 1997). Adolescent report has been found to be more reliable than child report (Weissman, Warner, & Fendrich, 1990). Validity data are generally in the moderate range. Agreement with behavior rating scales has been found to be in the moderate range (McCauley, Mitchell, Burke, & Moss,

1988) as well as with other semistructured interviews (Cohen, O’Connor, Lewis, & Velez, 1987). The K-SADS is the most widely used interview for the study of depression (Hodges, 1994). It shows strong agreement with alternatively determined diagnoses as well as with response to treatment (Fine, Forth, Gilbert, & Haley, 1991). The disadvantage of the K-SADS is also its advantage in that it requires a highly trained professional who is able to integrate clinical information and knowledge with the results of the K-SADS. Such discretion can lead to varying diagnoses depending on the training and inclination of the clinician. It is critical for the practitioner to use strict DSM-IV criteria as well as to understand the degree of importance each informant brings to the diagnostic process.

Interview Schedule for Children The ISC (Kovacs & Beck, 1977) is a semistructured interview for children ages 8–13. This interview was originally developed to assess the occurrence of depression longitudinally and focuses on current symptom ratings rather than diagnosis (Kovacs, 1984). Symptom probes are concise and are to be applied verbatim. The ISC should be administered to the parent and then the child. The clinician actually makes three ratings: parent, child, and an overall summary rating. It is recommended that the clinician use clinical history, demographics, psychological testing, and situational variables as well as the ISC to arrive at a diagnosis (Kovacs et al., 1984). Reliability of this interview has not been comprehensively studied. One of the few studies using a test–retest interval of only a few hours resulted in excellent reliability (Last, Strauss, & Francis, 1987). In this study, the reliability for major depression was .84 and for dysthymia, .66. The short time interval likely resulted in spuriously high results (Last, 1987). Moderate correlations between informants has been found for mood (r = .52), vegetative symptoms (r = .55), and poor correlations for concentration/attention (r = .32) (Kovacs et al., 1984). Moderate correlations between the ISC and the CDI (r = .33) and with the CBCL depression scale (r = .38) have been reported (Paulaskas & Kovacs, 1984). The

reyn2-12.qxd

6/6/2003

9:18 AM

Page 279

12. Assessment of Childhood Depression

ISC has been used to evaluate the course of depression and has been found to be a valid indicator for the presence or absence of depression over time (Kovacs et al., 1984). Caution should be used with the ISC not only because it requires a highly trained professional for administration but also because its psychometric properties are largely unresearched. The clinician must be highly versed in DSM-IV criteria. The strength of the ISC is the flexibility it offers in the questions asked as well as its use as a research instrument into the course of depression.

279

addition, the CAS and the K-SADS showed good concordance for parent informants (Verhulst, Altthaus, & Berden, 1987) and fair for child report (Hodges et al., 1987). The CAS is a promising instrument that allows for different interviews based on the child’s age. Moreover, it provides extensive reliability and validity data. The provision of a computer scoring program as well as the clinician input is an innovation that makes this instrument unusual among the various interviews available.

Child Assessment Schedule

Diagnostic Interview Schedule for Children—2

The Child Assessment Schedule (CAS; Hodges, 1990) has three versions. The initial version was designed for younger children (5–7), the second is for ages 7–12, and the third is for use with adolescents (Hodges, Kline, Stern, Cytryn, & McKnew, 1982). There are 11 topics that can be grouped by environment, self-concept, and mood. Questions are embedded in each group that query about the child’s peers, school relations, family, hobbies, fears/worries, self-concept, mood, anger, and reality testing. Everyday functioning is explored by the clinician with probes into how the child understands his or her difficulties and his or her ability to solve problems. Scale scores can be generated for interview, each symptom, and for the topic areas. There is a manual available as well as a computerized program to provide diagnoses and scale scores (Hodges, 1990). The CAS has been found to be reliable showing good test–retest reliability (r > 0.80) (Hodges, Cools, & McKnew, 1989) and internal consistency (values of 80 or above) (Hodges, 1993; Hodges, & Saunders, 1989; Hodges, Saunders, Kashani, Hamlett, & Thompson, 1990). Validity has been established through comparison of the CAS with self-rating scales as well as with other interview schedules. Comparing the CAS with the Birleson Depression Self-Rating Scale (Birleson, 1981) the CAS showed more sensitivity to depressive symptoms (Kashani, Rosenberg, & Reid, 1989). The CBCL depression scale has been found to correlate significantly with the CAS depression subscale and with the CDI (Hodges, Kline, Stern, Cytryn, & McKnew, 1982). In

The DISC was initially developed at the National Institutes of Mental Health and designed for ages 6–17. The DISC-2 is an updated version incorporating DSM-IV. The DISC-2 is a highly structured interview with specified questions. Clinical expertise is not needed, and probes are provided to ask for additional clarification of responses so that experienced clinicians can evaluate the responses at a later date. There are separate forms of the DISC-2 for parents, teachers, and child. The parent and child forms yield scores in 27 symptom areas coded on a scale of 0–2. Test–retest reliability of the original DISC has been found to be satisfactory, ranging from .63 for the child version and .72 for the parent version. Fair to good test–retest reliability has been found for the DISC-2 for children and adolescents with stronger reliability evident for adolescents and for shorter test periods present (Breton, Bergeron, Valla, Berthiaume, & St. Georges, 1998; Jensen, Roper, Fisher, & Piacentini, 1995). These findings have been replicated with better reliability reported for the parent than for the child interview (Schwab-Stone, Fisher, Piacentini, & Shaffer, 1993; Shaffer, Fisher, Dulcan, & Davies, 1996). Interrater reliability has also been found to be high using the DISC-2 with children ages 11–17 (Shaffer, Schwab-Stone, Fisher, & Cohen, 1993). Validity studies comparing the DISC with the K-SADS have found moderate agreement between the two interviews (Cohen et al., 1987). Comparison of the DISC-2 with clinician-generated diagnoses also showed moderate levels of agreement (Piacentini,

reyn2-12.qxd

280

6/6/2003

9:18 AM

Page 280

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Shaffer, Fisher, & Schwab-Stone, 1993). In a study that interviewed youths ages 9–17, the DISC-2 was found to show good validity across a number of diagnoses, including major depression and moderate validity for anxiety disorders and dysthymia (SchwabStone, Shaffer, Dulcan, & Jensen, 1996). The sensitivity of the DISC-2 for depression and suicidality was evaluated by King and colleagues (1997). The DISC-2 was found to correctly identify depressive symptoms and suicidality with good agreement between the DISC-2, the RADS, the CDRS, and the Suicide Ideation Questionnaire, suggesting good concurrent validity. The parent version has been found to be more sensitive than the child version, particularly for depression and suicidality (Fisher, Shaffer, Piacentini, & Lapkin, 1993; King et al., 1997). The DISC-2 has shown good reliability and validity. There have been several revisions of the DISC and it is important for the clinician to have the most recent version. Fewer studies for reliability and validity are present for younger children. The studies that do exist for the DISC are poor for the younger sample and it is likely that the DISC should not be used for children under the age of 12. Bird and colleagues (1987) raised the issue that children under the age of 11 appear to have difficulty with the interview and the results may yield overdiagnosis of depression in a child of this age. In fact, an epidemiological research study found an overdiagnosis of depression at all age levels using the DISC (Bird, Gould, Yager, Staghezza, & Canino, 1989). It is unclear whether this finding continues with the DISC-2. The DISC was purported to be appropriate for administration by lay interviewers; however, as Hodges (1994) rightly points out, most of the studies have used trained professionals. Thus, it is not established whether the DISC-2 can be used with lay interviewers. Given the concerns about the DISC and younger children it is likely is that it should not be used with these children, and for use with older youths, a trained professional is required.

Diagnostic Interview for Children and Adolescents The DICA was the first interview designed solely for children. It gives both current and

past diagnoses and is designed for children ages 6–17. There are separate forms for parent and child and it is to be administered by experienced clinicians. Specificity has been an area of concern with the original DICA in that it was found to discriminate well for overall pathology but poorly for individual diagnoses (Sylvester, Hyde, & Reichler, 1987). The DICA-R has separate interviews for children under age 12 and adolescents to age 17 as well as a parent interview. In the child and adolescent versions the wording is changed to assist the child in understanding the questions (Reich, 1988). An analysis of the difference between the child and adolescent versions found that a high level of psychological impairment was related to poorer test–retest reliability (Perez, Ascaso, Masson, & de la Osa, Chaparro, 1998). Moreover, lower reliability for the child form was found for the longest questions, internalizing disorders, report of the duration of the symptoms, and how accurately the child compared him- or herself with others. For the adolescent form, reliability was affected by internalizing content, report of the duration of the symptoms, and the evaluation of the degree of impairment involved in the disorder. Parent and child concordance has been difficult to establish for the DICA. Many studies have found poor concordance (Earls, Reich, Jung, & Cloninger, 1988; Kashani, Orvaschel, Burk, & Reid, 1985) with some finding fair concordance (Vitiello, Malone, Buschle, Delaney, & Behar, 1990; Welner, Reich, Herjanic, & Jung, 1987). These findings need to be replicated with the DICA-R. The agreement of the DICA with clinician-generated hypotheses was found to be low to moderate possibly due to the tendency for clinicians to evaluate symptoms in a different manner and the strictness they use in applying DSM-IV to the diagnosis (Ezpeleta, de la Osa, Domench, & Navarro, 1997). Conversely, the DICA-R has been found to show high agreement between psychiatrist’s diagnosis and DICA-R diagnosis (Boyle, Offord, Racine, & Sandford, 1993). Furthermore, Welner and colleagues (1987) found 81.5 % agreement between the DICA and clinician-generated diagnosis for externalizing behaviors with somewhat lower

reyn2-12.qxd

6/6/2003

9:18 AM

Page 281

12. Assessment of Childhood Depression

though adequate agreement for internalizing disorders. Thus, the DICA-R is a promising interview that needs additional validity research conducted in order for it to become more widely used. Data concerning dysthymia have not been forthcoming and further research is needed in this area (Hodges, 1994). The DICA-R has most frequently been used as a research tool with less use of the computerized program. Moreover, the use of the DICA-R with review by a trained professional has been found to increase the diagnostic specificity and accuracy and helps to offset a tendency to overdiagnose depression in children and adolescents (Kashani et al., 1987).

CONCLUSIONS In conclusion, it appears that the diagnosis of depression in childhood necessitates the use of various measures that tap into different areas of the child’s life. From the studies reviewed previously, it appears that the combining of cognitive assessment with behavioral data is necessary for an accurate diagnosis of depression. Particularly important is information from the child as to subjective feelings and thoughts to which parents and teachers may not be privy. Parental input is important in determining time frames and duration of possible depressive symptoms. Of recurring importance is the interaction of parental psychopathology with parental reports of depression in children (Poznanski, 1982). Parents who are themselves depressed may overidentify the problems their child has or withdraw from interaction with the child. It would appear important to obtain information from both parents to try to offset this tendency. In addition, it may be a valuable tool in therapy to discuss the various points of view garnered from separate clinical interviews with parents. Additional areas of concern are with the psychometric instruments themselves. The instruments reviewed earlier differ in their definition of depression, the criteria used to diagnose depression, the measurement of severity and duration of symptoms, the standardization on normal populations versus clinical populations, and the ability to

281

discriminate depression from other psychopathologies. In fact, this ability to distinguish depressed children from those with other types of psychopathology has repeatedly been cited as one of the weaknesses of most of the measures reported (Kazdin & Petti, 1982). Reliability and validity studies are also needed to ascertain the utility of some of the methods. Of great concern is the tendency of the various measures to validate one another against each other. The foremost and most widely used instrument, the CDI, is often used to establish the credibility of a new instrument. Unfortunately, the CDI has not been found to discriminate among conduct disorder, anxiety disorders, and depression in psychiatric populations unless the scores are at the extreme range. Although a manual has recently been published, there are several unanswered questions as to the psychometric properties of this instrument. Much of the depression research relies on identifying groups using the CDI or the CDRS-R. Both instruments have limited information as to their underlying validity and reliability. The use of structured and semistructured interviews appears to be more promising in the identification of depression in childhood. The questions involved in these interviews are problematic for younger children and such interviews are questionable at best. However, these interviews used with parents have provided diagnostic information that has been found to be helpful in identifying those children most at risk. As previously discussed, a comprehensive assessment of depression should include multi-instrument, multi-informant methods. However, recent studies examining assessment procedures show that comprehensive assessment is not widely practiced, and that more uniformity is needed among clinicians when identifying depression in children. Although the CDI was never intended to diagnose depression, a recent study shows that it is still the most often used self-report inventory, found in more than three-fourths of the studies that use self-report inventories (Fristad, Emery, & Beck, 1997). Although half of all studies on childhood depression use the CDI, two-thirds of these do not use an additional clinical structured interview for diagnosis. In 44% of the studies that

reyn2-12.qxd

282

6/6/2003

9:18 AM

Page 282

IV. SPECIFIC SYNDROMES AND SYMPTOMS

used the CDI alone, those children scoring high were referred to as “depressed” with no explanation of the CDI’s limitations. In a study that surveyed school psychologists about their understanding of childhood depression (Clarizio & Payette, 1990), only 51% used the DSM-III-R classification system for a diagnosis. Sixty-four percent reported using their own knowledge, alone or in combination with other criteria. Techniques used were primarily unstructured, informal interviewing (79%), with observation methods listed as second and self-report third. When instruments were listed, many school psychologists reported projective techniques (the Thematic Apperception Test, for example), particularly those with 5 or less years’ experience. Parent rating was only used by 3% of psychologists surveyed. Overall, these findings indicate that although school psychologists may use more than one assessment source, the sources are usually more subjective and informal than objective and structured. It would appear most appropriate to take a developmental stance in our understanding and evaluation for the presence of depression. The expression of depression repeatedly has been found to vary with age and in some research with gender. Multicultural studies are sadly lacking in this area and the applicability of standard instruments to varying populations needs to be carefully completed. The use of these instruments in various cultures is just beginning to be explored (Frias, del Barrio, & Mestre, 1991). Therefore, it appears that there are numerous researchable questions to be addressed in the area of assessment. Outcome research is scarce, and the comparison of differing types of therapeutic interventions is rare. The use of typically developing populations for standardization of the instruments is helpful. In addition, the use of various ages in the assessment of clinical populations showing anxiety and depression assists in our understanding of childhood affective disorders. Although the literature has generally concluded that childhood depression exists, the measures of this depression vary according to report source, assessment method used, cutoff scores used to delineate severity, and definition of symptomatology. All these variables

require more careful scrutiny. Studies with broad-based populations, both general and clinical, ideally will aid in our understanding of the underpinnings of childhood depression, and in our manner of intervention.

REFERENCES Abramson, L. Y., Metalsky, G. I., & Alloy, L. B. (1989). Hopelessness depression: A theory-based subtype of depression. Psychological Review, 96(2), 358–372. Abramson, L. Y., Seligman, M. E. P., & Teasdale, J. D. (1978). Learned helplessness in humans: Critique and reformulation. Journal of Abnormal Psychology, 87, 40–47. Achenbach, T. M., & Edelbrock, C. S. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington: University of Vermont. Allen, L., & Majidi-Ahi, S. (1989). Black American children. In J. T. Gibbs & L. N. Huang (Eds.), Children of color: Psychological interventions with minority youth (pp. 148–178). San Francisco: JosseyBass. Alloy, L. B., & Abramson, L. Y. (1979). Judgment of contingency in depressed and nondepressed students. Sadder but wiser? Journal of Experimental Psychology General, 108, 441–485. Alloy, L. B., Hartlage, S., & Abramson, L. Y. (1988). Testing the cognitive diathesis-stress theories of depression: Issues of research design, conceptualization, and assessment. In L. B. Alloy (Ed.), Cognitive processes in depression (pp. 31–73). New York: Guilford Press. Ambrosini, P. J., Metz, C., Prabucki, K., & Lee, J. (1988). Video tape reliability of the third revised edition of the K-SADS. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 723– 728. American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Asarnow, J. R., & Bates, S. (1988). Depression in child psychiatric inpatients: Cognitive and attributional patterns. Journal of Abnormal Child Psychology, 16(6), 601–615. Asarnow, J. R., & Carlson, G. A. (1985). Depression self-rating scale: Utility with child psychiatric inpatients. Journal of Consulting and Clinical Psychology, 53, 491–499. Asarnow, J. R., Tompson, M., Hamilton, E. B., Goldstein, M. J., & Guthrie, D. (1994). Family-expressed emotion, childhood-onset depression, and childhood-onset schizophrenia spectrum disorders: Is expressed emotion a non-specific correlate of child

reyn2-12.qxd

6/6/2003

9:18 AM

Page 283

12. Assessment of Childhood Depression psychopathology or a specific risk factor for depression? Journal of Abnormal Child Psychology, 22, 129–146. Barnett, D., Manly, J. T., & Cicchetti, D. (1991). Continuing toward an operational definition of psychological maltreatment. Development and Psychopathology, 3, 19–29. Bath, H. I., & Middleton, M. R. (1985). The Children’s Depression Scale: Psychometric properties and factor structure. Australian Journal of Psychology, 37, 81–88. Beck, A. T. (1967). Depression: Clinical, experimental, and theoretical aspects. New York: Hoeber. Beck, A. T. (1976). Cognitive therapy and emotional disorder. New York: International Universities Press. Beck, D. C., Carlson, G. A., Russell, A. T., & Brownfield, F. E. (1987). Use of depression rating instruments in developmentally and educationally delayed adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 97–100. Bemporad, J. R. (1994). Dynamic and interpersonal theories of depression. In W. M. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 81–96). New York: Plenum Press. Berard, R. M. F., Boermeester, F., Hartman, N., & Rust, A. L. (1997). The use of depression rating scales in an adolescent psychiatric population: Sex and age differences. International Journal of Adolescent Medicine and Health, 9, 313–320. Bernstein, G. A. (1991). Comorbidity and severity of anxiety and depressive disorders in a clinic sample. Journal of the American Academy of Child and Adolescent Psychiatry, 30, 43–50. Bidaut-Russell, M., Reich, W., Cottler, L. B., & Robins, L. N. (1995). The Diagnostic Interview Schedule for Children (PC-DISC v. 3. 0): Parents and adolescents suggest reasons for expecting discrepant answers. Journal of Abnormal Child Psychology, 23, 641–659. Bird, H. R., Canino, G., Gould, M. S., Riberta, J., Rubio-Stipec, M., Woodbury, M., Huertas-Goldman, S., & Sesman, M. (1987). Use of the Child Behavior Checklist as a screening instrument for epidemiological research in child psychiatry: Results of a pilot study. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 207–213. Bird, H. R., Gould, M. S., Yager, T., Staghezza, B., & Canino, G. (1989). Risk factors for maladjustment in Puerto Rican children. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 847–850. Birleson, P. (1981). The validity of depressive disorder in childhood and the development of a self-rating scale: A research report. Journal of Child Psychology and Psychiatry, 22, 73–88. Birleson, P., Hudson, I., Buchanan, D. G., & Wolff, S. (1987). Clinical evaluation of a self-rating scale for depressive disorder in childhood (Depression SelfRating Scale). Journal of Child Psychology and Psychiatry and Allied Disciplines, 28, 43–60. Bloom, B. L. (1985). A factor analysis of self-report measures of family functioning. Family Process, 24, 225–239. Bloom, B. L., & Naar, S. (1994). Self-report measures

283

of family functioning: Extensions of a factor analysis. Family Process, 33, 203–216. Boyle, M. H., Offord, D. R., Racine, Y., & Sanford, M. (1993). Evaluation of the Diagnostic Interview for Children and Adolescents for use in general population samples. Journal of Abnormal Child Psychology, 21, 663–681. Bravos, M., Woodbury-Farina, M., Canino, G. J., & Rubio-Stipec, M. (1993). The Spanish translation and cultural adaptation of the Diagnostic Interview Schedule for Children (DISC) in Puerto Rico. Culture, Medicine, and Psychiatry, 17, 329–344. Breton, J-J., Bergeron, L., Valla, J-P., Berthiaume, C., & St. Georges, M. (1998). Diagnostic Interview Schedule for Children (DISC-2. 25) in Quebec: Reliability findings in light of the MECA study. Journal of the American Academy of Child and Adolescent Psychiatry, 37, 1167–1174. Brown, L. K., Overholser, J., Spirito, A., & Fritz, G. K. (1991). The correlates of planning in adolescent suicide attempts. Journal of the American Academy of Child and Adolescent Psychiatry, 30, 95–99. Brunshaw, J. M., & Szatmari, P. (1988). The agreement between behavior checklists and structured psychiatric interviews for children. Canadian Journal of Psychiatry, 33, 474–481. Cantwell, D. (1983). Depression in childhood: Clinical picture and diagnostic criteria. In D. Cantwell & G. Carlson (Eds.), Affective disorders in childhood and adolescence (pp. 3–18). New York: Spectrum. Carey, M. P., Faulstich, M. E., Gresham, F. M., Ruggiero, L., & Enyart, P. (1987). Children’s Depression Inventory: Construct and discriminant validity across clinical and nonreferred (control) populations. Journal of Consulting and Clinical Psychology, 55, 755–761. Carlson, G. A., & Cantwell, D. (1980). A survey of depressive symptoms, syndromes, and disorders in a child psychiatric population. Journal of Child Psychology and Psychiatry, 21, 19–25. Carlson, G. A., & Kashani, J. H. (1988). Phenomenology of major depression from childhood through adulthood: Analysis of three studies. American Journal of Psychiatry, 145(10), 1222–1225. Cicchetti, D., Gaiban, J., & Barnett, D. (1991). Contributions from the study of high-risk populations to understanding the development of emotion regulation. In J. Garber & K. Dodge (Eds.), The development of emotion regulation and dysregulation (pp. 15–48). New York: Cambridge University Press. Cicchetti, D., & Schneider-Rosen, K. (1978). An organizational view of affect: Illustration from the study of Down’s syndrome infants. In M. Lewis & L. A. Rosenblum (Eds.), The development of affect (pp. 309–350). New York: Plenum Press. Clarizio, H. F. (1994). Assessment of depression in children and adolescents by parents, teachers, and peers. In W. H. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 235–248). New York: Plenum Press. Clarizio, H. F., & Payette K. (1990). A survey of school psychologists’ perspectives and practices with childhood depression. Psychology in the Schools, 27, 57–63.

reyn2-12.qxd

284

6/6/2003

9:18 AM

Page 284

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Cohen, P., O’Connor, P. Lewis, S., & Velez, C. N. (1987). Comparison of DISC and K-SADS-P interviews of an epidemiological sample of children. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 662–667. Cohen-Sandler, R., Berman, A., & King, R. (1982). Life stress and symptomatology: Determinants of suicidal behavior in children. Journal of the American Academy of Child Psychiatry, 21, 178–186. Cole, D. A., & Carpentieri, S. (1990). Social status and the comorbidity of child depression and conduct disorders. Journal of Consulting and Clinical Psychology, 58, 748–757. Cole, P. M., & Kaslow, N. J. (1988). Interactional and cognitive strategies for affect regulation: Developmental perspective on childhood depression. In L. B. Alloy (Ed.), Cognitive processes in depression (pp. 310–343). New York: Guilford Press. Cole, P. M., & Rehm, L. P. (1986). Family interaction patterns and childhood depression. Journal of Abnormal Child Psychology, 14, 297–314. Costello, E. J., Edelbrock C. S., & Costello, A. J. (1985). Validity of the NIMH Diagnostic Interview Schedule for Children: A comparison between psychiatric and pediatric referrals. Journal of Abnormal Child Psychology, 13, 579–595. Costello, E. J., Edelbrock, C. S., Dulcan, M. K., Kalas, R., & Klaric, S. H. (1984). Report on the NIMH Diagnostic Interview Schedule for Children (DISC). Washington, DC: National Institute of Mental Health. Coyne, J., & Downey, G. (1991). Social factors and psychopathology: Stress, social support, and coping processes. Annual Review of Psychology, 42, 401–425. Crowley, S. L., & Worchel, F. F. (1993). Assessment of childhood depression: Sampling multiple data sources with one instrument. Journal of Psychoeducational Assessment, 11, 242–249. Crowley, S. L., Worchel, F. F., & Ash, M. J. (1992). Self-report, peer-report, and teacher-report measures of childhood depression: an analysis by item. Journal of Personality Assessment, 59, 189–203. Cummings, E. M., & Cicchetti, D. (1990). Toward a transactional model of relations between attachment and depression. In M. T. Greenberg, D. Cicchetti, & E. M. Cummings (Eds.), Attachment in the preschool years: Theory, research, and intervention (pp. 339–372). Chicago: University of Chicago Press. Dalley, M. B., Bolocofsky, D. N., Alcorn, M. B., & Baker, C. (1992). Depressive symptomatology, attributional style, dysfunctional attitude, and social competency in adolescents with and without learning disabilities. School Psychology Review, 21, 444–458. Davis, N. (1990). The Reynolds Adolescent Depression Scale. Measurement and Evaluation in Counseling and Development, 23, 88–91. Dawson, G. (1994). Development of emotional expression and emotion regulation in infancy: Contributions of the frontal lobe. In G. Dawson & K. W. Fischer (Eds.), Human behavior and the developing brain (pp. 346–379). New York: Guilford Press.

Dawson, G., Frey, K., Panagiotides, H., Yamada, E., Hessl, D., & Osterling, J. (1999). Infants of depressed mothers exhibit atypical frontal electrical brain activity during interactions with mother and with a familiar, nondepressed adult. Child Development, 70(5), 1058–1066. de la Osa, N., Ezpeleta, L., Domenech, J. M., & Navarro, J. B. (1996). Interrater reliability of the Diagnostic Interview for Children and Adolescents (DICA-R). Psicothema, 8, 359–368. deMesquita, P. B., & Gilliam, W. S. (1994). Differential diagnosis of childhood depression: Using comorbidity and symptom overlap to generate multiple hypotheses. Child Psychiatry and Human Development, 24, 157–172. Di Giuseppe, R. A. (1981). Cognitive therapy with children. In G. Emery, S. D. Hollon, & R. C. Bedrosian (Eds.), New directions in cognitive therapy: A casebook (pp. 50–98). New York: Guilford Press. Doerfler, L. A., Felner, R. D., Rowlison, R. T., & Raley, P. A. (1988). Depression in children and adolescents: A comparative analysis of the utility and construct validity of two assessment measures. Journal of Consulting and Clinical Psychology, 56, 769–772. Earls, F., Reich, W., Jung, K. G., & Cloninger, C. (1988). Psychopathology in children of alcoholic and antisocial parents. Alcoholism: Clinical and Experimental Research, 12, 481–487. Eason, L. J., Finch, A. J., Jr., Brasted, W., & Saylor, C. (1985). The assessment of depression and anxiety in hospitalized pediatric patients. Child Psychiatry and Human Development, 16, 57–64. Edelbrock, C., Costello, A. J., Dulcan, M. K., Kalas, R., & Conover, N. C. (1985). Age differences in the reliability of the psychiatric interview of the child. Child Development, 56, 265–275. Epkins, C. C., & Meyers, A. W. (1994). Assessment of childhood depression, anxiety and aggression: convergent and discriminant validity of self-, parent-, teacher-, and peer-report measures. Journal of Personality Assessment, 62, 364–381. Ezpeleta, L., de la Osa, N., Domench, J. M., & Navarro, J. B. (1997). Diagnostic agreement between clinicians and the Diagnostic Interview for Children and Adolescents—DICA-R—in an outpatient sample. Journal of Child Psychology and Psychiatry and Allied Disciplines, 38, 431–440. Ezpeleta, L., de la Osa, N., Domenech, J. M., Navarro, J. B., & Losilla, J. M. (1997). Test–retest reliability of the Spanish adaptation of the Diagnostic Interview of Children and Adolescents. Psicothema, 9, 529–539. Fallon, T., & Schwab-Stone, 1994). Determinants of reliability in psychiatric surveys of children aged 6–22. Journal of Child Psychology and Psychiatry and Allied Disciplines, 35, 1391–1408. Field, T. M. (1984). Early interactions between infants and their postpartum depressed mothers. Infant Behavior and Development, 7, 517–522. Fine, S., Forth, A., Gilbert, M., & Haley, G. (1991). Group therapy for adolescent depressive disorder: A comparison of social skills and therapeutic support.

reyn2-12.qxd

6/6/2003

9:18 AM

Page 285

12. Assessment of Childhood Depression Journal of the American Academy of Child and Adolescent Psychiatry, 30, 79–85. Fine, S., Moretti, M., Haley, G., & Marriage, K. (1984). Depressive disorder in children and adolescents: Dysthymic disorder and the use of self-rating scales in assessment. Child Psychiatry and Human Development, 14, 223–229. Fisher, P. W., Shaffer, D., Piacentini, J., & Lapkin, J. (1993). Sensitivity of the Diagnostic Interview Schedule for Children, 2nd edition (DISC-2. 1) for specific diagnoses of children and adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 666–673. Frias, D., del Barrio, V., & Mestre, V. (1991). The Children’s Depression Inventory (CDI): Its psychometric properties in foreign and Spanish populations. Evaluacion Psicologica, 7, 377–391. Fristad, M. A., Emery, B. L., & Beck, S. J. (1997). Use and abuse of the children’s depression inventory. Journal of Consulting and Clinical Psychology, 65, 699–702. Garber, J., Braafladt, N., & Zeman, J. (1991). The regulation of sad affect: An information-processing perspective. In J. Garber & K. Dodge (Eds.), The development of emotion regulation and dysregulation (pp. 208–237). New York: Cambridge University Press. Garland, E. J., & Weiss, M. (1995). Subgroups of adolescent depression. Journal of the American Academy of Child and Adolescent Psychiatry, 34, 831. Gibbs, J. T., & Huang, L. N. (1989). Children of color: Psychological interventions with minority youth. San Francisco: Jossey-Bass. Gittelman-Klein, R., & Klein, D. F. (1973). School phobia: Diagnostic considerations in the light of imipramine effects. Journal of Nervous and Mental Disorders, 156, 199–215. Hagborg, W. J. (1992). Prevalence and correlates of self-reported depressive mood among seriously emotionally disturbed adolescents. Psychological Reports, 70, 23–26. Hains, A. A. (1992). Comparison of cognitive-behavioral stress management techniques with adolescent boys. Journal of Counseling and Development, 70, 600–605. Hammen, C., & Compas, E. B. (1994). Unmasking masked depression in children and adolescents: The problem of comorbidity. Clinical Psychology Review, 14, 585–603. Hammen, C. L., & Krantz, S. (1976). Effect of success and failure on depressive cognitions. Journal of Abnormal Psychology, 85, 577–586. Hammen, C., & Rudolph, K. D. (1996). Childhood depression. In E. J. Mash & R. A. Barkley (Eds.), Child psychopathology (pp. 153–195). New York: Guilford Press. Harrington, R., Rutter, M., & Fombonne, E. (1996). Developmental pathways in depression: Multiple meanings. Development and Psychopathology, 8, 601–616. Havey, J. M., & Dodd, D. K. (1992). Environmental and personality differences between children of alcoholics and their peers. Journal of Drug Education, 22, 215–222.

285

Herjanic, B., Herjanic, M., Brown, F., & Wheatt, R. (1975). Are children reliable reporters? Journal of Abnormal Child Psychology, 3, 41–48. Hodges, K. (1990). Depression and anxiety in children: A comparison of self-report questionnaires to clinical interview. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 376–381. Hodges, K. (1993). Structured interviews for assessing children. Journal of Child Psychology and Psychiatry and Allied Disciplines, 34, 49–68. Hodges, K. (1994). Evaluation of depression in children and adolescents using diagnostic clinical interviews. In W. H. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 183–208). New York: Plenum Press. Hodges, K., Cools, J., & McKnew, D. (1989). Test–retest reliability of a clinical research interview for children: The Child Assessment Schedule. Psychological Assessment, 1, 317–322. Hodges, K., & Craighead, W. E. (1990). Relationship of Children’s Depression Inventory factors to diagnose depression. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 489–492. Hodges, K., Gordon, Y., & Lennon, M. P. (1990). Parent-child agreement on symptoms assessed via a clinical research interview for children: The Child Assessment Schedule (CAS). Journal of Child Psychology and Psychiatry and Allied Disciplines, 31, 427–436. Hodges, K., Kline, J., Stern, L., Cytryn, L., & McKnew, D. (1982). The development of a child assessment interview for children: The Child Assessment Schedule (CAS). Journal of Child Psychology and Psychiatry, 31, 427–436. Hodges, K., McKnew, D., Burbach, D. J., & Roebuck, L. (1987). Diagnostic concordance between the Child Assessment Schedule (CAS) and the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS) in an outpatient sample using lay interviewers. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 654–661. Hodges, K., & Saunders, W. (1989). Internal consistency of a diagnostic interview for children: The Child Assessment Schedule. Journal of Abnormal Child Psychology, 17, 691–701. Hodges, K., Saunders, W., Kashani, J., Hamlett, K., & Thompson, R. (1990). Internal consistency of DSM III diagnoses using the symptom scales of the Child Assessment Schedule (CAS). Journal of the American Academy of Child and Adolescent Psychiatry, 29, 635–641. Ines, T. M., & Sacco, W. P. (1992). Factors related to correspondence between teacher ratings of elementary student depression and student self-ratings. Journal of Consulting and Clinical Psychology, 60, 140–142. Ivarsson, T., Lidberg, A., & Gillberg, C. (1994). The Birleson Depression Self-Rating Scale (DSRS): Clinical evaluation in an adolescent inpatient population. Journal of Affective Disorders, 32, 115–125. Ivens, C., & Rehm, L. P. (1988). Assessment of childhood depression: correspondence between reports

reyn2-12.qxd

286

6/6/2003

9:18 AM

Page 286

IV. SPECIFIC SYNDROMES AND SYMPTOMS

by child, mother and father. Journal of the American Academy of Child and Adolescent Psychiatry, 27, 738–741. Jensen, P., Roper, M., Fisher, P., & Piacentini, J. (1995). Test–retest reliability of the Diagnostic Interview Schedule for Children (DISC 2. 1): Parent, child, and combined algorithms. Archives of General Psychiatry, 52, 61–71. Jensen, P. S., Watanabe, H. K., Richters, J. E., Roper, M., Hibbs, E. D., Salzberg, A. D., & Liu, S. (1996). Scales, diagnoses, and child psychopathology: II. Comparing the CBCL and the DISC against external validation. Journal of Abnormal Child Psychology, 24, 151–168. Kahn, J. S., Kehle, T. J., & Jenson, W. R. (1987, March). Depression among middle-school students: Descriptive and correlational analyses. Paper presented at the annual convention of the National Association of School Psychologists, New Orleans. Kandel, D. B., & Davies, M. (1986). Adult sequelae of adolescent depressive symptoms. Archives of General Psychiatry, 43, 255–262. Kashani, J. H., Allan, W. D., Beck, N. C., Bledsoe, Y, & Reid, J. C. (1997). Dysthymic disorder in clinically referred preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 36(10), 1426–1433. Kashani, J. H., Beck, N. C., Hoeper, E. W., Fallahi, C., Corcoran, C. M., McAllister, J. A., Rosenberg, T. K., & Reid, J. C. (1987). Psychiatric disorders in a community sample of adolescents. American Journal of Psychiatry, 144, 584–589. Kashani, J. H., & Carlson, G. A. (1987). Seriously depressed preschoolers. American Journal of Psychiatry, 144(3), 348–350. Kashani, J. H., Holcomb, W. R., & Orvaschel, H. (1986). Depression and depressive symptoms in preschool children from the general population. American Journal of Psychiatry, 143(9), 1138–1143. Kashani, J. H., Orvaschel, H., Burk, J. P., & Reid, J. C. (1985). Informant variance: The issue of parentchild disagreement. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 701–706. Kashani, J. H., & Ray, J. S. (1983). Depressive related symptoms among preschool-age children. Child Psychiatry and Human Development, 13(4), 233–238. Kashani, J. H., Rosenberg, T., & Reid, J. (1989). Developmental perspectives in child and adolescent depressive symptoms in a community sample. American Journal of Psychiatry, 146, 871–875. Kaslow, N. J., Brown, R. T., & Mee, L. L. (1994). Cognitive and behavioral correlates of childhood depression: A developmental perspective. In W. M. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 97–122). New York: Plenum Press. Kaslow, N. J., Deering, C. G., & Racusin, G. R. (1994). Depressed children and their families. Clinical Psychology Review, 14(1), 39–59. Kaufman, J., Birmaher, B., Brent, D., & Rao, U. (1997). Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL): Initial reliability and va-

lidity data. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 980–988. Kazdin, A. E. (1981). Assessment techniques for childhood depression. Journal of the American Academy of Child Psychiatry, 20, 358–375. Kazdin, A. E. (1987). Children’s Depression Scale: Validation with child psychiatric inpatients. Journal of Child Psychology and Psychiatry, 28, 29–41. Kazdin, A. E. (1988). Childhood depression. In E. J. Mash & L. G. Terdal (Eds.), Behavioral assessment of childhood disorders (2nd ed., pp. 157–195). New York: Guilford Press. Kazdin, A. E. (1990). Childhood depression. Journal of Child Psychology and Psychiatry, 31(1), 121–160. Kazdin, A. E. (1994). Informant variability in the assessment of childhood depression. In W. M. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 249–271). New York: Plenum Press. Kazdin, A. E., French, N. H., & Unis, A. S. (1980). Child, mother, and father evaluations of childhood depression in psychiatric inpatient children. Journal of Abnormal Child Psychology, 11, 167–180. Kazdin, A. E., French, N. H., Unis, A. S., Esveldt-Dawson, & Sherick, R. B. (1983). The Hopelessness Scale for Children: Psychometric characteristics and concurrent validity. Journal of Consulting and Clinical Psychology, 51, 504–510. Kazdin, A. E., & Petti, T. A. (1982). Self-report and interview measures of childhood and adolescent depression. Journal of Child Psychology and Psychiatry, 23, 437–457. Kazdin, A. E., Rodgers, A., & Colbus, D. (1986). The Hopelessness Scale for Children: Psychometric characteristics and concurrent validity. Journal of Consulting and Clinical Psychology, 54, 241–245. Kendall, P. C., & Braswell, L. (1982). Assessment for cognitive-behavioral interventions in the schools. School Psychology Review, 11, 21–30. Kendall, P. C., Stark, K. D., & Adam, T. (1990). Cognitive deficit or cognitive distortion in childhood depression? Journal of Abnormal Child Psychology, 18, 267–283. Kerr, M. M., Hoier, T. S., & Versi, M. (1987). Methodological issues in childhood depression: A review of the literature. American Journal of Orthopsychiatry, 57, 193–198. King, C. A., Katz, S. G., Ghaziuddin, N., Brand, E, Hill, E., & McGovern, L. (1997). Diagnosis and assessment of depression and suicidality using the NIMH Diagnostic Interview Schedule for Children (DISC-2. 3). Journal of Abnormal Child Psychology, 25, 173–181. King, C. A., Raskin, A., Gdowski, C. L., Butkus, M., & Opipari, L. (1990). Psychosocial factors associated with urban adolescent female suicide attempts. Journal of the American Academy of Child and Adolescent Psychiatry, 29, 289–294. Knight, D., Hensley, V. R., & Waters, B. (1988). Validation of the Children’s Depression Scale and the Children’s Depression Inventory in a prepubertal sample. Journal of Child Psychology and Psychiatry, 29, 853–863. Kobak, R., Sudler, N., & Gamble, W. (1991). Attach-

reyn2-12.qxd

6/6/2003

9:18 AM

Page 287

12. Assessment of Childhood Depression ment and depressive symptoms during adolescence: A developmental pathways analysis. Development and Psychopathology, 3, 461–474. Kopp, C. B. (1989). Regulation of distress and negative emotions: A developmental view. Developmental Psychology, 25(3), 343–354. Kovacs, M. (1981). Rating scales to assess depression in school-aged children. Acta Paedopsychiatrica, 46, 305–315. Kovacs, M. (1984). The Interview Schedule for Children (ISC). Psychopharmocology Bulletin, 21, 991–994. Kovacs, M., & Beck, A. T. (1977). Maladaptive cognitive structures in depression. American Journal of Psychiatry, 135, 525–533. Kovacs, M., Feinberg, T. L., Crouse-Novack, M. A., Paulauskas, S. L., & Finkelstein, R. (1984). Depressive disorders in childhood: I. A longitudinal prospective study of characteristics and recovery. Archives of General Psychiatry, 41, 229–237. Kovacs, M., & Paulauskas, S. L. (1984). Developmental stage and the expression of depressive disorders in children: An empirical analysis. New Directions in Child Development, 26, 59–80. Lang, M., & Tisher, M. (1978). Children’s Depression Scale. Melbourne: Australian Council for Educational Research. Last, C. G. (1987). Developmental considerations. In C. G. Last & M. Hersen (Eds.), Issues in diagnostic research (pp. 201–216). New York: Plenum Press. Last, C. G., Strauss, C. C., & Francis, G. (1987). Comorbidity among childhood anxiety disorders. Journal of Nervous and Mental Disease, 175, 726–730. Lefkowitz, M. M., & Burton, N. (1978). Childhood depression. Psychological Bulletin, 85, 716–726. Leon, G. R., Kendall, P. C., & Garber, J. (1980). Depression in children: Parent, teacher, and child perspectives. Journal of Abnormal Child Psychology, 8, 221–235. Lewinsohn, P. M. (1974). A behavioral approach to depression. In R. J. Friedman & M. M. Katz (Eds.), The psychology of depression: Contemporary theory and research (pp. 53–89). New York: Wiley. Lewinsohn, P. M. (1975). The behavioral study and treatment of depression: In M. Hersen, R. M. Eisler, & P. M. Miller (Eds.), Progress in behavior modification (Vol. 1, pp. 19–64). New York: Academic Press. Lewinsohn, P. M., Roberts, R. E., Seeley, J. R., & Rogde, P. (1994). Adolescent psychopathology: II. Psychosocial risk factors for depression. Journal of Abnormal Psychology, 103, 302–315. Lyman, R. D., & Hembree-Kigin, T. L. (1994). Mental health interventions with preschool children. New York: Plenum Press. Lyons-Ruth, K., Zeanah, C. H., & Benoit, D. (1996). Disorder and risk for disorder during infancy and toddlerhood. In E. J. Mash & R. A. Barkley (Eds.), Child psychopathology (pp. 457–491). New York: Guilford Press. McCauley, E., Mitchell, J. R., Burke, P., & Moss, S. (1988). Cognitive attributes of depression in children and adolescents. Journal of Consulting and Clinical Psychology, 56, 903–908.

287

McCauley, E., & Myers, K. (1992). Family interactions in mood disordered youth. Child and Adolescent Psychiatric Clinics of North America, 1, 111–127. McKnew, D. H., Cytryn, L., Efron, A. M., Gershon, E. S., & Bunney, E. W. (1979). Offspring of patients with affective disorders. British Journal of Psychiatry, 134, 148–152. Miller, L., Warner, V., Wickramaratne, P., & Weissman, M. (1999). Self-esteem and depression: Ten year follow-up of mothers. Journal of Affective Disorders, 52, 41–49. Mokros, H. B., Poznanski, E., Grossman, J. A., & Freeman, L. N. (1987). A comparison of child and adolescent parent ratings of depression for normal and clinically referred children. Journal of Child Psychology and Psychiatry, 28, 613–627. Mullins, L. L., Siegel, L. J., & Hodges, K. (1985). Cognitive problem-solving and life event correlates of depressive symptoms in children. Journal of Abnormal Child Psychology, 13, 305–314. Nelson, W. M., & Politano, P. M. (1990). Children’s Depression Inventory: Stability over repeated administrations in psychiatric inpatient children Journal of Clinical Child Psychology, 19, 254–256. Nelson, W. M., Politano, P. M., Finch, A. J., Wendel, N., & Mayhall, C. (1987). Children’s Depression Inventory: Normative data and utility with emotionally disturbed children. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 43–48. Newman, J. R., & Garfinkel, B. (1992). Major depression in childhood and adolescence. In S. R. Hooper, G. W. Hynd, & R. E. Mattison (Eds.), Child psychopathology: Diagnostic criteria and clinical assessment (pp. 65–106). Hillsdale, NJ: Erlbaum. Nieminen, G. S., & Matson, J. L. (1989). Depressive problems in conduct-disordered adolescents. Journal of School Psychology, 27, 175–186. Nurcombe, B. (1994). The validity of the diagnosis of major depression in children and adolescence. In W. H. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 61–80). New York: Plenum Press. Orvaschel, H., & Puig-Antich, J. H. (1987). Schedule for Affective Disorders and Schizophrenia for School-Age Children (Epidemiologic version, 4th ed.). Pittsburgh, PA: Western Psychiatric Institute and Clinic. Paulauskas, S. L., & Kovacs, M. (1984). The Interview Schedule for Children (ISC): A symptom-oriented psychiatric interview. Paper presented at the annual conference of the American Psychological Association, Toronto. Pellegrino, J. F., Singh, N. N., & Carmanico, S. J. (1999). Concordance among three diagnostic procedures for identifying depression in children and adolescents with EBD. Journal of Emotional and Behavioral Disorders, 7, 118–127. Perez, R. G., Ascaso, E. L., Domench, J. M., & de la Osa-Chaparro, N. (1998). Characteristics of the subject and interview influencing the test–retest reliability of the Diagnostic Interview for Children and Adolescents—Revised. Journal of Child Psychology and Psychiatry and Allied Disciplines, 39, 963–972. Piacentini, J., Shaffer, D., Fisher, P. W., & Schwab-

reyn2-12.qxd

288

6/6/2003

9:18 AM

Page 288

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Stone, M. (1993). The Diagnostic Interview Schedule for Children—Revised version (DISC-R): III. Concurrent criterion validity. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 658–665. Poznanski, E. O. (1982). The clinical phenomenology of childhood depression. American Journal of Orthopsychiatry, 52, 308–313. Poznanski, E. O. (1984). Preliminary studies of the reliability and validity of the Children’s Depression Rating Scale. Journal of the American Academy of Child Psychiatry, 23, 191–197. Poznanski, E. O., Cook, S. C., & Carroll, B. J. (1979). A depression rating scale for children. Pediatrics, 64, 442–450. Puig-Antich, J., & Chambers, W. (1978). The Schedule for Affective Disorders and Schizophrenia for School-Age Children (Kiddie-SADS). New York: New York State Psychiatric Institute. Puig-Antich, J., Lukens, E., Davies, M., Goetz, D., Brennan-Quattrock, J., & Todak, G. (1985). Psychosocial functioning in prepubertal major depressive disorders: I. Interpersonal relationships during the depressive episode. Archives of General Psychiatry, 42, 500–507. Puig-Antich, J., & Rabinovitch, J. (1986). The relationship between affective and anxiety disorders in childhood. In R. Gittelman (Ed.), Anxiety disorders of childhood (pp. 108–120). New York: Guilford Press. Puura, K., Tamminen, T., Almquist, F., & Kresanov, K. (1997). Should depression in young school-children be diagnosed with different criteria? European Child and Adolescent Psychiatry, 6, 12–19. Radke-Yarrow, J., Cummings, E. M., Kuczynski, L., & Chapman, M. (1985). Patterns of attachment in two-and three-year-olds in normal families and families with parental depression. Child Development, 56, 884–893. Ramirez, O. (1989). Mexican-American children and adolescents. In J. T. Gibbs & L. N. Huang (Eds.), Children of color: Psychological interventions with minority youth (pp. 224–250). San Francisco: Jossey-Bass. Rawson, H. E., & Tabb, L. C. (1993). Effects of therapeutic intervention on childhood depression. Child and Adolescent Social Work Journal, 10, 39–51. Rehm, L. P. (1977). A self-control model of depression. Behavior Therapy, 8, 787–804. Reich, W. (1988). DIS Newsletter, 5, 8–9. Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: AGS. Reynolds, W. M. (1986). Reynolds Adolescent Depression Scale. Odessa, FL: Psychological Assessment Resources. Reynolds, W. M. (1987). Reynolds Adolescent Depression Scale: Professional manual. Odessa, FL: Psychological Assessment Resources. Reynolds, W. M. (1989). Reynolds Child Depression Scale. Odessa, FL: Psychological Assessment Resources. Reynolds, W. M. (1993). Self-report methods. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child

and adolescent assessment (pp. 98–125). New York: Plenum Press. Reynolds, W. M. (1994). Assessment of depression in children and adolescents by self-report questionnaires. In W. H. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 209–234). New York: Plenum Press. Reynolds, W. M. (1999). Adolescent Psychopathology Scale. Odessa, FL: Psychological Assessment Resources. Reynolds, W. M., & Graves, A. (1988). Reliability of children’s reports of depressive symptomatology. Journal of Abnormal Child Psychology, 17, 647–655. Reynolds, W. M., & Mazza, J. J. (1998). Reliability and validity of the Reynolds Adolescent Depression Scale with young adolescents. Journal of School Psychology, 36, 295–312. Reynolds, W. M., & Miller, K. L. (1985). Depression and learned helplessness in mentally retarded and nonretarded adolescents: An initial investigation. Applied Research in Mental Retardation, 6, 295–307. Ribera, J. C., Canino, G., Rubio-Stipec, M., & Bravo, M. (1996). The Diagnostic Interview Schedule for Children (DISC-2. 1) in Spanish: Reliability in a Hispanic population. Journal of Child Psychology and Psychiatry and Allied Disciplines, 37, 195–204. Roberts, R. E. (1994). An exploration of depression among Mexican-origin and Anglo adolescents. In R. G. Malgady (Ed.) Theoretical and conceptual issues in Hispanic mental health (pp. 92–112). Malabar, FL: Krieger. Roberts, R. E., & Chen, Y-W. (1995). Depressive symptoms and suicidal ideation among Mexicanorigin and Anglo adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 34, 81–90. Roberts, R. E., Chen, R., & Roberts, C. R. (1997). Ethnocultural differences in prevalence of adolescent suicidal behaviors. Suicide and Life-Threatening Behavior, 27, 208–217. Roberts, R. E., Chen, Y-W., & Solovitz, B. L. (1995). Symptoms of DSM-III-R major depression among Anglo, African, and Mexican-American adolescents. Journal of Affective Disorders, 36, 1–9. Roberts, R. E., Roberts, C. R., & Chen, Y. R. (1997). Ethnocultural differences in prevalence of adolescent depression. American Journal of Community Psychology, 25, 95–110. Roberts, R. E., Solovitz, B. L., Chen, Y-W., & Casat, C. (1996). Retest stability of DSM IIIR diagnoses among adolescents using the Diagnostic Interview Schedule for Children (DISC-2. 1C). Journal of Abnormal Child Psychology, 24, 349–362. Rochlin, G. (1959). The loss complex. Journal of the American Psychoanalytic Association, 7, 299–316. Rotundo, N., & Hensley, V. R. (1985). The Children’s Depression Scale: A study of its validity. Journal of Child Psychology and Psychiatry, 26, 917–927. Rubio-Stipec, M., Canino, G. J., Shrout, P., & Dulcan, M. K. (1994). Psychometric properties of parents and children as informants in child psychiatry epidemiology with the Spanish Diagnostic Interview

reyn2-12.qxd

6/6/2003

9:18 AM

Page 289

12. Assessment of Childhood Depression Schedule for Children (DISC. 2). Journal of Abnormal Child Psychology, 22, 703–720. Sacco, W. P., & Graves, D. J. (1984). Childhood depression, interpersonal problem-solving, and selfratings of performance. Journal of Clinical Child Psychology, 13, 10–15. Sattler, J. M. (2002). Assessment of children. San Diego, CA: Author. Saylor, C. F., Finch, A. J., Baskin, C. H., Saylor, C. B., Darnell, G., & Furey, W. (1984). Children’s Depression Inventory: Investigation of procedures and correlates. Journal of the American Academy of Child Psychiatry, 23, 626–628. Saylor, C. F., Finch, A. J., Spirito, A., & Bennett, B. (1984). The Children’s Depression Inventory: A systematic evaluation of psychometric properties. Journal of Consulting and Clinical Psychology, 52, 955–967. Schoenert-Reichl, K. A. (1994). Gender differences in depressive symptomatology and egocentrism in adolescence. Journal of Early Adolescence, 14, 49–64. Schwab-Stone, M. (1995). Discussion of: Do children aged 9 through 11 years understand the DISC version 2. 25 questions? Journal of the American Academy of Child and Adolescent Psychiatry, 34, 954–956. Schwab-Stone, M., Fallon, T., Briggs, M., & Crowther, B. (1994). Reliability of diagnostic reporting for children aged 6–21 years: A test–retest study of the Diagnostic Interview Schedule for Children—Revised. American Journal of Psychiatry, 151, 1048–1054. Schwab-Stone, M., Fisher, P. W., Piacentini, J., & Shaffer, D. (1993). The Diagnostic Interview Schedule for Children—Revised version (DISC-R): II. Test–retest reliability. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 651–657. Schwab-Stone, M. E., Shaffer, D., Dulcan, M. K., & Jensen, P. S. (1996). Criterion validity of the NIMH Diagnostic Interview Schedule for Children version 2. 3 (DISC-2. 3). Journal of the American Academy of Child and Adolescent Psychiatry, 35, 878–888. Schwartz, M., Friedman, R., Lindsay, P., & Narrol, H. (1982). The relationship between conceptual tempo and depression in children. Journal of Consulting and Clinical Psychology, 50, 489–490. Seligman, M. E. P. (1975). Helplessness: On depression, development, and death. San Francisco: Freeman. Seligman, M. E. P., Peterson, C., Kaslow, N. J., Tanenbaum, R. L., Alloy, L. B., & Abramson, L. Y. (1984). Attributional style and depressive symptoms among children. Journal of Abnormal Psychology, 93(2), 235–238. Semrud-Clikeman, M., & Hynd, G. W. (1991). Review of issues and measures in childhood depression. School Psychology International, 12, 275–298. Shaffer, D., Fisher, P., Dulcan, M. K., & Davies, M. (1996). The NIMH Diagnostic Interview Schedule for Children Version 2. 3 (DISC-2. 3): Description, acceptability, prevalence rates, and performance in the MECA study. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 865–877.

289

Shaffer, D., Schwab-Stone, M., Fisher, P. W., & Cohen, P. (1993) The Diagnostic Interview Schedule for Children—Revised version (DISC-R): I. Preparation, field testing, interrater reliability, and acceptability. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 643–650. Shain, B. N., Kronfol, Z., Naylor, M., Goel, K., Evans, T., & Schaefer, S. (1991). Natural killer cell activity in adolescents with major depression. Biological Psychiatry, 29, 481–484. Shain, B. N., Naylor, M., & Alessi, N. (1990). Comparison of self-rated and clinician-rated measures of depression in adolescents. American Journal of Psychiatry, 147, 793–795. Sroufe, L. A., & Rutter, M. (1984). The domain of developmental psychopathology. Child Development, 55, 17–29. Stark, K. D., Sander, J. B., Yancy, M. G., Bronik, M. D., & Hoke, J. A. (2000). Treatment of depression in childhood and adolescence: Cognitive-behavioral procedures for the individual and family. In P. C. Kendall (Ed.), Child and adolescent therapy: Cognitive-behavioral procedures (2nd ed., pp. 173–234). New York: Guilford Press. Stark, K. D., Humphrey, L. L., Crook, K., & Lewis, K. (1990). Perceived family environments of depressed and anxious children: Child’s and maternal figure’s perspectives. Journal of Abnormal Child Psychology, 18, 527–547. Stark, K. D., Reynolds, W. M., & Kaslow, N. J. (1987). A comparison of the relative efficacy of selfcontrol therapy and behavioral problem-solving therapy for depression in children. Journal of Abnormal Child Psychology, 15, 91–113. Sylvester, C., Hyde, T., & Reichler, R. (1987). The Diagnostic Interview for Children and Personality Inventory for Children in studies of children at risk for anxiety disorders or depression. Journal of the American Academy of Child and Adolescent Psychiatry, 26, 668–675. Takeuchi, K., Roberts, R. E., & Suzuki, S. (1994). Depressive symptoms among Japanese and American adolescents. Psychiatry Research, 53, 259–274. Thompson, R. J., Jr. Merritt, K. A., Keith, B. R., Murphy, L. B., & Johndrow, D. A. (1993). Mother– child agreement on the Child Assessment Schedule with nonreferred children: A research note. Journal of Child Psychology and Psychiatry and Allied Disciplines, 34, 813–820. Tisher, M., Lang-Takac, E., & Lang, M. (1992). The Children’s Depression Scale: Review of Australian and overseas experience. Australian Journal of Psychology, 44, 27–35. Trad, P. V. (1994). Depression in infants. In W. M. Reynolds & H. F. Johnston (Eds.), Handbook of depression in children and adolescents (pp. 401–426). New York: Plenum Press. Verhulst, F. C., Althaus, M., & Berden, G. F. (1987). The Child Assessment Schedule: Parent–child agreement and validity measures. Journal of Child Psychology and Psychiatry and Allied Disciplines, 28, 455–466. Vitiello, B., Malone, R., Buschle, P. R., Delaney, M. A., & Behar, D. (1990). Reliability of DSM III diag-

reyn2-12.qxd

290

6/6/2003

9:18 AM

Page 290

IV. SPECIFIC SYNDROMES AND SYMPTOMS

noses of hospitalized children. Hospital and Community Psychiatry, 41, 63–67. Weiss, B., Weisz, J. R., Politano, M., Carey, M., Nelson, W. M., & Finch, A. (1991). Developmental differences in the factor structure of the Children’s Depression Inventory. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3, 38–45. Weiss, B., Weisz, J. R., Politano, M., & Carey, M. (1992). Relations among self-reported depressive symptoms in clinic-referred children versus adolescents. Journal of Abnormal Psychology, 101, 391–397. Weisz, J. R., Sweeney, L., Proffitt, V., & Carr, T. (1993). Control-related beliefs and self-reported depressive symptoms in late childhood. Journal of Abnormal Psychology, 102, 411–418. Welner, Z., Reich, W., Herjanic, B., & Jung, K. G. (1987). Reliability, validity, and parent–child agree-

ment studies of the Diagnostic Interview for Children and Adolescents (DICA). Journal of the American Academy of Child and Adolescent Psychiatry, 26, 649–653. Willcutt, E. G., Hartung, C. M., Lahey, B. B., Loney, J., & Pelham, W. E. (1999). Utility of behavior ratings by examiners during assessments of preschool children with attention-deficit/hyperactivity disorder. Journal of Abnormal Child Psychology, 27, 463–472. Wirt, R. D., Lachar, D., Seat, P. D., & Broen, W. E., Jr. (2001). Personality Inventory for Children—2nd edition. Los Angeles, CA: Western Psychological Services. Zero to Three/National Center for Infants, Toddlers, and Families. (1994). Diagnostic classification: 0–3. Diagnostic classification of mental health and developmental disorders of infancy and early childhood. Arlington, VA: Author.

reyn2-13.qxd

6/6/2003

9:19 AM

Page 291

13 The Assessment of Attention via Continuous Performance Tests

CYNTHIA A. RICCIO CECIL R. REYNOLDS

Problems with attention or concentration in one form or another are common concerns, if not the primary concern, when children and youth are referred for comprehensive evaluation; indeed, such problems are common among children generally. Attention is a key component in cognitive functioning (Siegel, 2000) and the neural traces of attention form the foundation of memory (Cohen, 1993b; Sohlberg & Mateer, 1989); furthermore, attention can be viewed as the process that controls the flow of information processing in the brain (Siegel, 2000). Consistent with this view, Broadbent (1953, 1957) proposed that the individual’s capacity to take in information is limited, and, therefore, information that is not relevant needs to be filtered out. He further proposed that characteristics of the stimulus (i.e., intensity, importance, and novelty) determined whether specific information would be filtered out or attended to. With its presumed relation to cognition, information processing, and memory, it is not surprising that attentional problems are the most frequent type of cognitive impairments resulting from neurological disorders or dysfunction (Cohen, Malloy, & Jenkins, 1999; Mapou, 1999).

When parents and teachers indicate that a child’s attention or concentration is a concern, however, defining or measuring what is meant by attention is not a simple task. As used in common discourse, attention is not unitary but has been described as a highly complex construct (Mirsky, Fantie, & Tatman, 1995). Although many theories of attention have focused on arousal (Hebb, 1958; Moruzzi & Magoun, 1949; van Zomeran & Brouwer, 1994), current conceptualizations of attention are concerned with more than the individual’s arousal level (e.g., Posner & Peterson, 1990; Voeller, 1991). It has been suggested that attention should be conceptualized as having multiple components or elements. These components include (1) the initiating or focusing of attention; (2) sustaining attention or vigilance; (3) inhibiting responses to irrelevant stimuli or selective attention; and (4) shifting attention (Mirsky, 1989; Mirsky, Anthony, Duncan, Ahearn, & Kellam, 1991; Sohlberg & Mateer, 1989; Zubin, 1975). Due to the need for attentional shifts and attentional flexibility in the self-regulatory and inhibition processes, attentional processes held a central role in Luria’s (1966) model of normal and abnormal 291

reyn2-13.qxd

292

6/6/2003

9:19 AM

Page 292

IV. SPECIFIC SYNDROMES AND SYMPTOMS

brain function and are closely linked to executive control. Both executive control and attention are also necessary for initiation or generation of the response to a specific stimulus, for maintenance of the response or shifting of the response, and for the flexibility needed to meet changing task demands (Cohen, 1993c; Whyte, 1992). The multifaceted nature of attention and executive control makes it difficult to map these behaviors onto specific neurological structures. Current functional system models include cortical and subcortical structures as well as connecting pathways and projections with increasing levels of complexity and are reviewed elsewhere (see Cohen & O’Donnell, 1993; Riccio, Reynolds, & Lowe, 2001; van Zomeran & Brouwer, 1994). The complex functional system models that integrate both neuroanatomical and neurochemical influences on behavior provide a plausible explanation for the myriad manifestations of deficits in attention and executive control across disorders of disparate etiology (Voeller, 1991). Using these complex models, interference at any level of the functional system could lead to a cluster of clinically similar, and yet potentially different, behaviors depending not only on the location of the damage or dysfunction but also on the extent of that damage or dysfunction. Because compromise anywhere within the system may lead to attentional deficits regardless of etiology, the use of attention problems alone as a diagnostic consideration is suspect. At the same time, the assessment of attention may provide useful information for treatment planning across disorders. To this end, a number of tests have been developed for the purpose of measuring the many facets of attention (Cohen, 1993a, 1993c; Riccio & Reynolds, 2001). To rule out other confounds (e.g., cognitive ability), the task demands of measures of attention need to involve limited cognitive processing and, thus, need to be relatively simple. Similarly, to eliminate confounds with memory limitations, the memory load must be minimal as well. The specific components of attention that are of interest also will dictate task parameters. When sustained attention is the construct to be measured, repetition and duration of the task are key features. When focus or selec-

tive attention is of interest, the characteristics of the relevant and nonrelevant stimuli become more important. Parameters of task complexity, temporal demands, processing speed requirements, and task or target salience also need to be considered in the assessment of attention (Cohen et al., 1999). Computerized assessment of attention and executive control facilitates the accommodation of tasks to match these constraints. The continuous performance test (CPT) is one group of paradigms for the study of attention. Although not all CPTs are computerized, technology allows for virtually infinite variations to task parameters across differing CPTs. As we review the CPT literature and the multitude of CPT variations, we discuss the extent to which the parameters can change the task and influence the results obtained.

THE CPT PARADIGM The CPT is one group of paradigms for the evaluation of attention as well as response inhibition (or disinhibition). Most often, CPTs are used to obtain quantitative information regarding an individual’s ability to sustain attention over time. Of the myriad measures of attention available, the CPT is the most frequently used (DuPaul, Anastopoulos, Shelton, Guevremont, & Metevia, 1992). The original CPT was developed by Rosvold, Mirsky, Sarason, Bransome, and Beck in 1956 as a research tool to study vigilance. In the original version, letters were presented visually one at a time, at a fixed rate (interstimulus interval, or ISI) of 920 milliseconds. The individual was required to respond by pressing a lever whenever the letter “X,” designated as the target stimulus, appeared. At the same time, the individual was to inhibit responding when any other letter appeared; this is referred to as the X-type CPT. A variation of this task in which the target was the letter “X,” but only if the “X” was immediately preceded by the letter “A” (or an AX-type CPT) was used as well. Rosvold and colleagues found group differences to occur when comparing individuals with brain damage to controls on the X-type CPT; these between-group differences were magnified with the increased difficulty level of the AX-type CPT.

reyn2-13.qxd

6/6/2003

9:19 AM

Page 293

13. Continuous Performance Tests

CPT Variations and Effects Since 1956, the CPT has continued to be used in the study of attention as well as executive control, with multiple variations in the components of the task; today, the majority of CPTs are computer administered. The basic paradigm for the CPT consists of rapid presentation of continuously changing stimuli with a designated “target” stimulus or “target” pattern such that the individual is to respond (or inhibit responding) based on the stimulus presented. As such, it requires selective attention or vigilance for an infrequently occurring target or relevant stimulus; at the same time, the duration of the task is intended to be sufficient to measure sustained attention. Despite these general similarities, Halperin (1991) commented that there were as many versions of the CPT available as there were clinicians who used them, while Greenberg and Waldman (1993) suggested that there are more than 100 different versions of the CPT in use. Different CPTs include variations in the basic task or CPT type, the characteristics of the target, variations in the ISI, presence or absence of distractors, presence or absence of feedback following responses, modality of presentation, duration of the target presentation, duration of task, and so on (see Table 13.1). The effects of some of these possible variations and modifications to the CPT on performance have been reviewed elsewhere (see Ballard, 1996; Corkum & Siegel, 1993; Riccio et al., 2001) and are summarized here. Task Type The task may be the simpler X-type CPT, or an AX-type CPT, or a further modification of the AX such that the target must be preceded by itself (XX-type: e.g., Fitzpatrick, Klorman, Brumaghim, & Borgstedt, 1992) or where color and letter are critical features (e.g., orange T followed by blue S: Garfinkel & Klee, 1983). Another modification involves a change in the directions to respond except when the target is presented (not-X type) as in the Conners CPT (Conners, 1992, 1995) and the CPT II (Conners & MHS Staff, 2000). A number of studies have compared performance on the X-CPT to performance on the AX-CPT with consis-

293

tent findings that the AX-CPT is the more difficult task (e.g., Alexander, 1973; Goldstein, Rosenbaum, & Taylor, 1997). Similarly, when the X-CPT has been compared to the XX-CPT, findings supported the premise that the X-CPT is the easier task (Friedman, Vaughan, & ErlenmeyerKimling, 1978, 1981). Schachar, Logan, Wachsmuth, and Chajczyk (1988) compared the performance of 114 boys (7–11 years of age; clinic referrals) on an X-, XX-, and AX-CPT. Keeping all other parameters the same and counterbalancing to control for order effects, results indicated that there was a greater deterioration of performance over time on the XX-CPT and AX-CPT as compared to the X-CPT. In particular, Schachar and colleagues found decreased correct hits with the AX- and XX-CPT as compared to the X-CPT. In addition, results revealed faster reaction times for the AXCPT and increased commission errors (false alarms) on the XX-CPT (Schachar et al., 1988). Stimulus Characteristics Other differences include the type of stimulus (e.g., letters, numbers, shapes, words, and objects), or how the target is defined (e.g., position of small square within large square: Greenberg, 1988–1999). Drawing from Broadbent (1953, 1957), it would be expected that the characteristics of the target (i.e., novelty and semantic content) could affect the likelihood that irrelevant (nontarget) stimuli would be filtered out. Consistent with this, characteristics of the target (i.e., animal vs. letter, nonword vs. word, and verbal vs. nonverbal) have been found to have an impact on CPT performance (e.g., Earle-Boyer, Serper, Davidson, & Harvey, 1991; Harper & Ottinger, 1992; Shapiro, Morris, Morris, Flowers, & Jones, 1999). Another characteristic of the stimulus is the quality of the stimulus itself. In some studies, the stimulus is degraded in some manner. Evans (1988) compared performance on a standard X-CPT, the X-CPT with degraded (i.e., blurred) stimuli, and other variations. Findings revealed that the condition with degraded stimuli was the most difficult, while the standard X-CPT was the easiest. Clinicians need to keep in

reyn2-13.qxd

6/6/2003

9:19 AM

Page 294

294

IV. SPECIFIC SYNDROMES AND SYMPTOMS

TABLE 13.1. Variations in Continuous Performance Tests Variation

Examples

Possible effect(s)

Task type

X-type, AX-type, XX-type, not-X type, identical pairs

Difference in level of difficulty (e.g., Friedman et al., 1981; Schachar et al., 1988). Difference between initiation of response (e.g., on X-type) and inhibition of dominant response on not-X has not been studied.

Stimulus characteristics

Letters, numbers, shapes, words, objects, ear

Novelty and semantic content believed to affect the likelihood that irrelevant stimuli (nontargets) would be filtered out (e.g., Earle-Boyer et al., 1991; Shapiro et al., 1999).

Position on screen (or ear if No research available. auditory), change in position (or ear) within task Standard presentation degraded or blurred

Blurred or degraded presentation increased the difficulty level of the task (e.g., Evans, 1988).

Modality of presentation

Visual, auditory, or variable within task

Auditory presentation more difficult than visual (e.g., Baker et al., 1995; Sandford et al., 1995a, 1995b). No research available to compare single modality (visual or auditory) to variable within task modality.

Target frequency

High or low frequency

High frequency is believed to increase the likelihood of impulsive responding, while low frequency is believed to increase the likelihood of inattention (e.g., Beale et al., 1987).

Interstimulus interval (ISI)

Shorter, longer, variable, adaptive

ISIs that are either too short or too long have been found to yield increased error rates (e.g., Chee et al., 1989; Sykes et al., 1971)

Stimulus duration Shorter or longer

Duration of presentation was found to impact results (Chee et al., 1989).

Distractor conditions

Present or absent, same or different modality as stimulus

Distractors (auditory or visual) for visual CPTs resulted in increased errors (e.g., Crosby, 1972). Visual distractors resulted in poorer performance in other studies as well (e.g., Golier et al., 1997).

Task duration

Range of current CPTs is 3–30+ minutes

No comparative research available (i.e., how many minutes are sufficient? 9, 14, 23?).

Examiner presence

Present or absent

A decline in performance is associated with examiner absence (e.g., Draeger et al., 1986; Power, 1992)

Instructional set

Emphasis on speed, accuracy, or both

Instructional set has been found to affect performance (e.g., Leark et al., 1999; Sergeant & Scholten, 1985).

Provision of feedback or reinforcement

Feedback on performance, tangible reinforcers, response cost contingency

Research findings are equivocal (e.g., Corkum et al., 1996; Levy & Hobbes, 1988).

Equipment used

Size of screen, spacebar or mouse, type of mouse (if mouse), laptop or PC, headphones or speakers

No research is available on these differences.

reyn2-13.qxd

6/6/2003

9:19 AM

Page 295

13. Continuous Performance Tests

mind, however, that blurring may increase the information-processing component rather than, or in addition to, increasing attentional demands. Modality of Presentation Another variation involves changing the modality such that presentation may be visual as in the initial version, or it may be auditory (e.g., Earle-Boyer et al., 1991; Keith, 1994), or it may vary within the same task from auditory to visual stimuli (e.g., Sandford & Turner, 1994–1999). Modality differences have been investigated with associated differences found in CPT performance (e.g., Baker, Taylor, & Leyva, 1995; Draeger, Prior, & Sanson, 1986; Driscoll, 1994; Leark, Dupuy, Greenberg, Corman, & Kindschi, 1996; Sandford, Fine, & Goldman, 1995a, 1995b; Sandford & Turner, 1995). For example, Sandford and Turner (1995) reported better performance on visual than auditory tasks in their study, with children with attention-deficit/hyperactivity disorder (ADHD) making more commission errors when the stimuli were presented in the auditory format. Leark and colleagues (1996) reported comparisons between visual and auditory test formats. Results indicated that children made twice as many omission errors on the auditory task, but significantly more commission errors and faster reaction times on the visual task. At the same time, children demonstrated greater variability in response time on the auditory task (Leark et al., 1996). Finally, auditory CPTs have yielded higher correlations with verbal ability, language, and achievement as compared to visual CPTs (Kupietz & Richardson, 1978; Swanson & Cooney, 1989). Target Frequency Studies have varied the frequency of the target to provide a higher frequency or a lower frequency (e.g., Beale, Matthew, Oliver, & Corballis, 1987). Under the frequent target condition, a response set is created such that the dominant response is to press the mouse button, and, therefore, an individual who is impulsive is more likely to respond and to make commission errors. In contrast, the infrequent target condition requires vigilance

295

and response inhibition. It is believed that the stimulus–infrequent condition creates a situation in which a client who is inattentive is less likely to respond and, therefore, will make omission errors. If vigilance is not maintained during this segment of the test, the probability of omission errors will increase. The individual’s “readiness to respond” or arousal level may impact the number of omission errors as well (Beale et al., 1987; Sandford & Turner, 1995). ISI Differences The time lapse between presentations of the stimuli or the ISI has been varied with studies using a shorter interval or longer interval (e.g., Rueckert & Grafman, 1996; Sykes, Douglas, Weiss, & Minde, 1971); in at least some instances, the ISI has been found to affect CPT performance. For example, Sykes and colleagues (1971) assessed 40 children with hyperactivity and 19 control children (5–12 years of age) on multiple versions of the X- and AX-CPT with two ISIs (1,000 and 1,500 milliseconds). Results indicated a main effect for ISI for total correct responses and error scores such that both groups of children performed better when the ISI was longer regardless of other conditions. Related to ISI, yet discussed less frequently, is the stimulus onset asynchrony (SOA) which is the time between the onset of one stimulus and the next stimulus. The SOA is the sum of the duration of the stimulus presentation and the ISI, and, therefore, is sensitive to the effects on differences in duration of presentation as well as differences in ISI (e.g., Chee, Logan, Schachar, Lindsay, & Wachsmuth, 1989). Consistent with the findings for ISI, performance improved with longer SOAs in the Chee study. The ISI may vary within the task as well. Variable intervals may be preset such that for some blocks of trials the ISI is at one rate, while for other blocks the ISI is either longer or shorter (e.g., Conners, 1992, 1995; Conners & MHS Staff, 2000). In contrast, with an “adaptive” variable rate, the computer program automatically increases or decreases the ISI by 5% based on the accuracy of the last response (e.g., Brumm, 1994; Girardi et al., 1995; Rapoport et al., 1980). For CPTs with this format, the mean ISI (indicative of optimal

reyn2-13.qxd

6/6/2003

9:19 AM

296

Page 296

IV. SPECIFIC SYNDROMES AND SYMPTOMS

performance for that person) can be determined and is believed to reflect the information-processing speed of the individual.

interpret the results as indicative of sustained attention is directly linked to duration of the task.

Distractor Conditions

Examiner Presence or Absence

Other studies have included distractor conditions (e.g., Crosby, 1972; Golier et al., 1997; Hoy, Weiss, Minde, & Cohen, 1978). Crosby (1972) used visual X- and AX-CPTs with children under three conditions—no distractor, with auditory distractors, and with visual distractors. Results revealed deterioration in performance for all groups under the distractor conditions as compared to the no-distractor condition. Similarly, other studies (e.g., Golier et al., 1997; Hoy et al., 1978) consistently found a decline in performance for both the clinical and control groups under the distractor conditions.

Another factor found to affect CPT performance is examiner presence or absence during the administration of the CPT (e.g., Draeger et al., 1986; Power, 1992). Draeger and colleagues (1986) compared the CPT results for children under two conditions: examiner present or examiner absent. Results indicated that for children with ADHD as well as normal controls, a decline in performance occurred when the experimenter was absent; the deterioration in performance was greater for the children with ADHD. Results of the Power (1992) study supported this finding with some indications that children who were aggressive were more likely to demonstrate the deterioration of performance in the examinerabsent condition. Moreover, Power found that the deterioration in performance in the examiner absent condition was greater if the absent condition occurred second as opposed to first in the counterbalanced sequence.

Feedback/Reinforcement Still other studies examined the effects of providing feedback or reinforcement. As noted in Corkum and Siegel’s (1993) review, the provision of feedback (e.g., O’Dougherty, Nuechterlein, & Drew, 1984) and provision of tangible reinforcers (e.g., Levy & Hobbes, 1988) have been found to affect the overall performance of individuals. In at least one study, provision of feedback resulted in a higher hit rate (O’Dougherty et al., 1984). In another study, Levy and Hobbes (1988) found that a response cost contingency resulted in fewer errors of omission than either a no-feedback or reward contingency condition. In contrast, however, Corkum, Schachar, and Siegel (1996) found that the provision of incentives did not significantly affect CPT performance and concluded that motivational issues did not account for the vigilance decrement evidenced. Task Duration Another variable that differs across CPTs is the duration of the task itself. The condensed version of the MINI-CPT (Bremer, 1989) is probably one of the shortest CPTs with a duration of only 3 minutes. On the other hand, at least one CPT lasts over 30 minutes (Mansour, Haier, & Buchsbaum, 1996). Clearly, the extent to which one can

Instructional Set In addition to examiner presence or absence, differences in the instructional set or directions given to the individual have been investigated (e.g., Sergeant & Scholten, 1985; Tupler, 1989). Sergeant and Scholten (1985) administered the CPT under three different instructional set conditions—emphasis on speed, emphasis on accuracy, and emphasis on both speed and accuracy. Results indicated that CPT results were affected by the instructional set with decreased errors in the accuracy condition and increased errors in the speed condition. More recently, Leark, Dixon, Hoffman, and Huynh (1999) replicated these findings. They concluded that differences in performance could be elicited by changing the instructions given at the start of the CPT. Summary As can be seen from the effects of parameter differences on CPT performance, it is criti-

reyn2-13.qxd

6/6/2003

9:19 AM

Page 297

13. Continuous Performance Tests

cal to ensure that the normative data and clinical data are obtained under the same parameters with strict adherence to standardization procedures. Still other differences (e.g., the size of the computer screen, use of laptop vs. PC, use of mouse vs. space bar) remain to be investigated as possible confounds to be considered in the use of the CPT. Notably, in addition to the actual administration parameters, differing CPTs also provide different variables for interpretation of performance.

CPT Variables for Interpretation CPT variables reported often include correct hits (number or percent of correct responses to targets). In some cases, omission errors (i.e., number or percent of targets not responded to) are reported. Both correct hits and omission errors are interpreted as indicative of selective attention (e.g., Allen, 1993; Carter, Krener, Chaderjian, Northcutt, & Wolfe, 1995) as well as alertness (Gordon Systems, 1986). Commission errors (number or percent of responses to stimuli other than the target) are reported as an index of response inhibition. In some studies, relative accuracy (percent of correct responses of all responses made including incorrect responses) or total errors (combining omission and commission errors) may be reported (e.g., Bock, 1982). A number of researchers have investigated the types of commission errors that the individual makes on the CPT. One broad category of commission errors is referred to as target-related errors such that at least one item in the sequence is relevant to the correct target (e.g., X or A in the AX-CPT). Target-related commission errors are believed to be suggestive of anticipation or impulsivity (Gordon Systems, 1986; Halperin, Sharma, Greenblatt, & Schwartz, 1991). In contrast to target-related errors, random errors would be those responses to a sequence where neither item in the sequence was relevant to the target sequence. Random commission errors are not believed to be associated directly with inattention, impulsivity, or hyperactivity (Gordon Systems, 1986; Halperin, Wolf, Greenblatt, & Young, 1991) but, rather, to be reflective of dyscontrol (Halperin, Sharma, et al., 1991) or to be associated with problems in the individ-

297

ual’s level of arousal, or motivational level (Gordon Systems, 1986). Some programs provide a measure of multiple responses to the same stimulus (i.e., the individual responds more than once to the same stimulus). Multiple responses are believed to reflect hyperresponsivity (Sandford & Turner, 1995) and may be more consistent with motor disinhibition as opposed to impulsivity. Reaction time is another measure frequently reported with CPTs. Reaction time or response latency is believed to reflect the speed of processing as well as the speed of motor responding. For example, a child may demonstrate increased omission errors and a slower rate of responding without an associated increase in commission errors; this type of pattern may be interpreted as supporting a hypothesis of difficulty with the allocation of information-processing resources (Eliason & Richman, 1987). Reaction time is also important when considering the consistency or variability of the individual’s performance over time. Some CPT programs generate the standard deviation of the reaction time across blocks as a measure of consistency in responding and the ability to sustain attention over time. Alternatively, some researchers report the standard error of the reaction time (e.g., Conners, 1992, 1995; Conners & MHS Staff, 2000) as an indicator of the consistency or variability of responding over time. Still others use the standard deviation of the standard error over time as an indication of consistency (Levin et al., 1996). Although the use of variability of reaction time may be confounded to a large extent with IQ (see Jensen, 1982; Vernon, 1989), this variability statistic is nonetheless sensitive to central nervous system trauma and deserves considerably more study. Related to consistency, the vigilance decrement or the extent to which the individual’s accuracy declines over the course of the task is of interest as well (Parasuraman, 1984a, 1984b) and is considered an index of sustained attention (Cohen et al., 1999). Rather than using differences in the reaction time as a measure of consistency, some clinicians focus on comparisons of correct and incorrect responses over differing blocks of time within the same administration. To this end, some CPTs provide comparative information from blocks at the beginning of the

reyn2-13.qxd

6/6/2003

9:19 AM

298

Page 298

IV. SPECIFIC SYNDROMES AND SYMPTOMS

task and at the end of the task either directly (e.g., Conners, 1992, 1995) or via derived quotients (e.g., Allen, 1993; Levav, 1991; Sandford & Turner, 1995; Slicker, 1991). Signal Detection Theory As an alternative to direct performance scores, some clinicians incorporate signal detection theory (SDT) in generating performance indexes for interpretation (e.g., Klorman, Brumaghim, Fitzpatrik, & Borgstedt, 1991; Liu, Hwu, & Chen, 1997). The basic premise of SDT is that the decision to respond is based on the child’s setting a certain standard or criterion for responding. SDT variables of sensitivity (also referred to as d’ or d-prime) and response bias (also referred to as Beta) are based on signal to noise (i.e., target to nontarget) ratios (Cohen, 1993a). The calculation of these scores is based on the distributions of the child’s responses to both signal (targets) and noise (nontargets). For the CPT, sensitivity is derived from the mean distribution of responses to both targets and nontargets; it is equivalent to the difference between the sum of the distributions for both targets and nontargets, and nontargets alone (see Swets, 1973, 1984). Sensitivity is believed to represent the likelihood that the individual will detect the signal (respond to the target) when it is presented or the ability to discriminate targets from nontargets. As such, sensitivity is dependent on both the characteristics of the stimulus and the sensitivity of the individual. Related to sensitivity, response bias or response style is believed to reflect the extent to which the individual is being conservative or impulsive in responding. Response bias is presumed to relate to the strategy used in making the decision to respond or the individual’s response style. For example, if the individual tends to be cautious and conservative in responding, the individual is more likely to miss signals (increased omission errors) but to have fewer commission errors. On the other hand, if the response style is less cautious, the individual may have decreased omission errors but increased commission errors. It has been argued that the sensitivity and bias indexes may be more sensitive to differences in performance on the CPT than

omission or commission errors (Lam & Beale, 1991) and that SDT procedures may be particularly useful in neuropsychological assessment (Cohen, 1993c). SDT allows for the determination of response characteristics based on the frequency of the targets and is easily converted into indices for assessing the vigilance decrement and response inconsistency (Cohen et al., 1999). Although many clinicians and researchers use the measures of sensitivity and response bias in reporting CPT scores, concerns have been raised by others as to the applicability of SDT to the CPT paradigm (e.g., Jerison, 1967; Parasuraman, 1979). In particular, it has been argued that the CPT does not meet the suggested criterion for SDT application in that it does not require successive discrimination of stimuli, but relies on sequential processing (Parasuraman & Davies, 1977). Parasuraman (1984a, 1984b) also argued that for a decrement in sensitivity to occur, the task must be of sufficient duration (i.e., 30–45 minutes). Alternatively, R. A. Cohen (1993c) pointed out that those CPTs with an increased emphasis on reaction time and test-generated changes in the ISI are more like to result in vigilance decrements and should approximate the traditional SDT task.

Commercially Available CPTs As can be seen by the number of variations to the CPT paradigm and variables reported, the potential number of CPTs is limited only by one’s imagination. Not all the CPTs reported in the literature, however, are commercially available. From the commercially available CPTs, four have been selected for inclusion in this chapter based on the differences in CPT parameters among the four tasks. The four tasks are the Conners CPT-II (Conners & MHS Staff, 2000), the Gordon Diagnostic System (GDS; Gordon, 1983), the Integrated Visual and Auditory CPT (IVA; Sandford & Turner, 1994–1999), and the Test of Variables of Attention (TOVA®; Greenberg, 1988–1999) and TOVA®—Auditory (TOVA-A; Greenberg, 1996–1999). Of these, only the GDS is not computerized. Some of these measures have been reviewed in more detail elsewhere (see Dumont, Tamborra, & Stone, 1995; Lowe, Reynolds, Riccio, & Moore, 1999; Riccio et al.,

reyn2-13.qxd

6/6/2003

9:19 AM

Page 299

13. Continuous Performance Tests

2001). Although some of these programs provide an option for customization, description of the standard paradigm and scores generated is the focus here. Conners Continuous Performance Test—II The standard version of the CPT-II, like the earlier version (Conners, 1992, 1995) is a 14-minute visual CPT. A not-X CPT, the individual is required to respond when a letter appears on the computer screen, except for when the letter “X” (i.e., the target) appears; when the letter “X” appears, the individual is to inhibit the response. This format is intended to ensure a greater number of responses and therefore decreased chance error (Conners, 1992, 1995; Conners & MHS Staff, 2000). The targeted and nontargeted stimuli are randomly shown for 250 milliseconds. The CPT-II standard paradigm consists of six blocks, with each block divided into three subblocks. For the three subblocks within a block, the ISI may be 1,000, 2,000, or 4,000 milliseconds; the order of the three different ISI subblocks varies from block to block. There is no distractor condition on the CPT-II, and the stimuli are not blurred or degraded in any way. The CPT-II can be used for individuals from age 4 through adulthood. A short practice test is available to help the client become familiar with the paradigm (Conners & MHS Staff, 2000). Instructions are presented to the examinee on screen. Directions for task administration include an additional prompt to respond both quickly and accurately and the provision of a single prompt to redirect the examinee if needed. The manual clearly states that the examiner is to remain present during task administration. The CPT-II manual (Conners & MHS Staff, 2000) provides a description of the normative data (N = 1920) for the standard paradigm. The normative or general data are described as coming from 30 sites across the United States and including three provinces of Canada. A breakdown by gender for the general sample indicates that males under age 18 comprised 52.5% of the sample. Ethnicity of the general sample by age level is not provided; for the complete sample of adults and children, 47% of

299

the sample were white, 27% were black, and 21.4% were other. Additional information specific to the size of the population of the cities or type of setting (i.e., rural, suburban, and urban) where the data were collected, educational level, and socioeconomic status were not provided. The limited information on the sample by age level hinders the extent to which results can be generalized to populations of differing educational levels, socioeconomic status (SES), or ethnicity. In addition to the general sample, clinical data were collected for 271 children with ADHD ranging in age from 6 to 17 years. For this sample, 75.3% of the sample was male. Ethnicity, SES, or educational level are not provided for this clinical sample. A total of 13 indices are generated from the CPT-II for use in interpretation; due to the skewed distributions, computation of all variables use logarithmic transformations. Variables include the more commonly reported correct hits, omission errors, commission errors, and reaction time. The CPT-II also provides the variables from SDT of sensitivity (d’) and response bias (Beta). Several measures relate specifically to the consistency or variability of the individual’s performance across blocks and subblocks (e.g., standard error of reaction time by block, by ISI, and for the entire test). Finally, the CPT-II provides an overall index for the individual’s performance that is a weighted score derived from all the other indices as well as a Confidence Index that indicates the likelihood of attention problems. The default calculation of the Confidence Index (compared to the clinical sample) is designed to optimize the overall hit rate and assumes a base rate of 50% for clinical status (Conners & MHS Staff, 2000). The option is available, however, to select a classification criteria that minimizes either false positives or false negatives. The CPT-II indices are presented in raw score, percentile, and T-score formats. All variables are presented such that high scores indicate a problem in order to eliminate possible confusion. Data are clustered into eight age groups for both samples, with 2year age intervals for children and youth. For the normative data, subjects were not excluded based on prescreening, and at some sites, no prescreening was conducted.

reyn2-13.qxd

6/6/2003

9:19 AM

300

Page 300

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Thus, some cases in the general sample may have been clinically significant; this is believed to provide more accurate representation of the general population. Because of the potential inclusion of some clinical cases in the general sample, it is recommended that a T-score of 60 (or a percentile of 90) be used as the cutoff for determining impaired performances (Conners & MHS Staff, 2000). Gordon Diagnostic System Of the commercially available CPTs, the GDS (Gordon, 1983) has been available for the longest amount of time and is probably the most frequently used CPT in research studies. The GDS is a microprocessor unit, as opposed to a computer software program, that generates 11 separate tasks. Of these tasks, there are three basic paradigms—the delay task, distractibility task, and vigilance task. Both the vigilance and distractibility tasks are CPTs and there is more than one version of the distractibility and vigilance tasks available (Gordon, 1986a, 1986b; Gordon & Mettelman, 1988; Gordon Systems, 1991). The vigilance task is believed to tap into cognitive skills, behavioral regulation, and motivation (Gordon Systems, 1986). The prototype for the vigilance task is the AXCPT in a visual format with numbers as stimuli (Gordon & Mettelman, 1988; Gordon Systems, 1991). The children’s standard version of the vigilance task lasts 9 minutes and requires the child to press a button every time a two-number target combination (a “1” followed by a “9” or in the alternate version a “3” followed by a “5”) is presented. For both versions, the numerals are displayed for 200 milliseconds, with a 1,000-millisecond ISI (Gordon, 1986b; Gordon & Mettelman, 1988; Gordon Systems, 1991). Both of these versions are for use with children ages 6–16 years. Another version of the vigilance task is for use with preschoolers (i.e., 4- and 5-year-old children) and is shorter (6 minutes). In the preschool version, the vigilance task is an X-CPT. As with the children’s version, the stimuli are displayed for 200 milliseconds; the ISI for the preschool version is increased to 2,000 milliseconds (Gordon Systems, 1991). Regardless of the version (i.e., chil-

dren’s standard and alternate versions and preschool version), the vigilance task is divided into blocks, so that a child’s performance may be monitored every 3 minutes, thus providing insight into the vigilance decrement over time (Gordon & Mettelman, 1988; Gordon Systems, 1991). The second CPT of the GDS is the distractibility task. The distractibility task incorporates the AX-CPT of the vigilance task. At the same time that the stimuli are presented, however, digits are displayed simultaneously on either side of the target stimulus (i.e., distractors). As such, the distractibility task assesses the extent to which the individual can selectively attend to the target stimuli (Gordon Systems, 1991). Stimulus duration, task duration, target frequency, and ISI are the same on the children’s distractibility task as for the children’s vigilance task. There is no preschool version of the distractibility task. In addition to the two CPT-type tasks (vigilance and distractibility), the GDS includes one other task—the delay task. The delay task is administered prior to the vigilance and distractibility tasks. The delay task serves as a “warmup” for the vigilance and distractibility tasks. The delay task is not a CPT paradigm but is designed as a measure of impulse control. On this task the child can earn points by inhibiting a response. The delay task takes less than 9 minutes. There is no separate preschool version for the delay task. The manual for the GDS includes instructions for administration, tables (Gordon Systems, 1991), an interpretive guide (Gordon Systems, 1986), and a technical guide (Gordon Systems, 1987). The order in which the GDS tasks are to be administered and the standard instructional set that is to be used (Gordon & Mettelman, 1988) are provided. The GDS provides practice trials for the vigilance and distractibility tasks that are slower versions of the tasks. During administration of the GDS, directions clearly state that the examiner will remain present (Gordon Systems, 1991). Normative data for the preschool and children’s versions of the GDS can be found in the work of Gordon and Mettelman (1988) as well as in the manual (Gordon Systems, 1991) and is limited in scope. The normative sample for the GDS vigilance

reyn2-13.qxd

6/6/2003

9:19 AM

Page 301

13. Continuous Performance Tests

task (standard children’s version) consists of 1,019 nonreferred children, ranging in age from 6 to 16 years of age; when the preschool sample of 4- and 5-year-olds is included, it increases the normative sample to 1,266 children; children with psychological, learning, neurological, attention, and impulse control problems were excluded from the normative sample (Gordon & Mettelman, 1988). Unfortunately, the geographic region of the normative sample is restricted to Syracuse, New York (91% of the sample) and Charlottesville, Virginia (9% of the sample) areas (Gordon Systems, 1987). Information on the ethnic composition of the sample was not provided. For the vigilance and distractibility tasks, variables of performance include correct hits, omission and commission errors, and reaction time (latency) for each block as well as the average latency for the entire task. The slope of the reaction time across blocks (block variability) is used to assess consistency of performance (Gordon Systems, 1991). The examiner completes the test form, records the results from the microprocessor, and computes the block variances themselves. Tables include raw score, percentile, and thresholds. The threshold tables were derived based on the percentile information with scores divided into three categories: abnormal (5th percentile or lower), borderline (6th to 25th percentile), and normal (26th percentile and above). At the preschool level, the norms are provided in 1-year age intervals for 4- and 5-year-olds. For the children’s version, norms are in 2year age intervals for 6- to 11-year-olds, and in a single 5-year age interval for 12–16year-olds. As gender effects were reported to account for only 2% of the variance, separate gender-based norms were not deemed necessary (Gordon & Mettelman, 1988; Gordon Systems, 1987). Integrated (or Intermediate) Visual and Auditory Continuous Performance Test The IVA is a 13-minute X-CPT, and unlike other CPTs discussed here, the IVA incorporates both auditory and visual modalities within the same task. As such, the IVA requires the individual to shift attentional modalities within the same task. On the IVA, the child is required to respond to a vi-

301

sual or auditory target stimulus (i.e., the number “1”) and to refrain from responding when the nontarget stimulus (i.e., the number “2”) is presented in either a visual or auditory format. The order of auditory or visual presentation of stimuli is fixed and the target and nontarget stimuli are presented in a pseudo-random pattern with a 1,500 milliseconds ISI (Sandford & Turner, 1995). Because there is only a single nontarget stimulus, the discrimination task is simpler than that involved in the CPT-II or GDS. In addition to including auditory presentation, the IVA provides differing target frequency conditions. The target frequency changes by blocks in order to elicit omission and commission errors such that for alternating blocks of the task, there is a high frequency of targets; for the other half of the task, there is a low frequency of targets. The target to nontarget ratios as well as the visual and auditory stimuli changes occur in a counterbalanced design in order to reduce fatigue and to control for learning effects (Sandford & Turner, 1995). The normative sample for the IVA consists of 781 individuals (358 males and 423 females) ranging in age from 5 years through adulthood. The sample was comprised of individuals with no known learning, psychological, neurological, attention, or impulse control problems (Sandford & Turner, 1995; Turner & Sandford, 1995a, 1995b). Normative data can only be accessed through a “read only” file. Normative data are divided into 2–10-year intervals depending on the age. Information on the geographic region(s) where the normative data were collected, education level, SES, and ethnicity of the sample are not provided in the manual. In describing administration of the IVA when normative data were collected, the manual notes that the same type of mouse, 14-inch screen, and headphones were used for all subjects (Sandford & Turner, 1995). The manual includes specific directions with regard to examinee distance from the screen, center of the screen relative to examinee eye level, and so on. The extent to which variation from any of these conditions would affect resulting performance is not indicated. Directions are provided by following computer prompts and directions by the examiner. There is no specific state-

reyn2-13.qxd

302

6/6/2003

9:19 AM

Page 302

IV. SPECIFIC SYNDROMES AND SYMPTOMS

ment relating to examiner presence or absence during the test but the provision of additional prompts during the task would suggest that the examiner remains in the room during the test administration. A warm-up session is provided followed by a practice test to allow the child to become familiar with the operation of the mouse button. Following the practice test, the regular test is administered. A cool-down session follows the regular test. Reaction times are recorded during both the warm-up and cool-down sessions (Sandford et al., 1995a, 1995b; Sandford & Turner, 1995; Seckler, Burns, Montgomery, & Sandford, 1995; Turner & Sandford, 1995a, 1995b). The IVA has 11 raw score scales, 6 quotient scales, and 2 composite scores which are provided separately for visual and auditory portions presentation (Sandford & Turner, 1995; Seckler et al., 1995; Turner & Sandford, 1995a, 1995b). The two composite scores (Response Control Quotient, Attention Quotient) are combined to yield a Full Scale score as well. Each of the two composite scores is derived from three of the six quotient scores. The Response Control Quotient (RCQ) incorporates the Prudence (avoidance of commission errors), Consistency (minimal variability in response time), and Stamina (response time maintained across testing) quotients. In contrast, the Attention Quotient is derived from the Vigilance (avoidance of omission errors), Focus (number of response time outliers across testing), and Speed quotients (Sandford et al., 1995a, 1995b; Seckler et al., 1995; Turner & Sandford, 1995a, 1995b). Fine motor/hyperactivity is an additional scale and incorporates anticipatory responses or continued pressure on the mouse (i.e., failure to release). Additional variables include Comprehension (a measure of random errors), Persistence (decrement in performance related to motivation or arousal), and Sensory/Motor (slow reaction time that may impair performance); these are considered validity measures for the IVA. Finally, performance differences based on target frequency are reflected in the Readiness score, whereas differences in performance based on modality are reflected in the Balance score. Results of the IVA are presented in raw scores as well as in standard scores with a mean of 100 (SD = 15) with numeric as

well as graphic formats (Sandford & Turner, 1995). Test of Variables of Attention The TOVA is an X-CPT that is available in separate visual (TOVA) and auditory (TOVA-A; Greenberg, 1996–1999) versions (Cenedela, 1996; Greenberg & Kindschi, 1996; Greenberg, Kindschi, & Corman, 1999; Leark et al., 1996). The TOVA includes both a clinical version and a briefer screening, or preschool version. Both the TOVA and TOVA-A require an individual to respond whenever the target stimulus is presented. For the TOVA, the target stimulus is a colored square with a smaller square contained within and adjacent to the top edge of the larger square while the nontarget stimulus has the smaller inscribed square adjacent to the bottom edge of the larger square. Thus, the stimulus feature of importance is that of position or placement. For the TOVA-A, two audible tones are used as stimuli, one as the target, and one as the nontarget. As with the IVA, because there are only two stimuli (one target, one nontarget), the difficulty level for discrimination required is minimal. The stimuli on both the visual and auditory tasks differ from other CPTs in that the stimuli are non-language-based; this is considered important in that otherwise language-based learning problems could confound interpretation of CPT performance (Greenberg & Waldman, 1993). The clinical versions of the TOVA and TOVA-A are approximately 22 minutes in duration. As with other CPTs, the preschool version (for 4- and 5-year-olds) is shorter (11 minutes) in duration. The clinical versions are composed of four intervals whereas the preschool version has two intervals. For both the TOVA and TOVA-A, the stimuli are presented for 100 milliseconds with an ISI of 2,000 milliseconds. As with the IVA, target frequency is manipulated on the TOVA and TOVA-A. For the first half of the test, the target stimulus is randomly displayed on 22.5% of the trials (stimulus infrequent condition). In the second half of the test, the target stimulus is shown on 77.5% of the trials (Greenberg & Waldman, 1993; Leark et al., 1996). The three manuals (Cenedela, 1996; Greenberg et al.,

reyn2-13.qxd

6/6/2003

9:19 AM

Page 303

13. Continuous Performance Tests

1999; Leark et al., 1996) include standardized instructions to be given to the examinee (Leark et al., 1996). The examiner remains in the room during the administration of the task. A 3-minute practice test is administered prior to the beginning of the regular test (Leark et al., 1996). The normative sample for the TOVA is composed of two samples, the original sample and a second sample that was later added to the original sample. Taken together, the TOVA combined normative sample consists of 1,590 individuals, 712 males and 884 females, ranging in age from 4 years through adulthood. Separate norms are available by gender; however, norms stratified on the basis of SES and ethnicity are not provided. Greenberg and Crosby (1992) recommended the use of caution in interpreting the performance of individuals who are members of minority groups or who are from varying levels of SES. The current normative sample for the TOVA-A is limited to 2,551 children ages 6 to 19 years recruited from public schools in Minneapolis, Minnesota; the sample is described as predominantly (99%) Caucasian. As with the TOVA, SES and educational level are not reported. The norms for both the TOVA and TOVA-A are divided into 1-year intervals through age 19 (Greenberg et al., 1999; Leark et al., 1996). The TOVA and TOVA-A provide multiple variables of performance including omission and commission errors, response time, the standard deviation of the response time, and sensitivity (d-prime). It is important to note that “omission errors” on the TOVA is a measure of relative accuracy derived from the number of correct responses to the difference between the number of targets possible minus the number of anticipatory errors (Greenberg et al., 1999; Leark et al., 1996). The commission error score is based on the number of responses to nontargets relative to the total number of nontargets minus the number of anticipatory responses. For these computations, anticipatory errors are defined as those responses made within 200 milliseconds of the appearance of the stimulus regardless of whether the stimulus is a target or nontarget (Greenberg et al., 1999; Leark et al., 1996); the number of anticipatory errors are reported as well. The program also provides

303

the number of multiple responses and postcommission error response time (Cenedela, 1996; Greenberg et al., 1999; Greenberg & Waldman, 1993; Leark et al., 1996). Finally, the ADHD scale provides a comparison of the examinee’s scores to persons of the same age and gender in the database who had a diagnosis of ADHD (Cenedela, 1996). Results are reported as raw scores, percentages, standard scores (mean of 100, SD = 15), and standard deviations. Group statistics (means and standard deviations) are provided by age for both the TOVA and TOVA-A with separate norms provided for each age group by gender (Greenberg et al., 1999; Leark et al., 1996). An additional feature of both the TOVA and TOVA-A software is that the results can be saved and subsequent administrations compared for a given individual over to facilitate monitoring of treatment (Cenedela, 1996).

THE CPT AS A DIAGNOSTIC TOOL The increased use and the plethora of CPTs available suggest that the basic CPT paradigm has been accepted by clinicians and researchers as a measure of attention and executive control. Not only is some version of the CPT the most frequently used laboratory measure of attention (DuPaul et al., 1992), but the CPT has been described as the “gold standard” for measuring sustained attention (Fleming, Goldberg, & Gold, 1994, p. 205). In recent years, many third-party payers (e.g., Blue Cross/Blue Shield of Texas) seem to think the CPT is a form of “gold standard” or, at the very least, a powerful tool for use in diagnosis of psychopathology, and ADHD in particular. The extent to which the use of the CPT in diagnosis is appropriate will be explored next.

Sensitivity and Specificity Over the past 40 years, considerable research has been conducted using CPTs with children diagnosed with ADHD (e.g., August & Garfinkel, 1989; Barkley, Grodzinsky, & DuPaul, 1992; Chee et al., 1989; Halperin, Matier, Bedi, Sharma, & Newcorn, 1992), learning disabilities (e.g., Beale et al., 1987), traumatic brain injury (e.g., Kaufmann,

reyn2-13.qxd

304

6/6/2003

9:19 AM

Page 304

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Fletcher, Levin, & Miner, 1993), metabolic disorders (e.g., Anderson, Siegel, Fisch, & Wirt, 1969), autism (Garretson, Fein, & Waterhouse, 1990), low birth weight (Katz et al., 1996), conduct disorder (Chee et al., 1989), and various other disorders. When considering the usefulness of a measure or a family of measures in the diagnostic process, it is important to consider the sensitivity and specificity of the measure. Sensitivity refers to the ability of a diagnostic tool or procedure to detect a disorder when the disorder is present. Sensitivity can be expressed mathematically as the likelihood that a given method (e.g., the CPT) will detect a disorder when the individual engaging in the task in fact has a disorder (i.e., a true positive). For example, if the question is specific to ADHD, individuals with ADHD may have high rates of commission errors as compared to individuals who do not have ADHD. If sensitivity levels are too high, overdiagnosis (a high rate of false positives) is likely to occur; if sensitivity levels are too low, underdiagnosis (a high rate of false negatives) is likely to occur. The complement to sensitivity, specificity is the ability of a procedure to detect the absence of a specific disorder when it is not present. Mathematically, it is the likelihood that a given method will indicate accurately that a disorder is not present (i.e., a true negative). Specificity depends on the capability of the procedure (e.g., a CPT) to differentiate among disorders with overlapping symptoms. Many tests and procedures demonstrate high sensitivity but lack in specificity. This is a concern because in clinical practice the question is not usually as simple as “does this person have a disorder or is this person normal?” but to identify which of many disorders an individual may have. Unfortunately, most of the research on the diagnostic efficacy of CPTs has been conducted comparing a clinical group (e.g., children with ADHD) to a control, nonpsychiatric group. These studies may not provide information on specificity, but they do allow appraisal of the CPT as providing a criterion for the diagnosis of various disorders. ADHD and CPT Performance Given that the symptoms associated with ADHD in childhood include inattention,

hyperactivity, and impulse control problems, the notion of the CPT as a diagnostic tool for ADHD appears plausible on the surface. Of the studies reporting on CPT performance of children with ADHD, or attention deficit disorder (ADD), the majority compare children with ADHD to presumably normal children or nonclinical groups. In most of these studies, results fairly consistently yield significant between group differences in CPT performance on all or nearly all variables. Notably, on the CPT-II, the ADHD group was not found to make more commission errors than a nonclinical group (Conners & MHS Staff, 2000). When demographic variables (age, SES) and verbal ability were controlled for, however, the likelihood of finding differences between groups decreased somewhat (e.g., Koriath, Gualtieri, van Bourgondien, Quade, & Werry, 1985; Werry, Elkind, & Reeves, 1987). Few of the studies included classification rates (accuracy in identifying children as normal or ADHD based on CPT performance) or sensitivity and specificity coefficients. Importantly, when classification rates are reported, the samples are relatively small and this may result in inflated estimates of diagnostic accuracy (Willson & Reynolds, 1982). For example, using the Conners CPT (Conners, 1992, 1995), children with ADHD were compared to the general population (normative sample); results indicated a 13.0% false-negative rate and a 12.9% false-positive rate (Conners, 1992, 1995). Using the GDS with children with ADHD, children with other disorders, and a control group, results indicated approximately 70% agreement with parent and teacher ratings of children with ADHD depending on the age, the rater, and the scale (Gordon Diagnostic Systems, 1987). Although 70% agreement is better than chance, it should be noted that in 30% of the cases, results of the GDS were not consistent with diagnosis based on behavior ratings. Additional studies indicate that for children diagnosed with ADHD in comparison to a normal control group, the IVA was in agreement with group membership in 92% of the cases (Sandford & Turner, 1995). With the TOVA, depending on the cutoff used for predicted group membership, and children with either ADHD or no diagnosis, sensitivity quotients ranged from

reyn2-13.qxd

6/6/2003

9:19 AM

Page 305

13. Continuous Performance Tests

.60 to .76 (Leark et al., 1996). Using the Conners CPT, 70–75% classification accuracy has been reported for ADHD and other clinical groups (Czerny, O’Laughlin, & Griffioen, 1999). Specificity and sensitivity for the Conners CPT was reported to be 83% and 82%, respectively, for children and youth with ADHD as compared to a nonclinical sample (Conners & MHS Staff, 2000). Across studies, CPTs identified normal (non-ADHD) children with greater accuracy than they identified children with ADHD (Anastopoulos & Costabile, 1994; Barkley & Grodzinsky, 1994; Harper & Ottinger, 1992). Notably, few studies compared the performance of children with ADHD by subtypes using either the subtype system of the third edition of Diagnostic and Statistical Manual for Mental Disorders (DSM-III; American Psychiatric Association, 1980) or the DSM-IV (American Psychiatric Association, 1994). Results of these studies (e.g., Barkley, DuPaul, & McMurray, 1990; Barkley & Grodzinsky, 1994; Forbes, 1998; García-Sánchez, Estérez-González, SuárezRomero, & Junqué, 1997; Holcomb, Ackerman, & Dykman, 1985; Johnson, 1993) are equivocal with regard to the ability of CPTs to aid in subtype differentiation. Further, the majority of studies included children with a mean age between 6 and 12 years. Fischer, Newby, and Gordon (1995) compared the performance of younger and older children with ADHD relative to a normal control group. Results indicated that although there was a correct classification rate of 81% for the younger age group, this rate decreased to 20% for children ages 12–17 years. These findings have not been replicated; however, they suggest that the sensitivity of the CPT to ADHD may decline with increased age. The differentiation of ADHD from other disorders is complicated by the high level of co-occurrence of ADHD with other disorders such as learning disabilities and conduct disorder (e.g., Hynd et al., 1995; Riccio & Jemison, 1998). In addition to studies comparing children with ADHD to a nonclinical control group, a minority of studies have compared the performance of children with ADHD to other clinical groups including children with learning disabilities (e.g., Barkley et al., 1990, 1992; Barkley &

305

Grodzinsky, 1994; Chee et al., 1989; Riccio, Cohen, Hynd, & Keith, 1996; Richards, Samuels, Turnure, & Ysseldyke, 1990; Schachar et al., 1988; Tarnowski, Prinz, & Nay, 1986), conduct disorder (Chee et al., 1989; Halperin et al., 1990, 1993, 1995; Koriath et al., 1985), Tourette syndrome (Harris et al., 1995), schizophrenia (Erickson, Yellin, Hopwood, Realmuto, & Greenberg, 1984), and hypoxia (O’Dougherty et al., 1984). Thus far, results of studies comparing the CPT performance of children with ADHD to those with learning disabilities, for example, are highly inconsistent. This is not surprising in that many studies comparing children with learning disabilities to normal controls suggest that impaired CPT performance is associated with learning disabilities as well as ADHD with no synergistic effects of comorbid ADHD and learning disability (Chee et al., 1989; Robins, 1992). In that children with conduct disorder generally perform comparably to normal control groups (e.g., Koriath et al., 1985), CPTs are somewhat better in differentiating ADHD and conduct disorder; however, inattention and aggression could not be dissociated based on CPT performance (Halperin et al., 1990). Moreover, in other studies, children with conduct disorder were impaired on the CPT (e.g., Schachar et al., 1988). In comparison to other groups (e.g., Tourette syndrome and schizophrenia), children with disorders other than ADHD sometimes performed worse than or equally as poorly as children with ADHD. Children with Tourette syndrome, for example, demonstrate significantly slowed reaction times (e.g., Harris et al., 1995; Shucard, Benedict, Tekok-Kilic, & Lichter, 1997); this finding has potential heuristic value and should be studied further. Thus, although children with ADHD can be distinguished from some samples with reasonable consistency based on CPT performance, it is more difficult, if not impossible, to differentiate them from other diagnostic groups on a reliable basis. Other Disorders and CPT Performance Studies comparing the CPT performance of children with numerous disorders other than ADHD relative to normal controls or to children with other disorders are less numerous,

reyn2-13.qxd

6/6/2003

9:19 AM

Page 306

306

IV. SPECIFIC SYNDROMES AND SYMPTOMS

but add to what is known about the sensitivity and specificity of the CPTs. For example, in three out of four available studies on children with a diagnosis of schizophrenia, schizophrenia was found to be associated with impaired CPT performance (Erickson et al., 1984; Rund, Zeiner, Sundet, Oie, & Bryhn, 1998; Strandburg et al., 1990, 1994). With less consistency, children with a parent diagnosed with schizophrenia also have been found to demonstrate impaired CPT performance (Asarnow, Steffy, MacCrimmon, & Cleghorn, 1977; Erlenmeyer-Kimling & Cornblatt, 1978; Nuechterlein, 1983). Generally speaking, although a diagnosis of ADHD tends to be associated with increased commission errors, a diagnosis or family history of schizophrenia tends to be associated with increased omission errors. Although promising, these differences are not sufficiently consistent for diagnostic certainty (Riccio et al., 2001). Among clinical groups whose performance on the CPT have been studied, re-

search points to significant performance decrements for children with traumatic brain injury (e.g., Crosby, 1972; Katz et al., 1996, Kaufmann et al., 1993) as well as for children with seizure disorders (e.g., Levav, 1991; Miller, 1996). Children with neurofibromatosis also demonstrate deficits in CPT performance, predominantly in the form of slowed reaction time (Eliason, 1988) as with Tourette syndrome. In fact, significant evidence exists that any direct central nervous system (CNS) compromise will result in impaired performance on the CPT (Riccio et al., in press). Children with metabolic disorders such as phenylketonuria (PKU) have been found to perform poorly on CPTs and to evidence a vigilance decrement over time (Anderson et al., 1969; Brunner & Berry, 1987). Of the populations included in the extant literature, CPT performance generally was affected by most externalizing disorders, autism, and schizophrenia as well as most types of CNS dysfunction (see Table 13.2). Negative effects on CPT performance

TABLE 13.2. Summary of Disorders Associated with Impaired Performance on Continuous Performance Tests Disorder

Relevant Studies

ADHD (all subtypes, with or without a comorbid disorder)

Anastopoulos & Costabile (1994); August & Garfinkel (1989); Barkley & Grodzinsky (1994); Barkley et al. (1992); Chee et al. (1989); Conners (1992, 1995); Fischer et al. (1995); Forbes (1998); García-Sánchez et al. (1997); Halperin et al. (1992); Holcomb et al. (1985); Johnson (1993); Koriath et al. (1985); Robins (1992); Sykes et al. (1971); Teicher et al. (1996); Trommer et al. (1988); Werry et al. (1987) Garretson et al. (1990) Riccio et al. (1996) Mitchell & Quittner (1996) O’Dougherty et al. (1984) Hickey, Suess, Newlin, & Spurgeon (1995); Walker (1993) Katz et al. (1996) Crosby (1972); Kintslinger (1987) Eliason (1988) Anderson et al. (1969); Brunner & Berry (1987) Erickson et al. (1984); Rund et al. (1998); Strandburg et al. (1990, 1994) Levav (1991); Miller (1996) Harris et al. (1993); Shucard et al. (1997) Crosby (1972); Katz et al. (1996); Kaufmann et al. (1993) Romans et al. (1997)

Autism Central auditory processing disorder Hearing impaired/deaf Hypoxic/anoxic injuries Intrauterine toxic exposure Low birth weight Mental retardation Neurofibromatosis Phenylketonuria Schizophrenia Seizure disorder Tourette syndrome Traumatic brain injury Turner’s syndrome

reyn2-13.qxd

6/6/2003

9:19 AM

Page 307

13. Continuous Performance Tests

were associated with metabolic disorders and some medical conditions as well. Notably, CPTs were not found to be particularly sensitive to disorders of mood or affect, except during manic episodes or in the presence of psychotic features (Riccio et al., 2001). Equivocal findings exist for samples of children with learning disability, oppositional defiant disorder, and conduct disorder. Although further study is needed in this area, the obvious conclusion from the studies reviewed is that any disorder that results in a compromise of CNS integrity or function is likely to produce decrements in CPT performance. In fact, for this age range (5–17 years), the CPT appears quite sensitive to CNS compromise of varying etiology. If the diagnostic question is whether there is CNS compromise or not, then the CPT seems to do quite well. In practice, however, the question is more often which of a myriad of disorders or syndromes the child has. Across the literature available to date, when studies have included children with a variety of disorders, classification rates dropped precipitously. For example, Kintslinger (1987) found that CPT results resulted in 47.5% classification accuracy for children previously identified as either mentally retarded, learning disabled, behaviorally disordered, or normal. Notably, the best classification accuracy was for the normal children (19 of 20). Other studies with multiple clinical groups report similar findings (e.g., Halperin et al., 1992). In another study, CPT variables misclassified 28% of the non-ADHD group as ADHD and 20% of the ADHD sample as not ADHD (Forbes, 1998) As noted previously, multiple disorders include similar symptoms, and it would not be surprising to find that CPTs are sensitive to many such disorders. What may be apparent from this review, however, is that virtually any disorder of childhood that disrupts or compromises the integrity of the CNS is likely to result in impaired performance on the CPT (see Table 13.2). In fact, among disorders usually first identified in childhood and adolescence, the CPT is quite sensitive to problems of many varieties. Because of this sensitivity, and as noted by others (Forbes, 1998; Halperin et al., 1992; Riccio et al., 2001), the use of CPT perfor-

307

mance decrements as indicative of ADHD in particular would likely result in children with a variety of psychiatric (and medical) conditions being misdiagnosed if the comparison group is a normal control or nonclinical group. In reality, the more common task facing the clinician is to determine which of a number of similar disorders is the appropriate diagnosis for a client. The complexity of syndromes precludes the definition of any disorder based on one or two dimensions or a single measure without other tests and clinical and historical information. Although CPTs lack sufficient sensitivity and specificity for differential diagnosis, they demonstrate sensitivity to disorders that include behavioral symptoms associated with inattention and poor self-regulation (Riccio et al., 2001). CPT performance decrements are not disorder specific, but, rather, CPTs are sensitive to the presence of impulsivity and to the attentional deficits associated with brain dysfunction. As such, although lacking sufficient sensitivity and specificity for differential diagnosis, CPTs possess high degrees of usefulness for objective documentation of symptoms (including treatment monitoring) associated with multiple disorders as well as in ruling out disorders that involve brain dysfunction.

Validity and Reliability Issues Construct Validity If CPTs are to be interpreted as measures of attention or executive function, the results generated by the CPT would be expected to correlate significantly with results from other accepted measures; this is one method for establishing the construct validity of a measure’s suggested interpretation (Anastasi, 1988; Cohen & Swerdlik, 1999). At the same time, due to the multifaceted nature of attention, a one-to-one correspondence between CPT performance and other measures or methods would not be expected. Barkley (1998) asserted that the optimal method for assessing attention and self-regulation continues to be direct observations. Although direct observation may be the most ecologically sound means of assessing attention and executive control, observations in natural settings can be time-consuming. As a result, in practice, laboratory

reyn2-13.qxd

308

6/6/2003

9:19 AM

Page 308

IV. SPECIFIC SYNDROMES AND SYMPTOMS

measures of attention and executive control are used frequently. Another alternative to direct observation involves the use of behavior rating scales. If the CPT paradigm is measuring attention and executive control, it would be expected that CPT performance would be significantly correlated with results of direct observation, laboratory measures, and behavior scales. The majority of studies investigating the relation between CPT scores and results of direct behavioral observations yielded moderate to high correlations (Barkley, 1991; Garretson et al., 1990; Gordon, DiNiro, Mettelman, & Tallmadge, 1989; Harper & Ottinger, 1992; Kupietz & Richardson, 1978). Notably, higher correlations emerged in those studies with clinical samples of children regardless of diagnostic category (e.g., Barkley, 1991; Garretson et al., 1990; Harper & Ottinger, 1992; Kupietz & Richardson, 1978). The research further suggests that CPT performance is correlated with scores from other laboratory measures assessing shifts in attention (e.g., Allen, 1993; Suslow & Arolt, 1997), focused attention (e.g., Burg, Burright, & Donovick, 1995; Das, Snyder, & Mishra, 1992), selective attention (e.g., Das et al., 1992), motor activity (e.g., Allen, 1993; Trommer, Hoeppner, Lorber, & Armstrong, 1988), and impulsive behavior (Kardell, 1994; Slicker, 1991). The degree of association between CPT variables and a number of other measures varies depending on the CPT parameters and population. For example, omission errors on a visual X-CPT with typically developing children correlated moderately (r = .52) with the Stroop (Das et al., 1992). With a different visual X-CPT and a combined clinical and control group, however, the correlation between omission errors and the Stroop was negligible (r = .05). Across studies, however, the relative consistency of moderate correlations of CPT variables to direct behavioral observations and other measures of attention would support the notion that the CPT paradigm is measuring some aspect of attention. A number of studies have examined the association between CPT performance and various behavior rating scales (e.g., Allen, 1993; Das et al., 1992; DuPaul et al., 1992; Garretson et al., 1990; Gordon et al., 1989; Halperin, Wolf, et al., 1991; Harper & Ot-

tinger, 1992; Lam & Beale, 1991; Sandford et al., 1995b; Slicker, 1991; Teicher, Ito, Glod, & Barber, 1996; Wherry et al., 1993). Across studies, the relation between CPT performance and behavior scales is variable depending on the subscale, the rater, and CPT parameters. The degree to which hyperactivity scales or subscales were associated with CPT performance tended to be stronger than attention (or inattention) or impulsivity. There was a moderate degree of association found between commission errors and teacher ratings of hyperactivity (Barkley, 1991; Kupietz & Richardson, 1978). When comparing modality of the CPT administration, the correlation between hyperactivity and commission errors was stronger for the visual CPT format as compared to the auditory format (Kupietz & Richardson, 1978). Studies that included correlations of subscales of inattention or distractibility with CPT scores obtained correlation coefficients generally in the ± 0.45 range (Lowe et al., 1999). In contrast, correlation of CPT results to scales of impulsivity or dyscontrol tended to be lower. Coefficients were higher when impulsivity was combined in the scale with inattention or hyperactivity but were not as high as when “pure” attention or hyperactivity scales were used (e.g., Slicker, 1991; Teicher et al., 1996). Using the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992), Floyd’s (1999) results indicated substantial correlations of the Hyperactivity and Attention Problems subscales of the Parent Rating Scale (PRS) with omission errors. Attention Problems correlated moderately with commission errors as well. For the BASC Teacher Rating Scale (TRS), correlation coefficients were moderate for Hyperactivity and Attention Problems with commission errors. Consistent with the findings of sensitivity to a myriad of disorders, commission and omission error scores also correlated moderately with other scales, including aggression and depression (Floyd, 1999). At the same time, a number of studies have examined the relationship between cognitive ability and other functional domains with CPT performance (e.g., Aylward, Gordon, & Verhulst, 1997; Chae, 1999; Hoerig, D’Amato, Raggio, & Martin,

reyn2-13.qxd

6/6/2003

9:20 AM

Page 309

13. Continuous Performance Tests

1998; Kardell, 1994). Overall, CPT performance has not been found to be correlated significantly with verbal IQ, performance IQ, or Full Scale IQ. Campbell, D’Amato, Raggio, and Stephens (1991) concluded that CPTs measure attention or strategies associated with achievement to some extent; however, the level of association between CPT results and achievement test results varies depending in part on the modality of the CPT. Regardless of modality, the correlations are lower than expected for prediction of achievement from CPT performance. Moderate to strong levels of association were found with some motor tasks (e.g., Allen, 1993), suggesting that any motor difficulties may need to be considered when interpreting CPT results. Thus, the relationship between CPT performance and direct observations, rating scales, and laboratory measures of attention support the ecological validity of the CPT paradigm; at the same time, the level of association of CPT performance with other measures suggests that CPTs have some unique characteristics not shared with other measures and, thus, may provide information not otherwise available from traditional tests. One of the reasons so much emphasis is placed on attention is the belief that attention is the precursor to memory (Cohen, 1993b; Sohlberg & Mateer, 1989). Because of the presumed association between memory and attention, the degree of association between CPT variables and measures of memory would seem important, but to this point, there is a paucity of research in this area. Temporal Stability Independent researchers have conducted studies on the temporal stability of the scores derived from different CPT paradigms (e.g., Finkelstein, Cannon, Gur, Gur, & Moberg, 1997; Halperin, Sharma, et al., 1991; Harper & Ottinger, 1992; Rosvold et al., 1956). Reliability coefficients vary considerably depending on the CPT parameters, the CPT variable, and the test–retest interval. Of the variables of interest, the highest temporal stability has been found for omission errors or correct hits. For example, with a test–retest interval of at least 1 week, Harper and Ottinger (1992) reported test–retest reliability estimates for omis-

309

sion errors of .55 for an X-CPT and .80 for an AX-CPT. Halperin, Sharma, and colleagues (1991) reported moderate test–retest reliabilities for an AX-CPT with 34 nonreferred boys (7–11 years old) and an interval between test and retest of approximately 4.8 months. The test–retest reliability estimates for correct hits, reaction times, omission errors, inattention scores, and impulsivity scores ranged from 0.65 to 0.74, suggesting moderate temporal reliability. Across studies, test–retest reliability estimates for commission errors have been found to be lower than those for omission errors or correct hits (Halperin, Sharma, et al., 1991; Harper & Ottinger, 1992). Study of the temporal stability of the GDS children’s standard version is provided in the technical manual (Gordon Systems, 1987) and in Gordon and Mettelman (1988). The test–retest reliabilities were reported for two groups of children (nonreferred and referred). Results indicated that the GDS scores for vigilance and distractibility tasks have moderate to high test– retest reliability over the short term (2–22 days) and the vigilance task score has moderate to high reliability long term (1 year). Comparable data for the preschool versions were not available. For the CPT-II, temporal reliability coefficients ranged from low (e.g., hit reaction time by block, r = .28) to high (e.g., omission errors, r = .84) (Conners & MHS Staff, 2000) A temporal reliability study with 70 volunteers (mean age of 21.8 years) and test–retest intervals ranging from 1 to 4 weeks resulted in quite inconsistent results depending on the variable (Seckler et al., 1995), suggesting that the temporal stability of some scores is adequate while the stability of other scores is below expected levels. The extent to which these estimates apply for children is unknown. The TOVA scores have been found to have moderate internal consistency with reliability coefficients varying depending on the variable of interest. Similar results were found for the internal consistency of the TOVA-A scores (Leark et al., 1996). In an independent study, Llorente and colleagues (2000) investigated the internal consistency of the TOVA scores with a group of children (n = 63) with ADHD. Each half of the test was compared to the other half as well as the whole test;

reyn2-13.qxd

310

6/6/2003

9:20 AM

Page 310

IV. SPECIFIC SYNDROMES AND SYMPTOMS

results suggested strong internal consistency. Moderate temporal stability of the TOVA scores is reported in the manual for 24 presumed normal children with a test–retest interval of 90 minutes, an interval useful for same day testing of medication effects (Leark et al., 1996). With a longer (4-month) test–retest interval, Llorente and colleagues found temporal reliability estimates for the TOVA scores to range from adequate to moderate depending on the variable of interest.

Potential Moderator or Mediator Variables Developmental Issues and the CPT Because attention and executive control are subserved by neurological structures and systems, some understanding of the developmental trajectory of the CNS is critical. Although the primary cortical areas generally mature by birth (Luria, 1980), secondary and tertiary areas continue to develop postnatally. These include the functional systems involved in learning, memory, emotion, cognition, and language, as well as those systems involved in attention and executive control. Not only is there continued development of these areas, but primary areas as well as the pathways that connect structures within the primary areas are likely to change over time as myelination and cell specialization continue to occur (Merola & Leiderman, 1985; Rutter, 1981; Vygotsky, 1980). Although the knowledge base regarding neurodevelopment has increased in recent years, the majority of this knowledge is grounded on observations and informal assessment of individuals with identified brain damage as opposed to typically developing children (Reynolds, 1997). Developmental differences have been found to be associated with CPT performance in a number of studies across the lifespan (e.g., Conners, 1995; Greenberg & Crosby, 1992; Halperin, Sharma, et al., 1991; Holcomb et al., 1985; Klorman et al., 1991; Marks, Himelstein, Newcorn, & Halperin, 1999; Romans, Roeltgen, Kushner, & Ross, 1997; Sandford & Turner, 1995; Turner & Sandford, 1995a, 1995b). Developmental differences in children’s performance on CPTs have been reported by a

number of researchers using normative samples (e.g., Gordon & Mettelman, 1988; Greenberg & Waldman, 1993; Turner & Sandford, 1995a) as well as with clinical samples (e.g., Raggio & Whitten, 1994). Based on the available research base, after age 5, there is a gradual improvement in performance as a function of age which stabilizes in young adulthood to be followed by a decline in performance in later adulthood (e.g., Greenberg & Waldman, 1993; Raggio & Whitten, 1994; Sandford & Turner, 1995). Because of the developmental impact on CPT performance, it is important to incorporate developmental theory into the interpretation process. To meet this end, it has been asserted that normative data need to be provided in relatively small intervals (4- to 6-month intervals) for children and increasing to 1-year intervals in adulthood (Lowe et al., 1999). Gender and CPT Performance Gender, as a moderating variable, has been investigated minimally and research findings do not suggest substantial differences based on gender; however, the results are equivocal (e.g., Driscoll, 1994; Goldstein et al., 1997; Greenberg & Waldman, 1993; Levy, 1980; Wagner, 1987). For example, Levy (1980) administered the X-CPT to 120 boys and 110 girls (3 to 7 years old) and found no significant gender differences between the boys and the girls for omission errors, commission errors, or mean reaction times scores. In contrast, Wagner (1987) examined gender differences in CPT performance for a sample of 83 boys and 34 girls on the X-CPT and AX-CPT. The sample consisted children with ADHD and a control group of presumably typically developing children. Consistent with the findings of other studies (Greenberg & Crosby, 1992; Sandford & Turner, 1995), Wagner’s results revealed that boys made more errors of commission than girls. In other studies (e.g., Sandford & Turner, 1995), boys were found to exhibit faster reaction times than girls as well. Based on the potential for age and gender differences to affect CPT performance, the use of appropriate gender and age normative groups has been suggested (Sandford & Turner, 1995).

reyn2-13.qxd

6/6/2003

9:20 AM

Page 311

13. Continuous Performance Tests

Socioeconomic Status and the CPT A number of studies have been conducted to assess the relation between CPT scores and SES in the child population; however, as with gender, the findings are equivocal (Evans, 1988; Gordon & Mettelman, 1988; Levy, 1980; Levy & Hobbes, 1988). Levy (1980) found that the child’s ability to complete the CPT was dependent on both the child’s age and parental SES with children from the upper class able to complete the CPT at a younger age than those in the lower class. Similarly, Gordon and Mettelman (1988) found a significant relation between mother’s SES and total correct scores on the vigilance task whereas father’s SES was significantly related to commission error scores on the distractibility task. Although the correlations between CPT scores and parental SES were significant, SES accounted for a small percentage of the variance. In contrast, Evans (1988) found a nonsignificant relation between CPT performance and SES for 164 children (6 to 14 years old) when age effects were partialled out. Culture and the CPT Bauermeister, Berrios, Jimenez, Acevedo, and Gordon (1990) compared a large Puerto Rican sample to the GDS normative data (Gordon Systems, 1987). The Bauermeister sample consisted of 433 children and adolescents, ranging in age from 6 to 16 years in Puerto Rico. For purposes of comparison, the Puerto Rican children were then matched to randomly selected children from the U.S. normative sample on the basis of age and gender. The performance of the children from the two on the delay and vigilance tasks were compared; the distractibility task was not included. In comparison to the Puerto Rican children, U.S. children had higher efficiency ratios and total correct scores and committed fewer commission errors. Further analyses by age groups revealed statistically significant cultural differences in the 6- to 9-year-old age groups, with the U.S. children making more correct responses and committing fewer commission errors in comparison to the Puerto Rican children. Thus, if U.S. norms were to be applied to the Puerto Rican sample, then a

311

larger proportion of the Puerto Rican children would be identified as having potential attention and impulse control problems (Bauermeister et al., 1990; Lowe et al., 1999).

DISCUSSION AND CONCLUSIONS The complexity of the constructs of attention and executive control suggests that no single measure of the behavior will provide information that is sufficient to address all facets. Concerns with attention and executive control occur with high frequency regardless of the type of practice or setting or age of the client. The very nature of attentional and executive control functional systems makes it a formidable task to assess adequately and accurately the integrity of these systems. Further, it has been suggested that when one component of attentional processes is disrupted, other components of attention or executive control are more likely to be affected due to the interconnectedness of the these components (Cohen, 1993a). Assessment of attention and executive control must be multifaceted, paralleling the complexity of the functional systems involved. The basic paradigm for the CPT was initially designed almost 50 years ago for the assessment of attention and, to a lesser degree, response inhibition or dyscontrol. The best any single measure, including the CPT, can provide are data on specific aspects of attention and executive control that, in conjunction with other measures, can be considered in diagnostic hypothesis generation and in the monitoring of the treatment and rehabilitation process. Since Rosvold and colleagues (1956) introduced the original CPT over 40 years ago, a plethora of CPTs have been developed. Today, there are innumerable variations of the CPT used in clinical and research settings. At the same time, differing CPTs and clinicians report a myriad of scores for use in interpretation. Thus, different CPTs may place different demands on an individual’s attention, executive, and memory systems. When the possible numbers of differences is considered, it is clear that CPTs are not a unitary measure (Lowe et al., 1999) but a family of measures (Con-

reyn2-13.qxd

312

6/6/2003

9:20 AM

Page 312

IV. SPECIFIC SYNDROMES AND SYMPTOMS

ners, 1992, 1995) with different parameters and scoring indices. The extent to which the differences in parameters and measures across these four CPTs affect diagnostic considerations is unknown as there is no comparative research of the various CPTs across populations available. In using the CPT, as with any formal assessment measure, it is important for administration to be standardized consistent with the collection of normative data. As such, it is imperative for the CPT manuals to clearly state the conditions for standardized administration. Differences related to use of space bar versus mouse, sensitivity and response times of various mouse devices, different screen sizes, use of headphones as opposed to speakers, use of laptop versus standard PC, and so on have not been investigated; there is no evidence that these differences do not have an effect on the results obtained. Ballard (1996) reviewed a number of experimenter manipulated variables that may influence CPT scores and recommended that manuals address these issues (e.g., examiner presence or absence or instructional set). The potential for these changes to be confounds to test performance, possibly invalidating the interpretation of the results, requires adherence to standardization procedures in the administration of any CPT. In addition, given differences in task demands and parameters across CPTs, performance on any one of these CPTs can be interpreted based only on the normative data for those specific task parameters. For these reasons, although the capacity to customize the task may be beneficial for research purposes, clinical use of customized versions of CPTs is not recommended due to the lack of normative data. Ethical and professional standards demand that psychologists use measures that are technically adequate. As with the developers and publishers of other assessment tools, CPT developers and publishers need to ensure that the measures meet the standards of technical adequacy established by the joint committees and published by the American Educational Research Association (AERA; 1999). In keeping with these standards, the adequacy of the normative data need to be evaluated relative to the population for whom the measure is intended in order to ensure generalization of the inter-

pretation (e.g., Anastasi, 1988; Lowe et al., 1999). As such, the normative sample must be representative, of sufficient size to produce stable values and reduce potential sources of error, stratified across a number of demographic variables, and described in detail (e.g., Anastasi, 1988; Cohen & Swerdlik, 1999). In the past, limited research has been conducted on the relation between demographic variables and neuropsychological test scores (e.g., Lowe & Reynolds, 1999; Lowe et al., 1999; Reynolds, 1997, in press). Only a small number of CPT studies have been published with individuals from different racial or cultural backgrounds (e.g., Bauermeister et al., 1990). The results of the Bauermeister and colleagues (1990) study suggest that there may be a cultural confound to CPT performance. The use of the normative data (United States) with children from diverse cultures may be problematic and further study of the differences in performance that may be attributable to cultural differences, with appropriate development of separate norms for differing ethnic groups as necessary, seems warranted. At the same time, the research related to gender as a possible confound has been equivocal. An understanding of the relation between demographic variables and test scores is needed if test results are to be interpreted accurately (e.g., Lowe & Reynolds, 1999; Lowe et al., 1999; Reynolds, 1997). As is common among neuropsychological tests (Reynolds, 1997), a great weakness of CPTs generally is that studies of demographic variables have not been done. Studies of ethnic and of gender bias are sorely lacking for these tests, particularly with regard to specificity and sensitivity of CPT scores, and are desperately needed, particularly given their widespread use (see Reynolds, in press, for methods for such research). Another shortfall of the CPT is the limited number of reliability studies. Overall, moderate to high temporal stability and internal consistency were noted when reliability studies had been completed; however, as different versions of the CPT may be conceptualized as separate measures, and these various CPT paradigms may not measure the exact same construct, reliability estimates from studies using one CPT cannot

reyn2-13.qxd

6/6/2003

9:20 AM

Page 313

13. Continuous Performance Tests

be presumed to apply to other CPTs. Studies addressing the comparative reliability of CPT scores across demographic variables and diagnostic groupings are needed if CPT scores are to be used and interpreted with accuracy (Lowe et al., 1999). Without evidence of both internal consistency and temporal stability, any conclusions related to diagnostic considerations or treatment effectiveness become spurious. The fact that the manuals suggest the use of CPTs in the diagnostic process specific to ADHD as well as other disorders should not be interpreted as indicating high levels of sensitivity or specificity. Clinicians need to be cautious in their interpretation of CPT results. The CPT is only one measure, and multiple sources of information and multiple measures should be used when assessing attention and impulse control problems in order to corroborate CPT findings. Taken together with direct observation, behavior rating scales, and other psychometric tests, the CPT may provide useful information. The CPT is an objective measure that is not subject to rater bias or observer drift. Based on our review, the level of performance on CPTs may be helpful in ruling out or identifying attentional problems. However, reliance on CPTs as a primary diagnostic tool in determining the presence of a specific disorder (e.g., ADHD) is not warranted and will result in an unacceptably high number of false-positive errors or overdiagnosis of ADHD. In clinical practice, it is unusual for one to be asked simply to distinguish normal from abnormal, yet this has been the focus of most of the research with the CPT. The real issue is the need to adequately differentiate among the various psychopathologies in order to make appropriate diagnosis and treatment decisions. Given the myriad possible variations to the CPT paradigm, it is possible that some combination(s) will prove more helpful in differential diagnosis than others. Determining if diagnostic algorithms with CPTs exist, however, requires multisite research that includes multiple groups with relevant psychopathologies and at differing developmental levels using different CPT paradigms in conjunction with other measures. This research has yet to be conducted but is needed. In the meantime, the use of CPTs as one component of the as-

313

sessment process as well as a tool for monitoring attentional functioning holds tremendous promise.

REFERENCES Alexander, D. D. (1973). Attention dysfunction in senile dementia. Psychological Reports, 32, 229–230. Allen, L. F. (1993). Developmental delay of frontal lobe functioning: A possible cause of attention deficits in children. Unpublished doctoral dissertation, Texas A & M University, College Station, TX. American Educational Research Association. (1999). Standards for educational and psychological testing. Washington, DC: Author. American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan. Anastopoulos, A. D., & Costabile, A. A. (1994). The Conners’ continuous performance test: A preliminary examination of its diagnostic utility. The ADHD Report, 2(5), 7–8. Anderson, V. E., Siegel, F. S., Fisch, R. O., & Wirt, D. (1969). Response of phenylketonuric children on a continuous performance test. Journal of Abnormal Psychology, 74, 358–362. Asarnow, R. F., Steffy, R., MacCrimmon, D. J., & Cleghorn, J. M. (1977). An attentional assessment of foster children at risk for schizophrenia. Journal of Abnormal Child Psychology, 86, 267–275. August, G. J., & Garfinkel, B. D. (1989). Behavioral and cognitive subtypes of ADHD. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 739–748. Aylward, G., Gordon, M., & Verhulst, S. (1997). Relationships between continuous performance task scores and other cognitive measures: Causality or commonality? Assessment, 4, 325–336. Baker, D. B., Taylor, C. J., & Leyva, C. (1995). Continuous performance tests: A comparison of modalities. Journal of Clinical Psychology, 51, 548–551. Ballard, J. C. (1996). Computerized assessment of sustained attention: A review of factors affecting vigilance performance. Journal of Clinical and Experimental Neuropsychology, 18, 843–863. Barkley, R. A. (1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19, 149–178. Barkley, R. A. (1998). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed.). New York: Guilford Press. Barkley, R. A., DuPaul, G. J., & McMurray, M. B. (1990). Comprehensive evaluation of attention deficit disorder with and without hyperactivity as defined by research criteria. Journal of Consulting and Clinical Psychology, 58, 775–789.

reyn2-13.qxd

314

6/6/2003

9:20 AM

Page 314

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Barkley, R. A., & Grodzinsky, G. M. (1994). Are tests of frontal lobe functions useful in the diagnosis of attention deficit disorders? The Clinical Neuropsychologist, 8, 121–139. Barkley, R. A., Grodzinsky, G. M., & DuPaul, G. (1992). Frontal lobe functions in attention deficit disorder with and without hyperactivity: A review and research report. Journal of Abnormal Child Psychology, 20, 163–188. Bauermeister, J. J., Berrios, V., Jimenez, A. L., Acevedo, L., & Gordon, M. (1990). Some issues and instruments for the assessment of attention deficit hyperactivity disorder in Puerto Rican children. Journal of Clinical Child Psychology, 19(1), 9–16. Beale, I. L., Matthew, P. J., Oliver, S., & Corballis, M. C. (1987). Performance of disabled and normal readers on the continuous performance test. Journal of Abnormal Child Psychology, 15, 229–238. Bock, R. D. (1982). The role of arousal in Tourette syndrome. Unpublished doctoral dissertation, New York University, New York. Bremer, D. A. (1989). Mini-CPT: A continuous performance test program for the Tandy PC–8 pocket computer. Behavior Research Methods, Instruments, and Computers, 21, 11–14. Broadbent, D. E. (1953). Noise, paced performance, and vigilance tasks. British Journal of Psychology, 44, 295–303. Broadbent, D. E. (1957). A mechanical model for human attention and immediate memory. Psychological Review, 64, 205–215. Brumm, V. L. (1994). Neuropsychological and psychological correlates of marital violence in a clinical sample. Unpublished doctoral dissertation, University of Southern California, San Diego. Brunner, R. L., & Berry, H. K. (1987). Phenylketonuria and sustained attention: The continuous performance test. International Journal of Clinical Neuropsychology, 9, 68–70. Burg, J. S., Burright, R. G., & Donovick, P. J. (1995). Performance data for traumatic brain injured subjects on the Gordon Diagnostic System (GDS) tests of attention. Brain Injury, 9, 395–403. Campbell, J. W., D’Amato, R. C., Raggio, D. J., & Stephens, K. D. (1991). Construct validity of the computerized continuous performance test with measures of intelligence, achievement, and behavior. Journal of School Psychology, 29, 143–150. Carter, C. S., Krener, P., Chaderjian, M., Northcutt, C., & Wolfe, V. (1995). Abnormal processing of irrelevant information in attention deficit hyperactivity disorder. Psychiatry Research, 56, 59–70. Cenedela, M. (1996). TOVA: Test of Variables of Attention user’s manual IBM/PC version 7.0. Los Alamitos, CA: Universal Attention Disorders. Chae, P. K. (1999). Correlation study between WISCIII scores and TOVA performance. Psychology in the Schools, 36, 179–185. Chee, P., Logan, G., Schachar, R., Lindsay, P., & Wachsmuth, R. (1989). Effects of event rate and display time on sustained attention in hyperactive, normal, and control children. Journal of Abnormal Child Psychology, 17, 371–391. Cohen, R. A. (1993a). Attentional control: Subcortical

and frontal lobe influences. In R. A. Cohen (Ed.), The neuropsychology of attention (pp. 219–254). New York: Plenum Press. Cohen, R. A. (1993b). Introduction. In R. A. Cohen (Ed.), The neuropsychology of attention (pp. 3–10). New York: Plenum Press. Cohen, R. A. (1993c). Neuropsychological assessment of attention. In R. A. Cohen (Ed.), The neuropsychology of attention (pp. 307–328). New York: Plenum Press. Cohen, R. A., Malloy, P. F., & Jenkins, M. A. (1999). Disorders of attention. In P. J. Snyder & P. D. Nussbaum (Eds.), Clinical neuropsychology (pp. 541–572). Washington, DC: American Psychological Association. Cohen, R. A., & O’Donnell, B. F. (1993). Models and mechanisms of attention: A summary. In R. A. Cohen (Ed.), The neuropsychology of attention (pp. 177–188). New York: Plenum Press. Cohen, R. J., & Swerdlik, M. E. (1999). Psychological testing and assessment: An introduction to tests and measurement (4th ed.). Mountain View, CA: Mayfield. Conners, C. K. (1992). Conner’s continuous performance test user’s manual. Toronto: Multi-Health Systems. Conners, C. K. (1995). Conners’ continuous performance test user’s manual. Toronto: Multi-Health Systems. Conners, C. K., & MHS Staff. (2000). Conners’ continuous performance test—II user’s manual. Toronto: Multi-Health Systems. Corkum, P. V., Schachar, R. J., & Siegel, L. S. (1996). Performance on the continuous performance task and the impact of reward. Journal of Attention Disorders, 1, 163–172. Corkum, P. V., & Siegel, L. S. (1993). Is the continuous performance task a valuable research tool for use with children with attention deficit hyperactivity disorder? Journal of Child Psychology and Psychiatry, 34, 1217–1239. Crosby, K. G. (1972). Attention and distractibility in mentally retarded and intellectually average children. American Journal of Mental Deficiency, 77, 46–53. Czerny, J. A., O’Laughlin, E. M., & Griffioen, S. (1999). Diagnosing ADHD: A clinical comparison of two continuous performance tests. Manuscript submitted for publication. Das, J. P., Snyder, J. J., & Mishra, R. K. (1992). Assessment of attention: Teachers’ rating scales and measures of selective attention. Journal of Psychoeducational Assessment, 10, 37–46. Draeger, S., Prior, M., & Sanson, A. (1986). Visual and auditory attention performance in hyperactive children: Competence or compliance. Journal of Abnormal Child Psychology, 14, 411–424. Driscoll, S. M. (1994). The development and evaluation of objective auditory and visual attention instruments. Unpublished doctoral dissertation, Northwestern University. Dumont, R., Tamborra, A., & Stone, B. (1995). Continuous performance tests: The TOVA, Conners CPT, and IVA. Communique, 21(1), 22–24.

reyn2-13.qxd

6/6/2003

9:20 AM

Page 315

13. Continuous Performance Tests DuPaul, G. J., Anastopoulos, A. D., Shelton, T. L., Guevremont, D. C., & Metevia, L. (1992). Multimethod assessment of attention deficit hyperactivity disorder: The diagnostic utility of clinic-based tests. Journal of Clinical Child Psychology, 21, 394–402. Earle-Boyer, E. A., Serper, M. R., Davidson, M., & Harvey, P. D. (1991). Continuous performance tests in schizophrenic patients: Stimulus and medication effects on performance. Psychiatry Research, 37, 47–56. Eliason, M. J. (1988). Neuropsychological patterns: Neurofibromatosis compared to developmental learning disorders. Neurofibromatosis, 1, 17–25. Eliason, M. J., & Richman, L. C. (1987). The continuous performance test in learning disabled and nondisabled children. Journal of Learning Disabilities, 20, 614–619. Erickson, W. D., Yellin, A. M., Hopwood, J. H., Realmuto, G. M., & Greenberg, L. M. (1984). The effects of neuroleptics on attention in adolescent schizophrenics. Biological Psychiatry, 19, 745–753. Erlenmeyer-Kimling, L., & Cornblatt, B. (1978). Attentional measures in a study of children at high risk for schizophrenia. Journal of Psychiatric Research, 14, 93–98. Evans, C. A. (1988). A study of vigilance, memory processing speed, cognitive ability and the prediction of academic achievement in children 6–14 years old. Unpublished doctoral dissertation, Ohio State University, Columbus. Finkelstein, J. R. J., Cannon, T. D., Gur, R. E., Gur, R. C., & Moberg, P. (1997). Attentional dysfunctions in neuroleptic naïve and neuroleptic withdrawn schizophrenic patients and their siblings. Journal of Abnormal Psychology, 106, 203–212. Fischer, M., Newby, R. F., & Gordon, M. (1995). Who are the false negatives on continuous performance tests? Journal of Clinical Child Psychology, 24, 427–433. Fitzpatrick, P. A., Klorman, R., Brumaghim, J. T., & Borgstedt, A. D. (1992). Effects of sustained release and standard preparations of methylphenidate on attention deficit disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 226–234. Floyd, R. G. (1999, August). Convergent and divergent validity of the preschool checking task. Presented at the annual convention of the American Psychological Association, Boston. Forbes, G. B. (1998). Clinical utility of the Test of Variables of Attention (TOVA) in the diagnosis of attention deficit hyperactivity disorder. Journal of Clinical Psychology, 54, 461–476. Friedman, D., Vaughan, H., & Erlenmeyer-Kimling, L. (1978). Stimulus and response related components of the late positive complex in visual discrimination tasks. Electroencephalography and Clinical Neurophysiology, 45, 319–330. Friedman, D., Vaughan, H. G., Jr., & Erlenmeyer-Kimling, L. (1981). Multiple late positive potentials in two visual discrimination tasks. Neurophysiology, 18, 635–649. García-Sánchez, C., Estévez-Gonzáles, A., SuárezRomero, E., & Junqué, C. (1997). Right hemisphere

315

dysfunction in subjects with attention-deficit disorder with and without hyperactivity. Journal of Child Neurology, 12, 107–115. Garfinkel, B. G., & Klee, S. H. (1983). A computerized assessment battery for attention deficits. Psychiatry Hospitals, 14, 163–166. Garretson, H. B., Fein, D., & Waterhouse, L. (1990). Sustained attention in children with autism. Journal of Autism and Developmental Disorders, 20, 101–114. Girardi, N. L., Shaywitz, S. E., Marchione, K., Fleischman, S. J., Jones, T. W., & Tamborlane, W. V. (1995). Blunted catecholamine responses after glucose ingestion in children with attention deficit disorder. Pediatric Research, 38, 539–542. Goldstein, P. C., Rosenbaum, G., & Taylor, M. J. (1997). Assessment of differential attention mechanisms in seizure disorders and schizophrenia. Neuropsychology, 11, 309–317. Golier, J., Yehuda, R., Cornblatt, B., Harvey, P., Gerber, D., & Levengood, R. (1997). Sustained attention in combat-related posttraumatic stress disorder. Integrative Physiological and Behavioral Science, 32, 52–61. Gordon, M. (1983). The Gordon Diagnostic System. DeWitt, NY: Gordon Systems. Gordon, M. (1986a). How is a computerized attention test used in the diagnosis of attention deficit disorder. Journal of Children in Contemporary Society, 19, 53–64. Gordon, M. (1986b). Microprocessor-based assessment of attention deficit disorders. Psychopharmacology Bulletin, 22, 288–290. Gordon, M., & Mettleman, B. B. (1988). The assessment of attention: I. Standardization and reliability of a behavior-based measure. Journal of Clinical Psychology, 44, 682–690. Gordon, M., DiNiro, D., Mettelman, B. B., & Tallmadge, J. (1989). Observations of test behavior, quantitative scores and teacher ratings. Journal of Psychoeducational Assessment, 7, 141–147. Gordon Systems. (1986). Technical manual for the Gordon Diagnostic System. DeWitt, NY: Author. Gordon Systems. (1987). Interpretive manual for the Gordon Diagnostic System. DeWitt, NY: Author. Gordon Systems. (1991). Administration manual for the Gordon Diagnostic System. DeWitt, NY: Author. Greenberg, L. M. (1988–1999). The Test of Variables of Attention (TOVA). Los Alamitos, CA: Universal Attention Disorders. Greenberg, L. M. (1996–1999). The Test of Variables of Attention—Auditory (TOVA-A). Los Alamitos, CA: Universal Attention Disorders. Greenberg, L. M., & Crosby, R. D. (1992). A summary of developmental normative data on the T.O.V.A. ages 4 to 80+. Unpublished manuscript. Greenberg, L. M., & Kindschi, C. L. (1996). TOVA Test of Variables of Attention: Clinical guide. St. Paul, MN: TOVA Research Foundation. Greenberg, L. M., Kindschi, C. L., & Corman, C. L. (1999). TOVA Test of Variables of Attention: Clinical guide. Los Alamitos, CA: Universal Attention Disorders. Greenberg, L. M., & Waldman, I. D. (1993). Develop-

reyn2-13.qxd

316

6/6/2003

9:20 AM

Page 316

IV. SPECIFIC SYNDROMES AND SYMPTOMS

mental normative data on the Test of Variables of Attention (T.O.V.A.). Journal of Child Psychology and Psychiatry, 34, 1019–1030. Halperin, J. M., Matier, K., Bedi, G., Sharma, V., & Newcorn, J. (1992). Specificity of inattention, impulsivity, and hyperactivity to the diagnosis of attention deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 190–196. Halperin, J. M., Newcorn, J. H., Matier, K., Bedi, G., Hall, S., & Sharma, V. (1995). Impulsivity and the initiation of fights in children with disruptive behavior disorders. Journal of Child Psychology and Psychiatry, 36, 1199–1211. Halperin, J. M., Newcorn, J. H., Matier, K., Sharma, V., Mckay, K. E., & Schwartz, S. (1993). Discriminant validity of attention-deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 1038–1043. Halperin, J. M., Sharma, V., Greenblatt, E., & Schwartz, S. T. (1991). Assessment of the continuous performance test: Reliability and validity in a non-referred sample. Psychological Assessment, 3, 803–808. Halperin, J. M., Wolf, L. E., Greenblatt, E., & Young, J. G. (1991). Subtype analysis of commission errors on the continuous performance test in children. Developmental Neuropsychology, 7, 207–217. Harper, G. W., & Ottinger, D. R. (1992). The performance of hyperactive and control preschoolers on a new computerized measure of visual vigilance: The preschool vigilance task. Journal of Child Psychology and Psychiatry, 33, 1365–1372. Harris, E. L., Schuerholz, L. J., Singer, H. S., Reader, M. J., Brown, J. E., Cox, C., Mohr, J., Chase, G. A., & Denckla, M. B. (1995). Executive function in children with Tourette syndrome and/or attention deficit hyperactivity disorder. Journal of International Neuropsychological Society, 1, 511–516. Hebb, D. O. (1958). A textbook of psychology. Philadelphia: Saunders. Hickey, J. E., Suess, P. E., Newlin, D. B., & Spurgeon, L. (1995). Vagal tone regulation during sustained attention in boys exposed to opiates in utero. Addictive Behaviors, 2, 43–59. Hoerig, D. C., D’Amato, R. C., Raggio, D. J., & Martin, J. D. (1998, November). Comparing the TOMAL, CPT and WISC-III: Are memory, attention, and intelligence related? Paper presented at the annual meeting of the National Academy of Neuropsychology, Washington, DC. Holcomb, P. J., Ackerman, P. T., & Dykman, R. A. (1985). Cognitive event-related potentials in children with attention and reading deficits. Psychophysiology, 22, 656–667. Hoy, E., Weiss, G., Minde, K., & Cohen, N. (1978). The hyperactive child at adolescence: Cognitive, emotional, and social functioning. Journal of Abnormal Child Psychology, 6, 311–324. Hynd, G. W., Morgan, A. E., Edmonds, J. E., Black, K., Riccio, C. A., & Lombardino, L. (1995). Reading disabilities, comorbid psychopathology, and the specificity of neurolinguistic deficits. Developmental Neuropsychology, 11, 311–322.

Jensen, A. R. (1982). Reaction time and psychometric g. In H. J. Eysenck (Ed.), A model for intelligence (pp. 93–132). Berlin: Springer-Verlag. Jerison, H. J. (1967). Signal detection theory in the analysis of human vigilance. Human Factors, 9, 285–288. Johnson, B. D. (1993). Attention deficit hyperactivity disorder and undifferentiated attention deficit disorder: A study of neuropsychological differences. Unpublished thesis, University of Iowa, Iowa City. Kardell, E. G. (1994). Evaluation of the WISC-III in the assessment of attention and concentration processes in children. Unpublished doctoral dissertation, Pace University, New York. Katz, K. S., Dubowitz, L. M. S., Henderson, S., Jongmans, M., Kay, G. G., Nolte, C. A., & deVries, L. (1996). Effect of cerebral lesions on continuous performance test responses of school age children born prematurely. Journal of Pediatric Psychology, 21, 841–855. Kaufmann, P. M., Fletcher, J. M., Levin, H. S., & Miner, M. E. (1993). Attentional disturbance after pediatric closed head injury. Journal of Child Neurology, 8, 348–353. Keith, R. W. (1994). The auditory continuous performance test manual. San Antonio, TX: Psychological Corporation. Kintslinger, G. (1987). The use of the Gordon Diagnostic System for assessing attention deficit disorders as compared with traditional adaptive behavior measures in the public schools. Unpublished doctoral dissertation, University of Nebraska, Lincoln. Klorman, R., Brumaghim, J. T., Fitzpatrick, P. A., & Borgstedt, A. D. (1991). Methylphenidate speeds evaluation processes of attention deficit disorder adolescents during a continuous performance test. Journal of Abnormal Child Psychology, 19, 263–283. Koriath, U., Gualtieri, T., van Bourgondien, M., Quade, D., & Werry, J. (1985). Construct validity of clinical diagnosis in pediatric psychiatry: Relationship among measures. Journal of the American Academy of Child and Adolescent Psychiatry, 24, 429–436. Kupietz, S. S., & Richardson, E. (1978). Children’s vigilance performance and inattentiveness in the classroom. Journal of Child Psychology and Psychiatry, 19, 145–154. Lam, C. M., & Beale, I. L. (1991). Relations among sustained attention, reading performance, and teachers’ ratings of behavior problems. Remedial and Special Education (RASE), 12, 40–47. Leark, R. A., Dixon, D., Hoffman, T., & Huynh, D. (1999, November). The effects of malingering on the Test of Variables of Attention (TOVA). Paper presented at the annual meeting of the National Academy of Neuropsychology, San Antonio, TX. Leark, R. A., Dupuy, T. R., Greenberg, L. M., Corman, C. L., & Kindschi, C. L. (1996). TOVA Test of Variables of Attention: professional manual version 7.0. Los Alamitos, CA: Universal Attention Disorders. Levav, M. L. (1991). Attention performance in children affected with absence epilepsy and their first

reyn2-13.qxd

6/6/2003

9:20 AM

Page 317

13. Continuous Performance Tests degree relatives. Unpublished doctoral dissertation, University of Maryland, College Park. Levin, E. D., Conners, C.K., Sparrow, E., Hinton, S. C., Erhardt, D., Meek, W. H., Rose, J. E., & March, J. (1996). Nicotine effects on adults with attentiondeficit/hyperactivity disorder. Psychopharmacology, 123, 55–63. Levy, F. (1980). The development of sustained attention (vigilance) and inhibition in children: Some normative data. Journal of Child Psychology and Psychiatry, 21, 77–84. Levy, F., & Hobbes, G. (1988). The action of stimulant in attention deficit disorder with hyperactivity: Dopaminergic, noradrenergic, or both. Journal of the American Academy of Child and Adolescent Psychiatry, 27, 802–805. Liu, S. K., Hwu, H., & Chen, W. J. (1997). Clinical symptom dimensions and deficits on the continuous performance test in schizophrenia. Schizophrenia Research, 25, 211–219. Llorente, A. M., Amado, A. J., Voigt, R. G., Berretta, M. C., Fraley, J. K., Jensen, C. L., & Heird, W. C. (2000). Internal consistency, temporal stability, and reproducibility of individual index scores on the Tests of Variables of Attention (TOVA) in children with attention-deficit/hyperactivity disorder (AD/ HD). Archives of Clinical Neuropsychology, 15, 1–12. Lowe, P. A., & Reynolds, C. R. (1999). Age, gender, and education may have little influence on error patterns in the assessment of set-shifting and rule induction among normal elderly. Archives of Clinical Neuropsychology, 14, 303–315. Lowe, P. A., Reynolds, C. R., Riccio, C. A., & Moore, J. J. (1999). Psychometric characteristics of continuous performance tests (CPT). Manuscript in preparation, Department of Educational Psychology, Texas A&M University, College Station, TX. Luria, A. R. (1966). Higher cortical functions in man. New York: Basic Books. Luria, A. R. (1980). Higher cortical functions in man (2nd ed.). New York: Basic Books. Mansour, C. S., Haier, R. J., & Buchsbaum, M. S. (1996). Gender comparisons of cerebral glucose metabolic rate in healthy adults during a cognitive task. Personality and Individual Differences, 20, 183–191. Mapou, R. L. (1999, November). Clinical assessment of attention: Models and methods. Paper presented at the annual meeting of the National Academy of Neuropsychology, San Antonio, TX. Marks, D. J., Himelstein, J., Newcorn, H. H., & Halperin, J. M. (1999). Identification of ADHD subtypes using laboratory-based measures: A cluster analysis. Journal of Abnormal Child Psychology, 27, 167–175. Merola, J. L., & Leiderman, J. (1985). The effect of task difficulty upon the extent tot which performance benefits from between hemisphere division of inputs. International Journal of Neuroscience, 51, 35–44. Miller, J. M. (1996). Patterns of attentional impairment in children with generalized and partial epilepsy. Unpublished doctoral dissertation, Washington University, St. Louis, MO.

317

Mirsky, A. F. (1989). The neuropsychology of attention: Elements of a complex behavior. In E. Perecman (Ed.), Integrating theory and practice in neuropsychology (pp. 75–91). Hillsdale, NJ: Erlbaum. Mirsky, A. F., Anthony, B. J., Duncan, C. C., Ahearn, M. B., & Kellam, S. G. (1991). Analysis of the elements of attention: A neuropsychological approach. Neuropsychology Review, 2, 109–145. Mirsky, A. F., Fantie, B. D., & Tatman, J. E. (1995). Assessment of attention across the lifespan. In R. L. Mapou, & J. Spector (Eds.), Clinical neuropsychological assessment: A cognitive approach (pp. 17–48). New York: Plenum Press. Moruzzi, G., & Magoun, H. W. (1949). Brainstem reticular formation and activation of the EEG. Electroencephalography and Clinical Neurophysiology, 1, 455–473. Nuechterlein, K. H. (1983). Signal detection in vigilance tasks and behavioral attributes of schizophrenic mothers and among hyperactive children. Journal of Abnormal Psychology, 92, 4–28. O’Dougherty, M., Nuechterlein, K. H., & Drew, B. (1984). Hyperactive and hypoxic children: Signal detection, sustained attention, and behavior. Journal of Abnormal Psychology, 93, 178–191. Parasuraman, R. (1979). Memory load and event rate control sensitivity decrements in sustained attention. Science, 205, 924–927. Parasuraman, R. (1984a). The psychobiology of sustained attention. In J. S. Warm (Ed.), Sustained attention in human performance (pp. 61–100). New York: Wiley. Parasuraman, R. (1984b). Sustained attention in detection and discrimination. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 243–271). Orlando, FL: Academic Press. Parasuraman, K., & Davies, D. R. (1977). A taxonomic analysis of vigilance performance. In R. R. Mackie (Ed.), Vigilance theory, operational performance, and physiological correlates (pp. 559–574). New York: Plenum Press. Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, 25–42. Power, T. J. (1992). Contextual factors in vigilance testing of children with ADHD. Journal of Abnormal Child Psychology, 20, 579–593. Raggio, D. J., & Whitten, J. M. (1994). Raggio Evaluation of Attention Deficit Disorder READD manual. Unpublished manuscript available from D. J. Raggio, University of Mississippi Medical Center, Department of Pediatrics, Jackson, MS. Rapoport, J. L., Buchsbaum, M. S., Weingartner, H., Zahn, T P., Ludlow, C., & Mikkelsen, E. J. (1980). Dextroamphetamine: Its cognitive and behavioral affects in normal and hyperactive boys and normal men. Archives of General Psychiatry, 37, 933–943. Reynolds, C. R. (1997). Measurement and statistical problems in neuropsychological assessment of children. In C. R. Reynolds & E. Fletcher-Janzen (Eds.), Handbook of clinical child neuropsychology (2nd ed., pp. 296–319). New York: Plenum Press. Reynolds, C. R. (in press). Methods for detecting and evaluating cultural bias in neuropsychological tests.

reyn2-13.qxd

318

6/6/2003

9:20 AM

Page 318

IV. SPECIFIC SYNDROMES AND SYMPTOMS

In E. Fletcher-Janzen, T. Strickland, & C. R. Reynolds (Eds.), Handbook of cross-cultural neuropsychology. New York: Plenum Press. Reynolds, C. R., & Kamphaus, R. W. (1992). Manual for the Behavioral Assessment System for Children (BASC). Circle Pines, MN: American Guidance Service. Riccio, C. A., Cohen, M. J., Hynd, G. W., & Keith, R. W. (1996). Validity of the auditory continuous performance test in differentiating central processing auditory disorders with and without ADHD. Journal of Learning Disabilities, 29, 561–566. Riccio, C. A., & Jemison, S. (1998). ADHD and emergent literacy: Influence of language factors. Reading and Writing Quarterly, 14, 43–58. Riccio, C. A., & Reynolds, C. R. (2001). Continuous performance tests are sensitive to ADHD in adults but lack specificity: A review and critique for differential diagnosis. In J. Wasserstein, L. E. Wolf, & F. F. LeFever (Eds.), Adult attention deficit disorder: Brain mechanisms and life outcomes (pp. 113–139). New York: New York Academy of Science. Riccio, C. A., Reynolds, C. R., & Lowe, P. A. (2001). Clinical applications of the continuous performance test: Measuring attention and impulsive responding in children and adults. New York: Wiley. Richards, G. P., Samuels, S. J., Turnure, J. E., & Ysseldyke, J. E. (1990). Sustained and selective attention in children with learning disabilities. Journal of Learning Disabilities, 23, 129–136. Robins, P. M. (1992). A comparison of behavioral and attentional functioning in children diagnosed as hyperactive or learning disabled. Journal of Abnormal Child Psychology, 20, 65–82. Romans, S. M., Roeltgen, D. P., Kushner, H., & Ross, J. L. (1997). Executive function in girls with Turner’s syndrome. Developmental Neuropsychology, 13, 23–40. Rosvold, H., Mirsky, A., Sarason, I., Bransome, E. D. Jr., & Beck, L. H. (1956). A continuous performance test of brain damage. Journal of Consulting Psychology, 20, 343–350. Rueckert, L., & Grafman, J. (1996). Sustained attention deficits in patients with right frontal lesions. Neuropsychologia, 34, 953–963. Rund, B. R., Zeiner, P., Sundet, K., Oie, M., & Bryhn, G. (1998). No vigilance deficit found in either young schizophrenic or ADHD subjects. Scandinavian Journal of Psychology, 39, 101–107. Rutter, M. (1981). Psychological sequelae of brain damage in children. American Journal of Clinical Neuropsychology, 138, 1533–1544. Sandford, J. A., Fine, A. H., & Goldman, L. (1995a, November). A comparison of auditory and visual processing in children with ADHD using the IVA continuous performance test. Paper presented at the 1995 annual convention of CHADD, Washington, DC. Sandford, J. A., Fine, A. H., & Goldman, L. (1995b, August). Validity study of the IVA: A visual and auditory CPT. Paper presented at the 1995 annual convention of the American Psychological Association, New York. Sandford, J. A., & Turner, A. (1994–1999). Integrated

Visual and Auditory (IVA) continuous performance test. Richmond, VA: BrainTrain. Sandford, J. A., & Turner, A. (1995). Manual for the Integrated Visual and Auditory (IVA) continuous performance test. Richmond, VA: BrainTrain. Schachar, R., Logan, G., Wachsmuth, R., & Chajczyk, D. (1988). Attaining and maintaining preparation: A comparison of hyperactive, normal, and disturbed control children. Journal of Abnormal Child Psychology, 16, 361–378. Seckler, P., Burns, W., Montgomery, D., & Sandford, J. A. (1995, November). A reliability study of IVA: Intermediate visual and auditory continuous performance test. Paper presented at the 1995 annual convention of CHADD, Washington, DC. Sergeant, J. A., & Scholten, C. A. (1985). On resource strategy limitations in hyperactivity: Cognitive impulsivity reconsidered. Journal of Child Psychology and Psychiatry, 26, 97–109. Shapiro, M. B., Morris, R. D., Morris, M. K., Flowers, C., & Jones, R. W. (1999). A neuropsychologically based assessment model of the structure of attention in children. Developmental Neuropsychology, 14, 657–677 Shucard, D. W., Benedict, R. H. B., Tekok-Kilic, A., & Lichter, D. G. (1997). Slowed reaction time during a continuous performance test in children with Tourette’s syndrome. Neuropsychology, 11, 147–155. Siegel, D. J. (2000). Perception and cognition. In B. Sadock & V. Sadock (Eds.), Kaplan & Sadock’s comprehensive textbook of psychiatry (7th ed., pp. 386–402). New York: Lippincott. Slicker, E. K. (1991). Validation of an objective measure of perseveration in children with attention deficit hyperactivity disorder. Unpublished doctoral dissertation, Texas A & M University, College Station, TX. Sohlberg, M. M., & Mateer, C. A. (1989). Attention process training. Puyallup, WA: Association for Neuropsychological Research and Development. Strandburg, R. J., Marsh, J. T., Brown, W. S., Asarnow, R. F., Guthrie, D., & Higa, J. (1990). Event-related correlates of impaired attention in schizophrenic children. Biological Psychiatry, 27, 1103–1115. Strandburg, R. J., Marsh, J. T., Brown, W. S., Asarnow, R. F., Higa, J., & Guthrie, D. (1994). Continuous processing related ERPs in schizophrenic and normal children. Biological Psychiatry, 35, 525–538. Suslow, T., & Arolt, V. (1997). Paranoid schizophrenia: Non-specificity of neuropsychological vulnerability markers. Psychiatry Research, 72, 103–114. Swanson, H. L., & Cooney, J. B. (1989). Relationship between intelligence and vigilance in children. Journal of School Psychology, 27, 141–153. Swets, J. A. (1973). The relative operating characteristic in psychology. Science, 182, 990–1000. Swets, J. A. (1984). Mathematical models of attention. In R. Parasuraman & D. R. Davies (Eds.), Varieties of attention (pp. 183–242). New York: Academic Press. Sykes, D.H., Douglas, V. I., Weiss, G., & Minde, K. M. (1971). Attention in hyperactive children and the effects of methylphenidate (ritalin). Journal of Child Psychology and Psychiatry, 12, 129–139.

reyn2-13.qxd

6/6/2003

9:20 AM

Page 319

13. Continuous Performance Tests Tarnowski, K. J., Prinz, R. J., & Nay, S. M. (1986). Comparative analysis of attentional deficits in hyperactive and learning disabled children. Journal of Abnormal Psychology, 95, 341–345. Teicher, M. H., Ito, Y., Glod, C. A., & Barber, N. I. (1996). Objective measurement of hyperactivity and attentional problems in ADHD. Journal of American Academy of Child and Adolescent Psychiatry, 35, 334–342. Trommer, B. L., Hoeppner, J. B., Lorber, R., & Armstrong, K. (1988). Pitfalls in the use of a continuous performance test as a diagnostic tool in attention deficit disorder. Journal of Developmental and Behavioral Pediatrics, 9, 339–345. Tupler, L. A. (1989). The role of attentional resources in the memory performance of elderly adults: An investigation of the integration deficit hypothesis. Unpublished doctoral dissertation, Emory University, Atlanta, GA. Turner, A., & Sandford, J. A. (1995a, November). Developmental age and sex differences in auditory and visual processing using the IVA continuous performance test. Paper presented at the 1995 annual convention of CHADD, Washington, DC. Turner, A., & Sandford, J. A. (1995b, August). A normative study of IVA: Integrated visual and auditory continuous performance test. Paper presented at the 1995 annual convention of the American Psychological Association, New York. van Zomeran, A. H., & Brouwer, W. H. (1994). Clinical neuropsychology of attention. New York: Oxford Press.

319

Vernon, P. A. (1989). The heritability of measures of speed of information processing. Personality and Individual Differences, 10, 573–576. Voeller, K. K. S. (1991). Toward a neurobiologic nosology of attention deficit hyperactivity disorder. Journal of Clinical Neurology, 65, 52–58. Vygotsky, L. S. (1980). Mind in society: The development of higher psychological process. Cambridge, MA: Harvard University Press. Wagner, A. E. (1987). Gender differences in hyperactive school-age children. Unpublished master’s thesis, Michigan State University, East Lansing. Werry, J. S., Elkind, G. S., & Reeves, J. C. (1987). Attention deficit, conduct, oppositional, and anxiety disorders in children: III. Laboratory differences. Journal of Abnormal Child Psychology, 15, 409–428. Wherry, J., Paal, N., Jolly, J. Balkozar, A., Holloway, C., Everett, B., & Vaught, L. (1993). Concurrent and discriminant validity of the Gordon Diagnostic System: A preliminary study. Psychology in the Schools, 30, 29–36. Whyte, J. (1992). Attention and arousal: Basic science aspects. Archives of Physical Medicine and Rehabilitation, 73, 940–949. Willson, V. L., & Reynolds, C.R. (1982). Methodological and statistical problems in determining membership in clinical populations. Clinical Neuropsychology, 4, 134–138. Zubin, J. (1975). Problem of attention in schizophrenia. In M. L. Kietzman, S. Sutton, & J. Zubin (Eds.), Experimental approaches to psychopathology (pp. 139–166). New York: Academic Press.

reyn2-14.qxd

6/6/2003

9:21 AM

Page 320

14 Assessment of Attention-Deficit/ Hyperactivity Disorder

SHARI NEUL HEATHER APPLEGATE RON DRABMAN

Attention-deficit/hyperactivity disorder (ADHD) is described as a “persistent pattern of inattention and/or hyperactivity and impulsivity that is more frequent and severe than is typically observed in individuals at a comparable level of development” (American Psychiatric Association, 1994, p. 78). The behavioral signs are often evident in early childhood, are relatively chronic in nature, and are not readily accounted for on the basis of gross neurological, sensory/language/motor impairment, mental retardation, or severe emotional disturbance. Implied in this characterization is the notion that although the specific etiology of ADHD is not known, there are several causes of similar clusters of behavior that must be ruled out before an ADHD diagnosis is given. This concept, ruling out alternative explanations for ADHD-like clusters of behavior, is a crucial component of any proper assessment of ADHD. Successful undertaking of this task is usually more difficult than it appears, however, especially as the characteristics of ADHD may overlap considerably with other psychological disorders. Furthermore, the diagnosis of ADHD is controversial, especially as the etiology of the disorder is not known. ADHD is most likely a heterogeneous group of disorders

with multiple etiologies. However, significant disagreement remains regarding the underlying construct of ADHD. Consequently, the clinical diagnosis has been the most prevalent way of investigating abnormalities of attention (Taylor, 1998). In fact, Taylor (1998) argues that ADHD will increasingly be used to describe an area of investigation rather than a psychiatric disorder. This is not surprising given the fact that literature reviews regarding the characteristics and causes of ADHD have revealed significant diversity as to what constitutes ADHD. For instance, Goodman and Poillion (1992) conducted a review of the literature on the characteristics and causes of ADHD and found that a total of 69 characteristics and 38 causes were attributed to ADHD. Furthermore, there was no discernible pattern for identifying ADHD and little agreement regarding its etiology. The authors argued that the pattern emerging regarding the evolution of ADHD is similar to that of minimal brain dysfunction (MBD) in the 1960s (Goodman & Poillion, 1992). MBD was initially thought of as a medically based, organic syndrome, and a list of 99 associated characteristics was developed. Ultimately it was because of this proliferation of symptoms and vagueness of defini320

reyn2-14.qxd

6/6/2003

9:21 AM

Page 321

14. Assessment of ADHD

tion that the term was abandoned in 1966 (Coles, 1987). Clinicians are challenged to consider the possible underlying causes of pathological clusters of behavior before making a diagnosis. As Palmeri (1996) noted, this challenge is especially difficult because the current, fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994)(as well as past editions) is merely a nosology and “(it) unwittingly invites many clinicians to skirt the thoughtful pursuit of pathogenetic considerations” (p. 253). This chapter focuses on assessing ADHD in such a way as to increase the probability that other causes of ADHD-like clusters of behavior will either be identified or ruled out, as well as provide specific information regarding the ADHD child’s deficits and strengths to provide more comprehensive, effective interventions. The chapter covers five assessment-related topics: (1) differential diagnosis of ADHD and comorbidity with other disorders is discussed; (2) general issues related to the assessment of children are presented; (3) our model of psychoeducational assessment (including behavioral assessment and psychoeducational testing) is delineated; (4) information regarding ancillary assessments that may further define the problem and delineate specific areas for remediation is supplied; and (5) treatment recommendations that target the child’s specific deficits and use the child’s relative strengths are discussed.

DIFFERENTIAL DIAGNOSIS AND COMORBIDITY

321

Many psychological disorders are misidentified as ADHD. Teachers frequently mislabel children with mental retardation, borderline intellectual functioning, and learning disabilities as having ADHD (Landman & McCrindle, 1986), although ADHD can occur comorbidly with these conditions. Physical causes of ADHD-like clusters of behavior include impaired vision and hearing, seizures, traumatic brain injury, acute or chronic illness, poor nutrition, and sleep disorders, indicating that a thorough medical evaluation should be conducted before an ADHD diagnosis is considered (American Academy of Child & Adolescent Psychiatry [AACAP], 1997). Various emotional disorders may mimic ADHD as well, including anxiety, depression, sequelae of abuse and neglect, Tourette syndrome, bipolar disorder (BPD), conduct disorder (CD), and oppositional defiant disorder (ODD). Indeed, most psychiatric disorders may present with characteristics similar to ADHD. It is especially important that the best possible differential diagnoses are made because treatment options vary considerably depending on presumed etiology. In fact, inaccurate ADHD diagnoses may lead to treatments that are contraindicated (e.g., prescribing a stimulant for a child whose ADHD-like symptoms are the results of an anxiety disorder). In addition, early-onset BPD may be difficult to distinguish from ADHD, but once again, differential treatment makes it crucial that an appropriate diagnosis is given. Some distinguishing features of ADHD may be earlier age of onset, sustained clinical course, and family history (AACAP, 1997). The Mania Rating Scale may be useful as an adjunctive instrument (Fristad, Weller, & Weller, 1995) to discern symptoms of earlyonset BPD from ADHD. Children with ADHD frequently experience other psychological disorders as well. Comorbidity estimates range from 10% to 50% depending on the diagnosis and criteria used (AACAP, 1997). As many as 50% of clinically referred children with ADHD have an ODD diagnosis, 30–50% have a CD diagnosis, 15–20% have a mood disorder diagnosis, and 20–25% have an anxiety disorder diagnosis (Biederman, Newcorn, & Sprich, 1991; Newcorn & Halperin, 1994). Tourette syndrome, chronic tic disorder,

reyn2-14.qxd

322

6/6/2003

9:21 AM

Page 322

IV. SPECIFIC SYNDROMES AND SYMPTOMS

substance abuse, and speech/language delays commonly co-occur with ADHD, although estimates of the prevalence are not known (AACAP, 1997). When assessing a child with characteristics of ADHD it is extremely important to remember that several disorders show manifestations similar to ADHD. Determining that a child exhibits significant “inattention, hyperactivity, and impulsivity” is not sufficient to warrant an ADHD diagnosis. A functional assessment in which the “causes” (for lack of a more accurate term) of the ADHD behaviors are established is crucial if the best treatment is to be provided. Although DSM-IV does not take into account the etiology of a particular constellation of behaviors, there is sufficient evidence from the behavior analytic literature to suggest that the etiology of disorders is important for treatment purposes. Therefore, although a child exhibiting the classic ADHD “triad” of behaviors may meet technical criteria for a DSM-IV diagnosis, clinicians should be wary of automatically labeling the child as ADHD because treatment options vary considerably based on the underlying causes of the symptoms. Indeed, a cautionary statement in DSM-IV indicates that “the specified diagnostic criteria for each mental disorder are offered as guidelines for making diagnoses” (p. xxvii), suggesting that there are conditions in which an individual may meet criteria for a disorder but that it should be withheld.

GENERAL ISSUES IN THE ASSESSMENT OF CHILDREN Because children are referred for assessment and treatment by an adult, practitioners should address some important but practical issues. First, child behavioral assessments should be conducted within a developmental framework to determine whether the child’s behavior is within the expected developmental limits. However, guidelines for what is normative behavior may conflict with a parent’s view of what is acceptable for his or her child. When this is the case, educating the parent about normal child development may resolve the problem or change the parents’ perception of the severity of the problem. Second, assessment of a

child’s behavioral difficulties requires evaluation of the behavior of relevant persons in the child’s environment (e.g., parents, siblings, teachers, and classmates), and this requirement may be uncomfortable for the referring adult who views the child as the only source of the problem. Communicating with other relevant persons with regard to their own behavior and its impact on the child’s behavior necessitates considerable tact and adept interviewing skills. The theory underlying a behavioral approach to assessment such as that delineated in this chapter requires adequate sampling of relevant settings and stimuli to address the variability of children’s behavior across settings. This is best accomplished by using a multimodal assessment approach in which multiple informants are interviewed, problem behaviors targeted for intervention are observed in multiple settings, and a fluid process of hypothesis testing is employed regarding the nature of the problem, antecedent conditions, likely consequences without intervening, and expectations for treatment (Barkley & Edwards, 1998; Mash & Terdal, 1988).

PSYCHOEDUCATIONAL ASSESSMENT An assessment model based on a consideration of myriad biopsychosocial individual differences coupled with a proficient understanding of developmental processes is essential in conducting a thorough assessment and thereby designing an effective intervention. In our clinic, we adhere to a biopsychosocial model which stipulates that assessment and treatment should include consideration of the interaction of biological, social, and psychological factors. Impairments in functioning in any area may affect a child’s functioning in other areas with possible bidirectional effects. This model emphasizes integration of numerous variables and their interactions. It has particular utility in the assessment and treatment of children’s disorders, such as ADHD, that may result from impairments in these domains of functioning (Newcomb & Drabman, 1995). Using this model to guide and organize our multisource data collection, we assess variables in each domain (biological, social and psychological)

reyn2-14.qxd

6/6/2003

9:21 AM

Page 323

14. Assessment of ADHD

to distinguish those impairments most relevant for a particular child. This assessment is achieved via our psychoeducational approach, which consists of focused clinical interviews with the parent(s), teacher(s), and referred child; parent-completed rating scales; and administration of a thorough psychoeducational assessment battery.

Parent Interviews and Rating Scales The first step is conducting a focused clinical interview with the child’s parent(s) privately. We begin by eliciting a description of the presenting and related problems and then orally administer the Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983). Rather than administering it in its pencil-and-paper format, our oral administration often leads parents to volunteer information and elaborate on items. This format also allows us to ask for characteristics we might not have requested to better understand the nature of the problem (e.g., endorsement of “poor schoolwork” reveals that the child has poor penmanship but understands the material). This approach increases the accuracy of the data collected. Next, we obtain a thorough developmental history (medical, physical, social, and academic) of the referred child and complete a family history form to assess first- and second-degree biological relatives’ functioning in behavioral, emotional, addictive, and educational domains. Finally, we administer a Learning Styles Questionnaire developed in our clinic to assess learning difficulties represented by a list of 50 behavioral referents of the processing variables based on the Horn–Cattell Gf-Gc theory of cognitive processing (Waschbusch, Daleiden, & Drabman, 2000). Although we do not routinely use more traditional rating scales for assessing ADHD (e.g., Conners scales), many appropriate instruments could be used in conjunction with the CBCL, Teacher Report Form, and the Learning Styles Questionnaire. Many behavior rating scales have been designed to assess symptoms of ADHD and other behavior disorders. The most frequently used rating scales for assessing ADHD include the CBCL and its related Teacher Report Form (TRF) (both of which we use), Conners Rating Scales—Revised

323

(CRS-R), ADD-H Comprehensive Teacher Rating Scale (ACTeRS), Eyberg Child Behavior Inventory (ECBI), the Home Situations Questionnaire (HSQ), and the School Situations Questionnaire (SSQ). A discussion of the strengths and weaknesses of each of these questionnaires is not detailed here. However, clinicians using these instruments should familiarize themselves with the psychometric and normative properties of each scale before selecting a rating scale for use. Furthermore, rating scale use should be tailored to meet the needs of the assessment process in such a way as to provide useful information for diagnostic and treatment purposes, rather than administered simply because a rating scale is labeled “an ADHD assessment tool.” For example, if a parent interview yields vague information concerning when and where problem behaviors occur, then using the HSQ as an adjunct to pinpoint specifics may be helpful, particularly because the HSQ was designed to identify specific situations in which the problem behavior occurs (e.g., playing alone, at church, while watching TV, and in the car). In general, selection of rating scales should be determined by the specific information sought, should help define the problem in objective terms, and should assist in clarifying specific target behaviors for intervention.

Teacher Interviews In addition to the TRF, we phone interview the child’s teacher as a measure of reliability for our impressions of the child’s testing behavior, to assess motivation and attention span, gather more information regarding the child’s academic strengths and weaknesses, and evaluate the child’s rate of academic, social, and behavioral progression and strengths. This information is combined with the information from the parents to ascertain whether the child presents with a motivation/discipline problem at home or school, has difficulty with peer relationships, or has a personality conflict with a teacher. Another reason for interviewing the teacher is that children frequently behave differently in school than compared to at home. Interviewing the teacher will assist in assessing ADHD in that the clinician can get

reyn2-14.qxd

6/6/2003

324

9:21 AM

Page 324

IV. SPECIFIC SYNDROMES AND SYMPTOMS

specific information regarding behavioral problems that occur only in the school setting. Information that should be obtained from the teacher includes child behavior in a variety of contexts (e.g., classroom, hallway, lunchroom, and playground), teacher responses to child misbehavior (e.g., reprimand, ignoring, redirection, and detention), peer relationships, and academic performance. Assessing teacher behavior is crucial because the teacher may not handle many ADHD-like behaviors (e.g., off-task behavior) appropriately and thus sustain an otherwise modifiable problem behavior. Interviewing multiple teachers is also helpful, when possible. Junior high school students typically have more than one teacher, and occasionally we assess a child who is exhibiting significant behavior problems in one teacher’s class but not in another’s class. Sometimes this discrepancy is related to child variables (e.g., performs normally in physical education class but has difficulty in math); at other times the discrepancy is best explained by teacher variables (e.g., class structure, discipline style, and personality factors). When discrepant information is explained mostly by teacher variables, an ADHD diagnosis may not be warranted, in spite of the fact that a particular teacher consistently reports the standard ADHDtriad of inattention, hyperactivity, and impulsivity.

Child Interview We also interview the referred child. If the child is old enough, the Youth Self-Report (YSR) form of the CBCL or the Behavior Assessment Scale for Children (BASC) may be administered orally, with similar advantages to those previously noted with the parents. Next, the child is asked to draw a picture of his or her family members and of him- or herself. Then, for each family member, including self, the child is asked to list “three things that the person does that makes you happy, sad, and mad.” Then for the self picture, the question is “three things you do that make you happy, sad, and mad and three things you do that make others happy, sad, and mad.” Finally, once rapport is sufficiently established with the child, we seek the child’s view of the presenting problem.

Psychoeducational Testing The second part of our psychoeducational assessment involves extensive testing of the child using a battery of standardized intellectual and achievement tests (i.e., the Woodcock–Johnson Psycho-Educational Battery—Revised [WJ-R; Woodcock, 1989] and portions of the Wechsler Intelligence Scale for Children—Third Edition [WISCIII; Wechsler, 1991]), as well as additional measures of attention, memory processing, motor and perceptual capabilities. We administer the entire WJ-R battery because it is based on the hierarchical Gf-Gc (fluid and crystallized intelligence) model of cognitive abilities from the Horn–Cattell theory (McGrew & Flanagan, 1998). We believe that this model is the most valid and useful model for conceptualizing cognitive processing in the measurement of intellectual functioning. The main factors that we obtain from testing include comprehension-knowledge, fluid reasoning, visual processing, auditory processing, short-term acquisition and retrieval (visual and auditory), long-term storage and retrieval, and processing speed. The child’s standardized performance on these processing variables illuminates how he or she can best use information in learning situations which may lead to recommendations for changes in curriculum, program design, and presentation of academic tasks and assignments (Neul & Drabman, 1999). Review of the aforementioned information helps determine whether additional testing is required (e.g., anxiety or depression self-report measures) and whether formal behavioral observations and/or functional assessments are needed at home and/or school to clarify conflicting reports or have families demonstrate behavioral processes that they cannot aptly describe. Direct observation of the child’s behavior in a naturalistic setting (e.g., home and school) can provide important additional information and gives the clinician an opportunity to witness behaviors that parents and teachers are reporting. Direct observation of classroom behavior has been found to be a useful tool for accurately identifying behavior disorders in children. Skansgaard and Burns (1998) examined the agreement between teacher ratings and direct observation of 217 children with ADHD, CD, and

reyn2-14.qxd

6/6/2003

9:21 AM

Page 325

14. Assessment of ADHD

ODD. They reported that interrater agreement for the direct observations was higher for all disorders, suggesting that direct observation may play an important role in the proper assessment of ADHD. The accuracy of this type of measurement depends largely on the training and performance of the observers. Therefore, careful monitoring of the data for observer effects (e.g., reactivity, bias, or drift) is necessary. At times a semiformal functional assessment of specific problem behaviors may be needed in order to recommend appropriate interventions. In its simplest form, functional assessment is a type of direct observational assessment that involves observing and recording in sequence events that occur closest in time to the onset and to the ending of target behaviors. These data are then used to generate hypotheses about the possible function(s) or purpose(s) of the child’s behavior. The goal of identifying these functions or purposes is to teach the child an appropriate alternative behavior(s) that will get his or her needs met. For example, two different children may engage in “off-task” behavior while in the classroom but for very different reasons. For one child, off-task behavior may occur during difficult assignments because the child does not understand the material. For another child, off-task behavior may occur because it gets the teacher’s attention or because other children encourage “clowning.” In each of these cases, interventions would necessarily be different in order to increase on-task behavior. Ultimately, the data from these myriad sources are organized, interpreted, and then communicated in the form of an assessment report written at a level for ease of interpretation. This information is also communicated via a feedback session during which the main findings and impressions are reviewed with the parents and child (if mature enough) and recommendations made with specific directions on how to best implement them.

ANCILLARY ASSESSMENTS Up to this point, we have outlined a comprehensive behavioral assessment model that we use to address behavioral difficul-

325

ties, much like those observed in ADHD, of referred children. The phenomenon of ADHD is not entirely understood and its existence as a syndrome is sometimes disputed, as there are no definitive, biological measures available to make a diagnosis (Silver, 1999). Relatedly, many problem behaviors are often misdiagnosed as ADHD. Therefore, the main goal of assessment is to evaluate the referred child and the environments in which the child interacts to determine behavioral excesses and deficits versus determining the existence of the diagnosis, itself. This is achieved in part by conducting a thorough behavioral and psychoeducational assessment as described in the previous section. However, we would like to highlight the importance of evaluating certain child, parent, family, and environmental characteristics that are rarely included as a standard part of child behavioral assessments. We believe that assessing these factors on a routine basis is a crucial part of the assessment process. The factors that are discussed include child temperament, sleep disturbances, chronological age at school entry, school environment, and family issues. The importance of evaluating these factors is based on a combination of our clinical experience and empirical evidence.

Temperament Thomas and Chess (1977) defined temperament as a behavioral style, or the characteristic way that one experiences and responds to internal and external environmental factors. This behavioral style contributes to a child’s development and to her ability to navigate the social environment (McClowry, 1998). Temperament research has demonstrated a link between temperament characteristics and behavioral problems in early-elementary-age children (Garrison & Earls, 1987). For example, a behavior style may not be aberrant but viewed as such due to a “poor fit” between the child’s temperament and a parent’s or teacher’s expectations or own temperaments (Carey, 1998). Therefore, a child who exhibits behaviors associated with ADHD, such as hyperactivity, inattention, and impulsivity may be a child with a high-activity-level temperament that is at odds with parental and/or academic expectations. A lack of a “goodness of

reyn2-14.qxd

6/6/2003

9:21 AM

326

Page 326

IV. SPECIFIC SYNDROMES AND SYMPTOMS

fit” between a child’s innate abilities and environmental conditions thus produces a maintaining condition for problem behavior (Guevremont, DuPaul, & Barkley, 1993, p. 164). An intervention based on educating the parents about environment–temperament fit and adjusting the environmental demands, expectations, and opportunities to better fit the child’s temperament (McClowry, 1998) can significantly improve the child’s behavior without having to make a diagnosis of ADHD.

Sleep Disturbance Sleep disturbances are often associated with developmental disorders, such as mental retardation, learning disabilities, and emotional disorders (Day & Abmayr, 1998). The cardinal behavioral signs of ADHD: inattention, hyperactivity, and impulsivity (American Psychiatric Association, 1994) closely resemble symptoms of sleep deprivation; therefore, a child’s sleep–wakefulness patterns should be assessed (Corkum, Tannock, & Moldofsky, 1998). Previous versions of Diagnostic and Statistical Manual of Mental Disorders (DSM) (e.g., DSM-III; American Psychiatric Association, 1980) included sleep disturbances as a diagnostic criterion for ADHD; however, the current version does not include this criterion which attests to the controversy over whether and in what ways sleep problems are associated with ADHD (Day & Abmayr, 1998). A simple way to assess whether a child is getting enough sleep is to inquire about bedtime. Generally, young children (ages 4–8) should get 8 to 12 hours of sleep per night depending on the individual child’s need (University of Chicago Primary Care Group, 1995). Another issue to assess is the family’s and child’s television viewing habits, specifically whether the child has a television in his or her bedroom. A recent retrospective study revealed that 25% of the parents of kindergarten through fourthgrade children reported that their child had a television in his or her bedroom. Results demonstrated that increased daily television watching and, especially, bedtime watching in one’s bedroom were most significantly associated with sleep disturbances (Owens et al., 1999). Behaviors of hyperactivity and inatten-

tion are also associated with symptoms related to more serious sleep disorders such as sleep-related breathing disorders (e.g., apnea and snoring) and periodic limb movements (PLMs) (Chervin, Dillon, Bassetti, Ganoczy, & Pituch, 1997; J. Janusz, personal communication, September 4, 1999; Picchietti, England, Walters, Willis, & Verrico, 1998). In a group of children diagnosed with ADHD, habitual snoring was more frequently reported and excessive sleepiness and restless legs were relatively less frequently reported compared to non-ADHD psychiatric and general pediatric referrals (Chervin et al., 1997). In a group of children not previously diagnosed with ADHD, severity of hyperactivity and inattention was directly associated with snoring and excessive daytime sleepiness. These results suggest that symptoms of sleep disorders may actually cause inattention and hyperactivity. If such a relationship exists, the data suggest that treatment of snoring and sleeprelated breathing disorders might significantly decrease the prevalence of ADHD in children presenting with sleep-related symptoms. Finally, sleep disturbances, such as difficulty settling down, conflicts about bedtime rituals, and nighttime awakenings often occur and cause significant conflict in parent–child interactions (Day & Abamyr, 1998). These disturbances can contribute to problems the next day in terms of difficulty awakening on time and daytime sleepiness which may cause a child to be inattentive, irritable, hyperactive, and impulsive. Screening for these sleep disturbances in families reporting ADHD-like problems is crucial in determining the etiology of the presenting symptoms and the effectiveness of interventions designed to improve the problems.

Chronological Age Determination of school entry is based primarily on the chronological age of the child (e.g., 6 years old by October 1 of the academic year) (Tarnowski, Anderson, Drabman, & Kelly, 1990). If a child turns 6 years old just before this deadline, this child will be as much as 11 months younger than the oldest students in the same class. In a preliminary

reyn2-14.qxd

6/6/2003

9:21 AM

Page 327

14. Assessment of ADHD

(Drabman, Tarnowski, & Kelly, 1987) and follow-up, investigation by Tarnowski and colleagues (1990) found that younger children were disproportionately referred for psychological services with the youngest children in each class more often referred for academic and behavioral problems. However, no evidence emerged suggesting that the younger children were actually less competent (for their age) than the relatively older children. As a result, these young-forgrade children may be at a greater risk for inappropriate intervention. In a similar study examining young-for-grade children in fifth-grade classrooms in Virginia, LeFever, Dawson, and Morrow (1999) found that young-for-grade children were at an increased risk for medication use across all sex by race groupings. These results indicate that teachers, parents, and professionals may make inappropriate assumptions about young children’s behaviors (i.e., misunderstanding of developmentally appropriate inattention, impulsivity, and hyperactivity) and academic abilities leading to unnecessary referrals and use of stimulant medications.

School Environment As a practitioner, it is important to be familiar with the reputations of the public and private schools located in one’s referral area. Information regarding academic expectations and standards for students can provide a criterion by which a referred child can be compared to peers in terms of academic performance. A student referred for poor academic performance due to ADHDlike behaviors of inattention, hyperactivity, and/or impulsivity who maintains a C average may be viewed by a practitioner as experiencing problems in need of intervention. However, the school the student attends may have high academic standards in which a C average is the best the student can obtain. The presenting ADHD-like behaviors may be due to the child’s struggle with the coursework and anxiety over not being able to do as well as her peers. As mentioned previously, children referred for psychological services are often young-for-grade and are more likely to receive drug prescriptions (LeFever et al.,

327

1999). In the LeFever and colleagues (1999) study, three times as many boys as girls and twice as many Caucasian as African American students received drug therapy for ADHD behaviors. These statistics may be explained by the fact that more Caucasian parents have resources to send their children to private schools, resulting in increased academic pressure for parents, children and schools. However, it is not known whether private schools have proportionally more children taking medications such as Ritalin (methylphenidate). Therefore, it is important to be aware of the academic environment, as well as the referring parents’ academic standards and expectations when examining the child’s behavioral and related academic performance difficulties.

Family Issues The quality of a child’s family relationships and experiences with parents substantially impact the likelihood of clinical referral, severity of presenting symptoms, possibility of comorbidity issues, response to treatment, and prognosis (Woodward, Taylor, & Dowdney, 1998). For instance, children’s problem behaviors are associated with strained family relations among parents, siblings, and the referred child (Kaplan, Crawford, Fisher, & Dewey, 1998). In the assessment and treatment of children’s behavioral problems, problem behaviors must be considered in the context of their function (Erdman, 1998). To assess the function of problem behaviors, family contextual variables must be examined such as the general functioning of family members and as a familial unit, health of the marital relationship, nature of the parent–child attachment, presence of parental psychopathology, level of parenting skills and typical practices, parental attributions regarding their child’s behavior, and child–sibling relationship(s). Assessment of caregiving responsibilities and load may reveal information regarding the general functioning of the family unit. Specifically, assessing the mother’s employment demands, child workload, and support from father can illuminate potential areas for problems or evidence of protective factors (Barkley, 1981; Harvey, 1998). Disordered attachments (i.e., insecure,

reyn2-14.qxd

328

6/6/2003

9:21 AM

Page 328

IV. SPECIFIC SYNDROMES AND SYMPTOMS

avoidant, and/or ambivalent attachment relationships) are associated with disordered parent–child interactions, often leading to child noncompliance and coercive parenting practices (Erdman, 1998). These coercive patterns lead to parental mismanagement of behavior (i.e., parental attention given to a child’s negative behavior, thereby inadvertently reinforcing these negative behaviors) resulting in problem behaviors similar to those seen in children diagnosed with ADHD (Barkley, 1981). Effective behavioral strategies are not possible without considering the context in which these strategies are to be applied. If the parent–child relationship is not understood or addressed in the intervention program, then the problem behaviors may very well continue to exist, thus creating more relationship difficulties. Finally, parental attributions should be assessed because they are important sources of information and can alert the clinician to possible obstacles in designing interventions and ensuring the parents’ “buy into” the potential effectiveness of the interventions. Perhaps as important as the parent–child relationship is the relationship between a child and his or her sibling(s). Eighty percent of children in the United States have siblings (Dunn, 1996). Growing up with a friendly, supportive sibling(s) versus a hostile, antagonistic sibling(s) may have a significant impact on a child’s social and emotional development. Poor sibling interactions (especially aggressive interactions), early behavior problems, and disturbed parent–child relations lead to later disturbed behavior. Therefore, assessment of a referred child’s relationship with his or her sibling(s) is important for understanding the overall functioning of the family unit and particular difficulties within the family system. Other factors to assess are the child’s birth position, age difference between siblings, and the gender of the child and siblings. The magnitude of the age gap should be noted and considered within the context of how a child or parent describes the sibling relationships. For example, if a younger sibling is advancing beyond her older sister on academic tasks, the older sister may resent her younger sister, thus leading her to physically and/or emotionally mistreat her. This sibling relationship problem provides a context in which the younger sister’s behav-

ioral difficulties may be due to anxiety, anger, and/or frustration over her situation rather than being associated with ADHD. Also, the gender of the referred child and siblings should be noted. In early childhood, the influence of gender on sibling relationships is inconsistent. In middle childhood and early adolescence, gender appears to impact sibling relationships, with older sisters being more intimate and affectionate toward their younger siblings compared to older brothers (Buhrmester, 1992; Dunn, Slomkowski, & Beardsall, 1993). In addition to these sibling relationship patterns, the parent–sibling relationships should be assessed in terms of differential treatment of siblings by parents and involvement of parents in sibling conflict. Mothers who differentially treat their children often have children who have difficult, hostile sibling relationships (Dunn, 1996). In sum, assessing these family and sibling variables provides important supplementary information that should be considered when formulating hypotheses and designing formal interventions.

RECOMMENDATIONS ON FINDINGS: TREATMENT APPROACHES Just as a behavioral approach to assessment requires a multimodal approach, treatment and prevention programs for various child behavior difficulties (including ADHD) require a multimodal approach (Barkley, 1998a; Fee & Matson, 1993; Silver, 1999; Singh, Parmelee, Sood, & Katz, 1993). In fact, these multimodal treatment approaches are receiving much support as a “best practice” treatment approach (Johnston & Ohan, 1999). This type of approach is based on a combination of pharmacological, psychological (family parent, and child based), educational, and social skills training strategies (Barkley, 1990, 1998a, 1998b; Garber & Garber, 1998; Pelham, Wheeler, & Chronis, 1998; Schleser, Armstrong, & Allen, 1990). However, limited research data exist to support the efficacy of this multimodal approach (AACAP, 1997). This is due to the time, cost, and complexity associated with participant attrition, breadth and specificity of hypotheses, and large sample sizes needed for such investigations

reyn2-14.qxd

6/6/2003

9:21 AM

Page 329

14. Assessment of ADHD

to adequately assess a multimodal treatment program’s efficacy. Short-term efficacy of medication (e.g., Ritalin) and most types of behavior therapy have been well documented; however, the long-term cost-benefit analyses have yet to be studied for these types of treatment and their combinatory effect (Pelham et al., 1998). Specific to psychopharmacological intervention, efforts are currently underway to examine medication efficacy for new formulations. For example, pharmaceutical companies continue to introduce extended release (ER) forms of existing stimulant medications (e.g., Concerta as the ER form of methylphenidate), as well as develop new, nonstimulant-based medications, such as atomoxetine (ATM), which has recently been shown to be effective in managing symptoms of ADHD in children and adolescents (Michelson et al., 2002). The Multimodal Treatment Study of Children with ADHD (MTA Study; Arnold et al., 1997) and the FAST (Families and Schools Together) Track Program (Conduct Problems Prevention Research Group [CPPRG], 1992) are the most current systematic, long-term, multimodal treatment studies designed to address the foregoing, multimodal treatment issues (Hinshaw, Klein, & Abikoff, 1998). The outcome results are currently being compiled, but the goal of these multimodal programs is to directly target multiple functions over extended periods of time in order to positively influence posttreatment adjustment. Once again, similar to the approach of child behavioral assessment, the design of multimodal treatment programs should be based on a developmental perspective to identify the prime time points for introducing or refurbishing interventions for children, their parents, and their teachers (Johnston & Ohan, 1999). As mentioned at the beginning of this chapter, ADHD is most likely a heterogeneous group of disorders with multiple etiologies. Furthermore, the expression of ADHD is associated with environmental factors (Barkley, 1996, 1998a). As a result, treatment of ADHD and ADHDlike problems should be aimed at management of behavioral difficulties and any associated academic performance problems. As Johnston and Ohan (1999) proposed, treatments for ADHD will be most efficacious when they help the child perform specific be-

329

haviors in the natural environment. This proposal reflects Barkley’s (1998a) delineation of ADHD as a disorder of performance (e.g., “when” and “where” to perform a behavior) rather than a disorder of skill (e.g., “how” and “what” behavior to perform). Behavior management techniques are most useful in developing such a treatment approach. Specifically, the function of the problematic behaviors determined from a thorough, multidmodal assessment approach is altered by manipulating the associated antecedent and consequent events. Specific guidelines for developing the behavior-based and academic performance segments of the treatment program are to design interventions that directly alter the stimulus conditions that control the problematic behavior, as well as the pattern, timing, or reinforcement value of the consequences. Of course, these interventions must be applied across multiple situations within the home and school setting and must be administered for a sufficient length of time to prevent a return to pretreatment levels of symptoms. Most research has focused on improving vigilance and impulse control by applying contingent consequences in the form of reinforcement and punishment (usually response cost). Yet a paucity of research exists on altering the stimuli that may control or produce the problem behaviors. Decreasing the frequency of these problem behaviors should include altering the stimulus properties of the immediate environment and tasks assigned to these children. One method for altering the antecedent conditions of these problem behaviors is to generate hypotheses that will allow changes in the antecedents using the Horn–Cattell Gf-Gc theory of cognitive processing. Because Horn–Cattell theory has been useful in illuminating the cognitive processing capabilities in normally developing children, this theory may also be useful in understanding the cognitive processing of children with ADHD and ADHD-like symptoms (Waschbusch et al., 2000). For example, Barkley, DuPaul, and McMurray (1990) have found that predominantly hyperactive ADHD children have more difficulty with sustained attention and impulse control whereas predominantly inattentive ADHD children have difficulties with fo-

reyn2-14.qxd

330

6/6/2003

9:21 AM

Page 330

IV. SPECIFIC SYNDROMES AND SYMPTOMS

cused attention and information processing speed. Our assessment, based on Horn–Cattell theory, helps us to generate hypotheses about the child’s strengths and weaknesses and therefore assists us in designing interventions to change the appropriate antecedents which may decrease the necessity of using behavioral and/or medical techniques for children displaying ADHD-associated behaviors (Neul & Drabman, 1999). Some examples of altering the antecedent conditions may include increasing task novelty and reducing task difficulty to meet the child’s capability level, repetition of task instructions throughout the task’s duration, direct-instruction-based drills of important academic skills, and frequent shifting of tasks in both the home and school setting (Pfiffner & Barkley, 1998). Other methods for altering the antecedent conditions could include incorporating concrete cues for time limits and rules to maintain on-task behavior, such as allowing the child to use a portable timer on his or her desk or in the home within a visible range and/or use of “reminder” cards to be placed in the child’s work area delineating the rules for on-task behavior, organization, and study cues. These antecedent-based interventions should be paired with a consequence program in which successful use and application of the aforementioned techniques should be rewarded (e.g., with tokens, points, or extra recess time) (Pfiffner & O’Leary, 1993) and unsuccessful use or noncompliance should be punished (e.g., response-cost in the form of token or point loss or loss of play time at home contingent upon poor behavior reports from school) (Anastopoulos, Smith, & Wien, 1998). These specific techniques of the multimodal treatment program can be applied at home as well as at school. Successful management of the referred child’s problems is considerably more probable when contact between home and school exists (Pffifner & O’Leary, 1993). Parental communication with their child’s teacher(s) via notes using the child as a messenger can ensure that the behavior modification program is reliably administered between settings, new behavioral problems can be quickly addressed, and reporting of successes can be communicated to the child both at home and at school. This interaction will help maintain

the application of the interventions in both settings, thereby improving generalization of behavior improvements to other settings and longer-term maintenance of treatment gains. Once the actual program is developed, the first step in implementing a comprehensive treatment program is to assist the child’s parents in understanding that the treatment is multimodal and that ADHD and ADHD-like behaviors are managed rather than cured (Goldstein & Goldstein, 1989). It should be explained to the parents that a variety of treatment techniques will be applied in the management of their child’s behavioral problems, such as parent training in behavior modification principles to be used consistently at home, a schoolbased behavior management program to target classroom-related problem behaviors, and medication (if recommended). Depending on the parents’ view of medication use, the clinician should educate regarding the potential benefits and side effects of stimulant medication, with a specific focus on what they can expect the medication to do (increase on-task time, improve short-term performance) and not do (e.g., improve organization and increase task or good behavior motivation) for their child’s problems (DuPaul, Barkley, & Connor, 1998; Garber & Garber, 1998). With regard to the parent- and school-based treatment components, the clinician should determine the parents’ level of motivation and desire to learn the techniques and then monitor their implementation at home and at school (Barkley, 1988; Goldstein & Goldstein, 1989). The clinician should also assess the motivation and desire of the teacher(s) to implement and monitor the treatment recommendations. It should be explained to the parents and teacher(s) the importance of consistent and immediate application of behavior modification techniques, to expect some initial resistance and/or worsening of symptoms in the child as a reaction to the new environmental contingencies being imposed (e.g., an extinction burst effect) (Barkley, 1981), and the importance of having to adjust the program as needed to meet new behavioral challenges and/or adjust to improvements in behavioral symptoms. Some general principles should be followed to ensure effective implementation of

reyn2-14.qxd

6/6/2003

9:21 AM

Page 331

14. Assessment of ADHD

the treatment program and to increase the likelihood of treatment gains. First, when rewarding or punishing behavior, it should be done immediately and consistently to ensure that the child clearly learns the contingencies and their consequences. Second, a combination of positive reinforcement and punishment (e.g., time out or response cost) should be used. If a problematic behavior is punished to reduce its frequency, it must be replaced with an acceptable, appropriate alternative that can be rewarded to increase its frequency. Third, the overall goal in adjusting the environment as a part of any behavior-based treatment program is to set up the referred child for success. Doing so will increase the likelihood that the child will “buy into” the program creating a higher probability of treatment maintenance success.

SUMMARY AND CONCLUSIONS Current research suggests that ADHD is a heterogeneous group of disorders with multiple etiologies. Furthermore, researchers have noted that current characterizations of ADHD are subjective in nature, and at times contradictory (Goodman & Poillion, 1992). For example, such characteristics as “disorganized” (American Psychiatric Association, 1980; Bacon, 1982; Hunsucker, 1988) and “talks excessively” (American Psychiatric Association, 1987; Hunsucker, 1988; Ingersoll, 1988) are relative terms that may be defined differently by different people. Furthermore, some characteristics that have been used to describe children with ADHD are not observable, such as “accident prone” (Rutter, 1989) and “poor planner” (Kuehne, Kehle, & McMahon, 1987), and must be inferred from behavior, which is a highly subjective process. Furthermore, Goodman and Poillion (1992) concluded that 10% of the ADHD characteristics cited in the literature contradict one another. For instance, “underachievement” (Cohen, Caparulo, & Shaywitz, 1981) is contradictory to “no significant academic difficulties” (Kuehne et al., 1987), yet each is cited as characteristic of ADHD. As another example of this contradiction, Hunsucker (1988) reported that “normal IQ” was a characteristic of ADHD, whereas

331

Rutter (1989) stated that these children were of “below-average intelligence.” Hypothesized causes of ADHD are too numerous to list, and those that have been identified empirically have been based on correlational data (e.g., lead toxicity) (Goodman & Poillion, 1992). Furthermore, there is a general lack of agreement among researchers regarding the proposed etiologies, with approximately 50% of researchers in one study believing ADHD had a genetic cause, 36% believing ADHD was caused perinatally/prenatally, and 28% believing ADHD was the result of neurodevelopmental immaturity (Goodman & Poillion, 1992). Adding to this confusion regarding the etiologies and expression of ADHD is the frequency of the diagnosis. For instance, DSM-IV suggests that approximately 5% of children have ADHD, yet teachers believe that 25% of their students exhibit the clusters of behaviors that are associated with ADHD (Pelham, Gnagy, Greenslade, & Milich, 1992). Of course, teachers (via parents) make most of the referrals for treatment of ADHD. If DSM-IV statistics are correct, then five times as many referrals of individuals who should be diagnosed may be made. Because of the uncertainty of the nature of ADHD, we believe the diagnosis has limited usefulness at this time. Diagnosing a child as “ADHD” does not provide useful information regarding the most effective treatments given the heterogeneous characteristics and proposed etiologies of the disorder. More important than determining a “diagnosis” for a child is identifying the child’s relative strengths and weaknesses (both behaviorally and cognitively) that can be used to develop effective interventions. We believe that determining the functions of specific problem behaviors, as well as their antecedents and consequences, is currently the most effective strategy for improving child problem behaviors, regardless of the child’s diagnosis. Furthermore, because many children who exhibit ADHD-like behaviors experience academic difficulties it is necessary to evaluate the child’s cognitive processing. As mentioned earlier, we believe that Horn–Cattell theory is currently the most valid and useful model for conceptualizing cognitive abilities. We have found that the assessment of the factors associated with

reyn2-14.qxd

6/6/2003

332

9:21 AM

Page 332

IV. SPECIFIC SYNDROMES AND SYMPTOMS

Horn–Cattell theory has resulted in useful information regarding the child’s cognitive processing capabilities in a variety of domains that leads to specific recommendations. Children exhibiting the ADHD-like cluster of behaviors have varied problem behaviors and academic difficulties, as well as varied cognitive abilities. As such, assessment of these children must involve technologies that are likely to identify all combinations of the myriad problems these children face. We conclude this chapter with an analogy that helps clarify our beliefs regarding ADHD. A rash on one’s arm may be caused by internal (biological/genetic) or external (environmental) factors, or a combination of both. Discovering the rash’s cause can be essential to its cure. The same is true for the cluster of behaviors labeled “ADHD.” The cause may be internal, external, or both. Ideally, advances in brain imaging as well as psychological investigations using neuropsychological and Horn–Cattell theory will lead us to a better understanding of the variety of causes of this cluster of behaviors. Perhaps this understanding will lead us to more specific and therefore more effective treatment.

REFERENCES Achenbach, T. M., & Edelbrock, C. S. (1983). Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington: University of Vermont, Department of Psychiatry. American Academy of Child and Adolescent Psychiatry. (1997). Practice parameters for the assessment and treatment of children, adolescents, and adults with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 36(10)(Suppl.), 85–121. American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Anastopoulos, A. D., Smith, J. M., & Wien, E. E. (1998). Counseling and training parents. In R. A. Barkley (Ed.), Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed., pp. 373–393). New York: Guilford Press. Arnold, L. E., Abikoff, H. B., Cantwell, D. P., Conner, C. K., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H.

C., March, J. S., Newcom, J. H., Pelham, W. E., Richters, J. E., Schiller, E., Severe, J. B., Swanson, J. M., Vereen, D., & Wells, K. C. (1997). NIMH collaborative multimodal treatment study of children with ADHD (MTA): Design, methodology, and protocol evolution. Journal of Attention Disorders, 2(3), 141–158. Bacon, E. H. (1982). Assessment of children with attention problems. Paper presented at the Council for Exceptional Children’s Convention, Houston, TX. Barkley, R. A. (1981). Hyperactive children: A handbook for diagnosis and treatment. New York: Guilford Press. Barkley, R. A. (1988). Attention deficit hyperactivity disorder with hyperactivity. In E. J. Mash & L. G. Terdal (Eds.), Behavioral assessment of childhood disorders (2nd ed., pp. 69–104). New York: Guilford Press. Barkley, R. A. (1990). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment. New York: Guilford Press. Barkley, R. A. (1996). Attention deficit/hyperactivity disorder. In E. J. Mash & R. A Barkley (Eds.), Child psychopathology (pp. 63–112). New York: Guilford Press. Barkley, R. A. (1998a). Attention-deficit hyperactivity disorder. New York: Guilford Press. Barkley, R. A. (1998b). Attention-deficit/hyperactivity disorder. In E. J. Mash & R. A. Barkley (Eds.), Treatment of childhood disorders (2nd ed., pp. 55–110). New York: Guilford Press. Barkley, R. A., DuPaul, G. J., & McMurray, M. B. (1990). A comprehensive evaluation of attention deficit disorder with and without hyperactivity. Journal of Consulting and Clinical Psychology, 58, 775–789. Barkley, R. A., & Edwards, G. (1998). Diagnostic interview, behavior rating scales, and the medical examination. In R. A. Barkley, Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed., pp. 263–293). New York: Guilford Press. Biederman, J., Newcorn, J., & Sprich, S. (1991). Comorbidity of attention deficit hyperactivity disorder with conduct, depressive, anxiety, and other disorders. American Journal of Psychiatry, 148, 564– 577. Buhrmester, D. (1992). The developmental courses of sibling and peer relationships. In F. Boer & J. Dunn (Eds.), Children’s sibling relationships: Developmental and clinical issues (pp. 19–40). Hillsdale, NJ: Erlbaum. Carey, W. B. (1998). Temperament and behavior problems in the classroom. School Psychology Review, 27(4), 522–533. Chervin, R. D., Dillon, J. E., Bassetti, C., Ganoczy, D. A., & Pituch, K. J. (1997). Symptoms of sleep disorders, inattention, and hyperactivity in children. Sleep, 20(12), 1185–1192. Cohen, D. J., Caparulo, B. K., & Shaywitz, B. A. (1981). Neuropsychiatric disorders of childhood. Paper presented at the Research Forum on Children and Youth, Washington, DC. Coles, G. (1987). The learning mystique: A critical

reyn2-14.qxd

6/6/2003

9:21 AM

Page 333

14. Assessment of ADHD look at learning disabilities. New York: Pantheon Books. Conduct Problems Prevention Research Group. (1992). A developmental and clinical model for the prevention of conduct disorder: The FAST Track Program. Development and Psychopathology, 4(4), 509–527. Corkum, P., Tannock, R., & Modolfsky, H. (1998). Sleep disturbances in children with attentiondeficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 37(6), 637–646. Day, H. D., & Abmayr, S. B. (1998). Parent reports of sleep disturbances in stimulant-medicated children with attention-deficit hyperactivity disorder. Journal of Clinical Psychology, 54(5), 701–716. Drabman, R. S., Tarnowski, K. J., & Kelly, P. A. (1987). Are younger classroom children disproportionately referred for childhood academic and behavior problems? Journal of Consulting and Clinical Psychology, 55(6), 907–909. Dunn, J. (1996). Siblings: The first society. In N. Vanzetti & S. Duck (Eds.), A lifetime of relationships (pp. 105–124). Pacific Grove, CA: Brooks/Cole. Dunn, J., Slomkowski, C., & Beardsall, L. (1993). Sibling relationships from the preschool period through middle childhood and early adolescence. Developmental Psychology, 30, 265–284. DuPaul, G. J., Barkley, R. A., & Connor, D. F. (1998). Stimulants. In R. A. Barkley, Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed., pp. 510–551). New York: Guilford Press. Erdman, P. (1998). Conceptualizing ADHD as a contextual response to parental attachment. American Journal of Family Therapy, 26, 177–185. Fee, V. E., & Matson, J. L. (1993). Past developments and future trends. In J. L. Matson (Ed.), Handbook of hyperactivity in children (pp. 323–342). Needham Heights, MA: Allyn & Bacon. Fristad, M. A., Weller, R. A., & Weller, E. B. (1995). The mania rating scale (MRS): further reliability and validity studies with chidren. Annals of Clinical Psychiatry, 7, 127–132. Garber, M. D., & Garber, S. W. (1998). Beyond Ritalin: A multimodal approach to assessment and treatment of attention-deficit/hyperactivity disorder. In L. VandeCreek & S. Knapp (Eds.), Innovations in clinical practice: A source book (Vol. 16, pp. 109–126). Sarasota, FL: Professional Resource Press/Professional Research Exchange. Garrison, W. T., & Earls, F. J. (1987). Temperament and child psychopathology. In A. E. Kazdin (Series Ed.), Developmental clinical psychology and psychiatry: Vol. 12. Pittsburgh, PA: Sage. Goldstein, S., & Goldstein, M. (1989). Managing attention disorders in children. New York: Wiley. Goodman, G., & Poillion, M. J. (1992). ADD: Acronym for any dysfunction or difficulty. The Journal of Special Education, 26, 37–56. Guevremont, D. C., DuPaul, G. J., & Bakley, R. A. (1993). Behavioral assessment of attention deficit hyperactivity disorder. In J. L. Matson (Ed.), Handbook of hyperactivity in children (pp. 150–168).

333

Needham Heights, MA: Allyn & Bacon. Harvey, E. (1998). Parental employment and conduct problems among children with attention-deficit/hyperactivity disorder: An examination of child care workload and parenting well-being as mediating variables. Journal of Social and Clinical Psychology, 17(4), 476–490. Hinshaw, S. P., Klein, R. G., & Abikoff, H. (1998). Childhood attention deficit hyperactivity disorder: Nonpharmacological and combination treatments. In P. E. Nathan & J. M. Gorman (Eds.), A guide to treatments that work (pp. 26–41). New York: Oxford University Press. Hunsucker, G. (1988). ADD: Attention deficit disorder. Abilene, TX: Forrest. Ingersoll, B. (1988). Your hyperactive child: A parent’s guide to coping with attention deficit disorder. New York: Doubleday. Johnston, C., & Ohan, J. L. (1999). Externalizing disorders. In W. K. Silverman & T. H. Ollendick (Eds.), Developmental issues in the clinical treatment of children (pp. 279–294). Needham Heights, MA: Allyn & Bacon. Kaplan, B. J., Crawford, S. G., Fisher, G. C., & Dewey, D. M. (1998). Family dysfunction is more strongly associated with ADHD than with general school problems. Journal of Attention Disorders, 2(4), 209–216. Kuehne, C., Kehle, T. J., & McMahon, W. (1987). Differences between children with attention deficit disorder, children with specific learning disabilities, and normal children. Journal of School Psychology, 25, 161–166. Landman, G. B., & McCrindle, B. (1986). Pediatric management of nonpervasively “hyperactive” children. Clinical Pediatrics, 25, 600–604. LeFever, G. B., Dawson, K. V., & Morrow, A. L. (1999). The extent of drug therapy for attention deficit-hyperactivity disorder among children in public schools. American Journal of Public Health, 89(9), 1359–1364. Mash, E. J., & Terdal, L. G. (1988). Behavioral assessment of child and family disturbance. In E. J. Mash & L. G. Terdal (Eds.), Behavioral assessment of childhood disorders (2nd ed., pp. 3–68). New York: Guilford Press. McClowry, S. G. (1998). The science and art of using temperament as the basis for intervention. School Psychology Review, 27(4), 551–563. McGrew, K. S., & Flanagan, D. P. (1998). The intelligence test desk reference (ITDR): Gf-Gc cross-battery assessment. Needham Heights, MA: Allyn & Bacon. Michelson, D., Allen, A. J., Busner, J., Casat, C., Dunn, D., Kratochvil, C., Newcorn, J., Sallee, F. R., Sangal, R. B., Saylor, K., West, S., Kelsey, D., Wernicke, J., Trapp, N. J., & Harder, D. (2002). Once-daily atomoxetine treatment for children and adolescents with attention deficit hyperactivity disorder: A randomized, placebo-controlled study. American Journal of Psychiatry, 159(11), 1896–1901. Neul, S. K. T., & Drabman, R. S. (1999). Direct instruction and Gf-Gc model of cognitive abilities: Implications for improved educational outcomes. Ef-

reyn2-14.qxd

334

6/6/2003

9:21 AM

Page 334

IV. SPECIFIC SYNDROMES AND SYMPTOMS

fective School Practices, 18(2), 50–57. Newcomb, K. P., & Drabman, R. S. (1995). Child behavioral assessment in the psychiatric setting. In R. T. Ammerman & M. Hersen (Eds.), Handbook of child behavior therapy in the psychiatric setting (pp. 3–25). New York: Wiley. Newcorn, J. H., & Halperin, J. M. (1994). Comorbidity among disruptive behavior disorders: Impact on severity, impairment and response to treatment. Child and Adolescent Psychiatric Clinics of North America, 3, 227–252. Owens, J., Maxim, R., McGuinn, M., Nobile, C., Msall, M., & Alario, A. (1999). Television-viewing habits and sleep disturbance in school children. Pediatrics, 104(3), e27. Palmeri, S. (1996). Comment in: Journal of Developmental and Behavioral Pediatrics. Journal of Developmental and Behavioral Pediatrics, 17(4), 253–254. Pelham, W. E., Gnagy, E. M., Greenslade, K. E., & Milich, R. (1992). Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 210–218. Pelham, W. E., Jr., Wheeler, T., & Chronis, A. (1998). Empirically supported psychosocial treatments for attention deficit hyperactivity disorder. Journal of Clinical Child Psychology, 27(2), 190–205. Pfiffner, L. J., & Barkley, R. A. (1998). Treatment of ADHD in school settings. In R. A. Barkley, Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed., pp. 458–490). New York: Guilford Press. Pfiffner, L. J., & O’Leary, S. G. (1993). School-based psychological treatments. In J. L. Matson (Ed.), Handbook of hyperactivity in children (pp. 234–255). Needham Heights, MA: Allyn & Bacon. Picchietti, D. L., England, S. J., Walters, A. S., Willis, K., & Verrico, T. (1998). Periodic limb movement disorder and restless legs syndrome in children with attention-deficit hyperactivity disorder. Journal of

Child Neurology, 13(12), 588–594. Rutter, M. (1989). Attention deficit disorder/hyperkinetic syndrome: conceptual and research issues regarding diagnosis and classification. In T. Sagvolden & T. Archer (Eds.), Attention deficit disorder: clinical and basic research (pp. 1–24). Hillsdale, NJ: Erlbaum. Schleser, R., Armstrong, K. J., & Allen, J. S., Jr. (1990). Attention deficit hyperactivity disorder: New directions. In S. B. Morgan & T. M. Okwumabua (Eds.), Child and adolescent disorders: Developmental and health psychology perspectives (pp. 105–133). Hillsdale, NJ: Erlbaum. Silver, L. B. (1999). Attention-deficit/hyperactivity disorder: A clinical guide to diagnosis and treatment for health and mental health professionals (2nd ed.). Washington, DC: American Psychiatric Press. Singh, N. N., Parmelee, D. X., Sood, A. A., & Katz, R. C. (1993). Collaboration of disciplines. In J. L. Matson (Ed.), Handbook of hyperactivity in children (pp. 305–322). Needham Heights, MA: Allyn & Bacon. Skansgaard, E. P., & Burns, G. L. (1998). Comparison of DSM-IV ADHD combined and predominantly inattentive types: correspondence between teacher ratings and direct observations of inattentive, hyperactivity/impulsivity, slow cognitive tempo, oppositional defiant, and overt conduct disorder symptoms. Child and Family Behavior Therapy, 20, 1–14. Tarnowski, K. J., Anderson, D. F., Drabman, R. S., & Kelly, P. A. (1990). Disproportionate referrals for child academic/behavioral problems: Replication and extension. Journal of Consulting and Clinical Psychology, 58(2), 240–243. Taylor, E. (1998). Clinical foundations of hyperactivity research. Behavioural Brain Research, 94, 11–24. Thomas, A., & Chess, S. (1977). Temperament and development. New York: Brunner/Mazel. University of Chicago Primary Care Group. (1995). Getting a good night’s sleep [Online]. Available: http://uhs.bsd.uchicago.edu/uhs/infoline/sleep.htm1 Waschbusch, D. A., Daleiden, E., & Drabman, R. S. (2000). Are parents accurate reporters of their child’s cognitive abilities? Journal of Psychopathology and Behavioral Assessment, 22(1), 61–77. Wechsler, D. (1991). Manual for the Wechsler Intelligence Scale for Children (3rd ed.). New York: Psychological Corporation. Woodcock, R. W. (1989). Woodcock–Johnson Psycho-Educational Battery—Revised. Allen, TX: DLM Teaching Resources. Woodward, L., Taylor, E., & Dowdney, L. (1998). The parenting and family functioning of children with hyperactivity. Journal of Child Psychology and Psychiatry, 39(2), 161–169.

reyn2-15.qxd

6/6/2003

9:22 AM

Page 335

PART V

OBJECTIVE METHODS

reyn2-15.qxd

6/6/2003

9:22 AM

Page 336

This page intentionally left blank

reyn2-15.qxd

6/6/2003

9:22 AM

Page 337

15 Multisource and Multidimensional Objective Assessment of Adjustment: The Personality Inventory for Children, Second Edition; Personality Inventory for Youth; and Student Behavior Survey

DAVID LACHAR CHRISTIAN P. GRUBER

havior; collects observations from parents, teachers, and youth; and provides standard scores based on contemporary national samples. These measures support assessment that is then both multidimensional and multisource.

AN INTRODUCTION TO MULTIDIMENSIONAL AND MULTISOURCE ASSESSMENT With the publication of the teacher rating form Student Behavior Survey in 2000, the Personality Inventory for Children joined three other families of objective measures that provide separate measures completed by parent, teacher, and child (Lachar, 1998). This chapter introduces this family of measures used in the evaluation of school-age children: the Personality Inventory for Children, Second Edition (PIC-2); the Personality Inventory for Youth (PIY); and the Student Behavior Survey (SBS). These measures serve a function similar to that of two other groups of measures described in subsequent chapters of this volume and developed by Achenbach and by Reynolds and Kamphaus and to the measures comprising the revised Conners Rating Scales (Conners, 1997). Each of these four families of measures assesses multiple dimensions of problem be-

Multidimensional Assessment Multidimensional assessment refers to the practice of using a single instrument to conduct an assessment that evaluates a wide variety of child behavioral and adjustment characteristics. When it is well designed and well implemented, a multidimensional assessment can be seen to be both efficient and accurate. Important clinical phenomena measured using the same format and dimensions of adjustment are quantified using the same or a similar standardization sample. This approach contrasts with the one taken before these coordinated multidimensional instruments were available, one in which objective assessment applied several inde337

reyn2-15.qxd

338

6/6/2003

9:22 AM

Page 338

V. OBJECTIVE METHODS

pendent short scales that each measured one or a few limited dimensions, involved scale items that used different response characteristics, and used different normative samples. The results from these various scales were subsequently integrated into the evaluation of an individual student. Adoption of multidimensional assessment assumes the equal importance of symptom or problem presence and absence. This approach to assessment thus recognizes that a pattern of significant clinical problems often occurs in the same child. These problem constellations or pattern of diagnoses are often described as “comorbid.” For example, problem dimensions of anxiety and depression are often comorbid (Lonigan, Carey, & Finch, 1994), as are a variety of disruptive behavior problem dimensions (August, Realmuto, MacDonald, Nugent, & Crosby, 1996; Vaughn, Riccio, Hynd, & Hall, 1997). In each case, diagnosis depends on the ability to recognize the co-occurring problems and rule out competing hypotheses involving other influences (family problems, cognitive deficits, other psychopathology, etc.). Routine initial application of a multidimensional assessment measure also acknowledges the possibility that children referred for an evaluation for one reason often experience other problems. For example, application of a narrowly focused measure of attention-deficit/hyperactivity in the evaluation of a child hypothesized to exhibit these symptoms may prove problematic for two reasons. If clinically elevated scale scores are not obtained on such a measure, other explanations for observed inattention will not have been similarly investigated (i.e., alternative etiologies such as depression, anxiety, situational adjustment, learning disability, or acquired cognitive disability). In addition, application of such a narrowly focused measure may result in one or more clinically significant scale scores that provide no support for either the presence or absence of frequently comorbid conditions. It is therefore logical and efficient to apply a multidimensional measure of adjustment from the beginning and then subsequently focus on further differentiation of problem areas highlighted in the initial assessment effort. Such a “successive hurdles” approach recognizes the value of initial psy-

chometric information in the design of subsequent evaluative effort.

Multisource Assessment Multisource assessment refers to the practice of asking different people with different roles and perspectives to evaluate their impressions of a single child. Multisource assessment has recently become the preferred model for the evaluation of child and adolescent emotional and behavioral adjustment. Unlike the evaluation of adjustment in adults where self-report is generally deemed adequate, evaluations of school-age children have generally relied on the reports of adult caretakers—notably parents or teachers. Indeed, the context of assessment is fundamentally different for children and adolescents who are unlikely to refer themselves for evaluation or treatment and may not possess the academic, cognitive, or motivational status to complete a comprehensive self-report instrument. Young elementary school children, perhaps students in kindergarten through third grade, may be unable to describe themselves adequately through response to questionnaire statements. These children are unlikely to have mastered the range of vocabulary necessary to adequately describe dimensions of adjustment; such language competence is not usually attained before fourth or fifth grade. Another consideration is that youth are most often referred for evaluation because they are either noncompliant with the requests of the significant adults in their lives or exhibit problems in academic achievement, often presenting with inadequate reading skills. It is therefore not unusual that completion of a self-report inventory of several hundred items could present quite an assessment challenge even for a high school student (cf. Archer, Tirrell, & Elkins, 2001). In contrast, parents and teachers not only refer youth for assessment but are also the primary sources of useful systematic observation. Certainly adults are the most direct informants who can report the noncompliance of a child to their own requests. Parents are the only consistently available source for report of early childhood development and adult description of child behavior in the home. Teachers, on the other hand, offer the most accurate observations of the age

reyn2-15.qxd

6/6/2003

9:22 AM

Page 339

15. Multisource and Multidimensional Assessment

appropriateness of a child’s adjustment in the classroom and academic achievement, as well as the attentional, motivational, and social phenomena unique to the classroom and to the school. It is likely, however, that such observational accuracy decreases after the elementary school grades, as middle school and high school teachers in regular education classrooms have little continuous observation of students (usually only 45 minutes a day along with 30 other students). Despite the potential problems with youth self-description (Greenbaum, Dedrick, Prange, & Friedman, 1994; Jensen et al., 1996), this source of information still represents the most direct and accurate expression of personal thoughts and feelings. Note, for example, that Michael and Merrell (1998) recently demonstrated adequate short-term temporal stability for the selfreport of third- to fifth-grade students. At a minimum, child response documents the current characteristics and problems a child is willing to acknowledge—a useful component of a comprehensive evaluation. One of the provisos here, however, is that potential threats to response validity be identified, a topic that is discussed at greater length later in this chapter.

Issues in Using Multisource Assessments The availability of two or three independent sets of child descriptions provides a natural opportunity for comparison across informants. Achenbach, McConaughy, and Howell (1987) conducted a comprehensive literature review and found very limited concordance in general between the report of parent, teacher, and youth, although relatively greater agreement between sources was obtained for scales representing externalizing behaviors. A review of similar studies that evaluated the responses to objective interviews from parent and child concluded that greater agreement was obtained for visible behaviors and when children were older (Lachar & Gruber, 1993). Degree of agreement between these informants and evaluation of comparative utility continues to be explored (Duhig, Renk, Epstein, & Phares, 2000; Youngstrom, Loeber, & StouthamerLoeber, 2000). Although one reasonable approach to the interpretation of discrepancies between par-

339

ent, teacher, and youth is to assign such differences to situation-specific variation (e.g., the child is only oppositional at home, not in the classroom), other explanations are equally plausible. Cross-informant variance may reflect the reality that scales with similar or identical names may incorporate significantly different item content. On the other hand, the development and application of valid strictly content-parallel measures may limit instrument validity. Such use of only parallel item content in similarly named scales for different informants may restrict the diagnostic potential of each informant source by excluding the measurement of attributes that may be uniquely obtained from only one informant source. Such attributes might be measured through a parent-completed measure of developmental delay, a teacher-completed measure of classroom behavior, or a youth-completed measure of self-concept. The PIC-2, PIY, and SBS attempt in their structure and content to provide the opportunity both to compare similar scale content across informants as well as to measure phenomena that may be uniquely obtained from only one informant. Along with dissimilar scale content, another cause of poor across-informant agreement is the substantial effect of response sets on the accuracy of such information. The child or adolescent being assessed may not adequately comply with questionnaire instructions, due to either inadequate language or reading skills or lack of adequate motivation for the task. It is equally likely that a youth may not want to endorse a personal history of maladaptive behavior and current internal discomfort, although a negative presentation of parent adjustment and home conflict may be readily provided. At times, youth may also be motivated in an assessment context to admit to problems and symptoms that are not present. This same variety of motivations and conditions may also influence parent report. Indeed, there has been some concern that poor parent adjustment may compromise the validity of parent report (Achenbach, 1981). Subsequent review (Richters, 1992) and specific analysis of this issue with the Personality Inventory for Children (Lachar, Kline, & Gdowski, 1987), however, have found no empirical support for this consideration.

reyn2-15.qxd

340

6/6/2003

9:22 AM

Page 340

V. OBJECTIVE METHODS

The PIC-2 and the PIY incorporate validity scales to identify response sets. These scales are designed to measure random or inadequate response to scale statements, defensive denial of existing problems, and admission of symptoms that are unlikely to be present or exaggeration of actual problems (cf. Wrobel et al., 1999). Psychologists may interpret inconsistencies across informants in a variety of ways. At one extreme any evidence of symptom presence might be accepted from any informant source. At the other extreme, problems, symptoms, or characteristics may be recognized as present only when they are documented by at least two, or all three, informant sources. Though an optimal approach to the interpretation of multisource protocols has not been established (although the opinions of mental health professionals and parents have been studied; see Loeber, Green, & Lahey, 1990; Phares, 1997), there is a pragmatic advantage to an assessment system that includes separate measures standardized on parents, teachers, and youth. Conditions regularly occur in the conduct of psychological evaluations that make it difficult or impossible to obtain a parent, teacher, or youth report. Children may be too young, uncooperative, or language impaired. The evaluation may occur during the summer vacation, or the youth may have not consistently attended one classroom, or may have left school permanently. Parents may fail family appointments when their child is hospitalized, or children may be under agency guardianship. In such instances, a set of comprehensive parent, teacher, and self-report measures that can be applied independently of each other provide the flexibility to facilitate data collection. Common sense should be applied in the selection of assessment instruments, or the sequence in which they are collected and interpreted: First call on the source of referral to describe the child’s adjustment. If a parent or teacher observes problems and requests diagnostic or therapeutic services, the PIC-2 or SBS should be completed and interpreted. On the other hand, if a student approaches a school counselor regarding personal problems, a completed PIY will assist the counselor in determining the nature and seriousness of the student’s concerns

and in developing a plan for referral or intervention. In addition, teacher observation is often required to establish the presence of certain conditions, such as attention deficit. Also, any concern regarding disruptive behavior should be assessed to determine whether these problems generalize across home and classroom (cf. Power et al., 1998). Common sense should also guide the application of these measures in treatment planning and treatment evaluation. A diagnostic profile should be obtained from each proposed participant in the treatment process. When parent intervention is proposed, all relevant parent figures should be independently assessed (such as mother and stepfather or mother and grandmother) to determine the degree to which parents agree regarding the presence and severity of specific problems. In addition, child report of problems is central to a variety of interventions. If a child does not acknowledge the presence of the problems that motivated a parent to seek help, the resolution of this discrepancy will become a priority.

PERSONALITY INVENTORY FOR CHILDREN, SECOND EDITION Older Editions and the Historical Context The PIC in all its forms is a parent-completed questionnaire that has been used for more than 40 years. The initial administration booklet was published in 1958 and consisted of 600 statements—a number unthinkable in current practice, and indeed it was gradually reduced to 280 as administration options were developed over the years. These statements were written for 11 content areas (aggression, anxiety, asocial behavior, excitement, family relations, intellectual development, physical development, reality distortion, social skills, somatic concern, and withdrawal). These statements varied in format. Some items described historical fact; others reported the observations of others; still other items involved the direct report of parent impression or response. These direct statements describe behaviors and emotional states. Beyond the problems with the length of this early version, completion of the PIC administration

reyn2-15.qxd

6/6/2003

9:22 AM

Page 341

15. Multisource and Multidimensional Assessment

booklet was found to require a sixth- to seventh-grade reading level (Harrington & Follett, 1984). The first commercial version of the PIC, published in 1977, generated a profile of Tscores with 3 validity scales, 1 general screening scale, and 12 measures of child ability and adjustment and family function. These 16 scales were developed through a variety of empirical item selection techniques or through iterative content valid procedures (Wirt, Lachar, Klinedinst, & Seat, 1977). From the beginning, PIC interpretation placed little importance on the manifest content of endorsed inventory statements (except for the construction of a Critical Items set). Instead, an integrated program of research established external correlates and interpretive guidelines for individual profile scales (Lachar & Gdowski, 1979; Lachar, Gdowski, & Snyder, 1984) and replicated profile patterns (Gdowski, Lachar, & Kline, 1985; Kline, Lachar, & Gdowski, 1987; Lachar, Kline, Green, & Gruber, 1996; LaCombe, Kline, Lachar, Butkus, & Hillman, 1991). A profile interpretive procedure in which similarity coefficients are calculated between the individual profile to be interpreted and the mean profiles of students receiving specific special education services has also been developed (Kline, Lachar, Gruber, & Boersma, 1994). Special effort has also focused on demonstrating that PIC scale validity was not restricted by a child’s age, gender, or ethnicity status (Kline & Lachar, 1992; Kline, Lachar, & Sprague, 1985). This development of empirically derived interpretive guidelines for profile scales continues in the PIC-2, PIY, and SBS. The current manuals for these parent-, teacher-, and self-report measures indicate T-score ranges in which scales should be interpreted, provide interpretive hypotheses, and present a series of case studies in which these guidelines are applied. The 600-item version of the PIC was revised in 1982. The revised PIC (PIC-R; Lachar, 1982) applied a three-part, 420item booklet. The first 131-item part included one validity scale and four broad-band factor-derived scales (Lachar, Gdowski, & Snyder, 1982). The additional completion of the second part of the booklet (280 items) generated a full profile of 16 shortened

341

scales, while completion of all 420 items provided scores to the original full-length scales. A comprehensive review of the PIC and PIC-R 1977–1993 is available (Lachar & Kline, 1994) and a bibliography of over 350 relevant publications is presented in the 2001 PIC-2 manual.

PIC-2 Standard Format At the end of a 7-year development process, the PIC-2 was released in 2001. It represented the first complete revision of the PIC, involving changes to both the language of the test statements and the construction of the measures of response validity and the multidimensional content scales of the instrument. Completion of the full PIC-2 administration booklet now involves just 275 statements. The statements, substantially based on the original PIC statements, but modified to improve intelligibility, content coverage, and sensitivity to current social and family contexts, have been evaluated at a high third- to low fourth-grade level of readability. Scoring of results provides a profile of three validity scales, nine adjustment scales, and 21 adjustment subscales (Lachar & Gruber, 2001). The Standard Format profile provides gender-specific T-scores based on a representative normative sample of schoolage children, kindergarten through 12th grade (n = 2,306). This profile is quite similar to that of the PIY, facilitating comparison. The Standard Format also allows derivation of the PIC-2 Behavioral Summary, described later. During PIC-2 development and initial validation, an additional 1,551 protocols were collected on referred children. These cases often included PIY and SBS profiles, clinician ratings and assigned DSM diagnoses, and performance on psychometric instruments. A preschool version of the PIC is currently under development. The nine adjustment scales were constructed using an iterative process. Initial scale composition was based on either previous PIC or PIY item placement or substantive item content. Item-to-scale correlation matrices generated from a sample of 950 referred protocols were then inspected to establish the accuracy of these initial item placements. Each inventory statement retained on a final adjustment scale demon-

reyn2-15.qxd

342

6/6/2003

9:22 AM

Page 342

V. OBJECTIVE METHODS

strated a significant and substantial correlation to the scale on which it was placed. When an item obtained a significant correlation to more than one clinical dimension, it was placed in almost all cases on the dimension to which it obtained the largest correlation. In this manner, 94% of the 264 PIC-2 statements that comprise these nine scales are placed on only one scale. The 16 items that were placed on two of these final scales obtained substantial correlations to both of these dimensions and presented substantive content consistent with both scale dimensions. For example, “Others often say that my child is moody.” has been placed on DIS2: Depression (item-to-subscale correlation was .63) and DLQ2: Dyscontrol (comparable r = .61), as “moody” can signify both dysphoria and anger. The relatively scale-specific item composition of the nine PIC-2 adjustment scales is in contrast to the previous PIC-R structure. For example, in the PIC-R, 68% of Anxiety scale items also appeared on the Depression scale. In addition, considerable between-scale overlap occurred among the PIC-R cognitive triad scales: Achievement (56%), Intellectual Screening (37%), and Development (84%). Table 15.1 describes the PIC-2 adjustment scales and subscales. These scales average 31 items in length (range of 19–47 items) and obtain a median coefficient alpha of .89 (range of .81–.95). In a referred sample in which the PIC-2 was repeated 1 week after its first administration, these nine scales obtained a median reliability estimate of .90 (range of .88–.94). Similarity of PIC2 to the PIC-R and PIY clinical scales was measured by percent item overlap as well as correlation between PIC-2 and comparable PIC and PIY scales. PIC-2 scales on average obtain 66% overlap with PIC-R scales (range of 33–96%) and obtain a substantial median correlation of .94 (range of .81–.99) with the PIC-R equivalent. As would be expected, PIC-2 adjustment scales obtain substantial item overlap with PIY scales similarly named (average 79%, range of 51–100%). In spite of this substantial scale similarity, the difference in informants (parent to youth) resulted in only moderate concordance estimates (median correlation = .43, range of .28–.53). A major new element of the second edition is that the items of each adjustment scale are

partitioned into two or three subscales. Application of principal component factor analysis with varimax rotation guided the identification of two or three relatively homogeneous item subsets within each adjustment scale. PIC-2 subscales average 13 items in length (range of 6–21 items), with only three of 21 subscales incorporating less than 10 items. Table 15.1 provides coefficient alpha and temporal stability values for these subscales and lists two representative statements from each subscale. The majority of subscales demonstrate psychometric characteristics comparable to scales on shorter published questionnaires. In all instances the division of scales into subscales facilitates the interpretation process. For example, the actuarial interpretation of the PIC-R Delinquency scale (Lachar & Gdowski, 1979) identified T-score ranges associated with dimensions noncompliance, poorly controlled anger, and antisocial behaviors. These dimensions are each represented in PIC-2 DLQ subscales; their patterns of elevation represent the dominant endorsed content of this adjustment scale. (Note that there are comparable subscales on the PIY Delinquency scale.) The comprehensive PIC-2 manual provides empirically based interpretations for adjustment scales and subscales. Correlations between PIC-2 scale scores and clinician, teacher, and youth descriptions readily formed the basis of interpretive guidelines for the nine fundamental adjustment dimensions. Cognitive Impairment (COG) The statements that reflect limited general intellectual ability (COG1), problems in achieving in school (COG2), and a history of developmental delay or deficit (COG3) have been placed on this scale. COG2 elevation has been found to be associated with a broad range of inadequate academic habits and poor achievement in the classroom. Both teacher and clinician ratings demonstrate a strong relation between COG3 elevation and language deficits. Impulsivity and Distractibility (ADH) The majority of these items (21 of 27) appear on the first dimension. ADH1 (Disruptive Behavior) received substantial support

reyn2-15.qxd

6/6/2003

9:22 AM

Page 343

15. Multisource and Multidimensional Assessment

TABLE 15.1. PIC-2 Adjustment Scales and Subscales Cognitive Impairment (39 items, ␣ = .87, rtt = .94) PIC/R overlap: 74%, ACH: .86, IS: .66, DVL: .94 PIY overlap: 51%, COG: .43 COG1: Inadequate Abilities (13 items, ␣ = .77, rtt = .95) Others think my child is talented. My child seems to understand everything that is said. COG2: Poor Achievement (13 items, ␣ = .77, rtt = .91) It is hard for my child to make good grades. Reading has been a problem for my child. COG3: Developmental Delay (13 items, ␣ = .79, rtt = .82) At one time my child had speech difficulties. My child could ride a tricycle by age five years. Impulsivity and Distractibility (27 items, ␣ = .92, rtt = .88) PIC/R overlap: 33%, HPR: .81 PIY overlap: 63%, ADH: .31 ADH1: Disruptive Behavior (21 items, ␣ = .91, rtt = .87) My child jumps from one activity to another. My child cannot keep attention on anything. ADH2: Fearlessness (6 items, ␣ = .69, rtt = .86) My child will do anything on a dare. Nothing seems to scare my child. Delinquency (47 items, ␣ = .95, rtt = .90) PIC/R overlap: 53%, DLQ: .93 PIY overlap: 83%, DLQ: .53 DLQ1: Antisocial Behavior (13 items, ␣ = .88, rtt = .83) My child has been in trouble with the police. My child has run away from home. DLQ2: Dyscontrol (17 items, ␣ = .91, rtt = .91) When my child gets mad, watch out! Many times my child has become violent. DLQ3: Noncompliance (17 items, ␣ = .92, rtt = .87) My child often breaks the rules. My child tends to see how much he/she can get away with. Family Dysfunction (25 items, ␣ = .87, rtt = .90) PIC/R overlap: 96%, FAM: .99 PIY overlap: 100%, FAM: .44 FAM1: Conflict among Members (15 items, ␣ = .83, rtt = .90) There is a lot of tension in our home. Our family argues a lot at dinner time. FAM2: Parent Maladjustment (10 items, ␣ = .77, rtt = .91) One of the child’s parents drinks too much alcohol. The child’s parents are now divorced or living apart. Reality Distortion (29 items, ␣ = .89, rtt = .92) PIC/R overlap: 34%, PSY: .85 PIY overlap: 69%, RLT: .28 RLT1: Developmental Deviation (14 items, ␣ = .84, rtt = .87) My child often gets confused. My child needs protection from everyday dangers. (continued)

343

reyn2-15.qxd

6/6/2003

9:22 AM

344

Page 344

V. OBJECTIVE METHODS

TABLE 15.1. Continued Reality Distortion (29 items, ␣ = .89, rtt = .92) (cont.) RLT2: Hallucinations and Delusions (15 items, ␣ = .81, rtt = .79) My child thinks others are plotting against him/her. My child is likely to scream if disturbed. Somatic Concern (28 items, ␣ = .84, rtt = .91) PIC/R overlap: 79%, SOM: .95 PIY overlap: 86%, SOM: .43 SOM1: Psychosomatic Preoccupation (17 items, ␣ = .80, rtt = .90) My child is worried about disease. My child often has an upset stomach. SOM2: Muscular Tension and Anxiety (11 items, ␣ = .68, rtt = .88) Recently my child has complained of chest pains. My child often has back pains. Psychological Discomfort (39 items, ␣ = .90, rtt = .90) PIC/R overlap: 79%, D: .94, ANX: .90 PIY overlap: 79%, DIS: .42 DIS1:

Fear and Worry (13 items, ␣ = .72, rtt = .76) My child will worry a lot before starting something new. My child is often afraid of little things.

DIS2:

Depression (18 items, ␣ = .87, rtt = .91) My child has little self-confidence. My child hardly ever smiles.

DIS3:

Sleep Disturbance/Preoccupation with Death (8 items, ␣ = .76, rtt = .86) My child’s sleep is calm and restful. My child thinks about ways to kill himself/herself.

Social Withdrawal (19 items, ␣ = .81, rtt = .89) PIC/R overlap: 68%, WDL: .91 PIY overlap: 95%, WDL: .30 WDL1: Social Introversion (11 items, ␣ = .78, rtt = .90) My child is usually afraid to meet new people. Shyness is my child’s biggest problem. WDL2: Isolation (8 items, ␣ = .68, rtt = .88) My child does not like to be close with others. My child often stays in his/her room for hours. Social Skill Deficits (28 items, ␣ = .91, rtt = .92) PIC/R overlap: 75%, SSK: .96 PIY overlap: 82%, .49 SSK1:

Limited Peer Status (13 items, ␣ = .84, rtt = .92) My child often brings friends home. My child is very popular with other children.

SSK2:

Conflict with Peers (15 items, ␣ = .88, rtt = .87) My child seems to get along with every one. Other children make fun of my child’s ideas.

Note. Scale/subscale alpha (␣) values and PIC-2/PIC-R correlations based on a referred sample n = 1,551. PIC-2/PIY correlations based on a referred sample n = 382. One-week clinical retest correlation (rtt) sample n = 38. Selected material from the PIC-2 copyright © 2001 by Western Psychological Services. Reprinted by permission of the publisher, Western Psychological Services, 12031 Wilshire Boulevard, Los Angeles, California, 90025, U.S.A., www.wpspublish.com. Not to be reprinted in whole or in part for any additional purpose without the expressed, written permission of the publisher. All rights reserved.

reyn2-15.qxd

6/6/2003

9:22 AM

Page 345

15. Multisource and Multidimensional Assessment

from teacher ratings: Elevation on this subscale is associated with poor behavioral control in the classroom that disrupts the classroom process, while clinicians report impulsive, hyperactive, and restless behaviors associated with excessive attention seeking. The second dimension (ADH2, Fearlessness) appears to measure an aspect of bravado and obtained considerable correlation with teacher ratings of poorly controlled behavior. Delinquency (DLQ) DLQ1 (Antisocial Behavior) elevation is associated with behaviors readily associated with the total scale name. Subscale elevation correlates with documentation by clinician and admission by the student of a variety of unacceptable behaviors: truancy, alcohol and drug misuse, theft, running away from home, deceit, and association with other youth who are similarly troubled. DLQ2 (Dyscontrol) elevation suggests the presence of disruptive behavior associated with poorly modulated anger. Teachers note fighting and youth admit to similar problems (“I lose friends because of my temper.”). Clinicians describe these children as assaultive, defiant, argumentative, irritable, destructive, and manipulative. Their lack of emotional control often results in behaviors that demonstrate poor judgment. DLQ3 (Noncompliance) elevation emphasizes disobedience to parents and teachers, the ineffectiveness of discipline, and the tendency to blame others for problems. Youth agreement with this perception is demonstrated by a variety of PIY item correlates, including “I give my parent(s) a lot of trouble.” Family Dysfunction (FAM) This scale is divided into two meaningful dimensions. FAM1 (Conflict Among Members) reflects conflict within the family (“There is a lot of tension in our home.” “My parents do not agree on how to raise me.”). Clinicians note conflict between the child’s guardians and may be concerned that the child may have experienced emotional or physical abuse. The second FAM dimension (FAM2: Parent Maladjustment) more directly measures parent adjustment. Child descriptions that correlate with FAM2 in-

345

clude “One of my parents sometimes gets drunk and mean.” and “My parents are now divorced or living apart.” Reality Distortion (RLT) This content-valid scale is considerably different than the empirically keyed PIC-R Psychosis scale, while substantial overlap is obtained with the PIY RLT scale (RLT1: 57%, RLT2: 80%). RLT1 (Developmental Deviation) elevation describes intellectual, emotional, and social functioning usually associated with substantial developmental retardation or regression. RLT2 (Hallucinations and Delusions) describes symptoms and behaviors often associated with a psychotic adjustment. Somatic Concern (SOM) The first SOM subscale measures a variety of health complaints often associated with poor psychological adjustment. SOM1 (Psychosomatic Preoccupation) elevation is often associated with the child endorsement of similar complaints (“I feel tired most of the time.” “I often have headaches.” “I often have an upset stomach.”). The second SOM dimension (SOM2: Muscular Tension and Anxiety) appears to measure the somatic components of internalization. Psychological Discomfort (DIS) This relatively long scale of 39 items is best described as a measure of negative affectivity, divided as in the PIY into three meaningful dimensions. The first dimension (DIS1: Fear and Worry) measures fearfulness and worry and is associated with clinician description of anxiety, fear, and tearfulness as well as child complaint of fear and emotional upset. The second dimension (DIS2: Depression) is a general measure of depression that obtains considerable correlation with parent, teacher, and youth description. Teachers see students with an elevated DIS2 scale score as sad or unhappy, moody and serious, and not having fun. Clinicians note many of the classical symptoms of depression, including feelings of helplessness, hopelessness, and worthlessness. Demonstrating inadequate self-esteem, such children are overly self-critical and usually ex-

reyn2-15.qxd

6/6/2003

9:22 AM

Page 346

346

V. OBJECTIVE METHODS

pect rejection. The third dimension is similar to the PIY DIS3 dimension, combining report of problematic sleep and a preoccupation with death. Elevation of DIS3 (Sleep Disturbance/Preoccupation with Death) correlates with clinician documentation of concern regarding suicide potential and a wide variety of student admission of problems including sleep disturbance, dysphoria, and thoughts about suicide. Social Withdrawal (WDL) This is the shortest PIC-2 adjustment scale (19 items). The two WDL dimensions parallel those of the PIY: The first subscale (WDL1: Social Introversion) measures the personality dimension social introversion. Most items reflect psychological discomfort in social interactions. Clinician observation and youth self-report describe shyness and an unwillingness to talk with others. The second WDL dimension (WDL2: Isolation) is a brief subscale of eight items that describes intentional lack of contact with others. Social Skill Deficits (SSK) This scale consists of two dimensions. Both dimensions receive considerable support from self-report correlates. The first SSK subscale (SSK1: Limited Peer Status) reflects limited social influence. SSK1 elevation relates to self-report of few friends, a lack of popularity with peers, and little influence. Teachers note avoidance of peers and a lack of awareness of the feelings of others. SSK2 (Conflict with Peers) elevation, in contrast, measures problematic relations with peers. Student description documents this conflict, while clinicians observe poor social skills and a problematic social adjustment.

PIC-2 Standard Form Case Studies Table 15.2 presents the PIC-2 adjustment scales and subscales T-scores of five students seen for outpatient evaluation. Although interpretive guidelines should be applied within the context of each evaluation, these individual T-scores are highlighted based on actuarial interpretive guidelines (T >59 except for RLT2 and WDL2, T >69). The Case A PIC-2 profile presents only minimal elevations of FAM1, WDL, and

WDL1, each at 62T. This 6-year-old girl was assessed within the context of a custody evaluation; the grandmother who was the guardian completed this PIC-2. FAM1 reflects conflict between grandparents and biological parents and between biological parents. This relatively quiet young girl (WDL) obtained WISC-III (Wechsler Intelligence Scale for Children—Third Edition) and achievement scores within the normal range, and SBS teacher ratings did not document any academic or behavioral problems in the classroom. The Case B evaluation included a PIC-2 completed by this 9-year-old boy’s father. The PIC-2 was obtained as a screening measure to rule out problems in adjustment as part of a periodic reevaluation. This student was placed in a self-contained special education classroom for the cognitively impaired. Isolated elevation of COG and its subscales and minimal elevation on RLT1 were consistent with individual assessment performance in which the standard scores from measures of intellectual functioning, academic achievement, and adaptive behavior (except for socialization) were below 60. Case C’s PIC-2 was completed by his mother as part of a comprehensive outpatient evaluation. Just entering the third grade, this 9-year-old boy had been observed by his first- and second-grade teachers to demonstrate inadequate concentration, problematic participation in classroom activities, undercontrolled anger, and impulsive, disruptive behaviors. The test examiner reported a constant need to redirect C to the tasks at hand. Individual assessment demonstrated age-appropriate intellectual abilities and a 6- to 12-month delay in academic achievement. Profile C provided evidence of problems in academic achievement (COG2), ADHD-related classroom phenomena (ADH1), noncompliance, and poorly modulated anger (DLQ2, DLQ3), as well as poor peer relations (SSK2). An isolated DIS2 may reflect a dysphoric reaction to his entry into a newly established stepfamily. Even a cursory review of the profile for Case D reveals T-scores suggestive of severe developmental and behavioral disability. This 16-year-old girl had been diagnosed as autistic or with pervasive developmental disorder for the past 11 years. Cognitive

reyn2-15.qxd

6/6/2003

9:22 AM

Page 347

347

15. Multisource and Multidimensional Assessment

TABLE 15.2. Case Examples of PIC-2 Adjustment Scales and Subscales Case examples Scale/subscale

A

B

C

D

E

COG: Cognitive Impairment COG1: Inadequate Abilities COG2: Poor Achievement COG3: Developmental Delay

50 49 49 54

73 72 67 78

56 52 63 48

80 74 75 76

78 64 78 78

ADH: Impulsivity and Distractibility ADH1: Disruptive Behavior ADH2: Fearlessness

59 61 50

48 49 39

63 66 47

62 64 50

84 80 81

DLQ: Delinquency DLQ1: Antisocial Behavior DLQ2: Dyscontrol DLQ3: Noncompliance

46 46 43 49

45 54 46 43

69 46 78 66

65 46 58 72

89 71 98 79

FAM: Family Dysfunction FAM1: Conflict among Members FAM2: Parent Maladjustment

58 62 49

50 48 54

48 45 54

63 69 49

79 78 71

RLT: Reality Distortion RLT1: Developmental Deviation RLT2: Hallucination and Delusions

45 46 44

56 60 50

44 46 43

94 88 92

85 73 92

SOM: Somatic Concern SOM1: Psychosomatic Preoccupation SOM2: Muscular Tension and Anxiety

53 55 49

45 48 42

41 42 42

47 46 49

74 87 50

DIS: Psychological Discomfort DIS1: Fear and Worry DIS2: Depression DIS3: Sleep Disturbance/ Preoccupation with Death

45 51 43

41 43 42

59 49 67

75 71 67

97 76 99

45

43

43

81

73

WDL: Social Withdrawal WDL1: Social Introversion WDL2: Isolation

62 62 57

39 41 42

39 41 42

56 40 79

96 82 95

SSK: Social Skill Deficits SSK1: Limited Peer Status SSK2: Conflict with Peers

49 50 48

50 56 43

66 59 70

80 74 79

68 52 83

Note. T-scores in interpretive range set in bold to facilitate interpretation.

disability is reflected in substantial elevations of RLT1 and COG subscales. RLT1, WDL2, and SSK subscales suggest substantial deficits in adaptive behavior. Case D was described as fearful, emotionally labile, preoccupied with death, and demonstrating a sleep disorder (see DIS subscales). Case D was in constant need of supervision, often noncompliant and disruptive (DLQ3, ADH1). This family, and especially Case D’s parents, had experienced a great deal of stress in trying to meet this adolescent’s

needs and in planning for her future (FAM1). Profile E also documents severe psychopathology. (A review of validity scales ruled out the likelihood of either symptom exaggeration or inconsistent response.) Relevant history and adjustment during this evaluation and subsequent treatment were consistent with this profile. Case E’s mother completed the PIC-2. The referral questioned the presence of a learning disability in this 13-year-old boy. He had repeated

reyn2-15.qxd

6/6/2003

9:22 AM

Page 348

348

V. OBJECTIVE METHODS

kindergarten, received special education services for the learning disabled in grades 2 through 6, obtained declining academic grades in the seventh grade, and obtained borderline estimates of both intellectual ability and academic achievement in the current assessment (see COG subscales). Case E was disruptive at school (ADH and DLQ subscales) and suspicious of peers (SSK subscales). In the fifth grade, “voices” told him to hurt himself (see RLT subscales). This young adolescent was afraid to be separated from his mother and demonstrated a continuing preoccupation with death on both projective assessment and the PIY (elevated DIS subscales). Case E’s mother expressed despair in her attempts to help her son overcome his fears, because of the severity of his disabilities and her own psychological problems (FAM subscales). Elevations on SOM1 and WDL subscales reinforce the severity of these adjustment problems and demonstrate the lack of social support for this young adolescent. Treatment with weekly therapy, psychotropic medication, continued monitoring of suicide potential, and the need for psychiatric hospitalization were consistent with a diagnosis of major depression with psychotic features.

PIC-2 Behavioral Summary A major new component introduced in the PIC-2, the Behavioral Summary is based on the responses to the first 96 items of the PIC-2 administration booklet; these items are also presented in an integrated AutoScore form that supports the independent administration, scoring, and profiling of results. The Behavioral Summary can be derived from a completed administration of the PIC-2 Standard Form, by responding to only the first 96 booklet statements, or by completion of a PIC-2 Behavioral Summary Autoscore Form. The Behavioral Summary provides useful information that can be applied in a variety of situations. These applications include determination of the need for short-term intervention, the design of therapeutic interventions, the quantification of therapeutic progress, and in a variety of situations in which a short form or brief assessment would be preferred. Each Behavioral Summary item was chosen because it was written in the present

tense, had been frequently endorsed in the context of clinical assessment, and described phenomena that are often the focus of short-term therapeutic intervention. Using these guidelines, 12 items from each of eight PIC-2 adjustment scales were selected to become the short adjustment scales. Comparable items were not selected from the Cognitive Impairment scale because the majority of COG items demonstrate either historical content or were judged inappropriate targets for therapeutic effort due to the global or stable nature of the phenomena described in this dimension. For this reason, the COG scale—alone among all the PIC-2 Standard Form adjustment scales—is not represented among the shortened adjustment scales of the Behavioral Summary. Four summary or composite scales are derived from these 8 short adjustment scales and all 12 scales appear on the PIC-2 Behavioral Summary profile. The Externalization composite (EXT-C) represents the combined raw scores of the Impulsivity and Distractibility—Short and Delinquency—Short scales. The raw scores of the Reality Distortion—Short, Somatic Concern—Short, and Psychological Discomfort—Short scales are summed to form the Internalization composite (INT-C). The Social Adjustment composite (SOC-C) represents the combined raw scores of the Social Withdrawal—Short and the Social Skill Deficits—Short scales. In addition, all eight short adjustment scale scores are summed into the Total Score. The PIC-2 manual provides empirically derived interpretations of the short adjustment scales and the interpretive ranges for all 12 scales are 60–69T and >69T. The initial concurrent validity of the shortened adjustment scales was established through the correlation of these scale scores with clinician ratings, teacher descriptions, and self-report descriptions. Table 15.3 summarizes the number of external ratings identified from each source (clinician, teacher, and student) and provides examples from each rating source for each of these eight scales. Correlations between these shortened scales and their full-length versions are also presented. Table 15.3 documents that the short adjustment scales of the Behavioral Summary correlate substantially with their Standard Format versions (.92 to .96) and obtain in-

reyn2-15.qxd

6/6/2003

9:22 AM

Page 349

15. Multisource and Multidimensional Assessment

349

TABLE 15.3. Correlates of the PIC-2 Behavioral Summary Short Adjustment Scales Impulsivity and Distractibility—Short (ADH-S: .96a) Clinician ratings: total = 27 Argues (oppositional) Impulsive behavior Disobedient to parents/guardians Defiant (belligerent) Teacher ratings: total = 26 Disobeys class or school rules Disrupts class by misbehaving Sent to the office because of misbehavior Student ratings: total = 3 I often disobey my parent(s). Recently my school has sent notes home about my bad behavior. Teachers complain that I can’t sit still. Delinquency—Short (DLQ-S: .93) Clinician ratings: total = 48 Impulsive behavior Poorly modulated anger Poor judgment Disobedient to teachers Teacher ratings: total = 23 Angers other students Complains about the requests of adults Student ratings: total = 28 I often act without thinking. Sometimes I lie to get out of trouble. Several times I have said that I would run away. I sometimes swear (curse) at my parent(s). Family Dysfunction—Short (FAM-S: .93) Clinician ratings: total = 3 Conflict between parents/guardians Emotionally abused Teacher ratings: total = 0 Student ratings: total = 8 Our family enjoys being together more than most families do. (F) There is a lot of tension in our home. The child’s parents agree on how to raise the child. (F) Reality Distortion—Short (RLT-S: .94) Clinician ratings: total = 5 Auditory hallucinations Inappropriate emotion, affect Teacher ratings: total = 0 a

Reality Distortion—Short (RLT-S: .94) (cont.) Student ratings: total = 2 I need a lot of help from others. I hear voices that no one else can hear or understand. Somatic Concern—Short (SOM-S: .93) Clinician ratings: total = 3 Somatic response to stress Continually tired (listless) Teacher ratings: total = 0 Student ratings: total = 11 I often get very tired. I often have headaches. I often talk about sickness. Psychological Discomfort—Short (DIS-S: .92) Clinician ratings: total = 23 Irritability (easily upset) Anxious (tense/nervous) Depressed, sad, unhappy Inadequate self-esteem Teacher ratings: total = 2 Appears sad or unhappy Becomes upset for little or no reason Student ratings: total = 3 I tend to feel sorry for myself. I am not very sure of myself. I am often afraid of little things. Social Withdrawal—Short (WDL-S: .96) Clinician ratings: total = 1 Withdrawn Teacher ratings: total = 0 Student ratings: total = 1 Shyness is my biggest problem. Social Skill Deficits—Short (SSK-S: .95) Clinician ratings: total = 3 Poor social skills Isolated, few or no friends Teased by peers Teacher ratings: total = 0 Student ratings: total = 13 I wish that I were more able to make and keep friends. Other kids make fun of my ideas. Other kids are often angry with me. I am often rejected by other kids.

Correlation between Standard Format and Behavioral Summary versions of PIC-2 adjustment scales in a sample of 1,551 referred children. See text for explanation of correlate selection procedure. Note. Selected material from the PIC-2 copyright © 2001 by Western Psychological Services. Reprinted by permission of the publisher, Western Psychological Services, 12031 Wilshire Boulevard, Los Angeles, California, 90025, U.S.A., www.wpspublish.com. Not to be reprinted in whole or in part for any additional purpose without the expressed, written permission of the publisher. All rights reserved.

reyn2-15.qxd

6/6/2003

9:22 AM

Page 350

350

V. OBJECTIVE METHODS

dependent correlates from nonparent observers that match scale item content and diagnostic intent. Clinician ratings provided the greatest support for ADH-S, DLQ-S, and DIS-S, focusing on problems of disruptive and noncompliant behavior and intense and dysphoric affect that often form the basis of clinical referral. These analyses also demonstrate that ADH-S, as previously documented for the PIC Hyperactivity scale (Lachar & Gdowski, 1979), assesses those behaviors most related to problems in classroom adjustment. In addition, observations obtained directly from the student being evaluated provide those internal and subjective judgements that demonstrate the clinical value of the PIC-2 dimensions that did not receive robust correlates from clinicians or teachers.

PIC-2 Behavioral Summary Case Studies Table 15.4 presents the application of the Behavioral Summary in the evaluation of four youth. Case A’s pediatrician requested an evaluation to document either a learning disability or a significant behavior disorder. This boy’s kindergarten teacher reported that he had become increasingly disruptive in the classroom. Both of Case A’s parents completed the Behavioral Summary (see columns labeled Am and Af in Table 15.4). His mother noted that his disruptive behav-

iors were most noticeable in highly structured situations and that he was most responsive to his father’s discipline. Individual psychometric evaluation did not provide any evidence of a learning disability and considerable consistency was obtained between teacher SBS and maternal Behavioral Summary description. Because intervention at home was contemplated in addition to classroom modification, A’s father also completed the Behavioral Summary. It is important to note that the profile obtained from the mother identified problem dimensions of value in the development of a treatment plan and demonstrated that administration of EXT-C following an intervention would be an effective means of quantifying expected behavior change. In contrast, the father’s Behavioral Summary description suggested that A’s behavior reflected a problem-free adjustment. In addition to the general discrepancy between parent profiles, the mother’s considerable elevation on FAM-S (75T) also suggests that these differences in parent perception should be addressed as part of a therapeutic intervention. Case Study B presents the application of the Behavioral Summary to the design of an intervention and the subsequent measurement of its effectiveness (see columns labeled B1 and B2 in Table 15.4). This 8-yearold boy had recently entered third grade. For the previous 2 years his teachers had ex-

TABLE 15.4. Case Examples Using the PIC-2 Behavioral Summary Profile Case examples Scale

Am

Af

B1

B2

C

D

Short Adjustment Scales ADH-S: Impulsivity and Distractibility—Short DLQ-S: Delinquency—Short FAM-S: Family Dysfunction—Short RLT-S: Reality Distortion—Short SOM-S: Somatic Concern—Short DIS-S: Psychological Discomfort—Short WDL-S: Social Withdrawal—Short SSK-S: Social Skill Deficits—Short

70 67 75 43 63 62 41 58

50 57 54 43 49 44 41 43

63 71 46 43 43 57 41 63

53 57 46 43 43 53 41 53

66 71 75 55 56 53 64 72

80 82 64 55 42 48 55 44

Composite Scales EXT-C: Externalizing INT-C: Internalizing SOC-C: Social Adjustment TOT-C: Total Score

70 58 49 66

53 43 40 47

68 48 52 57

58 46 46 49

70 55 71 71

83 48 49 65

Note. T-scores in interpretive range set in bold to facilitate interpretation.

reyn2-15.qxd

6/6/2003

9:22 AM

Page 351

15. Multisource and Multidimensional Assessment

pressed concern regarding his ability to focus on classroom activities and to get along with his classmates. His mother, who completed both questionnaires, had requested an evaluation in hopes of identifying areas of academic skills for which home-based remediation would be helpful. The short adjustment scales identified problems in attention, cooperation, and poor relations with peers also supported by teacher rating with the SBS. Profile B2 documents improvement in child status following 3 months of stimulant medication and a coordinated effort between mother and teacher to improve his classroom performance. Case C is presented briefly to demonstrate that multidimensional objective assessment may bring clarity to an evaluation in a manner that may be inconsistent with the parent informant’s expressed personal opinion. In this case, this sixth-grade boy’s mother attributed poor emotional adjustment, more specifically depression, to family disruption secondary to a recent divorce. In fact, DIS-S and INT-C obtained from her descriptive responses both fall within the normal range. A full neuropsychological evaluation provided a more parsimonious alternative explanation for both his recent deteriorating behavioral adjustment to the classroom as well as his failure to improve through counseling and psychopharmacological treatment of his presumed “depression.” Case D is presented as an example of the value of routine addition of a PIC-2 Behavioral Summary to child assessment. This 15year-old girl was examined with the sole purpose of ruling out a learning disability. Although no psychometric evidence of a learning disability was obtained, the Behavioral Summary suggested areas of problematic behavioral adjustment consistent with teacher rating and self-report. This profile identified a need for treatment, specific problem dimensions, and an EXT-C of 83T suggested the value of a subsequent administration of the Behavioral Summary to quantify the effect of treatment.

PERSONALITY INVENTORY FOR YOUTH PIY Standard Profile The majority of PIY items were derived from rewriting the first 280 items of the

351

PIC-R administration booklet into first person format (see Table 15.5 for examples of PIY items). PIY self-report scales and subscales were constructed in the same manner as PIC-2 scales (Lachar & Gruber, 1993; Lachar & Gruber, 1995a, 1995b). The nine clinical scales were constructed with a uniform methodology, resulting in assignment of 231 items to only one scale, as well as a high degree of both scale content appropriateness and homogeneity. As in the PIC-2, each of these scales has been further divided into two or three nonoverlapping subscales that represent factor-guided dimensions of even greater content homogeneity. The pattern of scale and subscale elevation is a major focus of the PIY and PIC-2 interpretive process. Gender-specific linear T-scores have been derived from a national normative sample of 2,327 regular education students in grades 4 through 12, while a variety of analyses have been conducted using a large sample of clinically referred students (n = 1,178). PIY clinical scales average 26 items in length (range of 17–42 items) and the median coefficient alpha in referred protocols was .85 (range of .74–.92). Median test–retest reliability was .80 (range of .76–.91). The 24 subscales average 10 items in length (range of 4–16 items with five subscales of less than eight items in length) and the median coefficient alpha in referred protocols was .73 (range of .44–.84 with eight of 24 subscales less than .70). Comparable test–retest reliability estimates are a median of .73 and a range of .58–.88 (with 5 of 24 subscales less than .70). The PIY Administration and Interpretation Guide (Lachar & Gruber, 1995a) provides interpretive guidelines for scales and subscales as well as 15 case studies. PIY profile data are integrated into a demonstration of the effect of response sets and in a study of treatment effectiveness presented below. Differences in the nature of self-report and parent report are demonstrated when PIY scale and subscale content are compared to the PIC-2 equivalents. The PIY Cognitive Impairment scale includes only half of the items of the comparable PIC-2 scale. This difference reflects the exclusion of developmental or historical items in the self-report format (children are not accurate reporters of developmental delay), as well as

reyn2-15.qxd

6/6/2003

9:22 AM

352

Page 352

V. OBJECTIVE METHODS

TABLE 15.5. PIY Clinical Scales and Subscales Cognitive Impairment (20 items, ␣ = .74, rtt = .80) COG1: Poor Achievement and Memory (8 items, ␣ = .65, rtt = .70) I often forget to do things. School has been easy for me. COG2: Inadequate Abilities (8 items, ␣ = .67, rtt = .67) People say that I have common sense. I think I am stupid or dumb. COG3: Learning Problems (4 items, ␣ = .44, rtt = .76) I have been held back a year in school. Because of my learning problems, I get extra help, or am in a special class in school. Impulsivity and Distractibility (17 items, ␣ = .77, rtt = .84) ADH1: Brashness (4 items, ␣ = .54, rtt = .70) I brag a lot. I often nag and bother other people. ADH2: Distractibility/Overactivity (8 items, ␣ = .61, rtt = .71) I cannot wait for things like other kids can. Most of the time I run rather than walk. ADH3: Impulsivity (5 items, ␣ = .54, rtt = .58) I often act without thinking. I am often restless. Delinquency (42 items, ␣ = .92, rtt = .91) DLQ1:

Antisocial Behavior (15 items, ␣ = .83, rtt = .88) I sometimes skip school. I use illegal drugs.

DLQ2:

Dyscontrol (16 items, ␣ = .84, rtt = .88) People think that I am mean. I lose friends because of my temper.

DLQ3:

Noncompliance (11 items, ␣ = .83, rtt = .80) I often disobey my parent(s). Punishment does not change how I act.

Family Dysfunction (29 items, ␣ = .87, rtt = .83) FAM1:

Parent–Child Conflict (9 items, ␣ = .82, rtt = .73) I am unhappy about my home life. My parent(s) are too strict with me.

FAM2:

Parent Maladjustment (13 items, ␣ = .74, rtt = .76) My parents disagree a lot about how to raise me. My parents often argue.

FAM3:

Marital Discord (7 items, ␣ = .70, rtt = .73) My parent(s) always discuss things before they make a big decision. My parents’ marriage has been solid and happy.

Reality Distortion (22 items, ␣ = .83, rtt = .84) RLT1:

Feelings of Alienation (11 items, ␣ = .77, rtt = .74) I do strange or unusual things. I often get confused.

reyn2-15.qxd

6/6/2003

9:22 AM

Page 353

15. Multisource and Multidimensional Assessment

TABLE 15.5. Continued Reality Distortion (22 items, ␣ = .83, rtt = .84) (cont.) RLT2:

Hallucinations and Delusions (11 items, ␣ = .71, rtt = .78) I am afraid I might be going insane. People secretly control my thoughts.

Somatic Concern (27 items, ␣ = .85, rtt = .76) SOM1: Psychosomatic Syndrome (9 items, ␣ = .73, rtt = .63) I often get very tired. I often have headaches. SOM2:

Muscular Tension and Anxiety (10 items, ␣ = .74, rtt = .72) At times I have trouble breathing. Sometimes my heart pounds or races.

SOM3: Preoccupation with Disease (8 items, ␣ = .60, rtt = .59) I often talk about sickness. Being sick upsets me more than it does most others. Psychological Discomfort (32 items, ␣ = .86, rtt = .77) DIS1:

Fear and Worry (15 items, ␣ = .78, rtt = .75) Small problems do not bother me. I worry about things that adults worry about.

DIS2:

Depression (11 items, ␣ = .73, rtt = .69) I try to make the best of most things. I am often in a good mood.

DIS3:

Sleep Disturbance (6 items, ␣ = .70, rtt = .71) I have a lot of nightmares. I often think about death.

Social Withdrawal (18 items, ␣ = .80, rtt = .82) WDL1: Social Introversion (10 items, ␣ = .78, rtt = .77) Talking to others makes me nervous. I am often embarrassed. WDL2: Isolation (8 items, ␣ = .59, rtt = .77) I almost always play alone. I keep my thoughts to myself. Social Skill Deficits (24 items, ␣ = .86, rtt = .79) SSK1:

Limited Peer Status (13 items, ␣ = .79, rtt = .76) Other kids look up to me as a leader. People always listen when I speak.

SSK2:

Conflict with Peers (11 items, ␣ = .80, rtt = .72) I do not get along with the other students at school. I wish that I were more able to make and keep friends.

Note. Scale/subscale alpha (␣) values based on a clinical sample n = 1,178. Oneweek clinical retest correlation (rtt) sample n = 86. Selected material from the PIY copyright © 1995 by Western Psychological Services. Reprinted by permission of the publisher, Western Psychological Services, 12031 Wilshire Boulevard, Los Angeles, California, 90025, U.S.A., www.wpspublish.com. Not to be reprinted in whole or in part for any additional purpose without the expressed, written permission of the publisher. All rights reserved.

353

reyn2-15.qxd

6/6/2003

9:22 AM

Page 354

354

V. OBJECTIVE METHODS

the reality that fewer self-report items correlated with this dimension for youth. The PIY Impulsivity and Distractibility scale also incorporated fewer scale items (17) than its PIC-2 equivalent (27 items). Perhaps the report of ADH disruptive behavior would be more likely to be expected from an adult informant who finds such behavior distressful than from a student who may not find such behaviors disturbing. Such results suggest that the PIC-2 COG and ADH scales will demonstrate superior diagnostic performance in comparison to these PIY scales. In contrast, the other seven PIY clinical scales achieved a significant degree of similarity in content and length with the PIC-2 scale equivalents.

Abbreviated PIY Form The first 80 items of the PIY comprise a 32item screening scale chosen to provide an optimal identification of those regular education students who, when administered the full PIY, produce clinically significant results. These items also include three “scan items” for each clinical scale. Scan items were selected in such a manner so that students who endorse two or more of each set of three items would be those with a high probability of scoring > 59T on the corresponding clinical scale. Shortened versions of three validity scales can also be derived from these items.

PIY AND PIC-2 VALIDITY SCALES PIY and PIC-2 profiles include three comparable validity scales. The Inconsistency scale evaluates the likelihood that responses to items are either random or reflect in some manner inadequate comprehension of inventory statements or inadequate compliance with test instructions. The Dissimulation scale identifies profiles that may result from either exaggeration of current problems or a malingered pattern of atypical or infrequent symptoms. The third validity scale, Defensiveness, identifies profiles likely to demonstrate the effect of minimization or denial of current problems. The PIY also provides a fourth unique validity measure that consists of six items written so that either a true or false response would be highly

improbable, such as responding false to “I sometimes talk on the telephone.”

Inconsistency Scales The Inconsistency scales (INC) measure semantic inconsistency (Tellegen, 1988) through the classification of response to 35 pairs of highly correlated items drawn from all nine clinical scales (examples of statement pairs are “I have many friends.” versus “I have very few friends.” and “My child has a lot of talent.” versus “My child has no special talents.”). For each pair of statements, two response combinations are consistent and two are inconsistent (either True/True and False/False or True/False and False/True). Each inconsistent pair identified in a given protocol contributes one point to the INC raw score. Application of a cutting raw score of 12 correctly identified 92–96% of random protocols.

Dissimulation Scales The Dissimulation scales (abbreviated FB, representing “fake bad”) were empirically constructed through item analyses that compared clinical protocols and two sets of protocols completed by nonreferred regular education students or their parents. The PIY or PIC-2 was first completed with directions to provide an accurate or valid description. The same student or parent then completed the questionnaire a second time, now describing the student as in need of mental health counseling or psychiatric hospitalization. Selected FB items in the scored direction were infrequent in valid normal (PIY: 11%, PIC-2: 4%) and in valid clinical protocols (PIY: 18%, PIC-2: 15%), while frequent (PIY: 83%, PIC-2: 55%) in the fake bad or dissimulated condition. FB items reflect “erroneous stereotype” in that they reflect face-valid content by naïve informants but demonstrate no empirical validity (Lanyon, 1997). Examples of the 42 PIY FB items include “People are out to get me.” and “I do not care about having fun.” Examples of the 35 PIC-2 FB items include “My child is not as strong as most children.” and “My child often talks about sickness.” Application of one FB cutting score to PIY data correctly identi-

reyn2-15.qxd

6/6/2003

9:22 AM

Page 355

355

15. Multisource and Multidimensional Assessment

fied 99% of accurate, 98% of fake bad, and 96% of clinical protocols. Application of two potential cutting scores to comparable PIC-2 protocols revealed that both correctly classified 97–100% of accurate regular education student descriptions. A cutting score of >8 resulted in correct classification of 92% of dissimulated and 78% of clinical protocols (possible dissimulation), whereas a cutting score of >14 resulted in correct classification of 70% of dissimulated and 95% of clinical protocols (probable dissimulation). The pattern of FB and INC scale elevation facilitates the differentiation of inadequate from inaccurate response. A deliberate exaggerated response (or for that matter an accurate description of a severe or atypical psychopathological adjustment) would generate an elevated FB score and a nonelevated INC scale score. Protocols completed without adequate statement comprehension, in contrast, obtain raw INC and FB scores approximating 50% of each scale’s length; in this case both scales are clinically elevated (see Lachar & Gruber, 1995b, Figures 11, 12).

Table 15.6 presents the PIC-2 and SBS results that describe a hospitalized 7-year-old first-grade boy who had a history of multiple psychiatric hospitalizations. This boy had attended a self-contained special education classroom for the emotionally impaired. His current psychiatric hospitalization was due to reported verbal and physical aggression toward family members, noncompliance, auditory and visual hallucinations, agitation, running in front of moving cars, attempting to drown himself, hair pulling, running away from home, and oppositional defiant behavior. His parents were asking for assistance in obtaining an agency placement, as they were unable to cope with his undercontrolled behavior. Several psychiatrists had been sufficiently impressed by these parent descriptions to prescribe a variety of psychotropic medications (stimulants, neuroleptics, mood stabilizers, and antidepressants). In contrast to the parents’ presentation of severe emotional and behavioral psychopathology, it became known that this young boy had previously called Children’s

TABLE 15.6. Case E: Symptom Malingering and the PIC-2 Standard Form Profile Personality Inventory for Children, Second Edition Validity Scales Inconsistency Dissimulation Defensiveness

47 121 27

COG COG1 COG2 COG3

78 68 60 98

FAM FAM1 FAM2

82 81 71

ADH ADH1 ADH2

75 78 56

RLT RLT1 RLT2

102 92 104

DLQ DLQ1 DLQ2 DLQ3

89 54 102 81

SOM SOM1 SOM2

74 82 57

DIS DIS1 DIS2 DIS3

106 76 95 123

WDL WDL1 WDL2 SSK SSK1 SSK2

94 75 106

Student Behavior Survey Academic Performance Academic Habits Social Skills Parent Participation Health Concerns Emotional Distress Unusual Behavior Social Problems

46 58 57 — 59 38 43 35

Verbal Aggression Physical Aggression Behavior Problems Attention-Deficit Hyperactivity Oppositional Defiant Conduct Problems

89 77 88

40 43 39 40 38 43

reyn2-15.qxd

6/6/2003

9:22 AM

356

Page 356

V. OBJECTIVE METHODS

Protective Services to allege physical and emotional abuse by his parents. No indications of maladjustment were observed during his first week of hospitalization, and a telephone conversation with his teacher elicited that he was “a bright and cooperative student, obtaining excellent grades.” Subsequently the mother completed the PIC-2, the teacher completed the SBS, and this child was administered tests of intellectual ability and academic achievement. An obtained Full Scale IQ of 122 and SBS Academic Habits T = 58 and Social Skills T = 57 (positively worded statements reflect above-average adaptive behaviors in the classroom) are in remarkable contrast to his mother’s description of a severely disturbed child. A review of the PIC-2 Standard Format profile reveals an Inconsistency raw score of 4 (of 35, T = 47) and a Dissimulation raw score of 24 (a score equivalent to the top third of dissimulated protocols, T = 121). This pattern raises serious concerns about the validity of this profile. Indeed, the PIC-2 manual documents that the obtained pattern of response validity scales was the most common in malingered profiles (60.9%) and very uncommon in comparison samples (standardization = 0.6%, referred = 4.8%, random = 4.2%)(Lachar & Gruber, 2001, Chapter 3, Table 4). Of the nine broad adjustment dimensions, only two received any external support during this evaluation: FAM suggested considerable family conflict and parent maladjustment, while SOM (and the SBS Health Concerns scale) may reflect somatic components of stress experienced within this family. The pattern of 9 elevated adjustment scales and 18 elevated subscales is characteristic of both malingered and random PIC-2 protocols (Lachar & Gruber, 2001, Chapter 7, Table 47). Consistent with the generation of a malingered profile, subsequent weeks of observation during this hospital stay documented in a variety of ways that this child had been the scapegoat in this family and had experienced considerable emotional abuse.

Defensiveness Scales The remaining PIC-2 and PIY validity scales are labeled Defensiveness (DEF); each scale represents an expanded version of the PIC

Lie scale. DEF items represent denials of common problems (“Sometimes I put off doing a chore. False” “My child almost never argues. True”) and attributions of improbable positive adjustment (“My child always does his/her homework on time. True” “I am almost always on time and remember what I am supposed to do. True”). Such items represent inaccurate knowledge in the form of over-endorsement (Lanyon, 1997). DEF elevations above 59T, even in patients hospitalized on a psychiatric unit, result in profiles that either minimize current problems or consistently deny the presence of most or all problems in adjustment (Wrobel et al., 1999). A secondary interpretation of an elevated PIY DEF scale is that such a youth would be unlikely to be good candidate for talk therapy. Youth who respond with denial to items of an administration booklet are most likely to respond in a similar manner during a diagnostic interview. The INC/FB/DEF pattern readily identifies profiles in which caution must be applied to their interpretation. Table 15.7 presents three PIC-2 and PIY profile pairs. Case F was a high school senior who was evaluated at the request of her mother. Although interview did not identify any acute concerns, her mother identified several areas of continuing concern, including concentrating on school assignments and meeting obligations at home. Comparison of the PIC-2 and PIY profiles of Case F to each other and to the additional information gathered during this evaluation demonstrate both the value of the PIY and PIC-2 Dissimulation scale. FB is clinically elevated on the PIC-2, not the PIY, and the PIC-2 adjustment scales and subscales suggest problems that are not supported by the PIY or other evaluation. The PIC-2 emphasis on problems with cognitive status (COG1, RLT1), academic achievement (COG2), and inattention and disruptive behavior (ADH1) were not supported by neuropsychological evaluation or teacher description on the SBS. Neither the PIY nor individual interview of this adolescent provided any support for the presence of somatic concern (SOM1) or social discomfort (WDL1). In contrast, a variety of evidence was revealed that supported the presence of a clinical depression that required treatment. This depression was manifest in disruptive preoccu-

reyn2-15.qxd

6/6/2003

9:22 AM

Page 357

357

15. Multisource and Multidimensional Assessment

TABLE 15.7. Effect of Exaggeration and Defensiveness on PIC-2/PIY Profile Pairs

Scale/subscale

Case F __________________ PIC-2 PIY

Case G __________________ PIC-2 PIY

Case H _________________ PIC-2 PIY

Inconsistency Dissimulation Defensiveness

56 79 29

51 57 39

67 81 30

57 48 64

50 47 65

68 72 50

Cognitive Impairment COG1 COG2 COG3

63 62 69 43

49 62 40 43

67 68 70 48

57 52 45 85

43 49 41 43

47 42 45 67

Impulsivity and Distractibility ADH1 ADH2 ADH3

66 71 42 —

43 42 41 50

81 83 64 —

39 49 37 41

44 46 41 —

64 53 62 66

Delinquency DLQ1 DLQ2 DLQ3

71 46 67 76

51 49 49 53

86 63 98 76

41 43 41 43

42 46 43 42

49 49 56 42

Family Dysfunction FAM1 FAM2 FAM3

47 42 55 —

53 63 45 48

58 58 54 —

57 52 57 58

67 62 72 —

49 47 49 53

Reality Distortion RLT1 RLT2

69 66 68

54 63 48

73 83 56

48 53 41

50 55 44

79 74 79

Somatic Concern SOM1 SOM2 SOM3

65 77 42 —

54 59 52 45

74 76 65 —

49 38 53 59

41 42 41 —

65 59 65 65

Psychological Discomfort DIS1 DIS2 DIS3

88 66 91 93

75 66 77 68

97 81 92 83

63 64 52 63

53 51 49 63

68 66 58 68

Social Withdrawal WDL1 WDL2

90 84 81

56 49 65

85 72 88

41 46 38

46 49 42

59 53 65

Social Skill Deficits SSK1 SSK2

79 78 70

67 65 65

86 68 97

56 53 59

49 50 48

57 56 55

Note. T-scores in interpretive range set in bold to facilitate interpretation.

pation and problematic sleep (DIS3), withdrawal (WDL2), and a deteriorating and unrewarding social life (SSK). The profiles of Case G present a fairly common inconsistency between parent and youth obtained when the parent unilaterally seeks to hospitalize a child on a psychiatric unit. The only consistency in agreement is in the problem area of academic achievement (PIC-2 COG2, Poor Achieve-

ment, PIY COG3, Learning Problems). History and psychometric assessment document retention in grade, special class placement, and academic achievement substantially below assessed intellectual ability. Clearly the PIC-2 most accurately described this 12-year-old boy who was hospitalized for multiple behavioral and emotional problems. The elevation of the PIY DEF scale (T = 64) is the most likely explanation

reyn2-15.qxd

358

6/6/2003

9:22 AM

Page 358

V. OBJECTIVE METHODS

for this PIY profile that is essentially within normal limits. Indeed, review of this patient’s medical record details his repeated denial and minimizing of problems during hospitalization in an attempt to facilitate his early discharge from treatment. This child’s problems developed following a traumatic motor vehicle accident. He felt lonely and scared, he frequently cried, sobbed, shook, avoided others, and was preoccupied with excessive worries (DIS, WDL). He externalized his problems (DLQ) and had difficulties with peers and had little insight into his role in these conflicts (SSK2, Conflict with Peers). Case H is quite unusual in that the PIC-2 profile, with the exception of FAM1 (Conflict Among Members), FAM2 (Parent Maladjustment), and DIS3 (Sleep Disturbance/ Preoccupation with Death), is essentially within normal limits for this 12-year-old girl who was hospitalized on a psychiatric unit. Furthermore, this girl was hospitalized at the request of the mother who completed the PIC-2. The differences between PIC-2 and PIY on the Defensiveness scale are also consistent with a variety of PIY scale and subscale elevations that find no support on the PIC-2 profile. Case H presented with suicidal ideation, low self-esteem, depression, crying spells, poor appetite, and associated weight loss (DIS, WDL2). She actively demonstrated somatic concern and somatic symptoms in response to conflict during this hospitalization (SOM). H had told her therapist that she would not talk about her problems with her mother, because she was afraid such a discussion would distress her mother who was under psychiatric care (see PIC-2 FAM2, Parent Maladjustment). Clinicians were sufficiently concerned with H’s internalizing problems to assign discharge diagnoses of generalized anxiety disorder and depressive disorder not otherwise specified and to direct her to continue treatment with antidepressant medication following discharge. Why was H’s mother defensive in describing her daughter’s problems? The medical record documented that she worried that her daughter’s and her own psychiatric problems would demonstrate her inadequacy as a mother. Such a conclusion could result in her loss of her child to another adult family member.

STUDENT BEHAVIOR SURVEY The items of the SBS (Lachar, Wingenfeld, Kline, & Gruber, 2000) were developed through the review of established teacher rating scales and in the writing of new rating statements that focused on content appropriate to teacher observation. SBS items are not derived from the PIY or PIC-2. Unlike measures that provide separate parent and teacher norms for the same questionnaire items (see, e.g., the Devereux Scales of Mental Disorders; Naglieri, LeBuffe, & Pfeiffer, 1994), the SBS items demonstrate a specific school focus. Review of the SBS reveals that 58 out of its 102 items specifically refer to in-class or in-school behaviors and judgments that can only be made by school staff (Wingenfeld, Lachar, Gruber, & Kline, 1998). The SBS items are profiled onto 14 scales that assess student academic status and work habits, social skills, parental participation in the educational process, and problems such as aggressive or atypical behavior and emotional stress. Norms that generate linear T-scores are gender-specific and divided into two age groups: 5–11 and 12–18 years. As in the development and initial validation of the PIC-2 and PIY, a substantial number of SBS protocols were collected to describe regular education students (n = 2,612), as well as special education students or students referred for a psychological evaluation (n = 1,315). The SBS rating form presents descriptive statements and their rating options on both sides of one sheet of paper. These statements are sorted into content meaningful dimensions and are placed under 11 scale headings to enhance the clarity of item meaning rather than being presented in a random order. The SBS consists of two major sections. The first section, Academic Resources, comprises four scales that address positive aspects of the child’s adjustment to school, while the second section Adjustment Problems comprises seven scales that reflect the presence of various dimensions of psychopathology. Two slightly different rating methods are used to rate the SBS test items. On the first scale, Academic Performance (AP) the test items are eight areas of achievement such as reading comprehension or mathematics. For

reyn2-15.qxd

6/6/2003

9:22 AM

Page 359

15. Multisource and Multidimensional Assessment

these items, the teacher selects one of five ratings options (Deficient, Below Average, Average, Above Average, Superior) to describe each area of achievement. The remaining 94 items are rated on a 4-point frequency scale: Never, Seldom, Sometimes, and Usually. After the Academic Performance scale, the other items in the Academic Resources section are positively worded statements divided into three scales. The first two of these scales consist of descriptions of positive behaviors that describe the student’s adaptive classroom behaviors: Academic Habits (AH) and Social Skills (SS). The third scale consists of ratings of parent behaviors that are school specific: In Parent Participation (PP), the teacher is asked to evaluate by rating six descriptive statements the degree to which parents support the student’s educational program. The second major SBS section, Problems in Adjustment, provides seven scales that consist of negatively worded items: Health Concerns (HC), Emotional Distress (ED), Unusual Behavior (UB), Social Problems (SP), Verbal Aggression (VA), Physical Aggression (PA), and Behavior Problems (BP). Initial item and scale performance documented that 99 of 102 items statistically separated clinical and special education protocols from SBS protocols of regular education students. All items demonstrated that they had been placed on the scale with which each obtained the largest correlation. Scale scores of regular education and referred students obtained meaningful threefactor solutions (Wingenfeld et al., 1998). Additional effort (Pisecco et al., 1999) was applied to the construction of three additional 16-item nonoverlapping scales. These three scales consist of items nominated from several SBS content dimensions because they were consistent with characteristics often associated with one of three DSM-IV diagnoses (not necessarily specific diagnostic criteria): attention-deficit/hyperactivity disorder, combined type; oppositional defiant disorder; and conduct disorder. Item-to-scale correlations and a three-factor solution of these 48 SBS items empirically supported the placement of these items into scales labeled Attention Deficit Hyperactivity (ADH), Oppositional Defiant (OPD), and Conduct Problems

359

(CNP). Table 15.8 provides examples of SBS scale items, scale length, and the psychometric characteristics of coefficient alpha, test–retest reliability, and interrater reliability. Reflecting appropriateness of item placement, coefficient alpha values on these 14 scales ranged from .85 to .95 (median = .905). Table 15.8 presents the results of one of the four test–retest reliability analyses presented in the SBS manual. Using a short (.49 demonstrates across-informant construct validity for 11 of the 14 SBS scales. Teacherrated Academic Performance correlated substantially with COG2 (Poor Achievement, r = –.55). Academic Habits and Social Skills obtain similar correlations to PIC-2 subscales demonstrating that disruptive behavior and limited social adjustment at school correlate with noncompliant (DLQ3) and disruptive (ADH1) behaviors at home as well as parent report of poor school performance (COG2). Specific support was also achieved for Emotional Distress (DIS2, r = .49) and Social Problems (SSK2, r = .45; ADH1, r = .47, DLQ3, r = .46). Parent description of poorly modulated anger (DLQ2) correlated with both Physical Aggression (r = .51) and Verbal Aggression (r = .56) in the classroom, while ADH and other

reyn2-15.qxd

6/6/2003

9:22 AM

360

Page 360

V. OBJECTIVE METHODS

TABLE 15.8. SBS Scales, Their Psychometric Characteristics, and Sample Items Scale name (abbreviation)

Items/␣/rtt/r1,2

Examples

Academic Performance (AP)

8/.89/.78/.84

Reading comprehension Speech articulation

Academic Habits (AH)

13/.93/.87/.76

Completes class assignments Remembers teacher’s directions

Social Skills (SS)

8/.89/.88/.73

Helps other students Participates in class activities

Parent Participation (PP)

6/.88/.83/.68

Parent(s) encourage achievement Parent(s) meet with school staff when asked

Health Concerns (HC)

6/.85/.79/.58

Complains of headaches Talks about being sick

Emotional Distress (ED)

15/.91/.90/.73

Appears sad or unhappy Mood changes without reason

Unusual Behavior (UB)

7/.88/.76/.62

Says strange or bizarre things Seems disoriented or lost

Social Problems (SP)

12/.87/.90/.72

Angers other students Teased by other students

Verbal Aggression (VA)

7/.92/.88/.79

Argues and wants the last word Threatens other students

Physical Aggression (PA)

5/.90/.86/.63

Hits or pushes other students Destroys property when angry

Behavior Problems (BP)

15/.93/.92/.82

Disobeys class or school rules Lies to school personnel

Attention-Deficit/ Hyperactivity (ADH)

16/.94/.91/.83

Waits for his/her turn Talks excessively

Oppositional Defiant (OPD)

16/.95/.94/.86

Mood changes without reason Insults other students

Conduct Problems (CNP)

16/.94/.90/.69

Swears at school personnel Skips classes

Note. Scale alpha (␣) values based on a referred sample n = 1,315. Retest correlation (rtt) 5- to 11-year-old student sample (n = 52) with average rating interval of 1.7 weeks. Interrater agreement (r1,2) sample n = 60 fourth/fifth-grade team taught or special education students. Selected material from the SBS copyright © 2000 by Western Psychological Services. Reprinted by permission of the publisher, Western Psychological Services, 12031 Wilshire Boulevard, Los Angeles, California, 90025, U.S.A., www.wpspublish.com. Not to be reprinted in whole or in part for any additional purpose without the expressed, written permission of the publisher. All rights reserved.

DLQ dimensions also correlated with teacher rated Verbal Aggression. Teacherrated Behavior Problems in students also obtained substantial correlations with undercontrolled behaviors (ADH, r = .56; DLQ, r = .64) as described by their mothers. Health Concerns, Unusual Behavior, and Parent Participation obtained minimal if any support from this analysis. Review of the correlations between PIC-2 and PIY SOM subscales (Lachar & Gruber, 2001, Chapter 7, Table 34) suggests that parents and their children are more attuned to these

phenomena. Parent Participation is a dimension unique to the SBS; in contrast, parent (PIC-2) and student (PIY) dimensions of family status that share similar content are frequently correlated above .39. Unusual Behavior may represent phenomena that are infrequently observed in the classroom, even for students who receive mental health services. However, significant cross-informant agreement is also not obtained between parent and child for a comparable yet more parallel measure of maladjustment (RLT2).

reyn2-15.qxd

6/6/2003

9:22 AM

Page 361

361

15. Multisource and Multidimensional Assessment

TABLE 15.9. Correlations between SBS and PIC-2 Scale Scores SBS scale PIC-2 scale

AP

AH

SS

PP

HC

ED

UB

SP

VA

PA

BP

COG COG1 COG2 COG3

–.57 –.36 –.55 –.44

–.50 –.36 –.58 —

–.40 –.32 –.46 —

— — — —

— — — —

— — .31 —

.31 — — —

.37 .32 .39 —

.31 — .40 —

— — 31 —

.31 — .46 —

.46 .33 .52 —

.36 — .43 —

— — .38 —

ADH ADH1 ADH2

— — —

–.52 –.53 –.32

–.51 –.51 –.32

— — —

— — —

.36 .37 —

— — —

.47 .47 .31

.53 .50 .48

.41 .39 38

.56 .53 .49

.56 .57 .36

.57 .55 .43

.49 .44 .49

DLQ DLQ1 DLQ2 DLQ3

— — — —

–.43 — –.36 –.45

–.47 — –.44 –.47

— — — —

— — — —

.40 — .36 .38

— — — —

.45 — .39 .46

.60 .48 .56 .52

.54 .49 .51 .41

.64 .59 .54 .57

.45 — .38 .47

.60 .41 .54 .56

.64 .64 .56 .50

FAM FAM1 FAM2

— — —

— — —

— — —

— — —

— — —

— .30 —

— — —

— — —

.33 .34 —

.34 .33 —

.36 .34 —

— — —

.32 .33 —

.39 .37 .31

— –.33 —

–.36 –.34 –.32

–.37 –.34 –.34

— — —

— — —

.32 .30 .30

— — —

.38 .37 .32

.34 .30 .34

— — —

.30 — .32

.35 .34 .31

.36 .33 .34

— — .30

SOM SOM1 SOM2

— — —

— — —

— — —

— — —

— .30 —

.30 — —

— — —

— — —

— — —

— — —

— — —

— — —

— — —

— — —

DIS DIS1 DIS2 DIS3

— — — —

–.31 — –.33 —

–.38 — –.39 —

— — — —

— — — —

.47 — .49 .37

— — — —

.38 — .41 .30

.36 — .39 .33

.31 — .35 —

.33 — .39 .32

.30 — .31 —

.42 — .45 .35

.34 — .38 .32

WDL WDL1 WDL2

— — —

— — —

— — —

— — —

— — —

— — —

— — —

— — .34

— — —

— — —

— — —

— — —

— — .31

— — —

SSK SSK1 SSK2

— — —

–.34 — –.35

–.41 –.30 –.42

— — —

— — —

.35 — .34

.30 — —

.46 .36 .45

.30 — .37

— — —

— — .34

.36 — .39

.38 .32 .42

— — .30

RLT RLT1 RLT2

ADH OPD CNP

Note. Scale abbreviations explained in Tables 15.1 and 15.8. Correlations 1 SD, or 10 T-score points), although only two (WDL, SSK) present a pattern of a clinically elevated scale falling at reassessment within the normal range. Parent report documents improved social adjustment and improved affect and behavioral control. This adolescent, in contrast, was much more explicit in documenting change. In two PIY profiles in which validity scales fall within normal limits baseline assessment demonstrated nine clinically elevated subscales that fell to within normal limits at the second administration. Prominent changes were demonstrated on DIS and SOM subscales suggesting improved mood, sleep, and somatic state. Subscales also suggest improved behavioral control, less conflict with parents (FAM1), and improved relations with peers (SSK2).

COMMENTARY The second edition of the PIC, the development of a multidimensional teacher rating scale, and the collection of a national representative normative sample for each have gone a long way to respond to previous concerns raised regarding the PIC’s age (Kamphaus & Frick, 1996; Knoff, 1989; Merrell, 1994). It is expected that objective multidimensional assessment of youth will continue to gain popularity in a variety of applications, and the integration of such measures into a variety of settings will become the expectation. In light of such activity, critical evaluations of the PIC-2, PIY, and SBS manuals and the study of their demonstrated ability to evaluate emotional adjustment and to quantify response to in-

tervention will continue well into this new century. The emphasis on evaluating profile accuracy through the use of validity scales and the empirical determination of interpretive guidelines continues to characterize these measures. Many psychologists unconvinced of the importance of these psychometric issues may not value their contributions to assessment. A rather convincing demonstration of the effect of response sets was recently presented (Lachar, Morgan, Espadas, & Schomer, 2000). Adolescents newly admitted to a psychiatric unit were administered both the PIY and the self-report version of the Child Behavior Checklist, the Youth Self-Report (YSR; Achenbach, 1991). When the resulting profiles were sorted into two groups based on the presence of an elevated PIY DEF value (DEF > 59T), substantial differences were obtained for both selfreport measures. Of special interest were differences in estimates of undercontrolled behavior provided by the YSR, an instrument that does not incorporate any direct estimates of response validity. For example, the summary measure Externalizing Problems obtained an average value of 45T in the DEF > 59T group and an average value of 66T in the DEF < 60T group. Applying a conservative measure to classify individual protocols as demonstrating problem behavior (T > 64), 53% of nondefensive adolescents described a poor adjustment, while none of the defensive adolescents, as classified by the PIY validity scale, obtained a similar result. Certainly, when defensiveness, inadequate comprehension, or poor cooperation is likely in a given setting, or for a specific evaluation, the direct examination of response validity will often contribute to a full understanding of test results. Although the PIC-2 has been reduced from 420 to 275 items, some clinicians may still judge the length of this and the other similar questionnaires to be problematic. It is certain that the breadth and depth of a measure’s content establishes the potential boundaries of its utility. Even the 270 items of the PIY are easily completed in less than 45 minutes by children as young as fourth graders. PIC-2, PIY, and SBS efficiency has been improved by rejecting any item not ac-

reyn2-15.qxd

6/6/2003

9:22 AM

Page 365

15. Multisource and Multidimensional Assessment

tively used in the interpretive process as well as providing computer software for scoring and interpretation. The value of saving 10 or 15 minutes of teacher, parent, or youth effort should be balanced against what is lost in measure reliability and in the restriction of the variety of dimensions assessed. Professionals unmoved by the logic of this appeal may find routine application of the 96-item PIC-2 Behavioral Summary more attractive.

SUMMARY This chapter reviews the development and application of a “family” of parent-, teacher-, and self-report multidimensional inventories for use with school-aged children and adolescents (grades K–12). These objective questionnaires integrate a variety of psychometric components that improve efficiency and facilitate inventory interpretation, such as validity scales, subscalewithin-scale structure, and screening forms designed to be sensitive to treatment effects. The PIC-2, PIY, and SBS measure dimensions of internalizing and externalizing problem behaviors, family character, and cognitive ability. Each measure incorporates dimensions that are similar across informants as well as dimensions that are unique to a given informant source. These questionnaires can be applied independently or in combination. This chapter provides demonstration of instrument validity and application to case studies, as well as use of the PIC-2, PIY, and SBS to document treatment effectiveness.

REFERENCES Achenbach, T. M. (1981). A junior MMPI? [Review of Multidimensional description of child personality: A manual for the Personality Inventory for Children and Actuarial assessment of child and adolescent personality: An interpretive guide for the Personality Inventory for Children profile.] Journal of Personality Assessment, 45, 332–333. Achenbach, T. M. (1991). Manual for the Youth SelfReport and 1991 Profile. Burlington: University of Vermont, Department of Psychiatry. Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant corre-

365

lations for situational specificity. Psychological Bulletin, 101, 213–232. Archer, R. P., Tirrell, C. A., & Elkins, D. E. (2001). Evaluation of an MMPI-A short form: Implications for adaptive testing. Journal of Personality Assessment, 76, 76–89. August, G. J., Realmuto, G. M., MacDonald III, A. W., Nugent, S. M., & Crosby, R. (1996). Prevalence of ADHD and comorbid disorders among elementary school children screened for disruptive behavior. Journal of Abnormal Child Psychology, 42, 571–595. Conners, C. K. (1997). Conners’ Rating Scales-Revised technical Manual. North Tonawanda, NY: MultiHeath Systems. Duhig, A. M., Renk, K., Epstein, M. K., & Phares, V. (2000). Interparental agreement on internalizing, externalizing, and total behavior problems: A metaanalysis. Clinical Psychology: Science and Practice, 7, 435–453. Gdowski, C. L., Lachar, D., & Kline, R. B. (1985). A PIC profile typology of children and adolescents: I. An empirically-derived alternative to traditional diagnosis. Journal of Abnormal Psychology, 94, 346–361. Greenbaum, P. E., Dedrick, R. F., Prange, M. E., & Friedman, R. M. (1994). Parent, teacher, and child ratings of problem behaviors of youngsters with serious emotional disturbances. Psychological Assessment, 6, 141–148. Harrington, R. G., & Follett, G. M. (1984). The readability of child personality assessment instruments. Journal of Psychoeducational Assessment, 2, 37–48. Jensen, P. S., Watanabe, H. K., Richters, J. E., Roper, M., Hibbs, E. D., Salzberg, A. D., & Liu, S. (1996). Scales, diagnoses, and child psychopathology: II. Comparing the CBCL and the DISC against external validators. Journal of Abnormal Child Psychology, 24, 151–168. Kamphaus, R. W., & Frick, P. J. (1996). Clinical assessment of child and adolescent personality and behavior. Boston: Allyn & Bacon. Kline, R. B., & Lachar, D. (1992). Evaluation of age, sex, and race bias in the Personality Inventory for Children (PIC). Psychological Assessment, 4, 333–339. Kline, R. B., Lachar, D., & Gdowski, C. L. (1987). A PIC typology of children and adolescents: II. Classification rules and specific behavior correlates. Journal of Clinical Child Psychology, 16, 225–234. Kline, R. B., Lachar, D., Gruber, C. P., & Boersma, D. C. (1994). Identification of special education needs with the Personality Inventory for Children (PIC): A profile-matching strategy. Assessment, 1, 301–313. Kline, R. B., Lachar, D., & Sprague, D. J. (1985). The Personality Inventory for Children (PIC): An unbiased predictor of cognitive and academic status. Journal of Pediatric Psychology, 10, 461–477. Knoff, H. M. (1989). Review of the Personality Inventory for Children, Revised Format. In J. C. Connolly and J. C. Kramer (Eds.), The Tenth Mental Measurements Yearbook (pp. 624–630). Lincoln, NE: Buros Institute of Mental Measurements.

reyn2-15.qxd

366

6/6/2003

9:22 AM

Page 366

V. OBJECTIVE METHODS

Lachar, D. (1982). Personality Inventory for Children (PIC) Revised Format manual supplement. Los Angeles, CA: Western Psychological Services. Lachar, D. (1998). Observations of parents, teachers, and children: Contributions to the objective multidimensional assessment of youth. In A. S. Bellack & M. Hersen (Series Eds.) & C. R. Reynolds (Vol. Ed.), Comprehensive clinical psychology: Vol. 4. Assessment (pp. 371–401). New York: Pergamon. Lachar, D., & Gdowski, C. L. (1979). Actuarial assessment of child and adolescent personality: An interpretive guide for the Personality Inventory for Children profile. Los Angeles, CA: Western Psychological Services. Lachar, D., Gdowski, C. L., & Snyder, D. K. (1982). Broad-band dimensions of psychopathology: Factor scales for the Personality Inventory for Children. Journal of Consulting and Clinical Psychology, 50, 634–642. Lachar, D., Gdowski, C. L., & Snyder, D. K. (1984). External validation of the Personality Inventory for Children (PIC) profile and factor scales: Parent, teacher, and clinician ratings. Journal of Consulting and Clinical Psychology, 52, 155–164. Lachar, D., & Gruber, C. P. (1993). Development of the Personality Inventory for Youth: A self-report companion to the Personality Inventory for Children. Journal of Personality Assessment, 61, 81–98. Lachar, D., & Gruber, C. P. (1995a). Personality Inventory for Youth (PIY) manual: Administration and interpretation guide. Los Angeles, CA: Western Psychological Services. Lachar, D., & Gruber, C. P. (1995b). Personality Inventory for Youth (PIY) manual: Technical guide. Los Angeles, CA: Western Psychological Services. Lachar, D., & Gruber, C. P. (2001). Personality Inventory for Children, Second Edition (PIC-2) Standard Format and Behavioral Summary manual. Los Angeles, CA: Western Psychological Services. Lachar, D., & Kline, R. B. (1994). The Personality Inventory for Children (PIC) and the Personality Inventory for Youth (PIY). In M. Maruish (Ed.), Use of psychological testing for treatment planning and outcome assessment (pp. 479–516). Hillsdale, NJ: Erlbaum. Lachar, D., Kline, R. B., & Gdowski, C. L. (1987). Respondent psychopathology and interpretive accuracy of the Personality Inventory for Children: The evaluation of a “most reasonable” assumption. Journal of Personality Assessment, 51, 165–177. Lachar, D., Kline, R. B., Green, B. A., & Gruber, C. P. (1996, August). Contribution of self-report to PIC profile type interpretation. Paper presented at the annual meeting of the American Psychological Association, Toronto. Lachar, D., Morgan, S. T., Espadas, A., & Schomer, O. (2000, August). Effect of defensiveness on two selfreport child adjustment inventories. Paper presented at the annual meeting of the American Psychological Association, Washington, DC. Lachar, D., Wingenfeld, S. A., Kline, R. B., & Gruber, C. P. (2000). Student Behavior Survey (SBS) manual. Los Angeles, CA: Western Psychological Services.

LaCombe, J. A., Kline, R. B., Lachar, D., Butkus, M., & Hillman, S. B. (1991). Case history correlates of a Personality Inventory for Children (PIC) profile typology. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 13, 1–14. Lanyon, R. I. (1997). Detecting deception: Current models and directions. Clinical Psychology: Science and Practice, 4, 377–387. Loeber, R., Green, S. M., & Lahey, B. B. (1990). Mental health professionals’ perception of the utility of children, mothers, and teachers as informants on childhood psychopathology. Journal of Clinical Child Psychology, 19, 136–143. Lonigan, C. J., Carey, M. P., & Finch, A. J. Jr. (1994). Anxiety and depression in children and adolescents: Negative affectivity and the utility of self-reports. Journal of Consulting and Clinical Psychology, 62, 1000–10008. Merrell, K. W. (1994). Assessment of behavioral, social, and emotional problems: Direct and objective methods for use with children and adolescents. New York: Longman. Michael, K. D., & Merrell, K. W. (1998). Reliability of children’s self-reported internalizing symptoms over short to medium-length time intervals. Journal of the American Academy of Child and Adolescent Psychiatry, 37, 194–201. Naglieri, J. A., LeBuffe, P. A., & Pfeiffer, S. I. (1994). Devereux Scales of Mental Disorders manual. San Antonio, TX: Psychological Corporation. Phares, V. (1997). Accuracy of informants: Do parents think that mother knows best? Journal of Abnormal Child Psychology, 25, 165–171. Pisecco, S., Lachar, D., Gruber, C. P., Gallen, R. T., Kline, R. B., & Huzinec, C. (1999). Development and validation of disruptive behavior DSM-IV scales for the Student Behavior Survey (SBS). Journal of Psychoeducational Assessment, 17, 314–331. Power, T. J., Andrews, T. J., Eiraldi, R. B., Doherty, M. J., Ikeda, M. J., DuPaul, G. J., & Landau, S. (1998). Evaluating attention deficit hyperactivity disorder using multiple informants: The incremental power of combining teacher with parent reports. Psychological Assessment, 10, 250–260. Richters, J. E. (1992). Depressed mothers as informants about their children: A critical review of the evidence for distortion. Psychological Bulletin, 112, 485–499. Tellegen, A. (1988). The analysis of consistency in personality assessment. Journal of Personality, 56, 621–663. Vaughn, M. L., Riccio, C. A., Hynd, G. W., & Hall, J. (1997). Diagnosing ADHD (predominantly inattentive and combined subtypes): Discriminant validity of the Behavior Assessment System for Children and the Achenbach parent and teacher rating scales. Journal of Clinical Child Psychology, 26, 349–357. Wingenfeld, S. A., Lachar, D., Gruber, C. P., & Kline, R. B. (1998). Development of the teacher-informant Student Behavior Survey. Journal of Psychoeducational Assessment, 16, 226–249. Wirt, R. D., Lachar, D., Klinedinst, J. K., & Seat, P. D. (1977). Multidimensional description of child personality: A manual for the Personality Inventory for

reyn2-15.qxd

6/6/2003

9:22 AM

Page 367

15. Multisource and Multidimensional Assessment Children. Los Angeles, CA: Western Psychological Services. Wrobel, T. A., Lachar, D., Wrobel, N. H., Morgan, S. T., Gruber, C. P., & Neher, J. A. (1999). Performance of the Personality Inventory for Youth validity scales. Assessment, 6, 367–376.

367

Youngstrom, E., Loeber, R., & Stouthamer-Loeber, M. (2000). Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. Journal of Consulting and Clinical Psychology, 68, 1038– 1050.

reyn2-16.qxd

6/6/2003

9:24 AM

Page 368

16 The Minnesota Multiphasic Personality Inventory—Adolescent

ELLEN W. ROWE

The Minnesota Multiphasic Personality Inventory—Adolescent (MMPI-A; Butcher et al., 1992) is a broad-band, self-report instrument designed for the psychological assessment of adolescents ages 14–18. The measure can be administered once to provide evaluative and descriptive information, or it can be administered repeatedly to assess change over time (Archer, 1997b). The need to document change may result from developmental processes or intervention and treatment. Although the topic of this chapter is the MMPI-A, the chapter begins with a short review of the original Minnesota Multiphasic Personality Inventory (MMPI). The MMPI was first published in 1943 by Hathaway and McKinley (Graham, 1987) and was not revised until some 46 years later when the MMPI-2 was released (Butcher, Dahlstrom, Graham, Tellegen, & Keammer, 1989). After the MMPI-2 was published, the decision was made to develop an adolescent version of the MMPI. Though different from the MMPI, both the MMPI-2 and the MMPI-A are based on the MMPI. To understand the foundation and origins of the MMPI-A, therefore, one must begin with the MMPI.

FOUNDATION OF THE MMPI-A: THE ORIGINAL MMPI One of the primary reasons for the development of the MMPI was the authors’ wish to create an instrument that could be used to aid clinicians in efficiently and accurately diagnosing abnormal behavior (Graham, 1987). From the beginning, then, the scale was not intended to measure “normal” personality functioning. Instead, Hathaway and McKinley hoped to create an objective scale to assist in making accurate diagnostic decisions with adult inpatient psychiatric populations. Since its publication, however, the MMPI has been used clinically and for research purposes in many settings and with many different populations. The MMPI became popular almost immediately following its publication. Only 3 years after publication, practitioners ranked the MMPI as 15th among the most commonly used assessment instruments (Sundberg, 1961). By 1961 the MMPI rated as fifth. Furthermore, the MMPI has generated voluminous amounts of research. Thousands and thousands of articles and books exist on the MMPI. Buros (1974) reported 368

reyn2-16.qxd

6/6/2003

9:24 AM

Page 369

16. The Minnesota Multiphasic Personality Inventory—Adolescent

that by 1974 about 200 articles, books, or dissertations were being published annually on the MMPI. At the time of its publication, one of the most innovative aspects of the MMPI was Hathaway and McKinley’s approach to the creation of the instrument (Greene, 1980). In the 1930s and 1940s most scales were created using the rational method. With the rational method, items are selected for a scale or instrument if, based on the developer’s experience and opinion, they seem logically to measure the trait in question. Obviously, items that seem to measure a construct may or may not in fact measure the construct. In contrast to the rational method, Hathaway and McKinley used a procedure called criterion keying. Criterion keying was considered a big step forward in the advancement of empirical test construction. When using criterion keying, items can be initially created or selected using rational means. Hathaway and McKinley (1940) chose their original items from already existing tests, their experience, and psychiatric textbooks. Items are then given to two or more groups of individuals. One group is the criterion group that carries a particular diagnosis or trait the instrument is assumed to measure. The other group(s) serves as a comparison group(s) that does not exhibit the trait or diagnosis in question. Items are selected based on their ability to demonstrate differing response rates between the criterion and other group(s) (McKinley & Hathaway, 1940). The higher an individual’s number of positive responses, the more similar the individual’s responses are to the criterion group. For their criterion groups, Hathaway and McKinley selected eight clinical groups of patients (Graham, 1987). Each group was thought to represent a single diagnostic category and exhibit a relatively homogenous symptom profile. Response patterns by these eight groups yielded 8 of the 10 basic MMPI scales. These eight scales were named after the diagnostic labels of Hathaway and McKinley’s groups: Hypochondriasis (Hs, scale 1); Depression (D, scale 2); Hysteria (Hy, scale 3); Psychopathic Deviant (Pd, scale 4); Paranoia (Pa, scale 6); Psychasthenia (Pt, scale 7); Schizophrenia (Sc, scale 8); and Hypomania (Ma, scale 9). A group of homosexual males served as the

369

criterion group in creating the Masculinity–Femininity scale (Mf, scale 5) (Hathaway, 1956). Drake created the tenth and final standard scale of the MMPI in 1946. This scale, the Social Introversion–Extroversion scale (Si, scale 0), was the only scale created without a criterion group. The 10 basic clinical scales of the MMPI-A, though not composed of exactly identical items, follow the same order and carry the same labels as these 10 scales of the MMPI. Another aspect of the MMPI that contributed to its use and popularity was the creation and inclusion of four validity scales. Like the 10 clinical scales, validity scales with these same names appear on the MMPI-A. First was the Cannot Say (or ?) scale. The Cannot Say scale was simply the total number of items to which an individual did not respond or responded both True and False. Next was the F scale. According to Hathaway and McKinley (as cited in Archer, 1997b), a high score on the MMPI F scale was intended to alert clinicians to careless responding or extensive scoring errors. Another possible interpretation was that elevated F scale scores represented a tendency to exaggerate symptoms or “fake bad” (Graham, 1987). High F scale scores were also associated with high levels of pathology (Greene, 1980). The L or Lie scale originally consisted of 15 items that most people would acknowledge as faults or shortcomings (Graham, 1987). An elevation on this scale was intended to signal attempts by test takers to present oneself in a favorable manner or to look good. The last MMPI validity scale was the K scale. The primary purpose of the K scale was to increase the ability of the clinical scales to accurately identify psychopathology. The L, F, and K validity scales together with the 10 clinical scales were the 13 basic scales of the MMPI. In addition to the basic clinical and validity scales, researchers subsequently created many supplementary scales and subscales from MMPI items. These supplementary scales and subscales were intended to expand on the interpretive information available from the basic scales. As Graham (1987) points out, some of these scales were widely used and were well researched, while others were used only by those who developed them. Three of the most widely used

reyn2-16.qxd

370

6/6/2003

9:24 AM

Page 370

V. OBJECTIVE METHODS

supplementary scales were Welsh’s (1956) Anxiety (A) and Repression (R) scales and the MacAndrew Alcoholism Scale (MAC; MacAndrew, 1965). Welsh developed his A and R scales by starting with factor-analytic results from the basic scales. When factored, the basic scales typically yielded two core factors (Welsh, 1956). The A scale was created to measure the first factor and the R scale the second factor. MacAndrew (1965) created the MAC scale by selecting 51 MMPI items that distinguished outpatient male alcoholics from nonalcoholic male psychiatric outpatients. Higher scores on the MAC scale are associated with higher tendencies toward alcoholism. All three of these supplementary scales are interpretable with the MMPI-A. Because 8 of the 10 basic clinical scales were created using criterion keying, Hathaway and McKinley gave little attention to the homogeneity of items within each scale (Graham, 1987). As a result, the items within a particular scale could vary considerably in terms of content. Two test takers, then, could have had similar patterns of elevated scores that resulted from endorsing fairly different sets of items. To overcome this, a number of investigators analyzed subgroups of similar items within the basic scales to create subscales. The most commonly used subscales for the MMPI were the Harris and Lingoes (as cited in Graham, 1987) subscales. Harris and Lingoes developed subscales for 6 (scales 2, 3, 4, 6, 8, 9) of the 10 clinical scales. For example, Harris and Lingoes broke scale 2, Depression, into five subscales labeled Subjective Depression, Psychomotor Retardation, Physical Malfunctioning, Mental Dullness, and Brooding. It is worth noting that Harris and Lingoes adopted a logical approach to their creation of these subscales (Graham, 1987). Thus, items that seemed similar or seemed to measure a similar domain were grouped together on a subscale. Results for the Harris–Lingoes subscales are available with the MMPI-A testing and scoring materials. Other MMPI scales available for interpretation were the Wiggins (1966) content scales. Unlike the Harris–Lingoes subscales, which used the 10 basic clinical scales as a starting point and a logical approach to classifying items, Wiggins used the entire

MMPI item pool and followed both empirical and rational methods to develop his scales (Wiggins, 1966). Wiggins focused on devising scales with sound internal consistency and went through several iterations and revisions before arriving at his final scales. Content scales were created for the MMPI-2 and the MMPI-A at the time of their development, and several of these are similar to the Wiggins scales (Williams, Butcher, Ben-Porath, & Graham, 1992). Of course the MMPI was popular not just because of its design and structure but also because of its utility as a clinical tool. One of the most important aspects of the MMPI related to its use as a clinical tool, or in other words, to its interpretation. Although Hathaway and McKinley intended the MMPI to be used in diagnosing discrete psychiatric disorders corresponding to the 10 clinical scales, research did not show the instrument to be successful when applied in this manner (Graham, 1987). Instead, individuals with a given psychological disorder often evidenced elevations on more than one scale. Someone with depression, for example, might have elevations on that scale as well as others. Graham notes that this may have been partially the result of high intercorrelations among some of the scales. As a result, the MMPI was interpreted primarily in a descriptive manner. This does not mean MMPI results were not used to assist in making diagnostic decisions but merely that diagnostic outcomes typically did not correspond to a single scale elevation of the same name. To avoid the association between diagnostic categories and the scale labels, then, the 10 clinical scales were often referred to by number (1–0) rather than by label (i.e., Hypochondriasis). So it came to be that research on the MMPI guided clinicians’ interpretations, not the measure’s actual design or structure. Over the years, numerous techniques existed for MMPI interpretation. Many of these were based on the “profile” of an individual’s scale scores. Among the most common means of interpretation was the codetype method. Codetypes were used to interpret MMPI results both for adults and adolescents. According to Graham (1987), codetypes are simply a method of interpretation that considers more than one scale at a time. A 2-point codetype indicates the two

reyn2-16.qxd

6/6/2003

9:24 AM

Page 371

16. The Minnesota Multiphasic Personality Inventory—Adolescent

highest clinical scales in the profile. A 46 codetype, therefore, is one on which an individual received his or her highest score on scale 4 and the second highest score on scale 6. For the most part, codetypes are interchangeable so that a 46 can often be interpreted the same as a 64. Similar to 2-point codetypes, 3-point codetypes denote the three highest scales, but the 2-point codetype method was more commonly used. Decades of MMPI research and clinical use resulted in descriptors or correlates that were associated with codetypes. Codetype descriptors can be found in sources such as Graham (1987), Greene (1980), and Archer, (1987). One should note that codetypes reveal nothing about the absolute value of the scores, only the peaks of the profile. As Graham points out, though, the descriptors associated with a particular MMPI codetype were more likely to be appropriate when the T-scores for the codetype were above 70 and when these scores were significantly more elevated than the remainder of the profile. Even then, these descriptors or correlates are probabilistic statements that may or may not be accurate for a particular individual client (Greene, 1980). Although the MMPI was intended for use with adults ages 16 and older, it was used with adolescents both clinically and in research for decades (Archer, 1997b). Marks and Briggs (1972) developed the most frequently used normative data for adolescents, and publication of these norms opened the way for increased interpretive research with adolescent samples. In 1974, Marks, Seeman, and Haller published the first codetype descriptors or correlates specifically for adolescents based on research with adolescent samples, and in 1987, Archer published a comprehensive summary of MMPI research findings with adolescents. Archer, Maruish, Imhof, and Piotrowski (1991) documented the MMPI’s popularity for use with adolescent populations in their 1990 survey of clinicians. The MMPI was rated as the sixth most frequently used assessment instrument with adolescents. In fact, as the MMPI-A was not yet published, the previous edition of the Handbook of Psychological and Educational Assessment of Children (Reynolds & Kamphaus, 1990) includes a chapter on use of the MMPI with adolescents (Reilley, 1990).

371

Despite a burgeoning base of MMPI research with adolescents and the prevalence of its use with this population, the survey by Archer and colleagues (1991) pointed to several concerns regarding use of the MMPI with adolescents. The original MMPI consisted of 566 items, and clinicians in the survey listed the instrument’s length as a primary problem in evaluating adolescents. A second consideration was a normative sample. Some clinicians used adult norms with adolescents, whereas others relied on the Marks and Briggs (1972) normative data. Neither of these solutions was optimal. Archer (1987) reviewed the research on use of adult norms with adolescent samples and concluded that adolescent responses should be scored and interpreted using adolescent norms. At the same time, the Marks and Briggs data were not racially or nationally representative. Another problem listed by clinicians working with adolescents was the outdated or inappropriate language contained in some items. Finally, some clinicians reported that the lack of items or scales targeted specifically at adolescent problems or issues was a concern (Archer et al., 1991). The developers of the MMPI-A were aware of these shortcomings and sought to address them in the development of the MMPI-A (Butcher et al., 1992).

THE MMPI-A Development While the goals of the MMPI-A development team included production of an instrument that was sensitive to adolescent psychopathology and improvement of the shortcomings involved in using the MMPI with adolescents, the developers also endeavored to maintain as much continuity as possible with the original MMPI (Butcher et al., 1992). Consequently, the experimental version of the MMPI-A used in creating the final form of the instrument contained all 550 MMPI items (the 16 repeated items on the MMPI were not repeated on this experimental form). Another 154 items were added to the original 550 for a total of 704 items on the experimental version. Of the 154 additional items, 96 were items that related specifically to adolescent development

reyn2-16.qxd

6/6/2003

9:24 AM

372

Page 372

V. OBJECTIVE METHODS

or pathology. These new items covered domains such as school, teachers, relationships with parents and families, sexuality, and the negative influence of peers. The remaining 58 trial items were borrowed from the experimental version of the MMPI-2. In the end, a total of 478 items of the 704 were retained for the MMPI-A. This reduced number of items met the concern of a shorter instrument for use with adolescents.

Normative Sample To obtain the normative data, the MMPI-A experimental form was administered to approximately 2,500 adolescents in eight states (Archer, 1997b). The eight states of Minnesota, New York, North Carolina, Ohio, California, Virginia, Pennsylvania, and Washington were selected to provide a balance of rural and urban settings, geographic regions, and ethnic backgrounds (Butcher et al., 1992). Adolescents were recruited through their junior high and high schools, and the instrument was administered in groups. Exclusionary criteria were adopted to ensure that only complete and valid data were included in the normative sample. Exclusionary criteria included incomplete forms, age above 18 or below 14, F scale raw score above 25, or a Cannot Say score greater than 35. The final normative sample consisted of 805 males and 815 females, ages 14–18. The geographic, ethnic, age, grade, and parental education and occupation distributions of the final sample are provided in the MMPI-A: Manual for Administration, Scoring, and Interpretation (Butcher et al., 1992). The ethnic distribution is roughly similar to that of the 1980 census figures, but a few problems with the normative sample are worth noting. For example, the 18-year-old group consists of only 87 individuals. Furthermore, the parents of students in the normative sample tended to be better educated than one would predict from Census data, with about 50% of fathers and 40% of mothers reporting a college degree. As Black (1994) points out, the high levels of parental education bring into question the representativeness of the sample in terms of socioeconomic status. This in turn influences the degree of confidence one can place in the interpretation of results for adolescents with low so-

cioeconomic levels. Finally, because recruiting efforts were made through schools, the normative data are probably not representative of adolescents who drop out of school and those frequently absent (Butcher, et al, 1992). Once the normative data were collected, raw scores were transformed to linear Tscores for the validity scales, clinical scales 5 (Masculinity–Femininity) and 0 (Social Introversion), and for all of the supplementary scales (Archer, 1997b). However, the eight remaining clinical scale (1, 2, 3, 4, 6, 7, 8, and 9) scores and all the content scale scores raw score were transformed using uniform T-score procedures. Uniform T-score procedures have the advantage of maintaining the skewness of the underlying raw score distribution, while at the same time allowing for percentile comparability across scales (Butcher et al., 1992). Thus clinicians can consider percentiles on these scales comparably and make interpretations accordingly. Uniform T-score procedures were not used with scales 5 and 0 because these scales were created in a different manner than the other clinical scales and were more normally distributed (Archer, 1997b). Data were also obtained from a clinical sample of 420 boys and 293 girls, ages 14–18 (Butcher et al., 1992). The clinical sample was recruited from inpatient and outpatient treatment facilities and a special school program in Minnesota. The ethnic makeup of the clinical sample was less representative of Census data than the normative sample and closer to that found in the general Minnesota population.

Age Range As stated previously, the MMPI-A is normed for use with adolescents, ages 14–18. In the norming process, data were collected on 12- and 13-year-olds, but the data for adolescents under 14 revealed substantially higher F scale scores, suggesting a problem with validity (Butcher et al., 1992). Therefore, the decision was made to exclude data on 12- and 13-year-olds from the normative sample. In the MMPI-A: Manual for Administration, Scoring, and Interpretation, Butcher and colleagues (1992) suggest that mature boys or girls under the age of 14 who have good reading comprehension

reyn2-16.qxd

6/6/2003

9:24 AM

Page 373

16. The Minnesota Multiphasic Personality Inventory—Adolescent

may be able to respond reliably and validly to the MMPI-A. However, they caution that further research is needed with this age group and interpretations should be made with care. The reading level currently suggested for the MMPI-A is seventh grade (Archer, 1997b). With 18-year-olds clinicians have the choice of using either the MMPI-A or the MMPI-2. For this age, the recommendation in the manual is to use the MMPI-A with 18-year-olds who are in high school and the MMPI-2 with 18-year-olds who are in college, working, or living independently (Butcher et al., 1992). Shaevel and Archer (1996) compared the scores and profiles for a sample of 18-year-olds using both the MMPI-2 and the MMPI-A. The researchers found that using MMPI-2 norms as opposed to MMPI-A norms can result in considerably lower validity scale scores and higher clinical scale scores. The discrepancies reached as much as 15 T-score points, with higher range scores tending to be more discrepant. The authors conclude that in some cases, when the decision of which test to use seems arbitrary, the clinicians may want to use both the MMPI-A and the MMPI-2 norms in order to consider possible score and profile differences (Shaevel & Archer, 1996).

Format and Structure The MMPI-A consists of 478 items to which adolescents respond True or False. As on the MMPI, items are phrased as declarative statements in the first person. Clinicians have the options of administering the test in a paper-and-pencil booklet format where the adolescent reads the items, with an audiocassette, or on a computer. The 478 items yield the 13 basic validity and clinical scales, 4 new validity scales, 6 supplementary scales, the 28 Harris–Lingoes and 3 Si subscales, and 15 new content scales (see Table 16.1). The L, F1, and K validity scale scores and 10 standard clinical scale scores can be obtained from responses to the first 350 items. All but 58 of these 350 items appeared on the original MMPI. The majority of deleted items were from the F, Mf, and Si scales (Archer, 1997b). The last 148 MMPIA items are necessary for scoring the additional scales.

373

TABLE 16.1. MMPI-A Scales and Subscales Validity scales Cannot Say L (Lie) F, F1, F2 (Infrequency) K (Defensiveness) VRIN (Variable Response Inconsistency) TRIN (True Response Inconsistency) Basic clinical scales Scale 1 (Hs: Hypochondriasis) Scale 2 (D: Depression) Scale 3 (Hy: Hysteria) Scale 4 (Pd: Psychopathic Deviate) Scale 5 (Mf: Masculinity–Femininity) Scale 6 (Pa: Paranoia) Scale 7 (Pt: Psychasthenia) Scale 8 (Sc: Schizophrenia) Scale 9 (Ma: Hypomania) Scale 0 (Si: Social Introversion) Supplementary scales A (Anxiety) R (Repression) MAC-R (MacAndrew Alcoholism Scale— Revised) ACK (Alcohol/Drug Problem Acknowledgement) PRO (Alcohol/Drug Problem Proneness) IMM (Immaturity) Content scales A-anx (Anxiety) A-obs (Obsessiveness) A-dep (Depression) A-hea (Health Concerns) A-aln (Alienation) A-biz (Bizarre Mentation) A-ang (Anger) A-cyn (Cynicism) A-con (Conduct Problems) A-lse (Low Self-esteem) A-las (Low Aspirations) A-sod (Social Discomfort) A-fam (Family Problems) A-sch (School Problems) A-trt (Negative Treatment Indicators) Harris–Lingoes subscales D1 (Subjective Depression) D2 (Psychomotor Retardation) D3 (Physical Malfunctioning) D4 (Mental Dullness) D5 (Brooding) Hy1 (Denial of for Social Anxiety) Hy2 (Need Affection) Hy3 (Lassitude–malaise) Hy4 (Somatic Complaints) Hy5 (Inhibition of Aggression) Pd1 (Familial discord) Pd2 (Authority Problems) Pd3 (Social Imperturbability) (continues)

reyn2-16.qxd

6/6/2003

9:24 AM

Page 374

374

V. OBJECTIVE METHODS

TABLE 16.1. Continued Harris–Lingoes subscales (cont.) Pd4 (Social Alienation) Pd5 (Self-alienation) Pa1 (Persecutory Ideas) Pa2 (Poignancy) Pa3 (Naivete) Sc1 (Social Alienation) Sc2 (Emotional Alienation) Sc3 (Lack of Ego Mastery, Cognitive) Sc4 (Lack of Ego Mastery, Conative) Sc5 (Lack of Ego Mastery, Defective Inhibition) Sc6 (Bizarre Sensory Experience) Ma1 (Amorality) Ma2 (Psychomotor Acceleration) Ma3 (Imperturbability) Ma4 (Ego Inflation) Si subscales Si1 (Shyness/Self-consciousness) Si2 (Social Avoidance) Si3 (Alienation-self and Others)

Among the new scales are several validity scales. The F1 and F2 scales are subscales of the F scale. The other new validity subscales are the True Response Inconsistency scale (TRIN) and the Variable Response Inconsistency scale (VRIN). The MMPI-A supplementary scales available from the extended scoring materials include the traditional Anxiety scale (A), Repression scale (R), and MacAndrew Alcoholism Scale (MAC). Three new supplementary scales were developed particularly for the MMPI-A. These new scales are the Alcohol/Drug Acknowledgement scale (ACK), the Alcohol/Drug Problem Proneness scale (PRO), and the Immaturity scale (IMM). The Harris–Lingoes subscales from the MMPI are available with the MMPI-A. Item deletions on the Harris–Lingoes subscales parallel those on the basic clinical scales (Archer, 1997b). The three new MMPI-A Si subscales were developed originally for the MMPI-2 but can be interpreted with the MMPI-A as well. Although some of the 15 new MMPI-A content scales are similar to MMPI Wiggins (1966) scales, the Wiggins scales were changed significantly with the restandardization (Butcher et al., 1992). As a result, the MMPI-A content scales were developed from either MMPI-2 or MMPI-A data. Content scales that are unique to the MMPI-A include the Adolescent–Alienation scale, the Adoles-

cent–Low Aspirations scale, the Adolescent–Conduct Problems scale, and the Adolescent–School Problems scale.

Overview of MMPI-A Validity and Clinical Scales Traditional approaches to MMPI interpretation stressed consideration of scale combinations or profiles. At the same time, descriptions of and information on the individual scales were valuable in understanding the instrument and aided clinicians in generating hypotheses regarding interpretation. Following are descriptions of the MMPI-A validity and basic clinical scales, as well as some of the correlates that have been found with adolescent samples. The test–restest information for each scale is from that reported in the manual based on two test administrations, 1 week apart, with a subsample of 154 students from the normative sample (Butcher et al., 1992). Internal consistency coefficients of basic validity and clinical scales are provided in the manual for girls and boys separately based on results from the normative sample (Butcher et al., 1992). They are presented in the same format here. Except for the Cannot Say scale, scale scores are discussed in terms of T-scores with a mean of 50 and a standard deviation of 10. Validity Scales

Cannot Say (?) As on the MMPI, the MMPI-A Cannot Say scale is the total number of items to which the adolescent fails to respond or responds both True and False. The Cannot Say scale is not considered a formal MMPI-A scale due to the fact that it does not consist of a fixed set of items. Butcher and colleagues (1992) suggest that a protocol be considered invalid when Cannot Say scores are 30 or above. In a study by Archer and Gordon (as cited in Archer, 1997b), analyses revealed a significant relationship between Cannot Say scores and adolescents’ intellectual functioning/reading ability. Archer (1997b) concludes that high rates of nonrepsonding may often be a consequence of difficulties in reading or understanding instead of oppositional qualities. Clearly, reading ability and

reyn2-16.qxd

6/6/2003

9:24 AM

Page 375

16. The Minnesota Multiphasic Personality Inventory—Adolescent

intellectual functioning should be considered when deciding how to follow up on high Cannot Say scores.

L (Lie) The MMPI-A L scale retains 14 of the 15 items from the L scale on the MMPI, and recommended interpretations for elevations on this scale are similar to those for adults. Specifically, moderate (T-score 60–64) and high (T-score > 65) elevations may be seen as an attempt to present oneself in a positive light or to look good (Butcher et al., 1992). Archer (1997b) suggests that moderate elevations (in the 56–65 range) may indicate a tendency toward conformity. For adolescents in psychiatric settings, moderate elevations may denote a reliance on denial. Archer goes on to say that elevations of 66 and above are usually associated with extreme denial or lack of insight and sophistication. The test–restest reliability for the L scale is .61. The internal consistency reliability is .64 for boys and .58 for girls.

F, F1, and F2 (Infrequency) The F scale on the MMPI contained 64 statements reflecting unusual symptoms, ideas, or behaviors. Items were selected for the F scale if fewer than 10% of the original normative adult sample responded in the scored direction (Archer, 1997b). High F scores, then, alerted clinicians to possible validity concerns due to the rarity with which these items were typically endorsed in the deviant direction. However, adolescents tended to respond in the scored direction to many of these items much more frequently than adults. For example, approximately 26% of adolescents responded “True” to a statement about comics being the only interesting section of the news (Archer, 1997b). As a result, higher MMPI F scale scores were common among adolescents, and interpreting an elevated F scale score for adolescents was often problematic. Because of these issues, significant changes were made for the MMPI-A F scale. The MMPI-A F scale consists of 66 items that were endorsed in the scored direction by no more than 20% of adolescents in the normative sample (Butcher et al., 1992). Twenty-seven items were deleted from the

375

MMPI F scale, and 17 items on the MMPIA F scale are unique to that instrument. In terms of interpretation, Archer (1997b) notes that F scores in the moderate range (60–65) are typically associated with some signs of psychopathology. For scores in the marked range (66–89), Archer advises that all validity indicators be evaluated for possible patterns of an invalid profile. Scores in the extreme range (>90) often reflect an invalid protocol due to carelessness or faking bad. As always, decisions of validity should be make by considering other validity indicators. If the protocol is deemed valid, a score in this range may be a sign of an adolescent with severely disorganized thinking or psychotic symptomology. The test–restest correlation for the F scale is .55, and the internal consistency coefficient is .90 for boys and .82 for girls. In the normative sample the F scale correlated relatively highly with the Sc scale (.70 for males and females). The F scale is divided into two subscales, F1 and F2. Each subscale contains 33 items. All items on F1 are contained in the first 350 MMPI-A items, while those for F2 begin with item 242 and are found on the latter part of the instrument. Consequently, scores on the F1 scale can be used to evaluate response patterns for the basic scales, and scores on F2 can be used to evaluate response patterns for the supplementary and content scales. Comparison of the two scales can alert clinicians to individuals who change their approach to responding in the later part of the test. For instance, clinicians can hope to recognize adolescents who become tired or inattentive toward the end of the test and who begin responding carelessly.

K (Defensiveness) The MMPI-A K scale is composed of the 30 items from the MMPI K scale. These items were originally selected for the MMPI because of their ability to identify adults in psychiatric settings with significant pathology who produced normal MMPI profiles (Butcher et al., 1992). On the MMPI, the K scale was created as a correction for scores on five of the clinical scales. However, the K-correction procedure was never applied to adolescent MMPI profiles. A study by Alperin, Archer, and Coates (1996) com-

reyn2-16.qxd

376

6/6/2003

9:24 AM

Page 376

V. OBJECTIVE METHODS

pared MMPI-A standard norms versus MMPI-A norms with a K-correction in their ability to predict membership in the normative versus clinical samples. The authors found that use of the K-correction procedure did not substantially improve hit-rates for group membership. Alperin and colleagues conclude that K-correction procedures are not warranted for use with the MMPI-A. Butcher and colleagues (1992) recommend that the MMPI-A K scale be used as a basic validity indicator much like the L scale. They suggest that T-scores above 65 may reflect a defensive stance on the test. The K scale test–restest coefficient is .75. The internal consistency is .72 for boys and .70 for girls.

VRIN (Variable Response Inconsistency) and TRIN (True Response Inconsistency) Both the VRIN and TRIN scales were developed for the MMPI-2 and were adapted for the MMPI-A (Butcher et al., 1992). These two scales provide information on the degree to which an individual responds in a consistent manner to statements on the instrument. The VRIN scale is made of 50 pairs of statements that have either similar or opposite meanings. A raw score point is added to the VRIN total for each pair that is answered inconsistently. Thus, an elevated VRIN score is a sign that the adolescent may have responded to items carelessly, and the protocol may be invalid. A high VRIN score in conjunction with an elevated F scale score provides additional support to the conclusion that the adolescent responded randomly or carelessly. In the case of a high F score with a low VRIN, carelessness can be ruled out, and an interpretation of faking bad or actual pathology is more likely. The TRIN scale also consists of pairs of items, but on this scale, the pairs all have opposite meanings. This scale was created to identify protocols on which the individual tended to respond with a consistent True or False throughout. On the printout from the National Computer Systems (NCS) computer scoring, TRIN scale Tscores are followed by a “T” or “F” to indicate the direction in which the adolescent responded. Although research on the VRIN and TRIN scales has shown them to be useful in

the assessment of adults, there are limited data focusing on their use with adolescents. As a result, Butcher and colleagues (1992) warn that caution is warranted in their interpretation for this population. Butcher and colleagues do, however, offer the suggestion that profiles with VRIN or TRIN Tscores above 75 are probably characterized by significant inconsistency. Of course, scores on these scales that are within acceptable ranges do not necessarily mean the protocol is valid. A VRIN score within the acceptable range may mean the adolescent responded consistently but not necessarily accurately (Archer, 1997b). Furthermore, a TRIN score can be in the acceptable range even when an individual answers the items randomly. Clinical Scales

Scale 1 (Hs: Hypocondriasis) Scale 1 consists of 32 items, all of which are from Scale 1 on the MMPI. One item was dropped from the original 33-item MMPI scale for the MMPI-A. Scale 1 was initially developed to identify patients who reported symptoms indicative of the label hypochondriasis (McKinley & Hathaway, 1940). These symptoms include a preoccupation with illness, bodily health, and physical complaints. According to Graham (1987), the items on scale 1 appear to be the most homogenous of those on any MMPI basic scale, and Harris and Lingoes did not create subscales for scale 1. Items on this scale cover a variety of physical symptoms such as nausea, vomiting, numbness, dizziness, and chest pain. They range from mild and vague to specific and more severe (Butcher et al., 1992). Descriptors for individuals who have high scale 1 scores are a likelihood to respond to stress with somatic complaints, a likelihood to show little insight in therapy, and a reduced likelihood to take part in delinquent behaviors (Archer, 1997b). As one might expect, research has also documented elevations on scale 1 for adolescents with actual medical conditions, including epilepsy (Dodrill & Clemmons, 1984) and muscular dystrophy (Harper, 1983). When interpreting elevations on scale 1, then, clinicians should rule out the existence of a true medical problem. The test–restest reliability

reyn2-16.qxd

6/6/2003

9:24 AM

Page 377

16. The Minnesota Multiphasic Personality Inventory—Adolescent

for scale 1 is .79, and the internal consistency is .78 for boys and .79 for girls (Butcher et al., 1992).

Scale 2 (D: Depression) The MMPI-A scale 2 consists of 57 items from the original 60 items on the MMPI. Scale 2 was originally designed to evaluate symptoms of depression, including hopelessness and a general dissatisfaction with life (Hathaway & McKinley, 1942). The item content of this scale is broad and includes statements related to anxiety, sleep problems, health problems, teasing animals, self-esteem, concentration, and apathy. Archer, Gordon, Giannetti, and Singles (1988) report that clinicians found inpatient adolescents with MMPI scale 2 elevations as more likely to engage in therapy, to discuss their feelings, and to be more introspective. These adolescents were also seen as less impulsive, less hostile, and less likely to engage in rebellious behavior than other adolescent inpatients (Archer et al., 1988). In this study, adolescents with a scale 2 elevation were more likely to receive diagnoses of dysthymia or major depression (Archer et al., 1988) Harris and Lingoes (as cited in Butcher et al., 1992) created five subscales from scale 2 items. Butcher and colleagues (1992) recommend that clinicians use the subscale scores to refine their interpretations. The test–restest correlation for scale 2 is .78, while the internal consistency is .65 for boys and .66 for girls. These lower internal consistency coefficients may be a result of the diverse item content on the scale (Kamphaus & Frick, 2002).

Scale 3 (Hy: Hysteria) Scale 3 on the MMPI-A consists of the same 60 items as those found on scale 3 of the MMPI. The scale was created to identify patients who responded to stress with hysterical reactions (Butcher et al., 1992). A traditional definition of a hysterical reaction is a reaction characterized by loss of feeling or motor disorders that cannot be attributed to an organic cause. The items on scale 3 fall into two general categories (Greene, 1980). Items in one set relate to somatic concerns, while the other set indicates a sense of social well adjustment. According to the study by

377

Archer and colleagues (1988), therapists perceived inpatient adolescents who scored high on scale 3 as more inclined to respond to stress or anxiety with somatic complaints, more dependent, and more likely to vary their behavior to meet social expectations. These adolescents were also found to be more concerned about immediate needs and less likely to search for personal meaning in their lives. Butcher and colleagues (1992) report that girls in the MMPI-A clinical sample evidenced an association between scale 3 scores and somatic complaints. The Harris and Lingoes (as cited in Butcher et al., 1992) subscales for scale 3 are Denial of Social Anxiety, Need for Affection, Lassitude– Malaise, Somatic Complaints, and Inhibition of Aggression. The test–restest value is .70, and the internal consistency values are .63 for boys and .55 for girls. Also of note are the correlations among scales 1, 2, and 3 for both boys and girls. A number of the items on these three scales appear on two or more of the scales, and this is most likely a reason for the degree of correlation among the scales. The correlations for all scales are presented in Table C-1 of the MMPI-A: Manual for Administration, Scoring, and Interpretation (Butcher et al., 1992).

Scale 4 (Pd: Psychopathic Deviate) Scale 4 consists of 49 items. All the MMPI scale 4 items save one were used for the MMPI-A. The scale was developed to identify individuals with psychopathic personality (McKinley & Hathaway, 1944). Psychopathic personality is now labeled antisocial personality disorder. The individuals in McKinley and Hathaway’s (1944) criterion group exhibited behaviors such as lying, stealing, sexual promiscuity, truancy, and excessive drinking. Items on the scale include statements about family conflicts, wanting to leave home, a history of stealing, and unhappiness. Archer (1997b) notes that normal adolescents tend to endorse more scale 4 items than do adults, and high scale 4 scores are extremely common among adolescents in clinical settings. For example, in a study by Archer, Gordon, and Klinefelter (as cited in Archer, 1997b), 48% of adolescents in clinical settings produced a 2-point codetype in which one of the two elevations was on scale 4. Archer and colleagues (1988) found that

reyn2-16.qxd

6/6/2003

9:24 AM

Page 378

378

V. OBJECTIVE METHODS

inpatient adolescents with scale 4 elevations had a higher incidence of problems involving alcohol and drug abuse and poor judgment than did other inpatient adolescents. Therapists described these adolescents as unmotivated in therapy, and staff rated them as more hostile, rebellious, and incapable of learning from mistakes than other inpatients. Butcher and colleagues (1992) report that in the normative sample, scale 4 scores were associated with many problems in school including course failure, disciplinary infractions, and suspension from school. In the clinical sample, scale 4 scores were also correlated with ratings of delinquent behavior, aggressive behavior, and behavior problems totals (Butcher et al., 1992). The test–restest reliability for scale 4 is .80, and the internal consistency reliability is .63 for boys and .68 for girls.

Scale 5 (Mf: Masculinity–Femininity) Scale 5 on the MMPI was intended to identify homosexual males (Graham, 1987). Due to difficulties in obtaining an homogenous criterion group, however, the original criterion group consisted of only 13 homosexual males (Hathaway, 1956). For this and other reasons, interpretation of this scale has generated considerable debate and controversy, particularly in recent years (Archer, 1997b). Sixteen items were dropped from the original MMPI scale to create scale 5 on the MMPI-A. The resulting 44-item scale covers a range of topics including career interests, recreational interests, and relationships. As is the case in the MMPI-A: Manual for Administration, Scoring, and Interpretation (Butcher et al., 1992), descriptors associated with high or low scores on this scale are usually given separately for males and females. At the same time, Butcher et al. warn that because of inconsistencies in research findings, caution is warranted in interpreting this scale with girls. The test–restest correlation for scale 5 is .82. The internal consistency coefficients are .43 for boys and .40 for girls.

Scale 6 (Pa: Paranoia) All 40 of the items from the MMPI scale 6 were retained for this scale on the MMPI-A. The scale was designed to identify individu-

als deemed to have paranoid symptoms such as suspiciousness, feelings of persecution, ideas of reference, and grandiose selfconcepts (Hathaway, 1956). Some of the scale 6 items assess obvious psychotic symptoms, but other items cover subjects such as interpersonal sensitivity, moral virtue, and rigidity (Graham, 1987). As Graham (1987) points out, then, it is possible to obtain an elevated T-score without endorsing any of the statements relating to overtly psychotic symptoms. Lacher and Wrobel (as cited in Archer, 1997b) report that outpatient adolescent males who scored high on this scale were more likely to be described as distrustful and suspicious and more prone to evidence delusions of persecution. For males in the MMPI-A clinical sample, high scores were associated with being rated as hostile/withdrawn and as having neurotic/dependent behavior (Butcher et al., 1992). Archer observes that more research is needed concerning scale 6 descriptors for girls. Harris and Lingoes (as cited in Butcher et al., 1992) designated three subscales for scale 6, Persecutory Ideas, Poignancy, and Naivete. The test–restest reliability is .65, while the internal consistency is .57 for boys and .59 for girls.

Scale 7 (Pt: Psychasthenia) Scale 7 is composed of the same 48 items that formed scale 7 on the MMPI. The scale was originally developed to assess the constellation of symptoms labeled psychasthenia (Graham, 1987). The current diagnostic label most closely resembling psychasthenia is obsessive–compulsive disorder. Because this pattern of symptoms is relatively rare among inpatient populations, the original criterion group consisted of only 20 individuals (Archer, 1997b). The content of scale 7 includes statements about worries, health concerns, unhappiness, low self-esteem, and obsessive thoughts. Scale 7 high points were found to be relatively rare among adolescents in clinical settings (Archer, 1997b), but Butcher and Williams (1992) report that scale 7 elevations (>65), but not necessarily high points, were relatively common in the clinical sample (14% of boys and 19% of girls). According to Lachar and Wrobel (as cited in Archer, 1997b), outpatient adolescents who produce high scale 7 scores were

reyn2-16.qxd

6/6/2003

9:24 AM

Page 379

16. The Minnesota Multiphasic Personality Inventory—Adolescent

described as tense, nervous, anxious, restless, and excessively critical of themselves. Harris and Lingoes did not develop subscales for scale 7. The test–restest correlation for scale 7 is .83. The internal consistency values are .84 for boys and .86 for girls.

Scale 8 (Sc: Schizophrenia) Scale 8 has 77 items, which makes it the largest scale. One item was deleted from the original MMPI scale 8 for the MMPI-A. The scale was developed to identify patients diagnosed with schizophrenia (Hathaway, 1956). The items on scale 8 include statements relating to bizarre thought processes, unusual perceptions, social isolation, difficulties with impulse control, and disturbances in mood and behavior. Inpatient adolescents who score high on scale 8 were described as withdrawn, mistrustful, and vulnerable to stress (Archer et al., 1988). Staff reported that these adolescents tended to be interpersonally isolated and less adaptable to hospital expectations. In addition, these adolescents frequently displayed poor reality testing (Archer et al., 1988). Wrobel and Lachar (1992) found that adolescents with high scale 8 score were often made fun of or rejected by their peers. Archer (1997b) observes that scale 8 elevations may also reflect an adolescent’s experience with drugs, especially hallucinagens, and this should be considered in the interpretive process. Harris and Lingoes (as cited in Butcher et al., 1992) created six subscales for scale 8: Social Alienation; Emotional Alienation; Lack of Ego Mastery, Cognitive; Lack of Ego Mastery, Conative; Lack of Ego Mastery, Defective Inhibition; and Bizarre Sensory Experiences. For both boys and girls, this scale is highly correlated with scales 6 (.71 for boys and girls) and 7 (.83 for boys and .85 for girls) (Butcher et al., 1992). In part, these correlations are probably the result of items that scale 8 shares with scales 7 and/or 6. The test–restest reliability for scale 8 is .83, while the internal consistency reliability is .88 for boys and .89 for girls.

379

MMPI. The scale was designed to identify patients exhibiting hypomanic symptoms such as elevated and unstable mood, heightened motor activity, flight of ideas, irritability, and egocentricity (McKinley & Hathaway, 1944). The items on scale 9 cover these symptom areas. Adolescents who score high on scale 9 have been reported to be more likely to manifest impulsivity, insensitivity to criticism, unrealistic and grandiose goals, and rapidly changing moods (Archer, 1997b; Archer et al., 1988). There are four Harris and Lingoes (as cited in Butcher et al., 1992) subscales for scale 9. They are Amorality, Psychomotor Acceleration, Imperturbability, and Ego Inflation. The test–restest correlation is .70, and the internal consistency coefficient is .61 for boys and .79 for girls.

Scale 0 (Si: Social Introversion) Eight items were dropped from scale 0 on the MMPI to form the 62-item scale on the MMPI-A. As noted in the section on the MMPI, this scale was originally created after the other clinical scales, but it is considered one of the basic clinical scales (Graham, 1987). Items for scale 0 were selected by Drake (1946) based on their relationship to a measure of social extroversion/introversion. Higher scale 0 scores are associated with higher social introversion. Item content includes poor sociability, unhappiness, lack of interest in parties, shyness, difficulty making friends, and loneliness. Wrobel and Lachar (1992) report that scale 0 scores were correlated with parents’ ratings of their children as being very shy, having few or no friends, and tending to avoid relationships with their peers. Harris and Lingoes did not create subscales for scale 0, but three subscales were created for the MMPI2 and the MMPI-A based on factor-analytic results (Butcher et al., 1992). The three subscales are Shyness/Self-consciousness, Social Avoidance, and Alienation-Self and Others. The scale 0 test–restest correlation is .84, and the reliability coefficients are .79 for males and .80 for females.

Scale 9 (Ma: Hypomania)

MMPI-A Special Scales

Scale 9 on the MMPI-A consists of the same 46 that were used for this scale on the

As stated previously many, many special scales were created from MMPI items.

reyn2-16.qxd

6/6/2003

9:24 AM

Page 380

380

V. OBJECTIVE METHODS

These special scales were intended to expand on the interpretive information from the basic scales. On the MMPI-A, scores are available for 6 supplementary scales and 15 content scales. Three of the six supplementary scales were created for the MMPI-A, and the research on these scales remains limited. Likewise, more research is needed with the content scales also. Supplementary Scales

A (Welsh’s Anxiety) Welsh (1956) developed the original A scale to reflect the first dimension derived from factor-analytic studies with the MMPI. Thirty-five items from the original scale have been retained for the MMPI-A (Butcher et al., 1992). Archer, Gordon, Anderson, and Giannetti (1989) found that nursing staff ratings and self-reports for inpatient adolescents who scored high on the A scale indicated substantially more maladjustment than was found among other inpatients. Adolescents with high A scores were described as anxious, self-critical, overwhelmed, and fearful (Archer et al., 1989). High A scores in this study were also associated with higher rates of suicidal ideation and suicide attempts. The test–restest coefficient for the A scale is .81, and the internal consistency values are .87 and .89 for males and females respectively.

R (Welsh’s Repression) The R scale originated with the second factor found in MMPI factor-analytic studies (Welsh, 1956). The MMPI-A R scale consists of 33 of the original 40 items (Butcher et al., 1992). Inpatient adolescents with high R scores were described as inhibited, overcontrolled, and less likely to have problems with drug abuse (Archer et al., 1989). The R scale test–restest correlation is .65, while the internal reliability is .65 for boys and .52 for girls.

MAC-R (MacAndrew Alcoholism Scale—Revised) MacAndrew (1965) created the MAC scale from MMPI items that demonstrated effectiveness in distinguishing male alcoholics

from male psychiatric outpatients. Of the 49 items on the MMPI-A MAC-R scale, 45 are from the original scale. The MMPI MAC scale received considerable research attention with adolescent samples, and findings from this research generally supported the association between elevated MAC scores and substance abuse among adolescents (Archer, 1997b). Results from a study by Archer and colleagues (1989) indicate that adolescents with elevated MAC scores were described as undercontrolled, self-indulgent, assertive, and independent. These same adolescents were more likely to have a diagnosis of conduct disorder and to have an arrest record. High MAC scores have also been associated with certain profile codetypes (Archer & Klinefelter, 1992). For example, 79% of adolescents with a 49–94 codetype in a sample of adolescent male psychiatric patients had an elevated MAC score (using a criterion of raw score >27). However, Archer and Klinefelter caution that at least part of this association is a consequence of item overlap between the MAC scale and clinical scales. Archer (1997b) notes that high MAC scores among both adolescents and adults can signify drug abuse problems as well as alcohol problems. The MAC test–restest coefficient is .47. The internal consistency of the scale is .50 for boys and .45 for girls.

ACK (Alcohol/Drug Problem Acknowledgment) This scale is unique to the MMPI-A, and was developed to determine an adolescent’s willingness to acknowledge alcohol or drug problems (Butcher et al., 1992). Some of the items on the ACK scale relate directly to the use of drugs or alcohol, while others reflect behaviors or attitudes that correlate with drug use (Archer, 1997b). The test–restest correlation for the scale is .56, and the internal consistency coefficients are .64 (boys) and .66 (girls).

PRO (Alcohol/Drug Problem Proneness) The 36 items that comprise the PRO scale were selected due to their ability to distinguish adolescents in alcohol and drug treatment programs from adolescents receiving mental health services who did not have drug or alcohol problems (Butcher et al.,

reyn2-16.qxd

6/6/2003

9:24 AM

Page 381

16. The Minnesota Multiphasic Personality Inventory—Adolescent

1992). This scale, then, was created in much the same manner as the original MAC scale and is intended to identify drug and alcohol problems in adolescents (Archer, 1997b). Items on the PRO scale cover diverse issues such as negative peer influence, conflict with parents, bad judgment, and rule violation (Butcher et al., 1992). The test–restest correlation is .70, while the internal consistency is .63 for boys and .61 for girls.

IMM (Immaturity) Butcher and colleagues (1992) report that Archer, Pancoast, and Gordon developed the IMM scale for the MMPI-A. The scale was created out of a concern for the role of maturation in understanding adolescent psychopathology (Archer, Pancoast, & Gordon, 1994). The scale consists of 43 items that were selected using both rational and empirical methods. The items address issues such as externalization of blame, self-centeredness, living for the moment, and lack of insight (Archer, 1997b). Archer and colleagues (1994) report that higher IMM scores were associated with problems such as poor academic grades, school suspensions, and negative life events for adolescents in the nonclinical sample. Inpatient adolescent girls with elevated scores were more likely to be seen as defiant and angry, while inpatient boys were described as loud, threatening, and tending to bully others. For both male and female inpatients, higher IMM scores were associated with higher ratings of externalizing behavior (Archer et al., 1994). Findings from Imhof and Archer (1997) suggest that adolescents with higher IMM scores are less able to think in abstract terms and have difficulty perceiving the world from another’s point of view. The IMM test–restest correlation is .74, and the internal reliabilities are .80 (boys) and .82 (girls). Content Scales Of the 15 content scales on the MMPI-A, 11 were developed for the MMPI-2 and have a parallel scale on the MMPI-2. To differentiate them, all the content scales on the MMPI-A begin with Adolescent- followed by the scale name. The 11 MMPI-A scales also found in analogous format on the

381

MMPI-2 are: Adolescent-Anxiety (A-anx), Adolescent-Obsessiveness (A-obs), Adolescent-Depression (A-dep), Adolescent-Health Concerns (A-hea), Adolescent-Bizarre Mentation (A-biz), Adolescent-Anger (A-ang), Adolescent-Cynicism (A-cyn), AdolescentLow Self-Esteem (A-lse), Adolescent-Social Discomfort (A-sod), Adolescent-Family Problems (A-fam), and Adolescent-Negative Treatment Indicators (A-trt). The degree of overlap in items and content between the MMPI-2 and the MMPI-A for these scales ranges from total to moderate. For example, all the items on the A-lse are found on the MMPI-2 Low Self-Esteem scale, but A-fam shares only 15 of its 35 items with the MMPI-2 Family Problems scale. These 11 MMPI-A content scales are intended to measure the same constructs as those found on the MMPI-2 (Butcher et al, 1992). The remaining four content scales were created for the MMPI-A. These scales are Adolescent-Alienation (A-aln), Adolescent Conduct Problems (A-con), Adolescent Low Aspirations (A-las), and Adolescent School Problems (A-sch). All the content scales were developed using a combination of rational and empirical approaches with items found throughout the 478 MMPI-A items (Williams et al., 1992). One of the rational steps was a review of scale items that were thought to be developmentally inappropriate as indicators of adolescent psychpathology/personality. The empirical steps included repeated calculation of internal reliability coefficients for the scales with removal or addition of items that improved scale reliabilities. The developers of the content scales note that their approach to creation of the scales resulted in a reduction of the item overlap that is so prevalent among the basic validity and clinical scales (Williams et al., 1992). As Archer (1997b) points out, relatively little information is available with respect to MMPI-A content scale interpretation. Two studies have been conducted that provide empirically derived correlates for the content scales. The first, by Williams and colleagues (1992), involved the original MMPI-A clinical sample of 420 males and 293 females. The second, by Archer and Gordon (as cited in Archer, 1997b), was based on a sample of 64 male and 58 female adolescent inpatients. Descriptors, derived

reyn2-16.qxd

6/6/2003

382

9:24 AM

Page 382

V. OBJECTIVE METHODS

in part from these studies, can be found in Williams and colleagues (1992) and Archer (1997b). A study by Arita and Baer (1998) examined the validity of six of the content scales in relation to other measures designed to measure similar constructs. Arita and Baer found general support for the validity of the content scales, except for the A-anx and A-dep scales. They conclude that these scales did not discriminate well between the constructs of anxiety and depression. They observe, however, that none of the other measures used in their study were likely to make this distinction either.

SCORING Several options are available for scoring the MMPI-A. Clinicians can handscore the instrument, score it on the computer, or use mail-in scoring. When using any scoring procedure, the first step is to examine the answer sheet and check for unmarked items or items to which the adolescent responded True and False. To handscore the MMPI-A, clinicians need scoring templates for the scales, a VRIN/TRIN grid, and profile forms. These materials can all be purchased through National Computer Systems. To obtain raw score values for the scales, the clinician uses the scoring templates for each scale. Scoring templates are placed over the answer sheet in order to count responses. Once the raw scores have been determined, the clinician plots the raw scores on a profile sheet that provides corresponding T-score values. Tscore patterns and the profiles that emerge can then be interpreted. The handscoring procedure is slightly different for the VRIN and TRIN scales. To score these two scales, the clinician uses the VRIN or TRIN recording grid and transfers item responses for these scales from the answer sheet to the appropriate place on the recording grid. Directions for scoring these two scales, which are more complicated to score, are presented on the recording grids. For software scoring, NCS offers three score report choices. The Basic Score Report gives raw and T-score values for the basic validity and clinical scales. The Extended Score Report provides scores for the validity and basic clinical scales, the Harris–Lingoes subscales, the supplementary scales, and the content scales. The interpretive report in-

cludes scale scores as well as a narrative that describes symptom patterns and possible diagnostic and treatment considerations. Of course, the interpretive report is based solely on responses to the MMPI-A and incorporates no other information about an adolescent.

Interpetation Interpretation of the MMPI-A can be complex even for experienced users, and this chapter presents merely an overview of suggested procedures for interpretation. More comprehensive recommendations for interpretation can be found in sources such as the MMPI-A: Manual for Administration, Scoring and Interpretation (Butcher et al., 1992), Archer (1997b), Archer and Krishnamurthy (2002), and Butcher and Williams (2000). As is the case with any assessment data, the first step in interpreting the MMPI-A is consideration of additional information the clinician has about the adolescent. Additional information can include family history, cognitive abilities, health history, ethnic background, socioeconomic background, and history of psychiatric symptomology. All these factors can influence an individual’s responses to the MMPI-A. In the next step, the clinician evaluates the validity of the profile, based on the validity scales and indicators. The Cannot Say value, the VRIN, TRIN, F, L, and K scale scores, as well as the patterns of these scores, should all be reviewed. Once the validity has been assessed, the clinician turns to the 10 basic clinical scales and decides on the most useful approach for interpreting the clinical scores. For the clinical scales, Butcher and colleagues (1992) advise that T-score values of 60–64 be regarded as moderate elevations. According to Butcher and colleagues moderate elevations can be interpreted, though with less confidence than scores > 64. A T-score > 64 is the suggested level for determining clinically significant elevations (Butcher et al., 1992). If only one clinical scale is elevated, the clinician will focus on single-scale interpretation (Archer & Krishnamurthy, 2002). In this case, the descriptors and correlates associated with the elevated scale are considered for interpretation. The higher the scale’s elevation and the more it is elevated in relation to the remaining scales, the more

reyn2-16.qxd

6/6/2003

9:24 AM

Page 383

16. The Minnesota Multiphasic Personality Inventory—Adolescent

likely it will be that the descriptors are appropriate. At the same time, it is important to bear in mind that all descriptors and correlates are probabilistic statement that may or may not apply to an individual client (Greene, 1980). When more than one scale is elevated, the clinician has the choice of using a single-scale or a codetype approach. The codetype approach is like that used with the MMPI and discussed in the MMPI section of this chapter. Simply, the 2-point codetype method takes into account the pattern of scores and particularly the two highest clinical scales. Descriptors and correlates for codetypes are derived from individuals who produce a similar pattern or codetype. A 2-point codetype should be considered only if both scales are in the clinically significant range. One can also use the singlescale method with two scale elevations. In this case, the clinician takes into account a combination of the descriptors or correlates found with each individual scale, placing greater emphasis on the more elevated of the two scales. When more than two clinical scales are significantly elevated in a profile, Archer and Krishnamurthy (2002) recommend using a combination of 2-point codetype and individual scale interpretation. Following the review of clinical scales, the clinician can refine his or her interpretations through consideration of the Harris–Lingoes and Si subscales. As Archer and Krishnamurthy (2002) point out, subscales should be considered only for clinical scales with elevations in the clinically significant range. Further information can also be obtained from the supplementary and content scales and integrated into the final interpretation. The supplementary scales can contribute information regarding the adolescent’s overall level of maladjustment (the A scale), as well as information about alcohol and drug abuse problems (MAC-R, ACK, and PRO). The content scales can also provide insight into the adolescent’s functioning. However, Archer and Krishnamurthy advise that until more research on the content scales is available, content scales should interpreted only in light of findings from the basic scales. Archer and Krishnamurthy (1994) offer another approach to interpretation that can be used in conjunction with the steps described previously or as an alternative. The Structural Summary approach is based on

383

the factor-analytic results of the MMPI-A scales. A factor-analytic study by Archer, Belevich, and Elkins (1994) revealed eight dimensions, or factors, underlying the 69 MMPI-A scales and subscales. Archer and colleagues labeled these factors General Maladjustment, Immaturity, Disinhibition/ Excitatory Potential, Social Comfort (also called Social Discomfort), Health Concerns, Naivete, Familial Alienation, and Psychoticism. Archer and Krishnamurthy provide descriptors and correlates for each of the eight scales. As Archer and Krishnamurthy point out, the advantage of this approach is that it substantially reduces the redundancy often found among the MMPI-A scales and subscales and permits a more organized and succinct means of interpretation. Of course, further research is needed to evaluate this approach to interpretation and description of functioning. Before leaving the topic of interpretation, it is important to note that the most common profile for adolescents, even those with known psychological or behavioral problems, is the “no-code” or “within-normallimits” (WNL) profile (Archer & Krishnamurthy, 2002). A WNL profile is one in which all clinical scales are within normal limits. In fact, Archer and Krishnamurthy report that between 20 and 33% of adolescents in mental health settings produce profiles with no elevations in the clinically significant range. Similarly, Archer and Jacobson (1993) found that adolescents from the normative sample endorsed so-called critical items at almost the same rate as adolescents from the clinical sample. These findings highlight the importance integrating information from a variety of measures and sources in making clinical interpretations and in describing an adolescent’s functioning (Archer & Krishnamurthy, 2002).

CONCLUSIONS A 1998 survey documented the popularity of the MMPI-A among psychologists who work primarily with adolescents (Archer & Newsom, 2000). Among the 30 most frequently reported assessment instruments, the MMPI-A ranked fifth. Moreover, the MMPI-A was the only objective, self-report personality measure listed in the top 10.

reyn2-16.qxd

6/6/2003

384

9:24 AM

Page 384

V. OBJECTIVE METHODS

Thus, the MMPI-A continues in the tradition of its predecessor, the MMPI, as a popular tool in the assessment of adolescent functioning.

Strengths Respondents to the 1998 survey (Archer & Newsom, 2000) were also asked to list what they saw as the strengths of the new instrument. Among the responding clinicians, the most frequently mentioned strength was the MMPI-A’s ability to provide a comprehensive clinical description for adolescents (Archer & Newsom, 2000). The MMPI-A assesses a wide range of symptoms and areas of functioning, and this is clearly one of its primary advantages. Clinicians also recognized the value of the updated normative sample (Archer & Newsom, 2000). These norms are nationally representative, and they include minority populations. As mentioned previously, however, there are a few shortcomings. The level of parental education is high, and the actual number of adolescents from minority groups is small. All the same, the normative sample meets contemporary standards for assessment instruments and is an obvious improvement over the status of adolescent norms for the MMPI. Clinicians who responded to the survey noted ease of administration and psychometric soundness as two further strengths. Many of the newer scales, particularly the content scales, have excellent internal consistency. Though not mentioned by clinicians, another major asset of the MMPI-A is the ability to check for response biases and evaluate profile validity. The MMPI-A includes the validity scales that were strengths with the MMPI, namely, F, L, and K. Furthermore, the new VRIN and TRIN scales show promise for identifying inconsistent responding or a tendency for acquiescent/nonacquiescent responding. Last but not least is the strength of the MMPI-A’s foundation in the MMPI and the research base afforded by decades of MMPI investigations. As Archer (1997b) concludes, although the MMPI-A is not a direct replica of the MMPI, the degree of similarity between the instruments allows one to generalize the decades of research findings from the MMPI to the MMPI-A. Thus it is that many of the descriptors and correlates associated with MMPI-A clinical scales are

derived from a combination of research with the MMPI-A as well as numerous studies with the MMPI.

Weaknesses While the MMPI-A’s foundation in the MMPI is one of its greatest strengths, it is simultaneously one of its liabilities. In an effort to gain as much consistency as possible, most of the basic clinical scales were retained with little modification. As a result, several of these scales evidence low internal consistency, particularly when judged by today’s standards (Black, 1994). Furthermore, numerous items are found on more than one scale, and this, in turn, contributes to higher correlations among scales. High correlations are found among some of the clinical scales, as well as between some content and clinical scales. Both of these issues can lead to uncertainty and confusion in the process of interpretation. Also, as Claiborn (1995) points out, elevations on scale 5 are difficult to interpret due to the few and contradictory correlates found across studies. Another limitation of the MMPI-A is highlighted in a recent article by Archer, Handel, and Lynch (2001). These authors examined the ability of MMPI-A items to discriminate between normative and clinical samples. Archer and colleagues found that a significant number of items on both the basic and content scales do not demonstrate differential rates of endorsement between clinical and normative groups. This could be one explanation for the relatively high percentage of WNL basic profiles for adolescents receiving mental health services. According to Archer et al. the only two scales with a majority of items that did discriminate effectively were the Pd scale and the content scale, A-fam. Moreover, Alperin and colleagues (1996) found that neither K-correction procedures nor a reduction of the critical T-score value from 65 to 60 improved classification rates between the normative sample and a clinical sample. Clearly, these findings of limited sensitivity in identifying adolescent pathology seem a significant liability for the MMPI-A and warrant further investigation.

Future Directions In spite of its legacy with the MMPI, the MMPI-A is still a relatively new instrument.

reyn2-16.qxd

6/6/2003

9:24 AM

Page 385

16. The Minnesota Multiphasic Personality Inventory—Adolescent

As such, many areas and issues remain to be investigated and addressed. Archer (1997a) proposes six potential areas for inquiry in his article on future directions for the MMPI-A. A few of these are discussed here. One area for continuing research is the codetype approach to interpretation. This became the most accepted approach for MMPI interpretation, and the congruence between the MMPI and the MMPI-A appears to justify the generalization of research findings from the MMPI to the MMPI-A (Archer, 1997a). At the same time, the degree to which the codetype method of interpretation is applicable for the MMPI-A and adolescent samples remains open to question (Archer, 1997a). Furthermore, additional work with the actual MMPI-A, as opposed to the MMPI, in developing codetype correlates and descriptors is called for. Another issue for consideration is the utility of the MMPI-A Structural Summary for interpretive purposes. Because the Structural Summary is based on the factor structure or underlying dimensions of all 69 MMPI-A scales, it seems a natural approach to interpretation. However, basic questions remain. For example, research has not yet provided support for determining cutoff points for the eight factors (Archer, 1997a). In other words, how or in what range do we consider a factor to be elevated and worthy of interpretation. Finally, the issue of the MMPI’s accuracy in describing adolescent functioning and the presence of adolescent psychopathology deserves further attention. An obvious question related to the Archer and colleagues (2001) results is, Why don’t the MMPI-A items discriminate well when items from the MMPI do tend to identify pathology among adult populations? Archer and his colleagues suggest that one factor may be higher item endorsement rates for normal adolescents. Mean raw score comparisons appear to support this explanation. An alternative possibility to be explored is that adolescents in clinical settings report less symptomology than do adults (Archer et al., 2001). Given the findings of Archer et al., another direction for future research could be the development of a shorter form of the MMPI-A where item deletions are determined by item effectiveness. In conclusion, the MMPI-A is a popular instrument for the assessment of adolescents. The developers have clearly improved

385

many of the problems that were inherent in using the MMPI with adolescents, while maintaining many of the advantages of the original measure. Part of the MMPI-A’s heritage in the MMPI is a rich base of decades of research that has guided clinical applications. Researchers’ ongoing efforts to build a similar foundation for the MMPI-A will contribute to the continued use and popularity of this instrument.

REFERENCES Alperin, J. J., Archer, R. P., & Coates, G. D. (1996). Development and effects of an MMPI-A K-correction procedure. Journal of Personality Assessment, 67, 155–168. Archer, R. P. (1987). Using the MMPI with adolescents. Hillsdale, NJ: Erlbaum. Archer, R. P. (1997a). Future directions for the MMPIA: Research and clinical issues. Journal of Personality Assessment, 68, 95–109. Archer, R. P. (1997b). MMPI-A: Assessing adolescent psychopathology (2nd ed.). Mahwah, NJ: Erlbaum. Archer, R. P., Belevich, J. K. S., & Elkins, D. E. (1994). Item-level and scale-level factor structures of the MMPI-A. Journal of Personality Assessment, 62, 332–345. Archer, R. P., Gordon, R. A., Anderson, G. L., & Giannetti, R. A. (1989). MMPI special scale clinical correlates for adolescent inpatients. Journal of Personality Assessment, 53, 654–664. Archer, R. P., Gordon, R. A., Giannetti, R. A., & Singles, J. M. (1988). MMPI scale clinical correlates for adolescent inpatients. Journal of Personality Assessment, 52, 707–721. Archer, R. P., Handel, R. W., & Lynch, K. D. (2001). The effectiveness of MMPI-A items in discriminating between normative and clinical samples. Journal of Personality Assessment, 77, 420–435. Archer, R. P., & Jacobson, J. M. (1993). Are critical items “critical” for the MMPI-A? Journal of Personality Assessment, 61, 547–556. Archer, R. P., & Klinefelter, D. (1992). Relationships between MMPI codetypes and MAC scale elevations in adolescent psychiatric samples. Journal of Personality Assessment, 58, 149–159. Archer, R. P., & Krishnamurthy, R. (1994). A structural summary approach for the MMPI-A: Development and empirical corelates. Journal of Personality Assessment, 63, 554–573. Archer, R. P., & Krishnamurthy, R. (2002). Essentials of MMPI-A assessment. New York: Wiley. Archer, R. P., Maruish, M., Imhof, E. A., & Piotrowski, C. (1991). Psychological test usage with adolescent clients: 1990 survey findings. Professional Psychology: Research and Practice, 22, 247–252. Archer, R. P., & Newsom, C. R. (2000). Psychological test usage with adolescent clients: Survey update. Assessment, 7, 227–235.

reyn2-16.qxd

386

6/6/2003

9:24 AM

Page 386

V. OBJECTIVE METHODS

Archer, R. P., Pancoast, D. L., & Gordon, R. A. (1994). The development of the MMPI-A Immaturity scale: Findings for normal and clinical samples. Journal of Personality Assessment, 62, 145–156. Arita, A. A., & Baer, R. A. (1998). Validity of selected MMPI-A content scales. Psychological Assessment, 10, 59–63. Black, K. (1994). A critical review of the MMPI-A. Child Assessment News, 4(2), 1, 9–12. Buros, O. K. (Ed.). (1974). Tests in print II. Highland Park, NJ: Gryphon Press. Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). Minnesota Multiphasic Personality Inventory–2 (MMPI-2): Manual for administration and scoring. Minneapolis: University of Minnesota Press. Butcher, J. N., & Williams, C. L. (1992). Essentials of MMPI-2 and MMPI-A interpretation. Minneapolis: University of Minnesota Press. Butcher, J. N., & Williams, C. L. (2000). Essentials of the MMPI-2 and MMPI-A interpretation (2nd ed.). Minneapolis: University of Minnesota Press. Butcher, J. N., Williams, C. L., Graham, J. R., Archer, R. P., Tellegen, A., Ben-Porath, Y. S., & Kaemmer, B. (1992). Minnesota Multiphasic Personality Inventory—Adolescent (MMPI-A): Manual for administration, scoring and interpretation. Minneapolis: University of Minnesota Press. Claiborn, C. D. (1995). Review of the Minnesota Multiphasic Personality Inventory—Adolescent. In J. C. Conoley & J. C. Impara (Eds.), Twelfth mental measurement yearbook (pp. 625–628). Lincoln, NE: Buros Institute of Mental Measurement. Dodrill, C. B., & Clemmons, D. (1984). Use of neuropsychological tests to identify high school students with epilepsy who later demonstrate inadequate performance in life. Journal of Consulting and Clinical Psychology, 52, 520–527. Drake, L. E. (1946). A social I.E. scale for the Minnesota Multiphasic Personality Inventory. Journal of Applied Psychology, 30, 51–54. Graham, J. R. (1987). The MMPI: A practical guide (2nd ed.). New York: Oxford University Press. Greene, R. L. (1980). The MMPI: An interpretive manual. New York: Grune & Stratton. Harper, D. C. (1983). Personality correlates and degree of impairment in male adolescents with progressive and nonprogressive physical disorders. Journal of Clinical Psychology, 39, 859–867. Hathaway, S. R. (1956). Scales 5 (Masculinity–Femininity), 6 (Paranoia), and 8 (Schizophrenia). In G. S. Welsh & W. G. Dahlstrom (Eds.), Basic readings on the MMPI in psychology and medicine (pp. 104–111). Minneapolis: University of Minnesota Press. Hathaway, S. R., & McKinley, J. C. (1940). A multiphasic personality schedule (Minnesota): I. Construction of the schedule. Journal of Psychology, 10, 249–254. Hathaway, S. R., & McKinley, J. C. (1942). A multiphasic personality schedule (Minnesota): III. The

measurement of symptomatic depression. Journal of Psychology, 14, 73–84. Imhof, E. A., & Archer, R. P. (1997). Correlates of the MMPI-A Immaturity (IMM) scale in an adolescent psychiatric sample. Assessment, 4, 169–179. Kamphaus, R. W., & Frick, P. J. (2002). Clinical assessment of child and adolescent personality and behavior (2nd ed.). Boston: Allyn & Bacon. MacAndrew, C. (1965). The differentiation of male alcoholic out-patients from nonalcoholic psychiatric patients by means of the MMPI. Quarterly Journal of Studies on Alcohol, 26, 238–246. Marks, P. A., & Briggs, P. F. (1972). Adolescent norm tables for the MMPI. In W. G. Dahlstrom, G. S. Welsh, & L. E. Dalstrom (Eds.), An MMPI handbook: Vol 1. Clinical interpretation (pp. 388–399). Minneapolis: University of Minnesota Press. Marks, P. A., Seeman, W., & Haller, D. L. (1974). The actuarial use of the MMPI with adolescents and adults. Baltimore: Williams & Wilkins. McKinley, J. C., & Hathaway, S. R. (1940). A multiphasic personality schedule (Minnesota): II. A differential study of hypochondriasis. Journal of Psychology, 10, 255–268. McKinley, J. C., & Hathaway, S. R. (1944). The MMPI: V. Hysteria, hympomania, and psychopathic deviate. Journal of Applied Psychology, 28, 153–174. Reilley, R. R. (1990). Using the Minnesota Multiphasic Personality Inventory (MMPI) with adolescents. In C. R. Reynolds & R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of children: Personality, behavior, and context (pp. 324–342). New York: Guilford Press. Reynolds, C. R., & Kamphaus, R. W. (Eds.). (1990). Handbook of psychological and educational assessment of children: Personality, behavior, and context. New York: Guilford Press. Shaevel, B., & Archer, R. P. (1996). Effects of MMPI-2 and MMPI-A norms on T-score elevations for 18year-olds. Journal of Personality Assessment, 67, 72–78. Sundberg, N. D. (1961). The practice of psychological testing in clinical services in the United States. American Psychologist, 16, 79–83. Welsh, G. S. (1956). Factor dimensions A and R. In G. S. Welsh & W. G. Dahlstrom (Eds.), Basic readings on the MMPI in psychology and medicine (pp. 264–281). Minneapolis: University of Minnesota Press. Wiggins, J. S. (1966). Substantive dimensions of self-report in the MMPI item pool. Psychological Monographs, 80(22, Whole No. 630). Williams, C. L., Butcher, J. N., Ben-Porath, Y. S., & Graham, J. R. (1992). MMPI-A content scales: Assessing psychopathology in adolescents. Minneapolis: University of Minnesota Press. Wrobel, N. H., & Lachar, D. (1992). Refining adolescent MMPI interpretations: Moderating effects of gender in prediction of descriptions from parents. Psychological Assessment, 4, 375–381.

reyn2-17.qxd

6/6/2003

9:36 AM

Page 387

17 The Behavior Assessment System for Children

JENNIFER THORPE RANDY W. KAMPHAUS CECIL R. REYNOLDS

Bedecked in a long flowered dress, Anna came skipping through the clinic door, smiling from ear to ear, with her pigtails flopping side to side. She was full of energy, had trouble taking her gaze off her own reflection in the observation mirror, and was immediately conversant with the examiner. During the parent interview, her mother explained that Anna had difficulty paying attention in school, daydreamed, and sometimes did not complete schoolwork. Despite being generally a happy child, Anna was easily upset and this tendency sometimes interfered with cooperative play with friends. Her mother said she was worried that Anna’s standardized reading scores were well below that of her older sister at her age and was concerned that she might suffer from attention-deficit/hyperactivity disorder (ADHD). Anna’s older sister had always been one of the top students in her class. Her mother reported that it was difficult to avoid comparisons of her daughters, which she said were often “inevitable.” The results of Anna’s evaluation revealed scores in the average range on the Wechsler Intelligence Scale for Children—Third Edition (WISC-III Verbal IQ = 97; WJSC-IJI Performance IQ = 91) with achievement commensurate with these estimates of cognitive ability (WJ-R Reading Composite = 99; WJ-R Math Composite = 93). Anna’s mother and father re-

ported some attentional difficulties on the Behavior Assessment System for Children that fell in the borderline range (mother’s BASCPRS T-score = 63; father’s BASC-PRS T-score = 61). However, when her mother was queried using the ADHD module of the Structured Interview for the Diagnostic Assessment for Children, Anna’s symptoms did not meet criteria for diagnosis under the fourth edition of Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Associaiton, 1994). In addition, although her teacher reported some daydreaming in class during the interview, she did not report any elevations in Attentional Problems on the BASC (TRS T-score = 56), instead endorsing items on the Anxiety scale approaching clinical significance (TRS T-score = 69). No elevations were apparent on the Achenbach Teacher Report Form, which combines anxious and depressed symptoms in a single Anxious/Depressed scale (T-score = 50). Although still in the second grade, Anna had perfect grades in school and exhibited no behavioral problems at home or in school. Her teacher also reported that Anna had good social skills relative to her peers (TRS Social Skills T-score = 61) and average scores on other Adaptive composite scales (BASC TRS T-scores: Leadership = 57; Adaptability = 51; Study Skills = 59). Such be-

387

reyn2-17.qxd

388

6/6/2003

9:36 AM

Page 388

V. OBJECTIVE METHODS

havioral strengths are unexpected in children with ADHD. On the Parenting Stress Index Anna’s mother reported elevated levels of stress associated with Anna’s moods, distractibility, and acceptability. Anna did not meet criteria for ADHD diagnosis under DSM-IV; however, her mother had valid concerns regarding her ability to remain focused on academic tasks and to follow through on chores at home. The evaluation results revealed subclinical elevations in attention problems and possibly emergent anxiety that warranted continued monitoring, reevaluation within a year, and low-level behavioral interventions but did cross the categorical threshold for ADHD diagnosis. The clinician recommended that Anna be seen again in a year to evaluate her academic progress and short-term family therapy to address behavioral problems and expectations regarding Anna’s performance relative to that of her sister.

DIMENSIONAL ASSESSMENT WITH THE BEHAVIOR ASSESSMENT SYSTEM FOR CHILDREN Every clinician has experienced the quandary of children such as Anna who present with problem symptoms that nevertheless do not warrant DSM diagnosis: a child who cannot sit still but manages to make average grades in school, a child who seems to cry at the least provocation but does not meet criteria for mood disorder, a child who appears withdrawn but does not exhibit other identifiable difficulties. Behavior rating scales, such as those included in the Behavior Assessment System for Children (BASC), provide dimensional information on whether this child lies at the low or high range of the behavioral continuum. Because many or most highly prevalent disorders of childhood represent extremes of a continuum (Fergusson & Horwood, 1995; Scahill et al., 1999) (e.g., some children are more or less anxious, sad, social, inattentive, or active than others), diagnosis and treatment are highly dependent on determining that child’s location on the continuum relative to other children at his or her developmental level. Dimensional approaches ensure that children with subthreshold impairments can be monitored. Thus, elevations or a pattern of subclinical scale elevations can be a red

flag to the clinician that the child may not meet categorical criteria but is still experiencing impairment in particular domains that require tracking over time (Cantwell, 1996).

BRIEF HISTORY The BASC represents a blend of traditional behavioral methods and contemporary cognitive-behavioral approaches. At one time, behavioral assessment only dealt with clearly observable, overt behavior. With the rise of popularity of cognitive behaviorism throughout the 1980s, reports of covert behavior such as thoughts, feelings, and desires have come to be included as a significant component of behavioral assessment (e.g., Kratochwill, Sheridan, Carlson, & Laseck, 1999). However, in the use of selfreports of internal or otherwise unobservable phenomena, behavioral assessment does not draw deep-seated psychodynamic inferences nor does it seek determinants of character; the responses are viewed directly for what they represent—samples of behavior and reports of the frequency or occurrence of specific behaviors. In contrast to early conceptualizations of the behavioral assessment paradigm, however, clinicians do now recognize the appearance of chronic, long-standing characteristics such as anxiety and locus of control that generalize beyond highly specific settings. In fact, many traditional measures in use by psychologists are now used as components of behavioral assessments but with lower levels of inference involved in the interpretation of the results. (See Ramsay, Reynolds, & Kamphaus, 2002, for a review of the many different methods of behavioral assessment.) This chapter discusses the BASC, which offers practitioners a practical tool kit with complementary scales and interviews for assessing both positive and negative behaviors across home and school settings, using different informants and methods for measuring behavior. Interpretation and synthesis of multisource information is simplified through co-normed scales consistent across age levels and teacher and parent forms. Thus, use of the full Assessment System can capitalize on empirically supported strengths in raters, such as the general superiority of teachers to rate attention and hy-

reyn2-17.qxd

6/6/2003

9:36 AM

Page 389

17. The Behavior Assessment System for Children

389

peractivity (Loeber, Green, & Lahey, 1990; Verhulst, Koot, & Van der Ende, 1994); of children to report on their own internal moods such as depression or anxiety (Bird, Gould, & Stagheeza, 1992), and of parents to describe their child’s behavior specifically and differentially (Daniel, 1993). The provision of comprehensive, unique information using a variety of measurement techniques and sources is a strength of the system (Adams & Drabman, 1994). The BASC also offers a structured developmental history form to place behavior in context and an efficient Systematic Observation System (SOS) to allow for the real-time coding of classroom behaviors by trained observers.

the Externalizing and Internalizing Composite scores in predicting children’s school adjustment, adding complimentary predictions above and beyond these traditional indicators. These findings and others (e.g., diSibio, 1993) strongly support the standardized measurement of adaptive behaviors in making predictions regarding future prognosis (Thorpe et al., 2000). In Anna’s case her mother and teacher’s report of good adaptive skills on the BASC Parent Rating Scale (BASC-PRS) and the BASC Teacher Rating Scale (BASC-TRS) suggested little functional impairment in these domains, which is vital information in making diagnostic determinations.

SYSTEMATIC MEASUREMENT OF BEHAVIORAL STRENGTHS AND WEAKNESSES

EFFECTIVE DIFFERENTIAL DIAGNOSTIC TOOL

The BASC represents a departure from many typical rating scales in that it provides normatively referenced information on the child’s adaptive behaviors or strengths as reported by parents, teachers, and the children themselves. The lack of positive behavioral dimensions in the past has been a key limitation of behavior rating scales (e.g., see Kratochwill et al., 1999). Not only are these behaviors indicators of the child’s functional status, many states mandate that they be included in a diagnostic assessment of learning disabilities or emotional and behavioral disorders. Without the ability to reference the child to his or her peers on these dimensions, the practitioner must rely solely on subjective narrative reports of caregivers and teachers or his or her own one-time impression of the child in the clinic. In addition, the inclusion of positive items counterbalances negatively worded items, mitigates against response sets (Kamphaus & Frick, in press), and provides the child’s caregivers and teachers the opportunity to report favorably about the child. Thorpe, Kamphaus, Rowe, and Fleckenstein (2000) found that scores on the Adaptive Composite scale of the BASC, which includes adaptability (response to change in environment), study skills, social skills, and leadership subscales, predicted children’s behavioral and academic status in school as many as 2 years later. The Adaptive composite was as good or better than

The range of behavioral dimensions assessed by the BASC aids in making a differential diagnosis of specific categories of a disorder as denoted in the DSM-IV (American Psychiatric Association, 1994). The BASC rating scales are constructed to provide separate information on Attention Problems and Hyperactivity as well as separate information on Depression and Anxiety. This delineation allows for a better differential diagnosis and is particularly useful in making decisions regarding inattentive, hyperactive, or combined subtypes of ADHD that have very different implications for treatment (Doyle, Ostrander, Skare, Crosby, & August, 1997; Vaughn, Riccio, Hynd, & Hall, 1997). In Anna’s case, her teacher’s report of elevations on anxiety reached significance on the BASC but not on the Achenbach Child Behavior Checklist (CBCL) Anxious/Depressed subscales, perhaps due to the blending of these two constructs. In addition, although her mother reported elevations only on the Attention subscale of the BASC, she did not report problems on the Hyperactivity subscale (Tscore = 47), suggesting that Anna’s difficulties were restricted to attentional lapses. With the exception of item-level analysis, no differentiation of the two behaviors was possible using Achenbach CBCL, which provided a single Attention Problems subscale (T-score = 68) and also includes items relating to impulsivity. Research supports the advantage of the BASC subscales in ac-

reyn2-17.qxd

390

6/6/2003

9:36 AM

Page 390

V. OBJECTIVE METHODS

curately discriminating children with primary inattentive type from combined type (Vaughn et al., 1997).

COMPLIANCE WITH FEDERAL/STATE EDUCATIONAL ASSESSMENT REQUIREMENTS The BASC offers clinicians working in school settings or treating children with learning problems or behavioral problems an array of important measurement and assessment techniques to target these problems and to comply with standards for behavioral analysis necessary for tailoring individualized education plans (IEPs) to students with special needs as required under the Individuals with Disabilities Education Act (IDEA) (Flanagan, 1995). A unique School Problems composite on the BASCTRS provides clinicians with well-normed information regarding that child’s adjustment in a school setting. The BASC SOS facilitates effective functional behavioral analysis, also required in recent revisions to IDEA, as well as an easy-to-use method for tracking changes in frequency and duration of such behaviors over time. The BASC ADHD Monitor provides a shortened format for monitoring changes in Attention and Hyperactivity subsequent to medication and or targeted behavioral interventions. These components are described more fully later in the chapter. Components of the BASC also assess aspects of the federal definition of Emotional Disturbance (Flanagan, 1995; Reynolds & Kamphaus, 1992). In their review, Sandoval and Echandia (1994) call the BASC “one of the most useful and sophisticated of all the new measures available to those wishing to assess school-age children” (p. 425). Referring back to Anna’s case, the BASCTRS was used to reevaluate her school behavior 3 months later. The results, presented here, revealed both consistency and stability on the majority of scales as well as sensitivity to change. While Anna’s scores on Attention Problems scaled were identical to the earlier evaluation and her elevations on the Anxiety subscale are still apparent, her teacher’s report at follow-up reveals a sharp increase in somatic complaints, suggesting that Anna’s school performance concerns might be manifesting as physical ailments.

Scores in bold are composites that represent an overall index of their component subscales. Anna’s Case BASC-TRS T-scores Oct. 1999 Jan. 2000 Hyperactivity Aggression Conduct Problems Externalizing Problems composite Anxiety Depression Somatization Internalizing Problems composite Attention Problems Learning Problems School Problems composite Atypicality Withdrawal Behavior Symptoms Index

44 41 43 42

44 43 43 43

69 41 46 52

62 46 64 59

56 61 59

56 53 55

52 51 51

57 48 52

COMPONENTS AND USES The BASC is a multimethod, multidimensional approach to evaluating the behavior and self-perceptions of children ages 2½–18 years, and includes its new variant, the BASC ADHD Monitor (Kamphaus & Reynolds, 1998). The original BASC is multimethod in that it has five components, which may be used individually or in any combination: 앫 The Teacher and Parent Rating Scales (TRS and PRS), which are separate instruments that gather descriptions of the child’s observable behavior at home, in the community, and at school. 앫 The Self-Report of Personality (SRP), which the child uses to describe his or her behaviors, emotions and self-perceptions. 앫 The Structured Developmental History (SDH), which is used to collect biographical, demographic, historical, and developmental information from parents or other primary caregivers, and which can serve as the basis for a parent interview.

reyn2-17.qxd

6/6/2003

9:36 AM

Page 391

17. The Behavior Assessment System for Children

앫 The Student Observation System (SOS), a form for recording and classifying directly observed classroom behavior. The BASC components not only provide different sources of information but, in fact, use different methods, a factor important to assessing generalizability of results and validation in diagnosis. The SRP, which can be used with children from 8 to 18 years of age, provides an introspective report of behavior, feelings, attitudes, and cognitions. The BASC-TRS and -PRS provide holistic summaries of that child’s “typical” behavior on an array of positive and negative indicators as seen through the eyes of behavioral experts specific to the child in question. The SOS provides direct observation and counting of behavior, believed by many to be the sine qua non of behavioral assessment (e.g., see Ramsay, Reynolds, & Kamphaus, in press, Ch. 1, for a review). The SDH helps provide a context for the presenting problem and provides for a structured interview as an additional method of assessment. The BASC is multidimensional in that it measures numerous dimensions of behavior and personality, including positive (adaptive) as well as negative (clinical) dimensions and both internalizing and externalizing problems. The BASC Adaptive scales include Social skills, Study skills, Leadership, and Adaptability from the standpoint of the child’s parent or teacher and Interpersonal relations, Self-esteem, Self-reliance, and parental relations from the point of view of the child. The BASC assesses both overt and covert behavior along with attitudes, feelings, and cognitions as well as certain affective states (e.g., anxiety, depressed mood, and attributional states), giving a range of dimensions heretofore unavailable in a single system. Scales were developed based on comprehensive theoretical and empirical considerations (Reynolds & Kamphaus, 1992) and represent a synthesis of what is known about developmental psychopathology (Sandoval & Echandia, 1994). In their review of technical qualities of the BASC, Sandoval and Echandia (1994) remark that “authors have set the standard for test construction for this kind of scale to be used with the childhood population” (p. 421).

391

Teacher Rating Scales The TRS has three forms with items designed for three age levels: preschool (2½–5), child (6–11), and adolescent (12–18). The forms contain descriptors of behaviors that the respondent rates on a 4-point scale of frequency, ranging from Never to Almost always. The TRS takes 10–20 minutes to complete, although teachers who are familiar with the TRS seldom require more than 10 minutes. Evidence for temporal stability and convergent validity (Merydith & Joyce, 1998) of the TRS has been presented. The TRS assesses clinical problems in the broad domains of Externalizing Problems, Internalizing Problems, and School Problems. It also measures Adaptive Skills. Table 17.1 shows the scales for all levels of the TRS. The slight differences between levels are due to developmental changes in the behavioral manifestations of child problems. Nevertheless, scales and composites with the same name contain essentially the same conceptual content at all age levels, even though specific items change across age. Children simply do not show their problems the same way at all developmental levels. In addition to scale and composite scores, the TRS provides a broad composite, the Behavioral Symptoms Index (BSI) that assesses the overall level of problem behaviors. While we recommend that a teacher know a child at least 4 to 6 weeks before using the TRS, a recent study found little difference between ratings of a new teacher and the previous year’s teacher who had known the child for most of a school year. Hoover, Braver, Wolchik, and Sandler (2000) found that teacher ratings on the Teacher–Child Rating Scale (T-CRS; Hightower, 1986) were similar for a group of 240 elementary grade children who were part of a divorce intervention study. They concluded that neither the previous teachers’ nor the current teachers’ fall ratings were significantly different from the spring teachers’ pretest ratings. Thus, school psychologists may elect to obtain ratings from either the previous or current teacher early in the fall of a new academic year. The TRS may be interpreted with reference to national age norms (General, Female, or Male) or to Clinical norms. In addition, selected critical items may be interpreted individually. The TRS includes a

reyn2-17.qxd

6/6/2003

9:36 AM

Page 392

392

V. OBJECTIVE METHODS

TABLE 17.1. Composites and Scales in the TRS and PRS Teacher Rating Scales ______________________________________________________________

Composite/Scale

Parent Rating Scales __________________________________________________________

Preschool

Child

Adolescent

Preschool

Child

Adolescent

Externalizing Problems Aggression Hyperactivity Conduct Problems

* * *

* * * *

* * * *

* * *

* * * *

* * * *

Internalizing Problems Anxiety Depression Somatization

* * * *

* * * *

* * * *

* * * *

* * * *

* * * *

School Problems Attention Problems Learning Problems

*

* * *

* * *

*

*

*

* *

* *

* *

* *

* *

* *

Adaptive Skills Adaptability Leadership Social Skills Study Skills

* *

* * * * *

*

* * * *

* * * *

*

Behavioral Symptoms Index

*

*

*

*

*

*

Other Problems Atypicality Withdrawal

*

* * *

* *

Note. Italicized scales compose the Behavioral Symptoms Index. From Reynolds and Kamphaus (1992). Copyright 1992 by American Guidance Service, Inc. Reprinted by permission.

validity check in the form of an F (“fake bad”) index designed to detect an excessively negative response set on the part of the teacher completing the rating. The BASC software programs also yield a Patterning validity index that assesses deviant patterns such as alternating between choices on a consistent basis. This validity index is not typically in question because teachers and parents have little incentive to complete a rating scale carelessly. The consistency index produced by the software is of greater value in that it detects agreement among highly similar items. By doing so this index assesses more subtle response bias or may detect unreliability of a specific rater.

Parent Rating Scales The PRS is a comprehensive measure of a child’s adaptive and problem behaviors in community and home settings. The PRS uses the same four-choice response format as the TRS and also takes 10–20 minutes to complete. Like the TRS, the PRS has three

forms at three age levels: preschool, child, and adolescent. The age levels of the PRS are similar in content and structure. Table 17.2 shows the scale definitions of the PRS. The PRS assesses almost all the clinical problem and adaptive behavior domains that the TRS measures. However, the PRS does not have a School Problems composite, nor does it include the two TRS scales that are best observed by teachers (Learning Problems and Study Skills). The PRS offers the same norm groups as the TRS: national age norms (General, Female, and Male) and Clinical norms. Like the TRS, the PRS includes an F index, patterning, and consistency indexes as checks on the validity of the parent ratings and critical items that may signify behaviors that should be interpreted individually.

Self-Report of Personality The SRP is an omnibus behavioral and personality inventory specially designed for children and adolescents to report an array

reyn2-17.qxd

6/6/2003

9:36 AM

Page 393

17. The Behavior Assessment System for Children

393

TABLE 17.2. BASC TRS and PRS Definitions Scale

Definition

Adaptability

The ability to adapt readily to changes in the environment

Aggression

The tendency to act in a hostile manner (either verbal or physical) that is threatening to others

Anxiety

The tendency to be nervous, fearful, or worried about real or imagined problems

Attention Problems

The tendency to be easily distracted and unable to concentrate more than momentarily

Atypicality

The tendency to behave in ways that are immature, considered “odd,” or commonly associated with psychosis (such as experiencing visual or auditory hallucinations)

Conduct Problems

The tendency to engage in antisocial and rule-breaking behavior, including destroying property

Depression

Feelings of unhappiness, sadness, and stress that may result in an inability to carry out everyday activities (neurovegetative symptoms) or may bring on thoughts of suicide

Hyperactivity

The tendency to be overly active, rush through work or activities, and act without thinking

Leadership

The skills associated with accomplishing academic, social, or community goals, including, in particular, the ability to work well with others

Learning Problems

The presence of academic difficulties, particularly in understanding or completing schoolwork

Social Skills

The skills necessary for interacting successfully with peers and adults in home, school, and community settings

Somatization

The tendency to be overly sensitive to and complain about relatively minor physical problems and discomforts

Study Skills

The skills that are conducive to strong academic performance, including organizational skills and good study habits

Withdrawal

The tendency to evade others to avoid social contact

Note. The PRS does not include TRS composite scales of Learning Problems, Study Skills, or School problems. From Reynolds and Kamphaus (1992). Copyright 1992 by American Guidance Service, Inc. Reprinted by permission.

of thoughts, feelings, and behaviors relevant to psychological and interpersonal adjustment. The SRP, which takes about 20–30 minutes to complete, consists of a list of True/False statements to be completed by the child or adolescent. The two forms, one for children (ages 8–11) and one for adolescents (ages 12–18), have considerable overlap in scales, in structure, and in individual items. Both levels have identical composite scores: School Maladjustment, Clinical Maladjustment, Personal Adjustment, and an overall composite score, the Emotional Symptoms Index (ESI). The child level (SRPC) has 12 scales and the adolescent level (SRP-A) has 14 scales arranged into com-

posites (see Table 17.3). Unlike the BSI for the rating scales, the ESI is composed of both negative (clinical) scales and positive (adaptive) scales whose scoring has been reversed, because these are the scales that load highest on a general psychopathology factor. Like the rating scales, the SRP may be interpreted with reference to national age norms (General, Female, and Male) or to Clinical norms. Special indexes are incorporated to assess the validity of the child’s responses: the F index, patterning index, consistency index, the L (“fake good”) index for the SRP-A only, and the V index designed to detect invalid responses due

reyn2-17.qxd

6/6/2003

9:36 AM

Page 394

394

V. OBJECTIVE METHODS

TABLE 17.3. Composites and Scales in the SRP Composite/Scale

Child

Adolescent

Clinical Maladjustment Anxiety Atypicality Locus of Control Social Stress Somatization

* * * * * *

* * * * * *

School Maladjustment Attitude to School Attitude to Teachers Sensation Seeking

* * *

* * * *

Other Problems Depression Sense of Inadequacy

* *

* *

Personal Adjustment Relations with Parents Interpersonal Relations Self-Esteem Self-Reliance

* * * * *

* * * * *

Emotional Symptoms Index

*

*

Note. Italicized scales compose the Emotional Symptoms Index. From Reynolds and Kamphaus (1992). Copyright 1992 by American Guidance Service, Inc. Reprinted by permission.

to poor reading comprehension, failure to follow directions, refusal to respond seriously to the task, or poor contact with reality. Table 17.4 lists scale definitions of the SRP.

Structured Developmental History The SDH is an extensive history and background survey that may be completed by a clinician during an interview with a parent or guardian, or may be completed as a questionnaire by a parent, either at home or in the school or clinic. The SDH systematically gathers information that is crucial to the diagnostic and treatment process. Many developmental events and medical or related problems in the family may have an impact on a child’s current behavior. The SDH structures the gathering of the child and family history, both social and medical. Because it is comprehensive, the SDH should be an asset to any evaluation of a child, whether or not other BASC components are used. Areas

addressed in the SDH are noted in Table 17.5.

Student Observation System The SOS is a form for recording a direct observation of the classroom behavior of a child. The SOS uses the technique of momentary time sampling (i.e., systematic coding during 3-second intervals spaced 30 seconds apart over a 15-minute period) to record a wide range of children’s behaviors, including positive behaviors (such as teacher–student interaction) as well as negative behaviors (such as inappropriate movement or inattention). The BASC SOS may be used appropriately in regular and special education classes. It can be used in the initial assessment as part of the diagnostic process. It can also be used repetitively to evaluate the effectiveness of educational, behavioral, psychopharmacological, or other treatments.

Forms The TRS, PRS, and SRP forms come in two formats: handscoring or computer entry. The handscoring forms are printed in a convenient self-scoring format, allowing them to be scored rapidly without using templates or keys (requiring about 5 minutes each to score after practice with the forms). Each form includes a profile of scale and composite scores. The computer entry forms, which are simpler one-part forms, are designed to allow the user to key item responses into a microcomputer in about 5 minutes.

Computer Software A microcomputer program, BASC Plus, is available that offers on-line administration of the TRS, PRS, and SRP and computer scoring of a completed computer-scored or handscored form. The manual for BASC Plus explains how to use the program to administer, score, and report the TRS, PRS, and SRP. It includes additional interpretive text and a listing of target behavior not available on other computer programs. The BASC Enhanced ASSIST program offers users a simpler computer program that produces all possible scores, a graphical display

reyn2-17.qxd

6/6/2003

9:36 AM

Page 395

17. The Behavior Assessment System for Children

395

TABLE 17.4. SRP Scale Definitions Scale

Definition

Anxiety

Feelings of nervousness, worry, and fear; the tendency to be overwhelmed by problems

Attitude to School

Feelings of alienation, hostility, and dissatisfaction regarding school

Attitude to Teachers

Feelings of resentment and dislike of teachers; beliefs that teachers are unfair, uncaring, or overly demanding

Atypicality

The tendency toward gross mood swings, bizarre thoughts, subjective experiences, or obsessive–compulsive thoughts and behaviors often considered “odd”

Depression

Feelings of unhappiness, sadness, and dejection; a belief that nothing goes right

Interpersonal Relations The perception of having good social relationships and friendships with peers Locus of Control

The belief that rewards and punishments are controlled by external events or other people

Relations with Parents

A positive regard toward parents and a feeling of being esteemed by them

Self-Esteem

Feelings of self-esteem, self-respect, and self-acceptance

Self-Reliance

Confidence in one’s ability to solve problems; a belief in one’s personal dependability and decisiveness

Sensation Seeking

The tendency to take risks, to like noise, and to seek excitement

Sense of Inadequacy

Perceptions of being unsuccessful in school, unable to achieve one’s goals, and generally inadequate

Social Stress

Feelings of stress and tension in personal relationships; a feeling of being excluded from social activities

Somatization

The tendency to be overly sensitive to, experience, or complain about relatively minor physical problems and discomforts

Note. From Reynolds and Kamphaus (1992). Copyright 1992 by American Guidance Service, Inc. Reprinted by permission.

of results, and item responses, but does not allow on-line administration.

General Norms The General norms are based on a large national sample that is representative of the general population of U.S. children with regard to sex, race/ethnicity, clinical or special education classification, and, for the PRS, parent education. These norms are subdivided by age and, therefore, indicate how the child compares with the general population of children that age. For many applications, these norms (combining females and males) will be the preferred norms, and they are recommended for general use. Several of the scales of the TRS, PRS, and SRP show gender differences. Males tend to obtain higher raw scores on the Aggression, Conduct Problems, Hyperactivity, Attention

TABLE 17.5. Areas Addressed in the SDH 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.

Person Answering Questions Referral Information Parents Primary Caregivers Child Care Family History Brothers/Sisters Child’s Residence Family Relations Pregnancy Birth Development Medical History Family Health Friendships Recreation/Interests Behavior/Temperament Adaptive Skills Educational History Additional Comments

reyn2-17.qxd

6/6/2003

9:36 AM

Page 396

396

V. OBJECTIVE METHODS

Problems, and Learning Problems scales of the TRS and PRS and on the Sensation Seeking, Attitude to School, Attitude to Teachers, and Self-Esteem scales of the SRP. Females tend to score higher than males on the Social Skills, Study Skills, Leadership, and Depression scales of the TRS and PRS and on the Anxiety and Interpersonal Relations scales of the SRP. These differences in scores likely reflect real differences between males and females in the incidence of the indicated behavioral or emotional problems or strengths in adaptive skills. For these gender differences to be reflected in the normative scores, a common set of norms must be used for both males and females. The General combined-sex norms serve this purpose. General norms answer the question: How commonly does this level of rated or self-reported behavior occur in the general population at this age? Using General norms, more males than females will show high T-scores on Aggression, for example, and more females than males will have high T-scores on Social Skills. The combined gender or general norms preserve any observed gender difference in the shape and level of the raw score distributions. This is appropriate, and the general norms should be used if one believes that boys and girls are in fact different on various behavioral characteristics (i.e., observed differences are not due to psychometric artifacts). For example, girls score higher than boys on the SRP Anxiety scale (a common finding in research on anxiety, e.g., see Reynolds & Richmond, 1985). In determining which set of norms to use, the clinician must answer the question, “Are girls more anxious than boys, or are they simply more willing to admit to symptoms of anxiety?” If the former is true, the general norms are more appropriate, but in the latter case, the gender-specific norms are the correct choice. Reynolds and Kamphaus (1992) recommend the use of the general norms, a decision with which we continue to concur, but the individual clinician may disagree and opt for the other norms. This allows the clinician more latitude than typically occurs on other behavioral and self-report scales.

Female Norms and Male Norms These norms are based on subsets of the General norm sample; each is representative

of the general population of children of that age and gender. The effect of using these separate-sex norms is to eliminate differences between males and females in the distribution of T-scores or percentiles. For example, although raw score ratings on the Aggression scale tend to be higher for males than females, use of separate-sex norms removes this difference and produces distributions of normative scores that are the same for both genders.

Indexes of Validity and Response Set Several indexes are provided to help the BASC user judge the quality of a completed form. Validity may be threatened by any of several factors including failure to pay attention to item content, carelessness, an attempt to portray a child in a highly negative or positive light, lack of motivation to respond truthfully, or poor comprehension of the items. Information on the development of these indexes and the setting of cutoff scores is provided in Reynolds and Kamphaus (1992).

F Index The F index, included on all of the BASC rating-scale and self-report forms, is a measure of the respondent’s tendency to be excessively negative about the child’s behaviors or self-perceptions and emotions. The F index was developed using traditional psychometric methods associated with Infrequency scales (e.g., see Reynolds, 2001). On the PRS and TRS, the F index is scored by counting the number of times the respondent answered Almost always to a description of negative behavior or Never to a description of positive behavior. Because responses on the SRP are limited to True and False, items selected for that F index are either extremely negative items to which the child responded True or positive items to which the response was False. Items were selected for these scales that have a low probability of co-occurrence (i.e., they are seldom endorsed in concert with one another). The TRS, PRS, and SRP record forms show what levels of F index scores are high enough to be of concern. Detailed guidance to interpretation of the F index is given in Reynolds and Kamphaus (1992).

reyn2-17.qxd

6/6/2003

9:36 AM

Page 397

17. The Behavior Assessment System for Children

L Index The L index, offered for the adolescent level of the SRP, measures an adolescent’s tendency to give an extremely positive picture of him- or herself—what might be called “faking good.” The index consists of items that are unrealistically positive statements (such as “I like everyone I meet”) or are mildly self-critical statements that most people would endorse (such as “I sometimes get mad”). Individuals scoring high on this scale may also be giving the most socially desirable response or possibly are psychologically naive relative to their peers denying common, everyday problems or concerns. The SRP-A record form shows which L scores should be of concern.

V Index Each level of the SRP includes a V index made up of five or six nonsensical or highly implausible statements (such as “Superman is a real person”). The V index serves as a basic check on the validity of the SRP scores in general. If a respondent marks two or more of these statements as True, the SRP may be invalid.

BASC ADHD MONITOR The BASC ADHD Monitor fills a unique role in the assessment of children who are diagnosed with ADHD. The Monitor is the second step in an assessment regimen that is designed to enhance treatment planning and evaluation by thoroughly assessing the primary symptoms of ADHD on a continuing basis. Attention Problems and Hyperactivity constitute the core symptoms used by DSMIV to define the ADHD syndrome (Kamphaus & Frick, in press). Problems in one or both of these areas are used to differentiate the three subtypes of ADHD: ADHD, predominantly inattentive type; ADHD, predominantly hyperactive–impulsive type; and ADHD, combined type. Components of the original BASC system serve as the first step in the comprehensive assessment of children suspected of having ADHD. The BASC takes a broad sampling of child behavior to identify the full range of child problems, especially those that may

397

mimic the symptoms of ADHD. If the initial administration of the BASC reveals problems on the Attention Problems and/or Hyperactivity scales, the diagnosis of ADHD becomes a possibility. Of greater importance, however, is the necessity to use the BASC Teacher, Parent, and Self-Report Forms to rule out co-occurring problems, which can only be done with the initial use of a broad-based measure such as the BASC PRS or TRS (Kamphaus & Frick, in press). This process of ruling out other problems is particularly important for the diagnosis of ADHD, where so many comorbid disorders occur and where other disorders (e.g., childhood depression) may superficially appear to be ADHD. In fact, the use of narrowband scales of inattention or hyperactivity may result often in overdiagnosis of ADHD. The Monitor is concerned with treatment design for and evaluation of children with ADHD. The narrowly focused Monitor is designed to assess an expanded range of Attention Problems and Hyperactivity symptoms in a time efficient and practical manner. This additional detail allows the clinician to refine the diagnosis of ADHD and, of greater importance, to design a comprehensive treatment program aimed at reducing the core behavioral problems of inattention and hyperactivity. The Monitor also provides Internalizing and Adaptive Skills scales that further encourage comprehensive treatment planning and evaluation of treatment effectiveness by allowing clinicians to include these important constructs easily in the treatment plan. The BASC SOS and BASC ADHD Monitor represent a coordinated multiple-step assessment system that allows the clinician to proceed from referral for ADHD to diagnosis, treatment design, and treatment evaluation with greater ease and precision. In order to achieve these assessment objectives, the Monitor utilizes information provided by parents, teachers, and a classroom observer to assess the constructs listed in Table 17.6. Few tests are designed in a manner that facilitates the repeated collection and dissemination of child information to treatment providers (Kratochwill et al., 1999). The Monitor is designed to meet the unusual practical demands dictated by the need for the repeated assessment of the core

reyn2-17.qxd

6/6/2003

9:36 AM

398

Page 398

V. OBJECTIVE METHODS

TABLE 17.6. Multistep Assessment with the BASC, ADHD Monitor, and Student Observation System Component

Scales

Parent Monitor

Attention Problems Hyperactivity Internalizing Problems Adaptive Skills

Teacher Monitor

Attention Problems Hyperactivity Internalizing Problems Adaptive Skills

BASC SOS

Response to Teacher/Lesson Peer Interaction Work on School Subjects Transition Movement Inappropriate Movement Inattention Inappropriate Vocalization Somatization Repetitive Motor Movements Aggression Self-Injurious Behavior Inappropriate Sexual Behavior Bowel/Bladder Problems

symptoms of ADHD. The original BASC is quite sensitive to behavioral changes in individual children and it may be used repeatedly to evaluate treatment effects (Conoley et al., 2001), particularly if a child is found to have multiple problems (e.g., ADHD, depression, anxiety, and conduct disorder) that cannot be fully assessed by the Monitor. In the case of ADHD and its subtypes, however, the Monitor is constructed to allow clinicians to evaluate treatment with greater focus and time efficiency. The BASC ADHD Monitor is designed to: 1. Provide accurate and frequent feedback to the prescribing physician. The physician and other health care workers need accurate information to ensure that a child is receiving the most accurate psychotropic regimen and to adjust dosage. Information about the effects of medication on Hyperactivity, Attention Problems, Internalizing Problems, and Adaptive Skills can aid the physician in making crucial medical treatment decisions.

2. Ensure that the ongoing assessment of ADHD problems is efficient, timely, and cost-effective. Given the multiple time demands on parents, teachers, and others, little time remains to complete lengthy or unnecessarily complex rating scales that are not specifically targeted to the needs of the child with ADHD. On the other hand, the Monitor is designed to be adequately thorough in order to allow for the assessment of constructs in addition to the core dimensions of ADHD—Internalizing Problems and Adaptive Skills (Kamphaus & Frick, in press). All these assessment objectives must be achieved in an efficient way given the exigencies of health care. Accordingly, the Monitor is brief, yet it provides coverage of four important domains related to the functioning of the child with ADHD: Attention Problems, Hyperactivity, Internalizing Problems, and Adaptive Skills. 3. Provide a system of devices that allows for input from multiple informants. Teacher, parent, and clinician observations are all of potential importance for the treatment process, and communication among these individuals is crucial for effective treatment (Bender, 1997). Each Monitor form is designed to meet the specialized needs of each of these informants. 4. Emphasize the assessment of specific behavioral outcomes in order to demonstrate accountability for services. Increasingly, the effectiveness of child services is being challenged, thereby creating the need to assess outcomes. The Monitor assesses the DSM-IV criteria for ADHD and includes items that are written in clear behavioral terms. In addition, the Monitor software is designed to produce output that gives providers and administrators a clear indication of response to treatment. The Monitor is designed to provide clinicians with the information needed to adjust treatment whenever response to intervention is not optimal. 5. Link assessment to treatment planning and evaluation. The Monitor is designed to be practical enough to be considered central to the treatment process. Heretofore, physicians and other clinicians have often had difficulty acquiring the feedback needed to adjust treatment. The test and software design of the Monitor was guided throughout by the need to provide information relevant

reyn2-17.qxd

6/6/2003

9:36 AM

Page 399

17. The Behavior Assessment System for Children

to treatment. The selection of items and scales, test length, scoring and reporting systems, graphical output, and other Monitor characteristics were all guided by this central objective. ADHD Monitor interpretation can take several forms depending on the instrument(s) used, theoretical orientation of the clinician, the nature of the evaluation questions posed, and other factors. It is also important to keep in mind that the Monitor is designed to create and evaluate treatment plans. Therefore, interpretation of the scales as diagnostic devices is of considerably lesser importance. The initial step in evaluating monitor results is that the individual clinician asks whether or not significant change has occurred in response to treatment. For the Parent and Teacher Monitors four questions are generally posed: 1. Is treatment affecting symptoms of inattention? 2. Is treatment affecting symptoms of hyperactivity? 3. Is treatment affecting internalizing symptoms? 4. Is treatment affecting adaptive skills? The questions related to change are multitudinous and parallel for the SOS, where one may be assessing change at either the item or scale level (discussed in the last half of this chapter). Keep in mind that when scores change, they may show deterioration in some areas, not just improvement. For example, as a child’s symptoms of overactivity and inattention, come under control, comorbid symptoms of depression may become more prominent causing scores on the internalizing dimension to elevate. Even if a significant change in T-scores is apparent there are additional questions to consider. 5. Which scales have changed? 6. Is there a temporal (and potentially causal) relationship between the delivery (or lack thereof) treatment and the observed change? 7. Is the change of “clinical significance”? In other words, regardless of the amount of T-score change are parents or teachers

399

reporting change that is adequate to reduce functional impairment in their eyes? We think that establishing the temporal relationship between T-score change and delivery or withdrawal of treatments is of greatest importance. It is our experience that often this relationship is assessed via the speculation or supposition of the clinician. We think that a better way to draw a conclusion regarding the relationship of treatment to behavioral change is to conduct repeat assessments until the relationship is clear. For example, one could see a reduction in attention problems subsequent to the first month’s administration of medication. While this change represents a hopeful sign, this pretest–posttest design is probably insufficient to draw such a conclusion definitively. A third set of Monitor ratings taken a few months later that show the same trend would provide more assurance that the conclusion that medication is having an effect is warranted. A set of ratings yielding more symptoms when the child is not taking medication in the summer months or some other time would lend further support for medication effectiveness. We often find that an additional brief assessment clarifies our conclusions to a much greater extent than prolonged theorizing based on more limited data. Use of the BASC often precedes administration of the ADHD Monitor. There is, however, one important area of interpretive overlap between the BASC and the BASC ADHD Monitor Parent and Teacher Forms. A T-score baseline for treatment evaluation can be obtained from either set of measures. There are two administration scenarios that are most likely. 1. A clinician may administer either or both of the BASC Parent and Teacher Forms during the initial diagnostic evaluation. The obtained T-scores for the Hyperactivity, Attention Problems, Internalizing Problems, and Adaptive Skills scales may be entered into the BASC ADHD software and be used as the baseline against which subsequent administrations of the ADHD Monitor will be compared. 2. A clinician may administer either or both of the BASC ADHD Parent and Teacher

reyn2-17.qxd

6/6/2003

9:36 AM

Page 400

400

V. OBJECTIVE METHODS

Monitor Forms during the initial diagnostic evaluation. The obtained T-scores for the Hyperactivity, Attention Problems, Internalizing Problems, and Adaptive Skills scales will then be used as the baseline against which subsequent administrations of the ADHD Monitor Forms will be compared. It is important to establish a T-score baseline in a timely fashion regardless of the method used. In other words, we advise that a T-score baseline be collected during the evaluation phase and prior to implementation of treatment. The ADHD Monitor Tscores for Parent and Teacher rating scales serve as the most reliable indicator of behavioral change over time (see Kamphaus & Reynolds, 1998).

SOS Functional Behavioral Assessment with the SOS The SOS addresses some of the shortcomings inherent in the use of classroom observation techniques. Specifically, the SOS was developed to make practical the use of a momentary time-sampling procedure that adequately samples the full range of a child’s behavior in the classroom (Reynolds & Kamphaus, 1992). Several characteristics of the SOS exemplify this effort, including the following: 1. Both adaptive and maladaptive behaviors are observed (see Table 17.1); 2. Multiple methods are used including clinician rating, time sampling, and qualitative recording of classroom functional contingencies; 3. A generous time interval is allocated for recording the results of each time sampling interval (27 seconds); 4. Operational definitions of behaviors and time-sampling categories are included in the BASC manual (Reynolds & Kamphaus, 1992); and 5. Interrater reliabilities for the time-sampling portion are high which lends confidence that independent observers are likely to observe the same trends in child’s classroom behavior (Lett & Kamphaus, 1997).

These characteristics of the SOS have contributed to its popularity as a functional behavioral assessment tool. It is crucial, for example, to have adequate operational definitions of behaviors that, in turn, contribute to good interrater reliability. Without such reliability, clinicians will never know whether their observations are unique and potentially influenced by their own biases or idiosyncratic definitions of behavior. We also think that it is central for observations to simultaneously account for a child’s adaptive skills in the classroom. It is only by doing so that a clinician is able to recommend behaviors that should be targeted for instruction, intervention, or strengthening. Specifically, the BASC SOS Parts A, B, and C, and other components, can contribute to the functional assessment of behavior from multiple perspectives: 앫 Behavior Frequency. SOS Part A ratings of “never observed,” “sometimes observed,” and “frequently observed.” SOS Part B assesses frequencies by category of behavior problem and PRS and TRS ratings tally the frequency of behavior problems. 앫 Behavior Duration. SOS Part B ratings of percentage of time engaged in a particular behavior by category. 앫 Behavior Intensity. SOS Part A ratings of “disruptive.” SOS Part B ratings of frequency by category. 앫 Antecedent events to Behavior. SOS Part C descriptions of teacher position, behavior and other variables that precede misbehavior. 앫 Consequences of Behavior. SOS Part C descriptions of teacher behavior, peer behavior, and other variables that follow a behavior. 앫 Analysis of Behavior across Settings. SOS observations made at various times of day and classroom setting. The PRS may be used for the assessment of behavior in the community and home environments. Other components of the BASC, such as the PRS and TRS, may also be used as part of a functional behavioral assessment paradigm. Given the time-consuming nature of observations, it may be more practical to collect teacher ratings from classrooms

reyn2-17.qxd

6/6/2003

9:36 AM

Page 401

17. The Behavior Assessment System for Children

where an observation is not practical and parent ratings in order to assess differences across settings. Observations are central to the ongoing classroom problem-solving and consultation process that is frequently concerned with the ongoing assessment of a child’s behavioral adaptation in school as is discussed next (additional functional behavioral assessment guidance may be obtained at www.air.org/cecp/fbalproblembehavior/ strategies.htm#direct).

Monitoring with the SOS The SOS is the one component of the BASC ADHD Monitor that may be applied to all children regardless of their diagnosis or classification. In fact, we know of school districts that use the SOS and Monitor Software to evaluate progress toward TEP objectives, assess effects of prereferral intervention, and assess the effectiveness of various special education programming decisions. Some have used the SOS to assess the impact of social work or the services on classroom behavior. Perhaps more than any other BASC component, the SOS was specifically designed to serve the behavioral intervention and evaluation process in the classroom. We now discuss some possible scenarios and examples of applications of the SOS. Medical Effects Mary’s parents are opposed to the use of medication with their child in spite of the fact that numerous behavioral (psychotherapy, play therapy, token economy, etc.) and educational interventions (peer tutor, after school tutor, summer school, preferential seating, etc.) have failed. The SOS may help such reluctant parents gauge the effects of pharmacological interventions on Mary’s classroom behavior in a manner that they perceive as more objective than teacher ratings. In this example an independent, perhaps even case-blind, observer may take SOS observations presomatic therapy, at two or more points after initiation of somatic therapy (perhaps in as few as a couple of weeks to a month after the initiation of medication such as methylphenidate which reaches therapeutic levels rather quickly), or when-

401

ever dosage or medication is changed. The BASC Monitor Software can then graph Part B (momentary time sampling) results that can be shared with parents, physician, or other service providers and caregivers. Specific behaviors from Part A can be graphed as well but we would expect individual behaviors to be less reliable indicators of change overall. In this scenario it is crucial to be able to link somatic therapy to change. To do so, the SOS should be collected concurrently with changes in regimens. We think that the 15-minute time sampling is adequate for this purpose based on our experience and the fact that interobserver reliability did not differ for 15- or 45-minute observations (Lett & Kamphaus 1997). In addition, children receiving a variety of medications including psychostimulants, anxiolytics, antidepressants, and antipsychotic medications require careful monitoring of the effects of these drugs on classroom behavior. IEP Objectives Part A was designed specifically to enhance the development of IEP objectives. Behavior from Part A may then be tracked with the repeated rating of Part A and change graphed by Monitor Software. In fact, some statisticians who have expressed concern about the overreliance on significance testing have noted that graphing is one powerful alternative method for data analysis. We have noted how convincing a graph is to teachers, parents, and others. We suggest, however, that the clinician observe using Part B prior to completing Parts A and C. We think that the vigilance required to complete the momentary time sampling ensures careful observation that leads to a more accurate rating of the behavior intervention plans in an ongoing fashion. Finally, because 3 data points are advised to obtain a reliable trend line (Francis, Fletcher, Stuebing, Davidson, & Thompson, 1991), we recommend that, as a minimum, observations be collected at the outset of the school year (after the child has had 1 month to adjust to teachers, peers, etc.), at a midway point when it may be convenient to adjust intervention (certainly March or April of the academic year would be too

reyn2-17.qxd

6/6/2003

9:36 AM

402

Page 402

V. OBJECTIVE METHODS

late), and just prior to the annual evaluation of IEP goals. Prereferral Intervention The evaluation of such intervention can occur in the same framework advised for the annual evaluation of IEP objectives but on a shorter timetable. Again, a minimum of 3 data points are advised even if the intervention is designed to be brief (e.g., 1 or 2 months). Consider the following example: Shane is a victim of physical abuse by his mother, resulting in his being placed in foster care for 3 months. At the same time his mother is receiving treatment. He is initiating routine counseling sessions at school for the first time. Shane also has a history of distractibility and truancy at school. Shane’s truancy could be tracked by event recording during this period, and the SOS could assess his classroom behavior during monthly intervals. SOS results could be of some additive value in assessing the value and the effects of the foster care placement and counseling on his classroom behavior. Schoolwide Interventions While recognizing the impracticality of using the SOS on a large scale, we do think that it could be used for sampling purposes. For example, one or two children deemed to be at risk for aggression could be sampled from each classroom to evaluate the effects of the school’s violence prevention program. Good evaluation data are crucial for such programs as some evidence of iatrogenic effects have been noted. The SOS is designed specifically for classroom-based intervention. SOS results then should not be considered when evaluating home-based intervention unless home and school-based interventions are linked. For example, a homebound reinforcement program may be used to improve behavior at school. The SOS assesses the frequency of classroom behavioral problems. Consequently, SOS results from Parts A and B may be used to identify behaviors in need of intervention. Specifically, any behavioral problem that is exhibited or adaptive skill that is not exhibited becomes a potential candidate for intervention. Within these groups, problem

behaviors of higher frequency can be given priority for intervention. Analogously, lowfrequency adaptive skills also become candidates for intervention. The SOS is unique among Monitor components in that it allows clinicians to prioritize behaviors for classroom-based intervention. The SOS also measures the “bothersomeness” of a child’s behavioral problems via the disruptive category of Part A. Often children display a number of behavioral problems making it difficult to prioritize behaviors for intervention (Schwanz & Kamphaus, 1997). The ratings of disruptiveness can be used to identify behaviors that should be targeted first for treatment.

OTHER APPLICATIONS OF THE BASC Longitudinal Outcome Research Various components of the BASC are being used in longitudinal investigations to study the risk, onset, course, and progress of behavioral problems and psychopathology in childhood. Some studies have used the BASC as a measure of child outcomes or as the criterion variable of interest. Nelson, Martin, Hodge, Havill, and Kamphaus (1999) used the BASC TRS and PRS as outcome criteria to assess the predictive validity of early temperament. Nelson and colleagues predicted that preschool temperament would predict later functional behavioral status as assessed by the BASC. Their hypothesis was supported. They found that three temperament constructs rated by parents at age 3 are associated with BASC-TRS-C ratings at age 8. The assessment of problems early in development via teacher ratings may indicate early risk. These and related findings have considerable impact for the support of early screening and targeted prevention. As teachers use the BASC-TRS, they become quite adept at completing the forms, commonly completing them in 10 or so minutes. A teacher can then complete the BASC-TRS-P for an entire class in about 3 hours. There are few such efficacious approaches to screening for children at high risk for the development of behavioral and emotional difficulties at prereading age levels. The BASC-TRS-P and the -PRS-P are well suited to efficient

reyn2-17.qxd

6/6/2003

9:36 AM

Page 403

17. The Behavior Assessment System for Children

screening for identification of high-risk children in the 2½- to 5-year age range. CHAMPUS, the U.S. military civilian and retiree health care system, began a longitudinal study of adolescents placed in residential treatment centers (RTCs) in 1997. The study is under the direction of Dr. Richard Gaines. Although the data analyses are not yet complete, preliminary analyses and results are reported as being quite good. In this study, CHAMPUS was interested in predicting which adolescents referred for placement would actually benefit from the expensive RTC setting. Gaines (personal communication, 2001) reports that the BASC has been found to have “good predictive power” in this study, although details remain scant at this writing. We expect the BASC will continue to be used in such situations and that it will perform well due to its integrative development process (i.e., a combination of rational, theory-driven, and empirical methods). Merydith (2000) used the BASC-TRS-A to assess the effects of violence prevention programs conducted in schools. On the basis of the TRS-A and principal’s nominations, aggressive adolescents were identified for specific intervention programs. After treatment, the BASC scores showed substantial reductions in means for the treatment versus the control group on relevant scales—some exceeding a full standard deviation. These improvements are viewed as remarkable and suggest the TRS-A is sensitive to the effects of even brief intervention programs such as implemented here. The BASC has also been noted to be sensitive to interventions with Head Start children in the younger age range (see Reynolds & Kamphaus, 2002, for a review).

Forensic Applications of the BASC Reynolds and Kamphaus (2002) describe a variety of forensic or court-related applications of the BASC. They also note the many special features of the BASC that make it desirable in forensic settings. One key feature is the various validity scales on the BASC components and the ability to triangulate on behavior. An important factor for clinicians to consider in choosing instruments for forensic evaluations is the presence of scales designed to detect dissimula-

403

tion (Reynolds, 1997). Dissimulation is the act of making oneself (or in the case of rating scales, the person being rated) appear dissimilar or different in some way from one’s actual state. In the legal arena, individuals may have much to gain by appearing to have more or fewer problems than actually exist. Almost any behavioral or emotional disorder can be the subject of dissimulation. As Sattler (1998) notes in his extensive review, dissimulation, especially negative dissimulation or malingering, is difficult to identify. Objective methods are absolutely necessary for the accurate identification of dissimulation and the BASC provides the clinician with one of the few sets of measures for children to detect such problematic responding. Reynolds and Kamphaus (2002) review many applications of the BASC in the forensic arena. They suggest the BASC is especially useful in child custody, personal injury (particularly when posttraumatic stress disorder, traumatic brain injury, or emotional pain and suffering are at issue), juvenile certification, determining the needs of adjudicated delinquents, and documenting the need for special educational services.

SUMMARY From its SOS to the suite of behavior rating scales, the BASC provides multiple methods for gathering important information for making accurate assessments of children suffering from a wide range of diagnostic as well as subthreshold developmental difficulties. In the absence of a specific diagnostic determination, the BASC provides the ability to determine a child’s placement on a continuum of behavior relative to his or her peers, allowing clinicians to make judgments regarding probability of future problems. Because it provides a spectrum of information beyond that necessary for identifying clinical pathologies, the BASC is a useful instrument for professionals called on to make recommendations for children who require intervention plans tailored to both nurture their strengths and buttress their weaknesses. The BASC and other similar rating scales provide important dimensional information on profiles of child behavior that provide a more complete

reyn2-17.qxd

6/6/2003

9:36 AM

404

Page 404

V. OBJECTIVE METHODS

understanding of a child who may suffer functional impairment but may not meet strict diagnostic criteria under DSM-IV. Children in hospital, school, special education, and other similar settings often fit this profile, requiring accommodations or services without meeting categorical criteria for diagnosis. The BASC offers a variety of data gathering avenues for clinicians working in school settings who must comply with federal and state standards for educational assessment and monitoring of changes subsequent to intervention. Computerized programs and co-normed scales make cross-informant and multimethod information easy to compare, assimilate, and present to parents and educators. Importantly, the BASC allows clinicians to objectively assess a child’s adaptive strengths relative to peers, filling a large gap in available behavioral measurement tools. The BASC ADHD Monitor is a timely and efficient method for measuring medication and behavioral intervention effects in children with ADHD and is the newest element of this comprehensive assessment system. The BASC rating scales have also been used effectively in research, providing sensitive, accurate measurement in a number of longitudinal studies and proven application in forensic evaluations.

ACKNOWLEDGMENTS This chapter is adapted in part from Reynolds and Kamphaus (2002). Copyright 2002 by The Guilford Press. Adapted by permission. Portions of the chapter are reprinted or adapted from Reynolds and Kamphaus (1992) and Kamphaus and Reynolds (1998). Copyright 1992, 1998 by American Guidance Service, Inc. Reprinted/adapted by permission.

REFERENCES Adams, C. A., & Drabman, R. 5. (1994). BASC: A critical review. Child Assessment News, 4, 1–5. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Bender, W. N. (1997). Medical interventions and school monitoring. In W. N. Bender (Ed.), Understanding ADHD: A practical guide for teachers and parents (pp. 107–122). Upper Saddle River, NJ: Merrill. Bird, H., Gould, M., & Staghezza, B. (1992). Aggre-

gating data from multiple informants in child psychiatry epidemiological research. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 78–85. Cantwell, D. P. (1996). Classification of child and adolescent psychopathology. Journal of Child Psychology and Psychiatry and Allied Disciplines, 37, 3–12. Daniel, M. H. (1993, August). Diagnostic specificity of parents’ vs. teachers’ behavior ratings. Paper presented at the annual meeting of the National Association of School Psychologists, Washington, DC. diSibio, M. (1993). Conjoint effects of intelligence and adaptive behavior on achievement in a nonreferred sample. Journal of Psychoeducational Assessment, 11, 304–313. Doyle, A., Ostrander, R., Skare, S., Crosby, R. D., & August, G. J. (1997). Convergent and criterion related validity of the behavior assessment system for children–parent rating scale. Journal of Clinical Child Psychology, 26, 276–284. Fergusson, D., & Horwood, L. J. (1995). Early disruptive behavior, IQ, and later school achievement and delinquent behavior. Journal of Abnormal Child Psychology, 23, 183–199. Flanagan, R. (1995). A review of the Behavior Assessment System for Children (BASC): Assessment consistent with the requirements of the Individuals with Disabilities Education Act (IDEA). Journal of School Psychology, 33, 177–186. Flanagan, D. P., Alfonso, V. C., Primavera, L. H., Povall, L., & Higgins, D. (1996). Convergent validity of the BASC and SSRS: Implications for social skills assessment. Psychology in the Schools, 33, 13–23. Francis, D. J., Fletcher, J. M., Stuebing, K. K., Davidson, K. C., & Thompson, N.M., (1991). Analysis of change: Modeling individual growth. Journal of Consulting and Clinical Psychology, 59, 27–37. Hightower, A. D. (1986). The teacher–child rating scale: A brief objective measure of elementary children’s school problem behaviors and competencies. School Psychology Review, 15(3), 393–409. Hoover, H. V. A., Braver, S. L., Wolchik, S. A., & Sandler, I. N. (2000, August). Teachers’ ratings of children’s classroom behaviors: Time of year effects? Poster session at the annual meeting of the American Psychological Association, Washington, DC. Kamphaus, R. W., & Frick, P. J. (in press). Clinical assessment of child and adolescent personality and behavior (2nd ed.). Needham Heights, MA: Allyn & Bacon. Kamphaus, R. W., & Reynolds, C. R. (1998). BASC ADHD Monitor. Circle Pines, MN: American Guidance Service. Kratochwill, T., Sheridan, S., Carlson, J., & Laseck, K. (1999). Advances in behavioral assessment. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (pp. 350–382). New York: Wiley. Left, N. J., & Kamphaus, R. W. (1997). Differential validity of the BASC Student Observation System and the BASC Teacher Rating Scales. Canadian Journal of School Psychology, 13, 1–14. Loeber, R., Green, S., & Lahey, B. B. (1990). Mental health professionals’ perception of the utility of chil-

reyn2-17.qxd

6/6/2003

9:36 AM

Page 405

17. The Behavior Assessment System for Children dren, mothers, and teachers as informants on childhood psychopathology. Journal of Clinical Child Psychology, 2, 136–143. Merydith, S. P. (2000). Aggression Intervention Skill Training: Moral reasoning and moral emotions. NASP Communqiué, 28, 6–8. Merydith, S. P., & Joyce, E. K. (1998, August). Temporal stability and convergent validity of the BASC Parent and Teacher Rating Scales. Paper presented at the annual meeting of the American Psychological Association, San Francisco. Nelson, B., Martin, R. P., Hodge, S., Havill, V., & Kamphaus, R. (1999). Modeling the prediction of elementary school adjustment from preschool temperament. Personality and Individual Differences, 26, 687–700. Ramsay, M., Reynolds, C. R., & Kamphaus, R. W. (in press). Essentials of behavioral assessment. New York: Wiley. Reynolds; C. R. (1997). Detection of malingering in head injury litigation. New York: Kluwer Academic Press. Reynolds, C. R. (2001, October). Forensic neuropsychological evaluation. Workshop presented at the annual convention of the American Board of Forensic Examiners, Nashville, TN. Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: American Guidance Service. Reynolds, C.R., & Kamphaus, R.W. (2002). The clinician’s guide to the Behavior Assessment System for Children (BASC). New York: Guilford Press.

405

Reynolds, C. R., & Richmond, B. O. (1985). Manual for the Revised Children’s Manifest Anxiety Scale. Los Angeles, CA: Western Psychological Services. Sandoval, J., & Echandia, A. (1994). Behavior Assessment System for Children. Journal of School Psychology, 32, 419–425. Sattler, J. M. (1998). Clinical and forensic interviewing of children and families. San Diego, CA: Author. Scahill, L., Schwab-Stone, M., Merikangas, K. R., Leckman, J. F., Zhang, H., & Kasl, S. (1999). Psychosocial and clinical correlates of ADHD in a community sample of school-age children. Journal of the American Academy of Child and Adolescent Psychiatry, 38(8), 976–984. Thorpe, J. S., Kamphaus, R. W., Rowe, E., & Fleckenstein, L. (2000, August). Longitudinal effects of child adaptive competencies, externalizing, internalizing behavior problems on behavioral and academic outcomes. Poster session at the annual meeting of the American Psychological Association, Washington, DC. Vaughn, M. L., Riccio, C. A., Hynd, G. W., & Hall, J. (1997). Diagnosing ADHD subtypes: Discriminant validity of the Behavior Assessment System for Children (BASC) and the Achenbach parent and teacher rating scales. Journal of Clinical Child Psychology. 26, 349–357. Verhuist, F. C., Koot, H. M., & Van der Ende, J. (1994). Differential predictive value of parents’ and teachers’ reports of children’s problem behaviors: A longitudinal study. Journal of Abnormal Child Psychology, 22, 531–546.

reyn2-18.qxd

6/6/2003

9:37 AM

Page 406

18 The Achenbach System of Empirically Based Assessment

THOMAS M. ACHENBACH STEPHANIE H. MCCONAUGHY

The Achenbach System of Empirically Based Assessment (ASEBA) comprises a family of standardized instruments for assessing behavioral and emotional problems and adaptive functioning. A key feature of ASEBA instruments is that they assess functioning from multiple perspectives, including reports by parents, other collaterals, caregivers, teachers, youths, clinical interviewers, observers, and psychological examiners. A second key feature is that ASEBA instruments have been developed according to a “bottom-up” approach. In the bottom-up approach, large pools of items are tested for their ability to tap adaptive and maladaptive functioning, as scored by particular kinds of raters under particular conditions. The items are scored quantitatively to reflect the degree to which individuals manifest particular characteristics. Items are retained if they discriminate effectively between individuals who are not functioning well and demographically similar individuals who are functioning well. The items that tap behavioral and emotional problems are subjected to multivariate statistical analyses to identify syndromes of problems that co-occur. The term “syndrome” is used in its generic sense of “things

that occur together,” without implying a specific cause. Some syndromes may reflect primarily biological causes, others may reflect primarily environmental causes, and still others may reflect a mixture of causes. Figure 18.1 illustrates the bottom-up approach to deriving syndromes. The syndromes of co-occurring problem items are used to construct scales for scoring individuals to reflect the degree to which they manifest each syndrome. An individual’s score on a syndrome is computed by summing his or her scores on each item of the syndrome. To determine how the individual’s syndrome scores compare with scores obtained by similar individuals, the syndromes are displayed on profiles in relation to normative distributions of scores obtained by large samples of peers who were assessed with the same assessment procedures. The profiles display T-scores and percentiles for each syndrome, based on the relevant normative sample. The ASEBA originated in research to provide better differentiated assessment of child and adolescent psychopathology at a time when the American Psychiatric Association’s (1952) Diagnostic and Statistical Manual of Mental Disorders (first edition; DSM-I) had only the following two cate406

reyn2-18.qxd

6/6/2003

9:37 AM

Page 407

18. Achenbach System of Empirically Based Assessment

407

FIGURE 18.1. The “bottom-up” approach that derives syndromes from statistical associations among problems. For simplicity, only two problems (e.g., Can’t concentrate) are shown for each syndrome (e.g., Attention Problems). In practice, many items are included in the analyses and about 6 to 27 items are found to comprise syndromes.

gories for such disorders: (1) adjustment reaction of childhood and (2) schizophrenic reaction, childhood type. The first ASEBA publication (Achenbach, 1966) revealed many more syndromes of behavioral and emotional problems than were evident in the two DSM categories. Later publications provided standardized procedures for assessing problems and competencies according to parent, teacher, and self-reports (Achenbach, 1978; Achenbach & Edelbrock, 1983, 1986, 1987). Since then, revised versions have featured cross-informant syndromes that embody patterns derived from ratings by multiple informants who view the subjects from different perspectives (Achenbach, 1991; Achenbach & Rescorla, 2000, 2001, 2003). Because people’s functioning often varies from one context and interaction partner to another, comprehensive assessment requires data from multiple sources. To help users see specific similarities and differences between problems manifested in different contexts, the ASEBA software compares reports by multiple informants who contribute their own perspectives on the functioning of the person being assessed. In the following sections, we first describe forms that are completed by parents, caregivers, teachers, youths, and others who can

document the everyday functioning of the individuals being assessed. All the forms obtain quantitative scores for numerous items, plus specific details about individuals’ functioning. After we describe the forms, we outline the scales and profiles on which these forms are scored. Thereafter, we present assessment procedures for use by clinical interviewers, direct observers in group settings, and professionals who administer standardized tests. We then illustrate applications to clinical and special educational services, outcome evaluations, therapeutic interventions, cross-cultural comparisons, and longitudinal, epidemiological, and etiological research.

FORMS COMPLETED BY PARENTS, OTHER COLLATERALS, CAREGIVERS, TEACHERS, AND THE INDIVIDUALS THEMSELVES The forms described in this section are designed to be completed by people who do not have specialized training in assessment but who can document the functioning of the individuals in their usual environments, such as home and school. These forms are worded in nontechnical, idiomatic language that requires only fifth-grade reading skills.

reyn2-18.qxd

6/6/2003

9:37 AM

Page 408

408

V. OBJECTIVE METHODS

For respondents who cannot read English but are literate in other languages, translations of one or more ASEBA forms are available in 65 languages (Bérubé & Achenbach, 2003). Options are also available for using scannable forms, including reflective read and TELEform formats, for direct computer entry of data by respondents, and for Web-based administrations via WebLink (Achenbach & Rescorla, 2001).

Forms Completed by Parents and Other Collaterals The Child Behavior Checklist (CBCL) is completed by parents and others who see children under home-like conditions, including relatives, foster parents, and personnel in residential treatment facilities and group homes. CBCL/1½–5 One version of the CBCL spans ages 1½–5 (CBCL/1½–5; Achenbach & Rescorla, 2000). It has 99 items that describe behavioral and emotional problems, plus an open-ended item for adding problems that were not specifically listed. Examples include Avoids looking others in the eye and Cries a lot. For each item, the respondent circles 0 if it is not true of the child, 1 if it is somewhat or sometimes true, and 2 if it is very true or often true, based on the preceding 2 months. Respondents are asked to describe problems when warranted. Respondents are also asked to describe any illnesses or disabilities that the child has, what concerns the respondent most about the child, and the best things about the child. Because language development is a common reason for concern about young children, the CBCL/1½–5 includes the Language Development Survey (Rescorla, 1989), which is completed by parents of children under the age of 3. The Language Development Survey requests respondents to report up to five of the child’s multiword phrases and to circle words that the child uses on a list of 310 words that are among the most common in young children’s vocabularies. The respondent is also asked to provide information about factors that may be associated with language delays, such as premature birth, ear infections, relatives

with language delays, and multiple languages being spoken in the home. CBCL/6–18 Another version of the CBCL spans ages 6–18 (CBCL/6–18; Achenbach & Rescorla, 2001). Like the CBCL/1½–5, the CBCL/6– 18 has items that describe behavioral and emotional problems, which respondents score as 0, 1, or 2. Many of the items have counterparts on the CBCL/1½–5, but other items tap problems that are developmentally relevant to older ages. Respondents base their ratings on the child’s functioning over the preceding 6 months. To assess developmentally appropriate competencies, the first two pages of the CBCL/6–18 request information about the child’s functioning in sports, nonsports activities, organizations, jobs and chores, friendships, relations with significant others, playing and working alone, and school. As on the CBCL/1½–5, respondents are also asked to describe any illnesses or disabilities, what concerns the respondent most about the child, and the best things about the child. Adult Behavior Checklist Mental health and educational professionals who work with children are often involved in assessing adults, as well. This is because mental health and special educational services that begin in childhood or adolescence may continue beyond the age of 18. In addition, to properly evaluate the effects of services begun before age 18, it is often necessary to assess functioning after the age of 18. When assessing children, it may also be helpful to assess their parents using parallel instruments that are tailored to the parents’ developmental level. Based on longitudinal research on a nationally representative sample, as well as on other research in the United States and abroad, we developed the Adult Behavior Checklist for ages 18–59 (ABCL; Achenbach, 1997; Achenbach & Rescorla, 2003). The ABCL can be filled out by parents, relatives, spouses, partners, friends, and others who know an adult well. The overall format is similar to that of the CBCL. Many of the problem items have developmentally appropriate counterparts of the CBCL items, but

reyn2-18.qxd

6/6/2003

9:37 AM

Page 409

18. Achenbach System of Empirically Based Assessment

other items are more specific to adults. Longitudinal studies have shown that child and adolescent CBCL scores strongly predict ABCL scores in adulthood (Achenbach, Howell, McConaughy, & Stanger, 1995; Hofstra, van der Ende, & Verhulst, 2000, 2002).

Forms Completed by Day-Care Providers and Teachers Children’s functioning often differs between home and other settings. For assessment to be comprehensive, it should include data from people who see children in settings outside the home and who have perspectives that differ from parents’ perspectives. Teachers play vital roles in children’s lives, and children’s problems often involve their functioning in school. It is therefore important to obtain data from teachers whenever possible. Because increasing numbers of children attend day-care and preschool programs, comprehensive assessment of young children should include data from day-care providers and preschool teachers whenever possible. The following sections describe forms that we have developed to assess children from the perspectives of day-care providers and teachers. Caregiver–Teacher Report Form The Caregiver–Teacher Report Form (CTRF; Achenbach & Rescorla, 2000) is completed for 1½- to 5-year-olds by their daycare providers and preschool teachers. It has a format similar to that of the CBCL/1½–5, but 17 of the 99 problem items differ to reflect differences in home versus day-care and preschool environments. Respondents are asked to indicate whether they are primarily teachers or caregivers, the type of facility in which they see the child, the size of the child’s group or class, how many hours per week the child spends at the facility, how well the respondent knows the child, and whether the child has been referred for special services. If a child attends both daycare and preschool programs, it is helpful to obtain ratings from all relevant staff members in each setting to document similarities and differences in what is seen by different people across multiple settings. C-TRFs can then be scored on profiles for comparison

409

with each other and for comparison with profiles scored from each CBCL/1½–5 completed by parents and others who see the child in home-like settings. Teacher’s Report Form The Teacher’s Report Form (TRF) assesses the functioning of 6- to 18-year-olds in school settings, as seen by teachers, counselors, and other school personnel (Achenbach & Edelbrock, 1986; Achenbach & Rescorla, 2001). The TRF has many of the same problem items as the CBCL/6–18, but 23 of the 118 specific problem items differ to reflect differences in home versus school environments. The CBCL competence items are replaced by items assessing aspects of adaptive functioning that are evident in school. These include ratings of whether academic performance is below, at, or above grade level. They also include ratings of how hard the child is working, how appropriately the child is behaving, how much the child is learning, and how happy the child is. In addition, teachers are asked to provide achievement and ability test data if available, as well as to respond to open-ended questions about various aspects of the child’s functioning. By having each teacher complete a TRF, users can compare profiles that reflect variations in the child’s school functioning as seen by different teachers. Users can also compare the TRF profiles with CBCL profiles that reflect reports of the child’s functioning at home.

Self-Report Forms In addition to informants’ reports, comprehensive assessment requires data from the subjects themselves. Play sessions, interviews, and observations in group settings and during testing can be used to directly assess children who are too young to provide self-ratings and other standardized data about their own functioning. However, by the age of 11, most youngsters become cognitively capable of completing standardized forms analogous to the CBCL and TRF. Youth Self-Report To obtain self-reports from 11- to 18-yearolds, we have developed the Youth Self-

reyn2-18.qxd

6/6/2003

9:37 AM

410

Page 410

V. OBJECTIVE METHODS

Report (YSR; Achenbach & Edelbrock, 1987; Achenbach & Rescorla, 2001). The YSR has many of the same competence and problem items as the CBCL/6–18, but the items are worded in the first person. Items that would be developmentally inappropriate or difficult for youths to report about themselves are replaced with socially desirable items that most youths endorse. Youths are also asked to describe their illnesses and disabilities, concerns about school, other concerns, and the best things about themselves. Adult Self-Report The Adult Self-Report for ages 18–59 (ASR; Achenbach & Rescorla, 2003) parallels the ABCL but also has sections that assess adaptive functioning in areas that are relevant to various developmental paths that adults may follow. For all respondents, there are sections assessing relationships with friends and family. For respondents who are enrolled in educational programs and/or who are working or are in the military, there are items that assess functioning in these contexts. For respondents who are married or live with a partner, there are items to assess these relationships. In addition, there are items for assessing tobacco, alcohol, and drug use. When assessing children with ASEBA forms, practitioners may find it especially useful to have the children’s parents complete the ASR to describe their own functioning. Table 18.1 summarizes the ASEBA forms described in the foregoing sections.

NATIONALLY NORMED ASEBA PROFILES Each form listed in Table 18.1 is scored on profiles made up of scales that display an individual’s scores in relation to scores for national normative samples of peers who were rated by the same type of respondents. The scores and detailed descriptions for specific items are essential for comprehensive assessment of individuals’ functioning. In addition, comparisons with scores for large representative samples of typical individuals are also essential for judging whether particular scores are deviant from the normal range. To be truly representative, normative samples should be selected by scientific sampling to give all individuals in the target population approximately equal probabilities of being included in the sample. Such samples are called probability samples. The ASEBA profiles provide norms based on probability samples for which subjects were randomly selected according to scientific sampling procedures (for details, see Achenbach & Rescorla, 2000, 2001, 2003). A major purpose of the profiles is to enable users to distinguish between individuals whose scores are deviant enough to indicate a need for help and individuals who are in the normal range. Accordingly, the norms for the profiles are based on individuals from the national normative samples who were not referred for mental health or related services in the preceding year. In epidemiological terms, the normative samples were “healthy” samples, which were deemed to provide the most appropriate comparison

TABLE 18.1. Forms Completed by Parents, Other Collaterals, Caregivers, Teachers, and the Subjects Themselves Form

Completed by

CBCL/1½–5 CBCL/6–18 ABCL C-TRF TRF YSR ASR

Parents, relatives, surrogates Parents, relatives, surrogates Parents, spouses, partners, friends Daycare providers (caregivers), teachers Teachers, other school staff Youths Adults

Age range 1½–5 years 6–18 years 18–59 years 1½–5 years 6–18 years 11–18 years 18–59 years

Note. CBCL/1½–5, Child Behavior Checklist for Ages 1½–5; CBCL/6–18, Child Behavior Checklist for Ages 6–18; ABCL, Adult Behavior Checklist; C-TRF, Caregiver Teacher Report Form; TRF, Teacher’s Report Form; YSR, Youth Self-Report; ASR, Adult Self-Report.

reyn2-18.qxd

6/6/2003

9:37 AM

Page 411

18. Achenbach System of Empirically Based Assessment

groups for individuals who are being assessed for psychopathology and adaptive functioning. In the following sections, we describe ASEBA profile scales for scoring adaptive functioning and competencies. Thereafter, we describe scales for scoring behavioral and emotional problems.

Profiles for Displaying Adaptive Functioning and Competencies CBCL/1½–5 Language is one of the most crucial aspects of adaptive functioning for young children. Children’s first use of words usually delights their parents and opens the door to vast opportunities for communication, learning, and socialization. Concerns about language development are among the most common reasons for seeking professional help for young children. To help practitioners quickly determine whether young children’s use of language is within the normal range, parents’ responses to the Language Development Survey of the CBCL/1½–5 are scored on a profile that compares the child’s vocabulary with the vocabularies of a national sample of peers of the child’s age and gender. Cutpoints distinguish between the normal range and delayed vocabulary growth. Normative comparisons are also provided for the number of multiword phrases reported on the Language Development Survey. Whether referrals for help are prompted by concerns about language development or about behavioral and emotional problems, it is usually important to evaluate both. In some cases, delayed language may contribute to behavioral and emotional problems when children are frustrated or teased because of their inability to communicate. In other cases, behavioral and emotional problems contribute to language delays, or both may stem from a condition such as a pervasive developmental disorder (PDD). Because parents and surrogates are essential sources of information about children’s language and behavioral and emotional problems, it is usually cost-effective to have the Language Development Survey, as well as the problem portion of the CBCL/1½–5, completed early in the evaluation of young children.

411

CBCL/6–18, TRF, YSR After the preschool period, children are expected to develop competencies and adaptive skills for successful functioning in multiple contexts, such as the home, school, and peer group. Like lags in language development at earlier ages, lags in competencies may contribute to behavioral and emotional problems. Such lags may also stem from behavioral and emotional problems, or they may stem from conditions that contribute both to lags in competencies and to behavioral and emotional problems. Lags in competencies may thus be associated with behavioral and emotional problems for a variety of reasons. On the other hand, strong competencies may help to prevent behavioral and emotional problems or may offset the negative effects of problems. Comprehensive assessment should therefore include assessment of competencies and adaptive functioning, as well as problems. CBCL/6–18 Competence Profile On the CBCL/6–18, respondents list the child’s favorite sports and nonsports activities. Respondents also indicate how often and how well the child does each one, compared to others of the same age. In a similar format, respondents list the organizations, clubs, teams, and groups the child belongs to; how active the child is in each one; jobs and chores the child has; and how well the child does each one. Thereafter, respondents indicate how many close friends the child has; how often the child does things with friends; how well the child gets along with siblings, other children, and parents; and how well the child plays and works alone. For children who attend school, respondents report on the child’s performance in academic subjects, receipt of special remedial services, repetition of grades, and problems in school. The data provided by the respondent are scored on a profile that displays scales for Activities, Social, School, and Total Competence. Figure 18.2 shows a computer-scored version of the competence profile for a 13-year-old boy, Robert Morane (not his real name). Handscored profiles are also available. In Figure 18.2, the graphic display shows where Robert’s scores on the Activities, So-

reyn2-18.qxd 6/6/2003 9:37 AM Page 412

412 FIGURE 18.2. Computer-scored version of CBCL competence profile for 13-year-old Robert Morane.

reyn2-18.qxd

6/6/2003

9:37 AM

Page 413

18. Achenbach System of Empirically Based Assessment

cial, and School scales fall in relation to a national normative sample of 12- to 18year-old boys. The two broken lines printed across the graphic display demarcate a borderline clinical range, from the third to the seventh percentile of the national normative sample. Scores below the bottom broken line are in the clinical range, because they are lower than scores obtained from parents’ reports for 97% of the national normative sample of nonreferred 12- to 18year-old boys. Based on the CBCL/6–18 completed by Robert’s mother, Thelma, Figure 18.2 shows that Robert’s score for the Activities scale was just below the borderline clinical range. By looking to the left of the graphic display, we can see that Robert’s score is equivalent to a T-score of 30. By looking below the graphic display, under the title Activities, we can see that Robert’s total score for the Activities scale was 5.5. Beneath Robert’s total score for Activities is his T-score of 30, followed by the letter C. The C indicates that Robert scored in the clinical range. And beneath Robert’s Tscore,