An Overview of the Use of the Child Behavior Checklist within Australia: Report

An Overview of the Use of the Child Behavior Checklist within Australia An Overview of the Use of the Child Behavior Checklist within Australia provides a comprehensive review of selected Australian studies conducted over the past 20 years that have used the Child Behavior Checklist. The strengths and weaknesses of the CBCL for use in the Australian population are highlighted. In particular, the report discusses:
Suitability of the CBCL factor structure and normative data in Australian samples, with reference to large-scale Australian prevalence studies and smaller morbidity studies.
Use of the CBCL as a diagnostic tool for Anxiety Disorder, Attention-deficit Hyperactivity Disorder, Oppositional Defiant Disorder, Conduct Disorder and Depression.
Stability of behavioral and emotional problems within Australian samples.
Cross-informant stability between parents, children and teachers of behavioral and emotional problems.
The relationship between a range of psychosocial factors and CBCL ratings.

ISBN 0-86431-627-5

9 780864 316271

An Overview of the Use of the Child Behavior Checklist within Australia

Heather Siddons and Sandra Lancaster

An Overview of the Use of the Child Behavior Checklist within Australia

Report prepared by Ms Heather Siddons and Professor Sandra Lancaster, Victoria University

ACER Press

The publisher and authors wish to thank Professor Thomas M. Achenbach for his assistance in preparing this report. This publication has adopted the convention of spelling the words ‘behavior’ and ‘behavioral’ with ‘-or’ not ‘-our’, as is most common in Australia. This decision has been taken for the sake of consistency. Proper names, such as the names of publications, that use the ‘-our’ convention have been printed as published. First published 2004 by ACER Press Australian Council for Educational Research Ltd 19 Prospect Hill Road, Camberwell, Victoria 3124 Copyright © 2004 Australian Council for Educational Research All rights reserved. Except under the conditions described in the Copyright Act 1968 of Australia and subsequent amendments, no part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical. Photocopying, recording or otherwise, without the written permission of the publishers. National Library of Australia Cataloguing-in-Publication data: Siddons, Heather. An overview of the use of the child behavior checklist within Australia: report. Bibliography. ISBN 0 86431 627 5. 1. Child Behavior Checklist - Australia. 2. Behavioral assessment of children - Australia. 3. Child psychology Research - Australia. I. Lancaster, Sandra. II. Title. 155.4 Visit our website: www.acerpress.com.au

TABLE OF CONTENTS 1

THE CHILD BEHAVIOR CHECKLIST (CBCL) AND RELATED FORMS

1

1.1 The CBCL and Related Forms

1

1.2

CBCL, YSR and TRF Scales 1.2.1 Social Competence Scales 1.2.2 Problems at Various Levels

1 1 1

2

FACTOR STRUCTURE OF THE CBCL

4

2.1

Cross-Cultural Generalisability of the 8-Factor Cross Informant Model

4

2.2

Conclusions

5

3

USE OF CBCL ACROSS AUSTRALIA

6

3.1 The Western Australian Child Health Survey (WACHS) 3.1.1 Morbidity Rates

6 6

3.2 The National Survey of Mental Health and Wellbeing: The Child and Adolescent Component

8

3.3 The CBCL in a New South Wales Sample 3.3.1 Sample Description 3.3.2 Problem Behaviors 3.3.3 Cutoff Scores

9 9 9 10

3.4 Problems and Competencies Reported by Parents of Children in New South Wales and America 10 3.4.1 Problem Items 10 3.4.2 Competence Scales 11 3.5 The CBCL in a Melbourne Urban Sample 3.5.1 Sample Description 3.5.2 Level of problems 3.5.3 Comparisons between Sydney and American Data

11 11 11 12

3.6

Conclusions

12

4

MORBIDITY STUDIES IN SELECT POPULATIONS

14

4.1 Immigrant Children and Adolescents

14 i

4.2

Clinical Populations 4.2.1 A Melbourne Clinical Sample 4.2.2 A Sydney Clinical Sample 4.2.3 Western Australian Clinic Samples

15 15 15 15

4.3

Conclusions

16

5

DIAGNOSTIC UTILITY

17

5.1

Behavioral and Emotional Problems 5.1.1 A Western Australian Sample 5.1.2 A Melbourne Sample 5.1.3 A Brisbane Sample

17 17 17 18

5.2 Anxiety Disorders 5.2.1 Identification of Anxiety Disorders 5.2.2 Measure of Anxiety Severity

18 18 18

5.3 Attention Deficit Hyperactivity Disorder 5.3.1 Diagnostic Utility 5.3.2 CBCL Scores Across DSM-IV ADHD Subtypes

20 20 20

5.4 Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD) 5.4.1 Identification of an ODD and CD Factor 5.4.2 Symptom Level Among Children with ODD and CD

21 21 21

5.5 Depression 5.5.1 Development of a CBCL Depression Scale 5.5.2 Development of a YSR and CBCL Scale Equivalent to the CDI 5.5.3 Diagnostic Utility of Proposed Depression Scales

22 22 22 23

5.6

Comorbidity

24

5.7

Conclusions

25

6

STABILITY OF BEHAVIORAL AND EMOTIONAL PROBLEMS

27

6.1

WACHS Pilot Study

27

6.2

The Port Pirie Cohort Study

27

6.3

Conclusions

28

7

CROSS INFORMANT STABILITY

29

ii

7.1

South Australian Community and Clinic Samples 7.1.1 Mean Level of Problems in a Community Sample 7.1.2 Clinical Caseness 7.1.3 Mean Levels of Problems in Community and Clinic Samples

29 29 30 30

7.2

Victorian Clinical Sample

30

7.3

A Sydney Clinical Sample

31

7.4

Conclusions

31

8

BIOPSYCHOSOCIAL FACTORS AND CBCL REPORTS

33

8.1 School related problems 8.1.1 Academic problems 8.1.2 Bullying

33 33 33

8.2 Parental Mental Health

33

8.3 Child Gender

34

8.4 Demographic factors

35

8.5

Conclusions

36

9

FINAL REMARKS

37

10

REFERENCES

38

11

APPENDICES

42

11.1 Appendix 1: Use of CBCL Across Australia - Associated Tables

42

11.2 Appendix 2: Morbidity Studies in Select Populations – Associated Tables

51

11.3 Appendix 3: Diagnostic Utility – Associated Tables

55

11.4 Appendix 4: Stability of Behavioral and Emotional Problems – Associated Tables

63

11.5 Appendix 5: Cross Informant Stability – Associated Tables

65

11.6 Appendix 6: Biopsychosocial factors and CBCL reports - Associated Tables

69

11.7 Appendix 7: Bibliography of Published Australian Studies Using the Achenbach System of Empirically Based Assessment (ASEBA) 70 11.7.1 Diagnoses 70 11.7.2 Normative and Prevalence Studies 70 iii

11.7.3 Oppositional Defiance and Conduct Problems 11.7.4 Attention Problems and Hyperactivity 11.7.5 Depression 11.7.6 Delinquency and homelessness 11.7.7 Assessment Issues 11.7.8 Neuropsychological Assessment 11.7.9 Anxiety 11.7.10 Psychosocial Factors 11.7.11 Physical Illness 11.7.12 Sexual Abuse 11.7.13 Other

iv

71 72 73 74 74 75 76 77 78 80 80

1

The Child Behavior Checklist (CBCL) and Related Forms

1.1

The CBCL and Related Forms

The CBCL is a standardised questionnaire, completed by parents or primary caregiver, which provides a measure of behavioral and emotional functioning and social competence of children and adolescents. The CBCL has two sections, social competence and problem behaviors. The original normative data is based on a United States sample of 4,455 referred and non-referred children aged 4- to 16-years for the CBCL problem behaviors and a sample of 2,368 non-referred children aged 4- to 16-years for the competence scales (Achenbach & Edelbrock, 1983). Some revision to the CBCL factors was made and revised normative data for children aged 4- to 18-years were released in 1991 (Achenbach, 1991). An extensive review of the American normative data is provided in the CBCL manual (Achenbach, 1991; Achenbach & Edelbrock, 1983). Recently, the CBCL was updated “to incorporate new normative data, include new DSM-oriented scales, and to complement the new preschool forms” (Achenbach, 2002). The new version of the CBCL is suitable for children aged 6- to 18-years and the preschool version is appropriate for children aged 1½- to 5-years. The YSR and TRF are essentially parallel forms of the CBCL to be completed by the young person and teacher, respectively. The original versions of the YSR and TRF are normed for ages 11- to 18-years and 5- to 18-years respectively. The 2001 versions of the YSR and TRF are normed for ages 5- to 18-years and 6- to 18-years, respectively. The original versions of the CBCL, TRF and YSR contain 89 common items, thus allowing for cross-informant comparisons. Note, the 2001 editions have 93 items in common. The majority of research summarised in this report utilised the 1983 form or 1991 forms.

1.2

CBCL, YSR and TRF Scales

1.2.1

Social Competence Scales

This section contains 20 items and is designed to measure children’s positive adaptive functioning. Responses provide measures on 3 subscales: Activities: The amount and quality of participation in sports, hobbies, games, activities, jobs and chores Social: Friendships, how well child gets along with others, behaves, and plays and works alone School: Academic performance, special class, repeated grade, school problems 1.2.2

Problems at Various Levels

The CBCL contains 118 items describing a broad range of problems. There are also two items on which informants may provide additional information through open-ended responses. Responses are used to provide a measure of behavioral and emotional functioning on four different levels: Total problem score; Broad-band scores; Syndrome scale scores; Item scores. 1.2.2.1

Item Scores

The respondent is required to indicate how well each item describes their child’s behavior within the past 6-months, using a three-point scale (0 = not true, 1 = somewhat or sometimes true, 2 = very true or often true).

1

1.2.2.2

Syndrome Scale Scores

Factor analysis, conducted separately for each sex/age groups, identified the 8 syndrome scales, which are computed by summing responses to the relevant individual items. The initial factor analysis utilised varimax rotation, which means that the rotated factors are uncorrelated. There are 8 comparable syndrome scales that can be computed from the CBCL, TRF and YSR (see Table 1). Table 1. CBCL and Cross Informant Scales 1991 Version CBCL/4-18

Cross informant scales

Behavior scales: Withdrawn Somatic Complaints Anxious/Depressed Social Problems Thought Problems Attention Problems Delinquent Behavior Aggressive Behavior Sex Problems (age 4-11) Externalising problems Internalising problems Total problems scale Competence scales: Activities Social School Total competence

Behavior scales: Withdrawn Somatic Complaints Anxious/Depressed Social Problems Thought Problems Attention Problems Delinquent Behavior Aggressive Behavior Externalising problems Internalising problems Total problems scale Competence scales: Activities Social Total competence

2002 Version CBCL/1½-5 (caregiver/teacher form) Behavior scales: Withdrawn Somatic Complaints Anxious/Depressed Emotionally Reactive Aggressive Behavior Attention Problems Sleep problems Externalising problems Internalising problems Total problems scale

Competence scales: Language Development Survey

DSM-IV oriented scales: Affective Problems Anxiety Problems Pervasive Developmental Problems Attention Deficit/Hyperactivity Problems Oppositional Defiant Problems

1.2.2.3

CBCL/6-18 Behavior scales: Withdrawn/Depressed Somatic Complaints Anxious/Depressed Social Problems Thought Problems Attention Problems; Rule-Breaking Behavior Aggressive Behavior Externalising problems Internalising problems Total problems scale Competence scales: Activities Social School Total competence DSM-IV oriented scales: Affective Problems Anxiety Problems Somatic Problems Attention Deficit/Hyperactivity Problems Oppositional Defiant Problems Conduct Problems

Broad Band Scales

A second order principal factor analysis with varimax rotation of the correlations among the scale scores, conducted separately for each sex/age group, identified two broad band scales: Internalising problems and Externalising problems. The internalising factor reflects problems of withdrawal, somatic complaints, and anxiety/depression, whilst the externalising factor reflects delinquent and aggressive behavior. 2

The attention problems syndrome scale loaded highly on the Externalising factor (0.618). However, the loading was considered significantly lower than the aggressive and delinquent behavior loadings and therefore deemed inappropriate to include with the Externalising grouping. Neither the Social Problems nor the Thought Problems scales had consistently high loadings on either the Internalising or Externalising factor.

3

2

Factor Structure of the CBCL

This section reports research findings regarding the validity of using the CBCL factor structure with Australian children. Research examining the identification of new factors for oppositional defiant disorder, conduct disorder and depression are discussed in Section 5.

2.1

Cross-Cultural Generalisability of the 8-Factor Cross Informant Model

A recent study examined the cross-cultural generalisability of the 1991 8-factor cross-informant model of the CBCL for clinically referred children and adolescents from Australia, America and Holland (2000). Thus, confirmatory factor analyses were performed using only the 85 cross-informant items within each sample. The Australian sample comprised 2237 children (1523 boys, 714 girls) who had attended a mental health service within New South Wales during the period 1983-1997. Approximately 59% of the boys were aged less than 12-years, whilst the remaining boys were 12-years or older. Approximately 37% of the girls were aged less than 12-years, with the remainder of the girls 12-years or older. Ninety-percent of the informants were mothers, 5% fathers, 3% others, 2% unknown. The majority of participants were of Caucasian background. The American samples used were the CBCL 1991 clinical sample (n = 2110) (Achenbach, 1991) and a sample of 631 children and adolescents aged 8- to 18-years with severe emotional problems who had participated in a national treatment study (Dedrick, Greenbaum, Friedman, Wetherington, & Knoff, 1997, cited in Heubeck, 2000 #111). The Dutch sample comprised 2335 children and adolescents aged 4- to 18years recruited through mental health clinics. Heubeck (2000) utilised confirmatory factor analyses on the 1-factor and 8-factor models developed by Achenbach (1991). The analyses yielded important results. There was good support for a 1-factor model within the American, Dutch and Australian samples. Thus, overall the CBCL seemingly represents a basic psychopathology factor. The 8-factor model developed by Achenbach utilised a varimax rotation, which statistically ‘forces’ the factors to be independent. Using a varimax rotation, Heubeck’s results indicated that the uncorrelated 8factors model does not fit the Australian, American and Dutch data. However, Heubeck demonstrated that use of an alternative rotation method, which allows the factors to be correlated, results in an 8-factor model providing a better fit of the Australian, American and Dutch data compared to the 1-factor model. The confirmatory factor analyses (with correlated factors permitted) revealed that approximately 90% of the items loaded on the factors that they are purported to represent. Best convergent validity was shown for items measuring somatic complaints, anxious/depressed and aggressive syndromes, with the majority of items demonstrating a factor loading of at least 0.30 on the factors which they were assigned to by Achenbach (1991). The withdrawn, thought problems and delinquent syndromes also demonstrated good convergent validity using the Australian, Dutch and American data, though the confirmatory factor analysis identified additional items on each of these factors. Nevertheless, Heubeck advised that the withdrawn, somatic complaints, anxious/depressed, thought problems, delinquent, and aggressive behavior syndromes may be “used with some confidence” in Australia (p, 445). In contrast, relatively poor support was found for the CBCL factors alleged to measure attention and social problems (Heubeck, 2000). Using the Australian data, 4 of the items purported by Achenbach (1991) to measure attention received loadings of less than 0.30. In terms of attention, only 3 (8. 4

concentrate; 10. sit still; and, 41. impulsive) of the 14 items loaded on the same factor across the three countries. Heubeck (2000) suggested that the attention factor proposed by Achenbach may benefit from substantial revision, perhaps incorporating recent advances which have suggested that ADHD may be better defined along two dimensions, inattention and overactivity. Such revisions have been made in the 2001 revision of the correlated 8-factor model (Achenbach & Rescorla, 2001), which provides Inattention and Hyperactivity-Impulsivity subscales for scoring the TRF. The confirmatory factor analysis conducted by Heubeck (2000) provided very poor support for the social problems factor and Heubeck argued that the “social problems factor needs a major reconceptualisation” (p. 456). Heubeck’s analysis identified only 3 items purported to measure social problems (25. not get along with other kids, 38. teased, and 48. not liked) that had an adequate loading on the same factor using the Australian, American and Dutch samples. From an additional exploratory factor analysis, Heubeck identified a number of extra items that loaded on the social problems factor. The inclusion of these items on the social problems factors seemed to change the meaning of the factor, so that perhaps it better describes a child who may be “rejected, but who is mean, destructive, antisocial, and probably a bully” (p. 456), rather than a child who may be “immature and clumsy and who does not get along well with peers” (p. 456). Heubeck discussed the issue of items loading on more than one factor. The confirmatory factor analyses demonstrated that none of the five items purported by Achenbach to load on more than one factor actually did so, while items 45 (nervous) and 103 (sad) received substantial loadings on more than one factor using either the American, Dutch or Australian data. A number of other cross-factor loadings were also identified by Heubeck who advised that a revised version of the CBCL should incorporate such cross loadings. The revised correlated 8-factor model published in 2001 along with other changes, eliminated cross-loading items (Achenbach & Rescorla, 2001).

2.2

Conclusions

There is good support for the use of the CBCL 1-factor model (i.e. total behavior score) and 6 of the 8 CBCL syndromes (withdrawn, somatic complaints, anxious/depressed, thought problems, delinquent, and aggressive behavior) within clinic samples in New South Wales. Conversely, less confidence may be placed in use of the attention and social problems syndrome scales. However in terms of overall validity, the study by Heubeck (2000) is somewhat limited, as the Australian sample were drawn from mental health clinics within New South Wales and may not be representative of children from a non-clinical population and/ or children in other Australian states. Thus, the CBCL factor structure requires further validation to improve confidence that it is applicable to all Australian children.

5

3

Use of CBCL Across Australia

This section provides a summary of studies that have used the CBCL to assess morbidity of mental health problems among young Australians and/or the appropriateness of using the American norms with Australian populations. (Achenbach, Hensley, Phares, & Grayson, 1990; Bond, Nolan, Adler, & Robertson, 1994; Hensley, 1988).

3.1

The Western Australian Child Health Survey (WACHS)

The Western Australian Child Health Survey (WACHS) is a comprehensive study of the prevalence of mental health problems among children aged 4- to 16-years living in Western Australia (Garton, Zubrick, & Silburn, 1995). A range of measures was used to screen for mental health morbidity, including the CBCL and parallel forms (YSR, TRF). 3.1.1 3.1.1.1

Morbidity Rates Pilot Survey

A pilot study was conducted in 1992 on a random sample of 260 Perth metropolitan area households, of which 189 agreed to participate (Garton et al., 1995). Each household was provided with a CBCL, YSR and TRF to be completed and returned by post. The 189 households provided data for 321 young people (163 male, 158 female). Of the 189 households who agreed to participate, response rates were as follows: 96.3% CBCL, 94.5% YSR (12- to 14-years), and 93.0% (YSR 15- to 16-years). Morbidity of mental health problems was determined by including all children who scored greater than the 98th percentile (T score 70 or more) for at least one CBCL syndrome. Results indicated that 11.2% (n=36) of the pilot sample had deviant scores on at least one mental health syndrome. The prevalence of elevated scores on one or more syndromes increased to 19.3% if a cutoff at the 95th percentile (T score of 67 or more) was used. The mean CBCL raw scores and T scores by gender and age of the sample used in the WACHS pilot study revealed slightly, though not markedly, lower levels of total behavioral and emotional problems compared to the US 1991 norms (Table 2) (Garton et al., 1995). Table 2. Mean CBCL Raw and T Scores, by Gender and Age for the WACHS Pilot Sample (n = 321) and US Norms (Taken from Garton (1995)) Boys Girls US WACHS US WACHS US WACHS WACHS US 4-11 4-11yrs 4-11 yrs 12-16yrs 12-18 yrs 4-11yrs 12-16yrs 12-18 yrs yrs Raw Score Mean 22.6 24.3 21.3 22.5 17.9 23.1 17.7 22.0 SD 17.2 15.6 15.3 17.0 16.2 15.5 14.6 17.7 T Score Mean 48.6 50.1 49.5 50.0 45.5 50.1 46.9 50.0 SD 10.8 9.9 9.8 10.0 11.7 9.9 10.9 10.2

6

3.1.1.2

Main WACHS Survey

The main WACHS survey was conducted in 1993 and estimated morbidity rates of behavioral and emotional problems based on data on 2737 children aged 4- to 16-years living in Western Australia. Analyses yielded an overall prevalence of 17.7% for mental health morbidity. This figure was based on a participant being identified as a case by a T score equal to or greater than 60 on the CBCL and/or TRF. Prevalence of mental health morbidity for the entire WACHS sample is reported in Table 3. A greater proportion of boys than girls (20% versus 15.4%) were identified as having mental health problems (Zubrick et al., 1995). Table 3. Prevalence (%) of Mental Health Morbidity Within and Across Informants (Taken from Zubrick et al. (1997)) Age group Source

4-11 years

12-16 years

All children

Parent report

10.0

11.3

10.4

Teacher report

11.4

16.4

13.3

Not collected

35.7

24.6

Parent, teacher & youth reports combined

16.1

35.3

23.1

Parent & teacher reports combined

16.1

20.3

17.7

Youth report

Morbidity on individual syndromes was defined by having a T score on the CBCL and/or TRF equal to or greater than 67. Nearly 28% of children and adolescents were identified as being in the clinical range for at least one syndrome, 15.9% of whom were also identified as having overall behavioral/emotional problems. The morbidity rates for each CBCL syndrome are reported in Table 4. Table 4. Pecentage of Children With Mental Health Problems: Type of Problem, According to CBCL/TRF Reports (Taken from Zubrick (1995)) Sex Age group (years) Males

Females

4-11

12-16

All children

Delinquent problems

10.5

8.5

10.1

8.6

9.5

Thought problems

9.6

7.6

7.5

10.4

8.6

Attention problems

6.6

5.9

5.5

7.6

6.3

Social problems

7.0

4.7

5.7

6.0

5.9

Somatic complaints

7.0

3.1

4.7

5.6

5.0

Aggressive behavior

4.2

3.2

3.1

4.6

3.7

Anxiety/depression

4.7

2.6

3.0

4.8

3.6

Withdrawn

3.1

2.1

2.5

2.7

2.6

7

Adolescent suicidal ideation and deliberate self-harm were assessed via two items on the YSR. The WACHS report indicated that an estimated 15% of students had experienced suicidal ideation (22% adolescents aged 15-16 years and 12% adolescents aged 12-14 years) (Zubrick et al., 1995; Zubrick et al., 1997). A prevalence of 7.5% was reported for deliberate self-harm (8.6% adolescents aged 15-16 years and 6.8% adolescents aged 12-14 years). This figure received some validation from the results of a nation-wide survey conducted by the Centre for Diseases Control in USA, which reported a prevalence of 8.3 for deliberate self-harm among 11,000 high school students (Zubrick et al., 1997).

3.2

The National Survey of Mental Health and Wellbeing: The Child and Adolescent Component

One of the aims of the child and adolescent component of the National Survey of Mental Health and Wellbeing was to estimate the proportion of Australian children and adolescents with specific mental health disorders (Sawyer et al., 2000). The final sample surveyed was representative of Australian children and adolescents aged 4- to 17-years. Of those households identified as a having an eligible child, 86% agreed to participate, giving a response rate of 70%. Analyses revealed limited response biases. The CBCL was completed for 4083 children and adolescents (2082 male, 2001 female). The distributions of CBCL and YSR respective mean scores were consistent with the results reported in the WACHS (Sawyer et al., 2000). As in the WACHS, prevalence estimates of mental health problems were calculated using the cut-offs recommended by Achenbach (1991). Prevalence estimates (Sawyer et al., 2001) are reported in Table 5. Using the CBCL cutoffs, 573 children were identified as having a clinically significant mental health problem, whilst a further 500 were classified as having ‘sub threshold’ problems (CBCL T score 54-59). Table 5. Prevalence (%) of Mental Health Problems in 4-17-Year-Old Children (Taken from (Sawyer et al., 2001) Total % (n = 4083)

Males (n = 2082)

Females (n = 2001)

Total problems

14.1 (521 886)*

14.4

13.9

Externalising problems

12.9 (475 748)*

12.9

12.9

Internalising problems

12.8 (473 989)*

14.5

11.1

Somatic complaints

7.3

8.4

6.1

Delinquent behavior

7.1

7.1

7.1

Attention problems

6.1

6.5

5.6

Aggressive behavior

5.2

5.6

4.8

Social problems

4.6

5.6

3.6

Withdrawn

4.3

5.2

3.4

Anxious/depressed

3.5

3.9

3.2

Thought problems

3.1

3.3

2.8

CBCL Scale Broad band scales

Syndrome scales

8

*Population estimate The slightly lower prevalence estimate of 14% compared to the approximate WACHS 18% is accounted for by the fact that the WACHS prevalence estimate was based on combined parent and teacher reports, whereas the prevalence estimate of the National Survey was based solely on parent report. Comparison of parent and adolescent reported problems across surveys yielded very similar prevalence estimates (Sawyer et al., 2001). Correlations between informants were significant for clinical and sub-threshold caseness, and a range of other problems, after controlling for demographic factors (Sawyer et al., 2001). Some validity of the CBCL findings was obtained. Children with CBCL clinically significant problems were also rated by parents to have poorer self-esteem, more emotional and behavioral problems, poorer general health and greater pain and discomfort, as measured by the Child Health Questionnaire (CHQ), than children with sub-threshold problems. In turn, children with sub-threshold problems were reported to have more problems on the CHQ than non-clinical children. Children in the clinical and sub-threshold groups were also more likely to report suicidal ideation and behavior, and risk-taking behavior compared to the non-clinical group, even after controlling for operational confounds.

3.3

The CBCL in a New South Wales Sample

3.3.1

Sample Description

In an earlier study, Hensley (1988) used the CBCL with an Australian sample, comparable to the American 1981 normative sample with respect to age, gender and method of sample recruitment. The final sample for the study comprised 1300 children aged 4 through 16 years, with 50 children within each yearly age/sex group. Seventy-eight children who had received some form of psychological evaluation or treatment by a psychologist, psychiatrist or school counselor were excluded from the normative sample of 1300 non-referred children. In contrast to the American 1981 normative sample, the Australian sample was entirely metropolitan and urban (within Sydney, New South Wales). Efforts were made to replicate the American method of data collection. Ethnic variation within the Australian sample was comparable to the 1981 Census for the Sydney population, though there was a greater representation of immigrant families compared to the American normative sample. The non-inclusion of rural families and a larger proportion of immigrant families in the Australian sample may marginally limit the value of making comparisons to the American sample. The sample cannot be assumed to represent all areas of Australia (Achenbach et al., 1990) or the current Australian population . The proportion of the Australian sample within each socio-economic category generally fell between the proportion of the American clinical and the American normal samples. However, a significantly greater proportion of the Australian sample than the American normal sample fell within the unskilled category (which included the unemployed, single mothers and invalid pensioners). 3.3.2

Problem Behaviors

Results indicated that morbidity of total behavior problem, internalising and externalising problems for each age by gender group was significantly greater among the Australian sample compared to the American sample. Though somewhat less striking, there was a trend for the Australian sample to have poorer social competence, with the exception of activities, where 4- to 5-year old Australian boys and 6to 11-year old Australian girls were involved in more activities than their American counterparts. 9

Tables 1 to 6 in Appendix 1 display the mean raw scores of the problem behavior broad-band scales, total behavior problem score, social competence subscales and total scores, and t-values for contrasted pairs of means. Using a significance level of 0.01, t-values greater than 2.57 indicate significant differences between the Australian and American mean scores. It is unclear why the Australian children were rated to have significantly higher levels of behavior problems and poorer social competence. Subsequent analyses suggest that the differences could not be accounted for by disparate distributions of socioeconomic status between the countries. A more plausible explanation is the geographic location. Both the Australian and American samples were recruited from metropolitan and urban areas. However, a proportion of the American sample was also recruited from semi rural environments. International and Australian prevalence studies have reported higher incidence of psychiatric problems in urban and metropolitan areas (Connell, Irvine, & Rodney, 1982 and Rutter, 1975 #158, cited in Hensley, 1988 #6). Nevertheless, a study conducted by Bond et al. (1994) (described later) showed Sydney children to have higher levels of problem behaviors than Melbourne children living in urban metropolitan regions. 3.3.3

Cutoff Scores

Hensely (1988) also reported adjusted clinical cut-offs at the 90th percentile using the Australian data (Appendix 1, Tables 7 to 17) for the total and broad-band scales and competence scale. Similarly adjusted cutoffs were suggested for the syndrome subscales. However, Hensley urged that the modified cutoff points could not be used with confidence until a large clinically based Australian study is conducted.

3.4

Problems and Competencies Reported by Parents of Children in New South Wales and America

In a subsequent study, Achenbach, Hensley, Phares and Grayson (1990) examined item responses to compare problems and competencies reported for the sample of 1300 non-referred children from Sydney and the American non-referred normative sample (n = 1300). 3.4.1

Problem Items

A series of ANCOVA’s (SES as the covariate) were computed to examine differences between the Australian and American samples on every problem item and the total problem behavior score. As high statistical power makes it possible to detect very small effects, the authors utilised Cohen’s criteria for judging the magnitude of each effect, rather than merely statistical significance. An effect size accounting for 1-5.9% of the variance is considered small. An effect size accounting for 6.0-13.8% of the variance is considered medium. An effect size accounting for >13.8% of the variance is considered large. The Australian sample showed higher scores than the American sample on 80 specific items, the two open-ended items and the total behavior problem score. These differences were generally applicable across age and gender however the pattern of sex differences among the Australian sample was similar to the pattern of sex differences in the American sample. Also, 54 of the differences in problems endorsed were judged as having small nationality effects, whilst 23 accounted for less than 1% of the variance. Only one item (96: Thinks about sex too much) was identified as having a large nationality effect, with the item being endorsed by 39% of the Australian parents versus 2% of the American sample. Four items 10

and the total behavior problem scores showed medium nationality effects (see Table 18, Appendix 1). Nationality interacted with age for item 60 (Plays with sex parts too much), whereby the difference between the Sydney and American scores was less at ages 10- to 11-years and 16-years. 3.4.2

Competence Scales

The American children were rated as having significantly better social competence scores for 10 of the 20 items, and the Social Scale and Total Competence Scores (see Table 19, Appendix 1). Importantly, the majority of differences between the Sydney and American samples on the competency items yields a small effect size or accounted for less than 1% of the variance. Medium difference effects were demonstrated for two items: American parents reported their children to have significantly more contact with friends and to be involved in more sports; and Sydney parents reported their children as having significantly more friends than American children. Exclusion of the children who were rated within the clinically significant range on the Total Behavior Problem Score (29% Australian, 10% American) did not significantly alter the nationality difference in the number of problem behaviors. Sydney parents endorsed a significantly greater number of problem items than American parents, even when coding the items responses dichotomously. Moreover, Sydney parents endorsed a significantly greater number of items as occurring frequently than did American parents. The items endorsed by Sydney and American parents were not specific to either internalising or externalising problems.

3.5

The CBCL in a Melbourne Urban Sample

3.5.1

Sample Description

Bond et al. (1994) conducted a study to compare CBCL scores for a sample of non-referred Melbourne children with the revised 1991 American normative data and the New South Wales data (Hensley, 1988). The study was part of a larger study on asthma prevalence and morbidity. All children attending Years 2, 7 and 12 (7-, 12-, and 15-year-olds, respectively) in a random selection of Government, Catholic and independent schools in metropolitan Melbourne and surrounding areas were invited to participate. In total, 1,774 households completed the CBCL (46% response rate). The sample was reduced to reflect the prevalence of asthma, leaving 1051 children of whom 228 were asthmatic. The final sample, after excluding children who had been seen by a psychologist or psychiatrist, consisted of 1009 children (564 Year 2, 250 Year 7 and 195 Year 10; 63% boys). It is important to acknowledge that the study conducted by Bond et al. (1994) did not permit conclusions about prevalence rates because of the high attrition rate. Comparisons between the Melbourne sample and the Sydney and American samples is somewhat limited by the lack of representation of children at each age level. 3.5.2

Level of problems

Analyses of the Melbourne sample indicated that overall girls were reported to have higher levels of internalising problems, whilst boys had higher levels of externalising problems. The 7-year-olds were rated to have greater levels of total behavior problems and externalising problems than older children. The 12-year-olds were also rated higher than 7-year-olds on the activities and social subscale and the overall competence score. The 12-year olds were also rated as having better social competence, but also 11

had higher levels of externalising problems than the 15-year-olds. Mean scores are presented in Table 20, Appendix 1. 3.5.3

Comparisons between Sydney and American Data

The data collected by Bond et al. (1994) was compared to the 1991 revised American norms and to the Sydney data (re-scored using the revised scales). Means are presented in Appendix 1, Tables 21 and 22. The children in Years 7 and 10 were combined and overall, Sydney children were rated as having more behavior problems than Melbourne children, even after controlling for differences in socio-economic status. The differences between the Sydney and Melbourne samples were larger than the differences between the American and Melbourne samples on the total problem, internalising and externalising scores. Comparable to the Sydney and American item differences, a significant difference in mean item ratings between the Sydney and Melbourne data was found for 77 items. The Melbourne sample scored significantly higher on 8 items only, many of which referred to somatic problems. This may be reflective of the deliberate inclusion of children with asthma, whose parents may be more sensitive and likely to report physical problems. In particular, the Sydney sample was identified as scoring significantly higher than the Melbourne sample on the following items: 113. Other problems; 83. Stores up unneeded things; 96. Thinks about sex too much; 112. Worries; 86. Stubborn; 38. Is teased; 45. Nervous; 109. Whining; 93. Talks too much; and, 27. Jealous. All of these items (excluding item 27 ‘jealous’) were also identified by Hensely (1988) as being more problematic for the Sydney children compared to the American children. National differences emerged for the competence scales, where both Sydney and Melbourne children were reported to be more involved in activities and less involved in social organisations than American children.

3.6

Conclusions

The CBCL has been used to provide morbidity estimates among non-referred Australian children and adolescents. Reported estimates (using a criterion of T score ≥ 60 for total problem scale) have ranged from 10.0 to 20.3, depending on the whether the informant was parent, teacher, or combination. Somewhat higher morbidity estimates (24.6 to 35.7) have been reported using the YSR. The WACH also estimated 28% morbidity on at least one syndrome. This estimate was based on a T score ≥ 67 on either the CBCL or TRF. The morbidity estimates are likely to be over-inclusive, as Achenbach’s cutoffs for borderline levels of clinical problems were used. The WACHS indicated that 15% of children and adolescents endorsed the item assessing suicidal ideation and 7.5% endorsed the item assessing deliberate self-harm. These rates were higher than those indicated in parent and teacher reports, highlighting the importance of asking the young person about their wellbeing. Research has also shown the potential use of the CBCL to estimate odds of having particular problems, such as academic problems, bullying and familial problems. The similar distribution of problems across the WACHS and Child and Adolescent Component of the National Survey of Mental Health and Well-Being provide support for the reliability of the CBCL in Australian populations. Research using the original CBCL factor structure (Achenbach & Edelbrock, 1983) suggests a higher mean level of parent-reported behavioral and emotional problems and poorer social competence among children in New South Wales than America, though the patterns of sex differences were similar. Thus, 12

use of American norms may identify a greater number of Australian children as having clinically significant problems. Adjusted norms have been proposed using a non-clinical sample drawn from New South Wales. However, this may not be warranted as the size of cross-cultural differences was generally of small effect. In contrast, there is minimal evidence indicative of significant differences in parent reported problem behaviors between the Melbourne and Western Australian non-clinical samples and the American nonclinical normative sample, using the revised CBCL scales (Achenbach, 1991). However, as with the American sample, Melbourne children were generally shown to have fewer behavioral and emotional problems than children in New South Wales. Differences in methodology, sample demographics, and duration of studies mean that caution must be exerted when using either the American or suggested Australian norms for Australian samples. The study using a New South Wales sample utilised the original version of the CBCL (Achenbach & Edelbrock, 1983), whilst the Melbourne study and the WACHS used the 1991 revisions (Achenbach, 1991). However, it is highly unlikely that these changes would have altered the main finding that morbidity rates for problem behaviors and poor social competence are significantly higher among Sydney children than the American normative sample. As stated by Bond et al. (1994), despite the average increase of 3 points for the problems scores using the 1991 revision, the data presented by Hensley remains significantly higher than the American norms. Since these studies were conducted, the CBCL has undergone yet further revisions. Clearly, research is required to examine the utility of the most recent American norms within Australia and also to examine the utility of American YSR and TRF norms within Australia.

13

4

Morbidity Studies in Select Populations

The previous section provided an overview of studies that used the CBCL to estimate morbidity of mental health problems amongst Australian children and adolescents. This section summarises some research that used the CBCL to estimate morbidity rates using samples of Australian immigrants and children attending psychiatric services.

4.1

Immigrant Children and Adolescents

The level of behavioral and emotional problems and competencies among Australian immigrant children has been examined using the CBCL and YSR (Davies & McKelvey, 1998; Goldney, Donald, Sawyer, Kosky, & Priest, 1996). The study conducted by Goldney and McKelvey (Davies & McKelvey, 1998; Goldney et al., 1996) utilised a sample of 209 adolescents aged 12- to 16-years living in Perth and attending mainstream schools. An additional 53 adolescents were recruited though an Intensive Language Centre in Perth. In total, the CBCL was completed for 255 adolescents. The YSR was also available for 211 participants. Ninety-four (36.9%) participants were born overseas. The mean age of the immigrant adolescents was 13.9 years, whilst the mean age of the non-immigrant adolescents was 14.13 years. According to parent reports immigrant adolescents had fewer externalising problems than the nonimmigrant adolescents. In contrast, parent reports indicated that immigrant adolescents had fewer social competencies than non-immigrant adolescents. Similarly, self-reports (using YSR) completed by immigrant adolescents indicated fewer externalising and overall problems, but also fewer social competencies than non-immigrant adolescents. There was no reported difference in levels of internalising problems between immigrant and non-immigrant adolescents. Immigration status remained predictive of YSR (but not CBCL) externalising and competencies scores, even after controlling for SES, family composition, age, gender, parental immigration status, primary language and school setting. Goldney, Donald, Sawyer, et al. (1996) compared the levels of parent and self-reported behavioral and emotional problems of Indonesian adoptees (23 males, 11 females) with a mental health clinic population (68 males, 32 females) and a community sample of 100 two-parent families. Within the community sample the YSR report was completed by 116 males and 117 females, and the CBCL was completed for 121 males and 120 females. The average age at adoption for the Indonesian sample was 1 year 5 months. All children were aged 14- to 15-years at the time of participation in the study. Results showed no significant differences in levels of behavioral and emotional problems between the community and adoptee samples, according to both parent and self-report. Overall, both community and adoptee samples were shown to have significantly fewer behavioral and emotional problems than the clinic sample (see Appendix 2, Tables 1 to 4). The adoption sample did not differ from the clinic sample for female self-report of externalising problems, thought disorder and aggressive behavior, and parent report for females on the schizoid and delinquent subscales. The lack of significant differences was most likely due to limited power related to the very small sample size of adopted females. Consistent with other studies, children reported greater levels of problems than did parents.

14

4.2

Clinical Populations

4.2.1

A Melbourne Clinical Sample

4.2.1.1

Clinical Cutoff

The applicability of the US normative CBCL cut-off scores for the classification of behavior disorders was assessed using a Melbourne sample of 1342 children referred to a mental health outpatient service between July 1991 and October 1992 (Nolan et al., 1996). A Melbourne community sample (described previously) was used as a comparison sample (Bond et al., 1994). According to Achenbach (1991), on the behavior scales, a T score less than 60 is classified as non-clinical, a T score of 60-63 is borderlineclinical, and a T score of >63 is clinical. The mean T scores for referred children were above the clinical range across age and gender (see Table 5, Appendix 2). On the competence scales, a T score below 30 is considered clinical (and below 33 is considered borderline-clinical), as low competence scores are clinically important. Compared to the American clinical data, a greater proportion of clinically referred Melbourne children scored above the CBCL clinical cutoff (T score 60+) for the total behavior, externalising, internalising, and syndrome scale scores. Moreover, a greater proportion of clinically referred Melbourne children scored below the CBCL clinical cutoff (T score 30) for the total social competence score, and activities and social subscales. Conversely, a greater proportion of the clinically referred American children scored above the cutoff for school problems. A comparison between a sub-sample of the Melbourne clinical (n = 1342) and Melbourne non-clinical (n = 1009) samples indicated a number of demographic risk factors among the referred sample, including lower maternal education, lower socio-economic status, and blended/split families. 4.2.2

A Sydney Clinical Sample

Rey, Grayson, Mojarrad and Walter (2002) retrospectively examined the rate of diagnosis of major depression in a sample of 1310 adolescents aged 12- to 17-years referred to a mental health service in Sydney between 1993 and 1997. Mean CBCL and YSR raw scores for the total problems scale, externalising scale, internalising scale and anxious/depressed scale and proportions of DSM diagnoses are reported in Table 6, Appendix 2. The study did not report correlations between the CBCL scores and DSM diagnoses. However, the CBCL and YSR total scores were seemingly non-specific to DSM diagnoses (Rey et al., 2002), as a wide variety of diagnoses were made. This is not surprising given that the CBCL total score provides an overall measure of behavioral and emotional problems. The study also indicated that the mean levels on the anxious/depressed subscale were consistent among children referred across the study time period (see Table 6, Appendix 2), suggesting that rates of anxious/depressed symptomatology did not change between 1993 and 1997. 4.2.3

Western Australian Clinic Samples

Paterson, Bauer, McDonald and McDermott (1997) compared data from a sample of 58 consecutive psychiatric inpatient children and adolescents (mean age 11.3 years, range 8- to 16-years) to the Western Australian Child Health Survey data on mean levels of total behavior and emotional problems according to the CBCL, YSR and TRF. Results are reported in Appendix 2, Table 7. The mean level of psychopathology was significantly higher and number of ‘cases’ identified significantly greater among the psychiatric inpatients according to all informants. Consistent with other studies examining cross 15

informant stability inpatient children reported significantly fewer problems than their parents, whilst the opposite is true for the normative sample. Paterson et al. also reported an approximately equal prevalence of internalising and externalising problems within the psychiatric sample. A study conducted by McDermott, McKelvey, Roberts and Davies (2002) compared levels of behavioral and emotional problems in children receiving treatment for behavioral and emotional problems in one of four treatment settings (inpatient care, day treatment, outpatient care and consultation only) in Western Australia. Both the CBCL and YSR were completed. There were significant differences across treatment settings in parent report of problems (see Appendix 2, Table 8), with inpatient and day treatment children rated as having significantly more total and externalising problems than children receiving treatment via an outpatient service or through consultation only. Inpatient children were rated as having significantly greater internalising problems than children receiving treatment via an outpatient service or through consultation only. The severity of problems reported in the inpatient and day treatment samples are comparable to the previously described inpatient sample (Paterson et al., 1997). Similar patterns emerged based on the YSR, which was completed by all children and adolescents aged 11- to 17 years, with the main difference being that young people in a day treatment program did not report more problems than children receiving outpatient care or consultation only. Competency levels were significantly lower in the inpatient and day treatment care than the outpatient care or consultation only, as indicated by both parent and self-report (see Appendix 2, Table 8).

4.3

Conclusions

The CBCL has been used to assess levels of problems among immigrant children living in Australia. Overall, immigrant children and adolescents are reported to have similar or fewer behavioral and emotional problems than Australian born non-referred children and adolescents and significantly lower levels of problems than referred Australian born children. The CBCL reports suggest that immigrant children may have poorer social competence than non-immigrant children. However, this may be due to cross-cultural differences in social behavior rather than poorer social competence, per se. Studies using clinical samples have reported mean total raw scores in excess of 60. Research has shown a greater proportion of Melbourne clinically referred children than the American clinical normative sample score within the clinical range and that parents of referred Australian children may report higher levels of problems than teachers. However, the mean level of both parent and teacher-reported problems are significantly greater among clinical samples compared to non-clinical samples. The difference between the level of problems of clinically referred children and adolescents and non-referred children and adolescents is much smaller when using the YSR. The mean level of parent- and child-reported behavioral and emotional problems and competencies has been shown to vary across treatment settings. Generally, greater problems were reported among inpatients compared to children receiving outpatient care or consultations only. While the studies reviewed provide some indication of morbidity rates and levels of problems among Australian samples, they are not conclusive. Samples are not necessarily representative of their respective populations (e.g. clinical, immigrant). Also, some of the studies were conducted quite some time ago and may not be indicative of current morbidity rates for their respective population. Further research regarding morbidity is required using normative samples and the most recent version of the CBCL.

16

5

Diagnostic Utility

5.1

Behavioral and Emotional Problems

5.1.1

A Western Australian Sample

5.1.1.1

WACHS Pilot Survey

The WACHS pilot survey conducted a clinical calibration study, which enabled evaluation of the CBCL as a screening diagnostic tool. Approximately 6-months after the initial pilot study, the Semi-Structured Clinical Interview for Children (SCIC), the Diagnostic Interview for Children and Adolescents – Revised (DICA-R), and clinical diagnosis following interviews were completed for the sample of 40 children (Garton et al., 1995). Using these assessments 20 cases and 20 non-cases were identified. Participants were also asked to complete the CBCL or YSR. Children who scored over the 98th percentile on at least one syndrome, irrespective of source (i.e. CBCL, YSR or TRF) were classified as a case. The ‘cases’ were randomly matched to ‘non-cases’ on age. There was a significant association between a high score on the CBCL (or alternate forms) and receiving a clinical diagnosis 6 months later. Receiver Operating Characteristics (ROC) Analysis was used to examine the sensitivity and specificity of the CBCL. Sensitivity is defined as the percentage of individuals correctly classified as a case. Specificity refers to the percentage of individuals correctly classified as a non-case. Both sensitivity (0.86) and specificity (0.72) were very good, suggesting that a large proportion of children were correctly classified as a case or non-case using the CBCL (or alternate form). The positive predictive value of the CBCL however was somewhat lower (0.65). There were no significant age or gender differences. 5.1.1.2

WACHS Main Survey

The WACHS main survey also conducted a clinical calibration study on a sub-sample of 166 ‘cases’ and a random sample of 80 ‘non-cases’ (Zubrick et al., 1997). Cases were defined as scoring at or above the 98th percentile on a CBCL syndrome score. The composition of cases was as follows: 33 attention problems, 34 delinquency or aggression, 64 depression/anxiety, 35 somatic. Overall, 139 (56.5%) were male and the average age at interview was 12.4 years (range 5-17 years) (Zubrick et al., 1997). Appropriate forms of the DICA were used with the parent and child/adolescent, the results of which were used to make clinical diagnoses. Sensitivity and specificity rates for CBCL and DICA classifications of certain disorders are reported in Appendix 3, Table 1. Classification rates of cases and non-cases were better than chance across all disorders (except classification of non-cases for somatisation). The best classification results were achieved for dysthymia based on the CBCL anxious/depressed subscale (0.88 sensitivity and 0.81 specificity). Unfortunately, the report does not specify the CBCL cut-off points used for classifications. 5.1.2 5.1.2.1

A Melbourne Sample Sensitivity and Specificity

The sensitivity and specificity of the CBCL was assessed using a Melbourne sample of 1342 children referred to a mental health outpatient service between July 1991 and October 1992 (Nolan et al., 1996). Relatively high sensitivity and specificity was reported, particularly for the total behavior score (0.77 17

sensitivity and 0.83 specificity), suggesting that the CBCL may be used as a tool to screen for children with clinically significant behavioral and emotional problems (see Appendix 3, Table 2) (Nolan et al., 1996). The sensitivity and specificity is similar to that for the American sample (Achenbach, 1991; Nolan et al., 1996). The authors suggest that “even if the community sample is less representative than desirable, the discriminability of the CBCL would remain at an acceptable level” (Nolan et al., 1996, p 410). 5.1.3

A Brisbane Sample

A random sample of 64 boys and 56 girls aged 12- to 14-years in Grade 8 at a Catholic school in Brisbane completed the YSR and participated in the Diagnostic Interview for Children, Adolescents and Parents (Johnson, Barrett, Dadds, Fox, & Shortt, 1999). Thirteen children were identified as meeting criteria for a DSM-IV disorder based on diagnostic interview. Though the number of children who met criteria for an internalising or externalising disorder was small, the respective mean YSR scales were significantly higher than a random selection of children who did not meet criteria for a diagnosis.

5.2

Anxiety Disorders

5.2.1

Identification of Anxiety Disorders

Johnson, Barrett, Dadds, et al. (1999) examined the utility of the CBCL to discriminate between a sample of 57 children and adolescents aged 6- to 16-years, recruited through the referral service at an anxiety disorders clinic in Brisbane. Using the Diagnostic Interview for Children, Adolescents and Parents, 40 children were found to have an anxiety disorder. Fifty-two mothers and 40 fathers completed the CBCL. Both parents reported higher levels of internalising problems than externalising problems; mother reported a mean level of 59.83 (SD = 12.73) for internalising problems and 49.50 (SD = 10.65) for externalising problems, whilst fathers reported a mean level of 56.35 (SD = 12.06) for internalising problems and 48.60 (SD = 10.07) for externalising problems. Discriminant function analyses revealed that internalising scores (as reported by mother and father) significantly discriminated children with an anxiety disorder from those without an anxiety disorder. In contrast, neither mother nor father reports of externalising problems discriminated between the groups. A greater proportion of children with an anxiety disorder (92% based on mother report and 96% based on father report) than without an anxiety disorder (71% based on mother report and 70% based on father report) were correctly classified. 5.2.2

Measure of Anxiety Severity

The CBCL has been utilised in studies examining anxiety disorders in children and adolescents (Barrett, Duffy, Dadds, & Rapee, 2001; Cobham, Dadds, & Spence, 1999; Dadds et al., 1999). A study conducted by Cobham, Dadds and Spence (1999) showed that the mean levels of maternal reported CBCL internalising problems was significantly higher amongst a sample of 33 children diagnosed with a DSM-IV anxiety disorder compared to a clinical control group of 20 children diagnosed with either Opposition Defiant Disorder, Attention-Hyperactivity Disorder or Conduct Disorder, who in turn scored significantly higher than a non-clinical sample of 20 children (see Appendix 3, Table 3). The CBCL internalising score was within the clinical range for children with anxiety whose parents also had anxiety. The CBCL internalising score was within the borderline range for the other two clinical groups. Self reported anxiety (using the RCMAS) was also significantly higher among the 20 children diagnosed with an anxiety disorder compared to the clinical and non-clinical control groups. However, the RCMAS 18

did not differentiate between the clinical control and non-clinical control groups. These results suggest that unlike the RCMAS, which is a specific anxiety measure, the CBCL may be useful in identifying internalising problems among children with primary externalising DSM diagnoses. An intervention study included 128 children aged between 7 and 14 years, recruited through schools in metropolitan Brisbane, who were identified as having anxiety problems (DSM-IV diagnosis of features) using the Anxiety Disorders Interview Schedule for Children - Parent Version (ADIS). Clinicians rated the severity and interference of the child’s problems on an 8-point scale (Dadds et al., 1999). Correlations of the clinician severity rating with the CBCL Internalising Scale at pre-treatment, posttreatment, and long-term follow-up (6- and 12-months) assessed the validity of the diagnostic interview. Dadds, Holland, Laurens, et al. (1999) concluded that the moderate level correlations (0.36 pre-treatment, 0.25 post-treatment, 0.51 6-month follow-up and 0.40 12-month follow-up) indicated acceptable validity of diagnostic ratings. Dadds et al. (1999) also examined the chronicity of anxiety problems by identifying predictors of anxiety diagnostic status at post-treatment and 24-month follow-up. A higher CBCL internalising score at pretreatment was predictive of having an anxiety disorder at post-treatment and 24-month follow-up. The CBCL externalising scale was not predictive of anxiety diagnostic status. Other significant predictors included being female and having a higher clinician severity rating at pre-treatment. Pre-treatment CBCL score was also predictive of clinician severity rating at post-treatment and 24-month follow-up. Thus, it appears that the CBCL internalising scale may be useful in identifying children at risk of ongoing anxiety problems, though the inclusion of other factors is likely to improve sensitivity. Barrett, Duffy, Dadds and Rapee (2001) used the CBCL to detect changes in levels of internalising problems amongst a group of 52 children and adolescent who participated in an intervention program for anxiety disorders. The Fear Survey Schedule for Children – Revised (FSSC-R), the Revised Children’s Manifest Anxiety Scale (RCMAS) and the Children’s Depression Inventory (CDI), self-report measures of fears, chronic anxiety and depression in children, respectively, were also employed to assess change from pre-treatment to 1-year and 6-year follow-up (age 14- to 21-years). Regardless of treatment condition (CBT only or combined CBT and family anxiety management), maternal and paternal reports of child internalising and externalising problems were significantly lower at 1-year and 6-year follow-ups compared to pre-treatment (see Appendix 3, Table 4). Though not significant, there was a slight increase in CBCL-I scores at 6-year follow-up compared to 1-year followup for the CBT+FAM treatment group. Similar results were reported for the FSSC-R and RCMAS, with reductions in self-reported fears and chronic anxiety at the 1-year follow-up. As with the CBCL, slight increases in self-reported fears, anxiety and depression were found for the CBT-FAM treatment group at the 6-year follow-up. Additionally, the CBT only treatment group also reported slight increases in fears, anxiety and depression at the 6-year follow-up. Use of the CBCL internalising clinical cutoff (T ≥ 65) also revealed that the majority of participants (83% based on mother report and 85.4% based on father report) fell within the non-clinical range. Overall, these results provide support for the use of the CBCL as an appropriate measure of change over a long period of time and as a potential method for identification of anxiety problems at a clinical level. It is important to note that non-inclusion of a control group limits the confidence in conclusions, as change in levels of internalising and anxiety problems may have partly been a function of age. A further 19

limitation of the study was the use of CBCL and self-report anxiety measures for individuals older than the measures’ normative samples.

5.3

Attention Deficit Hyperactivity Disorder

5.3.1

Diagnostic Utility

Rey, Morris-Yates and Stanislaw (1992) examined the accuracy of the CBCL hyperactivity factor as a method of identifying ADHD in adolescents. The sample comprised 385 boys aged 12- to 16-years attending an adolescent psychiatric unit in Sydney. Seventy-nine boys received a DSM-III diagnosis of ADHD and 306 boys were diagnosed with another DSM-III disorder. The mean CBCL hyperactivity scores of the ADHD sample was significantly higher than the non-ADHD group (14.81 compared to 9.45), lending some support for using the hyperactivity subscale as an indicator of ADHD. ROC analysis was used to assess the diagnostic utility of the hyperactivity subscale (Rey, Morris-Yates et al., 1992). ROC analysis calculates a statistic called ‘area under the curve’ (AUC). The AUC summarises the diagnostic utility of a scale as a diagnostic tool, in that an area greater than 0.50 indicates that the scale predicts diagnosis at a level better than chance. Rey et al. reported an AUC estimate of 0.83, thus supporting the validity of the CBCL hyperactivity scale and suggesting that it may be useful in diagnosing ADHD in adolescent boys aged 12- to 16-years. The ROC analysis also produces sensitivity and specificity statistics, based on selected cutoff points. In relation to the reported study, sensitivity refers to the proportion of adolescents with ADHD who were identified using the scale as having ADHD. Specificity refers to the proportion of adolescents without ADHD who were identified using the scale as not having ADHD. Sensitivities and specificities according to varying cutoff points along the CBCL hyperactivity subscale have been reproduced in Appendix 3, Table 5 (Rey, Morris-Yates et al., 1992). Clearly, there is a marked tradeoff between sensitivity and specificity. For example, a cutoff of 12 on the hyperactivity subscale will result in correct identification of 87% of adolescents with ADHD, whilst 13% of cases will be categorised as not having ADHD. However, a cutoff of 12 means that 65% of adolescents will be correctly categorised as not having ADHD, whilst 35% will be incorrectly categorised as having ADHD. 5.3.2

CBCL Scores Across DSM-IV ADHD Subtypes

The discriminant validity of the DSM-IV ADHD subtypes was assessed using a sample of 3,597 children and adolescents age 6- to 17-years who participated in the Child and Adolescent Component of the National Survey of Mental Health and Well-Being in Australia (Graetz, Sawyer, Hazell, Arney, & Baghurst, 2001). Parents participated in a diagnostic interview (parent version of the Diagnostic Interview Schedule for Children) and completed the CBCL as a measure of behavioral and emotional problems and the Child Health Questionnaire as a measure of quality of life. The overall prevalence of DSM-IV ADHD was 7.5%, with 133 children diagnosed with Inattentive type, 68 with Hyperactivity-Impulsive type and 67 with Combined type (Graetz et al., 2001). On all CBCL scale scores (excluding Somatic Complaints), children diagnosed with ADHD (any type) scored significantly higher than the control children (see Appendix 3, Table 6). On all CBCL scales, children with combined type scored significantly higher than children with inattentive or hyperactivity-impulsive types. Importantly, the inattentive group scored significantly higher than the hyperactivity-impulsive group on the CBCL scales measuring attention, anxiety/depression, somatic complaints and overall 20

internalising problems, whilst the hyperactivity-impulsive group scored significantly higher than the inattentive group on the externalising scales. Scores on the Child Health Questionnaire provided some validity for the finding that children with ADHD have significantly higher levels of behavioral and emotional problems. Furthermore, scores on the Child Health Questionnaire exhibited a similar discrimination pattern to the CBCL (on comparative scales) amongst the ADHD subtypes.

5.4

Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD)

5.4.1

Identification of an ODD and CD Factor

Exploratory factor analysis on 22 CBCL items (see Appendix 3, Table 7) corresponding to DSM-III ODD and CD criteria was conducted to examine the factor structure of conduct problems corresponding to symptoms of DSM-III-R ODD and CD (Rey & Morris-Yates, 1993). CBCL data for 528 adolescents (aged 12- to 16-years; 58% male) who had been referred to an adolescent psychiatric unit in Sydney was used for the analyses. One-hundred and eighty nine adolescents (126 boys) were diagnosed by a clinician as having CD, 75 (45 boys) with ODD, and 264 (138 boys) with another diagnosis, according to DSM-III. The factor analysis identified four factors (see Appendix 3, Table 7), labeled aggression, delinquency, oppositionality and escapism. One factor corresponded very closely to DSM-III-R ODD, with all CBCL items representing the factor related to DSM-III-R criteria. On the basis of the factor analysis and subsequent cluster analysis, Rey and Morris-Yates suggested the presence of a broad CD construct, with an underlying multidimensional structure, whereby some adolescents were characterised as ‘traditional’ CD (high scores on all four factors), and other adolescents were characterised by severe aggression and oppositional behavior and lower delinquency and escapism. The authors also argued for a phenomenological distinction between ODD and CD, based on a group of adolescents scoring high on the ODD factor only. 5.4.2

Symptom Level Among Children with ODD and CD

The CBCL has been utilised as a screening measure for the identification of children with conduct problems and as a measure of outcome following treatment for children with conduct problems (Luk, Staiger, Mathai, Field, & Adler, 1998). In a sample of 176 children referred to a regional child and adolescent mental health service in Victoria, 96 were identified as having at least three symptoms suggestive of oppositional defiant/conduct disorder according to the CBCL and/or TRF. Of these, 15 children had not exhibited conduct symptoms for more than six months and 34 children also met DSMIII-R criteria for ADHD (based on teacher telephone interview). A final sample of 32 children received a form of treatment for conduct problems. The mean level of CBCL externalising problems was within the clinical range (T score 71.2). Some validation for the CBCL externalising scale as a measure of oppositional and conduct problems was provided in this study, as the mean levels of the severity of oppositional defiant/conduct problems and irritability/aggressive behavior (using the Eyberg Child Behaviour Inventory and Rowe Behavioural Rating Inventories, respectively) were also above the clinical cutoff scores. In addition, following treatment each measure showed significant reductions with means falling below the clinical cutoffs. Rey, Bashir, Schwarz, et al. (1988) compared a group of 25 adolescents diagnosed with oppositional defiant disorder (ODD) to 43 adolescents diagnosed with conduct disorder (CD) who attended an adolescent unit in Sydney on the CBCL social competence, internalising, externalising and total problems scale. The groups were also compared on DSM Axis V rating, a measure of chronic adversity, and 21

demographic variables (age, gender, social class). Univariate analyses indicated that children with CD had significantly more externalising and total problems and poorer social competence than children with ODD. The mean levels of externalising and total problems within the CD group fell within the clinical range, while mean levels on the respective scales for the ODD group fell within the borderline range. The CD group was also rated by clinicians as having greater stressors within the past year according to DSM Axis V. There were no group differences on the CBCL internalising scale or demographic measures. Results are presented in Appendix 3, Table 8.

5.5

Depression

5.5.1

Development of a CBCL Depression Scale

The 1991 CBCL provides an anxious/depressed score, but not a validated depression subscale. However, Nurcombe, Seifer, Scioli, et al. (1989) conducted a principal components analysis and cluster analysis of the CBCL items using data gathered from 216 adolescent inpatients. Results from these analyses identified a depressive cluster of 22 CBCL items (see Appendix 3 Table 9). Nurcombe et al. demonstrated significant differences in scores on the Children’s Depression Inventory (in the expected direction) between 23 patients who scored high versus 23 patients (matched by age and gender) who scored low on the proposed CBCL depression scale, thus providing some support for the validity of the proposed scale. Importantly, the 21st Century CBCL scales include DSM-oriented Affective (depressive) Problems scales, which have been demonstrated to correlate with DSM diagnoses and with the Behavioural Assessment System (Reynolds & Kamphaus, 1992) for Children Depression scale. 5.5.2

Development of a YSR and CBCL Scale Equivalent to the CDI

Initial development of the YSR identified a 33-items depression subscale for females and a 20-item subscale for males, both of which were reported to correlate highly with the Children’s Depression Inventory (CDI) (Kovacs, 1981), which is a widely used self-report depression measure (Achenbach & Edelbrock, 1987). Hepperlin, Stewart and Rey (1990) examined the potential of using the CBCL and YSR to extracT scores obtained on the CDI. That is, they attempted to identify scales comprising CBCL or YSR items, which more closely corresponded to the CDI than the existing YSR depression subscale. The study utilised a clinical sample of 207 adolescents aged 11- to 18-years (126 boys) referred to a psychiatric unit in Sydney for assessment from February to December 1996. Items forming CBCL and YSR depression scales that were comparable to the CDI were selected on the basis of statistical analyses and face validity (refer to Hepperlin et al. (1990) for details). Fifteen YSR items (see Appendix 3 Table 10) were selected to comprise the ‘YSR-CDI scale’ (Hepperlin et al., 1990). The corresponding CBCL items were selected to comprise the ‘CBCL-CDI scale’. All of these 15 YSRCDI items were included in the original YSR depression subscale for females, whilst only 3 items also appear on other factors. For boys, 9 of the YSR-CDI items are included in the original YSR depression scale, while 6 items are included on the unpopular subscale and 5 items on the self-destructive/identity problems subscale. Internal consistency of both the YSR-CDI and CBCL-CDI scales was 0.81, whilst split half reliability was 0.78 and 0.79 respectively. There was a strong correlation (0.76) between the YSR-CDI and CDI. The CBCL-CDI and CDI were relatively poorly correlated (0.23). This may be due to different use of 22

informants across measures (as the CBCL-CDI was completed by the parent and CDI by the adolescent), or differences in the measures per se. Though the original YSR male and female subscales are somewhat longer than the 15-item scale identified by Hepperlin et al. (1990), they have the advantage of available normative data. Also, the correlations between the YSR depression subscales for females and males with the CDI are 0.75 and 0.65, respectively, which is comparable to the YSR-CDI and CDI correlation. Nevertheless, the study by Hepperlin et al. suggests that the use of the CDI may be redundant if assessment also incorporates the YSR. 5.5.3

Diagnostic Utility of Proposed Depression Scales

Rey and Morris-Yates (Rey & Morris-Yates, 1991, 1992) used ROC analysis to assess the accuracy of the depression scale proposed by Nurcombe et al. (1989) (discussed in Section 2), as well as five other CBCL and/or YSR subscales, in identifying adolescents with and without major depression. Data was obtained from a cohort of 667 adolescents (387 males) aged 12- to 16-years who had been referred to an adolescent unit in Sydney for psychiatric assessment between 1983 and 1986. Table 6 outlines the scales assessed, whilst items composing each scale are listed in Table 11, Appendix 3. Table 6. CBCL and YSR Measures of Depression (Taken from Rey and Morris-Yates (1991)) Scale name

Scale description

CBCL-NUR

CBCL 22-item scale identified by Nurcombe et al. (1989)

YSR-CDI

YSR 15-item scale identified by Hepperlin et al. (1990)

YSR-DEPB

YSR 20-item depression factor for boys extracted by Achenbach and Edelbrock (1987)

YSR-DEPG

YSR 32-item depression factor for girls extracted by Achenbach and Edelbrock (1987) Cross-informant anxious/depressed factor extracted by Achenbach, Connors and Quay et al. (1989) Addition of CBCL-NUR and YSR-CDI, divided by two

Anxious/depressed Composite

Extensive file reviews conducted by senior clinicians identified four diagnostic groups: 23 adolescents with major depression, 62 with dysthymia, 57 with separation anxiety, and 634 with ‘other’ diagnoses. The sample of adolescents with major depression scored significantly higher than the sample with dysthymia, separation anxiety, or ‘other’ diagnosis. Furthermore, adolescents with major depression scored higher than the group with dysthymia on the CBCL-NUR, YSR-DEPB and Composite scales. Mean score for each depression scale is reported in Appendix 3, Table 12. ROC analyses indicated that each depression scale discriminated between patients diagnosed with major depression and patients with ‘other diagnoses’, dysthymia and separation anxiety (Rey & Morris-Yates, 1991, 1992). However, the overlap of confidence intervals indicated no statistical difference in the accuracy of discrimination between each diagnostic group. Sensitivity and specificity estimates were not routinely reported for each scale. However, estimates that were reported indicated that the sensitivity (proportion of depressed patients identified by the scale as depressed) and specificity (proportion of non-depressed patients identified as not depressed) statistics for the CBCL-NUR scale were less impressive than overall accuracy. For example, Rey and Morris-Yates (1991) reported that a cutoff of 15 when differentiating between depressed patients with other diagnoses resulted in a sensitivity of 0.83 and a specificity of 0.55. Thus, whilst 83% of depressed patients were identified as such, 17% of depressed patients were incorrectly classified as not depressed. Furthermore, a 23

specificity of 0.55 means that 45% of non-depressed patients with ‘other’ diagnoses were actually identified as depressed. Rey and Morris-Yates (1991; 1992) concluded that the accuracy of the proposed depression scale to differentiate between patients with and without depression is comparable to other measures. However, given the moderate sensitivity and specificity statistics, it is probably safer to use the proposed depression scale as an indicator of possible depression rather than as a diagnostic tool.

5.6

Comorbidity

The CBCL has been used to examine the prevalence of comorbid disruptive disorders and depression. Rey (1994) used a sample of 840 adolescent girls and 1252 adolescent boys referred to a adolescent unit in Sydney for psychiatric assessment between 1983 and 1991. CBCL data was available for at least 90% of the cohort. Comparison data came from the American clinical cohort (adolescents aged 12- to 16years) used in the original CBCL factor analysis (Achenbach & Edelbrock, 1983). The CBCL was used as a diagnostic indicator of Depression, Hyperactivity (ADHD), Oppositional Defiant Disorder (ODD) and Conduct Disorder (CD). The CBCL Depression Scale (Nurcombe et al., 1989) and the original Hyperactivity Scale (Achenbach & Edelbrock, 1983) were used. The items listed in Table 45 were summed and a cutoff score equal to or greater than 22 for Depression and 14 for Hyperactivity were indicative of respective diagnoses. The CDD and OD scales were developed on the basis of discriminant function analysis, which identified CBCL items that demonstrated good discrimination between patients referred for ODD and CD. Items are listed in Table 7. It is noteworthy that not all items corresponded to DSM-III-R criteria. On the ODD scale, a cutoff of 15 indicated a diagnosis of ODD. On the CD scale, a cutoff of 7 indicated a diagnosis of CD.

24

Table 7. CBCL Items Making up the Different Scales (Taken from (Rey, 1994)) Oppositionality Item Item content 3. Argues 16. Cruelty 22. Disobedient, home 23. Disobedient, school 37. Fights 57. Attacks people 68. Screams 86. Stubborn 90. Swears 94. Teases 95 Tantrums 97. Threatens

Conduct Item Item content 15. Cruel to animals 21. Destroys property 43. Lying 72. Sets fires 81. Steals at home 82. Steals at school 67. Runs away 101. Truancy 105. Alcohol, drugs 106. Vandalism

Hyperactivity Item Item content 1. Acts too young 8. Can’t concentrate 10. Hyperactive 23. Disobedient, school 41. Impulsive 44. Bites fingernails 45. Nervous 61. Poor school work 62. Clumsy 74. Shows off

Depression Item Item content 13. Confused 14. Cries 18. Harms self 30. Fears school 31. Fears doing bad 32. Has to be perfect 35. Worthless 42. Alone 47. Nightmare 50. Fearful 52. Guilty 54. Overtired 56b.

Headaches

75. 77. 80. 91. 100. 102. 103. 111. 112.

Shy Sleeps more Stares blankly Suicidal Trouble sleeping Underactive Unhappy Withdrawn Worrying

Note: Item content is summarised

As with some other Australian prevalence studies, the mean levels of problems and prevalence of individual and comorbid disorders was substantially higher among the Australian population compared to the US normative sample (see Appendix 3, Table 13). However, the pattern of comorbidity across countries was similar. The odds of having comorbid Depression-ADHD or comorbid Depression-ODD were significantly greater than comorbid Depression-CD (see Appendix 3, Table 14). Analyses indicated that having a diagnosis of depression did not increase the risk of a diagnosis of CD, even in the presence of a diagnosis of ADHD or ODD. A clear limitation of this research is the lack of an external criterion and the fact that the CBCL scales used have not demonstrated 100% accuracy in classification of the disorders.

5.7

Conclusions

Sensitivity and specificity estimates are often used to assess the diagnostic utility of instruments. Sensitivity refers to the percentage of individuals correctly classified as a case. Specificity refers to the percentage of individuals correctly classified as a non-case. These statistics have been used in research to assess how well the CBCL can predict diagnosis in Australian samples. The WACHS demonstrated good sensitivity and specificity estimates for syndrome scales. All estimates indicated a better than random classification result (except specificity of somatisation). The best 25

classification results were achieved for a diagnosis of dysthymia based on the CBCL anxious/depressed subscale. Relatively good sensitivity and specificity estimates have been reported using the total behavior score with a Melbourne clinical sample. The hyperactivity subscale has been used to predict diagnosis of ADHD, with sensitivity and specificity estimates better than random. While best results were achieved with a cutoff of 12 or 14, a significant number of children were also misclassified. Combinations of various CBCL items have been shown to discriminate between adolescents with Major Depression and other mood and anxiety disorders. However, there seemed to be a large trade-off between sensitivity and specificity estimates and the overlap of confidence intervals indicated no statistical difference in the accuracy of discrimination between each diagnostic group. Another method of examining the diagnostic use of the CBCL has been to compare the mean levels of reported problems between samples of children and adolescents with and without specific disorders. Children with anxiety disorders have been reported to have significantly higher levels of CBCL internalising problems than children without an anxiety disorder. Research has shown the clinical utility of the CBCL internalising scale as a measure of change following treatment for anxiety disorders. However, anxiety specific measures are likely to provide a more sensitive measure of anxiety and change in levels of anxiety. It is also imperative to acknowledge that children with disorders aside from anxiety have also shown at least borderline levels of internalising problems on the CBCL (and related forms). Thus, the internalising scale may be limited as a diagnostic measure of anxiety. Children with ADHD have been reported to have significantly higher mean levels on the CBCL hyperactivity syndrome scale than children and adolescents with other disorders. Significant differences have been shown in levels of behavior problems between children with different subtypes of ADHD. In particular, compared to children with inattentive or hyperactive type, children with combined type have been shown to score significantly higher on the total problem scale, externalising scale, internalising scale and the majority of syndrome scales. Levels of externalising problems have been shown to be in the clinical and borderline ranges for children with CD and ODD, respectively. Children with CD have also been shown to have significantly poorer social competence according to the CBCL than children with ODD. The CBCL has also been used to examine the prevalence of comorbid disruptive disorders and depression. However, the lack of an external criterion limits the application of research findings. Similarly, mean levels of depression symptoms (using various combinations of CBCL items) have been shown to be significantly higher among adolescents diagnosed with Major Depression than for those diagnosed with Dysthymia, Separation Anxiety Disorder and ‘other’ disorders. Research has even suggested the use of YSR items, which correspond to the CDI (self-report depression scale), may replace the use of the CDI. However, use of the combination of YSR items corresponding to the CDI and the CDI are somewhat limited, as the scales do not have norms based on representative probability samples. In sum, despite the seemingly adequate sensitivity and specificity results, a number of children were also misclassified in the studies reviewed. In a clinical setting the rate of misdiagnosis is likely to be unacceptable. On the other hand, the CBCL may provide a useful indicator of the severity of several types of problems. It is therefore recommended that the CBCL total, broad-band and syndrome scales should not be used as a diagnostic tool, but rather as one of many tools in the assessment process. Hopefully, further research using Australian samples will be conducted to examine the diagnostic utility of the recent CBCL DSM derived scales (Achenbach, 2002). 26

6

Stability of Behavioral and Emotional Problems

The CBCL has been used to assess the stability of behavioral and emotional problems over varying lengths of time.

6.1

WACHS Pilot Study

As part of the WACHS pilot study, some participants completed the CBCL (n=37) or the YSR (n=18) on two occasions, yielding 8-week test-retest correlation coefficients of 0.87 and 0.76, respectively (Garton et al., 1995). For the main WACHS survey, the average 6-month test-retest reliability was 0.75 (Zubrick et al., 1997). These figures are comparable to reliability estimates reported by Achenbach (1991, cited in Zubrick, 1997 #80)

6.2

The Port Pirie Cohort Study

The Port Pirie Cohort Study commenced in 1979 with a sample of 723 infants (686 families) who lived in or around the non-metropolitan region of Port Pirie, South Australia (Sawyer, Mudge, & Carty, 1996). The main aim of the study was to investigate the impact of low-level lead exposure on child development. The CBCL was used to provide a measure of behavioral and emotional problems when the children were aged 5-years (n = 444 families). Cross-informant data (based on the CBCL, YSR and TRF) was obtained for 147 girls and 130 boys, aged 11- to 12-years. Analyses indicated no response bias with respect to the total number of behavioral and emotional problems, externalising problems, internalising problems, child sex, or paternal occupational class at age 5-years. The stability of behavioral and emotional problems over the 6-year period was assessed using Pearson r correlations between the CBCL at age 5-years and the CBCL, YSR and TRF at age 11- to 12-years. All correlations between maternal CBCL ratings across the time points were significant at p