Raising Standards or Raising Barriers?: Inequality and High Stakes Testing in Public Education

&lai&vup

or ^Sarrcers/P

Of^

^nuisif^^a cwa/jieca- <Srw S/estin^/uvSceS/lcy A series of Century Foundation Books developed in conjunction with the Civil Rights Project at Harvard University. The volumes in the series assess the prospects for justice and equal opportunity under law for racial and ethnic minorities in the United States.

(Sducatlony

GARY O R F I E L D AND M i N D Y L . KORNHABER, editors

Religion, Race, and Justice in a. Changing America, Gary Orfield and Holly J. liebowitz, editors

A CENTURY FOUNDATION BOOK 2 0 0 1 • THE CENTURY FOUNDATION PRESS • NEW YORK

The Century Foundation sponsors and supervises timely analyses of eco­ nomic policy, foreign affairs, and domestic political issues. Not-for-profit and nonpartisan, it was founded in 1919 and endowed by Edward A. Filene.

(oofitenfis/ BOARD OF TRUSTEES OF THE CENTURY FOUNDATION

H. Brandt Ayers Peter A. A. Berle Alan Brinkley, Chairman Joseph A. Califano, Jr. Alexander Morgan Capron Hodding Carter III Edward E. David, Jr. Brewster C. Denny Charles V. Hamilton Matina S. Horner Lewis B. Kaden James A. Leach Richard C. Leone

Jessica Tuchman Mathews Alicia H. Munnell P. Michael Pitfield Richard Ravitch Alan Sagner Arthur M. Schlesinger, Jr. Harvey I. Sloane, M.D. Theodore C. Sorensen Kathleen M. Sullivan James Tobin David B. Truman Shirley Williams William Julius Wilson

Richard C. Leone, President

Acknowledgments

vii

Preface

ix

1. High-Stakes Testing Policies: Examining Their Assumptions and Consequences by Mindy L. Kornhaber and Gary Orfield 2. The Development and Impact of High-Stakes

LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA

Raising standards or raising barriers? : inequality and high-stakes testing in public education / Gary Orfield and Mindy L. Kornhaber, editors. p. cm. — (Civil rights in a new era) Includes bibliographical references and index. ISBN 0-87078-451-X (hardcover: alk. paper) — ISBN 0-87078-452-8 (pbk.: alk. paper) 1. Educational tests and measurements—Social aspects—United States. 2. Educational equalization—United States. 3. Test bias—United States. I. Orfield, Gary. II. Kornhaber, Mindy L. III. Series. LB3051 .R324 2001 371.26'01'3—dc21 00-012741 Cover design and illustration: Claude Goodwin Manufactured in the United States of America. Copyright © 2001 by The Century Foundation, Inc. All rights reserved. N o part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photo­ copying, recording, or otherwise, without the prior written permission of The Century Foundation, Inc.

1

3.

Testing by Gary Natriello and Aaron M. Pallas

19

High-Stakes Testing and Economic Productivity by Henry M. Levin

39

4. The Impacts of Minimum Competency Exam Graduation Requirements on College Attendance and Early Labor Market Success of Disadvantaged Students by John H. Bishop and Penan Mane

51

5. The Adverse Impact of High-Stakes Testing on Minority Students: Evidence from One Hundred Years of Test Data by George Madaus and Marguerite Clarke 6.

85

Do High-Stakes Graduation Tests Improve Learning Outcomes? Using State-level NAEP Data to Evaluate the Effects of Mandatory Graduation Tests by Monty Neill, with Keith Gayler

107

Contents

VI

7. The Harmful Impact of the TAAS System of Testing in Texas: Beneath the Accountability Rhetoric by Linda McNeil and Angela Valenzuela 8. Should We End Social Promotion? Truth and Consequences by Robert M. Hauser 9. High-Stakes Testing and Civil Rights: Standards of Appropriate Test Use and a Strategy for Enforcing Them by Jay P. Heubert Notes

195

Index

237

About the Contributors

247

T

he editors are grateful to have worked with the authors who have contributed papers to this volume. Each of them has long been involved in thoughtfully examining the role of high-stakes testing in the context of America's very disparate public schools, and it has been an exciting experience for us to see their newest work and insights take form in their chapters. The ideas behind this book originated at a con­ ference on high-stakes testing and K-12 education, which had many orga­ nizational parents to whom we owe our thanks: Teachers College, Columbia University, hosted our original conference. Our meeting there was cosponsored by Teachers College, Columbia Law School, and Harvard University. The Spencer Foundation provided funds to the Civil Rights Project to launch this conference and the chapters stemming from it. We are also grateful to the Carnegie Foundation, The John D. and Catherine T. MacArthur Foundation, and the Charles Stewart Mott Foundation for their support of work on K-12 education issues at the Civil Rights Project. Staff members of the Civil Rights Project, Marilyn Byrne, Lori Kelley, Michal Kurlaender, Suenita Lawrence, and Christina Safiya Tobias-Nahi, have helped in innumerable, ways. We deeply appre­ ciate their care, intelligence, and dedication. The Century Foundation invested its energies, support, and staff in this project. We are grateful to them for publishing this book and helping to build greater public under­ standing of high-stakes testing in our system of public education.

Vll

(£teface/

w

hile this book was in process, a new administration, headed by George W. Bush, came into power after a campaign in which both candidates strongly supported accountability through testing. Over the course of the campaign and thereafter, major battles have erupted in a number of states over the enforcement of testing policies. Some of those policies are being shelved or delayed while others are becoming even more strict. In short, the issue of educational testing has become even more prominent. The intense focus on testing that George W. Bush advocated in Texas has now become the model for Title I, the federal program for education of poor children. This focus has been underscored by the choice of Rod Paige, the former superintendent of Houston and a strong testing advo­ cate, to head the Department of Education. In his education policy statement, "No Child Left Behind," President Bush concludes that testing will do much to "close the achievement gap between disadvantaged students and their peers": Schools must have clear, measurable goals. . . . Requiring annual state assessments in math and reading in grades 3-8 will ensure that the goals are being met for every child, every year. Annual testing in every grade gives teachers, parents and policymakers the information they need to ensure that children will reach Aca­ demic success. The president, like many other test advocates, thinks that the combi­ nation of information, accountability, and sanctions will enable stu­ dents to reach high standards. He feels certain enough about this to justify the imposition of annual testing requirements on the states, even in an administration committed to increasing state autonomy in other areas. IX

X

Preface

However, as is too often the case, statements like the president's sim­ ply assume that testing and accountability systems will lead to educa­ tional improvements. Testing advocates take for granted that tests change the behavior of students and teachers in a positive way and that those changes produce more learning. They often equate a given test score with actual academic achievement, despite inconsistencies that occur in scores even for the same person in successive experiences with the same test. They also often assume that policies aimed at producing higher test scores will produce a stronger economy, and they allude to the economic dangers of a workforce whose test scores are lower than those of other nations. Test advocates treat these assumptions as self-evident facts. They are instead issues that should be examined through empirical investigations. Nearly all of the authors in this volume have documented that certain assumptions used to justify a heavy reliance on high-stakes tests are flawed. They largely find that the evidence is inadequate to demonstrate that test policies will motivate the unmotivated; solve problems created by inadequately trained teachers or weak administrators; close gaps in achievement among students from different racial, ethnic, and economic backgrounds; lead to better job candidate selection; or alter the national economy. Some contributors to this volume go further, claiming that the misuse of test scores to impose drastic sanctions without equalizing opportunity to learn actually can make bad situations worse and can harm the educational attainment of the most vulnerable students. Some studies in this book document that an overemphasis on testing leads to drastically narrowed curricula and to increases in dropout rates. The stakes, or consequences, for students and schools are very high in the intense but often uninformed debates over testing now taking place in Congress, in the courts, in state capitals, and in many school dis­ tricts. Too often the complex issues involved in improving teaching and learning are reduced to slogans and sound bites. Assessment is treated as if the right tests and sanctions can solve all the problems. The benefits of high-stakes testing are assumed, and the. costs—in terms of money, instructional time, curricula, retention, and dropping out—are ignored. This volume makes clear that high-stakes tests, even when appropri­ ately used, are not sufficient to promote strong schools. Well-constructed and appropriately used tests can help to detect problems, but they do not, in themselves, solve them. Advocates and critics alike agree that many other factors besides tests are important. States and districts that are regarded as exemplars of achievement gains have implemented a complex set of interacting policies, such as teacher professional development, smaller class sizes, and increased funding. Testing is not the sole or pri­ mary reason for actual improvements in learning.

Preface

XI

If we are to work toward genuine efforts to foster higher levels of learning on a more equitable basis, the limits of test-driven reforms need to be acknowledged and addressed. We therefore make the following recommendations: • We urge policymakers to use testing to inform, rather than replace, decisionmaking. A test score reveals only a very limited amount of information about individual students. Therefore, all major profes­ sional associations involved in educational testing, as well as the National Research Council, emphasize that decisions about student promotion, retention, program or curricular placements, and gradu­ ation must be based on more than a single test score. These decisions also should be informed by "other relevant information," such as grades, teacher recommendations, and results of other tests.1 It is not objective to rely on a single test score or test scores alone in making important educational decisions; it is a misuse of testing technology. • If we are going to put a serious emphasis on tests, it is very important that they be good tests and that they assess material that students actually have been taught. Many existing state tests are not valid tools for making decisions about retention, placement, or graduation. In addition, high-stakes tests are often imposed on schools where there is no guarantee that the students have been exposed to the material or have teachers who know the material tested. Minimum scores are often set as a result of political considerations about acceptable rates of failure and without any study of the feasibility of various groups of students and schools reaching the specified levels of proficiency. Students should not be denied diplomas or be retained in grade on the basis of scores from tests that have not been appro­ priately validated or when they have not been taught the material. • We urge policymakers to reach beyond test policies to implement reforms that actually build good learning environments. Research shows that poor and minority youths are more likely to be subjected to high-stakes tests, but they are also far less likely to attend schools American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, Standards for Educational and Psychological Testing (Washington, D.C.: American Psychological Association, 1999), pp. 146-47; National Research Council, Committee on Appropriate Test Use, High Stakes: Testing for Tracking, Promotion, and Graduation, Jay P. Heubert and Robert M. Hauser, eds. (Washington, D.C.: National Academy Press, 1999).

Xll

Raising Standards or Raising Barriers?

with experienced and certified teaching staffs, adequate libraries, up-to-date technologies, and high-achieving peer groups. We need policies that increase the pool of highly qualified teachers and that address the shortage of these teachers in districts and schools serving those most in need. Poorly designed test-use policies actually may have the perverse impact of encouraging good teachers to leave already challenged schools. We also need policies that foster smaller class sizes, especially for those most in need; smaller school sizes; adequate libraries; and rich and engaging curricula. Testing policies cannot substitute for, or accomplish the work of, policies aimed at building strong learning environments.

CHAPTER ONE

&GgAs*JbaAe&' ^e&tin^ (Sjccuninin^/

Policies/

£7heir ■ . ■ " . " * ■„

B QB D

n"

fl B



CO

to

a

0 tj oo-B

c c

to X

CN

X

K

'I 2

R

NO

0 ON

u a0

Z o

II to

°=: '5 « 2

SP 2f oo 3

TO EARNINGS AND UNEMPLOYMENT OF MALES

u ^
d C (U

ID

TABLE 4.4. RELATIONSHIP OF PROSE LITERACY AND SCHOOLING

Literacy

n

O

59

than higher-order problem-solving skills, writing skills, computer skills, occupation-specific skills, or affective competencies that are presumed more important. They argue that tests similar to the MCEs used by many states have weak relationships with wages and youth labor market suc­ cess. Where is the sense, they ask, in threatening to deny a credential that employers reward very handsomely—the high school diploma—in order to induce teachers to teach and students to learn basic reading and math literacy skills that employers do not reward by paying higher wages. The problem with this argument is that it is not consistent with employer behavior during the 1990s and mistakes form (a diploma) for substance (the skills and knowledge diplomas signal). When literacy, schooling, and earnings are all measured from the same period, simple tests assessing literacy have at least as strong a rela­ tionship with unemployment and earnings of adults as years of school­ ing. Table 4.4 presents evidence for this assertion from the National Adult Literacy Survey. Adults in the top prose literacy group earn roughly 3 times as much as those in the bottom literacy group and have onefifth the chance of being unemployed. College graduates, by contrast, earn 2.35 times as much as high school dropouts and have two-fifths the chance of being unemployed.

EARNINGS ($)

tN co -


Academic achievement: and economic performance, assumption regard­ ing, 5-6; gender and, 100; longterm effects of, 60-61; negative trends in, false claims regarding, 4-5; retention and, 168-73; and wages, 60, 61; See also Achievement gap Accelerated Schools, 49 Achievement gap, 5, 86-92; estimates for closing, 15; NAEP results and, 86-87, 87/; 88/", 89/; in Texas, 119, 136 ACT. See American College Test AFQT. See Armed Forces Qualification Test African American students: ACT per­ formance of, 89, 91/; effective teachers of, 143; failure rates for, 180; grade retention for, 165/, 166/, 1 6 7 / 168/; high school com­ pletion rates for, 57, 58t; mathe­ matics achievement of, 86-87, 8 7 / 8 8 / 89/; NELS performance of, 88, 90/; Regents diplomas for, 29, 3 0 / 30t; SAT performance of, 86, 9 2 / Age at school entry, 1 5 7 / 157-58 Age-grade retardation, 159; recent his-

tory of, 159-61, 160/; See also Retention Aid, state, 74, 75 Alexander, Karl L., 170-71 Alienation: examinations and, 99; TAAS system and, 139^10 Altonji, Joseph, 6, 60, 77 America 2000 education proposal, 3, 4 American College Test (ACT): court rulings on, 235«73; vs. NAEP lev­ els, 210«10; racial/ethnic group differences on, 89, 9 1 / Anderson, Douglas K., 174 Armed Forces Qualification Test (AFQT) scores, and labor market success, 60-61 Asia, economic growth in, explanations for, 48 Asian American students, test perfor­ mance of, 88, 9 0 / 9 1 / 9 2 / Attrition rates, 214«42; See also Dropout rates Authentic assessment, 94, 149; belief underlying, 213«37; impact on performance, 100-101 Barriers, test-related, 2; reducing, 16-18 Barton, Paul E., 119

Note: Page numbers followed by letters / t, and n refer to figures, tables, and notes, respectively.

237

Index

238 Berlin, Gordon, 45 Berliner, David, 4-5 BIA. See Bureau of Indian Affairs Biddle, Bruce J., 4 - 5 , 22 Bishop, John, 6, 9, 12-13 Black, Paul, 18, 122 Black students. See African American students Bowles, Samuel, 44 Bureau of Indian Affairs (BIA), 124 Bush, George, 3 Bush, George W.: on education system, 4; on grade retention, 153 California: grade retention in, 153; Long Beach School District, 229«70 Campbell, Don, 93 Cannell, John, 21Owl6 Carnegie units, 56; effects on enroll­ ment rates, 56-57 Cause, proper attribution of, principle of, 17, 185-86 Celebrezze, Anthony, 209«9 Cheating, examinations and, 98 Chicago, grade retention in, 154, 173-74, 176, 2 2 4 K 1 0

Civil rights, high-stakes testing and, 179-94 Civil Rights Act of 1964, Title VI of, 190-91 Civil rights law, and standards of appropriate test use, 17 Civil rights movement, and testing boom, 21 Clarke, Marguerite, 8, 9-10, 18 Classroom-based assessments, 122-25 Clinton, Bill: education policy of, 3-4, 195«5; on social promotion, 152-53, 224«13 Clinton, Hilary, \95n5 Cognitive skills: and economic produc­ tivity, 6, 4 1 ; jobs demanding, trend toward, 206n27; and wage gap, 43-44

Coleman, James, 42 Coley, Richard S., 119 College admission: considerations in, 204«7; MCEs and, 63-64, 8If; Regents program and, 71; regres­ sion models predicting, 206«26 Committee for Fiscal Equity, 32 Competition, examinations and, 98 Conceptual learning, effects of testing on, 9-10 Cortines, Ramon, 67 Courses: easier, students' preference for, 52; number of, and dropout rates, 56; See also Curriculum CPS. See Current Population Survey Cramming schools, 98 Crew, Rudy, 153-54 Cultural tracking, 1 4 4 ^ 5 Culturally relevant pedagogy, 142-43 Current Population Survey (CPS), 156 Curriculum: and graduation require­ ments, discrepancy between, 183-84; high-stakes testing and, 93-96, 110, 121, 129; narrowing of, 10, 93-94, 98-99, 117, 211«17; TAAS impact on, 132, 137-40, 142-43; See also Courses; Instruction Cut-score methodologies, arbitrary nature of, 90-92 Darling-Hammond, Linda, 26 Dauber, Susan L., 170 Davis, Gray, 153 Debra P. v. Turlington, 234«65 DeCicca, Phillip, 56 Disabilities, students with: exemptions from testing, 23-24, 28, 34; fail­ ure rates for, 181, 230«13; in large-scale state assessments, 229«10 District of Columbia, retention rates in, 155 Dropout rates: event, 57; grade reten­ tion and, 103, 173-74; high-

Index stakes testing and, 13, 101—4; MCEs and, 56-57, 101-2; race/ethnicity and, 57, 58*, 165, 169'/"; Regents Examinations and, 56, 75-77; socioeconomic status and, 102; status, 57; TAAS and, 102-3 Dworkin, A. G., 171-72 Earnings. See Wages Economic productivity, tests as predic­ tors of, 39^49; assumption regard­ ing, 5-7 Effective treatment, principle of, 17, 186-88 Elementary and Secondary Education Act of 1965, 209«9 Ellwein, Mary Catherine, 176 Employee selection, use of test results in, 40; danger of, 48^19; MCEs' role, 61-62 Employment success: literacy and, 57-59, 59*; MCEs and, 57-65 English proficiency, limited. See Limited English proficiency Enrichment remediation programs, 73 Enrollment rates: Carnegie units and, 56-57; determinants of, 78t-79t; MCEs and, 63; in Texas, 102-3 Entwisle, Doris R., 170 Ethnicity. See Race/ethnicity Event dropout rates, 57 Expenditures, TAAS system and, 10-11, 136-38 External examinations, consequences of, 98 FairTest, 110 Falk, Beverly, 26 Family background, and test scores, 44 Fassold, Mark A., 102 Flanagan, Ann, 118, 119 Fordham, Signithia, 122 Fordice v. U.S., 235«73

239 Formative assessment, 18, 122-25 From Gatekeeper to Gateway: Transforming Testing in America, 104-5 GATB. See General Ability Test Battery GED program, transfers to, Regents examinations and, 76 Gender: and academic performance, 100; and age at school entry, 158; and grade retention, 164, 164/; and MCEs' impact, 63 General Ability Test Battery (GATB), 41, 45-46 Gintis, Herb, 44 Glass, Gene V., 176 Gore, Al, on education system, 4 Grade promotion. See Promotion Grade retention. See Retention Grading committees, benefits of serving on, 74 Graduation: relevant information for, 184; testing tied to, development of, 21-22 Graduation rates: determinants of, 78t-79t; high-stakes testing and, 12-15; race/ethnicity and, 57, 5St; Texas, drop in, 119-20 Graduation tests: and curriculum, dis­ crepancy between, 183-84; NAEP gains in states with, 111-13, lilt, 117; NAEP gains in states without, 113, 114*, 117 Grissmer, David, 118, 119 Grissom, James B., 173 Haney, Walt, 12, 14, 102, 120 Hauser, Robert, 15, 22 Heisenberg's uncertainty principle, 210«15; social version of, 93 Heubert, Jay, 16, 17, 22 High school: academic achievement in, long-term effects of, 60-61; dropout/graduation from. See Dropout rates; Graduation rates

Index

240 High School and Beyond (HSB), 63; racial/ethnic group differences on, 90/ High school graduation tests: develop­ ment of, 21-22; NAEP gains in states with, 111-13, 112?, 117; NAEP gains in states without, 113, 114*, 117 High-stakes testing: alternatives to, 122-25; assumptions underlying, 5-12; and civil rights, 179-94; criticism of, 1-2, 22; development of, 19-22; and dropout rates, 13, 101—4; and economic productiv­ ity, 5-7, 39-49; and employment decisions, 48-49; and graduation rates, 12-15; impact of, 12-16, 22, 85-86; impact on minority students, 7, 94-96, 119-20; impact on teaching and learning, 93-96; interpretations of purpose and impact of, 36-38; and learn­ ing outcomes, 107-25; monitor­ ing of, need for, 105; and motivation, 96-100; as perfor­ mance-monitoring process, effi­ ciency of, 2 0 - 2 1 ; political climate for, 2 - 5 ; public opinion on, 1,3; range of programs, 85; recom­ mendations for improved use of, 38; and retention, 15-16; support for, 1, 22, 85; systemic effects of, 11-12 Hispanic students: failure rates for, 180; grade retention for, 165/, 166/, 167/, 168/; high school com­ pletion rates for, 57, 58t; mathe­ matics achievement of, 86-87, 87/, 88/, 89/"; Regents diplomas for, 29, 31/", 3l£; test performance of, 88, 90/", 91/", 9 2 / Hoffer, Thomas, 42 Holmes, C. X, 15, 168,169 Holmes, Edmond, 93 HSB. See High School and Beyond

IALS. See International Adult Literacy Survey Illinois. See Chicago Immigrant population, characteristics of, 145-46 Immigration, and boom in testing, 21 Incentive(s): inversion of, TAAS and, 138; retention as, 208w43 Industry, use of external motivation techniques in, 99 Instruction: alternatives in, 122-23; impact of high-stakes testing on, 93-96, 110, 121; quality of, and test performance, 185; supplytype exams and, 94; test informa­ tion used to enhance, recommendations for, 18; testing and improvement in, assumption regarding, 9-12; See also Curriculum Instructional narrowing, testing and, 10, 93-94, 98-99, 117, 211«17 International Adult Literacy Survey (IALS), 43 Intervention programs, vs. promo­ tion/retention, 177-78, 228w69 Iowa, grade retention in, 153 Iowa Test of Basic Skills (ITBS), 154, 176,224K10

IQ test scores, higher, trend toward, 5 Ireland, primary school examination in, 103 Israel, impact of high-stakes tests in, 211«17 ITBS. See Iowa Test of Basic Skills Job performance. See Worker produc­ tivity Johnson, William R., 43-44 Karweit, Nancy L., 158, 171 Kasim, Rafa M., 42, 45 Kauffman, Albert H., 120 Kindergarten, retention in, 157/, 157-58

Index Ladson-Billings, Gloria, 143 "Lake Wobegon" effect, 210ral6 Larson, Katherine A., 174 Learning: impact of high-stakes testing on, 93-96; on multiple levels, 121; principles of, 140; single indicator assessing, problems with, 131; test information used to improve, rec­ ommendations for, 18; testing and improvement in, assumption regarding, 9-12 Learning disabilities, and exemption from TAAS, 23-24 Learning Record, 124 Learning Research and Development Center, proposed examination sys­ tem, 98 Legal challenges: to educational test­ ing, 190-91, 234«65; to Minnesota Comprehensive Assessments, 35-36; to Regents Examinations, 32; to Texas Assessment of Academic Skills (TAAS), 25-26 LEP. See Limited English proficiency Lerner, Barbara, 57 Levin, Henry, 6, 7 Levy, Frank, 42 Lillard, Dean, 56 Limited English proficiency (LEP), stu­ dents with: exemptions from test­ ing, 23-24, 34; Regents diplomas for, 32, 34/; TAAS and, 144-47 Linn, Robert L., 47 Literacy: and labor market success, 57-59, 59t; NALS conceptualiza­ tion of, 42 Madaus, George, 8, 9-10, 18 MALDEF. See Mexican American Legal Defense and Educational Fund Mane, Ferran, 6, 9, 12-13 Marion, S. E, 171 Massachusetts Comprehensive Assessment System (MCAS), 91-92

241 Mastery tests, vs. placement tests, 183, 230w21 Math scores: cohort growth in, 119; as earnings predictor, 4 1 ; race/ethnic­ ity and, 86-88, 87/, 88/, 8 9 / 91/, 92/ Math studies, TAAS system and, 136 MCAS. See Massachusetts Comprehensive Assessment System MCEs. See Minimum competency examinations McLure, Gail T , 89 McNeil, Linda, 10 Measurement validity, principle of, 16-17, 182-84 Melancthon, Philip, 93 Mexican American Legal Defense and Educational Fund (MALDEF), on TAAS, 25-26 Mexican American students: ACT per­ formance of, 89, 91/; effective teachers of, 142-A3; NELS perfor­ mance of, 88, 90/; SAT perfor­ mance of, 92/; Texas schools for, characteristics of, 2 2 1 K 1 1

Mexican immigrants, characteristics of, 145-46 Miedel, Wendy T , 173 Mills, Richard, 26 Minimum competency examinations (MCEs), 53; changes in, 65; and college attendance, 63-64, Sit; criticism of, 57-59; and disadvantaged students, impact on, 51-83; and dropout rates, 56-57, 101-2; and earnings, 64-65, SOt; eco­ nomic benefits for graduates, 6-7; and employment success, 57-65; high schools requiring, 54t; and minority students, impact on, 62, 64; negative consequences of, 54-56; public opinion on, 53-54, 55t; race/ethnicity and impact of, 63-64, 82t-83t; skills assessed by, importance of, 57-59, 61;

242

Index socioeconomic status and impact of, 63, 64, 82t-83t; support for, 61; and wages, 64, 77, SOt, 2 0 1 K 4

Minnesota Comprehensive Assessments, 33-36; exemptions from, 34-35; legal challenges to, 35-36; racial/ethnic disparities in performance, 35, 36/", 3 7 / Minority students: disparities in test scores, 8; failure rates for, 180; gains of, 5; high-stakes testing and, 7, 94-96, 119-20; improved performance of, 89; MCE gradua­ tion requirement and, 62, 64; motivation issues for, 8; psycho­ logical stress affecting, 8-9; Regents diplomas for, 14, 29-32; retention of, 15-16, 174-76; TAAS system and, 10,14, 131-32; test performance of, 86; See also Race/ethnicity Monitoring, of high-stakes testing, need for, 105 Motivation: 18th and 19th century views of, 96, 212«22; complexity of, 96-97; differences in, 8; highstakes testing and, 96-100; obsta­ cles to, 97; testing and, assumption regarding, 1, 7-9 Murnane, Richard, 4 1 , 43 NAACR See National Association for the Advancement of Colored People NAEP. See National Assessment of Educational Progress NALS. See National Adult Literacy Survey Narrowing of curriculum, testing and, 10, 93-94, 98-99,117, 211«17 A Nation at Risk, 3, 5, 47 National Adult Literacy Survey (NALS), 42 National Assessment of Educational Progress (NAEP), 109; achieve­

ment gap in, 86-87, 87/, 88/, 89/; achievement scores, in states with and without high school gradua­ tion tests, 113-16, 1151; descrip­ tion of national math attainment, 91; on failure rates, predictions regarding, 181; gains in states with high-stakes graduation test­ ing, 111-13, 112J, 117, 130; gains in states without high-stakes graduation testing, 113, 114t, 117; as indicator, 11; mandatory tests and scores on, 12; vs. SAT and ACT levels, 210«10; signifi­ cance of gains in, 116-17; strengths and limitations of, 108-11, 120 National Association for the Advancement of Colored People (NAACP), on TAAS, 25 National Board on Educational Testing and Public Policy, 105 National Center for Educational Statistics, 155-56 National Center on Education and the Economy, proposed examination system, 98 National Commission on Testing and Public Policy, 104-5 National Educational Longitudinal Study (NELS), 63, 86, 88; racial/ethnic group differences on, 88, 9 0 / National Research Council (NRC): Committee on Appropriate Test Use, 155; study of high-stakes test­ ing, 180 Native Americans, mathematics achievement of, 87, 91/, 9 2 / Natriello, Gary, 14 Neal, Derek A., 43-44 Nebraska, testing policies in, 117 Neill, Monty, 18, 118 NELS. See National Educational Longitudinal Study

Index New York City: grade retention in, 153-54; Promotional Gates pro­ gram in, 175 New York Performance Standards Consortium, 123-24 New York State, Regents Examinations in. See Regents Examinations North Carolina: as model for other states, 107; NAEP framework and tests in, 109; NAEP gains in, 112f, 118 NRC. See National Research Council Occupation, literacy scores and, 42-43 OCR. See Office for Civil Rights OECD. See Organization for Economic Cooperation and Development Office for Civil Rights (OCR), and standards of appropriate test use, 181-82, 192, 236«83 Ogbu, John, 8, 122 Organization for Economic Cooperation and Development (OECD), International Adult Literacy Survey (IALS), 43 Pallas, Aaron M., 14 Parents, Regents program and, 71-72 Payzant, Thomas, 121 Performance: authentic assessment and, 100-101; focus on, vs. focus on learning, 98; See also Academic achievement Persuasive essay (TAAS), 134-35 Pierret, Charles, 6, 60, 77 Placement, validity of test scores for, 186-87 Placement tests, 231«32; vs. mastery tests, 183, 2 3 0 K 2 1

Political climate, for high-stakes testing, 2-5 Poor students: dropout rates for, 102; high-stakes testing and, 7; Regents diplomas for, 14, 32, 3 3 /

243 Presidential campaigns, education issue in, 3-4 Price, Hugh, 51 Primary Language Record, 124 Principals, TAAS program and, 127, 129, 138 Productivity. See Worker productivity Professional development: formative assessments and, 123; Regents pro­ gram and, 73, 74; TAAS displac­ ing, 137 Professional discipline, and educational testing, 188-90 Promotion, 151-78; guidelines for, in all-Regents program, 73-74; rele­ vant information for, 184; vs. retention, 15; test-based, recent proposals for, 152-54; use of achievement tests to end, current enthusiasm for, 154; validity of test scores for, 186-87 Psychological stress, effect on test scores, 8-9 Public education, decline in quality of, political rhetoric emphasizing, 2-4 Puerto Rican students, test performance of, 90/, 91/, 9 2 / Pullout programs, 73 Race/ethnicity: and ACT performance, 89, 91/; and dropout rates, 57, 58i, 165, 169/; and grade reten­ tion, 164-65, 165/, 166/, 167/, 168/, 174-76; and mathematics achievement, 86-88, 87/, 8 8 / 89/; and MCE impact, 63-64, 82*-83*; and Minnesota Comprehensive Assessments performance, 35, 36/, 37/; and NELS performance, 88, 90t; and Regents Examinations performance, 29-32, 30/, 30f, 31/, 3It; and SAT performance, 86, 92/; and TAAS performance, 24t, 24-25, 25t; and wage gap, 4 3 ^ 4 , 45; See also Minority students

244 Raudenbush, Steven W., 42, 45 RCTs. See Regents competency tests Reading: instruction, impact of highstakes testing on, 211wl8; scores, as earnings predictor, 4 1 ; TAAS and, 133 Reagan, Ronald, on education system, 2-3 Reardon, Sean, 174-75 Regents competency tests (RCTs), 67, 76 Regents Examinations, 26-32, 65-77; alternative to, proposal for, 123-24; beneficiaries of, 71; and dropout rates, 56, 75-77; exemp­ tions from, 28-29; experience with, 68-71; failure rates, predic­ tions regarding, 75; generating support for, 68-69; history of, 66-67; implications for state and local educational policies, 71-74; legal challenges to, 32; projected impact of, 13-14; racial/ethnic dis­ parities in performance, 29-32, 30/; 30f, 31/; 31*; schedule for implementation of new standards, 28; shift to all-Regents curriculum, 67-68; and struggling students, 69-70, 71, 74; and teachers, impact on, 70-71, 74, 76; writing prompts in, 207w33 Rehabilitation Act of 1973, Section 504 of, 28 Remedial education, tests and, 177 Retention, 151; and academic achieve­ ment, 168-73; after school entry, 161-63, 162/, 163/; alternatives to, 177-78; and dropout rates, 103, 173-74; effectiveness of, tests as sole measure of, 176; harmful impact of, 15, 103, 187; highstakes testing and, 15-16; in kindergarten, 157/, 157-58; in primary and secondary grades, 159-61, 160/; research on,

Index frustrations of, 173; social differ­ ences in, 163-66; sources of data on, 155-56; test-based, recent pro­ posals for, 152-54; threat of, as incentive to study, 208«43; trends and differentials in, 154-66 Reynolds, Arthur J., 173 Rumberger, Russell W., 174 Safety Net Study Group, recommenda­ tions for, 72-74 Sanctions, attached to tests, negative consequences of, 1-2 Scholastic Assessment Test (SAT): vs. NAEP levels, 210«10; private vs. public school performance on, 42; racial/ethnic group differences on, 86, 9 2 / School day/year, lengthening of, 74 School entry: age at, 157/, 157-58; retention after, 161-63, 162/, 163/ Schooling: additional years of, and gain in earnings, 44, 45; immigrant youths' orientation toward, 146; noncognitive effect on earnings, 44; relationship to earnings and unemployment, 59, 59*; subtractive, 142; time out of school and impact of, 60-61; years of, payoff in relation to, 206n27 Science instruction, TAAS system and, 136 Scores: cut-score methodologies, arbi­ trary nature of, 90-92; inherent imprecision of, ways of reducing, 184 Sheed, Wilfred, 98 Sheldon, Kennon M., 22 Shepard, Lorrie A., 158, 171, 173 Singham, Mano, 122-23 Smith, Adam, 212«22 Smith, Mary Lee, 171 Sobol, Thomas, 26 Social indicators, impact on social process, 93

Index Social promotion. See Promotion Social studies, TAAS system and, 136 Socioeconomic status: and dropout rates, 102; and grade retention, 174-76; and MCE impact, 63, 64, 82t-83t; and test scores, 44 Spaulding, F. T., 211nl7 Special education students: exemption from Regents exams, 76; exemp­ tion from TAAS, 23 Standards, new educational, 46-47 Standards, of appropriate test use, 16-17, 182-88; enforcement problems, 188-91; importance of, 191-94 Standards movement, advocates of, 85 Stanford 9, vs. TAAS results, 108-9 State(s): control over schooling, 20; with high-stakes graduation test­ ing, NAEP gains in, 111-13, 112*, 117, 130; without high-stakes graduation testing, NAEP gains in, 113, 114*, 117; See also specific states Statistically significant results, vs. eco­ nomically significant results, 46 Status dropout rates, 57 Steele, Claude, 8, 37, 122 Sternberg, Robert, 44 Stock responses, 94, 95 Struggling students, Regents Examinations and, 69-70, 71, 74 Subtractive schooling, 142 Sum, Andrew, 45 Summer schools, mandatory, recom­ mendation for, 73 Sun, Anji, 89 Supervisory ratings: as productivity measure, 41; test scores and, 45-46 Supply-type exams, influence on instruction, 94 Systemic effects, of high-stakes testing, 11-12

245 TAAS. See Texas Assessment of Academic Skills Teachers: cultural awareness of, 142—43; job rewards under TAAS, 129; low expectations of, criticism of, 5 1 ; monitoring of performance of, TAAS and, 127; and policy makers, gap between, 149; Regents program and, 7 0 - 7 1 , 74, 76; TAAS and, 127, 129, 138 Teaching. See Instruction Temple, Judy A., 173 Tennessee, NAEP gains in, 112*, 118, 119 Test(s): independent research on, need for, 148-50; performance on, com­ peting explanations of, 185-86; placement vs. mastery, 183, 230«21; power of, 93; standards of appropriate use, 16-17, 182-88; standards, enforcement problems, 188-91; standards, importance of, 191-94; See also High school graduation tests; High-stakes testing Test scores: cut-score methodologies, arbitrary nature of, 90-92; inher­ ent imprecision of, ways of reduc­ ing, 184 Texas: achievement gap in, 119, 136; enrollment rates in, 102-3; grade retention in, 153; history of educa­ tional achievement in, 127; minor­ ity students in, high-stakes testing and, 119-20; as model for other states, 107; NAEP framework and tests in, 109; NAEP gains in, 112*, 118,130 Texas Assessment of Academic Skills (TAAS), 10-11, 23-26, 127-50; artificial curriculum fostered by, 129, 133; drill time for, 132, 135; and dropout rates, 102-3; educational impact of, 132-50;

246 exemptions from, 23-24; expen­ ditures under, 1 0 - 1 1 , 136-38; generic curriculum of, 142; and grade retention, 172; harmful effects of, 129-30; implications of experience with, 14-15; legal challenges to, 25-26; low-level skills emphasized by, 138-40; meaninglessness of results on, 140; and minority students, 10, 14, 131-32; policy rhetoric sur­ rounding, 131; popularity of, 128; vs. principles of learning, 140-42; progression rates under, 14; racial/ethnic disparities in performance on, 24f, 2 4 - 2 5 , 25t; vs. Stanford 9 results, 108-9; and students with limited English proficiency, 144-47; subjects not tested by, reduction in quality and quantity of, 136; subjects tested by, reduction in quality and quantity of, 132-36; as ticket to nowhere, 147-50 Third International Mathematics and Science Study (TIMSS), 103-4 Tinkelman, Sherman, 66 Tracking: definition of, 230«22; nega­ tive consequences of, 187; relevant information for, 184 Triesman, Uri, 122-23 Unemployment, literacy and schooling and, 59, 59t United Kingdom, authentic assessment in, 100-101 U.S. v. Fordice, 235n73

Index Valenzuela, Angela, 10, 144, 148 Validity, measurement, principle of, 16-17, 182-84 Valiga, Michael J., 89 Wages: academic achievement and, 60, 61; cognitive skills and, 43-44; lit­ eracy and schooling and, 59, 59t; MCEs as predictor of, 64-65, 77, 80i, 201M4; as productivity mea­ sure, 40; racial gap in, 43-44, 45; regression models predicting, 206«26; test scores and, correla­ tion between, 6, 7, 41-45 Watts, James, 229w70 West Virginia, testing policies in, 117 White students: grade retention for, 165/", 166f, 167/", 168/; high school completion rates for, 57, 58t; test performance of, 88, 90/ Wiliam, Dylan, 18, 122 Willett, John, 42 Winters, Lynn, 229n70 Worker productivity: academic achieve­ ment in high school and, 60; com­ parisons of, difficulty with, 40; factors affecting, 48; minimum competency examinations (MCEs) and, 61; supervisory ratings as measure of, 45-46; tests as predic­ tors of, 6-7, 39-46; using new standards to predict, 47; using test scores to predict, danger of, 49-50 Workshops, study in, 123 Writing: impact of high-stakes testing on, 95; Regents Examinations and, 207«33; TAAS and, 133-35

tyc/tout? the/

(jo/itm/wtor®

John H. Bishop is a member of the Department of Human Resource Studies at the New York State School of Industrial and Labor Relations, Cornell University. He is also executive director of the Educational Excellence Alliance, a consortium of three hundred high schools that are studying ways to improve school climate and student engagement. Prior to coming to Cornell in 1986, he was director of the Center for Research on Youth Employability and associate director, research, at the National Center for Research in Vocational Education. He has served on numerous advisory committees, and he has published numerous articles on education reform and hiring and training policies. Marguerite Clarke is assistant professor of research in the Lynch School of Education at Boston College and associate director of the National Board on Educational Testing and Public Policy. She is a former Fulbright scholar and was a consultant to the National Research Council for their volume, High Stakes: Testing for Tracking, Promotion, and Graduation (National Academy Press, 1999). Keith Gayler was a doctoral student at the Harvard Graduate School of Education. Robert M. Hauser is Vilas Research Professor of Sociology at the University of Wisconsin-Madison, where he directs the Center for Demography of Health and Aging. He is a member of the National Academy of Sciences and a fellow of the National Academy of Education. His current research interests include trends in educational progression and social mobility in the United States among racial and ethnic groups and the uses of educational assessment as a policy tool. He is the editor, 247

248

About the Contributors

About the Contributors

249

Jay P. Heubert is associate professor of education at Teachers College, Columbia University, and adjunct professor of law at Columbia Law School. He is the coeditor (with Robert M. Hauser) of the National Research Council volume High Stakes: Testing for Tracking, Promotion, and Graduation and Law and School Reform: Six Strategies for Promoting Educational Equity (Yale University Press, 1999). In May 2000 he was named a Carnegie Scholar, one of twelve nationally and two in education.

and research, since 1988. She has taught at the Harvard Graduate School of Education and has been a visiting scholar at the Stanford University School of Education. She served as the vice-president of the Curriculum Studies Division of the American Educational Research Association and is the editor of the Social and Institutional Analysis Section of American Educational Research. She is the author of numerous publications on curriculum, teaching, and urban schooling, including Contradictions of Control: School Structure and School Knowledge (Routledge, 1986) and Contradictions of School Reform: The Educational Costs of Standardized Testing (Routledge, 2000).

Mindy L. Kornhaber is the director of research for K-12 education at the Civil Rights Project at Harvard University and is a reseach associate at the Harvard Graduate School of Education. Her work focuses on how institutions and social policies enhance or impede the development of individual potential. She is coauthor, with Howard Gardner and Warren Wake, of Intelligence: Multiple Perspectives (Harcourt Brace College Publishers, 1996).

Gary Natriello is professor of sociology and education in the Department of Human Development at Teachers College, Columbia University. He is the editor of the Teachers College Record and the director of the Evaluation Center at Teachers College. He is the author of numerous articles and books, including (with William Firestone and Margaret Goertz) From Cashbox to Classroom: The Struggle for Fiscal Reform and Educational Change in New Jersey (Teachers College Press, 1997).

Henry M. Levin is the William Heard Kilpatrick Professor of Economics and Education at Teachers College, Columbia University, and the direc­ tor of the National Center for the Study of Privatization in Education, a nonpartisan entity. He is also the David Jacks Professor Emeritus of Higher Education and Economics at Stanford University.

Monty Neill is the executive director of the National Center for Fair & Open Testing (FairTest, www.fairtest.org). He has directed FairTest's work on testing in the public schools since 1987 and has taught and administered in preschool, hich school, and college. His publications include Implementing Performance Assessments: A Guide to Classroom, School and System Reform and Testing Our Children: A Report Card on State Assessment Systems.

with Jay P. Heubert, of the National Research Council volume High Stakes: Testing for Tracking, Promotion, and Graduation.

George Madaus is the Boisi Professor of Education and Public Policy in the Lynch School of Education at Boston College and a senior fellow with the National Board on Educational Testing and Public Policy. He was the executive director of the National Commission on Testing and Public Policy, which produced the 1990 report, From Gatekeeper to Gateway: Transforming Testing in America. Ferran Mane is an assistant professor at the Rovira i Virgily University (Spain). He has been visiting fellow at Essex University (England) and at Cornell University. He has published several articles and chapters on vocational education, the effects of on-the-job training on workers' pro­ ductivity, changes in the American education system, and the effects of technological change on education and occupational structures. Linda McNeil is a professor of education and has been codirector of the Rice University Center for Education, a center for teacher development

Gary Orfield is professor of education and social policy at Harvard University. He is codirector (with Christopher Edley, Jr.) of the Civil Rights Project at Harvard University. His most recent books are (with Susan Eaton) Dismantling Desegregation: The Quiet Reversal o^Brown v. Board of Education (New Press, 1996); (with Edward Miller) Chilling Admissions: The Affirmative Action Crisis and the Search for Alternatives (Harvard Educational Publishing Group, 1998), and (with Michal Kurlaender) Diversity Challenged: Evidence on the Impact of Affirmative Action (Havard Educational Publishing Group, 2001). Aaron M. Pallas is professor of sociology and education in the Department of Human Development at Teachers College, Columbia University. He is the editor of Sociology of Education and a past chair of the Sociology of Education Section of the American Sociology Association.

250

About the Contributors

He studies the interrelations of educational stratification, the social orga­ nization of schools, and the life course of individuals. Angela Valenzuela is associate professor of education and of Mexican American studies at the Univesity of Texas at Austin. Prior to this posi­ tion, she taught at Rice University. Her book Subtractive Schooling: U.S. Mexican Youth and the Politics of Caring (State Uniyesrity of New York Press, 1999) was awarded the American Educational Research Association Outstanding Book Award, the highest prize in the United States in education research. She also has been the recipient of a Ford Foundation Post-Doctoral Fellowship.

?>

Advance praise l o r Raising Standards

OP Raising

Rarriers?

"This is the best book I have read on high-stakes testing. Essays by distinguished social scientists and educators point out the weaknesses, inequalities, and misdi­ rection of current policy and practice. This is urgently needed today." —ArthUP Levine, President, Teachers College. Columbia University

Supporters and critics of the present trend will have to deal ' arguments put forth in this provocative collection —Howard Gardner, Hobbs Professor of Cognition and E Harvard University; Author of The I cipliiiecl Mind "Is high-stakes testing a panacea or a Pandora's Box? This book s comprehensive body of evidence on these questions should be read by every policymaker and practitioner of education. There will be a test at the end of the book, and it carries extraordinarily high stakes for students, schools, and our American democracy itsel(vn —Linda Darling-Hammond, Charles E. Ducoinnuui Professor of Education, Stanford University

The contributors to this volume are: John H. Bishop ♦ Marguerite Clarke ♦ Robert M. Hauser ♦ Jay P. Heubert Mindy L. Kornhaber ♦ Henry M. Levin ♦ Linda McNeil ♦ George Madaus Ferran Mane ♦ Gary Natriello ♦ Monty Neill with Keith Gayler Gary Orfield ♦ Aaron M. Pallas ♦ Angela Valenzuela

To order, call 1 (800) 552 5450 In Washington, DC (202) 707-0258 The Century Foundation Press 41 East 70th Street, NY, NY 10021

"HE CENTURY FOUNDATION

ISBN 0 - 8 7 0 7 8 - 4 5 2 - S

780870Y84521