Springer Series on Evidence-Based Crime Policy
Series Editors: Lawrence W. Sherman Heather Strang
Crime prevention and criminal justice policies are domains of great and growing importance around the world. Despite the rigorous research done in this field, policy decisions are often based more on ideology or speculation than on science. One reason for this may be a lack of comprehensive presentations of the key research affecting policy deliberations. While scientific studies of crime prevention and criminal policy have become more numerous in recent years, they remain widely scattered across a wide range of journals and countries The Springer Series on Evidence-Based Crime Policy aims to pull this evidence together while presenting new research results. This combination in each book should provide, between two covers (or in electronic searches), the best evidence on each topic of crime policy. The series will publish primary research on crime policies and criminal justice practices, raising critical questions or providing guidance to policy change. The series will try to make it easier for research findings to become key components in decisions about crime and justice policy. The editors welcome proposals for both monographs and edited volumes. There will be a special emphasis on studies using rigorous methods (especially field experiments) to assess crime prevention interventions in areas such as policing, corrections, juvenile justice, and crime prevention. Published in Cooperation with the Campbell Crime and Justice Group
For further volumes: http//www.springer.com/series/8396
wwwwwwwwww
Faye S. Taxman
●
Steven Belenko
Implementing Evidence-Based Practices in Community Corrections and Addiction Treatment
Faye S. Taxman Criminology, Law and Society George Mason University Fairfax, VA, USA
[email protected] Steven Belenko Department of Criminal Justice Temple University Philadelphia, PA, USA
[email protected] ISBN 978-1-4614-0411-8 e-ISBN 978-1-4614-0412-5 DOI 10.1007/978-1-4614-0412-5 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2011939835 © Springer Science+Business Media, LLC 2012 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
Preface
For most of our careers, we have devoted our work to generating knowledge about “what works,” mostly aimed at improving criminal justice outcomes for offenders. Both of us have advanced new ideas and concepts in judicial and corrections programs to address the criminogenic needs of offenders, primarily substance abusers and high risk offenders. Steven Belenko conducted extensive research on treatment needs and substance abuse treatment interventions for drug-involved offenders including prosecutorial diversion programs, drug courts, and prison treatment. Faye Taxman has devoted extensive work toward developing, testing, and refining models of continuum of care and seamless system in probation, prison, and jail settings. And, she has worked on testing new models for facilitating the advancement of researcher–practitioner partnerships as well as interventions to promote organizational change. We both have worked extensively in collaboration with practitioners and policy makers to demonstrate that the criminal justice system can deliver effective interventions that improve the quality of life for offenders, their families, and communities. The implications of our work, along with other colleagues, are that it is important to offer services within the corrections and criminal justice settings as a tool to improve both public safety and public health. We strongly believe that the corrections system should be part of the service delivery system in the USA, including integrated behavioral and health services that promote boundaryless systems. The high concentration of offenders with substance use disorders, mental health problems, and infectious diseases demands a new paradigm. Along the way, we both realized that the issues were increasingly less about the research on “what works” (or identifying services that improve behaviors) and more about: (1) the degree to which the community corrections and other criminal justice organizations and society at large believe that research findings are useful for crime prevention efforts; (2) the techniques that community and institutional corrections agencies can use to reduce widespread barriers and resistance to organizational change; (3) the willingness of corrections and other justice agencies to consider effective ideas and strategies from other disciplines that seek to change behavior; and (4) the willingness of addiction treatment, mental health, and other behavioral services agencies to provide care for offenders. v
vi
Preface
The “aha” moment evolved slowly as both of us pursued our own work during an era of increasingly punitive sentencing policies and enormous increases in incarceration and community control. This era coincided with a growing recognition that the general punishment tools of the corrections system did little to promote a drugor crime-free lifestyle; in fact, in many ways it contributed to the growing cynicism about the intentions and effectiveness of the justice system. We noted that many of the offenders affected by these punitive policies had underlying drug abuse disorders. Our own work evolved as a result of being exposed to a health services framework and becoming practicing “health services criminologists.” The growing research on behavioral health interventions transformed our own contributions to criminology and criminal justice as we learned more about the stages of behavioral change and how different types of interventions or techniques can facilitate such change. Through this lens, we began to understand that the lessons from health services research and intervention science were applicable to the problems of interventions in targeted settings like the corrections and judicial systems. The growth of new research and tools to advance organizational change, and the emergence of a new science of implementation, also caught our attention. At the same time, the addiction treatment field was undergoing its own metamorphosis. This specialty field of care was the subject of several important and critical reports by the Institute of Medicine. The IOM “Bridging the Gap” report in 1996 emphasized the need to improve the type and quality of care offered to those that suffered from an addiction disorder. This report triggered many new studies devoted to the quality of addiction treatment, the infrastructure needs of the field, and the barriers to advancing clinical practice. The focus of the IOM report on improving the adoption of evidence-based practices in the addiction treatment system coincided with the needs and demands of the corrections and larger justice systems – the unmet needs of service delivery agencies given the unserved client-level needs. In our view, the similarities between the needs for change in addiction treatment and criminal justice/community correction agencies were glaring. More importantly, while a number of research-based and effective interventions to reduce recidivism and drug use had been identified over the years, it was clear that these evidencebased practices were not being disseminated widely, or being sustained over time once they were implemented. As the evidence-based practices and treatment methods gained momentum, it was apparent that renewed attention to implementation and methods to advance organizational change processes were sorely needed. This book actually started as part of an idea that Steven Belenko had to establish the Center on Evidence-based Interventions for Crime and Addiction (CEICA) (during his tenure at the Treatment Research Institute, and codirected by Drs. Belenko and Harry K. Wexler). They organized a CEICA conference in December 2006 that brought together researchers, practitioners, and policy makers to discuss the challenges of implementing effective addiction treatment programs for offenders. Shortly thereafter, the National Institute of Corrections provided a grant (PI, Dr. Belenko, Drs. Taxman and Wexler co-PIs, Cooperative Agreement 06PEI06GJN8) to produce a white paper on this topic that served as an impetus for this book and our ongoing work on implementation.
Preface
vii
At the same time, Faye Taxman was working with the Maryland Division of Probation and Parole on an implementation project revamping probation supervision, to include officers using adapted motivational interviewing and cognitive restructuring techniques as well as using risk and need assessments to inform supervision plans. This project is part of a 20 year agreement that Dr. Taxman has with this agency to implement innovations (EBP) and to engage in technology transfer efforts, and has brought to light many of the challenges confronting community corrections agencies trying to move from an enforcement model to EBP. Both of us have ongoing research that propelled further work in this area. Steven Belenko (Temple University) and Faye Taxman (George Mason University) each direct centers engaged in implementation research to test various implementation, organizational change and process improvement strategies to increase the use of evidence-based practice in corrections agencies; these are part of the Criminal Justice Drug Abuse Treatment Studies (CJDATS) cooperative agreement, funded by the National Institute on Drug Abuse. Dr. Taxman also directed the national surveys of the criminal justice agencies that revealed many of the issues addressed in this book regarding the low uptake of EBP, and the organizational dynamics that affect the inability for science to penetrate traditional practice. Our careers are turning now as we embark on new studies that build on this framework of intervention science and implementation science. This budding field builds upon our own training, and a research framework incorporating Patton’s (1987) utilization focused evaluation methods, action research, and policy focus. But our new research agendas allow for more attention to the methods and techniques to support organizational and systems change. This book is the culmination of many years of funding and research that we both share. Some of our funders have been the National Institute on Drug Abuse, National Institute on Alcohol Abuse and Alcoholism, Center for Substance Abuse Treatment, National Institute of Justice, National Institute of Corrections, Bureau of Justice Assistance, State Justice Institute, Edna McConnell Clark Foundation, and Robert Wood Johnson Foundation. Steven Belenko acknowledges the following people who have influenced his research and thinking about how to improve criminal justice practices and policies and reduce the harms caused by drug abuse: Tom McLellan, Richard Dembo, Jeremy Travis, Dean Fixsen, Ken Schoen, the late Jack Novik, Nancy Wolff, Joseph A. Califano, Jr., Anne Swern, his CJDATS colleagues, and the many drug court judges with whom he has had the pleasure to work. Faye Taxman acknowledges the following people that have shaped her own views including Judith Sachwald, Ernest Eley, Jasper Ormond, Tom Williams, Peter Luongo, Justin Jones, and others that each day try to import EBP into the justice and addiction treatment settings. Dr. Taxman’s mentors and collaborators in the field include Edward Latessa, Todd Clear, Tom McLellan, Doug Anglin, the late Doug Longshore, James Byrne, Dwayne Simpson, and the late Don Gottfredson. Navigating the pathway to study the issues related to adopting EBP has been Peter Friedmann, Doug Young, Craig Henderson, Jeff Bouffard, Susan Turner, David Weisburd, Danielle Rudes, Shannon Portillo, Karen Cropsey, and CJDATS collaborators, to name a few. We both acknowledge Redonna Chandler for her insights and
viii
Preface
contributions to our work. Both authors shared equally in the conceptualization, background research, and writing of this book. This book would have not been possible without the assistance of Judith Sachwald, who was an invaluable editor. Doctoral students Jennifer Lerch and Jill Viglione at George Mason University’s Criminology, Law and Society department and George Mason University law student, Carolyn Watson, assisted us in preparing figures, exhibits, references, and other parts of this book, and we thank them immensely. It is not surprising that implementation studies are receiving increasing attention. This book is dedicated to the practitioners, policy makers, and scholars that seek to improve the quality of addiction treatment and other behavioral health programming to reduce recidivism and relapse to drug use among the enormous numbers of offenders in need of services. Theirs is a noble venture, and one that demands our full attention and support. We are committed to advancing knowledge in this field and hope that this book stimulates new research, new synergies to implement EBP, and new organizational efforts to deliver the most effective services possible to the greatest number of people. The realization that the corrections system should be a service delivery system would fulfill our aspirations for an effective crime control policy that recognizes the value of community corrections and addiction treatment services. Fairfax, VA Philadelphia, PA
Faye S. Taxman Steven Belenko
Contents
1
2
Introduction ............................................................................................. 1.1 The Promise of Science in Public Policy ......................................... 1.2 Why Community Corrections and Addiction Treatment? ............... 1.3 Building Knowledge in Community Corrections ............................ 1.4 The Focus on Technology Transfer.................................................. 1.5 Community Corrections Presents Unique Challenges for Addiction Treatment Interventions............................................. 1.6 Multistage Conceptual Model for Identifying and Selecting EBPs .......................................................................... 1.7 Evidence-Based Interagency Implementation Model (EB-IIM)...... 1.8 Conclusions and Outline of the Book .............................................. References ................................................................................................. Identifying the Evidence Base for “What Works” in Community Corrections and Addiction Treatment ........................ 2.1 Introduction and Overview .............................................................. 2.2 Basic Definitions and Concepts ....................................................... 2.2.1 Hierarchy of Levels of Evidence.......................................... 2.3 Efficacy vs. Effectiveness ................................................................ 2.4 Frameworks for Determining the Evidence Base ............................ 2.4.1 The Food and Drug Administration (FDA) Model .............. 2.4.2 Applying the FDA Model to Behavioral Interventions......................................................................... 2.4.3 Synthesizing Across Research Designs ............................... 2.4.4 Consensus Processes ............................................................ 2.4.5 Systematic Reviews and Meta-Analyses ............................. 2.5 Evidence-Based Repositories........................................................... 2.5.1 Cochrane Collaboration/Cochrane Reviews ........................ 2.5.2 Campbell Collaboration–Crime and Justice Group ............. 2.5.3 National Registry of Evidence-Based Programs and Practices ........................................................................
1 4 5 7 8 10 11 12 14 15 19 19 22 22 24 26 26 27 27 28 29 32 32 33 34 ix
x
Contents
2.5.4 Blueprints for Violence Prevention ...................................... 2.5.5 Washington State Institute for Public Policy ....................... 2.6 NIDA Principles of Effective Drug Treatment ................................ 2.7 Defining “What Works” in Community Corrections ....................... 2.8 Standards of Evidence in Community Corrections and Addiction Treatment ................................................................. 2.9 Conclusions ...................................................................................... References ................................................................................................. 3
4
Theories of Organizational Change and Technology Transfer ........... 3.1 The Implementation Quandary ........................................................ 3.2 Understanding Organizational Approaches: Three Different Models.................................................................... 3.2.1 Diffusion Models ................................................................. 3.2.2 The Conceptual Model ......................................................... 3.3 Expanding the Concept of Implementation ..................................... 3.4 Moving Past Initial Implementation: The Concept of Sustainability.......................................................... 3.5 Building Interagency Collaborative Supports: The Availability, Responsiveness, and Continuity (ARC) Model ............................... 3.6 Attention to Performance: Quality Improvement Processes, Performance Contracts, and Benchmarking .................................... 3.6.1 Plan-Do-Study-Act (PDSA)................................................. 3.6.2 Network for Improvement of Addiction Treatment (NIATx): Quality Improvement Processes ........................... 3.6.3 COMPSTAT and Feedback Loops ....................................... 3.7 Total Organizational Change Processes ........................................... 3.8 Conclusion ....................................................................................... Appendix: List of Organizational Change Models ................................... References ................................................................................................. Organizational Change – Technology Transfer Processes: A Review of the Literature ..................................................................... 4.1 Systematic Reviews of Change Strategies ....................................... 4.2 Outer Setting: The Environmental Context for Change................... 4.3 Inner Setting: Within a Specific Organization ................................. 4.3.1 Readiness for Change........................................................... 4.3.2 Alignment of Values............................................................. 4.3.3 Structure ............................................................................... 4.3.4 Professionalism and Staffing................................................ 4.3.5 Resources ............................................................................. 4.3.6 Summary of Inner Setting Findings ..................................... 4.4 Organizational-Level Models of Technology Transfer .................... 4.4.1 Preparing the Organization for the Change.......................... 4.4.2 Staff-Level Concerns Regarding Technology Transfer ............................................................
37 39 40 46 49 50 52 57 57 59 61 62 68 70 76 79 80 81 82 83 85 86 87 91 92 96 98 100 101 101 102 103 104 104 108 109
Contents
4.4.3 Client Factors ....................................................................... 4.4.4 Change Actors ...................................................................... 4.4.5 Training ................................................................................ 4.5 Dissemination Efforts: Specialized Training by Researchers .......... 4.5.1 Piloting ................................................................................. 4.5.2 Fidelity and Program Integrity ............................................. 4.5.3 Performance Monitoring ...................................................... 4.5.4 Quality Improvement Models .............................................. 4.5.5 Summary .............................................................................. 4.6 Conclusion ....................................................................................... Appendix: Summary of Major Findings from Organizational Studies in Behavioral Health Examining Inner Setting Issues............................... References ................................................................................................. 5
6
Community Corrections Addiction Treatment: Strategies to Adopt, Implement, and Sustain Effective Practices ........................ 5.1 Current State of Evidence-Based Practice in the Addiction Treatment Field ................................................................................ 5.2 Improving Treatment Processes ....................................................... 5.2.1 Federal and National Initiatives to Disseminate EBP in Treatment Agencies ......................................................... 5.2.2 Federal Initiatives to Define Quality and Key Outcome Measures for Treatment Agencies ........................ 5.3 Disseminating EBP: Lessons from the Centers for Disease Control and Prevention’s REP and DEBI Models for HIV Interventions ............................................................................ 5.3.1 Replicating Effective Programs ........................................... 5.3.2 Diffusion of Effective Behavioral Interventions (DEBI) ..... 5.4 Adoption of EBP in Community Addiction Treatment ................... 5.4.1 State Initiatives ..................................................................... 5.5 Conclusions ...................................................................................... References ................................................................................................. Current State of EBP in the Community Corrections Field ............... 6.1 EBP in Community Corrections Agencies: Results from the National Criminal Justice Treatment Practices Survey .............. 6.1.1 Best, Evidence-Based, or Strongly Supported Practices ............................................................................... 6.1.2 Adopting EBP ...................................................................... 6.1.3 Adopting EBP in Community Corrections Settings ............ 6.1.4 Important Constructs in Understanding Adoption Patterns ................................................................................. 6.1.5 What Factors Affect the Likelihood of Adopting EBP in Corrections Settings? ....................................................... 6.1.6 How Do Corrections Administrators Handle the Competing Values of Providing Treatment and Other Services? .............................................................
xi
110 110 112 113 114 115 115 116 116 118 119 123 129 129 133 135 136
141 142 143 144 144 146 147 151 152 153 154 154 155 157
159
xii
Contents
6.1.7
What Type of Reform Strategy Advances the Use of EBP in Corrections Settings? ........................................ 6.1.8 Conclusions from NCJTP Survey Findings ....................... 6.2 NIC Initiative to Expand the Use of EBP by Community Corrections Agencies ....................................................................... 6.2.1 Dissemination Materials for the Field ............................... 6.2.2 Initial NIC/CJI Adoption Sites: Maine and Illinois ........... 6.2.3 NIC Framework Garnering Support from Sister Organizations .................................................. 6.3 View from the Field: Results from Key Informant Interviews ........ 6.3.1 How Are Evidence-Based Practices Identified? ................ 6.3.2 Factors that Affect the Use of Evidence-Based Practice in Corrections Agencies ....................................... 6.3.3 Current Use of EBP ........................................................... 6.3.4 Current Level of Corrections Staff Knowledge about EBP .......................................................................... 6.3.5 What Constitutes Evidence in EBP?.................................. 6.3.6 Key Gaps in Knowledge about Evidence-Based Practice .............................................................................. 6.3.7 Are State and Federal Regulators Requiring EBP? ........... 6.3.8 Key Steps Required for Successful Implementation of EB Treatment................................................................. 6.3.9 Key Challenges for Implementation and Sustainability of EBP .................................................. 6.3.10 Organizational Changes and Conditions Needed for Implementing and Sustaining EBP............................... 6.3.11 Necessity of Protocol Fidelity ........................................... 6.3.12 Are Outcomes Regularly Obtained and Reported to Demonstrate Effectiveness? ........................................... 6.4 Conclusion ....................................................................................... References ................................................................................................. 7
The Idiosyncrasies of the Corrections and Treatment Environments........................................................................................... 7.1 Recognizing the First Hurdle ........................................................... 7.2 Opening the Door to Offender Change as a Goal of Corrections................................................................................... 7.2.1 Looking Through an Offender-Based Lens ....................... 7.2.2 Lack of Infrastructure, Knowledge, and Skills .................. 7.3 The Deficiencies of Addiction Treatment Programs........................ 7.3.1 Overcoming EBP Implementation Barriers to Achieve Improvements in Addiction Treatment .............................. 7.4 Conclusion ....................................................................................... References .................................................................................................
160 160 161 162 164 167 169 171 171 172 174 175 176 177 178 179 181 182 184 185 186 189 190 192 193 196 196 199 201 202
Contents
8
9
Making Good Choices: A Multistage Conceptual Model for Identifying and Selecting Evidence-Based Practices ..................... 8.1 Overview of the Evidence Mapping Process ................................... 8.1.1 Tests of Scientific Rigor ....................................................... 8.1.2 Tests of Transportability ...................................................... 8.1.3 Tests of Organizational Capacity ......................................... 8.2 The Challenges Associated with Scientific Studies ......................... 8.2.1 Limitations of RCT and Quasi-Experimental Designs ........ 8.2.2 Selection Effects, Targeting, and Penetration ...................... 8.2.3 Population Impact and Penetration ...................................... 8.2.4 Statistical Significance and Effect Size ................................ 8.3 Transportability Assessment ............................................................ 8.3.1 Setting and Populations........................................................ 8.3.2 Fidelity and Program Integrity ............................................. 8.3.3 Incorporating Clinician and Other Staff Input ..................... 8.4 Organizational Capacity Assessments ............................................. 8.4.1 Perceived Value of EBP: Balancing Public Safety and Public Health ................................................................. 8.4.2 Improve Interagency Efforts Through Goal Alignment ....... 8.4.3 Inner Setting Issues .............................................................. 8.4.4 Building Interagency Systems of Care................................. 8.4.5 Considering Clients’ Perspectives and Treatment Needs.................................................................................... 8.4.6 Intervention Costs ................................................................ 8.4.7 Outer Setting ........................................................................ 8.5 Making the Decision to Adopt an EBP ............................................ 8.6 Conclusions ...................................................................................... References ................................................................................................. Conceptual Model: Evidence Based Interagency Implementation Model ........................................................................... 9.1 The Evidence-Based Interagency Implementation Model (EB-IIM) ............................................................................... 9.1.1 Develop Knowledge Stage ................................................... 9.1.2 Building Foundation (Improving the Capacity at Individual and Organizational Level)............................... 9.1.3 Set Expectations: The Use of Benchmarks for Performance.................................................................... 9.1.4 Align: Using the Pilot as a Learning Stage .......................... 9.1.5 Renovate and Sustain: The Ultimate Goals ......................... 9.2 Setting the Stage for Implementation: Core Components to Manage the Process ..................................................................... 9.2.1 Creating a Culture of Change: Techniques and Strategies ....................................................................... 9.2.2 Strategies to Build a Broad Base of Support: The Example of Social Marketing .......................................
xiii
207 209 210 211 211 212 212 213 215 217 218 219 220 222 223 223 224 225 226 227 228 229 231 232 234 239 240 244 248 254 256 258 261 262 263
xiv
Contents
9.3
10
Techniques to Manage the Change Process ................................... 9.3.1 Executive Leadership........................................................ 9.3.2 Staff .................................................................................. 9.3.3 Facilitators of the Change Process.................................... 9.3.4 Implementation Change Teams Through Cross-Sectional or “Vertical Slice” Working Teams................................... 9.3.5 Stakeholders...................................................................... 9.3.6 Project Management Activities ........................................ 9.3.7 Clear Performance Goals and Measures Provide Objective Feedback on Progress....................................... 9.4 Conclusion ..................................................................................... References .................................................................................................
265 265 266 267
Evidence-Based Implementation Agenda ............................................. 10.1 Implementation Tools .................................................................... 10.1.1 Intervention or Evidence-Based Practice/Treatment ........ 10.1.2 Inner Setting ..................................................................... 10.1.3 Outer Setting ..................................................................... 10.1.4 Process: Evidence Mapping and Implementation Models .............................................................................. 10.2 Advancing a Research Agenda on Implementation in Community Corrections Settings ............................................... 10.2.1 Intervention Strategies ...................................................... 10.2.2 Implementation Strategies ................................................ 10.3 Council for Public Health-Safety Evidence-Based Practice .......... 10.4 Conclusion ..................................................................................... Appendix A: Interventions Checklist........................................................ Appendix B: Inner Setting Checklist ........................................................ Appendix C: Outer Setting Checklist ....................................................... Appendix D: Process Management Issues Checklist ................................ References .................................................................................................
275 281 281 284 286
267 268 268 269 270 272
288 289 292 295 298 299 301 304 306 308 310
Index ................................................................................................................. 315
Chapter 1
Introduction
With nearly eight million adults under correctional control in the United States, the numbers of persons incarcerated and under community supervision have exploded in the United States. This reflects public concerns over increasing crime rates, shifts toward incapacitative and just deserts sentencing policies, and frustration over the perceived failures of other crime prevention policies and programs. The latter perception implies that the only effective response to criminal behavior is to arrest more people and lock them up in prison or jail for longer periods of time. Robert Martinson’s famous “Nothing Works” paper (1974) is credited for fueling some of the discontentment with programs and services for offenders. His rejoinder a few years later that “some things work” (Martinson, 1979) did not alter the political landscape. The real story reported by Martinson was that few correctional programs had been implemented as intended, and even if implemented the program was not very different from existing services. But both of his treatises emphasized the bottom line of effectiveness – the failure to improve offender-level outcomes. The concerns about implementation were largely ignored in this seminal work. The emphasis on offender outcomes has dominated the last few decades in which punishment and incapacitation have been the focus of criminal justice policy. Treatment and other correctional programs have been viewed as luxuries with little potential for reducing recidivism. Although the research literature continues to highlight that evidence-based programs and treatments are effective in reducing recidivism and other negative outcomes (see MacKenzie, 2006; Nagin, Cullen, & Jonson, 2009), implementation of effective treatments and programs has taken a backseat in correctional practice. Even though some states report the expanded use of communitybased alternatives to incarceration, the programs that are in place are grossly underutilized and have a low capacity: fewer than 10% of offenders under correctional supervision are involved in programming on any given day (Taxman, Perdoni, & Harrison, 2007). The incarceration preference has come at a stiff price. Nearly eight million adults are under correctional control (Glaze, 2010), the United States holds nearly 25% of the world’s incarcerated population (Pew Center on the States, 2009), and nearly F.S. Taxman and S. Belenko, Implementing Evidence-Based Practices in Community Corrections and Addiction Treatment, Springer Series on Evidence-Based Crime Policy, DOI 10.1007/978-1-4614-0412-5_1, © Springer Science+Business Media, LLC 2012
1
2
1
Introduction
70% of offenders recycle through the system within three years (Langan & Levin, 2002). Many states spend more funds on prisons and jails than college education (Pew Center on the States, 2008). The growing corrections population reflects law enforcement policies and mandatory sentencing laws over the past 25 years that have largely affected drug-involved offenders (Mauer & King, 2007; Tonry & Farrington, 1995). As a result of these policies and the close connection between substance abuse and crime (White & Gorman, 2000), 75% of offenders have substance abuse problems (Belenko & Peugh, 2005). Related health issues include extremely high rates of HIV/AIDS and other sexually transmitted diseases compared with the general population (CDC, 2005, 2006; Glynn & Rhodes, 2005; Maruschak, 2009), and high prevalence of co-occurring mental health disorders (Belenko, Lang, & O’Connor, 2003; James & Glaze, 2006; National Institute of Mental Health, 2008; Teplin et al., 2006). One impact of the expanded crime control policies that focus on arrest and incapacitation is that more people with social problems are being handled by the correctional system. During the same era that saw increased use of incarceration for drug-related behaviors, there has been a growing body of science about effective crime prevention policies. The report to Congress on Preventing Crime: What Works, What Doesn’t, What’s Promising (Sherman et al., 1997), along with a substantial body of research on effective interventions for inmates and other offenders (Andrews, Zinger, Hoge, & Bonta, 1990; MacKenzie, 2006; Mitchell, Wilson, & MacKenzie, 2007; Wilson, Mitchell & MacKenzie, 2006), proferred that drug treatment and certain interventions are effective tools to reduce criminal behavior. Moreover, a well-articulated body of research exists for evidence-based treatments for substance abuse disorders (National Institute on Drug Abuse, 2006) that provides a roadmap for reducing recidivism and drug use. Reducing drug abuse through effective treatment could have positive impacts on public safety and reduce the strains on state and local criminal justice systems occurring from the overuse of incarceration. But, as Martinson indicated more than three decades ago, merely making treatment available is not sufficient to achieve the delivered results. Guidance is needed as to which treatments are effective, which offenders would benefit from what types of treatment, how to improve access to effective treatment, how to change organizations to expand treatment capacity and access for offenders, how to implement treatment programs that can deliver the desired results, and how to overcome factors that negatively affect the delivery of effective addiction treatment in the justice setting (Taxman & Bouffard, 2000; Taxman et al., 2007). Accordingly, the failure to adequately implement interventions for drug-involved offenders has enormous implications for public safety and public health. The challenges associated with changing organizational culture and processes and improving program implementation are ever present. Little is known about how to expeditiously and successfully implement effective evidence-based treatments in real-world settings, including behavioral health and community corrections settings. An emerging science of implementation is designed to answer questions of how to best disseminate, implement, and sustain evidence-based treatments and practices in organizations and systems that typically struggle to do so. Theorists and
1
Introduction
3
researchers are now examining the process of changing organizational cultures, integrating behavioral health services across different systems, and improving the implementation of health services. Over the past three decades, in many areas of health and social services – medicine, education, social work, addiction treatment, mental health, and criminology – there has been increasing focus on the importance of promoting and assuring the use of evidence-based practices (EBP). EBP represent the idea that there is sufficient science in a given area to declare that a particular practice or treatment is efficacious and can be implemented as part of routine practice. The EBP movement has been facilitated by the development of: (1) scientific methods to summarize the state of knowledge and impacts of interventions, practices, and policies; (2) statistical tools to synthesize results across different studies and to measure the size of the effect; and (3) theoretical frameworks to classify studies to improve the practices. The term evidence-based practices is, in essence, a catchphrase or slogan for interventions or practices that should be widely used because research indicates that they positively alter human behavior. EBP are considered the holy grail for achieving desired outcomes. But, is the solution as simple as merely using science to identify and designate specific interventions or practices as evidence-based? When the EBP movement began, the assumption was that the main challenge was to conduct research to determine if a program was effective – did it improve outcomes compared with treatment as usual? Once an EBP was identified, it was believed that agencies and organizations would enthusiastically rush to implement these practices. The reality has turned out to be far more complex. The lessons from healthcare fields are that it takes many years for evidence-based medicine to work its way into routine practice, if it ever does. The same is true for behavioral health services, education, social services, and criminal justice settings where the institutional imperative is to continue to do business using familiar practices and processes. This book addresses the connected issues of knowledge development and utilization in the context of adopting and implementing EBP, particularly addiction treatment programs, in community corrections agencies. The intersection of EBP in community corrections and addiction treatment presents unique challenges because it affects two disciplines and requires community corrections systems to articulate and embrace a secondary mission of offender change (rehabilitation). A great need exists for researchers, organizations, and policymakers to understand the process for determining EBP and the even greater need to consider the viability of an EBP, given the organizational context and the alignment with the environment. The transportability of an EBP is seldom discussed, yet given the history of poor implementation in correctional settings (Gendreau, Goggin, & Smith, 1999), it is critical to consider and rethink the process toward routinization and sustainability of EBP. This book assesses the EBP movement in terms of its contribution to identifying practices and policies that should lead to improved outcomes at the individual and system level, and examines the research surrounding the adoption and implementation of EBP using an organizational lens. The organizational lens requires a commitment to the utilization of EBP in these settings as well as presenting models
4
1
Introduction
for identifying and selecting EBP and for advancing the integration of EBP into practice. In many ways this book expands the concept of EBP to consider evidencebased organizational research as well as management strategies to advance the uptake of EBP in community corrections settings.
1.1
The Promise of Science in Public Policy
The EBP framework is in many ways a common sense, practical perspective that a rational, data-driven strategy will help decipher what treatment or practice is worthwhile to use, what should be abandoned, and what needs to be improved. The whats can refer to a variety of practices in an organization – ranging from normal business practices to special programs. The EBP framework is theoretically apolitical. Decisions about what works are not (and should not be) guided by philosophical orientations, historical practices, politics, staff preferences, or managerial preference; instead the EBP framework emphasizes effectiveness, which is defined as the degree to which the improvement outperforms existing practices. In reality, these factors often intrude to prevent EBP from being more broadly implemented. Science is objective, but the scientific mind is framed in the community to which one belongs and includes the values, norms, and methods accepted within that community. Thomas Kuhn, in his famous Structure of Scientific Revolutions (1966), developed the concept of the paradigm to indicate that science grows by replacing one paradigm with another. Each paradigm provides the means to test existing theories or ideas. This process then serves to advance knowledge. In many ways, EBP represents the new paradigm since it provides a framework to safeguard objectivity by using science to define the practices or products that should improve the outcomes. The EBP paradigm gauges what is useful in the real world through scientific studies that measure objective outcomes. Evidence-based practice, however, is fraught with certain realities inherent in the process of defining and using knowledge. With respect to knowledge development, the objective questions are: (1) how many studies need to be conducted for a practice to be declared an EBP; (2) how many studies of a certain practice/intervention/ treatment have been conducted in which populations and settings?; (3) are the studies of sufficient scientific quality; and (4) are the components of the practice/ intervention/treatment adequately identified or defined so they can be replicated in the field? That is, the evidence-based foundation can only be constructed when there is sufficient knowledge about the setting, population, and/or intervention. Although the EBP field has determined that a minimum of two studies with similar findings constitutes sufficient evidence, this standard may or may not address the variation in setting, population, nature of the intervention, and adherence to the original intervention design. The potential to determine whether a research finding is robust is substantially increased when there are more studies across varied settings and populations. And, scientific knowledge is built on the presumption that replication increases confidence that research findings are not random or subject to biases.
1.2
Why Community Corrections and Addiction Treatment?
5
Knowledge utilization pertains to how practitioners and policymakers apply scientific knowledge, whether it is within an organization or within the stakeholder community. Application refers to the distillation of key concepts and core components, the alignment or fit within the organizations’ processes or culture, and likelihood of addressing recognized problem areas. The importation and continued use of new knowledge, whether it is a foreign concept/idea or a refinement of existing practice, is a field of study that is slowly evolving. A related concept is the notion that the knowledge will evolve or be modified in a manner to fit the environment. But an important concern is whether the evolving practice will be similar to the science-based practice that was studied. “Technology Transfer (TT),” the process of introducing new technologies, programs, or practices into organizations, addresses the area of knowledge utilization. An expanded concept of technology transfer is the new, emerging field of implementation science: the study of how organizations effectively implement and sustain new or modified practices. This is an area of increasing interest because the time lag between science development and utilization is estimated to be as much as 20 years (Proctor et al., 2009); there is, thus, great interest in increasing the rate of adoption and implementation of EBP in organizational settings to improve overall outcomes.
1.2
Why Community Corrections and Addiction Treatment?
The numbers are staggering. Nearly six million offenders are under community corrections (probation or parole) supervision in the United States, or about 1:23 adults 18–65 years old (Glaze, 2010; Taxman et al., 2007). This is about 3 times the number of offenders who are in prison (West & Sabol, 2010). Equally compelling is the estimate that nearly five million probationers and parolees have some type of substance use disorder, and nearly half of the probationers/parolees have supervision orders for drug treatment services (Taxman et al., 2007). Probationers and parolees have drug abuse disorders that are 4 times greater than the general population (Substance Abuse and Mental Health Services Administration, 2009), and 40–80% of the referrals to community substance abuse treatment services are from criminal justice agencies. More than one-third (35%) of state prison commitments are due to violations of parole or other conditional release (West & Sabol, 2010), mainly related to substance use and abuse. Stated simply, the largest concentration of substance abusers is found in the community corrections system, and they drive much of the recidivism and parole/probation violations. Research indicates that providing effective treatment services for offenders reduces recidivism, with estimates ranging from 10% to 30% depending on the type of treatment provided (Drake, Aos, & Miller, 2009; Mitchell et al., 2007; Taxman, Byrne, Pattavina, & Ainsworth, 2010; Wilson et al., 2006). The challenge is how best to provide effective treatment services in community correction settings. In response to the high numbers of offenders with substance abuse disorders, a number of treatment interventions have been implemented over the past decade in
6
1
Introduction
the criminal justice system, that have achieved reductions in recidivism; these include drug courts, treatment diversion, prison and jail treatment, and postincarceration aftercare (Belenko, 2002; Friedmann, Taxman, & Henderson, 2007; Knight, Simpson, & Hiller, 1999; Prendergast, Hall, Wexler, Melnick, & Cao, 2004; Taxman, 1998; Taxman et al., 2007; Wexler, De Leon, Thomas, Kressel, & Peters, 1999; Wexler, Melnick, Lowe, & Peters, 1999). However, such treatment has been able to serve relatively few offenders with drug problems (Belenko & Peugh, 2005; Bhati & Roman, 2010; Taxman et al., 2007). Only 24% of state and 8% of jail inmates reported receiving any treatment; including non-clinical interventions such as self-help groups or drug education (Belenko & Peugh, 2005). Among probationers, only 25% with histories of drug use receive treatment while on probation (Mumola & Bonczar, 1998). The percentage of parolees receiving clinically-based substance abuse treatment is unknown, but is likely to be low relative to need, given the low percentages of released inmates who engage in aftercare treatment following release from prison (Knight et al., 1999; Prendergast et al., 2004; Wexler et al., 1999). Despite the large number of drug courts (Huddleston, Marlowe, & Casebolt, 2008), most are small and have been estimated to serve well under 5% of the eligible population (Belenko, 2002; Belenko, Fabrikant, & Wolff, 2011; Bhati & Roman, 2010; Taxman et al., 2007). Given these high rates of substance abuse treatment needs but low rates of treatment access, it is likely that providing expanded access to evidenced-based interventions could contribute to public safety and to the reduction of the growing substance-involved population in the criminal justice system. As Petersilia (1999) has emphasized, achieving meaningful recidivism reduction is difficult without providing effective treatment and rehabilitation services. This is especially true in community corrections settings, where caseloads are high and substance-abuse related violations are common (Taxman et al., 2007), and research on the integration of treatment and community supervision has been lacking (Marlowe, Festinger, Lee, Dugosh, & Benasutti, 2006; Taxman, 1998; Taxman & Bouffard, 2000; Taxman & Thanner, 2004). The need to provide effective treatment services to offenders is clouded by consideration of the extent of support for the belief that addiction treatment services are appropriate and needed for substance abusing offenders. The concept of appropriate opens the door to personal opinions, political philosophies, societal values, and other factors that frame how those involved in the corrections system are handled. That is, even if corrections administrators supported the expansion of treatment for offenders, the external support from the criminal justice community (i.e., judges, prosecutors, defenders), treatment providers, and the community at large is required for substance abuse treatment to be part of the delivery system. Clarifying values must be part of any effort to advance the use of EBP in corrections settings. In many ways, it is the competing values that inhibit the adoption, implementation, and sustainability of EBP in community corrections settings. This is why an organizational approach that addresses both the internal (organization) and external (stakeholders) needs is crucial to the EBP debate.
1.3 Building Knowledge in Community Corrections
1.3
7
Building Knowledge in Community Corrections
The question of what works in corrections has captivated the attention of policymakers, practitioners, and researchers for nearly 50 years as the United States has struggled with finding the most appropriate strategies that were tough and punitive for punishing wrongdoers, held offenders accountable, and protected the community. The what works movement introduced in the late 1980s (Andrews et al., 1990), now subsumed under the EBP label, emphasizes the nature of correctional practices that should be in operation. By having science provide objective information about effective practices that change offender behavior, policymakers and practitioners can use that data-driven strategy to improve impacts and net economic benefits as a tool to frame informed decisions. The what works phenomenon offered this consensusbuilding approach to reduce the more reactive and crisis-driven decision making that has often characterized policy promulgation in the criminal justice system. Since the what works approach emerged, the methodology and techniques have evolved with the advent of statistical tools to facilitate meta-analyses or systematic reviews. Systematic reviews are now fairly common, including those supported by the Campbell Collaboration (www.campbellcollaboration.org), journals that are devoted to dissemination of experimental findings (i.e., Journal of Experimental Criminology), consensus and systematic reviews that are devoted to identifying best practices and EBPs, national repositories of information about evidence-based addiction treatment and prevention, and state legislation that requires the use of EBP to qualify for state funding. Complementary resources are available to the public that translates research into operational practice. The National Institute on Drug Abuse (2006) Principles of Drug Abuse Treatment for Criminal Justice Populations identifies 13 EBPs. The Substance Abuse and Mental Health Services Administration (SAMHSA) hosts the National Registry of Evidence-based Programs and Practices (NREPP) that identifies EBP in substance abuse and mental health prevention and treatment programs (http://www.nrepp.samhsa.gov/index.asp). Finally, the National Institute of Corrections has produced numerous publications on EBP (Crime & Justice Institute, 2004, 2010; Taxman, Shepardson, & Byrne, 2004). These resources illustrate how science and practice have merged for federal, state, and local governments to try to ensure that provided services are designed to achieve desired outcomes. The accumulation of knowledge in community corrections has paralleled the addiction treatment field – the stars have aligned in the identification of potential treatments that are applicable to people with addiction disorders, regardless of whether they are involved in the corrections system. A number of systematic reviews and meta-analyses have identified drug courts, in-prison therapeutic communities with aftercare, cognitive behavioral therapy, and other treatment approaches as likely to reduce relapse and recidivism. This is the good news. But the unanswered and necessary question is how to improve and expand their adoption and implementation of EBP in community corrections settings.
8
1.4
1
Introduction
The Focus on Technology Transfer
To a large extent, the efforts to date have focused on the dissemination and diffusion of knowledge about policies and practices that can improve offender outcomes. The diffusion process – the mechanism by which new or existing practices are communicated through certain channels within a social system and/or organization over time – has received less attention in the corrections field. Models of diffusion have been built primarily within the substance abuse field echoing the work of Everett Rogers (1983) to innovate, implement, and measure progress, and sustain the efforts. Different methods have been proposed with varying emphases, such as process improvements (Hoffman, Ford, Choi, Gustafson, & McCarty, 2008; McCarty, Gustafson, Wisdom, Ford, Choi, & Molfenter, 2007); organizational models of change (Addiction Technology Transfer Center, 2004; Klein & Sorra, 1996); addressing staff needs in technology transfer (Markus, 1983); and other organizational strategies such as fidelity and adherence models, value clarification, and leadership support. These models were built to be used within a specific discipline – primarily substance abuse – where the public health mission and emphasis on reducing drug abuse are clear, and the overall goals tend to be unified. In addiction treatment, the goals are to improve the health and well-being of a person. When considering TT across public health (addiction treatment) and public safety (community corrections) systems, a need exists to clarify mutual goals and to ensure that the stakeholders have similar views of these goals. Otherwise, these models are likely to fall short and need modification. Thus far, most dissemination efforts focus on the formative stages of defining EBP with the publication of systematic reviews or meta-analyses, listing evidencebased programs or best practices, providing manuals, or providing one-time training on EBP. These efforts are focused primarily on building knowledge by identifying EBP. Little attention is given to the more difficult issues related to dissemination of policies and practices, the feasibility of implementing key features of the EBP, sustainability (compromised by staff turnover, budget constraints, and staff skill levels), alignment with current policies and practices, and associated policy changes in real-world settings (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Rogers, 1995; Taxman et al., 2004). The National Institutes of Health (NIH) has highlighted the importance of moving clinical findings to the field and transferring research to practice by emphasizing translational research in its Roadmap for Medical Research (NIH, 2006). The NIH has also introduced a policy to make research findings more available in the public domain through open access to scientific journal articles. An emphasis on EBP utilization would require significant attention to the challenges of innovation diffusion (Panzano & Roth, 2006; Patton, 1987) that ranges from developing organizational and staff capacity, disseminating EBP, translating evidence into operational components, retraining staff in key skills needed to conduct the intervention, enhancing organizational leadership at all levels to sustain change, monitoring fidelity and performance, and establishing new business practices that comport with the translation of research. As stated by Rogers in his seminal work on diffusion of innovations (Rogers, 1983, 1995, 2003), the implementation
1.4
The Focus on Technology Transfer
9
of a new practice(s) requires ensuring that the innovation fits within existing business processes. There is a need to synthesize the scientific knowledge in the context of practice and policy to move toward sustainability in real-world settings. Researchers, practitioners, and policymakers must be cognizant of the importance of sustainability, ongoing intervention fidelity (i.e., “ecological validity”), and transportability to different settings and populations. Consistent with the mission of public safety and accountability, community corrections agencies must balance the priorities of supervision with expanding referrals to evidence-based addiction treatment. However, identifying and using evidencebased addiction treatment is not typically a community corrections priority given that substance abuse is generally viewed as a public health issue. Community corrections agencies generally benefit from efforts in the public health system to identify effective treatments. Alignment of goals and missions between the community corrections and public health systems has progressed very slowly, and is often haphazard rather than deliberate. Community corrections administrators may also believe that the selection of a treatment program should be the decision of the treatment agency(s). In summary, the challenges of identifying EBP treatments that address offenders’ substance abuse and other criminogenic needs (i.e., antisocial values and peers, impulsivity and decision-making), fall between the cracks of the two systems. Attempting to bridge the chasm between the systems results in greater challenges in building organizational capacity to implement and sustain EBP and provide offenders under community supervision with evidence-based addiction treatment. As the value attached to EBP increases, the need to encourage an active dialogue among researchers, policymakers, practitioners, and clients becomes more important to the TT process. To date, the articulation of EBP has been primarily a top-down process, more aimed at meeting scientific concerns than clinical applications. Prior efforts to identify and disseminate EBP in criminal justice addiction treatment (such as NIDA’s Principles of Drug Abuse Treatment for Criminal Justice Populations – NIDA, 2006) have generally lacked a balanced perspective that integrates criminal justice practitioners’ perspectives with the views of researchers, other practitioners, policymakers, and clients. And, identification and dissemination of EBP in community corrections (such as the National Institute of Corrections/ Crime and Justice Institute’s model (2004)) has generally lacked the involvement of scientists and researchers in the process. Scant attention has been given to organizational and other barriers to implementation and sustainability (Brown & Flynn, 2002; Fixsen et al., 2005; Roman & Johnson, 2002; Simpson, 2002). Careful exploration is needed of the TT process for evidence-based treatment of offenders under community supervision, and the identification of potential new approaches to increasing TT. The translation of existing dissemination and diffusion models to addiction treatment in community corrections, with the conflicting and contradictory goals of public safety and health, has not been adequately considered. This book addresses the diffusion and implementation of evidence-based community corrections and addiction treatment and practices within the community corrections environment by presenting
10
1
Introduction
a framework that reflects the unique characteristics and needs of both systems. In considering the limitations of simply having scientists designate EBP and assume that these programs and practices will be widely adopted while ignoring the steep uphill climb of implementation, this book outlines processes for defining EBP and for importing innovations into this setting. The Evidence-Based Interagency Implementation Model (EB-IIM) serves to outline a conceptual framework for fostering organizational change for expanding effective addiction treatment in community corrections settings.
1.5
Community Corrections Presents Unique Challenges for Addiction Treatment Interventions
The EB-IIM evolved from our review of organizational change processes and models in the corrections and substance abuse fields, as well as the general literature on adoption and implementation of new business practices. These efforts included reviews of : (1) the literature on EBP in community corrections and addiction treatment, including the criteria used to define evidence; (2) dissemination, diffusion, and organizational change research across disciplines; (3) studies on organizational factors that affect the quality of implementation of innovations in addiction treatment and correctional agencies; (4) survey findings on addiction treatment and corrections practices; (5) the lessons learned from the National Institute of Corrections Evidence-Based Practices project (Crime & Justice Institute, 2010) and other efforts at implementation of EBPs; and (6) interviews with key informants in the field regarding the implementation process. Taken together, this research indicated to us that an effective and useful TT model must address the unique facets related to the intersection of the addiction treatment and community corrections settings. This intersection is of much less concern in other disciplines. Stated simply, the interagency nature of the problem is not typically addressed in existing change processes where the missions are clearly aligned and participating organizations have a clearly defined role in delivering services. Neither of these two conditions exists when the community corrections and addiction treatment systems intersect; therefore a TT model must address the following issues: 1. New EBP need to be aligned into existing processes or procedures, which will require either adaptations of EBP or modifications in existing processes (i.e., the quality improvement model; see Ford, Green, Hoffman, Wisdom, Riley, & Bergmann, 2007). Often the introduction of an EBP will require an assessment of existing processes with an eye toward modifications, reductions in duplication, or clarity of importance assigned to these processes. This process is needed to overcome organizational barriers to innovation and changes in work practices. 2. Staff working in corrections agencies (i.e., managers, supervisors, and line staff) requires both the development of knowledge and the skills to use these practices. More attention needs to be given to an organizational strategy to engage the
1.6 Multistage Conceptual Model for Identifying and Selecting EBPs
11
organization as a whole in the change process. Work requirements in corrections settings generally mean that staff do not have the basic competencies in interviewing and interaction skills that are present in other disciplines (e.g., substance abuse, mental health), and that are needed to support rehabilitation efforts and offender change. The need exists to give more attention to the development of basic and advanced skills in EBP areas. 3. Improvements in feedback loops are critical to “incentivize” the system, and to provide the organization and stakeholders with information about the advances made and to fuel the momentum to change. Feedback loops can include management information systems, benchmarks, performance monitoring, testimonials, or public media messages to address the social networks associated with diffusion, and to ensure that the feedback builds momentum. An important component in this model is to identify performance objectives that can help the agencies monitor their progress. Within the community corrections setting, attention needs to be given to internal (i.e., staff, supervisors, other agencies) and external (i.e., judges, prosecutors, defenders, legislators, public health officials) stakeholders that are interested in improving outcomes.
1.6
Multistage Conceptual Model for Identifying and Selecting EBPs
Given the complexities of implementing evidence-based addiction treatment in community corrections settings, it should not be surprising that current models for designating EBP often fall short of providing a road map for selecting EBP that will both fit into local settings and will have the same positive impacts found in research settings. Although the scientific basis for designating an intervention/practice as EBP requires determination of effectiveness through multiple replications, few if any EBPs have been tested in a broad array of settings and populations. Very few addiction treatment EBPs have been tested with community corrections populations. In addition, the evidence base can sometimes have limitations due to concerns with selection bias, statistical vs. clinical significance, feasibility in real world settings, and applicability to a broader population. Aside from a careful review of the evidence base, two other issues need to be considered when agencies make decisions to adopt and implement an EBP. The first concerns whether the EBP is transportable to the local setting (Schoenwald & Hoagwood, 2001) – are the setting and population similar to those used in the research studies? Can the EBP be implemented with fidelity? Does the target population resemble the population that was studied? The second issue is the organizational capacity to implement the particular EBP. Are managers and staff ready to embrace this EBP and change their business practices to accommodate the innovation? Organizational capacity includes such factors as whether the staff perceives value in the EBP, resources and processes exist to implement the intervention, external influences are supportive, the costs are reasonable, and the organization is ready,
12
1
Introduction
willing, and able to incorporate the EBP into the existing system. If the local conditions are not ripe for the implementation of EBP, it is not likely to be implemented well and achieve the desired outcomes. The decisive question is not just whether the local conditions are already present but rather can they be developed through use of sound organizational change strategies. We developed a multistage model that considers these issues and provides a framework for identifying EBP, critically assessing the evidence base, assessing transportability, and assessing and improving organizational capacity for EBP. By examining these issues during the phases of adopting, implementing, and sustaining an EBP, organizations can be more confident that the intervention or practice will be implemented in a manner that is likely to achieve the desired outcomes.
1.7
Evidence-Based Interagency Implementation Model (EB-IIM)
Our EB-IIM evolved from the existing literature and the unique features of correctional and addiction treatment agencies. It provides a framework for managing the change process through six actions: (1) knowledge development; (2) foundation building; (3) expectation setting; (4) alignment; (5) renovation; and (6) sustainability. Although the model is presented in a linear fashion, as shown in Fig. 1.1, it is recognized that it is not necessarily linear and organizations may have different starting places for different EBP areas. For example, an organization that contracts with a well-established substance abuse treatment provider that already delivers evidencebased interventions (e.g., cognitive behavioral therapy, family-focused interventions) with dedicated treatment slots for parolees, or one that has been using a second generation risk tool (i.e., Wisconsin Risk and Need), may need less work in knowledge development, given their prior efforts. And, this model underscores the processes of change that involve active leadership, staff involvement in change processes, staff with common knowledge in a given area, staff engagement in facilitated meetings and workshops to develop strategic and implementation plans,
Fig. 1.1 Conceptual model for evidence-based interagency implementation
1.7 Evidence-Based Interagency Implementation Model (EB-IIM)
13
project management that is clearly defined with authority and timeframes, working teams at multiple levels, cross-systems organizational collaborations, and clear performance goals and measures. The EB-IIM also recognizes an important lesson from the literature that effective TT must engage both the agency (inner) and stakeholder (outer) settings – the inner refers to the corrections agency whereas the outer refers to the layers of stakeholders and interested parties that influence the corrections agency. The TT process directly affects the community corrections agency but partners shape correctional initiatives including long-term sustainable change. Unlike mission-specific (public health) TT models, the EB-IIM preparation phase is separated into two components that include knowledge building and foundation setting. Knowledge building is designed to familiarize the organization with the EBP(s), including the nature of the scientific studies, the type of intervention, the measures used, and the key components of the practices. The goal is to build an appreciation for knowledge and a better understanding of EBP. The development of the skills of the individual and the organization to implement or support EBP is an important ingredient in this model. In some cases, the community corrections agencies will implement the innovation; in other cases, they will support a treatment agency. New skills such as motivational interviewing, use of geographical information systems, or other technologies may be important for the agency. Similarly, the organization will need to review its work processes and procedures and refine them to integrate the EBP. The third phase is expectation setting. Each new practice or innovation is coupled with a rationale and logic about the improvements that will occur. Expectation setting is designed to allow internal and external agencies to establish desired goals that are grounded in realistic benchmarks. This phase addresses the problem of unrealistic expectations such as organizations that promote EBP and expect recidivism reductions in the range of 50%. It is important to take modest, small steps in building the resiliency of the organization to implement EBP. Alignment is similar to a pilot that adjusts the agency’s processes and procedures to accommodate the change, and the impact of the changes on the offenders. The pilot or phase one introduction allows for all parties to work toward a common goal, and to address the barriers to full implementation of EBP. Both internal and external stakeholders should be involved in aligning the policies and procedures to create a renewed focus on desirable, and achievable, outcomes. Finally, renovation and sustainability are critical to lasting practices. This model recognizes that community corrections agencies must receive support from external agencies to implement EBP and treatments, and to develop internal capacity for change. Sustaining the innovation over time requires steadfast belief that the goals and objectives are worthy of continued investment. The initial evidence will become a launch pad for adopting and implementing other EBP. Refinement is a part of sustainability. The goal is to refine the practice to fit within the organization’s business process. Refinement focuses mainly on integration into the fabric of the organization, as well as system processes. The emphasis on sustainability throughout the process is focused on ensuring that innovation does
14
1
Introduction
not get diluted or morphed into another process. Refinement allows the agency to routinize the practice so that it is in place for the long term. The unique aspect of this model is that it outlines the inter- and intra-agency efforts that are important to adoption of EBP in this setting: clarity of mission and goals, interagency work groups, performance measures, social media campaign, and resources. These are important glues to bind the processes together. Based on the unique features of interagency efforts to adopt, implement, and sustain EBP, the major activities need to focus on the binding components to solidify support for EBP.
1.8
Conclusions and Outline of the Book
This book was motivated by a desire to provide new models to facilitate and encourage adoption, implementation, and sustaining the use of evidence-based addiction treatment in community corrections settings. The goal is to advance efforts to improve organizational strategies to implement sound, evidence-based addiction treatment and related programming. Ultimately, our interest is in improving the outcomes from the justice system, particularly substance-involved offenders. The model recognizes that addiction treatment providers and community corrections agencies are very different, and that there is a need for different implementation dynamics that must be addressed to facilitate change. These include: • The complexity of the community corrections field must be addressed in any implementation model. • The mission and goals that encompass EBP must be clarified to demonstrate cohesion with public safety and public health, and to demonstrate how offender change (abstinence from drug use as a result of successful treatment) improves public safety. • As a whole, basic skills that involve the dynamics of human services (e.g., assessment, communication/interviewing, engagement, treatment placement) are needed within corrections and treatment environments, and it cannot be assumed that these are currently available. • Performance measures provide wholesale support to the organization regarding the change model by ensuring that managers and staff link the change to the goals of the organization and continue the momentum associated with the change. • Community corrections agencies and treatment providers must be cross-trained and co-trained in order to more closely align their goals and expectations for offender management and progress, and to enable more informed choices about treatment placement and responses to offender progress in treatment. The value of this approach will be determined when community corrections agencies begin to systematically test the model as part of their strategies to implement EBP for addiction treatment.
References
15
This book contains nine additional chapters that analyze the issues raised in this introduction. Chapter 2 summarizes the techniques to identify EBP and discusses the strengths and weaknesses of these various approaches. Chapter 3 focuses on the theories of organizational change, and the major approaches in public sector organizations. Following the theories of change, Chapter 4 is devoted to distilling the research on organizational factors that impact the successful adoption of innovations. These include staff and managerial issues, characteristics of the innovation, characteristics of the organization, stakeholder involvement, and policy-related matters. Together, this set of chapters summarizes the importance of the organization in the discussion of using EBP in real world settings. Chapters 5 and 6 provide detailed updates on the state of EBP in addiction treatment and community corrections. The goal of these chapters is to detail advances in each field related to identifying, adopting, and implementing EBP. Chapter 7 then provides a summary of the unique and challenging environment issues that affect the adoption of innovations in addiction treatment and community corrections agencies. Chapter 8 provides a model for identifying the scientific, transportability, and organizational capacity issues that affect the identification and suitability of mapping evidence-based research into practice settings. Recognizing that the scientific basis for designating EBP has some limitations, and organizational, staff, and client factors affect the fit of an EBP into real world settings, this model moves the field further by incorporating scientific robustness, transportability, relevance, and capacity into the discussion. Chapter 9 presents an interagency evidence-based implementation model designed to address the needs of community correction and addiction treatment agencies. We present a six step process as well as managerial, facilitation, and policy level factors that affect the use of EBP. The chapter presents the heuristic implementation model as well as important planned change strategies to address implementation issues. Chapter 10 lays out an implementation and research agenda. This is a budding field that will proliferate in the future with proper nurturing, guidance, and assistance. Our goal is to identify the next steps needed to foster more attention to the importance of implementation strategies and implementation science in improving the integration of or increasing the utilization of effective addiction treatment programs by community corrections agencies.
References Addiction Technology Transfer Center (ATTC). (2004). The change book (2nd ed.). Kansas City: ATTC National Office. Andrews, D. A., Zinger, I., Hoge, R. D., & Bonta, J. (1990). Does correctional treatment work – A clinically relevant and psychologically informed meta-analysis. Criminology, 28, 369–390. Belenko, S. (2002). The challenges of conducting research in drug treatment court settings. Substance Use & Misuse, 37(12–13), 1635–1664.
16
1
Introduction
Belenko, S., Fabrikant, N., & Wolff, N. (2011). The long road to treatment: Models of screening and admission into drug courts. Criminal Justice and Behavior, 38, 27–48. Belenko, S., Lang, M., & O’Connor, L. (2003). Self-reported psychiatric treatment needs among felony drug offenders. Journal of Contemporary Criminal Justice, 19(1), 9–29. Belenko, S., & Peugh, J. (2005). Estimating drug treatment needs among state prison inmates. Drug and Alcohol Dependence, 77(3), 269–281. Bhati, A. S., & Roman, J. K. (2010). Simulated evidence on the prospects of treating more druginvolved offenders. Journal of Experimental Criminology, 6(1), 1–33. Brown, B. S., & Flynn, P. M. (2002). The federal role in drug abuse technology transfer: A history and perspective. Journal of Substance Abuse Treatment, 22, 245–257. Centers for Disease Control and Prevention. (2005). Hospitalization by first diagnosis, all ages: U.S., 2002–2004. Atlanta: U.S. Department of Health and Human Services, CDC. Centers for Disease Control and Prevention. (2006). Sexually transmitted disease surveillance 2005 supplement. Atlanta: U.S. Department of Health and Human Services, CDC. Crime & Justice Institute. (2004). Implementing evidence-based practice in community corrections: The principles of effective intervention. Washington: National Institute of Corrections, Community Corrections Division. Crime & Justice Institute. (2010). Putting the pieces together: Practical strategies for implementing evidence-based practices. Washington: National Institute of Corrections. Drake, E., Aos, S., & Miller, M. (2009). Evidence-based public policy options to reduce crime and criminal justice costs: Implications in Washington State. Victims & Offenders, 4(2), 170–196. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Ford, J. H., II, Green, C. A., Hoffman, K. A., Wisdom, J. P., Riley, K. J., & Bergmann, L., (2007). Process improvement needs in substance abuse treatment: Admissions walk-through results. Journal of Substance Abuse Treatment, 33(4), 379–389. Friedmann, P. D., Taxman, F. S., & Henderson, C. E. (2007). Evidence-based treatment practices for drug-involved adults in the criminal justice system. Journal of Substance Abuse Treatment, 32, 267–277. Gendreau, P., Goggin, C., & Smith, P. (1999). The forgotten issue in effective correctional treatment: Program implementation. International Journal of Offender Therapy and Comparative Criminology, 43(2), 180–187. Glaze, L. E. (2010). Correctional populations in the United States, 2009. Washington: Bureau of Justice Statistics. Glynn, M., & Rhodes, P. (2005). Estimated HIV prevalence in the United States at the end of 2003. In National HIV Prevention Conference, Atlanta, June 2005 (Abstract T1-B1101). Accessed July 21, 2007. Hoffman, K. A., Ford, J. H., II, Choi, D., Gustafson, D. H., & McCarty, D. (2008). Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment. Drug and Alcohol Dependence, 98(1–2), 63–69. Huddleston, C. W., Marlowe, D. B., & Casebolt, R. (2008). Painting the current picture: A national report card on drug courts and other problem-solving court programs in the United States. Alexandria: National Drug Court Institute. James, D. J., & Glaze, L. E. (2006). Mental health problems of prisons and jail inmates (NCJ 213600). Washington: Department of Justice, Bureau of Justice Statistics. Klein, K. J., & Sorra, J. S. (1996). The challenge of innovation implementation. Academy of Management Review, 21, 1055–1080. Knight, K., Simpson, D. D., & Hiller, M. L. (1999). Three-year reincarceration outcomes for inprison therapeutic community treatment in Texas. The Prison Journal, 79(3), 337–351. Kuhn, T. S. (1966). The structure of scientific revolutions (3rd ed.). Chicago: University of Chicago Press.
References
17
Langan, P. A., & Levin, D. J. (2002). Recidivism of prisoners released in 1994. Federal Sentencing Reporter, 15(1), 58–65. MacKenzie, D. L. (2006). What works in corrections? Reducing the criminal activities of offenders and delinquents. Cambridge: Cambridge Press. Markus, M. L. (1983). Power, politics, and MIS implementation. Communications of the ACM, 26, 430–444. Marlowe, D. B., Festinger, D. S., Lee, P. A., Dugosh, K. L., & Benasutti, K. M. (2006). Matching judicial supervision to clients’ risk status in drug court. Crime & Delinquency, 52(1), 52–76. Martinson, R. (1974). What works? – Questions and answers about prison reform. The Public Interest, 35, 22–54. Martinson, R. (1979). New findings, new views: A note of caution regarding sentencing reform. Hofstra Law Review, 7(Winter), 243–258. Maruschak, L. (2009). HIV in prisons, 2007–08 (NCJ 228307). Washington: U.S. Department of Justice, Bureau of Justice Statistics. Mauer, M., & King, R. S. (2007). A 25-year quagmire: The war on drugs and its impact on American society. Washington: The Sentencing Project. McCarty, D., Gustafson, D., Wisdom, J., Ford, J., Choi, D., & Molfenter, T. (2007). The Network for the Improvement of Addiction Treatment (NIATx): Enhancing access and retention. Drug and Alcohol Dependence, 88, 138–145. Mitchell, O., Wilson, D. B., & MacKenzie, D. L. (2007). Does incarceration-based drug treatment reduce recidivism? A meta-analytic synthesis of the research. Journal of Experimental Criminology, 3(4), 353–375. Mumola, C. J., & Bonczar, T. P. (1998). Substance abuse and treatment of adults on probation, 1995. Rockville: Bureau of Justice Statistics. Nagin, D. S., Cullen, F. T., & Jonson, C. L. (2009). Imprisonment and reoffending. In M. Tonry (Ed.), Crime and justice: A review of research (pp. 1–91). Chicago: University of Chicago Press. National Institute of Mental Health. (2008). Statistics. Retrieved March 18, 2008, from http:// www.nimh.nih.gov/health/topics/statistics/index.shtml. National Institute on Drug Abuse. (2006). Principles of drug abuse treatment for criminal justice populations. (NIH Publication No. 06–5316). Rockville: National Institute on Drug Abuse. National Institutes of Health. (2006). Fact sheet: NIH roadmap for medical research. Bethesda: National Institutes of Health. Panzano, P. C., & Roth, D. (2006). The decision to adopt evidence-based and other innovative mental health practices: Risky business? Psychiatric Services, 57(8), 1153–1161. Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park: Sage. Petersilia, J. (1999). Parole and prisoner reentry in the United States. Crime and Justice, 26, 479–529. Pew Center on the States (2008). One in 100: Behind bars in America. Washington, DC: The Pew Charitable Trusts. Retrieved April 20, 2011, from http://www.pewcenteronthestates.org/ initiatives_detail.aspx?initiativeID=56212#2011. Pew Center on the States (2009). One in 31: The long reach of American corrections. Washington, DC: The Pew Charitable Trusts. Retrieved April 20, 2011, from http://www.pewcenteronthestates.org/initiatives_detail.aspx?initiativeID=56212#2011. Prendergast, M. L., Hall, E. A., Wexler, H. K., Melnick, G., & Cao, Y. (2004). Amity prison-based therapeutic community: 5-year outcomes. The Prison Journal, 84(1), 36–60. Proctor, E., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34. Rogers, E. M. (1983). Diffusion of innovations (3rd ed.). New York: The Free Press. Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: The Free Press. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: The Free Press.
18
1
Introduction
Roman, P. M., & Johnson, J. A. (2002). Adoption and implementation of new technologies in substance abuse treatment. Journal of Substance Abuse Treatment, 22(4), 211–218. Schoenwald, K. S., & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services, 52, 1190–1197. Sherman, L. W., Gottfredson, D., MacKenzie, D., Reuter, P., Eck, J., Bushway, S. (1997). Preventing crime: What works, what doesn’t, what’s promising. A Report to the U.S. Congress. Washington: U.S. Department of Justice. Retrieved from http://www.ncjrs.gov/works/. Simpson, D. D. (2002). A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment, 22, 171–182. Substance Abuse and Mental Health Services Administration. (2009). Treatment Episode Data Set (TEDS) Highlights – 2007 National Admissions to Substance Abuse Treatment Services. OAS Series #S-45, HHS Publication No. (SMA) 09–4360, Rockville. Taxman, F. S. (1998). Reducing recidivism through a seamless system of care: Components of effective treatment, supervision, and transition services in the community. Washington: Office of National Drug Control Policy. Taxman, F. S., & Bouffard, J. A. (2000). The importance of systems issues in improving offender outcomes: Critical elements of treatment integrity. Justice Research and Policy, 2, 9–30. Taxman, F. S., Byrne, J., Pattavina, A., & Ainsworth, S. (2010). Analysis of criminogenic needs of offenders. Fairfax: Center for Advancing Correctional Excellence. Taxman, F. S., Perdoni, M., & Harrison, L. D. (2007). Drug treatment services for adult offenders: The state of the state. Journal of Substance Abuse Treatment, 32, 239–254. Taxman, F. S., Shepardson, E., & Byrne, J. (2004). Tools of the trade: A guide for incorporating science into practice. Washington: National Institute of Corrections. Taxman, F. S., & Thanner, M. H. (2004). Probation from a therapeutic perspective: results from the field. Contemporary Issues in Law, 7(1), 39–63. Teplin, L. A., Abram, K. M., McClelland, G. M., Mericle, A. A., Dulcan, M. K., & Washburn, J. J. (2006). Psychiatric disorders of youth in detention. U.S. Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention. (NCJ-213600). Tonry, M., & Farrington, D. P. (1995). Strategic approaches to crime prevention. Crime and Justice, 19, 1–20. West, H. C., & Sabol, W. J. (2010). Prisoners in 2009. Washington: Bureau of Justice Statistics. Wexler, H. K., De Leon, G., Thomas, G., Kressel, D., & Peters, J. (1999). The amity prison TC evaluation: Reincarceration outcomes. Criminal Justice and Behavior, 26(2), 147–167. Wexler, H. K., Melnick, G., Lowe, L., & Peters, J. (1999). Three-year reincarceration outcomes for Amity In-Prison Therapeutic Community and Aftercare in California. The Prison Journal, 79(3), 321–336. White, H. R., & Gorman, D. M. (2000). Dynamics of the drug-crime relationship. In Crime and Justice 2000, vol. 1: The nature of crime: Continuity and change (NCJ 182408, pp. 151–218). Washington: U.S. Department of Justice, National Institute of Justice. Wilson, D. B., Mitchell, O., & MacKenzie, D. L. (2006). A systematic review of drug court effects on recidivism. Journal of Experimental Criminology, 2(4), 459–487.
Chapter 2
Identifying the Evidence Base for “What Works” in Community Corrections and Addiction Treatment
2.1
Introduction and Overview
The evidence-based practices (EBP) movement emerged in the 1990s as a strategy to reduce the gap between science and practice in many disciplines that involve the delivery of services: medicine, education, social services, substance abuse, mental health, and criminal justice. In the past, standards varied regarding the definitions of “best practices” and what might constitute an EBP; these were often defined in an idiosyncratic manner. The more recent interest in EBP focuses on improving outcomes by ensuring that direct service providers use proven techniques and technologies (defined by the results from scientific studies) in their daily practices. Professional organizations and federal agencies have been actively promoting the identification and implementation of EBP in health care, mental health (National Advisory Mental Health Council, 2006), substance abuse (NIH, 2004), community corrections (NIC, 2004), and other areas. This interest has spurred the creation of numerous initiatives to disseminate scientific knowledge through comprehensive literature reviews that synthesize and quantify the results using recognized metaanalysis and systematic review procedures. Synthesized findings are disseminated to the field via information sessions or websites, practitioner training, and publications. Prominent examples include the Office of Juvenile Justice Prevention and Delinquency (OJJPD) Blueprints for Violence Prevention, Substance Abuse and Mental Health Services Administration’s (SAMHSA) National Registry of Evidence-Based Programs and Practices (NREPP) (http://www. nrepp.samhsa.gov/AboutNREPP.aspx), Cochrane Reviews (http://www.cochrane. org), and Campbell Collaboration (http://www.cochranecampbellcollaboration.org). These efforts focus primarily on methods for reviewing the literature, establishing criteria for labeling a treatment or practice as evidence-based, identifying and rating interventions, and producing summary papers that systematically review research findings. An example of efforts to simply translate findings and then disseminate to the field is the National Institute of Corrections Evidence-Based Practices for Community Corrections (http://cjinstitute.org/projects/integratedmodel). F.S. Taxman and S. Belenko, Implementing Evidence-Based Practices in Community Corrections and Addiction Treatment, Springer Series on Evidence-Based Crime Policy, DOI 10.1007/978-1-4614-0412-5_2, © Springer Science+Business Media, LLC 2012
19
20
2
Identifying the Evidence Base for “What Works” in Community Corrections…
The term “evidence-based practice” has many definitions but generally requires a thorough review of the research for a given intervention or practice to identify studies that found positive outcomes in real-world settings. The preference is for scientifically rigorous studies using randomized controlled trial (RCT) designs or high quality quasi-experimental designs. The accepted standard of an EBP is that there must be at least two rigorous studies (i.e., randomized designs or high quality quasi-experimental designs) with similar findings on key outcomes. Two examples of EBP definitions are as follows: • “Evidence-based practice is the integration of best research evidence with clinical expertise and patient values… Patient values refers to the unique preferences, concerns, and expectations that each patient brings to the clinical encounter” (Institute of Medicine, 2001). • “Evidence-based practices are interventions for which there is consistent scientific evidence showing that they improve client outcomes” (Drake et al., 2001). Unlike the fields of medicine and other health care professions, identifying EBP in correctional practice and behavioral health (i.e., substance abuse, mental health) is a much more complex undertaking given that the findings are subject to more debate and controversy. This is because the degree of improvement in symptoms can be subjective as to whether an intervention or practice is “effective,” whether the positive effects are clinically meaningful, and whether the findings are statistically significant. For example, few studies are longitudinal in nature, and the findings often reflect short-term outcomes of 12 months or less. This raises concerns about the significance of the study findings given the limited information on duration of effect, with some arguing that progress in 12 months (or less) is significant while others find this timeframe too limited to make a judgment about effectiveness. In addition, one must consider that many treatment counselors and criminal justice practitioners are highly invested personally and professionally in delivering services or using clinical techniques with which they are familiar and comfortable; the criteria that the counselors or staff use may differ from the research findings. A core challenge for the EBP field involves the practicalities of conducting field research in behavioral health, substance abuse, and correctional interventions/ programs. The demarcation of a practice or treatment as an EBP is a lengthy and sometimes cumbersome process due to the difficulty and expense of conducting rigorous well-controlled scientific studies (primarily using randomized controlled trials) that have sufficient follow-up periods to detect differences in client outcomes. There is also considerable debate whether mental illnesses and substance abuse can be eradicated or whether reductions in symptoms are sufficient for a treatment to be deemed effective. The same is true for criminal conduct. Long-term abstinence for drug abusers can be quite difficult to achieve (McKay, 2001). For correctional interventions, the problems are even more exacerbated because justicerelated funding agencies at the federal (e.g., U.S. Department of Justice) and state levels generally do not provide funding for studies that exceed a few years. Criminal justice stakeholders are often reluctant to approve RCTs with offender populations because of concerns about due process, public safety, and interference with judicial, correctional, or prosecutorial authority. That is, in justice settings there is
2.1
Introduction and Overview
21
much concern that randomized trials sacrifice the nuances of decisions made regarding the delivery of services and programs in correctional settings, and that correctional staff should not be bound to a RCT design, especially when that would constrain criminal justice decision making or when offender behavior may jeopardize public safety. The emphasis on RCTs, complicated by the realities of ensuring public safety, increases the complexity of conducting such studies in community corrections settings. RCTs and long-term follow-up studies are more common in addiction treatment research. Typical grants funded by the National Institutes of Health, particularly the National Institute on Drug Abuse and the National Institute of Mental Health, are five years in duration whereas funding from the National Institute of Justice tends to be under three years. Even with fewer barriers to conducting rigorous studies in addiction treatment research, the addiction treatment field struggles with relying totally on a scientific basis for recommending particular treatments or therapies. The addiction treatment profession still has a strong and influential organizational culture that values individual clinician experience and viewpoints in determining what type of treatment might work best for different types of clients in different treatment settings (Capoccia et al., 2007). The field is caught in a “Catch 22” given that clinical trials may reveal new therapies but the tendency is for the addiction treatment field to greatly value clinical experience and judgment over science-based research findings (Norcross et al., 2005). Counseling staff may be reluctant to adopt and implement an EBP that contradicts or interferes with the type of counseling or services they were trained on or have been delivering; this can often result in drift from the EBP-defined intervention and poor implementation of an intervention that in turn reduces the effectiveness of the EBP. Counselors in recovery may be motivated to use techniques that helped them overcome their substance abuse problems, regardless of whether those techniques have been designated as EBP. The question of whether clinician or patient input is necessary to designate an intervention or practice as evidence-based is controversial, as discussed in detail below. Another frequently mentioned concern is whether the strategies that researchers use in RCTs compromise some of the “real-world” conditions that can affect the delivery of EBP. These issues are unresolved but exemplify a significant tension that affects adoption of evidence-based practices in the field. For this reason, clinical researchers have proffered the need to conduct efficacy trials (using RCTs) to demonstrate the ability of a practice or treatment to improve outcomes followed by an effectiveness trial to demonstrate that the practice delivers similar outcomes in real-world settings (see below for more discussion). A similar tension exists in correctional and criminal justice settings where practitioners feel that their expertise and professional judgment are not adequately integrated into RCT or high quality quasi-experimental research designs. Regardless of strains between science-based identification of “evidence” and clinical practice, the correctional and substance abuse fields have defined a set of principles for identifying evidence-based practices and have designated some practices as EBP. This has allowed both fields to create taxonomies to designate practice or treatments according to the degree of scientific rigor. In this regard, clinical practice refers to accepted or consensus guidelines by the disciplines and best practices
22
2
Identifying the Evidence Base for “What Works” in Community Corrections…
refers to practices that are well-respected by the field(s) and where some research has been conducted to affirm efficacy and effectiveness.
2.2
Basic Definitions and Concepts
Technology Transfer (TT) is about taking the findings from science (the laboratory) and applying them in real-world settings in a way that leads to meaningful change in practices and treatments provided. Knowledge development and knowledge utilization are processes embedded within TT. TT helps process the science (findings from studies) and churns it through organizational mechanisms to become reality, while maintaining the integrity of the originally defined intervention or practice. Many factors influence the TT process (e.g., sociopolitical environments, leadership, staffing, severity of the crime problem in the community, interagency efforts, historical efforts, resources) in ways that shape the resulting product, practice, or intervention. It is necessary to fully understand how the organizational, interagency, and personnel processes affect the outcomes both in terms of the nature of the intervention/practice and the outcomes at the organizational and client level. A new field of study, implementation science, is geared toward providing a scientific process to understand how to maximize implementation to achieve adherence to the science-based interventions/programs and to better understand the components of effective implementation processes (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Proctor et al., 2009). Because many interventions enter into clinical or correctional practice and become well accepted over time without rigorous scientific evidence to support the intervention, TT must address the issue of compatibility of EBP with existing practices. Although this reflects that many clinicians and practitioners greatly value their own observations and experiences in assessing whether an intervention works, it also reflects the reinforcement that the treatment and justice/corrections staff receive from their supervisors, funding agencies or the public on the services they provide. However, there is scientific consensus that observation, anecdotes, and personal experiences, although important, furnish relatively low levels of evidence to support a determination that an intervention or practice is effective because such techniques rely on subjective and nonreproducible assessments of impacts. Figure 2.1 below shows the various levels of evidence ranked from lowest to highest scientific strength.
2.2.1
Hierarchy of Levels of Evidence
In addition to the variety of research designs (e.g., quasi-experimental and randomized experiments), there are different levels of evidentiary strength for drawing conclusions about research findings on interventions and practices. The highest standard, gold, has traditionally required randomized trials (preferably theory-driven)
2.2 Basic Definitions and Concepts
23
Multivariate Longitudinal Research Replications and Multisite Randomized Controlled Trials Single Site Experiment
Quasi-Experiments
Process Evaluations Qualitative information including focus groups, expert panels, key informant activities
Participant and Program Staff Observations, anecdotes
Fig. 2.1 Gradations of scientific methods and approaches
with multiple replications in different sites, controls for sample attrition, significant and sustained reductions in risk behaviors, and a preponderance of evidence supporting effectiveness across multiple studies. A less rigorous standard, silver, would include the same outcomes and replications as the gold standard but using a quasi-experimental design with strong statistical controls. Quasi-experimental designs include case control, statistically matched samples, regression discontinuity, time series, and single sample pre-post designs with or without longitudinal observations (Campbell & Stanley, 1963). A bronze standard would use matched comparison groups but without adequate statistical controls, and the lowest standard would entail inadequate research designs. The rigor of the studies signifies the degree to which the findings are subject to error or bias due to the methods or the variables, and to the level of internal validity. Despite the broad acceptance by research funders that RCTs are the gold standard for determining the effectiveness of interventions, some theorists and statisticians have raised warnings about the limitations of RCTs (Brown et al., 2009; Manski, 2011; Sampson, 2010). These concerns include the difficulty of drawing causal inferences about intervention effects from many RCTs (Sampson, 2010), the commonality of selection bias in experimental studies (Belenko, Fabrikant, & Wolff, 2011; Berk, 2005; Sampson, 2010), the lack of research on the effective components (or “active ingredients”) of interventions (Taxman & Thanner, 2006), concerns about small and homogeneous samples in the typical RCTs (Taxman & Rhodes, 2010),
24
2
Identifying the Evidence Base for “What Works” in Community Corrections…
sample attrition problems and lack of information on the fidelity of the intervention (Taxman & Friedmann, 2009), sample contamination between experimental and control groups (Taxman & Rhodes, 2010), follow-up periods that are too brief to determine long-term intervention effects (Taxman & Rhodes, 2010), and the limitations of RCTs for informing policy (Sampson, 2010). To yield two or more studies of similar findings requires a sufficient time to conduct numerous RCTs to determine that an intervention is effective. Added to this is the need to extend the RCT to disparate populations in different settings to replicate the findings. For example, a single-site RCT with a sample of 150 offenders and a 2-year follow-up can take 5 years to complete; this can limit the value of the RCT model for policymakers who need to make relatively quick decisions about which interventions to adopt. For these reasons, some have called for a greater reliance on multicenter trials where there are simultaneous RCTs in different settings and with different populations (Weisburd & Taxman, 2000), rigorous observational studies, mixed methods approaches (Palinkas, Horwitz, Chamberlain, Hurlburt, & Landsverk, 2011), and multivariate longitudinal studies (Sampson, 2010; Tucker & Roth, 2006). The latter (longitudinal) design involves following one cohort through and after treatment, collecting frequent and comprehensive data that may allow the researcher to isolate the components of effective treatment and the factors that affect treatment success and failure along different time points. This model is specifically useful for substance abuse or mental health disorders, or criminal behavior, where the duration of the treatment impact may be affected by factors other than the specific intervention. Nonetheless, despite the above concerns, RCTs remain the standard exemplar by federal funders, researchers, and evidence-based repositories. These sources frequently make decisions about which interventions or practices are effective and evidence-based. The Food and Drug Administration (FDA) and NIH models retain their primacy and are likely to remain the preferred models for the foreseeable future given the focus on reducing harms to individuals, as discussed below. The FDA model requires at least two clinical trials with similar outcomes to indicate a medication or procedure is ready for public consumption. NIH employs a similar standard.
2.3
Efficacy vs. Effectiveness
Well-established standards for the scientific process also describe several stages that are needed to develop knowledge about the evidence base for interventions and practices. Efficacy refers to evidence that a treatment/practice has beneficial effects when delivered under carefully controlled conditions designed for experimentation. In efficacy studies (Phase II trials, see below), the researcher exerts considerable control over sample selection, delivery of the intervention, and the settings in which the intervention takes place. This best replicates the laboratory environment in the natural sciences where the scientist exercises the most control over every aspect of an experiment. Effectiveness refers to evidence that a treatment has beneficial effects when delivered to heterogeneous samples of clinically referred individuals treated in diverse clinical settings by clinicians rather than researchers (Phase III trials; see below).
2.3
Efficacy vs. Effectiveness
25
Efficacy trials usually involve randomized clinical trials while effectiveness studies may also include traditional evaluations as well as multisite replications using randomized trials. The question about whether an effective intervention is transportable (Schoenwald & Hoagwood, 2001) has been raised and has spurred interest in the resiliency of outcomes as the intervention or practices move from efficacy to formal effectiveness trials to more general use (diffusion and dissemination). A thorough discussion of transportability issues is in Chap. 8. Sustainability is another important issue, and refers to the extent to which an intervention remains effective over time and continues to be implemented with fidelity (Fixsen et al., 2005). When an intervention is sustainable, staff embraces it as being effective and preferable to previous or alternative approaches. To sustain an intervention, it is also usually necessary that local resources be used after the initial external or grant funding is completed. Comparative effectiveness research (CER) is a relatively new approach to health care research that seeks to compare evidence on the effectiveness and potential harms of different treatment options simultaneously (Sox & Greenfield, 2009). In contrast to the more traditional RCT model of comparing a new intervention to a placebo or standard care, CER seeks to use evidence from existing published research, including systematic reviews and meta-analyses, to provide information about the relative impacts of different (often comparable) treatment options or models. CER can also be conducted using new studies that randomize patients into two or more different treatment options to determine the effectiveness of either approach, and to ascertain whether one treatment is better suited for one type of patient. CER can fulfill an important goal for the federal government and health care professional in generating timely information about different treatments and disseminate the results in a way that is easily understood and usable by clinicians, policymakers, and patients. To promote RCTs that compare different treatment interventions to one another, the Agency for Healthcare Research and Quality of the U.S. Department of Health and Human Services outlines seven steps for using CER to compare treatments and increasing the public health impact of the findings (http://www.effectivehealthcare.ahrq.gov/index.cfm/what-is-comparative-effectiveness-research1): 1. Identify new and emerging clinical interventions. 2. Review and synthesize current medical research. 3. Identify gaps between existing medical research and the needs of clinical practice. 4. Promote and generate new scientific evidence and analytic tools. 5. Train and develop clinical researchers. 6. Translate and disseminate research findings to diverse stakeholders. 7. Reach out to stakeholders via a citizen forum. CER can help clinicians and policymakers make more informed choices about which intervention to use with which population. Thus, although a relatively new approach, CER has potential utility for helping community corrections and addiction treatment agencies make decisions about which program or practice to implement with their population. The American Recovery and Reinvestment Act of 2009 created the Federal Coordinating Council for Comparative Effectiveness Research to coordinate CER throughout the federal government.
26
2
Identifying the Evidence Base for “What Works” in Community Corrections…
In theory, and certainly in practice, all research strategies have some value for advancing science and knowledge. To be most useful for practice and policy decisions, researchers should maximize scientific rigor and use a systematic process for developing knowledge, starting with observation and qualitative research through randomized clinical trials with appropriate statistical analyses. The basic approach that underlies all levels of inquiry is systematic observation and objectivity and a set method for collecting and analyzing data. From a methods perspective, what hierarchically differentiates research designs is the extent to which potential threats to internal validity (i.e., nature of the intervention, techniques to recruit patient or collect data, comparability of experimental and control groups, sample attrition and missing data) and external validity (i.e., generalizability) are controlled. Moving up the scientific scale, process and implementation evaluation allows measurement of how an intervention is operating and its effects upon participants as well as fidelity of implementation (adherence to the original intervention and study design). Finally, rigorous designs that include well-designed control groups range from single site experimental and quasi-experimental studies to the gold standard of multisite randomized clinical trials. Adding multiple waves of data collection to increase the length of follow-up in longitudinal designs also strengthens the study findings by examining the duration of the effect or the patterns of decay in outcomes.
2.4
Frameworks for Determining the Evidence Base
A number of processes are available to determine the evidence base. This section will identify the different approaches for determining the evidence base and scientific processes to synthesize information. What are the procedures and process used to decide that an intervention or practice is effective (i.e., evidence-based)? What are the benefits and drawbacks of these methods? How is information about EBP disseminated to the field? What are some of the types of efforts used to promote the adoption and implementation of EBP? In this section, we address these questions.
2.4.1
The Food and Drug Administration (FDA) Model
The evidence determination process has its roots in the FDA Model for reviewing scientific evidence to evaluate the effects of pharmaceutical treatments (FDA, 2010). The FDA guidelines dictate that in order for a medication and/or device to be considered appropriate and safe for public use, a series of clinical trials need to be completed. The evidence needs to be based in strong science and research design, and be able to determine whether the drug/product is both safe and effective. The FDA model requires: (1) methodological quality of the evidence; (2) findings of a positive treatment effect that are relevant to appropriate target groups (e.g., by gender, age categories, and disease); (3) findings replicated in a minimum of two different studies; and (4) an overall consistency of the evidence in terms of the
2.4
Frameworks for Determining the Evidence Base
27
direction of the effect. After assessing the totality of the scientific evidence, the FDA determines whether there is “Significant Scientific Agreement” to support the hypothesized effect. Given the overarching importance of scientific rigor, the FDA model calls for multiple, replicated randomized controlled trials before a drug/ product can be designated as effective and safe for human consumption. The resulting evidence is then used to develop information for both dosages and impacts.
2.4.2
Applying the FDA Model to Behavioral Interventions
In translating the FDA model into behavioral health interventions, Rounsaville, Carroll, and Onken (2001) proposed a staged model of intervention research. The process begins with intervention development, followed by a pilot randomized trial of intervention efficacy in one site under carefully controlled conditions (Stages 1a and 1b). The National Institutes of Health guidelines expand on this model and define four stages of clinical trials research for treatment interventions to determine whether an intervention is effective. The trials at each phase have different purposes and help scientists answer different questions. In Phase I trials, researchers conduct initial tests of an experimental drug or treatment in a small group of people (20–80) in order to evaluate its safety, feasibility and acceptability, determine dosages, and identify any unanticipated negative effects. Phase I trials provide early evidence of efficacy. In Phase II trials, the experimental treatment is given to a larger group of people (100–300) to determine efficacy in a controlled setting with a relatively focused target population and to further evaluate its safety and side effects. In Phase III trials, the experimental intervention is tested in large groups of people (1,000–3,000) in multiple settings and locations, with less researcher control over the intervention or the selection of the study subjects. This is referred to as a multisite trial, often using multisite longitudinal data to determine effectiveness over time. Phase III trials determine whether an intervention is effective, examine any unanticipated negative consequences, and compare the intervention to other commonly used treatments. The replicated RCTs can serve to determine the benefit–risk relationship of the intervention and assess its effects in different populations. Phase IV trials continue obtaining data on long-term effects of the treatment, assessing effectiveness in different populations, assessing costs and benefits, identifying optimal dosage, and measuring “active ingredients” (e.g., dismantling studies).
2.4.3
Synthesizing Across Research Designs
The FDA and modified behavioral interventions models are based almost exclusively on randomized clinical trials. But, in many disciplines like criminal justice and education, it is common for studies to use a broader range of methods that vary in rigor. In criminal justice studies, the Maryland Scientific Methods Scale (Sherman et al., 1997) and similar schemes have been used to accommodate the varying
28
2
Identifying the Evidence Base for “What Works” in Community Corrections…
Table 2.1 Levels of evidence in the Maryland Scale Level 1 (weakest Correlation between a crime prevention program and a measure of crime evidence) or crime risk factors at a single point in time Level 2 Temporal sequence between the program and the crime or risk outcome clearly observed, or the presence of a comparison group without demonstrated comparability to the treatment group Level 3 Comparison between two or more comparable units of analysis, one with and one without the program Level 4 Comparison between multiple units with and without the program, controlling for other factors, or using comparison units that evidence only minor differences Level 5 (strongest Random assignment and analysis of comparable units to program and evidence) comparison groups
designs while rigorously assessing the evidence base. In this scheme, various studies are combined to determine the level of knowledge that exists across studies that range in design from no control groups to randomized trials. The Maryland Scientific Methods Scale was developed from a consensus process whereby researchers developed techniques to combine studies, regardless of rigor, to inform policymakers of the state of knowledge in a given area. This framework forms the basis for designation of interventions as evidence-based, as shown in Table 2.1, and has been used or adapted in various evidence-based repositories discussed in Section 2.5. A tension exists between internal validity (integrity of the intervention) and external validity (generalizability to broader populations in different settings). As one moves up the scale of scientific rigor, threats to internal validity of the evidence decrease, yielding more confidence in the findings. Relatively few addiction treatment programs and practices have been designated as evidence-based in criminal justice settings, as discussed below. This is largely due to the difficulty of conducting studies in justice settings that meet the highest standard of scientific rigor. For example, it is perceived to be unethical and sometimes legally impossible to randomly assign offenders to prison or probation, and ethically problematic to withhold treatment from an offender if that treatment could help the individual avoid incarceration. Accordingly, these issues as well as other factors (e.g., cost, feasibility and acceptability, transportability) may lead policymakers to adopt interventions with lower levels of evidentiary strength. This contributes to the selection and continued use of interventions that do not have the strongest evidence base.
2.4.4
Consensus Processes
The scientific process is different from consensus approaches. Consensus approaches can involve activities such as focus groups, panels of experts and key informant surveys that access the richness of clinical experience but do not include rigorous hypothesis testing. Consensus approaches are important from many dimensions, and in fact can be used in conjunction with the scientific process. Consensus approaches can be used to enhance the research by including stakeholders in the
2.4
Frameworks for Determining the Evidence Base
29
definition of interventions to study and to synthesize clinical and scientific information. Such strategies can be used to identify potential new interventions and practices, to identify key outcome measures as well as possible moderators and mediators, and to ascertain whether the findings are feasible and sustainable in real-world settings. Such approaches are insufficient for testing whether the idea or concept can actually affect client outcomes but they can garner support for the utilization of the findings after a study is completed. Although one dilemma is the sometimes nonrigorous process associated with the consensus approach, it is recognized that the consensus approach allows the field to have input, particularly when there are inconsistent findings or concern that the intervention or practice tested may not be suitable for the field. Many groups that support EBP to guide policy and practice recognize the value of the consensus model, and often include stakeholders in the EBP designation process. The general perception is that such a process will ease dissemination efforts and contribute to greater utilization of the research findings. Stated simply, the consensus approach is part of a process to reduce the gap from research to practice, a core goal of TT as well as the EBP movement.
2.4.5
Systematic Reviews and Meta-Analyses
Increasingly, scholars and EBP repositories are relying on systematic reviews and meta-analyses to rigorously summarize the conclusions that can be drawn from the empirical literature about the effects of specific programs or interventions for offenders in specific settings. Systematic reviews incorporate methodological criteria for synthesizing information across various studies that vary in terms of quality of the design and statistical methods. These reviews are important because there may be numerous existing program evaluations and intervention studies, including studies that have not been published in the scientific journal-based, peer-reviewed literature (referred to as gray literature). The challenge of the synthesis process is to draw meaningful and defensible conclusions across a number of studies where the quality of studies varies substantially including different instrumentation and measures, target populations, and statistical analysis methods; different studies may reach different conclusions about efficacy; studies may be done in different settings with different populations; different studies may include different types of bias that raise questions about the internal or external validity of the findings; or the number of publications and journals may be overwhelming and difficult to sift through or not be readily accessible to program staff or policymakers. For these reasons, program developers, policymakers, and researchers need help in understanding what can reasonably be concluded from existing research findings, what is unknown, and what is unclear. Systematic reviews are of increasing importance in the health care and justice fields. These reviews synthesize studies, using agreed upon standards for addressing methodological weaknesses, in a meaningful way that is also very “customer friendly.” The impetus for systematic review methods came from the Cochrane Collaboration, an international organization that seeks to improve health care decisions through the preparation, maintenance, and
30
2
Identifying the Evidence Base for “What Works” in Community Corrections…
dissemination of systematic reviews of the risks and benefits of health care interventions (http://www.cochrane.org; see Sect. 2.5). These reviews involve complex procedures, and require many judgments to be made (Oxman, 1994); systematic reviews may introduce another set of biases given the decision criteria required in the synthesis process (see below). A good review needs to be explicit about the selection criteria, search strategies, coding methods, and study quality ratings. Even when the review is explicit, it is clear that the researchers are making decisions about which studies to include and the decision criteria to make determinations about “effectiveness.” Two sample definitions of systematic reviews are: “[Systematic] reviews … use rigorous methods for locating, appraising, and synthesizing evidence from prior evaluation studies. They contain a methods and results section and are reported with the same level of detail that characterizes high-quality reports of original research” (Farrington, Petrosino, & Welsh, 2001, p. 340). “Systematic reviews … answer a clearly formulated question, employing systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from the studies that are included in the review” (Mowatt, Grimshaw, Davis, & Mazmanian, 2001, p. 55).
General guidelines for summarizing results from the systematic review (see Box 2.1) include (Oxman, 1994): • Draw conclusions only from the evidence reviewed alone. • Recommendations should be linked to the strength of the evidence, based on design quality (but also relevance and concerns about attrition and missing data). • The review should be explicit about values and preferences. • Subgroup analyses should be interpreted with caution (these may have been post hoc, the subgroups may not be randomly selected) subgroups may not be appropriate targets for the intervention, etc. • Different statistical analyses may result in different conclusions. • Sensitivity analyses should be conducted if possible (e.g., unpublished vs. published studies, by rigor of included studies). How sensitive are the results to the methods used for the review, how robust are the findings across methods, populations? Provide confidence intervals around the effects to provide a good indication of the precision of the findings.
Box 2.1 Options for Reporting Systematic Review Findings • Percentage of studies (within study quality groupings, populations, settings, other subgroups, perhaps) that found a significant difference in outcomes. • Calculating the average effect size (a standardized measure of the difference in an outcome between the experimental and control groups). • Calculating the Odds Ratio (a standardized measure that indicates the direction and size of the impact between experimental and control groups). • A forest plot showing the range of effect sizes across studies. One can also calculate a weighted estimate of treatment effect but the plot illustrates the trend across studies.
2.4
Frameworks for Determining the Evidence Base
31
Systematic reviews and especially meta-analyses typically consolidate findings across disparate studies by calculating outcome differences into standard effect sizes. Some argue that effect size is more important than probability values, statistical significance or hypothesis testing (Grimshaw, Eccles, & Tetroe, 2004), since it standardizes differences between the experimental and control groups across different measures. However, the relative value of effect size vs. statistical significance has generated much debate in the field (Weisburd, Lum, & Yang, 2003). Statistical significance indicates the degree to which one can have confidence in the findings based on the probability that the difference between the groups was not due to chance, the size of the sample, and the variance. But effect sizes are easier to interpret given that they can be translated into a measure of magnitude of the effect:
