An Introduction to Quality Assurance in Health Care

An Introduction to Quality Assurance in Health Care This page intentionally left blank An Introduction to Quality A...

Author: Avedis Donabedian

400 downloads 1835 Views 7MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

An Introduction to Quality Assurance in Health Care

This page intentionally left blank

An Introduction to Quality Assurance in Health Care AVEDIS DONABEDIAN Edited by Rashid Bashshur

OXPORD UNIVERSITY PRESS

2003

OXFORD UNIVERSITY PRESS

Oxford New York Auckland Bangkok Buenos Aires Cape Town Chennai Dar es Salaam Delhi Hong Kong Istanbul Karachi Kolkata Kuala Lumpur Madrid Melbourne Mexico City Mumbai Nairobi Sao Paulo Shanghai Taipei Tokyo Toronto

Copyright © 2003 by The American University of Armenia Corporation Published by Oxford University Press, Inc. 198 Madison Avenue, New York, New York 10016 http://www.oup-usa.org Oxford is a registered trademark of Oxford University Press All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of Oxford University Press. Library of Congress Cataloging-in-Publication Data Donabedian, Avedis. An introduction to quality assurance in health care / Avedis Donabedian. p. cm. Includes bibliographical references and index. ISBN 0-19-515809-1 1. Medical care —Quality control. I. Title. RA399.A1 D653 2003 362.1'068'5-dc21

2002070383

246897531 Printed in the United States of America on acid-free paper

To all our loved ones who, scarcely known to us, and innocent of all fault, perished at the hands of a cruel enemy because they were Armenian


Foreword

A testament to the intellectual commitment and caliber of the man, Avedis Donabedian wrote this book when most of us would have renounced any interest in the external world and focused exclusively on a rapidly declining state of health. There was only one argument, no other, that could have persuaded him to commit himself to the arduous task. Haroutune Armenian, President of the American University of Armenia, a dear friend and cherished colleague, knew the argument and used it. Haroutune argued that the best way for Donabedian to influence the quality of medical care in Armenia and other countries around the world would be to write a book for the general readership of health-care professionals. With deep admiration for Armenian's dedication to his homeland, his scholarship and his fortitude, Donabedian accepted willingly. This would be the last time he spoke to us through the professional written word, and this is his lasting gift. As he had done many times before, Donabedian wrote this book for anyone seriously interested in improving the quality of medical care. He wrote for the reader who would stay with him from cover to cover. His style in this volume is conversational. He wanted to talk to the reader, to tell the reader in succinct and personal form the whole story of quality from beginning to end, as he knew and experienced it. Donabedian had a deep appreciation for the reader who would scrutinize his words and understand them fully. So, he chose them carefully and deliberately to convey his message, yet he expressed himself in a very personal fashion.

viii

FOREWORD

He wanted to develop a sense of affiliation with the readers, whom he considered fellow seekers of truth and knowledge on a quest to improve the human condition. Donabedian knew there would be no opportunity for him ever to visit his original homeland, so he wrote for fellow Armenians and others who would want to learn about quality of care and how to improve it. He knew the book would be published posthumously, so he dedicated it to the people of Armenia, the homeland of his parents who suffered unspeakable hardships and sorrow in their personal trail of tears when forcibly removed. Readers very familiar with Donabedian's work on the theory, concept, and measurement of medical care quality may not find new revelations in this book. They will, however, find new insights and dimensions as he presents, discusses, and assesses them in a systematic and probing personal statement. Perhaps just as important, they will appreciate the last words from the teacher, the scholar, and the friend they have so admired. Those readers new to Donabedian's expertise will no doubt come away much better informed and equipped to face and explore the problem of improving the quality of medical care, as have many previous generations of health-care professionals. Avedis Donabedian the person was unique in many ways. As one of his former students and closest friends, Julio Frenk, put it, "His secret, I believe, was his capacity to love." He summarized the nature of Donabedian, the man and his work as, " . . . a love for ideas, a love for words, and a joy in the happy conjunction of both. At the peak of his maturity, he came to define quality precisely as love." Avedis Donabedian wrote this volume during the end stages of combat with an avaricious cancer that weakened his musculature but left his mind untouched. Faced with failing health and imminent death, he comforted and gave strength to those about him as he savored every moment with his beloved wife, Dorothy, and his cherished friends. Avedis wondered how much time he had left, yet he was certain that the next phase into which he was passing would be peaceful, free of pain, and full of good company.

Foreword

ix

Rather than writing this book, I think Avedis would have rather spent his last days and hours reading and writing poetry. In the week before his death, he gathered his strength to leave us this last farewell, which I share with the reader: Departure I do not go, as the poets say, Raging against the dark. There is no darkness where I go Only sweet light Pure as a summer's day. My one regret, love of my life, Parting from you, and you Dear Friend, and you All others, cherished, loved, Sinew and sap of my old life, And, in my new, recurrent gifts, As I wait upon its blessed shore The joy of your arrivals. —Avedis

Avedis Donabedian passed away with a smile on his face on November 9, 2000, at the age of 81. His departure leaves a deep void in my life that I will feel until my turn comes. Ann Arbor, Michigan

Rashid Bashshur


Preface

It was with great reluctance that I undertook to write this book. I have been in retirement for many years now. And although during this time I have traveled a fair amount, lectured in many places, and made many friends, as time went on and my health began to fail, I let my past slip by, and began to look to a brighter future beyond. From this happy state I was recalled by the urgent need, insistently and repeatedly brought to my attention, for a brief, coherent account of quality assurance in health care for use by students of the subject in my native Armenia. It was difficult to resist for long the persuasive representations of a dear friend whose sacrificial devotion to the cause of education in Armenia was a shining example. That, and a sense of obligation to my own people, gradually wore down my defenses. I must also admit that I was enticed by the prospect of leaving behind as a modest legacy a self-contained summary of my work, written not for experts, but for the kinds of commencing students who, during much of my professional life, had been my particular concern. But would that legacy be a body of thought already old and worn out? Would I be seen as out of step, no longer relevant? Would my very words, so little attuned to the fashionable vocabulary of today, betray me? As I confronted these doubts, I was sustained by the conviction that fundamentals do not often change; or, when they do, it is very slowly. I believed that the new is many times, as I have tried to show elsewhere, mostly a continuation of the old.1 And, sometimes, what seems shiningly new is no more than a

xii

PREFACE

reinvention of the formerly known, perhaps with an added emphasis or twist, or possibly under a new, more alluring, name. So that, if nothing else, by weaving new threads into the old fabric, I could perhaps convey a sense of the essential continuity, the organic growth of our field, in its values, concepts, and methods. To this end, I have tried to fold into my narrative the more recent developments I was aware of. In doing so I have avoided a polemic that would have only confused and distressed a beginner. I have, however, in a few places, permitted myself a scattering of mildly ironical remarks —a foible I hope forgiven in an octogenarian! The degree and nature of the documentation I have provided in this book (or failed to do) deserves some comment. What may seem to some to be an excessive reliance in the bibliography on my own work is intentional. I have conceived of this text as a personal testament, although brief, of what I have learned and believed, and taught. Because it is no more than a summary, I wished to offer the reader an opportunity to consult the more detailed expositions to be found in the three volumes of my Explorations . . .2 and in the other references I cite. I have also harbored the hope that a compendium of the latter may at some time be assembled and, perhaps, be translated into Armenian as well. I have tried, at the same time, to acknowledge the work of others, but, I admit, not as fully as it has been my custom to do. I did not wish an elementary introduction to be overly cluttered with the trappings of scholarly protocol. Those who wish to trace the origins of my thinking can find them meticulously documented in my other published work. I have already commented in passing on the terminology I have used. Out of loyalty to my predecessors and to my own past, I chose the words I was familiar with. I did not wish to assume a veneer of modernity I was not entitled to. But I have bowed (happily, I might add) to political correctness by avoiding, to the extent possible, gender-specific terminology. And by using words such as "clinician," "practitioner," and "care-giver," I have intended to include among my audience not merely physicians, but other health workers as well. By contrast, "patients" I have firmly embraced, meaning by the term not "those content uncomplainingly to wait," but "those who suffer." It is to their welfare that, first as a practitioner, and then as a professor, I have dedicated myself.

Preface

xiii

I can only hope that this modest work of mine will inspire my readers as well to commit themselves to the noble cause.

A.D.

References 1. Donabedian, A., "Continuity and Change in the Quest for Quality." Clinical Performance and Quality in Health Care 1: 9-16, January/February/March, 1993. 2. Donabedian, A., Explorations in Quality Assessment and Monitoring. Volume I: The Definition of Quality and Approaches to Its Measurement; Volume II: The Criteria and Standards of Quality; Volume III: The Methods and Findings of Quality Assessment and Monitoring —An Illustrated Analysis. Ann Arbor, MI: Health Administration Press, 1980, 1982, and 1985, respectively. Volume I contains, together with other material, a more detailed exposition of the structure-process-outcome model. Volume III contains brief descriptions, accompanied by graphical illustrations, of many of the methods of performance monitoring described in this text, and of many other studies of quality as well. As a service to the reader, I take the liberty of saying that copies may be obtained from Health Administration Press, American College of Healthcare Executives, One North Franklin Street, Suite 1750, Chicago, IL. 60606-3491. For inquiries: phone 312-845-9869; fax, 312-424-0703; email, [email protected]. Web site URL: www.ache.org/hap.


Acknowledgments

I am indebted to Haroutune Armenian, president of the American University of Armenia and professor of epidemiology at the Johns Hopkins University School of Hygiene and Public Health, who, besides urging me to undertake this work, inspired me with his own example of selfless service. No doubt, as this project unfolds, I shall have others to thank, including those who will help amend and improve my text, those who will undertake the difficult task of translation, and those who will verify and refine it. These I shall be happy to gratefully acknowledge.


Contents

LIST OF APPENDICES, xxi INTRODUCTION, xxiii What This Book Is About, xxiii The Meaning of Quality Assurance, xxiii The Targets of Quality Assurance, xxiv The Components of Quality Assurance, xxv The Quality Monitoring Cycle, xxvii Formal and Informal Monitoring, xxviii Some Foundations of Quality Assurance Through Monitoring, xxix The Meaning of Quality in Health Care, xxxi 1. THE COMPONENTS OF QUALITY IN HEALTH CARE, 3 Efficacy, 4 Effectiveness, 5 Efficiency, 9 Optimality, 11 Acceptability, 18 Choosing a Definition of Quality, 24 Steps in Monitoring and Improving Clinical Performance, 26

CONTENTS

xviii

2. DETERMINING WHAT TO MONITOR, 29 Externally Required Monitoring Activities, 30 Internally Motivated Monitoring Activities, 30 3. DETERMINING PRIORITIES IN MONITORING, 39 Importance to Patient Welfare, 40 Additional Institutional Interests, 40 Feasibility, 42 4. SELECTING APPROACHES TO ASSESSING PERFORMANCE, 45 Definitions of Structure, Process, and Outcome, 46 Comments on the Structure-Process-Outcome Model, 47 Relative Usefulness of the Three Approaches, 49 Conclusions About the Choice of an Approach, 56 5. FORMULATING CRITERIA AND STANDARDS, 59 Definitions of Terms, 60 Importance of Criteria and Standards, 61 Some Attributes of Criteria and Standards, 61 6. OBTAINING THE NECESSARY INFORMATION, 77 Medical Records, 78 Surveys, 84 Financial Records, 86 Statistical Reports, 87 Direct Observation, 87 Test Situations, 89 7. CHOOSING WHEN AND HOW TO MONITOR, 91 When to Monitor, 92 How to Monitor, 93

Contents

xix

8. CONSTRUCTING A MONITORING SYSTEM, 117 Key Elements, 118 9. BRINGING ABOUT BEHAVIOR CHANGE, 123 Readjustments in the System, 124 Educational and Motivational Activities, 126 10. THE EFFECTIVENESS OF QUALITY MONITORING, 133 Contextual Factors, 134 Operational Factors, 136 Some Concluding Remarks, 137 REFFERENCES, 139 APPENDICES, 145 INDEX, 197


List of Appendices

1. Pediatric Appropriateness Evaluation Protocol (AEP), 145 2. Staging Criteria for Cancer of the Breast and Cholecystitis, 151 3. Pneumonia: Severity Levels, 153 4. MEDISGRPS Method for Classifying Cases by Severity and Assessing Progress Towards Improvement, 155 5. The Apache II Severity of Disease Classification System, 159 6. Specialty-Specific Indicators Proposed for Concurrent Review by the American Hospital Association, 1989, 163 7. General Screens Proposed by the Empire State Medical, Scientific, and Educational Foundation, 179 8. Clinical Indicators Proposed for Further Field Testing, Joint Commission on Accreditation of Health Care Organizations, 1989, 183 9. A More Modest List of Performance Measures for Hospitals Proposed by the Joint Commission on Accreditation of Health-Care Organizations, February 2000, 193


Introduction

What This Book Is About This book is not an exhaustive review of what is known or said about its subject matter. It is, rather, a personal statement of my own views and conclusions. I shall, therefore, permit myself a certain informality of style, imagining that I am talking, most of the time, to one of my friends or students. My purpose in writing is to offer a concise and I hope a readily understood introduction to the ways quality in health care can be defined, measured, and improved. But in my presentation I shall not follow this reasonable order. Aiming to be "practical," I shall begin with the immediate task we face, which is to maintain and improve quality, an activity that goes under the name of "quality assurance," among others.

The Meaning of Quality Assurance An author who sets out to write a treatise on quality assurance must begin by saying what he understands that term to mean. I shall define quality assurance to mean all actions taken to establish, protect, promote, and improve the quality of health care. Strictly speaking, one cannot assure or guarantee quality. One can only increase the probability that care will be "good" or "better." Because

INTRODUCTION

xxiv

quality cannot be guaranteed, many, myself included, have criticized the term assurance. Suggested alternatives have been improvement or, better still continuous improvement, terms meant to remind us that no given level of quality can be fully satisfactory1; one should always try to do even better, progressing to ever higher levels of goodness.2 Another alternative, the term quality management, is useful in suggesting that the quality of health care is subject to managerial oversight. Unfortunately, the term may also be understood to mean that there can be several levels of quality that cost more or less and, therefore, can be offered at correspondingly higher or lower prices. If so, one must consider the social and ethical consequences of this viewpoint, as I shall try to do later in this work. If one understands the several terms I have mentioned (and others I have not) and is aware of their implications, one may still choose to use the term quality assurance, as I have done, because it is firmly established and widely used.

The Targets of Quality Assurance By targets I mean the functions and activities that are subject to quality assurance. Traditionally, in health care, quality assurance has been meant to apply predominantly, or even exclusively, to health care itself as provided directly to patients by legitimate health care practitioners. Removed one level, we include other services that directly affect the ability of practitioners to perform well, meaning such things as radiological, pharmaceutical, and laboratory services. In recent years, under the rubric of "total quality management," it has become popular to include in the idea of quality assurance almost every function or activity of a health care organization or system, including such things as the performance of housekeeping crews, secretarial and clerical services, the billing and fee-collecting office, and so on. It is reasonable to say that the quality of the immediate environment of health care, including the managerial activities in it, have an influence of the

Introduction

xxv

quality of care — either directly, by influencing the performance of practitioners or indirectly, by influencing the convenience, comfort, or safety of patients. Therefore, it is quite legitimate to adopt this broader view, as signified by "total management," provided one does not take one's eye off the key object of "quality assurance," which, I believe, should pertain to the performance of practitioners as they care for patients. The functions and activities subject to quality assurance are broadened even more when we include how patients care for themselves and are cared for by family members and friends. There is good reason for this broadening of scope. First, it allows one to infer how well health care practitioners have prepared patients to care for themselves. Second, it allows one to obtain a complete picture of the health care actually executed by all participants in it. Third, with this information in hand, one can know where the failures have been and possibly take steps to improve care.

The Components of Quality Assurance As I show in Figure Intro. 1, quality assurance activities can be divided into two parts. One I call "system design and resources," and the other "performance monitoring and readjustment." It is obvious that without resources of sufficient quantity and good quality we cannot offer the best care we are potentially capable of. Moreover, the system of care should be designed so that it helps the delivery of good care rather than creating obstacles to it. System design corresponds to what later I shall call "structure." It includes professional recruitment, education, training, and certification. It also includes the number, distribution, equipment, organization, and licensure of hospitals and other health care facilities. It includes, furthermore, the testing and marketing of drugs and biologicals, the financing of care, access to health services, legal protection of consumer and provider interest, and so on. This book will deal only secondarily with how a system of health care should be set up and run. It will be concerned mainly with the second component of quality assurance, the one I have called "perfor-

xxvi

INTRODUCTION

Figure Intro. 1. Components of quality assurance.

mance monitoring and readjustment." This, in fact, is what quality assurance is generally taken to mean. It is an activity by which we obtain information about the level of quality produced by the health care system and, based on an interpretation of that information, take the actions needed to protect and improve quality.

Introduction

xxvii

This action, as shown in Figure Intro. 1, can take one of two forms: activities meant to educate and motivate persons directly, or readjustments in system resources and design. The readjustments made are those that are expected to influence people's behavior indirectly. The people whose behavior one hopes to influence directly (for example, by education) or indirectly (for example, by a change in the system) are primarily health care practitioners, but also include patients and managers —in fact, anyone involved directly or indirectly in health care. The two components of quality assurance I have described are obviously interrelated. By system design we set the average level of performance as well as the degree of variation above or below that average. By quality monitoring we aim to obtain a more precise adjustment of quality to the level we desire. This is done by raising the average and reducing the fluctuations around it. Quality monitoring (the activity by which we keep the quality of care under constant observation) can be thought of as the eyes and ears of the system of health care. Without it, we do not know where we are or where we are going.

The Quality Monitoring Cycle We can infer from Figure Intro.2 that monitoring and readjustments is a continuing, cyclical activity. The main steps in the monitoring cycle are seen to be: 1. Obtaining data on performance. 2. Pattern analysis; an activity that is essentially epidemiological in nature. 3. Interpretation; which means advancing hypotheses that might explain the patterns observed. 4. Taking preventive, corrective, or promotive action based on the causal hypotheses that have been advanced. 5. Obtaining data on subsequent performance to determine the consequences of the actions taken.

xxviii ii

INTRODUCTION

In Figure Intro.2,1 have included some details that can be for the moment ignored. We shall return to them later on. Moreover, the particular way I have depicted the monitoring cycle is my own; and it is only one of many from which the reader can freely choose. What matters is not the particular formulation but the underlying message. That message is: observe, interpret, do something, assess what you have done, and never rest!

Formal and Informal Monitoring In this book I shall be dealing almost exclusively with quality monitoring as a formally established activity within an organization or a system of

Figure Intro.2. The quality monitoring cycle.

Introduction

xxix

health care. There is, however, another, much subtler, informal process that occurs when colleagues observe each other at work and adjust their behavior to prevalent professional norms. For that reason, the quality of care is greatly influenced by any activity that makes the work of practitioners visible to one another, to their students, and to other health-care workers. These activities include collaborative work, referrals and consultations, clinical rounds, clinical pathology conferences, and so on. The opposite effect occurs when professionals practice in isolation, removed from interaction with their peers. In such cases the failures and successes of the practitioners are both obscured. In a reputable organization, such as a hospital associated with a medical school, the informal influences I have mentioned are conducive to better care. But they could have the opposite effect in an institution where the prevalent norms are inimical to quality. Bad practice by some encourages bad practice by others. In either case, I believe that the informal influences I have mentioned are as important as the formal mechanisms of quality assurance I intend to describe. But it is also necessary to have a formal, predictable monitoring activity as well —one that is uniformly implemented and is acceptable to those whose practices it is meant to scrutinize.

Some Foundations of Quality Assurance Through Monitoring It is difficult, even impossible, for quality monitoring to be introduced and to flourish in a health-care institution or system obstinately opposed to it. A certain degree of readiness to accept monitoring must either exist or be developed. In Table Intro. 1, I offer some of the conditions that are needed for this to happen. This is only an introduction to a subject that I shall deal with more fully later, mainly when I discuss the effectiveness of quality monitoring. Undoubtedly, the most important single requisite is a commitment to quality: an unequivocal desire and determination to dedicate oneself to the best one is capable of, despite every obstacle. This commitment

INTRODUCTION

XXX

Table Intro. 1. Some Foundations of Quality Assurance Through Monitoring A. COMMITMENT TO QUALITY 1. Genuine commitment 2. Internally motivated commitment, not simply a response to external pressures 3. A commitment by everyone, at all levels in an organization B. INSTITUTIONALIZATION OF THAT COMMITMENT 1. Specification of institutional goals 2. Establishment of an organizational structure for performance monitoring a. Specification of responsibility for monitoring b. Allocation of resources: personnel, money, etc. 3. Design and implementation of a set of formal monitoring activities a. Routine and occasional activities b. Centralized and decentralized activities 4. Establishment of mechanisms for communicating information and implementing action 5. Creation of a tradition or culture C. AGREEMENT ON THE MEANING OF QUALITY

must arise from values and impulses within an organization itself. It should not be only the consequence of forces that impinge upon it from the outside. This is not to say that external forces cannot either support or oppose the aspirations that originate within an organization. I merely want to say that external pressures are insufficient to truly motivate individuals and organizations unless they are correspondingly predisposed. Besides being self-motivated, the commitment to quality must pervade the institution at all its levels and in all its aspects. Particularly important is the role of recognized, legitimate professional leaders. Their support, not only in words but also in deeds, is essential. In Table Intro. 1, under the heading of "institutionalization," I mention some of the activities by which the commitment to quality becomes manifested. If present, these activities testify to the genuineness of that commitment; if absent, they lead one to suspect the opposite. An institution committed to quality takes steps to specify, codify, and make known the goals it has pledged itself to achieve, making sure that

Introduction

xxxi

the pursuit of quality has a commanding place among them. Then it proceeds from this declaration to actions demonstrating that more than lip service is being paid to quality. If the commitment to quality is genuine, and if the statement of institutional goals is more than window dressing, an institution will develop the mechanisms for implementing its commitment and attaining its goals. There must be, for example, an organizational mechanism charged with the responsibility to monitor, and adequately supported with the needed resources. Various monitoring activities will be discernible, some routine and others occasional; some carried out by the centralized authority and others delegated. There will be well-defined avenues for communicating the findings to those responsible for taking action in response to these findings, it will be evident that appropriate action has been taken, and it will be known how effective any action has been. All this amounts to a fundamental change in the ethos of an institution. There is now a tradition or a "culture" where the pursuit of quality occupies its rightfully commanding position. This tradition or culture, if it is to be coherent and effective, must perforce include an agreement on what quality in health care means. To this fundamental consideration we must now direct our attention.

The Meaning of Quality in Health Care Some believe that quality in health care is too abstract and nebulous a concept to be precisely defined or objectively measured. It is said that a competent, experienced practitioner can almost intuitively recognize it, if it exists, and offer a equally intuitive measure of its magnitude. And it is similarly asserted that different persons differ in what they perceive quality to be, and how much of it there is. Consequently, some claim that there can be no definition or measure of quality that everyone will accept. While I agree with some of these views to some degree, I must wholeheartedly reject them. If it is true that quality is some ill-defined image in the eye of each beholder, it would be difficult to set it apart as

xxxii

INTRODUCTION

a goal an individual or an organization can aspire to. Precisely at this point the quality assurance enterprise would crumble into nothingness and this book would end. I believe, on the contrary, that the concept of quality can be rather precisely defined, and that it is amenable to measurements accurate enough to be used as a basis for the effort to monitor and "assure" it. This is the conviction that I must now justify.

An Introduction to Quality Assurance in Health Care


1 The Components of Quality in Health Care

4

AN INTRODUCTION TO QUALITY ASSURANCE IN HEALTH CARE

t is possible to conceive of quality as the product of two factors. One is the science and technology of health care, and the second is the application of that science and technology in actual practice. The quality of care achieved in practice is the product of these two. As is shown in Figure 1.1, that product can be characterized by several attributes that include efficacy, effectiveness, efficiency, optimality, acceptability, legitimacy, and equity. These, taken singly or in a variety of combinations, constitute a definition of quality and, when measured in one way or another will signify its magnitude. As a prelude to what is to follow, I give in Table 1.1 brief definitions of each of the attributes of quality that I have just mentioned. A more detailed, but concise, description follows.

I

Efficacy Efficacy is the ability of the science and technology of health care to bring about improvements in health when used under the most favorable circumstances. Some further considerations should be noted. First, it is not always possible to specify what "the most favorable circumstances" I have postulated are. Therefore, it is possible to substitute "under specified circumstances" for the vaguer concept of "most favorable." Second, it follows from what I have just said that efficacy is the standard against which any improvement in health achieved in actual practice is to be compared. As suggested in Figure 1.1, efficacy is not itself subject to monitoring when the quality of practice is being assessed. Rather, it is given to us, a priori, as a product of research, experience, and professional consensus. Third, as I show in Figure 1.1, the science and technology of health care set the standard not only for efficacy but for the other attributes of quality as well. This means that actual performance in all its aspects is compared to what our science and technology, at its best, is expected to achieve. This formulation has a consequence also alluded to in Figure 1.1. It requires that "science and technology" be defined broadly to in-

The Components of Quality in Health Care

5

Figure 1.1. Components of quality in health care.

elude not only biological factors but the behavioral sciences as well. It is true that these sciences may not be, as yet, sufficiently developed to offer us clear guidelines and precise standards. It is hoped, however, that they will be able to do so as they mature. And finally, we should remember that not all standards of performance derive from what we are accustomed to call "science and technology." Some standards are set by social and individual preferences, and some others by ethical and moral considerations.

Effectiveness Effectivenessss isis the degree to which improvements in health now attainable are, in fact, attained. This implies, as I have already said, a compar-

6


Table 1.1. Components of Quality: Definitions 1. EFFICACY The ability of the science and technology of health care to bring about improvements in health when used under the most favorable circumstances. 2. EFFECTIVENESS The degree to which attainable improvements in health are, in fact, attained. 3. EFFICIENCY The ability to lower the cost of care without diminishing attainable improvements in health. 4. OPTIMALITY The balancing of improvements in health against the costs of such improvements. 5. ACCEPTABILITY Conformity to the wishes, desires, and expectations of patients and their families. 6. LEGITIMACY Conformity to social preferences as expressed in ethical principles, values, norms, mores, laws, and regulations. 7. EQUITY Conformity to a principle that determines what is just and fair in the distribution of health care and its benefits among members of the population.

ison between actual performance and the performance that the science and technology of health care, ideally or under specified conditions, could be expected to achieve. This formulation is presented graphically in Figure 1.2. To simplify my presentation, I assume in Figure 1.2 that we have in mind a mostly self-limiting disease; for example, a moderately severe upper respiratory infection, not threatening to life. On the ordinate axis I have a measure of health status; for example, the ability to perform the activities of daily living. On the abscissa is plotted the passage of time.


7

Figure 1.2. Graphical presentation of effectiveness in a self-limiting disease.

As the figure shows, the person in question (or the average of a group of persons) begins with a specified level of health close to "wellness." When illness strikes, as shown by the solid line in the figure, health deteriorates for a while and then, because the disease has been assumed to be self-limiting, begins to improve, finally attaining a level similar to that present at the beginning. With this formulation, the area A in Figure 1.2 represents the effect of treatment. Areas A and B combined represent the effect of the best treatment. Effectiveness can now be represented by the fraction (A) -j- (A + B), which is the ratio of the health improvement achieved in actual practice to the health improvement that could have been achieved had the best treatment been given. Effectiveness is, therefore a relative concept, which can be defined as follows:

8

AN INTRODUCTION TO QUALITY ASSURANCE IN HEALTH CARE Improvements in health expected from the care to be assessed Relative httectiveness = Improvements in health to be expected

from the "best" (or "standard" care)

In this regard, it is important to note a number of considerations. 1. Figure 1.2 is offered as only one example among several that could be constructed. It could be, for example, easily modified to represent a progressive disease or one characterized by remissions and relapses. In each case the comparison would be between what is actually achieved and what, under better circumstances, could have been achieved. 2. According to this formulation, effectiveness is only relative to what the science and technology of health care can accomplish. At any given time it is limited by that capacity; and as that capacity improves, the standard against which effectiveness is to be judged is raised accordingly. Clearly, the notion of "continuous improvement" is built into the model, even if some have not seen it. 3. The model, and consequently, the meaning and measure of effectiveness (and of quality as a whole) depend on how one defines and measures health. Given only partial concepts and measures of health, one obtains correspondingly partial measures of effectiveness; with inclusive concepts and measures of health, one obtains similarly inclusive measures of effectiveness. 4. The model represents probabilities, not certainties. We measure the effectiveness of care not by what has occurred in any given case, or a small number of cases, but by what we can expect to occur if an adequate sample of cases were to receive a specified kind of care. If in any given case a practitioner gives care that is known to give the best results on the average in that kind of case, the care is judged to have been good even if that particular patient experiences little improvement, suffers complications, or even dies. 5. The model requires, if it is to be fully implemented, a great deal of information that, alas, is often not available. Very often there


9

are no good measures (or even agreed-upon definitions) of health status; the course of untreated illness is often unknown; what "the best treatment" means or can achieve is equally obscure; and even what happens under actual treatment in groups of patients is imperfectly documented. These deficiencies, when the model reveals them, should stimulate clinical trials and epidemiological studies to rectify them. While we wait for more complete information, we can use less-rigorous but still useful measures of effectiveness. Instead of an inclusive measure of health, we could use a specific symptom or sign such as cough or dyspnea in asthmatics, or a set of blood pressure readings in hypertensives. We could compare the improvement in health during corresponding illnesses receiving different kinds of treatment. 6. When resources are limited we may have to set two (or more than two) standards of comparison. One would be what is achievable with the best use of the resources actually at hand. Another could be what would be achieved with the best use of resources we need, and would like to have. The object is to set immediate goals while we aim for what is better in the future.

Efficiency "Efficiency" is the ability to lower the cost of care without diminishing attainable improvements in health. Expressed as an equation: Improvements in health expected from the care to be assessed Efficiency = 1 he cost ot that care

This means that efficiency is increased if, for a given cost, health improvement is increased or if the same degree of health improvement is attained at a lower cost. It follows that the mere reduction in cost does not denote efficiency unless health benefits are either unaffected or are improved.

10


There are three ways of improving efficiency in health care. 1. One way is for health-care practitioners to prescribe and implement care that does not include harmful, useless, or less effective remedies or methods. This kind of efficiency I have called clinical efficiency because it depends on clinical knowledge, judgment and skill. 2. Another way of improving efficiency is to produce more efficiently the goods and services that are used in providing care. For example, costs are lowered if a hospital is run at a higher occupancy rate, or if a nurse is not asked to do things that an aide or secretary could do just as well. Perhaps one could also include in this category changes in procedure (for example for the preparation and administration of drugs) that reduce the frequency of errors that might cause injury to health and, consequently, higher costs. This kind of efficiency I have called production efficiency.y It has also been called managerial efficiency y because it depends on organizational and managerial decisions in which clinicians do not play the decisive role. It should be recognized, nevertheless, that without the participation and consent of clinicians, some of these changes would be difficult to introduce. 3. Still another way of making care more efficient is to distribute it among different classes of patients (characterized by age, sex, ethnicity, economic status, place of residence, kind of illness, etc.) in a way proportionate to expected improvements in health. In other words, resources are allocated to population subgroups who are perhaps sicker or are more likely to benefit from care, and do so for longer periods of time, and at proportionately lower cost. By doing so one aims for what is called "distributional efficiency." This is, as we shall see, an aspect of quality at the societal level. To cite specific, documented examples of these three kinds of efficiency would take us too far afield in a brief review such as this book is meant to be. But because clinical efficiency is so closely allied to patient care, I shall mention a few illustrative examples.


11

The literature contains many examples of the effort to devise new strategies of care that lower cost while outcomes remain unchanged or even improve. A recent report demonstrates that, by codifying rules that guide the physical examination of patients who have suffered injuries of the ankle or foot, it is possible to reduce the number of X-ray examinations of these parts by 30% without causing harm. The resulting savings, assuming this procedure were universally adopted, is estimated at $150 million for the United States.' Still another study has demonstrated the consequences of implementing a procedure (which the investigators call a critical pathway) for managing patients who seek care for "community acquired pneumonia." The procedure or protocol in question specifies how the severity of the pneumonia is to be judged, which patients are to be admitted to the hospital and which treated at home, the treatment to be given in each of these two places, and, for hospitalized patients, when the patient is judged ready to be discharged. It is estimated that, without injury to patients, the cost to the hospital is reduced by $1,700 per patient, an amount thought to be much greater than the added cost of treatment at home.4

Optimality Optimality is the balancing of improvements in health against the cost of such improvements. This definition implies that there is a "best" or "optimum" relationship between costs and benefits of health care, a point below which more benefits could be obtained at costs that are low relative to benefits, and above which additional benefits are obtained at costs too large relative to corresponding benefits. Figure 1.3 will help illustrate this somewhat abstract concept. To generate the figure it is necessary to engage in a "mental experiment": an experiment rather difficult to duplicate in actual practice, but nonetheless one from which emerges a fundamental principle very relevant to medical practice and social policy. To begin with, we assume that we have an ideal physician, one who has perfect knowledge of both health-care improvements and cost. This

Figure 1.3. Hypothetical relations between health benefits and costs of progressively more elaborate care, when care is clinically most efficient.

12


13

physician is, furthermore, a person who will never prescribe unnecessary care. This physician is given successively larger sets of resources to use in health care. The upper panel in Figure 1.3 shows that if the physician has nothing to work with, there is no improvement in health (referred to as "benefits" in the figure) that could be attributed to care. But as the physician receives more resources (obviously costing more) progressively more improvements can be achieved. Accordingly, the curve of improvements attributable to care rises, as does cost, except that improvements rise faster than costs do. Subsequently, however, as more and more resources are available to the physician, the improvements, though they continue to occur, become relatively smaller and smaller. Eventually, there comes a point when no further improvements can be obtained no matter what resources are available to the physician. At this point, the "ideal physician" recognizes that no further improvement can be accomplished and will not call for additional resources. "Maximally effective" care has been achieved. In the lower part of Figure 1.3 there is an attempt to compare improvements in health against the cost of these improvements. To make such a comparison, we must express costs and improvements in corresponding units of measurement, perhaps dollars. When health improvements are translated to their corresponding equivalents they are called benefits, as the figure shows. How this conversion can be made, and precisely how benefits and costs are compared, is a subject to be taken up in health economics. Here I only assume that it can be done in a socially acceptable manner. And to simplify the matter further, I only subtract costs from benefits. When this is done, there is a point in the lower graph that corresponds to the matching two points in the graph in the upper pane of the figure. The curve thus generated goes upward to a point and then begins to decline. It goes upward as long as additional benefits exceed additional costs. It goes downward when additional benefits are smaller than additional costs. The point of inflection represents "optimally effective" care. I provide Table 1.2 as a partial illustration of the notion of optimality. The table shows some findings of a study of the costs and effects of pneumococcal vaccination. 5 On the left-hand side of the top panel are the net

Table 1.2. Cost-Effect Estimates for Pneumococcal Vaccination Cost (Dollars) Losses Added cost of vaccination

Effects (Quality-adjusted Years) Losses

Gains

Gains

Reduced cost of caring for pneumonia

Increased quality of life as a result of prevention of pneumonia

Added cost of caring for complications of vaccination

Reduced quality of life as a result of complications of vaccination

Cost of treatment of illnesses other than pneumonia as a result of extended life

Increased life expectancy as a result of avoiding death from pneumonia

Discounted at 5% annually

Vaccination Age

2-4 5-24 25-44 45-64 65 plus

Reduced quality of life as a result of extended life

Measured as quality-adjusted years of life (as follows) and discounted at 5% annually Death = 0.0 Bed disability = 0.4 Other disability = 0.6 Full function = 1 . 0

Net Effect: Quality-adjusted Days per Person Vaccinated

Cost per Quality-adjusted Year of Life

0.05 0.07 0.15 0.43 1.59

$77,200 35,300 22,900 5,700 1,000

From Willems et al., "Cost effectiveness of vaccination against pneumococcal pneumonia," New England }. ofMed. 303: 553-559, September 4, 1980.

14


15

costs of vaccination. These are the losses caused by vaccination minus the financial gains from the procedure. Note, in particular, that if persons do not die of pneumonia and as a result live longer and consequently incur other costs due to illness, this is counted as loss attributed to vaccination. This attribution illustrates an important principle: namely, that improvement in quality can result in added costs because of resulting longevity. The 5% discount rate applied to the monetary gains and losses of vaccination embodies still another principle or presumption advanced by economists: namely, that losses and gains further and further in the future have a progressively lower value to persons living in the present. It is not clear how much lower that value is. The figure of 5% is a conventional estimate, the rationale for which I shall not attempt to describe. In the right-hand portion of Table 1.2 the effects of vaccination are shown. These appear as both gains and losses. Once again we find that some improvements in quality, while they enhance the quality of life, can also have the opposite effect. In order to add up these various effects into a single sum, a method of measurement is needed that can be compared to the single measure of monetary cost — whether that cost is expressed in dollars or some other unit. The single measure of effects used in Table 1.2 is the "qualityadjusted year of life." This measure is derived by attributing different weights to different states of health so that a total can be computed. The weights in the table are subjective estimates. This, in addition to the roughness in the categorization of health states and the relative arbitrariness of the discount rate, introduces yet another element of uncertainty to the analysis. To this uncertainty must be added the error that generally pervades all estimates of loss and gain because information about them, as they occur over a lifetime, is usually incomplete. But these defects, important as they are factually, need not concern us too much, because what we are after now is to demonstrate a way of thinking about costs and effects rather than an effort in actual measurement. Assuming that these estimates of the costs and effects of vaccination are reasonable, we can compare them as shown in the lower panel of Table 1.2. This is a comparison of the costs and effects of vaccination at

16


different ages. In the young, health gains are relatively small and costs high. In the aged, costs are lower and gains higher. Such a comparison can help in deciding to which age groups it would be socially appropriate to offer vaccination based on how much society is willing to pay for a given gain in a year of quality-adjusted life. The comparison does not, however, fix the optimal point below which gains exceed losses and above which losses exceed gains. To do that, it would be necessary to set a monetary value for each year of life gained at each age. So that benefits (or "gains") expressed in dollars can be compared to losses, also expressed in dollars. I am not prepared to describe the methods by which economists have proceeded to convert years of life into their equivalent in dollars. I must, however, warn the reader that these methods invariably rest on assumptions that have serious ethical implications. Now that the reader has a better understanding of the concept of optimality, it is time to move on to some further implications of the model shown in Figure 1.3. One consequence of this model is the presence of two standards of quality: "maximally effective care" and "optimally effective care." Which of these two standards is the one to be used in defining and "assuring" the quality of care? As I shall show a little further on, when I discuss the attribute of "social legitimacy," the answer depends on whether individual or social interests are to be safeguarded. Still other implications of the model presented graphically in Figure 1.3 have a bearing on important aspects of the relationship between cost and quality. First, as I have already shown, if optimality is chosen as the standard of quality, any expenditures beyond those required to achieve the optimum can be regarded as wasteful. In other words, beyond the optimum, quality can be considered to be lower rather than higher. Second, if maximally effective care is taken as the standard of quality, expenditures below that point are justified. Only expenditures beyond that point are considered wasteful. And if they result in harm, care is not only wasteful but also of lower quality. It will be recalled that in order to generate the curves in Figure 1.3 it was assumed that the care given was that by an "ideal physician" who consistently avoided all useless care. In actual practice, in any community, most physicians depart from the ideal, sometimes to a remarkable degree.


17

Consequently, the improvement in health is not as rapid or as large, and where resources are plentiful (in a relatively wealthy country, for example) practitioners are thought to offer care not only beyond the optimal but beyond the maximal as well. Useless or even harmful care is piled upon care beyond the point of any reasonable hope for improvement. In such situations it is believed that restrictions on the cost of care can produce a significant improvement in quality. But this is only true if the consequent reductions in care are judicious, so that only useless or harmful care is omitted while everything useful remains — a consequence by no means certain. By contrast, where resources are limited and, frequently, improvements in health are below the optimal, more care, and costlier care, rather than less are needed. But even in such situations, what resources are available could be more effectively used if clinical and production efficiency were improved. This brings us to still another question. Are considerations of cost relevant to the definition of quality? Or, on the contrary, should quality be defined independently of cost considerations? In other words, is cost an ingredient in quality itself or is it simply the price at which quality is bought? On this point opinions differ; and it does not matter much what position one takes as long as cost and quality are assessed simultaneously. I have argued that, in some ways, cost is an ingredient in the definition of quality itself. This is obviously true when practitioners give care more likely to be harmful than useful. It could be argued that this is also true if the care given is not harmful but merely useless. Besides questioning that there is such a thing as "merely useless," (since almost everything we do could be harmful) we could argue that such care shows inattention, ignorance, or carelessness on the part of the practitioner, and for those reasons, can be considered to be of poor quality. Moreover, because "useless" care for some leaves less for others who could have benefited from it, such care is socially irresponsible and therefore reprehensible. Having completed our discussion of efficiency and optimality we are ready to move on to the fifth component or attribute of quality, as listed in Figure 1.1.

18


Acceptability Acceptability is defined as conformity to the wishes, desires, and expectations of patients and responsible members of their families. I shall develop this definition in five parts: 1. Accessibility 2. The patient-practitioner relationship. 3. The amenities of care 4. Patient preferences regarding the effects, risks, and cost of care 5. What patients consider to be fair and equitable I shall briefly comment on each of these. Accessibility By accessibility I mean the ease with which persons can obtain care. This depends on spatial factors such as distance from the sources of care and on the availability and cost of transportation; on organizational factors such as the days and hours when sources of care are open to receive patients; on economic factors such as income and the possession of health insurance; on social and cultural factors such as the ethnic or religious preferences of those who seek care or the biases of those who provide it.6 We could debate whether or not accessibility is properly an aspect, attribute, or component of quality or, on the contrary, whether it is only an attribute of care separate from quality. Irrespective of such debate, it is true that people continue to be vitally concerned about their ability to get care when they want it and with how easily and conveniently it can be obtained. And, as we shall see soon, accessibility is also a critical component in the social acceptability of care. The Patient-Practitioner Relationship It is not easy to list all the properties that stand for goodness in the patient-practitioner relationship. As a beginning, mainly to stimulate the


19

reader's thinking, I offer the attributes listed in Table 1.3. I shall not go over these in detail. To put it briefly: personal concern, empathy, respectfulness, avoidance of condescension, willingness to take time, effort to explain, attention to the patient's preferences, honesty, truthfulness, and plain good manners are essential ingredients in good care. To begin with, these attributes are desirable in their own right. They embody cherished values of the health care professions, and include desirable characteristics of every interaction in a civilized society. Moreover, when such attributes are present, patients are not only pleased, but also reassured. They regard their presence as evidence (which they understand) that technical care (which they do not understand so well) will also be good. For these reasons, when patients have a choice, and when competition among providers of care is allowed, the attractiveness of the patient-practitioner relationship becomes a key to success among competing organizations. To emphasize this point, some have taken to using the terms client or customer, rather than patient, to designate those who Table 1.3. Some Attributes of a Good Patient-Practitioner Relationship Congruence between therapist and client expectations. Adaptation and flexibility: the ability of the therapist to accept his or her approach not only to the expectations of the client but also to the demands of the clinical situation. Mutuality: gains for both therapist and client. Stability: a stable relationship between client and therapist. Maximum client autonomy, freedom of action, and movement Maintenance of family and community communication and ties. Maximum egalitarianism. Active client participation through shared knowledge concerning the health situation, shared decision making and participation in carrying out therapy. Empathy and rapport without undue emotional involvement of the therapist. A supportive relationship without undue dependency. Confining therapist and client influence and action to the boundaries of their legitimate social functions. Avoidance of client and therapist exploitation economically, socially, or sexually. Maintenance of client and therapist dignity and individuality. Privacy. Confidentiality. From Donabedian, A., "Models for Organizing the Delivery of Personal Health Services and Criteria for Evaluating Them." Milbank Memorial Fund Quarterly 50: 103-153, October 1972, Part 2.

20


are likely to need care or actually to seek it. In this way, one recognizes the health care system as a market rather than a social organism motivated mainly by its internal commitments and values. Whether those who seek care are called clients, customers, or patients, it is reasonable to expect a link between the goodness of care and the goodness of the patient-practitioner relationship. This is because this relationship is the vehicle by which technical care is implemented. A good relationship motivates the practitioner to do well. It also motivates the patient to cooperate, so that the effectiveness of care is enhanced. In fact, in many cases, the proper management of the patient-practitioner relationship is in itself the most important technique of care. Finally, the patient-practitioner relationship is perhaps the most sensitive indicator of the persistence of differences adverse to the underprivileged in the organization and delivery of care. Such differences are the last to disappear when the goal is to put all patients on an equal footing with regard to the care offered to them.

The Amenities of Care The amenities of care are the desirable aspects of the circumstances under which care is given. They include properties such as convenience, privacy, comfort, restfulness, cleanliness, the availability of adequate parking (in a motorized society), the availability of refreshments, good food, and so on. These features are adjuncts to the desirable aspects of the patientpractitioner relationship and, like the latter, help make the experience of receiving health care either pleasant and rewarding or unpleasant and humiliating. Because I have put so much emphasis on the patient-practitioner relationship and the amenities of care, I should also mention some dangers of an over-reliance on these in judging the quality of care. One should be alert to the possibility that attractive surroundings and a pleasant "bedside manner" might cover up, or substitute for, deficiencies in technical care because it is expected that patients will readily appreciate and be gratified by the former but will be less likely to understand and ap-


21

preciate the latter, especially when the consequences or success or failure in technical care are delayed and responsibility for them obscured. It is, moreover, difficult to set up preconceived specifications of the patient-practitioner relationship as attributes of its goodness. This is because patients differ so widely in their expectations and preferences. Some would prefer a cool, distant, authoritarian practitioner, whereas others would wish to have one who is warm, empathetic and friendly. Some patients prefer to be told; others wish to be consulted. This means that the criterion of quality in the patient-practitioner relationship could be its congruence with the patient's expectations and preferences, rather than with a list of preconceived attributes. But one cannot completely relinquish the latter. The expectations and preferences of patients are heavily conditioned by personal experiences in privileged or underprivileged circumstances. Therefore, norms of what the patient-practitioner relationship, and of the amenities of care as well, should be, continue to be necessary. There is still another difficulty in being guided only by what patients like or desire. In some cases, patients wish to have treatments or procedures that are not good for them, or are unnecessary, or are socially undesirable. The conscientious practitioner cannot relinquish the standards of what is right and moral. The cost of care, and the priorities imposed by it, is still another consideration. Some amenities can be costly to institute and maintain. The patient-practitioner relationship, if it is to be satisfactory, cannot be hurried; it requires time and is therefore costly. The benefits derived from incurring these added costs will have to be compared with those of the alternative uses resources could be put to.

Patient Preferences Regarding the Effects, Risks, and Cost of Care Patients agree with their practitioners in recognizing the effects on health as the primary attribute of quality in health care. And, as their practitioners also do, patients compare the expected improvements in health to the risks associated with care. But, it is also important to recognize that patients may value each of these consequences of care (effects, risks, and

22


cost) in a way that differs from the valuations of their practitioners. Moreover, patients differ greatly from one another in their valuations. When these differences arise out of ignorance of the consequences of the care contemplated or received, education is the remedy. When they arise out of economic considerations, the social financing of care can help reduce the differences. But even then, differences among patients persist. For example, some are anxious to avoid an immediate risk (let us say, from a surgical operation) whereas others are willing to accept that risk if the prospect of subsequent longevity is improved.7 Similarly, some patients would prefer a shorter life of higher quality, whereas others would prefer longer survival even if the quality of life is relatively low.8 The consequence of the preceding considerations is that practitioners should take time to explain to patients (or their relatives when patients are unable to fully understand the situation) the expected cost, risk, and effects of alternative methods of care, and be guided by the informed opinion of these parties.9

What Patients Consider to Be Fair or Equitable Patients have decided opinions about how fairly or equitably they are treated when it comes to such things as access to care, the patientpractitioner relationship, the amenities, and the expected benefits from care. Because the concepts of fairness or equity, even when individually assessed, are matters of social concern, I shall deal with the subject when I consider the social aspects of quality.

Legitimacy Legitimacy is defined as conformity to social preferences, as expressed in ethical principles, values, norms, laws, and regulations. Briefly, it can be regarded as social acceptability, the property that corresponds to "acceptability to individuals." In a democratic society, where patients and potential patients can express their wishes through a representative political process, individual and social concerns can be expected to correspond,


23

and, in general, this is the case. Nevertheless the specification of quality can differ depending on whether it is viewed by individuals making decisions for themselves or by society when it seeks to advance the welfare of an entire community. Consequently, society can arrive at different conclusions in determining which kinds of care are most effective, efficient, optimal, or equitable. How does that happen? The disparity between individuals and collectivities in their views of efficiency and optimality arise because society has different estimates of costs and effects. As regards costs, individuals are concerned with what they themselves pay, whereas society takes into account as well the costs it assumes in the shape of government programs, health insurance, and the like. As regards effects, individuals are concerned with what pertains to themselves. They are less mindful of harm or benefit to others (beyond their own families) when they themselves either receive care or do not receive it. Society takes the broader view. For example, it may be important to society that individuals be immunized in sufficient numbers to prevent the spread of disease even when some individuals do not wish it. In other words, society takes into account the "externalities" (as economists call them) of the actions of individuals. Moreover, society may differ from individuals in its assessment of effects by placing greater or lesser valuations on the health and longevity of certain individuals than of others. One example is greater attention to the health and welfare of children as compared to that of old people because children have an entire lifetime ahead of them; or more attention to wage earners because they contribute more to the economy. The consequence of such disparities as I have described is conflict between the interests of individuals and the public, at least in the short term. This conflict can be resolved only by political and social consensus in a free society. But even if this occurs, a heavy burden may be placed on health care practitioners if they are expected, individually, to carry out social policy as they care for individual patients, allowing more care for some while denying some useful care to others. Such rationing decisions should be made by regulations that transcend the particular patientpractitioner transaction.

24


Equity Equity is defined as conformity to a principle that determines what is just and fair in the distribution of health care and of its benefits among the members of a population. Although equity is an important determinant of individual and social acceptability, I have thought it worthy of separate mention as a component of quality. Obviously, equity depends first, on access to care, and second, on the effectiveness and acceptability of the care received. As a general rule, the aim is to erase all differences in these regards between population groups characterized by age, sex, income, social class, ethnic origin, place of residence, and so on. But, once again, we must recognize that individual and social preferences can intrude to disrupt the balance of what could be considered equitable. As I have already said, individuals have decided views of what equitably should be theirs. But society may determine, in the pursuit of equity, that persons already privileged should have less care than they want, so that others, heretofore deprived, should receive more. Attention to what is effective, efficient, or optimal can also intrude on determinations of social equity. For example, it may be concluded that certain categories of persons should have more care because the results in health improvement are significantly greater when compared to cost. The relative emphasis on prevention as compared to treatment is another example of this kind of reasoning. One can ask, therefore, whether or not everyone should receive equal care relative to "need," whether need is determined by what individuals want or what health professionals estimate. One can also ask whether, rather than distributing care according to "need," it should be distributed according to the expected benefits from care. These issues are to be settled by social consensus in a free society.

Choosing a Definition of Quality I have already said that agreement on a definition of quality is one of the foundations of an effort to assure quality. But, having seen how many


25

different attributes go into the definition of goodness, one must wonder about the relative priority to be assigned to each of the several attributes — what to include or exclude, where to begin. In my opinion, there is no general answer to these questions. Much depends on the context for assessment; on what is most relevant, on what one is responsible for, on what one can control. Merely as an illustration of this viewpoint, I offer Figure 1.4, which is meant to suggest what components of quality are most pertinent at successively more inclusive levels of attention, responsibility, and control. I use as a visual device a set of concentric semicircles, at the core of which is the patient-practitioner

Figure 1.4. Quality assessment at successively more inclusive levels.

26


interaction. Beyond the core, in order, is first, the care implemented by the patient, and then, the care received by the community. This order is not invariant. I adopt it because it is congenial to a health care practitioner. With regard to the care provided by physicians and other practitioners, attention focuses on the effectiveness and efficiency of technical care, and on the patient-practitioner relationship. Technical care depends on the knowledge, judgment, and skill of those who offer it. The amenities of care are at one remove from the health-care transaction itself. They depend on factors generally beyond the direct control of the practitioners, unless these also own the firm that provides care or, in some other way, can influence it. When one is concerned with the domain that surrounds the core transaction, something is added. That is, attention to the care implemented by patients and their families when these are removed from the direct influence or control of their practitioners. Clearly, the practitioners may perform faultlessly but care can be ineffective if patients do not do their share. When attention focuses on the care received by a community as a whole, a new set of concerns is added to those already mentioned. These include access to care, with due regard to effectiveness and equity. And, as I have already described, new specifications of optimality are likely to emerge. The figure I have offered is, of course, nothing more than a sketchy illustration. It is far from an exhaustive description of how the attributes of quality that are most salient vary from situation to situation. The lesson to be learned is that one must focus on what one is responsible for, what one can improve, and what is most in need of improvement.

Steps in Monitoring and Improving Clinical Performance Again, my intent in this little book is to describe briefly how one may assure the quality of health care by monitoring clinical performance and


27

improving it when necessary. The steps one might take in carrying out this intention can be listed as follows: 1. 2. 3. 4. 5. 6. 7. 8. 9.

Determining what to monitor Determining priorities in monitoring Selecting an approach (or approaches) to assessing performance Formulating criteria and standards Obtaining the necessary information Choosing when to monitor Choosing how to monitor Constructing a monitoring system Bringing about behavior change

I shall now deal with each of these in turn.


2 Determining What to Monitor

30

I


shall deal with my first subject, which is determining or finding what to monitor, by going through a series of logical steps toward making that determination.

Externally Required Monitoring Activities Quite often, and increasingly so, certain monitoring activities are required by governmental agencies that provide care, directly or indirectly; pay for it; and assume responsibility for its quality. When private insurance is available, the insurers themselves may require certain kinds of monitoring to control the cost of care and possibly also to supervise its quality. It is obvious that any monitoring system will find it necessary to comply with such requirements. But, in my opinion, the more important and meaningful activities are those that the providers of care establish and carry out themselves as part of their own responsibility to provide good care.

Internally Motivated Monitoring Activities It is the purpose of all monitoring activities to identify instances or situations where the quality of care falls below the level expected or desired. These pose a "problem" to be solved. Some have wished, rather, to focus not on "problems" but on so-called "opportunities for improvement." Possibly the difference is more a matter of words than of realities; or this phrase could represent a meaningful difference in orientation. In the first instance we could be saying: "We are not doing well; we should correct our errors." In the second instance we could be saying: "We are doing well, but could we be doing better?" I shall leave it to the reader to decide if the difference is important or not. As I proceed, I shall deal with both problems and opportunities, but most of the time I shall use "problems" to stand for both. I conceive of problem identification to be of two kinds. The first I shall call troubleshooting, a popular American expression to denote action

Determining What to Monitor

31

in response to a clear problem by someone qualified to solve it. In this case the problem imposes itself; it finds us. The second approach to identifying problems or opportunities for improvement I shall call planned reconnaissance. This means that we take action to find such problems and opportunities that we did not know about or we only suspected but needed to confirm and document. Troubleshooting Troubleshooting is the action taken by clinicians or administrators when a problem is presented to them by some untoward event. Things go wrong; people complain; there are unpleasant, even tragic, events. When a seemingly healthy person unexpectedly dies or the wrong limb is amputated, the problem arrives at our doorstep unsolicited. But to encourage the reporting of other, less notable adverse events, it is important to establish an environment conducive to reporting. To create such an environment, we should be prepared to ask questions, and when told we should be willing to listen. The reporting of problems, whether solicited or unsolicited, should never be met with immediate denial, displeasure, anger, argumentation, accusation, retaliation, or punishment. One should listen, explain if there has been only a misunderstanding, or promise to investigate when this seems justified. I believe it would be helpful to ask the bearer of the report his or her view of how the problem may have arisen and how it might have been prevented. When possible and appropriate, the person reporting could be invited to participate in the investigation that follows. At the very least, those who have made a report should be informed of what the subsequent investigation has revealed and what corrective action has been taken. If this procedure is ignored, reports and suggestions will simply cease, even as dissatisfaction and apathy persist and grow. When adverse events are reported, there is pressure to deal with each event as it arises. But it is important, at the same time, to try to understand the more general significance of discrete events. One should look for the possible presence of a "mechanism" that may underlie a series of events, and if so, aim to deal with that root cause.

32


Planned Reconnaissance I have already defined planned reconnaissance as action taken to reveal problems or opportunities for improvement. In this case, we take the initiative to find the problems or opportunities. The kinds of action we might take can be classified as problem identification (1) by group discussion and study, and (2) by routine surveillance. Problem identification by group discussion and study. Small groups of health-care professionals, joined perhaps by administrators and other support personnel, can meet to review the work of a department or unit and to suggest ways that may improve performance. Such groups have been called "quality circles" or "quality improvement teams," or given other names as the fashion of the times dictates. A quality circle was intended by Ishikawa,10 the originator of this social device, to be a rather small group of not more than ten workers, all in the same workshop, who meet voluntarily at least twice a month, under a leader whom they themselves elect, to study the principles and methods of quality monitoring in general, and to engage in quality control activities specific to their own work. These activities include identifying problems of quality, understanding their causes, proposing and implementing corrective action, and evaluating the results of such action. Although the activities of each group are limited to a small unit in the organization and are ostensibly voluntary, the activities of the several quality circles in an organization are promoted, supported, and coordinated by the quality assurance directorate of that organization. Moreover, the quality circles in any one organization interact with the quality circles in other similar organizations through a national organization of quality circles and its regional branches. Thus, quality circles assume the character of a movement whose purpose is to enhance the self-image and status of line workers through their involvement in quality improvement activities. Because this model was developed for industrial settings, such as factories, its applicability to the health-care system might be questioned, unless the model is significantly modified. Perhaps for that reason, or out

Determining What to Monitor

33

of a lack of vision, I know of no instances of the model's thoroughgoing application to health care. Nowadays, almost any group that undertakes quality improvement activities, even if it does not comply to the original model, can be called a "quality circle" or some other corresponding name. Routine surveillance. The second form of planned reconnaissance can be called routine surveillance, a category that includes two subdivisions: opinion surveys and performance monitoring. Opinion surveys are an established method of social research. There could be surveys of the opinions of patients, other clients, practitioners, managers, and anyone else who can contribute to an understanding of how a health-care system has performed. Surveys may be conducted by telephone, mail, or personal interview, using highly structured or loosely structured questionnaires. The former asks for answers to specific questions by providing spaces which a respondent checks. The latter allows opportunities for the respondents to describe their thoughts or experiences. The survey can be conducted at home or during a visit to a healthcare facility; and it can pertain to different periods in the course of the care received. However it is conducted, the design and interpretation of a survey, if it is to be free of error and bias, requires advice and direction from a qualified social scientist. As to performance monitoring, the second category subsumed by planned reconnaissance, it is perhaps useful to think of it as taking one of two forms: "clinical or anecdotal" or "statistical or epidemiological." Clinical or anecdotal monitoring takes the form of case reviews such as clinical audits. One example would be the review of medical charts by a group of physicians in order to see if the care, as recorded, has been acceptable or, in some respects, could have been improved. The other form of performance monitoring, the form I have called "statistical or epidemiological," envisages a flow of information regarding critical activities and significant outcomes of care, so that unusual patterns can be detected and studied. The "tracer method" as an example of planned reconnaissance. The tracer method begins with an assumption that the quality of

34


a bundle of related clinical activities can be represented by one activity in the bundle or, at most, by a very small number. If this is true, one can select one tracer, or a few, to represent the entire category they belong to. For example, one could select screening for visual defects in children to represent the entire category of screening activities in children. Similarly, immunization against measles, for example, could stand for all immunizations. And in adults, screening for breast cancer in women and for prostate cancer in men could stand for case-finding as a whole.11 In order to use this presumption to map the quality of care in a larger context, one would need, as a first step, to construct a matrix, or map of the terrain to be explored. In the above-mentioned examples, I have assumed that care could be divided into preventive and therapeutic, into care for children and adults, and, among adults, into care for males and females. This illustration of mapping is, of course, only an example. The "terrain of care," if one can use the term, can be mapped in a large variety of ways, depending on what precisely the objectives of monitoring are. I shall now offer a more fully developed application of the tracer method. We shall assume that our object is to monitor the quality of care in the emergency service of a hospital. If so, one might begin by visualizing what activities go on in such a setting, perhaps by constructing a diagram that represents the flow of these activities. Such a diagram, offered merely as an illustration, is shown in Figure 2.1. The figure is constructed by assuming that patients come to the emergency service in one of two ways: by walking in or by ambulance. For simplicity, I shall follow the progress of the walk-in patient. First, there is a sorting process (or triage) that separates out patients who need immediate stabilization (which is carried out) and those who do not. This is the first phase. In the second phase, a diagnosis is made and interim treatment started accordingly. In Phase III, after the patient's progress has been observed, more definitive (basic) treatment decisions are made; and subsequently, in Phase IV, a decision is made as to whether the patient is discharged or, alternatively, referred to one of the facilities shown in the figure. Having constructed such a flow-diagram one can begin to consider

Figure 2.1. One possible representation of the progression of care in a hospital's emergency unit. (From Rhee, K. J., Donabedian, A., and Burney, R. E., "Assessing the Quality of Care in a Hospital Emergency Unit: A Framework and its Application," Quality Review Bulletin, January, 1987.)

35

36


what the objectives of care are at each phase of a patient's progression and, in general, throughout all phases. In Table 2.1 there is an illustrative listing of such objectives. These are used to indicate the basic aspects of care that should be considered in making a judgment about quality. In Table 2.2 one finds an illustration of one possible tracer matrix; many other mappings are possible. In this partial mapping, the stub of the table shows the "phases" in the flow of patients, as well as the activities during each phase, as already described. The headings of the table specify aspects of care as "physical-physiological," "psychological," and "socialenvironmental." Let me again emphasize that many other headings could have been devised. The intersection of the categories in the stub of the table with those in the headings creates a matrix of 24 cells. Now we can select in each of the cells a tracer or tracers that are important on their own account and moreover are thought to represent other conditions in that cell. The next step could be to select cases that fall in any one of the cells of the matrix, and, based on a review of the patients' medical records, by direct observation, or in some other way, we can obtain information about what was done, how promptly, and with what results, using the

Table 2.1. Possible Objectives of a Hospital Emergency Unit 1. Expeditiousness, timeliness, and duration of care. 2. Appropriateness of diagnostic and therapeutic interventions as judged by the greatest net benefit at the lowest cost. 3. The validity of diagnostic decisions. 4. Skill in the execution of diagnostic and therapeutic interventions. 5. Reliability and validity of diagnostic information and monitoring data. 6. Appropriateness of referral. 7. Maintenance of continuity-in-care through linkage and transfer of adequate information to a more stable source of care. 8. Appropriate recording and management of information. 9. Patient education and motivation with a view to prevention. 10. Discharge of legitimate organizational and social obligations with due regard to responsibilities toward individual patients. From Rhee, K. J., Donabedian, A., and Burney, R. E., "Assessing the quality of care in a hospital emergency unit: a framework and its application." Quality Review Bulletin, January, 1987.

Table 2.2. An Illustrative Tracer Matrix for Assessing the Quality of Care in a Hospital Emergency Unit Aspects of Illness, Health and Health Care

Phases and Components of the Process of Care

Physical-Physiological

Phase I: Sorting and stabilization

Cardiac arrest, multiple trauma

Acute behavioral disturbance

Acute behavioral disturbance

Cardiac arrest, multiple trauma Chest pain, abdominal pain, musculoskeletal pain, sick child

Acute behavioral disturbance Depression, acute psychosis, bronchial asthma

Acute behavioral disturbance Child abuse, bronchial asthma

Phase II: Diagnosis and interim management Interim treatment Diagnosis Phase III: Basic treatment decisions Therapeutic Preventive

Phase IV: Completion of disposition Referral-linkage

Societal functions

Information management

Psychological

Social, Environmental

Bronchial asthma, poisoning Bronchial asthma, poisoning, upper respiratory, infections, lacerations Poisoning, foreign body, hypertension Poisoning, foreign body, bronchial asthma

Bronchial asthma, rape, assault Poisoning, foreign body, bronchial asthma, child abuse, spouse abuse

Any of the above, specified illness among vulnerable population groups (e.g., aged, poor, isolated) Reportable infections, conditions requiring genetic counseling, acts of violence, motor vehicle accidents Any of the above

Any of the above, vulnerable population groups, substance abuse Child abuse, spouse abuse, rape, assault, suicide

From Rhee, Donabedian, and Burney, Quality Review Bulletin, January 1987.

Any of the above vulnerable population groups Cases requiring commitment

Any of the above

Any of the above

38


objectives of care listed in Table 2.1 as a guide. Clearly, to explore the entire matrix as depicted would be a horrendous task. I know of no instance of such a complete exploration. What is more likely to be done is to select only a few conditions to begin with and perhaps at some later date to select a few others. The task of selection is made easier by observing that some conditions appear in several of the cells of the matrix and, therefore, can cast light on more than one aspect of emergency care. But even then, one needs to know what is more important and what is less, the more important being a candidate for immediate attention and the less important postponed. To this pressing subject of importance rating we must now turn.

3 Determining Priorities in Monitoring

40


A

nyone setting out to monitor the quality of care is likely to be confronted with many things vying for attention; so many that one may not know where to begin. It is important, therefore, to have some guidelines for selection. In this section of my exploration, I shall try to offer such a guide. For convenience of discussion, I shall organize it as follows: importance to patient welfare . . . additional institutional interests . . . feasibility. These I shall take up in order.

Importance to Patient Welfare The rule that governs importance to patient welfare may be summarized by the expression "maximum achievable benefit," a guideline introduced by John Williamson. Accordingly, the characteristics of a problem that determine its importance are as follows:12-13 1. The problem is believed to occur frequently. 2. Error or failure in performance is known or believed to occur frequently. 3. When it occurs, such error or failure in performance is believed or known to have serious consequences to health, and is costly. 4. The error or failure in question can be rather easily corrected. I know of no generally acceptable formula that assigns weights to these several properties so that they can be summarized by a single index of importance. That is a matter for informed judgment. Briefly, one could say: select what is frequent, grievous, and correctable.

Additional Institutional Interests Because the welfare of patients is the primary responsibility of providers, whether these are institutions or individuals, "importance," as described above, continues to be the paramount concern of all providers. That said, one should be aware that other considerations can also influence insti-

Determining Priorities in Monitoring

41

tutional decisions on what requires more urgent attention. I list and discuss some of these as follows.

Meeting Externally Imposed Requirements By virtue of their legal and administrative consequences, such requirements acquire almost automatic priority even when, if the requirement did not exist, the institution would have preferred to direct its attention elsewhere.

Risk Reduction There is, of course, a significant degree of correlation between the adverse experiences of patients and consequent risk to the organization. Nevertheless, in assessing risk, attention focuses mainly on whether or not the institution is likely to be sued, attract the unfavorable attention of regulators or insurers, or lose its good reputation. In countries where litigiousness is rampant, the prevention of being sued is the major consideration in risk management. If a suit is successful, large penalties may be assessed. And even if unsuccessful, a suit is costly in legal fees and other expenses, raises the cost of insurance against malpractice (actual or alleged), and damages the institution's reputation.

Institutional Enhancement The reduction of risk contributes to patient welfare, reduces cost to the institution, and protects its reputation. In a previous section I also pointed out that cost reduction could be attained by the exclusion of harmful or "useless" care, a goal quality monitoring should contribute to. The reduction of risk and cost, as well as the improvement of quality, enhance the reputation of an institution and attract patients, practitioners, and other resources to it. The very fact that an institution demonstrates its dedication to quality by having established a mechanism to watch over it, if made known to the public, recommends the institution to those who may wish to use it.

42


Representativeness It is not often that truly representative sampling, in the statistical sense, is used in quality monitoring. Reasonable precautions against bias, for example in selecting medical records for review, are generally sufficient. Perhaps this is because quality monitoring is not perceived as research. It is, ordinarily, a much less rigorous effort to collect information that reveals the kinds of care that need attention in a specific setting. Precise quantification of problems is not needed, nor is there any effort to generalize. With that having been said, monitoring systems should have in mind all the clinical activities of an institution and attempt to include all these as actual or potential subjects for performance assessment. This calls for some scheme of what may be called illustrative sampling, of which the tracer method already described could be one example.

Fairness A plan for quality reconnaissance should also be alert to yet another principle: that of fairness. I mean that monitoring should not be concentrated so heavily on some units, functions, or individuals to be seen as unduly biased or bordering on the persecutory, unless there is clear reason for such concentration of attention. One advantage of aiming for representativeness is to avoid the perception of unfair targeting.

Feasibility Not every kind of monitoring one might wish to institute is feasible or practicable. There could be a number of obstacles or, on the contrary, facilitators, that influence when monitoring begins and what directions it might take. These include organizational readiness and operational feasibility. Organizational readiness. It is best, when one considers where and what to monitor, to search for units in which there is a perception that quality needs to be improved, a desire to improve, and a willingness to

Determining Priorities in Monitoring

43

institute monitoring. Often this will depend on the presence within a unit (or the institution as a whole) of one or more respected leaders who are willing to cooperate. It is all the better if they themselves request that monitoring be instituted. Operational requirements. A number of "operational" or "technical" requirements can also influence the way one begins and continues to monitor. What one chooses to assess should lend itself to valid judgments of quality because the science of health care is so well developed that one knows what is good or not good to do or accomplish. This is reflected, as we shall see, in one's ability to formulate valid criteria of quality. Moreover, there should be accurate and rather easily available information concerning the activities and results of care. And if the organization desires to judge primarily the performance of the practitioners themselves, the care implemented and the results obtained should not depend too much on the cooperation of the patient. In other situations, of course, the degree of patient cooperation would be important to include, or it could be the major focus of attention.


4 Selecting Approaches to Assessing Performance

46


A

s the reader will recall, selecting approaches to assessing performance was the third in my list of steps to be taken in order to .institute a monitoring activity. In other words, we need to have some way of finding if the quality of care has been "good," "fair," or "poor." More than 30 years ago, I suggested that there are three approaches to assessing the quality of care. These I called "structure," "process," and "outcome": a triad that, perhaps because of its simplicity and almost intuitive persuasiveness, has gained widespread acceptance, even if it has not always been well understood or properly used. So I shall begin by saying what I take each of these terms to mean and, then expand on the subject, albeit briefly.

Definitions of Structure, Process, and Outcome Structure This is meant to designate the conditions under which care is provided. These include: 1. Material resources, such as facilities and equipment 2. Human resources, such as the number, variety, and qualifications of professional and support personnel 3. Organizational characteristics, such as the organization of the medical and nursing staffs, the presence of teaching and research functions, kinds of supervision and performance review, methods of paying for care, and so on.

Process This is taken to mean the activities that constitute health care — including diagnosis, treatment, rehabilitation, prevention, and patient education — usually carried out by professional personnel, but also including other contributions to care, particularly by patients and their families.

Outcome These are taken to mean changes (desirable or undesirable) in individuals and populations that can be attributed to health care.

Selecting Approaches to Assessing Performance

47

Outcomes include: 1. Changes in health status 2. Changes in knowledge acquired by patients and family members that may influence future care 3. Changes in the behavior of patients or family members that may influence future health 4. Satisfaction of patients and their family members with the care received and its outcomes. A more complete and detailed classification of outcomes appears in Table 4.1. I shall now expand on these rather sketchy descriptions of the three approaches by commenting on their implications, discussing their relative uses and limitations, and offering some conclusions on the choice of one or more of them.

Comments on the Structure-Process-Outcome Model Because the three-part classification of approaches to assessment is not always properly understood, I need to make some brief comments on what it is and is not. 1. Structure, process and outcome are not attributes of quality. They are only kinds of information one can obtain, based on which one can infer whether quality is good or not. 2. Inferences about quality are not possible unless there is a predetermined relationship among the three approaches, so that structure influences process and process influences outcome, as shown in the following simple diagram: P process —> P outcome Structure —>

Of course, the linear relation depicted above is a much simplified version of a much more complex reality in which certain causes have their effects, themselves become causes to subsequent effects, and so on

Table 4.1. Classification of Outcomes A. CLINICAL 1. 2. 3. 4.

B.

C.

D.

E.

F.

G.

Reported symptoms that have clinical significance Diagnostic categorization as an indication of morbidity Disease staging relevant to functional encroachment and prognosis Diagnostic performance — the frequency of false positives and false negatives as indicators of diagnostic or case finding performance PHYSIOLOGICAL-BIOCHEMICAL 1. Abnormalities 2. Functions a. Loss of function b. Functional reserve — includes performance in test situations under various degrees of stress PHYSICAL 1. Loss or impairment of structural form or integrity—includes abnormalities, defects, and disfigurement 2. Functional performance of physical activities and tasks a. Under the circumstances of daily living b. Under test conditions that involve various degrees of stress PSYCHOLOGICAL, MENTAL 1. Feelings — includes discomfort, pain, fear, anxiety (or their opposites, including satisfaction) 2. Beliefs that are relevant to health and health care 3. Knowledge that is relevant to healthful living, health care, and coping with illness 4. Impairments of discrete psychological or mental functions a. Under the circumstances of daily living b. Under test conditions that involve various degrees of stress SOCIAL AND PSYCHOLOGICAL 1. Behaviors relevant to coping with current illness or affecting future health, including adherence to health-care regimens, and changes in health-related habits 2. Role performance a. Marital b. Familial c. Occupational d. Other interpersonal 3. Performance under test conditions involving varying degrees of stress INTEGRATIVE OUTCOMES 1. Mortality 2. Longevity 3. Longevity, with adjustments made to take account of impairments of physical, psychological or psychosocial function: "full-function equivalents" 4. Monetary value of the above EVALUATIVE OUTCOMES Client opinions about, and satisfaction with, various aspects of care, including accessibility, continuity, thoroughness, humaneness, informativeness, effectiveness, and cost

From: Donabedian, "Explorations in Quality Assessment and Monitoring," Volume II, pgs. 367-368.

48

Selecting Approaches to Assessing Performancee

49

in a single chain or a chain with two or more branches. Therefore, it becomes somewhat arbitrary to say where structure ends and process begins, or where process ends and outcomes begin. To some, this ambiguity is distressing, but it should not be. What is more important than a precise separation is the ability to specify how a series of causes and effects is configured. This having been done, one can decide what kinds of information are needed to pass a judgment on quality, and how easily that information can be obtained. After all, the structure-process-outcome model is only a servant, not a master. The reader will find in Figures 4.1 and 4.2 a partial demonstration of the points I have tried to make. 3. It should be remembered that the relations postulated to exist between adjacent pairs in the structure-process-outcome model are not certainties. Rather they are probabilities, hence the lower-case "p" that is placed over each of the arrows in the diagram shown above. These probabilities may be large or small, and they may be well established by scientific evidence or largely presumed. The higher the probabilities are, and the more firmly established they are by scientific evidence, the more credible our judgments of quality can be. On the contrary, the weaker the probabilities and the more imperfectly supported by the evidence, the more tentative the judgments of quality will be. At the extreme, if nothing is known or surmised about the relations in question, no reasonable judgments can be made using this model. 4. A further limitation of the structure-process-outcome model is that it was developed to assess clinical practice. For that purpose, it performs reasonably well. But when the model is used to evaluate activities other than clinical practice, it may work as is, may perform only if modified, or may even fail.

Relative Usefulness of the Three Approaches The three approaches to assessing quality that I have described are not alternatives that yield equivalent results. Each has its own uses and limitations.

50


Figure 4.1. Some hypothetical relationships between characteristics of structure, process, and outcome.

Structure It is generally agreed that structure, meaning the way a health care system is set up, has an important bearing on how persons in that system behave and, consequently, on the quality of care offered and enjoyed. In some ways, structure could be the major determinant of the quality of care that the system, on the average, is able to offer. But detailed variations in many


57

Figure 4.2. Elements of structure, process, and outcome in diagnosis and treatment.

system characteristics have a rather weak relationship to corresponding variations in quality. Therefore, variations in system characteristics, unless they are unusually large, yield only presumptive judgments on the quality of care. Counterbalancing advantages are that some attributes of structure are more readily observable and more easily documented. They also tend to be more stable.

52


Process By contrast, the detailed characteristics of health-care processes can provide discriminating and valid judgments about the quality of care. In some ways, this assertion is self-evident because "quality of care" can be taken to mean "quality of the process of care." But this identity notwithstanding, the judgment of quality is not inherent to the characteristics of the process itself. It derives from a relationship, established in advance, between process and outcome, a relationship that should have been verified by antecedent medical research. In other words, we say such and such characteristics of process signify quality because we know (or believe) that they contribute to desirable outcomes. And, on the contrary, that such and such characteristics of process signify poor quality because they are known (or believed) to result in undesirable outcomes. We see that processes of care are more directly related to outcomes than are the characteristics of structure. They are, by the same token, more able to identify smaller variations in quality. Another advantage is that processes of care are what could be called "contemporaneous"; they are taking place in the now and consequently offer current, even immediate, indications of quality. Moreover, that information is rather easily obtained; for example, from the medical record, or by questioning patients, or by direct observation if the care is supervised. In this respect, it is unfortunate that the medical record does not usually contain information about the patient-practitioner relationship. And while the record may tell us when and by whom something has been done, it cannot say how skillful the execution of care has been.

Outcome There has been much argumentation about the relative merits of outcome as compared to process as means for assessing the quality of care. Outcome assessment is favored because it can be asserted, quite justifiably, that what matters most is the effect of the care on the patient's health and well being. But, it should be remembered that outcomes have been defined as consequences attributable to antecedent care. In other words,


53

if one is to use an outcome to make a judgment on quality, one should be able to say that the care given was responsible for the outcome observed. It is often difficult to make this connection between antecedent care and subsequent outcomes. I have called this the "problem of attribution." The problem of attribution arises, in part, because the relationship between process and outcome is often imperfectly known, and when known, it is a probability that could be small. The probabilistic nature of the relationship between process and outcome means that, in a given case, or in a small number of cases, we cannot be certain that a given set of processes eventuated in one or more specified outcomes. Larger numbers of cases are needed to establish the relationship. Even when a large number of cases has been observed, there is a further difficulty to overcome. This is because patients vary in their medical, social, psychological, and genetic characteristics; features that can, in themselves, influence outcomes either independently of process or by interacting with process. This means that before some outcomes can stand for quality, one must correct for differences among patients in such characteristics, extraneous to the process of care — a procedure known as "case-mix adjustment." I cannot stop now to describe the many methods of case-mix adjustment that have been developed.14 Although these adjustments are always useful, even necessary, when outcomes are assessed, I believe we do not as yet have a method for case-mix adjustment so complete that one can rely entirely on outcome measurement to assess the quality of care. These difficulties are counterbalanced, however, by several advantages in outcome assessment. The difficulty of attribution when outcomes are used is offset by the advantage of what I think of as inclusivity. I mean by this that outcomes gather in themselves, so to speak, the contributions of all inputs into care from whatever source, including the contributions of patients and their families, while assigning to each input a weight proportionate to its contribution. Process assessment cannot easily do this. Another advantage to outcome assessment is that outcomes reflect not only what was done for patients but also how skillfully that was done, an aspect of care that process assessment can also reveal, but only with

54


an added effort to seek the needed information. And many outcomes are subject to being felt and evaluated by patients, so that they can judge the quality of care they have received. But this can also lead to faulty conclusions, because not all outcomes are visible, and outcomes likely to appear further on in the future are as yet unknown to the patient, even if practitioners, with some accuracy, can predict them. The availability of relevant information and the nature of that information should also be considered in assessing the usefulness of outcome as a measure of quality — or, for that matter, of process as well. Outcomes that occur during the course of care can of course be easily observed and assessed. In fact, it is by evaluating these concurrent or intermediate outcomes that practitioners ordinarily guide the conduct of care. But when the relevant outcomes appear after care has been terminated, special effort is needed to obtain the necessary information; for example, by questioning patients or by calling them back for examination. In some cases it is only after years have elapsed that the more remote consequences of care can be observed. And the longer the time elapsed the more there has been an opportunity for factors outside health care to intervene, rendering the relationship between past care and a remote outcome even more questionable. One must also choose how outcomes are to be measured. In this respect, outcomes could be classified into "partial, diagnosis specific," or "inclusive, generic." Partial, specific outcomes are meant to tell us whether or not, in defined situations, specific clinical objectives have been attained — for example, if in hypertensives the blood pressure has been controlled or after a prostatectomy urinary incontinence has resulted. Such measures are more dependent on the quality of care and more sensitive to variations in it. Inclusive, generic measures are meant to provide an estimate of health status without regard to diagnosis.15'16 Mortality and its opposite, longevity, are two such measures. But longevity alone does not give a full picture of health. Consequently a great deal of effort has gone into developing measures of longevity that also take into account various degrees of functional performance or disability, and other aspects of what is called "the quality of life."17'18 The reader has seen an example of one such


55

measure, although crude, illustrated in Table 1.2. A more detailed exploration of what kinds of these measures exist, how they are constructed, and what assumptions they are based on, is beyond the scope of this book. We only need remember that inclusive, generic measures of health and welfare are not very sensitive to variations in the quality of care. Rather, they represent the general state of health and welfare in a society, reflecting a large variety of factors — health care being only one ingredient, and not necessarily the most important. To conclude this brief discussion of outcome as an indicator of quality, I offer the following guidelines as to how it is best used. 1. The outcome selected should be relevant to the objective of care; it stands for what the clinician is aiming for. 2. The outcome must be achievable by good care. This means that the methods for doing this are available and under the control of the health-care system. 3. The outcome, whether good or bad, must be attributable first to health care, and then, to the contribution of the practitioner (or other person) whose performance is being assessed. 4. The duration of the outcome as well as its magnitude should be taken into account. 5. As a corollary, the trade-off between levels and durations of alternative outcomes may be considered. For example, a shorter life at a higher level of function may have to be weighed against a longer life with greater disability. 6. As another corollary, information on the relevant outcome must be available, which is not an easy matter, especially when obtaining the information requires follow-up over long periods. 7. It is necessary to track not only the consequences of taking action but also the consequences of not taking action in order to obtain a complete picture of performance. 8. Finally, the outcome cannot stand alone. The means used to achieve the outcome also have to be considered, unless it is assumed that resources are unlimited, which almost always is far from true.

56


Conclusions About the Choice of an Approach It should be evident from the preceding discussion that each of the three approaches to quality assessment has advantages and disadvantages. This leads to the conclusion that perhaps the best course of action is to use a combination of approaches, the precise mix of which is to be determined by the nature of the problem to be studied and the availability of the information needed. I recommend such a strategy for the following reasons. 1. A combination of approaches allows one to obtain a more complete assessment of quality, because each of the several categories of information (on structure, process, and outcome) perhaps is more indicative of a given aspect of quality than is another category. If so, the combination can help explore more fully the several aspects of quality. 2. A combined strategy can help identify the causes of failures in quality, attributing them to structure, process, or both. This can suggest what corrective measures should be taken to improve quality. 3. If the inferences about quality based on structure, process, and outcome agree with one another, the agreement tends to confirm that the method of assessment is appropriate and the inferences are valid. 4. But, on the contrary, if the conclusions based on structure, process, and outcome do not agree, one is alerted to the possible presence of one or more of the following problems. a. The data on one or more of structure, process, or outcome assessments are incomplete, inaccurately measured, or falsified. b. More specifically, measurements may have been made at the wrong time or in an insufficient number of cases, or without adequate case-mix standardization. This is particularly true of outcomes, but could be true of processes as well.

Selecting Approaches to Assessing Performancee

57

c. The relations postulated to exist among structure, process, and outcome, and used as a basis for inferences concerning quality, have been erroneous or faulty. This could be because existing scientific knowledge has been misapplied or because the existing knowledge, even properly applied, is fundamentally insufficient or flawed. Only further research can remedy this last defect.


5 Formulating Criteria and Standards

60


T

he fourth of the steps in monitoring clinical performance is the formulation of criteria and standards. Clearly, we cannot measure anything, the quality of care included, unless we have something to measure with — a meter stick, for example, or a balance, and a set of weights. Criteria and standards are the tools by which the quality of care is measured.19'20

Definitions of Terms Various meanings have been assigned to the terms criteria and standards. Because there is no agreed-upon usage, it is necessary to define what each of us means by these terms. I define criterion as an attribute of structure, process, or outcome that is used to draw an inference about quality. For example: • A criterion of structure could be the staffing of the intensive care unit. • A criterion of process could be whether or not blood transfusion has been used during surgery. • A criterion of outcome could be case fatality. I define standard to mean a specified quantitative measure of magnitude or frequency that specifies what is good or less so. For example: • A standard for the staffing of an intensive care unit could be: not less than one registered nurse per two occupied beds. • A standard for the frequency of blood transfusions could be: not less than 5% and not more than 20% in surgeries of specified kind in a specified category of patients. • A standard for case fatality could be: no more than 0.1% for a specified procedure (or a set of procedures) in a specified category of patients. Needless to say, the standards I have given are merely illustrative. They should not be taken either as actual or recommended. I should also point out that every criterion is accompanied by a standard that pertains to it.

Formulating Criteria and Standards

61

There is no accepted single word that designates this bonded pair. Sometimes I shall refer to it in hyphenated form as criteria-standards. Norm is yet another term often used in the literature of evaluation that comes to add to the confusion in our nomenclature. As far as I can understand, a norm may be one of three things: (1) a general rule of goodness — for example, to reduce mortality to a minimum; (2) a description of current practice or current outcomes, such as a mean, median or a mode; and (3) a standard based on such practice.

Importance of Criteria and Standards Criteria and standards are vehicles by which the general concepts and attributes of quality that I have described (or others that the evaluator may wish to advance) are translated to actual measurements. The reader could think, for example, about how one might proceed to measure acceptability or equity. It follows from the above that the criteria of quality should correspond as faithfully as possible to the concepts of quality that they are intended to measure. Then the criteria and standards should be specified in a manner that is easy and accurate to measure. The construction and application of criteria and standards are technical matters of considerable complexity. Accordingly, there is a large body of literature that anyone seriously interested in evaluation needs to become familiar with.19'20 In this introductory book, I shall be able to touch on only a few issues of conceptual or methodological interest. Moreover, I shall write only about the criteria and standards of technical care, because it is here that most of the work has been done, to the relative neglect, unfortunately, of acceptability and equity. But I believe that a study of the criteria and standards for technical care can, to some degree, be applicable to the other dimensions of quality as well.

Some Attributes of Criteria and Standards The attributes of criteria and standards are the characteristics they can be distinguished by, and to some extent evaluated. I shall deal with these

62


attributes under seven headings: derivation, degree of specification and explicitness, validity, adaptability to case variation, recordability, stringency, and screening efficiency.

Derivation I shall distinguish two kinds of derivation: normative and empirical. Normative derivation. By normative derivation I mean that the criteria and standards are based on what is known or agreed to be good or acceptable and what is not so. In other words, normative criteria and standards are derived either from direct knowledge of the scientific literature and its findings, or from the agreed-upon opinions of experts and leaders, an opinion presumably based on knowledge of the pertinent literature as well as on clinical experience. Because we are concerned primarily with the monitoring of clinical performance, we tend to confine our attention to the clinical sciences. They tell us what are the acceptable or recommended processes of care and the outcomes consequent to these. But there are, as well, other sciences or bodies of knowledge from which criteria and standards could be derived. The criteria of the goodness of physical structure come, I suppose, from the sciences of engineering and architecture. Presumably, these are pertinent to environmental safety, salubriousness, efficiency of operation, comfort, and esthetic merit. In various ways, these and related properties are no doubt pertinent to acceptability by patients and to health-care personnel as well. Increasingly, architects and engineers have become interested in the relationship of the design of a "microenvironment" (the obstetrical delivery unit, for example) and how clinicians and patients behave. The criteria and standards of organizational structure derive from the relevant organizational and administrative sciences. Presumably these tell us how best to set up and run a health-care system or facility, so as to obtain the most effective, efficient, and acceptable performance.


63

In the same vein, one can ask whether or not there is a science of the interpersonal process from which judgments on the quality of the patient-practitioner relationship could be derived. Perhaps the behavioral sciences could tell us something about what features of the interpersonal process are most related to the effective conduct of care. Other factors that influence personal and social acceptability are more likely to reflect personal preferences and social values. As I shall describe in a subsequent part of this exposition, the validity and persuasiveness of the criteria and standards one uses to assess quality depend heavily on the validity of their scientific foundations and on social consensus. Empirical derivation. By empirical derivation I mean that the criteria and standards are based on existing practice. Briefly, empirical criteria and standards are derived from "what there is"; by contrast, normative criteria and standards are derived "what there should be." Prevalent practice could be used as one source for empirical standards. One could establish the mean, median, or mode of such practice, or alternatively, the top, second, third, or fourth quartiles, and so on. By such determinations one is enabled to say where, in relation to other providers, one's own performance is located — for example above or below the mean; in the topmost quartile or the lowest; among the top 10%; and so on. Further refinements are possible and often needed. For example, in making comparisons one should want to compare like with like, meaning that one should compare one institution with others of similar type. This comparison among peers can occur also within an institution — for example, when aspects of the performance of several physicians (say, prescribing patterns) are ranked from top to bottom and the location of each physician in the array is compared with all the others. Clearly, empirical standards tend to be lower than those derived from normative sources. To correct for this disparity, the standard of performance could be set as the practice of leading professionals or the practice in teaching institutions. It is also possible to compare one's own practice with each of two sets of standards: one empirically derived and one nor-

64


matively derived. The former tells us where we stand relative to others like ourselves; the latter tells where we should be — what we should be aiming for. One hears often nowadays of something called "benchmarking." I understand this to mean a comparison among similar institutions to see which has the best outcomes or the lowest cost when caring for similar patients, and, by examining how practice is conducted in the most successful institutions, to learn how one may improve practice in one's own. (In passing, the reader may note how an old idea can gain allure by being arrayed in a new raiment!)

Degree of Specification and Explicitness The degree to which the criteria and standards are specified and made explicit can be perceived as a continuum ranging from almost none to as complete as can be achieved. This continuum can be roughly divided into four categories: implicit, guided, inferred, and explicit. Definitions. Implicit criteria and standards are not specified. They are the criteria and standards present in the minds of experts who are, for example, given a set of medical records and asked to judge, based on what they know or believe, whether the care in each case has been good, fair, or poor. Sometimes the process of assessment by implicit criteria-standards is "guided" to various degrees by specifying what is to be judged. For example, one could ask: "Is the admission to the hospital justified?" "Is the length of stay appropriate?" "Have the necessary investigations been done?" "Has the treatment been appropriate?" "Are the outcomes (to the extent they have been observed) reasonable?" In particular, one could ask if any aspect of care has been excessive, unnecessary, ill advised or harmful. The reviewers might also be asked to judge the timeliness of care or, as an indication of inefficiency, whether or not there have been unexplained pauses and delays in the progress of care.


65

Those who are asked to review the care in this manner may be asked to simply indicate, all things being considered, if the care can be described qualitatively as good, fair or poor. They might be asked, moreover, to rate performance in each of the several categories (such as the ones I have mentioned) on a scale of 0 to 10 or of 0 to 100. The final judgment of quality could be left to the expert reviewers, individually or collectively, or there could be instructions as to how the several categories are to be weighed and summed. Whether the criteria and standards have been totally "implicit" or "implicit but guided," the experts conducting the assessment could be asked after they make their judgments to record why, or on what grounds, these judgments were made. From these descriptions or justifications one can "infer" what the heretofore implicit criteria and standards have been, and perhaps also judge how credible the judgments have been. Furthermore, one can learn from the nature of the "inferred criteria standards" how more-explicit criteria-standards might be constructed. Explicit criteria-standards are those specified, in various degrees of detail, before the assessment of quality is made. Usually they apply to the process of care, but they can apply to its outcomes as well (and certainly to many structural characteristics also, as attested to by requirements for accreditation). When criteria and standards are considered to be fully specified there is almost no room left for the opinions of reviewers. All that is needed is to determine whether, in fact, the requirements specified have been met or not. And the manner in which a total score of quality is to be derived is also frequently specified by assigning weights to each item in the list of criteria-standards, so that they can be summed up to arrive at a total numerical score. Explicitness and specification should not lead one to believe that clinical judgment has been excluded as a factor in evaluation. The selection of criteria and standards involves judgments of different degrees of subjectivity. And we do not have as yet an objective method for assigning weights to the criteria. These are the product of subjective clinical estimates.

66


Implicit and explicit criteria. The reader will have already concluded that the implicit and explicit formats I have described have their particular advantages and limitations.21 Very briefly, implicit criteria-standards are very flexible; they can be adapted to all the detailed characteristics of any given case, assuming the reviewer is expert, attentive, and thorough. Presumably, the reviewer would have given consideration not only to the appropriateness of what care was included but also to what had been left out. But the application of such criteria and standards is costly. It needs highly qualified reviewers who ought to be given time to do their work. Another drawback is that, quite often, there are large differences of opinion among several reviewers who independently assess any given record of care. This means considerable unreliability in the judgments obtained. Unreliability is reduced by having reviewers who are expert not only in clinical matters but also in the task of conducting reviews, who are highly motivated, and who have received the necessary training. When highly reliable and valid judgments are required, several judges are used to assess independently each record of care; and where significant differences in judgments occur, the several experts are asked to meet and discuss their differences, aiming in that way to arrive at a consensus. By comparison, explicit criteria-standards are difficult to formulate but, once formulated, relatively inexpert persons who have been trained to abstract from medical records the information pertinent to the criteria and standards can easily and inexpensively apply them. Given good records and well-trained abstractors, a high degree of reliability (reproducibility) can be accomplished, to a level considerably superior to that obtained by using implicit criteria-standards. But one can question whether or not the judgments made are as valid. This is because explicit criteriastandards are generally formulated to apply to the "average case" in any given category of patients usually distinguished by diagnosis and, consequently, do not adapt well to variations in case characteristics within the chosen diagnostic category. Moreover, the explicit criteria-standards list only what should be done for care to be rated "good." They do not usually permit a judgment on what may have been done erroneously, since the


67

class of possible errors or excesses is too large to pre-specify other than in exceptional situations. Very soon I shall be returning to reconsider the subject of validity and adaptation to case variation. Now, I would like to suggest that the strengths of each of the explicit and implicit criteria-standards can be exploited and their weaknesses mitigated by a sequential use of the two. First, a rather simple set of explicit criteria can be used to screen cases so that they are separated into those that "fail" and those that "pass." As a second step, the relatively few cases that have failed are assessed in greater detail using the implicit method of review, or the explicit method with more detailed and inclusive criteria-standards, or both together. To know what failures in quality may have been missed, one should also review more thoroughly a sample of the cases that have passed the initial screen. This will yield a more complete picture of failures in quality and may suggest improvements in the screen. Validity. The validity of the criteria and standards could be called their "truthfulness": the accuracy with which they represent the quality of care without contamination by error or irrelevancy. Validity, therefore, is the single most important attribute of criteria-standards, whether these are implicit or explicit. I have already touched upon issues of validity in previous sections of this book. Here, I shall try to amplify. The validity of criteria-standards can derive from two sources: (1) the science that generates them and (2) consensus among the experts who vouch for them. Consequently, the validity of the criteria-standards can be only as strong as the validity of the two sources they derive from. I have also pointed out that the relationship between structural characteristics and clinical performance, with few exceptions, is poorly understood. The behavioral and social sciences are equally unable to vouch with certainty for what is going to be individually or socially acceptable. Even the clinical sciences, which seem to us more powerful and definitive, are not always so. Much that is done in clinical practice, even if widely accepted, has not been established by unequivocal scientific evidence.

68


The conclusion I wish to draw from all this is not that we are unable to judge the quality of care. I wish only to emphasize that the certainty with which we can make our affirmations depends wholly on the strength of the evidence that supports them. When the evidence is firm, our judgments are correspondingly firm. If the evidence is weak, our judgments are correspondingly tentative. The only remedy for this uncertainty in our judgments is in further scientific study and research. There is a widespread belief that outcomes have an inherent validity of their own and are, therefore, independent of prior scientific evidence when used as criteria of the quality of care. In this context, this is merely an illusion. The scientific evidence that connects processes to subsequent outcomes is the same evidence that connects outcomes to antecedent processes. From this iron rule of reciprocal dependence there is no escape.22 It is true, however, that in scientific research (as distinct from quality monitoring) outcomes are the only validators of quality, assuming that the outcomes have been appropriately selected and measured and that they have been achieved in a legitimate manner. 23 Does this mean that, until our science is fully mature, only "evidence-based medicine" is to be practiced, as some have advocated? I think that is too extreme a position to take. It is a stance, I suspect, advocated by some who wish to strip medicine to its bare bones, controlling in that way what they believe to be its too burdensome cost — unless it is intellectual arrogance that impels them. To some degree, the opinions and experience of expert clinicians can help us remedy the shortcomings of our sciences. But opinion and experience provide only provisional validation of our judgments; they are only interim guides, awaiting the more definitive pronouncements of scientific progress. Until then, the opinions of experts will serve us well. And to the extent the experts we have selected are widely recognized and respected, the criteria-standards also gain in persuasiveness and acceptability to a broader range of practitioners. In addition to the foregoing fundamental considerations, some properties of the criteria-standards themselves affect the validity of the judgments they lead to. Some of these properties I have already described


69

when the reliability of implicitly derived and explicitly derived criteriastandards were compared. One of these properties, that of adaptability to case variation, is the next object of our more searching attention.

Adaptability to Variation The review of care by the "implicit" method offers opportunities for the reviewer to adapt his or her judgments to every clinical peculiarity of every individual case, noting with equal ease the presence or absence of indicated elements of care, as well as the presence of those not needed or contraindicated. These adaptations contribute to the validity of the resultant judgment. By contrast, explicit criteria-standards are, as I have said, often formulated for the "average case" and moreover cannot foresee, and consequently do not list, all the interventions that are unnecessary or contraindicated in any given case. Such criteria-standards can be described as "all-or-nothing," meaning that 100% of cases should conform to what the criteria-standards prescribe. The resultant inability to respond to variations in clinical characteristics among individual cases may result, on one hand, in a diminution of validity, and on the other hand, if clinicians know what the criteria-standards prescribe, in unnecessary interventions and increased cost. In response to these drawbacks the design of criteriastandards has undergone several modifications, some of which I shall now briefly describe. One modification of the criteria-standards has been to specify what have been called adherence ratios. The underlying presumption is that complete adherence to the criteria-standards cannot be expected, even when the quality of care is "perfect," because of individual variations in the clinical characteristics of cases in any given category. There may also be the presumption (expressed or implicit) that perfection cannot be expected, in any case, in the real world. I suppose adherence ratios can be expressed in several ways. For each item in a list of criteria, a range of what is acceptable may be specified; for example, adherence of no less the 80% and no more than 95%. One

70


could also specify what percent of cases (always less than 100%) in a given category can be expected to adhere to at least a given percentage (say 75%) of the criteria-standards. Such percentages have considerable arbitrariness to them, but they may also reflect what has been observed to happen in the care given by leading professionals in the more prestigious institutions. Note, however, that when percentages of this kind require that there be a group of cases to which they are applied, one can imagine that a lucky confluence of errors of commission and omission might result in a deceptively acceptable average. Another modification of the traditional lists of criteria-standards is to introduce what have been called exceptions. This means that to every criterion-standard there is an attached list of one or more specified clinical situations (called "exceptions") that permit or require a departure from that criterion-standard. Still another modification that responds to case variation is simply to classify the entities to be assessed into narrower and narrower classes: for example, subdivisions of diagnoses; of stage and severity; of age and sex; and so on. For each of these classes separate criteria and standards are formulated. Because the subclasses are more homogeneous, it can be assumed that more of the criteria and standards will apply to more of the cases. If one is willing to multiply the lists of criteria and standards, the process of sub-classification can go on and on. Finally, one may arrive at an algorithmic presentation of a given clinical situation. The algorithm begins with a presenting sign or symptom and then, in stepwise fashion, specifies what should be done, and depending on the results of what is done, what next step or steps are open to one. Sometimes these stepwise progressions and the contingencies they embody are called branching criteria, or criteria maps. They are, I believe, the final refinement that explicit criteria and standards are subjected to. At the same time, they mirror more or less faithfully the thought processes that underlie the "implicit method" for judging the quality of care.24"26 Recordability. Still another consideration may govern whether or not some items are included among the evaluative criteria-standards. That


71

is recordability, meaning how likely the items in question are to be noted in the medical record, if complied with and, perhaps, also noted as omitted, when this is the case. This is important because in most cases the medical record is the primary source of information about the process of care and its immediate outcomes. But the attribute of recordability can also be generalized to include the ease with which information can be obtained, from any source, when needed for quality assessment. From a practical point of view, one could argue that it is useless to include in a list of criteria-standards an item about which information is expected not be available. But, on the contrary, one could say that if the information called for is important, its inclusion will encourage better recording or otherwise make the information more accessible. Stringency. Stringency is the level of performance required by the criteria-standards of the quality of care. One could argue that, ideally, stringency should be of the highest. But other considerations may intervene. For example, one needs to take into account the resources and skills available in any given situation. If these are limited, a lower level of performance may be acceptable, provided the adopted level envisages the best use of what is available. At the same time, a higher level of performance could be set as a goal one tries to progress toward. And in any situation the level of stringency should not be so high that almost everyone is found to have failed. Nor should it be set so low that everyone passes. The ability to discriminate among levels of performance depends on how the criteria-standards are calibrated. Screening efficiency. The level of the stringency of the criteriastandards is related to yet another property; namely that of screening efficiency. This is the success with which the criteria-standards can separate cases of acceptable quality from those that are not acceptable. What one aims for is to have a set of criteria-standards that identifies almost every case of unacceptable quality but without including among the "unacceptable" any cases of acceptable quality. Because this requirement is a special case of the more general epidemiological concepts of sensitivity

72


and specificity, I shall refer the reader to the appropriate textbooks on the subject. I will only remark that a high degree of screening efficiency will make it easier to implement the two-stage evaluation procedure that I described when I compared the strengths and weaknesses of evaluation by implicit and explicit criteria.

Steps in Formulating Normatively Derived, Explicit C riteria - S tandards It may be useful at this point to describe very briefly how normatively derived, explicit criteria-standards can be derived.27 I shall proceed in steps. Delineating the field of inquiry. This means a definition of what the criteria and standards are meant to evaluate: for example, what aspects of quality of care; what components of performance; at what level of aggregation (of individuals, of institutions, of a community); and so on. Selection of panels of experts. These are the groups of persons who are to formulate the criteria-standards. In selecting these persons, two considerations are to be kept in mind. The first, of course, is recognized expertise in the aspects of care to be assessed. The more impeccable and commanding the credentials of the experts are, the more accepted their recommendations would be. But, besides expertise, one needs to consider what could be called the "politics" of any given situation. Because the health-care system has many internal divisions and corresponding centers of power and influence (for example specialists and generalists, one specialty versus another vying for a domain; institutional versus community practice; physicians versus non-physicians) the panels of experts should represent the more important of the several constituencies that are relevant. This is doubly important if there are several competing, and reasonably established, traditions that would claim to define quality in any given situation. The aim is to obtain endorsement of the criteria-standards by the segments of the health-care establishment that matter, so that the


73

criteria-standards gain in legitimacy, acceptability, and likelihood of implementation. It is important that the final recommendation of the panel of experts represent a true consensus, a genuine agreement, freely accomplished, not forced. In general, one of three methods may be used to achieve this. The first method is the familiar committee guided, led, and perhaps controlled by a chair holder chosen by the members of the committee or otherwise assigned. The dynamics of such a committee cannot be predetermined. Some members may be more assertive than others, and the chair holder permissive or controlling. Therefore, other methods of arriving at consensus have been proposed, and often used. The nominal-group process is a procedure meant to remedy what could be the dysfunctional features of the deliberations of the traditional committee. As a first step in the nominal-group process, each member of the group gives an independent opinion, recorded anonymously and in secret, for example by an entry on a card. Then these opinions are assembled, disclosed (for example by entries on a blackboard), and discussed by the group. At the next step each member records for the second time, in secret and anonymously, a personal opinion that may or may not have been influenced by the preceding group discussion. Subsequently, the mean, mode, or median of this final round is taken to represent the position of the group as a whole. A third method for reaching consensus is the "Delphi" method (named for its oracular antecedents!) or a modification of it. The procedure differs from the preceding two in that the members of the panel do not meet face-to-face. Opinions are expressed at a distance, by mail or a more contemporary electronic method. It follows that a larger number of experts can participate. The first step in the Delphi method is to ask each member of the extended panel to express an independent and anonymous opinion. Then these opinions are assembled and tabulated, the position of the group being represented by a frequency distribution and some measure of central tendency. The next step is to send this tabulation back to each member of the participating panel asking for a revised opinion in the light of

74


what the group as a whole has said. These second, possibly modified or unmodified, opinions are reported back and as before, tabulated and summarized. If there has been very little shift in the opinions expressed, the procedure can now end. Otherwise, the step described above is repeated until the position of the group has been stabilized. Some statistical representation of this final position is then constructed.

Selecting the "Referent" of the Criteria-Standards The fourth step in constructing the criteria and standards is to select (using one of the consensus methods I have described) what I have called the referent for the criteria-standards. This is the entity or subject to which the criteria-standards are to apply. The referent could be a diagnostic category, such as diabetes mellitus or, more specifically, one or more of several clinically distinguishable subgroups of the disease. Alternatively, the referent could be a condition such as chest pain or headache. It could also be a procedure: medical or surgical; preventive, diagnostic, therapeutic, or rehabilitative; use of radiological procedures, for example, or of blood transfusion, or of antibiotics; and so on. The referent could even be an administrative procedure such as that used for admission to the hospital or discharge from it. One could go even further afield and choose as a referent the procedures used for billing and payment collection (assuming one has adopted the model of what is called total quality management).

Specification of the Criteria and Standards The preceding more or less preliminary steps bring us to the heart of the matter, which is the specification of the criteria and standards themselves. To do this, one uses one of the methods of consensus already described and, in somewhat schematic form, proceeds as follows. 1. Assemble an inclusive list of criteria-standards by requesting suggestions from each member of the expert panel. It is helpful to


2.

3. 4.

5. 6.

7.

75

have provided to these experts in advance a review of the pertinent literature. Criteria and standards proposed and used by others could be added to the lists the panel members have provided. Have each member of the panel rate each criterion-standard on the basis of its perceived importance to good care and good outcomes by assigning to it a numerical score — for example, 1 to 10, or 1 to 100. Sometimes a second numerical score is assigned to each criterion-standard to reflect recordability. Reduce the original inclusive set of criteria-standards to a smaller subset based on the proportions of panelists who have rated each item highly for importance and recordability, doing so by using agreed-upon cut-off points on the scales of scores. Sometimes there is an added requirement, which is that not too many members of the panel object to the inclusion of an item. As a hypothetical example: more than 75% of panelists rate an item as 80 or more on importance and recordability, and not more than 15% rate the item 20 or less. Weights of importance are assigned to each item, based on the information already available or by a new weighting effort. Sometimes the panel members are asked to specify what has been called (by Brook and his associates) the time window. This means the time during treatment, or after it, when measurements of a specified outcome in a specified disease most clearly and reliably indicate differences in the quality of care. (It seems to me that the timing of certain elements of process would be equally critical and revealing and therefore should also be specified.) Next is the standard to be achieved with regard to each criterion. For examples: in a specified type of case, 90% are expected to undergo a specified procedure; or no more than 20% should require a specified drug; or all symptoms should have been relieved within a week; or mortality within one year should not exceed .05%.

76


8. It is necessary to specify where the information concerning adherence to the criteria-standards is to be found; what the procedures and rules for abstracting the information (e.g., from the medical record) are to be; how a given symptom (e.g., an asthmatic attack) is to be identified and rated for severity; and so on. 9. It is important to test the criteria-standards in a pilot study before they are released for general use in order to check feasibility (perhaps including cost of implementation), reliability, accuracy of judgments, acceptability to those who are to be assessed, and so on. 10. Because the science of health care, and the social conditions that influence its implementation, undergo continual change (one hopes, for the better) any set of criteria-standards must be continually revised to conform to changing conditions. If not, they become an obstacle to progress. Once more, we find the notion of continuous improvement inherent in the traditions of our professions. It is not true that until recently we were unmindful of it. But perhaps we were in danger of taking it for granted.

6 Obtaining the Necessary Information

78


wo observations I wish to make at the outset. First, information is the life-blood of quality assessment (as it is of practice also). Without it nothing can be done. If the information is incomplete, invalid, or intentionally distorted, a false picture of quality inevitably results. Second, as I have already remarked, there is a reciprocal relation between criteria-standards and the information one needs and seeks. The criteria and standards one formulates are contingent on what information is available concerning the degree of adherence to them. But it is also hoped that if the criteria and standards are put into effect, the needed information will be forthcoming. I could add a third observation. Setting up an information system and caring for and maintaining it require advanced skills, especially now when so much is computerized. These are skills I do not have. Therefore, I shall deal mainly with traditional sources of information, and with these only sketchily. The sources of information I shall touch upon are the following: medical records, surveys, financial records, statistical reports, direct observation, and test situations.

T

Medical Records The medical record kept by health-care practitioners for each patient under their care is the most frequently used source of information about the process of care and about the outcomes during care or soon afterward. Good records are essential for good care and for credible assessments of quality as well. Unfortunately the records often suffer, among other deficiencies, from incompleteness, sometimes from untruthfulness, and, not infrequently, from difficulties of interpretation.

Some Deficiencies of Medical Recording In this section I shall deal with some deficiencies of medical recording in a little more detail.

Obtaining the Necessary Information

79

Incompleteness. The completeness of medical records varies according to type of information and site of care. The medical record is particularly deficient in ambulatory settings, especially when practitioners work alone, far from an institutional source of care they are accountable to. In an organized ambulatory setting (e.g., an outpatient clinic) records are more complete, but less so than for inpatients. Nursing homes and convalescent homes almost scandalously suffer from poor recording. An additional problem when care is provided at a number of sites is the difficulty of communication among sites, and the difficulty, if the care is being assessed, of piecing together a comprehensive picture of the care the patient received during the entire course of an illness. Completeness also varies by type of information about care. By the nature of things, information about outcomes is confined to those that appear during the course of care or at its conclusion. The information, moreover, is confined to physical and physiological outcomes. There is a corresponding gap in recording the processes of care: there is no information about the management of the patient-practitioner relationship. The entries are restricted to aspects of technical care such as the chief complaint, laboratory tests, prescribed drugs, interventive procedures, and the positive findings on physical examination. Examinations that are done but yield negative results are often not mentioned at all, so that one has no proof that they were done. Doubtful veracity. The truthfulness and accuracy of the entries in the medical record can also be open to question. One hopes that deliberate falsification does not occur often. But the temptation to modify positive findings, to tamper with their timing, or, perhaps most often, simply not to mention embarrassing occurrences or errors is always present, especially in an atmosphere of blame, punishment, or legal action. A more pervasive and sometimes important deficiency occurs unintentionally and unawares. It is the "observer error" that distorts the information even before it is recorded. This can occur when a nurse records the blood pressure, a physician describes the heart sounds, the laboratory reports a low hemoglobin value, the pathologist reports the presence of pre-cancerous cells, or a radiologist reports a lung lesion. None of these

80


kinds of entries can be assumed to be accurate. Many studies have shown a stunning unreliability in all such observations: several persons observing the same phenomenon often disagree widely.28 Such ambiguities may, of course, influence treatment decisions. Therefore, the quality evaluator faces two alternatives. One is to assess the quality of care, assuming the recorded data are accurate. The other, where possible (for example there is a film, a tracing, a tissue sample) is to verify the accuracy of the evidence as well. A subset of errors is the inaccuracy of diagnostic classification and of coding in records and abstracts of records. Depending on how detailed the coding has been, an error of 20% to 40% in diagnostic classification may occur.29 And when payment for care is dependent on such classification (as when diagnosis-related groups — DRGs — are constructed) there is always the temptation to move, by subtle shifts, a case into a class for which more payment can be expected. These misclassifications disturb quality assessment by influencing the closeness of the match between the criteria-standards and their intended referents, including appropriateness of admission and length of stay. Difficulty of interpretation. Still another problem encountered when records (or other sources) are used to obtain information about the process of care is the difficulty in reconstructing the intent and train of thought of the practitioner, including what were the purposes of the various recorded investigations, treatments, etc., especially in complex cases where several health problems coexist. Because the essence of quality assessment is to express an opinion on the practitioners' clinical judgment, the handicap I have mentioned could be severe. One could also argue that, if no coherent plan of action can be inferred from the record, this is itself an indication of poor performance. Besides the preceding, more fundamental deficiencies of medical records, there are several lesser problems attributable to the design, maintenance, and management of the records and their governance. Quite often records are difficult or impossible to find. When found they can be partly or wholly illegible. If entries are unsigned or show a signature that only the one who made it can decipher, the reviewer cannot tell who did


81

what. As the record grows in volume it becomes more difficult to use as a guide for subsequent care and equally difficult to assess. Some Consequences of Recording Deficiencies One consequence of poor recording has been a widespread skepticism about the reliability and validity of judgments of quality based on record review. It is often said that such assessments are judgments of the record rather than of the care that was actually given. A more fundamentally vitiating consequence is that what is included in the criteria and standards is limited (as I have pointed out) by what one expects to find in the record. Even more important, the concept of quality can itself be impoverished by the almost uniform exclusion from the record of information bearing on important aspects of quality — the patient-practitioner relationship, for example, or patient preferences when alternative strategies of management are available. Some Responses and Remedies One can respond to criticisms of the record as a basis for judging quality by offering reasonable justifications; more fundamentally by improvements in the record; and, where possible, by seeking additional information. Justifying responses. One can argue that a judgment on recording is a reasonable reflection on the quality of care because the record is a necessary tool for providing care. It is needed to assure continuity and coordination as care evolves. It is also needed as a medico-legal document should the quality of care be challenged in court; it can either exonerate or assign responsibility for failure. And the record is of course indispensable when reimbursement is linked to the diagnosis made and the services provided. That the record will be subsequently used to assess quality of care is itself a potent argument in favor of good recording. Besides the kinds of arguments I have offered, one could also point to some studies that have shown good recording to be correlated with the

82


quality of care, when quality was judged by information obtained independently of the record — for example, by direct observation. Unfortunately, there are few such studies; and in those reported, the correlation between the quality of care and the quality of recording has been rather low. One must conclude that good care can coexist with poor records. I do not know if the reverse also holds: namely, that poor quality can coexist with good records. Verification of recorded information. What appears in the record can be checked when there is an independent source of information such as a tracing, a film, or a pathological slide. Review of this information by several experts can lead to a reasonably firm conclusion. A more fundamental remedy for poor record-keeping is to establish in the services that provide such information (laboratory, radiology, electrocardiography, etc.) the necessary quality control systems. One can also compare what appears in the record with what one learns from the other sources of information. Supplementing the record. Within a given site, say a hospital, there often are other sources of information that can supplement what the medical record contains. These could be, for instance, daily notes kept by the nursing staff documenting the patient's condition, procedures carried out, drugs administered, and so on. The clinical laboratory should have its own records against which the medical record can be compared. Sometimes hospital records can be amplified by obtaining information about ambulatory care received before and after hospitalization. Eventually, a system of what has been called "record linkage" can be envisaged. This requires that each patient be identified by a unique number that appears on records of care no matter where the care is received. The compilation of the considerable amount of information pertaining to each unique number would probably require a computerized system. Modifying the assessment procedure. One could say that a remedy to the general poverty of recording is to adapt the choice of referents and criteria-standards to the record. While this may be necessary, it is a dys-


83

functional adaptation rather than a remedy. It is much better to improve the record itself. Improving the record. The record should be improved by insisting that existing guidelines be enforced. But one should also offer help (e.g., by providing dictating machines and secretarial help) to lighten what practitioners often see as a heavy burden. And perhaps practitioners will be further motivated to keep good records when they see that judgments of their performance depend on what they have written down. Above all, however, what practitioners admit to, in case a justifiable error has occurred, should not be punished. Good recording cannot flourish in an atmosphere of fear and recrimination. Good recording can also be promoted by improvements in record design. For example, by having preformulated checklists (as of the recommended schedule of immunizations, or of items in an acceptable physical examination) one can partly alert the practitioner to what is expected and partly allow recording by checkmarks that indicate what was done or found — and what was not. The "problem-oriented record" developed and popularized by Lawrence Weed has been recommended as a tool for improving both the conduct and assessment of care. In this kind of record, the practitioner enters a list of the patient's problems, preferably in order of importance, notes what is done for each, and visit after visit, records what progress has been made, which problems have been solved, and what new ones have appeared.30 In redesigning the record, one should have in mind the components in the definition of quality that one would like to have information about, if at all possible. For example, there have been attempts to discover what patients think about their care by having them record their daily impressions, either separately or in the record itself, to which they would have access.31'32 Clearly, much can be done to improve existing records. But one should remember that recording is both burdensome and timeconsuming. At the extreme, recording could interfere with the more important activities of patient care. Therefore, in attempting to improve the

84


record one should exercise good judgment and restraint. Perhaps the introduction of wholly or partly computerized records will allow the desired combination of greater completeness, ease of recording, and improved availability when needed for patient care or assessment. This I shall leave to more qualified persons to deal with. Made-to-order recording. In some instances, usually to conduct research, the records of care routinely kept are considered too heterogeneous and incomplete to use. Instead, the selected practitioners who agree to participate in a project are asked to keep, for a limited time, specified kinds of information, perhaps on separate forms. What is gleaned from these more uniform records can then be compared, for example, before and after the introduction of some intervention, or can be compared across practitioners with different degrees of training, and so on. The uniformity and completeness in recording obtained this way is counterbalanced, however, at least to some degree, by the likelihood that practitioners so closely guided and observed will change the usual manner of their practice. Therefore, the effort to find substitutes and supplements for the medical record continues.

Surveys I use the term surveys to indicate a variety of ways of obtaining information from patients and members of their families, from practitioners, and from populations in communities, using mailed questionnaires or interviews, the latter by telephone or face-to-face. The information can be entirely new or can be a verification or supplementation of what has appeared in the medical record. Much can be learned from rather simple, informal surveys of those who have received care. But the conduct of more formal surveys, especially of a community, requires considerable specialized skill: for instance, to design questionnaires, to draw probability samples, to control for accuracy and completeness of responses, to analyze and interpret data, and


55

so on. These are matters I shall not touch upon. Anyone contemplating such scientific surveys should seek expert assistance.

Surveys of Patients and Family Members Talking to or questioning family members tells us how pleased they are with the care they have received and with the other experiences they have had before, during, and soon after any episode of care. These opinions can be obtained on how matters could have been improved. And in some cases, specific items that have appeared in the medical record (or have failed to appear) can be verified. Opinions obtained while the patient is awaiting care in a medical setting may be distorted by the patient's unwillingness to offend or antagonize. It is better to seek information in a more neutral setting, but not so long afterward that memory has begun to fade. In all cases, confidentiality and anonymity must be maintained. Surveys of Practitioners Questioning practitioners or talking to them about the care they have given or failed to give can be used to flesh out what has appeared in the record. Because practitioners tend to explain away their deficiencies, the information obtained may not be valid enough if research is the object of the survey. But when the purpose is to monitor the quality of care, it is essential to allow practitioners to explain what they have done or failed to do, so that the rationale underlying their behavior can be understood. The discussion that may follow can be educational to the practitioner and to the assessor as well. In no case should a final determination concerning performance be made without such a discussion. And if the performance of an entire institution is at issue, the institution should also have an opportunity to justify itself. Practitioners can also be questioned, not just about the care they themselves have provided, but about the quality of performance, more generally, in an institution or a community; about the conditions of their work, material and organizational; and about their satisfactions and dissatisfactions.

86


I have not included practitioner satisfaction as an attribute of quality, as some have done. To keep practitioners happy is not among the objectives of care. Nevertheless, practitioner satisfaction is an important attribute of what I have called structure or the environment of care. It influences how well practitioners perform.

Surveys of Populations I have put surveys of populations in a separate category partly because a population includes persons who, during a given period of time, may not have had any care, or may have received care from several sources so that their experience can not be attributed to any one source. A population may be delimited by the area in which it lives — a neighborhood, a town, a region, a country. Subsets of populations may be chosen for special purposes — mothers, children, the aged, the poor. When a given source of care (a hospital, a health center, etc.) is given responsibility for the care of those living in a given geographic area or for those who have enrolled in a given program, then all those who may potentially receive care from that source are its population. Because surveys of populations include those who have not received care, one can obtain a better picture of the difficulties of access and of inequity in distribution. Also one can find about need and unmet need for care." Information about socioeconomic status and living conditions can also be sought. All of this information tells us, in part, how well a health-care system has functioned, and also how one may plan to do better. In some countries, there are continuous or periodic population surveys that publish information a health-care planner and evaluator can use. Alternatively, a given provider of care may conduct a survey, on its own, of the population for which it is responsible. But it should be remembered that to do so properly, one will need expert assistance.

Financial Records In some health-care systems, payment is made case by case, the payment varying by type of illness, type of procedure, length of stay under care,


87

and so forth. To receive payment the provider submits a claim. The paying agency requires that the claim form provide some information about the patient. This includes verification of entitlement to care; personal information, such as age, sex, place of residence; and medical information, such as diagnosis and some details of the care received. Such data are accessible, especially if computerized, and they provide information about some aspects of the appropriateness of care and its quality, and the cost implications of these. But because the claim form is by necessity brief, and often faulty or incomplete, what one learns from it is limited and of questionable validity. Still, the statistical analysis of the information can reveal general patterns, point to pervasive abuse or possible weakness, and if the identity of the patients and providers is made known, can lead to more thorough investigation of quality using the corresponding medical records and other means.

Statistical Reports Health-care institutions and systems often regularly collect information about their own operations and in some cases about their accomplishments. A statistical analysis of this material can reveal information pertinent to appropriateness, quality, and cost. I shall say more about these matters in a subsequent part of this book.

Direct Observation Direct observation can be used to obtain information about structure, process, and outcome.

Observation of Structure It is customary for an inspection team or a team of evaluators to visit a hospital, a nursing home, an ambulatory care facility, or a doctor's office, to see how safe and hygienic such a place is; whether or not the facilities

88


meet standards; whether or not the needed equipment is available and kept in good repair; whether or not the medical records are properly kept and managed; and so on. Organizational structure is more difficult to observe, but aspects of it can be gleaned from interviews with key personnel, perusal of organizational charts, and perhaps actual attendance at some meetings —for example, of the medical staff, of the infection control committee, or of a quality circle.

Observation of Outcomes The verification of outcomes may require direct observation in some cases. One can tell, for example, whether a fracture has been properly set, or how far the range of motion has been preserved or restored. Other outcomes may require more-detailed examinations and investigations. Direct observation of a dentist's work is a standard method of evaluation; what was done, and how well, are clearly evident in the patient's mouth!

Observation of Process The process of care is observed and, when necessary revised, whenever teachers guide their students, or supervisors exercise authority over their subordinates. Among colleagues, an informal process of reciprocal observation and an automatic adjustment to behavioral norms are constantly taking place. When the norms of behavior conform to standards of quality, this "visibility" of care to one another is an important safeguard of good care. Sometimes, observation of practice has also been used as a formal procedure to assess the work of colleagues, especially in ambulatory-care settings where medical records are generally rather poorly kept.34'35 The procedure for observation has been something like the following example. A qualified practitioner visits a colleague who has given previous consent to participate in the study. The observing practitioner is introduced to the patient and permission obtained for the observer to remain in the examining room. There the observer stays, as unobtrusively as pos-


89

sible, without offering any comments or suggestions. The observer only notes how the colleague interacts with the patient, what is done or omitted, and so on. Afterward, the observer writes down what he has seen and heard and records an opinion of the quality of care. (To do so in a more regular fashion, a form specifying what aspects of care need to be noted and assessed may guide the observer.) The quality of care for each item observed is recorded and a final rating assigned to the care as a whole. The rating may be qualitative ("exceptional," "good," "satisfactory," "not satisfactory") or quantitative (a number between 1 and 100). In addition to, or in place of, personal observation, videotaping may be done as the information-gathering device. The Royal College of General Practitioners in England has used a modification of this method as a testing and educational tool for its members who wish to be assessed. A special feature is the free discussion between the examiners and the practitioner being assessed that follows each such experience.36

Test Situations So far, I have described methods by which one tries to assess the quality of care actually practiced in ordinary, "real life" situations. Some other methods have been used to find out the capacity to practice well, as distinct from what is actually practiced. Creating a test situation to see how the practitioner would conduct care under the circumstances defined by the test does this. Sometimes an actual patient is introduced to the practitioner, who is then monitored as she or he goes through the sequence of care. Perhaps more often the process I have described is simulated by interaction with a computer programmed to respond to the clinical activities undertaken by the practitioner. In either case, the practitioner knows that this is a test and therefore can be expected to try to excel. Later, I shall describe something I have called challenge monitoring, a procedure to test a practice or a system of care, with prior consent, but without knowledge of when the test is conducted or what it is to be. Presumably, what is revealed is the ordinary, everyday pattern of care or system operations.


7 Choosing When and How to Monitor

92


When to Monitor he sixth in the steps that mark the progression of a monitoring effort might be, as I see it, a determination of when the monitoring is to take place. There are, in the main, three possibilities: (1) prospective or anticipatory monitoring, (2) concurrent monitoring, and (3) retrospective monitoring.

T

Prospective Monitoring Prospective or anticipatory monitoring is the attempt to pass judgment on a proposed clinical event or intervention before the event occurs or the intervention is implemented. For example, approval may be needed before a person is admitted to the hospital or a surgical operation is undertaken; assuming, of course, there is no emergency. Interventions for apparently urgent conditions can be monitored subsequently.

Concurrent Monitoring Concurrent monitoring occurs during the course of care, either periodically as a routine, or when something happens that triggers a review of the way a case is being handled. Clinicians are constantly engaged in concurrent evaluation of their own work as they care for each patient, judging whether or not the management is on course and effective by how the patient responds. In this way the health-care practitioner decides if the diagnosis needs to be revised or the method of treatment altered. "Concurrent monitoring" as part of a quality assurance system is simply a formal plan by which, at critical junctures in the care of patients, someone other than the caregiving practitioner assesses the progress of the case and, if needed, offers advice or perhaps actually intervenes. To implement such a system, it would be necessary to arrange for periodic or continuous flow of information from the clinical domain to the monitoring apparatus — a matter of some complexity.

Choosing When and How to Monitor

93

Retrospective monitoring. Retrospective monitoring is the method most frequently used. Generally, it is conducted by reviewing a sample of medical records and — based on the record as well as on additional information, if needed — making a judgment on the quality of care. Prospective monitoring prevents care of certain kinds from ever being given. Concurrent monitoring allows one to intervene, and if needed, to interrupt care or alter its course. Retrospective monitoring cannot do either of these, but it does teach one, when errors in care are discovered and their causes are understood, how to do better in the future. There can be, of course, combinations of these timing strategies.

How to Monitor The seventh step in the progressive unfolding of a monitoring plan brings us to the actual monitoring methods one may choose from. This is a vast subject. I shall attempt only a classification of methods, accompanied by rather brief descriptions. Controls over the Process of Care Controls over the process of care are examples of prospective or anticipatory monitoring. I shall mention two examples: certification and recertification of hospital admissions and continued stays, and second surgical opinion programs. Certification and recertification of admissions and stays. Because hospital care is expensive, a procedure may be established for ensuring that a patient truly needs to be admitted to the hospital to begin with, and to review the need for continued stay thereafter." In Figure 7.1 I show schematically one procedure for accomplishing this purpose. Of course, it is important to make sure the patient's condition is urgent when admission is requested, and that equally good care cannot be provided at lower cost elsewhere — at home or in a clinic, for example.

Figure 7.1. Certification of hospital admissions and continued stay. (Source: Prepared by A. Donabedian, based on: Goran, Michael J., et al., "The PSRO Hospital Review System," Medical Care, Vol. 13, Supplement, April 1975, p. 1 and information provided by D. F. Slebert and Alan Reider, Office of Professional Standards Review Organizations, June 1980.)


95

Briefly, the steps in Figure 7.1 can be described as follows. 1. Certain prespecified categories of patients are admitted automatically. At admission they are assigned a prespecified length of stay considered appropriate for each category. 2. All other patients are assessed either before admission or within three days after that in order to determine if admission were appropriate. If it is judged to be appropriate, a prespecified length of stay (called a "checkpoint") is assigned. A "coordinator," usually a nurse, who is guided by a list of relevant criteria-standards, assesses appropriateness of admissions and stays. These can be specific to a patient's diagnosis or may be geared to clinical features present in the patient. (See Appendix 1 for an example of appropriateness criteria that take account of clinical status and treatments needs — "level of care" — rather than diagnosis.) 3. When the assigned duration of stay expires, the patient is assessed once again and either discharged or assigned an additional permissible stay, at the end of which there is another reassessment leading either to discharge or to another specified extension of stay, and so on, until the patient is judged ready to leave —the judgment being made, at this point and at every preceding point, by the kinds of criteria I have mentioned. 4. It is important that the procedure I have described include a number of safeguards that protect the patient and the patient's doctor, and also check the performance of the coordinator. For example, at each certification and recertification, the patient's doctor can request a review of the decision made by the coordinator, by appealing first to a physician advisor appointed for the purpose, and then, if this is judged insufficient, to a committee of physicians. Furthermore, a sample of the coordinator's decisions is reviewed, even if these decisions have not been questioned, in order to check compliance to the criteria-standards and, if needed, to revise the criteria-standards themselves. Finally, the performance of the patient's doctor can be reviewed to determine how appropriate the doctor's admission and stay decisions have been. 5. What happens when a hospital stay fails to be certified would depend on how the health-care system is organized. In some cases all that follows is that the insurance program refuses to pay for care. It is also possible that a doctor found to be regularly mistaken in the decision to

96


hospitalize, suggesting either ignorance, carelessness, or deliberate abuse, will be dropped from the roster of practitioners who care for patients for whom the insurer (or governmental payer) has accepted responsibility. As a more immediate step, it is conceivable that the hospital stay will be denied or terminated by administrative action. With the reader's permission, I shall now digress a little, my purpose being to illustrate the relationship between the structural features of a health-care system and consequent monitoring activities directed at the process (as in this case) or outcomes. The procedure for certification and recertification that I have described, although anticipatory with respect to future care, is also concurrent as regards recertification. It is, of course, time-consuming and expensive to implement. It is subject to misjudgments by the coordinator and the physician advisor. It grates on the sensitivities of clinicians who take pride in their autonomy; and it could rob patients of insurance protection. It is necessary to ask, therefore, if the achievements of the procedure justify the effort it requires. This is the first message I wish to convey; the need to evaluate any monitoring procedure one may introduce. The second point to consider is whether one could avoid using a repressive monitoring procedure by introducing a structural change that would eliminate the need for it. It is no longer feasible to say that if every person paid for his or her care, other monitoring would no longer be necessary. As an alternative, insurers and governmental payers could alter the method of reimbursing hospitals (doing so, e.g., by case or with a yearly budget) so that the hospitals themselves will carry the burden of monitoring. But even so, because the hospitals seek solvency (and in some cases profit) while the payers want patients to receive the care they need, the payers will have to continue to monitor, even if the behaviors and outcomes to review may have changed because hospitals have begun to operate under a new set of financial incentives. Second surgical opinions. Another method of certification has been applied to recommendations for surgery for conditions not considered to be urgent. Each initial recommendation is reviewed by another, wellqualified surgeon chosen from a designated panel. The second opinion


97

of this surgeon may either confirm or fail to confirm the original recommendation. If necessary a third opinion may be sought. What happens when the original recommendation is not confirmed would depend on the nature of the relevant health-care system. The patient may have the option of refusing surgery or of accepting it without incurring a penalty. Sometimes a penalty may be imposed if the patient has an operation despite its having been unconfirmed. In any case, it is important to keep non-confirmed patients under observation to make sure that no adverse consequences occur. In Figure 7.2 I show the consequences of one of the earlier secondsurgical-opinion programs in the United States. The findings of subsequent studies can be summarized as follows: 1. There is considerable variation in non-confirmation, depending on the diagnosis of the condition surgery has been initially recommended for. In one set of studies, non-confirmation ranged from a low of 10% for breast surgery to a high of 34% for orthopedic operations. 2. Some persons whose recommendations for surgery are confirmed do not subsequently have the operation, while others who have not obtained a confirmation nevertheless go ahead and have the surgery originally recommended. Overall, there is an estimated 5% to 10% reduction in elective surgery as a direct consequence of the program. 3. It is believed that there is also an indirect deterrent effect because surgeons are more careful in recommending surgery when they know their recommendations will be subsequently reviewed. This so-called sentinel effect may account for a further 10% reduction in elective surgery. 4. When the costs of the program are compared to the savings that result from its implementation, second-surgical-opinion programs appear to be cost-effective. There is perhaps a net reduction of 5% in the cost of elective surgery. 5. The earlier studies have found no evidence that patients are harmed when the need for surgery is not confirmed, provided they are kept under observation so that appropriate treatment, surgical or nonsurgical, can be given if needed. I should remind the reader that the experience I have summarized and the percentages I have cited apply to the United States only; else-

98


Figure 7.2. Some findings of second surgical opinion studies. (Source: E. G. McCarthy, "Mandatory and Voluntary Second Opinion Programs in the Greater New York Area, with National Implications," Dorothy E. Eisenberg Lecture, Harvard Medical School, June 12, 1976.)

where the results would not necessarily be the same. Nor is it certain that the controls over unnecessary surgery I have just described are always appropriate, seeing they are meant to apply where surgeons are mostly in private practice, are paid for each operation, and therefore have financial (as well as professional) incentives to operate. But it should also be noted that the use of confirmation by a second opinion applies beyond surgery to any important and costly procedure or regimen of treatment.

Reminders: Computerized and Other It is believed that quite often failures or errors in clinical care result, not from ignorance or dereliction, but mostly from inattention and forgetfulness brought about by the pressure of medical work and its constantly


99

increasing complexity. Consequently, methods have been developed, often depending on computerized recording to remind clinicians of the procedures that are appropriate to each patient's situation.39 For example, depending on the patient's circumstances, the clinician may receive a reminder that a certain immunization has come due, or that a procedure for screening or diagnosis is to be carried out, laboratory tests requested, or certain medications either prescribed or withheld if there is a contraindication. The response to such reminders appears to be favorable, provided the rules that govern them have been formulated by the clinicians themselves, and that the reminders are not rules to be obeyed but recommendations only — suggestions that a clinician may not ignore but can contravene after recording a reason for doing so. If the clinician does not record a negative opinion, the reminders will be repeated, and eventually there could be a personal visit from someone in authority who can demand an explanation. Status-Progress Monitoring It goes without saying that clinicians adjust the care they give, and assess its consequences, by paying close attention to how each patient under their care progresses, or otherwise responds, by getting better or worse. Similarly, a scheme that would classify each patient according to the degree of progression or severity of illness would have several important uses in health-care planning, assessment, and "control," as follows. Severity or stage of illness can determine what kinds of care a group of patients will need, where that care is to be given, what resources it will require, how much it is likely to cost and, consequently, how much is to be paid for it. This information is necessary for the rational planning of health-care services. Classifying patients by stage or severity of illness also has a number of important applications to quality assessment and assurance. For example, the stage at which a disease is encountered when a patient who has it seeks care for the first time, or is first admitted to the hospital, or to any other facility, tells us something about ease of access to care as

100


well as the quality of care prior to admission to the hospital. Similarly, progression toward improvement, or the lack of it, during the course of care can lead to inferences about how good the care has been. And, finally, because the degree of success depends to some degree on how seriously ill a patient was when first encountered, the severity and stage of the illness must be taken into account when the outcomes of care are compared or otherwise assessed. This is done by what is called a case-mix adjustment, a subject I have already mentioned in connection with outcome assessment. In addition to accounting for severity and stage of illness, case-mix adjustment also corrects for differences in age, sex, ethnicity, social class, and economic status — in short, for any characteristic that is known to, or believed to, influence the outcome and cost of care. Many schemes have been advanced for classifying patients by severity or stage of illness; some designated by fanciful acronyms, and some patented to be sold for a price by their originators. These schemes include the "Staging Criteria" originated by Joseph Gonnella;40-41 the "Computerized Severity Index (CS1)" developed by Susan Horn;42 the MEDISGRPS (pronounced "Medisgroups") advanced by MediQual Systems, Inc.; and the "Acute Physiology and Chronic Health Evaluation," known as the "APACHE," developed by William Knaus and his associates for use in intensive care units.43 A more recent example (which I cite mainly to show that the effort continues) comes to us from France in the shape of a "Simplified Acute Physiology Scale (SAPS II)."44 A detailed description of these classification methods is beyond the scope of this book, but to give the reader some notion of what they look like, several examples are given in Appendices 2-5. A perusal of these samples will show that the severity or stage of an illness or trauma are designated by one or more of the following: physical findings, laboratory and other diagnostic test results, and physiological function. These are, of course, the kinds of observations that guide clinical management in everyday practice. What the more formal classifications provide is a procedure for checking the progress of a patient by an outside observer as well. To do so in a timely manner so that, if necessary, the course of care can be interrupted and altered, one needs to institute almost daily review and feedback to the monitoring system. While this can be done by visiting


101

patients on an almost daily basis (or at predefined critical points), a computerized recording system is almost a necessity. I am compelled to remind the reader yet once more that if the principles of this method are understood, one can develop a classification system of one's own, a scheme even more suitable to local circumstances than anything purchased from elsewhere.

Aberrance Monitoring I use the term aberrance monitoring to designate a continuous or periodic attention paid to unwanted adverse events whose occurrence leads one to suspect that health care has not met standards of accessibility or of quality, or both. Of course, to serve in this manner, the events in question must be, to a significant degree, preventable by good planning and care; inevitable occurrences will not qualify. Preventable adverse events can be observed in a community at large. Examples are mortality, morbidity, and disability. Maternal, perinatal, and infant mortality, while often inevitable, are known to include a sufficiently large proportion of preventable deaths to serve as reasonably sensitive indicators of social conditions in general, and of health care in particular. The occurrence of an infectious disease that could have been prevented by adequate, timely immunization would be an even more sensitive and specific indicator of a failure in health care. Such occurrences, and others like them, have been called "sentinel events,"45 this is because they correspond to the alarm raised by someone who stands guard over the safety and welfare of a community. Preventable adverse events can also occur, needless to say, during the course of clinical care or soon after; and when they do, they are even more direct evidence of failure than are events occurring in a community. While such events have always been known to occur, and have been used to monitor the quality of care and to improve it, they have attracted renewed attention, and even outrage, under the unfortunate designation of "errors." To call an event an "error" suggests that individual practitioners are at fault, whereas, quite often, a weakness in the system of care is mainly responsible.46

102


The wholly or partially preventable adverse events that occur in connection with clinical care include the more sensitive mortalities already mentioned, as well as other case fatalities and complications. Other examples of such unwanted outcomes are adverse drug reactions, transfusion mishaps, infections acquired in the hospital, injuries to patients from falls and other accidents, bed sores, and so forth. In addition to such outcomes, faulty processes can also be considered adverse events. Antibiotics may have been improperly prescribed; blood and blood products may have been administered in improper amounts or without adequate precautions; laboratory tests and other diagnostic procedures may have been performed too often or not often enough, and when performed, their findings may have been ignored; specimens of tissue removed at surgery may not have been sent for pathological examination or, if sent, the tissue is too often found to have been normal. Certain behaviors of patients and staff members can also serve as adverse events worthy of monitoring. For instance, patients may become noisy and troublesome; or they may leave the hospital or clinic against advice; or there might be a high turnover in the nursing or medical staffs. I do not intend for this list to be in any way complete. I only wish to suggest the almost endless roster of events that are candidates for monitoring. All that is required is that the event have an important bearing on the quality of care, that it be observable, and that it be well-defined enough to be identified, and counted or measured. As to the methods of aberrance monitoring, I have found it convenient to distinguish two varieties. The first I call anecdotal or clinical, and the second statistical or epidemiological. "Anecdotal" or "clinical" aberrance monitoring. In some instances an adverse event (sometimes called a "critical incident") is so serious, or is so likely to have been preventable, that any occurrence of it needs to be noted or investigated. Thus, every maternal death is worthy of analysis aiming to find what may have gone wrong and how the error, if any, could in the future be prevented. In other, somewhat less serious and more frequent occurrences (such as postoperative infections or non-


103

fatal drug reactions) rather than study every case, a sample of cases is drawn and the cases included in the sample are studied. The principle underlying the anecdotal or clinical approach is that no failures of the kinds described should occur in a well-conducted system of care. The motto is "zero errors," even when it is known that this is unattainable. The method of investigation is called "clinical" because it consists of a detailed review of the records of care, supplemented when needed with interviews with members of the health-care team and with family members and patients. "Statistical" or "epidemiological" aberrance monitoring. The principle underlying this approach is that monitoring should be guided, not by the mere occurrence of adverse events, but by the average frequency of these events (as measured by a mean or a median, for example) and by the deviations above and below that average as time elapses. Two kinds of patterns are distinguished. One is called common-cause variation and the other special cause-variation. Common-cause variation is the degree of variability inherent in any system of production (in this case, a health-care system or subsystem). It characterizes that system and is the consequence of the interaction of many factors, some known and others unknown. Consequently, the variation observed is close to random, and generally assumes a normal distribution or perhaps another form reasonably well-defined. As long as the variability of observations falls within that distribution (usually defined by a measure of dispersion such as a standard deviation), one assumes that the system in question is "in control," meaning that it continues to function as one would expect. Only when the system fails to function in this way does one suspect that some special cause has intervened to upset the established pattern, a cause that needs to be identified and if necessary eliminated. Special-cause variation is suspected to exist whenever the distribution of adverse events deviates significantly from the random pattern already established by past experience. There may be, for example, a sudden increase in the frequency of postoperative infections significantly beyond expectation. Other patterns may also show a loss of randomness. These

104


include the occurrence of several observations in an unbroken sequence, above or below the pre-established measure of central tendency. Or, there could be a succession of observations of progressively rising or falling values, indicating a trend. A pattern of periodicity would also suggest the intervention of a special cause, such as when anesthetic mishaps increase during weekends (let us assume because of inadequate staffing) and are lower during the rest of the week. The preceding exposition is an almost ludicrously condensed and vague account of a set of concepts and methods that characterize the highly developed field of statistical control as applied to industrial production.47'48 A reader who wishes to learn more should consult an appropriate text or seek help from an expert. The more immediate question is whether or not this industrial model also applies to the "production" of health care services and of health. There are, it seems to me, many occasions when a variant of statistical control could apply to health care. I have already mentioned postoperative infection rates and anesthetic mishaps as examples. Perinatal mortality rates in each of several hospitals of a community could be another example, as Bundesen partly demonstrated many years ago.49 Graphical plots of such events that show both the average of values and the variation above and below the average during a sufficiently long period of time would reveal the customary performance of the system in question, as well as non-random departures from the customary. Such a graph is called a statistical control chart. The measure of central tendency could be a mean or a median. The values plotted could be successive mean values of a series of observations (such as monthly averages of the number of transfusions) as well as the ranges of such observations (the difference between the highest and lowest monthly values). The plotted values could be proportions rather than averages (e.g., the proportion of the patient population admitted with chest pain who received specified tests). The plotted values could also be individual observations rather than the means (such as the daily number of chest X-rays that were found to be normal). As the occasion demands, other forms of control charts can be constructed.


105

I have intended to offer a representative rather than an exhaustive list of examples. And to add a little concreteness to my account, I offer Figure 7.3 as a crude visual representation of the material I have described. In constructing this entirely hypothetical figure, I have assumed that adverse occurrences (like postoperative infections) have been observed long enough to yield a stable mean and standard deviation. The control limits have been set at plus and minus two sigma, assuming that any deviation beyond these limits is not likely to be random. Subsequent observations, each being the mean occurrence of the adverse event during six months (for example) are plotted. A rising trend or a close approach to, or overstepping of, the upper control limit is a signal that infection control is likely to have failed. There is no difficulty, it seems to me, in responding to non-random increases in the occurrence of an adverse event by initiating an investi-

Figure 7.3. Hypothetical control chart.

106


gation of why this has happened. Perhaps non-random reductions would merit equal attention. Investigation of their causes might reveal an error in reporting, or even, one hopes, an improved method of care that could be more widely adopted. The search for an explanation of a special-cause variation could include construction of control charts for sub-parts of a system (e.g., of separate clinical services). The search could also include reviews of clinical records as well as discussions with persons involved in care and in other day-to-day operations. Established procedural routines could be reassessed. Perhaps there has been a failure in equipment maintenance. Possibly the characteristics of the patients treated have changed. Our attitude toward common-cause variation is much more problematical. This is because the "culture" of health care is heavily imbued with the notion that no errors should occur. Therefore, it is difficult to be reconciled to the idea that some degree of variability that could represent "error" is to be tolerated as an established feature of any production process. To this difficulty, there could be one of several responses. 1. We could assume that common-cause variation does not mean "error" as just described. Rather, it may reflect uncontrollable random events, including, perhaps, variations in patient characteristics one has no control over, but that influence the patient's response to care. 2. On the contrary, we could take it for granted that common-cause variation is the product of at least some preventable error. If so, efforts need to be made to reduce the average occurrence of such error, as well as the variability around that average. 3. Alternatively, we could conclude (as John Williamson has suggested) that at least some of common-cause variation reflects preventable error, but decide that present circumstances do not allow major remedial changes in the system of care, or that the effort needed to bring about such changes could more appropriately be used in other ways. I shall draw on a recent report by Saturno et al.50 to demonstrate an application of statistical control aiming to remedy an established pattern of unacceptably high variability and low average performance. The problem faced was the length of time it took for a person showing evidence


107

of serious myocardial infarction to receive the necessary thrombolytic (clot-dissolving) treatment at the intensive care unit the patient was referred to. The relevant data are shown in Figure 7.4, in which are displayed the successive control charts depicting the time lag experienced by each patient during three phases of the study. It is clear that during the initial one-year period of observation there was a pattern of extreme variability, often approaching and once exceeding the upper control limit set at 3 sigma. There was also an unacceptably high average value, well beyond what the study team had set as an acceptable goal. Further analysis and group discussion suggested causes and remedies, the latter including the installation of an elevator between the emergency facility and the intensive care unit, a modification of architectural design that, incidentally, serves to highlight for us the relation between the phys-

Figure 7.4. Time elapsed between admission to the emergency service and receipt of thrombolytic therapy in the intensive care unit by patients suspected to have serious myocardial infarction, during each of three phases of observation in one hospital in Murcia, Spain. Control limits in each chart are set at 3 sigma.

108


ical structure of a facility and the quality of care practiced in it. More important, one observes in the figure the subsequent reduction of the average time elapsed between admission and the receipt of required therapy, as well as a remarkable reduction in the variability around that average, the variability remaining well within the control limits set at 3 sigma. Equally instructive is that further monitoring (during the third phase) improved this pattern even more. Despite what I have just described, statistical control has not been used much, as far as I know, in the monitoring of health care. The degree of its applicability and its usefulness are yet to be established. The attention I have given it is meant mainly to stimulate interest and experimentation. But, fortunately, statistical control and analysis in health care include much more than the rather restrictive "industrial" model. The broad range of epidemiological methods more familiar to us are available as sentinels of aberrance and as means for localizing it in "time," "place," and "person." For example, analysis by time would include fluctuations and more regular periodicities that could reflect events in the system of care (such as shifts in staffing or in the availability of investigative or therapeutic resources) or variations (seasonal and other) in the characteristics of patients who present for care. It would also include the study of secular trends, a necessary tool in verifying the effectiveness of interventions to improve care. Analysis by person would include populations and subdivisions of populations, as well as practitioners in functional groups or even as individuals. Analysis by place would include defined geographic areas that are, in some way or another, more or less exclusively connected with one or more sources of care whose performance is at issue. Small-area analysis is an example of this. More often, place would refer to an institution such as a hospital that is being compared with other hospitals like itself. As I have emphasized before, in all such comparisons it is necessary to correct for the effect of factors other than health care that might influence the process of care and its outcome. There is one more condition to be met before statistical or epidemiological methods can be used. There needs to be a regular flow of accurate data about variables (of process, of outcome, or of both) that can


109

be used to maintain surveillance or to conduct studies. In a previous section I have described the main sources of information that one could draw upon. But the problem of assuring a constant flow of good information remains. The nature of the information needed would of course vary according to the kinds of problems one wishes to address. For routine use there could be a predetermined list of key events, perhaps adapted to each specialty or type of illness, which could be collected, tabulated, and graphed on a regular basis. Such events go under names such as "clinical indicators," "performance indicators," "generic screens," and the like. I list some examples in Appendices 6-9. The reader will notice that some of these indicators are keyed to diagnoses, others to procedures or specialties, while still others, called "generic," cut across these. Of course, I give these examples merely as illustrations. It is best for any monitoring enterprise to formulate its own. And, the aim should be to have few rather than many, and to confine them to matters that are critical to good care. Computerized records are particularly suitable for providing information about such indicators or screens, in easily usable form. In the absence of computerization, a system of self-reporting may be set up; but this is not always satisfactory because what is reported may not adhere to strict definitions, and it also tends to be incomplete, especially if reporting could attract unfavorable attention to those who do the reporting. As an alternative, it is possible to have each record abstracted by personnel trained for the purpose, using a form that contains the predetermined items of information deemed useful for assessing quality, as well as items of managerial importance. These abstracts can then be computerized and used to generate the necessary tabulations and graphic displays. Still another method is to have one or more competent persons visit each person at frequent intervals to glean from the record and, if needed, by talking to the patient and staff, the predetermined items of information. It is a time-consuming and costly undertaking, and it is not clear that the results are worth the effort unless the items sought are few, are quite critical to care, and there is adequate review and follow-up leading to remedial ac-

110


tion. Masses of data that clog the monitoring system so that they are mostly ignored do no good. Goal-Attainment Monitoring Goal attainment as a method of monitoring enjoys several advantages. It fixes attention on the more important objectives of care and, by revealing whether or not these objectives have been met, signals either success or failure. If the latter, it becomes, one hopes, a spur to improvement. In the hands of John Williamson, who devised and advocated the method, and spent a decade testing and refining it, the method which (after several trials) Williamson called "health accounting," can be described as follows.51-52 As a first step, a committee of physicians and other members of the health care team, including administrators if relevant, meets to identify the conditions that need the most attention in the monitoring effort. This recognizes two key principles: that at any given time one must focus attention on what is most important, and that it is necessary to involve a representative cross section of personnel in the quality assurance enterprise, allowing the members of the group to participate on a reasonably equal footing. To allow for this to happen, Williamson recommended that the "nominal-group process" be used to arrive at the group's decisions, and that the criteria of importance were to be the frequent occurrence of the "problem"; the perceived frequency and seriousness of the deficiency in its management; and the potential for improvement through corrective action. As a second step, the committee, using the recommended method for arriving at a consensus, specifies the outcomes that are to be the criteria of failure or success. These could be, as Williamson called them, "diagnostic outcomes" and "therapeutic outcomes," with the latter subdivided into effects on health status and outcomes other than such effects. Diagnostic outcomes were meant to include the accuracy of the diagnosis reported and the percentage of cases identified among those who are known or suspected to have the diagnosis in question. The first of these requires simply the further testing of those already under care. The second


111

diagnostic outcome is more difficult to verify; it requires at least a sample of persons who report for care for any reason, or a population, so that they may be screened for the presence of that diagnosis. "Therapeutic outcomes," as Williamson defined them, included patient satisfaction, the acquisition by patients of knowledge about their illness, and changes in behavior that are conducive to health. Therapeutic outcomes also included, of course, changes in health status. These could be measured by outcomes specific to any given disease (such as by blood pressure changes in hypertensives or blood-sugar levels in diabetics). They could also be measured by a functional scale made up of the following progression: (1) asymptomatic, minimal risk; (2) asymptomatic, high risk; (3) symptomatic; (4) restricted; (5) dependent; (6) dead. Nowadays, there are, of course, more refined measures of overall health status that could be used.53 The advantages of such scales as Williamson used and of their more recent elaborations are the same no matter what the patient's diagnosis might be, and they purport to measure the quality of life more inclusively. They are not, however, as specific or sensitive as the outcomes more particularly pertinent to each diagnosis. The third step in the progression of goal-attainment monitoring is actually to measure the selected outcomes after a prespecified interval has elapsed. This may be done by mailed questionnaire, by an interview, or by having a qualified person (whom Williamson called a "health accountant") examine the patient at home, or by having the patient come back to the clinic to be interviewed and examined. The fourth step in goal-attainment monitoring is taken if it is found that the outcomes achieved have been unacceptably below the prespecified and hoped-for standards. If so, a study is initiated to determine the causes of failure. This may include a case-by-case review of the care recommended and implemented in order to determine how the care has fallen short of implicit or explicit criteria of quality. The fifth step in this progression is to take appropriate action to correct whatever defects in care have been detected; and the sixth step is to repeat the entire sequence, in order to determine if the remedial action taken has been effective.

112


I have already mentioned the advantages of this method if it is fully implemented. The disadvantages include limited knowledge of what outcomes to hope for as a result of acceptably good care and, consequently, the temptation to set one's goals too low, to avoid disappointment. One safeguard is to compare one's own results with those reported in the literature or by institutions whose performance one could take as a benchmark. One additional drawback of Williamson's method is that the measurements it proposes are expressed as percentages in a relatively large population. It is, in this regard, "epidemiological" rather than "clinical." A remedy to this problem is to specify a minimum outcome that every person in a group must attain after the lapse of a specified period of time. This is precisely the principle underlying the "problem-status index" proposed by Mushlin and Appel, two of Williamson's students.54

Cohort-Trajectory Monitoring We owe to Robert Brook a method of monitoring that is both simple and instructive. I have called it cohort-trajectory monitoring because it envisages a group of patients who have a given diagnosis or condition who are followed as they progress through a health-care system. When this is done, one is apt to find failures at every successive step in that progression, each failure revealing a deficiency that needs correction; and the succession of failures amount to an overall estimate of the effectiveness or goodness of the system in question. For example, in one study of patients who reported at an outpatient clinic with gastrointestinal symptoms and were referred for X-ray examination, it transpired that only 67% came back to be examined; 55% had an "adequate" X-ray examination; there were abnormal findings in 27%; and at least "minimal" treatment was received by 10%. In summary, the patients who were treated when they had abnormal X-ray findings, plus those who were not treated because the X-ray findings were normal, amounted to 38% of the patients initially referred for radiography. Brook has suggested that this is the measure (even if provisional) of "effectiveness or success."55


113

In this study, as in other similar studies, the "progression" of the cohort selected for observation is reconstructed from past events as depicted in the medical record. It is of course possible, with greater effort, to actually follow patients as they progress, prospectively, from day to day. I believe that much can be learned from a simple study in which a few patients are observed at intervals as they journey through a care-giving system. What one might see could be astonishingly revealing. And it would be even more so if the observer were also to be the patient! I could also suggest that the device of trajectory monitoring could be expanded to include other occurrences that depend on appropriate follow-up to be successfully managed. An example of a monitoring procedure often used is the ascertainment of what happens when a practitioner receives notice of an abnormal laboratory (or other investigative) finding. Too often one discovers that there is no indication in the patient's record that the medical attendant noted the abnormality; or, if it was noted, that some action was taken; or, if action was taken, that it was even partially appropriate. I am sure the reader can think of many other examples. Among these, it would be most revealing to find out what happens to patients after a hospital has discharged them. Besides revealing how carefully (if at all) a hospital prepares for patient discharge, one would learn how adequate community services and resources are. Challenge Monitoring I have grouped under the heading of challenge monitoring actions taken to present a health-care system with a predefined stimulus in order to reveal how the system reacts to the challenge of that stimulus. The nature of the challenge depends on what aspect of care one is interested in exploring. Challenge monitoring is an established procedure in testing the quality of diagnostic services. From time to time a standardized specimen is submitted for testing. It is unknown to the diagnostic service, but those submitting the specimen know it. The challenge for the diagnostic service is to arrive at an accurate identification of the unknown.

114


Similar in principle is the attempt to find out what happens when a phone call is put through requesting an appointment, or help for a sick child (for example), or for a more serious emergency. The purpose is to find out how speedily and how appropriately the system responds. An interesting example of challenge monitoring is the use of socalled simulated, programmed, or standardized patients. Such a "patient" is, in fact, healthy, but has been instructed and trained to pretend to have the history and symptoms of a specified medical condition. Unknown to a prospective provider of care, such a "patient" arrives at the clinic and requests care. What happens is gleaned from the records of care and also from the observations reported by the pseudo patients who have been trained to make the needed observations. It is possible in this way to assess not only the technical competence of a caregiver, but also to assess the features of the personal interaction between patient and practitioner.55'56 Needless to say, challenge monitoring can provoke opposition, anger, and even outrage, if it is conducted without previous consent. Those to be tested in this way must have agreed in advance, but of course they will not know when the test is to occur. And even if prior consent has been obtained, it must be made clear to all that the purpose of the test is not to trap and punish or humiliate, but to learn and improve. Case Studies The idea of case studies should not be new to the reader. In one way or another, it has been referred to several times in the preceding narrative. It consists of a patient-by-patient assessment of the quality of care, based on information in the medical record and supplemented, if needed, by interviews with caregivers and patients. The assessment of quality is made by expert personnel using criteria that are implicit, implicit but guided, or explicit. What these categories mean, and what their comparative advantages and limitations are, have been described earlier in this text. Primary case studies: as their name implies, these are studies that are undertaken as a first step in the process of assessment, unprompted by antecedent monitoring activity. Let us say one is interested in finding out how well patients with suspected coronary occlusion are managed. A sam-


115

pie of cases is drawn from among those admitted with that diagnosis to the hospital. The care received by these patients is reviewed in detail, as I have described. Or, if one wishes to begin earlier in the process of care, one can sample cases that call an ambulance service, or present at the emergency service of the hospital. The reviewer would be interested in verifying the diagnosis, in judging whether or not the requisite diagnostic tests were done, the treatment was appropriate, and follow-up adequate. Secondary case studies: these are studies undertaken as a result of questions raised by antecedent monitoring activity. Let us assume, for example, that aberrance monitoring has identified a potentially preventable adverse event, such as a reaction to a transfusion or a drug, or a postoperative complication or infection, or even a death. To determine whether or not that event was indeed preventable, and to find out what may have gone wrong, a more thorough review of each case, or of a sample of cases, would have to be conducted. In fact, such case studies are the usual concluding event to all less-definitive monitoring activities.


8 Constructing a Monitoring System

118


I

have already spoken of the several strategies one might use to identify problems or "opportunities for improvement" as a first step in the process of monitoring the quality of care. Some problems are forced upon us; others require an effort on our part to find them. Among the tools for finding out problems or opportunities for improvement are the methods of monitoring I have just described. The additional thought I want to introduce now is that these strategies and methods should not be used haphazardly. They should be viewed rather as a system of interrelated parts conducted in an organized manner. The many parts allow for stepwise growth and also for a mutuality of support and interaction among parts that adds to efficiency and effectiveness. The organizational structure of the system could also be simple or elaborate, depending on the scale of the monitoring effort. It should have the following characteristics.

Key Elements There should be, it seems to me, an organizational unit that assumes responsibility for the planning, coordination, direction, and implementation of the monitoring effort as a whole. The leadership of the monitoring enterprise should be entrusted to someone who, by virtue of knowledge, experience, and organizational position, commands respect, and who also understands the mission and methods of quality assurance in health care. Consequently, in settings where personal health care is the key function, the leader of the quality assurance effort should also be a competent and respected clinician. The additional skills needed seem to me to belong in the field of epidemiology. It is here that one acquires the skills to gather, analyze, and interpret the data quality assurance depends on, and, moreover, to design and conduct the studies often needed to test causal hypotheses and to verify failure or success when corrective action has been taken. With these thoughts in mind, I have conceived of the quality assurance professional as a "clinical performance epidemiologist," a role not as yet widely recognized.58

Constructing a Monitoring System

119

The qualifications I have just enumerated should be supplemented by the leader's ability to work amicably with people at all levels of an organization, to inspire trust, and to set a personal example. Additionally, the leader should have access to the highest level of the executive hierarchy in the system or organization. Only in this way can the information relevant to quality assurance help bring about the needed changes in institutional policies and procedures. And it would, moreover, contribute to the prestige and credibility of the monitoring enterprise if its leader were to be advised and supported by a committee of senior members of that organization. In most cases, the monitoring activity is likely to begin on a modest scale, but it should be designed with a view to stepwise expansion as conditions demand and allow. And the several components, in place or projected for the future, should be designed to complement each other. Thus, they become an interdependent set, more efficient and effective as a whole. Moreover, the monitoring activities should include, where possible, some that are the responsibility of the central unit, whereas others are delegated to the units that actually care for patients. Delegation is important because the monitoring effort benefits from the knowledge and skills of those who provide care. Besides, it fosters the feeling among the rank-and-file that the monitoring effort is partly their own, that they can influence it, can use its findings to improve their ability to help their patients and, in doing so, to help themselves as well. At the same time, the central unit continues to assist, guide, and supervise all delegated activities and, when necessary, verify their findings and effectiveness. All information concerning quality, no matter how generated, should ultimately reach the central unit where it is evaluated, interpreted, and passed on with appropriate recommendations to those in the organization who can take action to bring about change. The ability to perform this mediating function so that everyone in the organization can reap its benefits is the foundation on which the legitimacy of the monitoring activity must, ultimately, rest. Whenever a more fully developed, comprehensive, quality monitoring activity is introduced into an organization or institution, it must be

120


coordinated with other activities already in place that are directly or indirectly pertinent to quality assurance. If these more traditional activities do not already exist, they should be introduced. These more traditional methods of quality assurance include postoperative tissue review, mortality review, infection control, and review of the use of drugs and biologicals. Equipment purchasing and maintenance and the assurance of sterility of supplies are also germane. So is risk management, since risk and the quality of care are so intimately related. The linkage between continuing education and quality monitoring deserves special attention because it is believed that education designed to address recognized deficiencies in performance is more likely to command attention. Similarly, research activities can result from a need to answer questions that the quality assurance effort has raised. In planning and implementing such research, the well-trained clinical performance epidemiologist can be a valuable asset. In addition to fostering internal linkages, the quality monitoring effort in an organization or institution is required to respond to demands made upon it from the outside; for example, by governmental agencies or nongovernmental bodies that inspect and credential the care-providing institutions. But while the providers of care respond to these external demands, they should also proceed in ways that respond more specifically to their own circumstances and aspirations. I offer Figure 8.1 as a partial example of what I have in mind when I speak of interrelated activities that support and reinforce each other. The top tier of entries in the figure shows the several ways that problems or opportunities may be identified and, perhaps, selected for further study. These include (reading from left to right) statistical information systems; generic or other screens that one can use to identify potentially preventable aberrant events; the activities of "quality circles"; problems identified by external agencies such as insurance companies, accrediting agencies, or governmental bodies; problems or opportunities for improvement identified by the chiefs and staff of clinical departments; problems brought to the attention of the members of the board or managers of a hospital; and complaints made and suggestions offered by consumers as well as workers in an institution.

Figure 8.1. A hypothetical quality monitoring system. * Detailed audit uses implicit criteria, perhaps supplemented by explicit criteria. Information additional to that in the chart is often required.

122


The lower part of the figure shows some activities that may result from antecedent problem identification. In this instance it is assumed that something has suggested a deficiency in the management of patients who have had surgery for a hernia. The method of further investigation is to select a sample of such cases and, using a screen of explicit criteria, to assess the medical record to divide the cases into two categories: those that pass, and those that fail. All cases that fail are reviewed in greater detail in order to identify what went wrong and why. A sample of cases that passed the original screen of explicit criteria should also be studied in detail to see how successful the screen has been in separating out cases of less-than-good quality. This last procedure, which is meant to test the screening capacity of the explicit criteria, might lead to a revision of the criteria themselves. As a consequence of the detailed auditing carried out, one is able to find, one hopes, the causes of the deficiencies observed, to take corrective action, and subsequently to see if the corrections have been successful. In this way, the quality assurance cycle described very early in this text is brought into play. The figure I have just described, as well as the other features of a quality monitoring system I have recommended, have had in mind a rather developed enterprise in a large, amply financed institution. Smaller institutions with limited resources will be unable to meet the requirements I have detailed unless, perhaps, they form a coalition that permits them as a group to pool their resources to meet the monitoring needs of all. When even this is impracticable, one should not lose heart. Wellplanned efforts on a small scale by truly dedicated persons can succeed, sometimes even better than more elaborate and costly mechanisms run by an institution more for show than substance. For example, even the smallest of institutions could form a quality circle or a quality improvement team that would try to find ways to improve the quality of care in a given clinical unit. What is needed is a dedication to quality and a modicum of common sense. Rather than waiting for direction from above, one could start to burrow from below!

9 Bringing about Behavior Change

124


e are now at the ninth and last of the steps in the plan for monitoring clinical performance. At the same time, we are nearing the end of the work that has kept us together all this while. But before we part, there are some difficult questions to be asked. Does quality monitoring work? Does it improve care? What must be done to enhance its power to do good and, perhaps, to reduce the likelihood of its doing harm? I shall try to answer these difficult questions in two steps. First, I shall describe some activities and interventions meant to bring about changes in the behavior of those who deliver care — changes that, if they occur, can be expected to improve quality. And then, I will offer some opinions about the factors that may influence whether or not the desired behaviors will occur and bring about the hoped-for improvements in care. But, before I begin, I must warn the reader that a great deal of what I am about to say will be speculative, a matter of opinion rather than demonstrable fact. Sadly, in what concerns these questions, perhaps the most important of all to be asked about quality assurance, we have mostly only bits of experience and flashes of insight. There is no encompassing theory of effectiveness I am aware of. The interventions meant to bring about behavioral change I shall also describe under two headings: readjustments in the system and educational and motivational activities. Each of these will be further subdivided into long-term and short-term efforts. This scheme I shall use as a general guide to organize my presentation, while allowing myself some freedom to depart from a strict adherence to it when it seems useful to do so.

W

Readjustments in the System By starting our consideration of behavioral change with readjustments in the system of health care, we recognize, as we have done from the start and repeatedly thereafter in this work, the relationship between structure and process. We also shift the focus of attention from deficiencies in personal performance to defects in the system that may have permitted

Bringing about Behavior Change

125

these deficiencies to occur. Consequently, we are less likely to accuse individuals and more likely to create an environment more conducive to good care.

Long-term Readjustments Some system readjustments take a long time to accomplish; accordingly, their salutary effects will be delayed. These readjustments I call long-term. The most fundamental of these is the provision of resources adequate in quantity and high in quality. These resources are both material and human. It is true that whatever resources are available can be used poorly or well. But to believe that over the long haul one can continue to provide good care with insufficient, substandard resources is to live in a dreamland. Among the more important long-term readjustments I must also include a restructuring of incentives that allow good performance to be recognized and rewarded while substandard performance is also recognized and, if possible, corrected. Ideally, there should be a link between the quality of performance and progress in a professional career, from positions of lesser rank to those that are higher, more responsible, more prestigious, more highly rewarded. But one should not believe that incentives are only economic. Professionals respond to other incentives as well, some concrete (such as more leisure to study and do research) and some symbolic (such as special mention and appreciation for excellence). The astute manager will try to understand what each professional person values most, and design the incentives accordingly.

Short-term Readjustments Short-term readjustments in the system are changes that can be accomplished more rapidly, more easily; they can be expected, therefore, to show early results. Of these readjustments I shall mention four: facilitation, routinization, circumvention, and direct intervention. By facilitation I mean the identification and removal of obstacles to good care. It is difficult, for example, to take blood pressures routinely or

126


to examine a specimen at the bedside if the apparatus or the microscope is locked up. It is difficult to keep detailed notes if there are no dictating machines. And so on. One should not expect professional personnel to behave heroically in the discharge of their daily responsibilities. Where possible, their way should be smoothed. Routinization is another relatively simple change that can smooth and regulate both administrative and clinical processes. The introduction of algorithms, protocols, or guidelines is an option.59'60 The algorithm will detail the steps that the professional staff have agreed should be taken in the investigation and management of a specific clinical condition. And, the medical record could be redesigned in ways that make it easier to implement the agreed-upon algorithm. It is assumed, of course, that in any given case the algorithm may be departed from, provided the reason for doing so is clearly recorded. By circumvention I mean the creation of mechanisms and procedures that can get around obstacles that cannot, themselves, be eliminated. For example, if it becomes clear that physicians cannot or will not take time to explain to patients the nature of their illnesses or the drug regimen they are to follow, a nurse can be assigned to do the former, while the pharmacist who dispenses the drugs can be expected to do the latter. As to direct intervention, the fourth of the short-term system readjustments on my list, little need be said. The reader has already encountered descriptions of it in connection with surgical second opinions and the certification of hospital admissions and stays, as well as some aspects of status-progress monitoring.

Educational and Motivational Activities I shall use, once again, the device of dividing these activities into two categories: long-term and short-term. The former, as I have already mentioned, take time to implement and to show results, whereas the latter are more easily introduced and can be expected to show early results. Long-term activities. Of the long-term activities I shall mention two: (1) professional education, training, and socialization; and (2) the edu-


127

cation of consumers and their more active participation in the conduct of care. Under the heading of professional education, training; and socialization, I place all activities meant to improve the competence of healthcare professionals of every category, so that they become more expert and also more observant of professional norms. I would also argue that administrators and managers of health-care systems should also be professionalized, so that they are able to understand more completely the processes of health care as well as the values and culture of the health-care professions, thus becoming better equipped to lead and to manage more effectively and with greater finesse. Insofar as clinicians are concerned, a thorough grounding in the science and technology of health care is an obvious necessity. But, in addition, their education should include a thorough familiarity with the values and methods of quality assurance. The subject should be introduced early in the educational progression, beginning with theoretical presentations, progressing to participation in projects designed and conducted by the students themselves, and finally eventuating in participation in the everyday quality assurance activities of the institution where the clinicians are being trained. In this way one would hope to produce a generation of practitioners to whom quality assurance becomes a friend and ally, rather than an unwanted intrusion. Other benefits will also ensue. It is fortunate that the point of view and methods of quality assurance are also those of problem-solving, quantitative analysis, and research in a more general sense. There is no better vehicle than a quality assurance project to teach a practitioner these necessary orientations and skills. The education of consumers and their more effective participation in the process of care is the second long-term intervention under the heading of "educational and motivational activities." I mean by "consumers" primarily "patients," as well as others responsible for them. I am not concerned with those in an organization whom others have called "internal customers" because they occupy intermediate positions in the chain of activities that constitute health care. No doubt these also need to learn, but I include them under the care-giving apparatus. My main concern now is with those whom every practitioner must be dedicated to; namely,

128


patients. But it should be understood that patients are not to be merely passive recipients of care. They should also be active participants. By being so in appropriate ways, consumers help not only themselves but also those who care for them. It is proper, therefore to consider how they can do this.61 It seems to me that there are four ways in which patients and those responsible for them (including those who purchase care on their behalf) can influence the conduct of health care: they do so (1) through traditional market mechanisms, (2) through organizational mechanisms, (3) through interpersonal exchanges during the process of care, and (4) through political action. Market mechanisms come into play whenever consumers have a choice to receive care from one provider instead of another, and also when they can change providers when these do not meet their expectations.62 But to do this in a way that encourages good care in its technical as well as its interpersonal aspects, consumers need to know more about the quality of the former. This kind of information has been difficult to assemble and to provide so that it can be understood and used when needed.63 Still, the effort to inform consumers continues; for example, through the publication of so-called report cards that have a bearing on the quality of care offered by health maintenance organizations, as well as by other providers of care.64'65 It is more likely that associations of consumers and those that purchase care for them will have the skills necessary to interpret the kinds of published information I have described. Several organizational mechanisms can be used to encourage consumer participation. As I said in the section of this book dealing with problem identification, consumers should be encouraged to express their opinions, either spontaneously or through direct solicitation. Perhaps there should be an ombudsman, an officially appointed but neutral intermediary, to whom consumers can confide their worries and disappointments, and who in turn can transmit these to the appropriate authorities in the organization. Still another device is to have regularly scheduled meetings of consumers and their health-care practitioners. It is best for such meetings to be held in settings and circumstances that reduce the disparity in power and prestige between practitioners and patients. The


129

purpose is to encourage social interaction as well as a free exchange of views, each side learning from the other. Whether or not consumers can participate in the activities of the quality monitoring enterprise itself has been a matter of speculation. The purely technical aspects of care are perhaps beyond the competence of the average consumer (but perhaps not beyond the well-versed consumer representative). Yet it seems to me that consumers could perhaps contribute to the formulation of general policy as well as to assessing the aspects of care that affect their comfort, convenience, satisfaction, and aspirations for a better life. It is perhaps through their participation in the interpersonal exchanges of the health-care process that consumers can best influence the quality of their own care. Therefore, just as health-care practitioners need to be educated to understand their patients, the patients themselves need to understand what is expected of them in a mutually rewarding interaction with their caregivers. In particular, they should learn to recognize, as far as possible, what good care is, and demand that they get it. In this regard, prior experience of good care, especially in recurrent situations (pregnancy, for example) is, I think, the best possible preparation. Political action is the fourth and last item in my list of ways consumers can influence the quality of care. Of political action I know little. But it is reasonable to say that for such action to be successful, consumers must be organized and informed, and the political system must allow free expression of public opinion and be responsive to it. Alas, this is not always the case; and even when it is, those who contend for political power are eminently skilled in confusing and manipulating public opinion. It takes insight, skill, and fortitude to dispel the smoke screens with which politicians cunningly conceal themselves and their private purposes. With these brief comments on political action I come to the end of the discussion of long-term educational and motivational activities meant to bring about behavioral change. As I have said, these activities take time to implement, and their effects are slow to be felt. Short-term activities. The short-term activities I want to mention are: (1) reminders; (2) feedback; (3) continuing education; (4) rewards; and (5) penalties.

130


I need not say more about reminders, because I have already described them as one of the methods of performance monitoring. Practitioners who are reminded to do things that they have agreed in advance to do can be expected to alter their behavior accordingly. Feedback in its purest form is simply conveying information to groups of individuals about their performance as revealed by earlier quality monitoring activities. This implies that practitioners know what ought to be done but have become inattentive or careless. Therefore, by simply being told how they have performed, they will be led to do better in the future. There is reason to believe that the details of the feedback procedure will influence its effectiveness. Eisenberg,66 who has made an extensive review of the effectiveness of behavior-modifying interventions, concludes that (1) feedback, besides describing the behavior of a group in general, should also inform each individual of his or her performance, as compared with that of the group; (2) that it is more effective if provided faceto-face by a respected colleague, rather than impersonally, such as a mailed report or tabulation; and (3) that the standard for assessing individual performance should be, first, the performance of peers in the same group, and subsequently, perhaps the performance of peers in other comparable institutions. Standards derived from the literature seem to be less persuasive to those who see themselves engaged in the "front lines" of the "real world!" There is, moreover, a linkage between feedback and continuing education, which is the next to be considered in our roster of short-term activities.67 It is believed that continuing education is more useful when it addresses failures that the quality monitoring activity has identified, as contrasted with subjects of a more general nature. According to Eisenberg, continuing education is more efficacious under the following conditions: (1) the subject accepts that he or she has a need to learn, as documented by the performance monitoring mechanism; and (2) the educational activity is specific to that need, is conducted face-to-face and if possible oneon-one, and is conducted by an "influential" who is a person whom the subject trusts and respects. Rewards are still another means of influencing individual behavior. I make a distinction between rewards and incentives; I hope legitimately.


131

I take incentives to mean features of an organization that promote a consistent pattern of good performance. By contrast, rewards are meant to recognize discrete, individual, meritorious acts. For example, it would be a reward if a person who makes a money-saving suggestion or innovation were to receive in return (as happens in some organizations) a percentage of the money saved by implementing that suggestion or innovation. Of course, the institution of a provision for rewards is part of the more general system of incentives in an organization. But rewards can also create dissatisfaction and personal rivalries, especially if in some units opportunities for earning rewards are not as great as in others, or if they are equal, but some are singled out and some are not. Consequently, some have argued that rewards, whether monetary or other, should recognize the performance of groups or teams rather than of individuals. Penalties, the last entry in my list of short-term educational and motivational activities, are of course the very opposite of rewards. Some would question whether penalties should ever be used as motivators of behavior. The opposition to penalties is based partly on the belief (in my opinion not fully justified) that most failures in quality are the result of weaknesses in systems rather in individuals. And besides, penalties are deeply resented, tend to demoralize, and to encourage evasion and concealment, when the aim should be to acknowledge error and to strive to improve. Some conclude that penalties are misdirected (by targeting individuals rather than systems) and self-defeating. I am, in general, sympathetic to this viewpoint, but I also believe that individual malfeasance does occur, and that a reasonable system of penalties, judiciously applied, is a necessary component of a quality assurance system. But the system of penalties should be graduated to the seriousness of the offense; it should be fair; it should be consistently applied; and it should have earned prior approval by those who are likely to endure it. Above all, it should be rehabilitative rather than destructive. Those who incur a penalty (the loss of some surgical privileges, for example) should be able by subsequent effort to vindicate themselves. That notwithstanding, those who prove recalcitrant and incorrigible should be dismissed or, when a serious offense has been committed, referred to law

732


enforcement agencies. It does no good at all if a health-care system is perceived as a protector of malfeasance within an inner circle of cronies. I would like to conclude this section with two observations. First, I am not sure that the distinction between adjustments of the system and modifications in individuals is as sharp as I (and others) have made it seem. The reader will have noticed that, in my own text, I have not always observed the distinction. Furthermore, is it not true that individuals (whether they are practitioners or consumers) are also part of the "system" of health care? If that is so, it follows that reasonably stable changes in individuals can be viewed as system adjustments as well. In any event, that is what I believe. Here is the second point I want to make. This account of remedial interventions is meant only as an introduction to the several alternatives one can choose from, either singly or in mutually reinforcing combinations, as the occasion demands. In any single instance of failure (or error), the most fruitful approach is to understand (and perhaps schematically visualize) the sequence of activities during which the error occurred. This should help one localize the failure and therefore help determine how and where to introduce the remedial or preventive modification in that sequence of care. So much the better if one can so modify that sequence that the likelihood of human error is at least lessened or, ideally, eliminated.68 The ability to make such a modification is one determinant of the effectiveness of quality monitoring. What other determinants there might be is the next subject I must deal with.

10 The Effectiveness

of

Quality Monitoring

134


s quality monitoring effective? Does it work? Does it produce results? These are the kinds of questions we must now address.69'70 In an introductory account such as this, we cannot afford to review the voluminous literature that bears on these questions. And if we did, no clear answer would emerge. We would conclude that sometimes quality monitoring and the subsequent effort to influence behavior do in fact succeed, at least for a limited degree, during the rather short periods of time during which observations have been made, and in some circumstances. At other times there is no success or, if it is achieved, it is small and evanescent. This is true of every one of the monitoring and remedial methods I have described. There is no one method (so far as I know) that in and of itself is uniformly effective or ineffective. Therefore, it seems to me that the secret of success is neither entirely in the method itself, nor entirely in the circumstances within which the method is implemented. The secret, rather, is perhaps in the interaction between the circumstances and the method; it is in the fit between the two. Therefore, I shall proceed from this vantage point, basing what I shall say partly on empirical observations reported in the literature, partly on theory-based expectations, and partly on informed speculation. My method will be to classify the factors that are likely to influence effectiveness into two categories: those that are contextual, and those that are operational.

I

Contextual Factors By context I mean the general characteristics of the situation into which quality assurance through monitoring is to be introduced, and in which it is to operate. I shall mention the following factors: (1) culture, beliefs, values, and assumptions; (2) leadership; (3) sponsorship; and (4) organization.

Culture, Beliefs, Values, and Assumptions At its most general, the context is defined by a set of properties that go under the name of "culture." This means, as used here, a prevailing set

The Effectiveness of Quality Monitoring

135

of beliefs and assumptions about how important quality is compared to other competing objectives, how "quality" is to be defined, who is responsible for it, and what methods of improving it are either acceptable or unacceptable to those who have the power to facilitate or to hinder change.

Leadership Leadership is a significant feature of the culture of an organization. It can also, if judiciously and persistently employed, modify the culture itself. Two properties affect how successfully leadership can be exerted. The first is authority, which includes the ability to influence careers, to reward or censure. The second, for want of a better word, could be called charisma, namely the ability to persuade, to motivate, to inspire trust, to set a personal example of commitment to improving the quality of care.

Sponsorship Sponsorship by the relevant professional association is still another factor that may add to the probability of implementing quality monitoring and to its subsequent success. Sponsorship is, in effect, a declaration that the objectives and methods of the monitoring effort are approved and, therefore, deserve participation and compliance.

Organization Both leadership and sponsorship imply an organizational "home" in which they can come into full play. Therefore, formal organization can be added to the contextual factors that contribute to the successful implementation and operation of quality monitoring and readjustment. Formal organization also confers operational possibilities, in the sense that it can muster the necessary skills and tools to do the job of monitoring.

736


Operational Factors To help me organize my discussion of the operational factors I intend to mention, I shall draw on a more general formulation of the determinants of health behavior (the health belief model)71 to offer the following rather crude progression. 1. There is a demonstrable, consequential need. Performance monitoring and readjustment is easier to initiate when there is already dissatisfaction with the quality of care. And the monitoring effort itself should be able to demonstrate, using credible data properly presented and interpreted, the magnitude and the nature of the need to improve. 2. Something can be done to meet the need. It is essential for everyone in an organization, and especially for those who provide care, to believe that the quality monitoring effort will be taken seriously and that its findings will be acted upon. If this is not the case, there will follow disappointment, loss of confidence in, and resistance to the monitoring effort. 3. The right thing is done in the right way. It should be fully clear to everyone that whatever is done in response to the findings of the performance monitoring effort is the right thing, done in the right way. Monitoring is openly resented or covertly subverted if participants believe that it will lead to loss of professional discretion, or to repressive, punitive interventions at odds with professional ideals and values. Every effort should be made to demonstrate by word and deed that this is not the case; that the purpose is to help professionals do what they would want to do to serve their patients. 4. There are demonstrably useful results. The monitoring effort should be able to demonstrate useful results free of harmful consequences. There is nothing more persuasive to skeptics and, therefore, more likely to garner support, than to show that quality monitoring can bring about changes that are helpful to the conduct of more successful care. At the same time, there should be no consequences adverse to the attainment of that goal. 5. Adverse effects are minimal or absent. Whether or not quality monitoring can have consequences harmful to the conduct of good health

The Effectiveness of Quality Monitoring

137

care is a matter of debate. The more frequent consequence, if monitoring is unwanted, is for it to be subtly opposed, circumvented, weakened, and neutralized into impotence. Still, under certain circumstances, when monitoring is misapplied, it could do unintentional harm. The following are some unwanted consequences to be guarded against at all costs. Excessive managerial control could distort clinical judgment and dilute professional responsibility. If explicit criteria and protocols are too rigidly enforced, practice could become stereotyped and innovation hampered, especially if existing guidelines are not being constantly revised. Where malpractice litigation is prevalent, the consequence of the above could be an increase in legal hazard. Most importantly, if the corrective actions taken are punitive, unfair, or otherwise misdirected, the result would be an ambience of fearfulness that leads to resistance, evasion, concealment, and ultimate demoralization.

Some Concluding Remarks Performance monitoring and readjustment, the subject upon which I have lavished so much attention in this work, does not in and of itself create a commitment to quality; it only serves that commitment. A genuine, persistent, unshakable resolve to advance quality must come first. If that is present, almost any reasonable method for advancing quality will succeed. If the commitment to quality is absent, even the most sophisticated methods will fail. Some, acutely aware of the ineradicable presence of human fallibility, have put their faith in redesigning the processes of care in ways that substitute technology for human decisions and actions. It is a worthy and useful approach. But to design and implement such modifications requires the prior commitment to quality that I have already emphasized. Besides, there is a limit to how far this process of "mechanization" can be taken. Another approach to improving the quality of care is to redesign the system of care itself in ways that channel patients, depending on what they suffer from, to the most appropriate and most highly qualified source

138


of care in a regional system of institutions. This, too, is useful and, in some cases, necessary.72 But even in the most specialized institutions, the need for monitoring and readjustment will remain. That the pursuit of quality depends ultimately on human will has been, for me, a source of hope rather of despair. It is exhilarating, even ennobling, to believe that every obstacle to good care can be removed or circumvented if only we were ardently to wish it. I have looked in many places for the source of the commitment that would bring about such dedication to quality. I have seen it in professional values and traditions. It is implicit in the contract between the professions and society — a contract that confers privileges on the former in return for responsibilities toward the latter.73 I have wished to see it in each practitioner's need for the respect of colleagues and the gratitude of patients; in the quest for acceptance, for success, for the joy in virtuosity. Often I have cast the commitment to quality in moral terms, preferring to see it as the ethical imperative that must govern the conduct of all caregivers. More recently, as I contemplate the imminent end of my every activity, a religious, a transcendent element has crept into my thinking. The secret of quality, I wish to believe, is love: love of one's profession, love of one's fellow man, and love of God. I cannot, of course, impose any of my own feelings and beliefs upon the reader. I can only hope that each reader of these pages will extract from them what seems most pertinent and useful and, perhaps, go away strengthened and inspired. And now, farewell.

References

1. Berwick, D.M., Continuous Improvement As an Ideal in Health Care. N Engl 1 Med 320: 53-56, January 5, 1989. 2. Imai, Masaaki, Kaizen. New York: McGraw Hill Publishing Company, 1986. 3. Stiell, I.G., et al. Decision Rules for the Use of Radiography in Acute Ankle Injuries: Refinement and Prospective Validation. JAMA 269: 1127-1132, March 3, 1993. 4. Marrie, T.J., et al. Controlled Trial of a Critical Pathway for Treatment of Community-Acquired Pneumonia. JAMA 283: 749-755, February 9, 2000. 5. Willems, J.S., et al. Cost Effectiveness of Vaccination Against Pneumococcal Pneumonia. N Engl J Med 303: 553-559, September 4, 1980. 6. Donabedian, A. "Accessibility," in Aspects of Medical Care Administration: Specifying Requirements for Health Care. Cambridge, MA: Harvard University Press, 1973. 7. McNeil, B.J., et al. Fallacy of the Five-year Survival in Lung Cancer. N Engl J Med 299: 1397-1401, December 21, 1978. 8. McNeil, B.J., et al. Speech and Survival: Tradeoffs between Quality and Quantity of Life in Laryngeal Cancer. N Engl J Med 305: 982-987, December 22, 1981. 9. McNeil, B.J., et al. On the Elicitation of Preferences for Alternative Therapies. N Engl J Med 306: 1259-1262, May 27, 1982. 10. Ishikawa, K. What Is Quality Control: The Japanese Way. Englewood Cliffs, N.J.: Prentice Hall, 1985. 11. Kessner, D.M., et al. Assessing Health Quality —The Case for Tracers. N Engl J Med 288: 189-194, January 25, 1973. 12. Williamson, J.W., et al. Priorities in Patient-Care Research and Continuing Education. JAMA 204: 93-98, April 22, 1968.

739

140

REFERENCES

13. Donabedian, A. "Health Status As a Determinant of Medical Care Priorities," in Aspects of Medical Care Administration: Specifying Requirements for Health Care. Cambridge, MA: Harvard University Press, 1973. 14. Thomas, J.W.; and Longo, D.R. Applications of Severity Measurement Systems for Hospital Quality Management. Hospital and Health Services Administration 35: 221-243, Summer 1990. 15. Donabedian, A. "Measuring Health," and "Evaluational Uses of Measures of Morbidity and Health," in Aspects of Medical Care Administration: Specifying Requirements for Health Care. Cambridge, MA: Harvard University Press, 1973. 16. Ware, J.E.; and Sherbourne, C.D. The MOS 36-item Short-Form Health Survey (SF-36) Conceptual Framework and Item Selection. Med Care 30: 473-483, June, 1992. 17. Leplege, A.; and Hunt, S. The Problem of Quality of Life in Medicine. JAMA 278: 47-50, July 2, 1997. 18. Wilson, I.E.; and Cleary, P.O. Linking Clinical Variables with Health-Related Quality of Life: A Conceptual Model of Patient Outcomes. JAMA 273: 5965, January 4, 1995. 19. Donabedian, A. The Criteria and Standards of Quality. Volume II of Explorations in Quality Assessment and Monitoring. Ann Arbor, MI: Health Administration Press, 1982. 20. Donabedian, A. Criteria and Standards for Quality Assurance and Monitoring. QRB: Quality Review Bulletin 12: 99-108, March 1986. 21. Donabedian, A. Advantages and Limitations of Explicit Criteria for Assessing the Quality of Health Care. Milbank Q 59: 99-106, Winter 1981. 22. Donabedian, A. The Role of Outcomes in Quality Assessment and Assurance. QRB: Quality Review Bulletin 18: 356-360, November 1992. 23. Donabedian, A. The Assessment of Technology and Quality: A Comparative Study of Certainties and Ambiguities. Int } Technol Assess Health Care 4: 487-496, No. 4, 1988. 24. Peterson, O.L.; Barsamian, E.N.; and Eden, M. A Study of Diagnostic Performance: A Preliminary Report. / Med Educ 4: 797-803, August 1966. 25. Greenfield, S., et al. Peer Review by Criteria Mapping: Criteria for Diabetes Mellitus: The Use of Decision-Making in Chart Audit. Ann Intern Med 83: 761-770, December 1975. 26. Donabedian, A. Using Decision Analysis to Formulate Process Criteria for Quality Assessment. Inquiry 18: 102-119, Summer 1981. 27. Donabedian, A. Methods for Deriving Criteria for Assessing the Quality of Medical Care. Med Care Res Rev 37: 635-698, Fall 1980. 28. Koran, L.M. The Reliability of Clinical Methods, Data, and Judgments. N Engl 1 Med 293: 642-646 and 655-701, September 25 and October 2, 1975.

References

141

29. Demlo, L.K.; and Campbell, P.M. Improving Hospital Discharge Data: Lessons from the National Hospital Discharge Survey. Med Care 19: 1030-1040, October, 1981. 30. Weed, L.L., Medical Records, Medical Education, and Patient Care. Chicago: Year-Book Medical Publishers, Inc., 1971. 31. Westin, A.F. Medical Records: Should Patients Have Access? Hastings Cent Rep 6: 23-28, December 1976. 32. Stevens, D.P. What Happens When Hospitalized Patients See Their Own Records. Ann Intern Med 86: 474-477, April 1977. 33. Donabedian, A., "The Assessment of Need." Chapter III in Aspects of Medical Care Administration: Specifying Requirements for Health Care. Cambridge, MA: Harvard University Press, 1973. 34. Peterson, O.L., et al. "An Analytical Study of North Carolina General Practice, 1953-1954." / Med Educ 31: Part 2, 1-165, December 1956. 35. Clute, K.F., The General Practitioner: A Study of Medical Education and Practice in Ontario and Nova Scotia. Toronto: University of Toronto Press, 1963. 36. Royal College of General Practitioners, What Sort of Doctor: Assessing Quality of Care in General Practice. London: The Royal College, July 1985. 37. Goran, M.J., et al. The PSRO Hospital Review System. Med Care Volume 13, Supplement, April 1975. 38. McCarthy, E.L., Second Opinion Elective Surgery. Boston: Auburn House Publishing, 1981. 39. McDonald, C.J., et al. Reminders to Physicians from an Introspective Computer Medical Record. Ann Intern Med 100: 130-138, January 1984. 40. Gonnella, J.S.; and Goran, M.J. Quality of Patient Care —A Measurement of Change: The Staging Concept. Med Care 13: 467-473, June 1975. 41. Gonnella, J.S., editor, Clinical Criteria for Disease Staging. Santa Barbara, CA: Sysemetrics, Inc. 1982. 42. Horn, S., CSI, Computerized Severity Index Software. Baltimore, MD: Center for Hospital Finance and Management, Johns Hopkins University, 1987. (Also see Horn, S., Measuring Severity: How Sick Is Sick? How Well Is Well? Healthcare Financial Management September, 1986.) 43. Knaus, W.A. Apache II: A Severity of Disease Classification System. Crit Care Med 13: 818-829, October 1985. 44. Le Gall, Jean-Roger, et al. A New Simplified Acute Physiology Score (SAPS II) Based on a European/North American Multicenter Study. JAMA 270: 2957-2963, December 22-29, 1993. 45. Rutstein, D.D., et al. Measuring the Quality of Medical Care: A Clinical Method. N Engl } Med 294: 582-588, March 11, 1976.

142

REFERENCES

46. Leape, L.L. Error in Medicine. JAMA 272: 1851-1857, December 21, 1994. 47. Ishikawa, K., Guide to Quality Control. White Plains, NY: Quality Resources, 1971. (A simple introduction to methods of data presentation and interpretation. See chapters 7 and 8, pages 61-85, on Control Charts.) 48. Farnum, N.R., Modem Statistical Control and Improvement. Belmont, CA: Duxbury Press, 1994. 49. Bundesen, H.N. Effective Reduction of Needless Hebdomadal Deaths in Hospitals. JAMA 157: 1384-1399, April 16, 1955. 50. Saturno, P.J., et al. Reducing Time Delay in the Thrombolysis of Myocardial Infarction: An Internal Quality Improvement Project. Am / Med Qual 15: 85-93, May/June, 2000. 51. Williamson, J.W. Evaluating Quality of Patent Care: A Strategy Relating Outcome and Process Assessment. JAMA 218: 564-569, October, 1971. 52. Williamson, J.W., Assessing and Improving Health Care Outcomes: The Health Accounting Approach to Quality Assurance. Cambridge, MA: Ballinger Publishing Company, 1978. 53. See note 16. 54. Mushlin, A.I.; and Appel, F.A. Testing an Outcome-Based Quality Assurance Strategy in Primary Care. Supplement to Med Care 18: 1-100, May 1980. 55. Brook, R.H.; and Stevenson, R.L., Jr., Effectiveness of Patient Care in an Emergency Room. N Engl J Med 283: 904-907, October 22, 1970. 56. Barrows, H.S., Simulated Patients (Programmed Patients): The Development and Use of a New Technique in Medical Education. Springfield, IL: Charles C. Thomas, Publishers, 1971. 57. Kinnersley, P. Potential of Using Simulated Patients to Study Performance of General Practitioners. Br J Gen Pract 43: 297-300, July 1993. 58. Donabedian, A. Specialization in Clinical Performance Monitoring: What It Is and How to Achieve It. Qual Assur 5: 114-120, November 1990. 59. See note 4. 60. Grimshaw, J., et al. Developing and Implementing Clinical Practice Guidelines. Quality in Health Care 4: 55-64, 1995. 61. Donabedian, A. Quality Assurance in Health Care: Consumer's Role. Quality in Health Care 1: 247-251, 1992. 62. Donabedian, A. Perspectives on the Free Choice of the Source of Personal Health Care. Milbank Q 59: 586-595, Fall 1981. 63. Marshall, M.N., et al. The Public Release of Performance Data: What Do We Expect to Gain? A Review of the Evidence. JAMA 283: 1866-1874, April 12, 2000. 64. U.S. General Accounting Office, Health Care Reform: "Report Cards" Are Useful But Significant Issues Need to Be Addressed. Washington, D.C.: General Accounting Office, 1994. (Publication NEHS-94-219).

Referencess

143

65. Ziegenfuss, J.T., Jr., Health Care Quality Report Cards Receive Grade — Incomplete. Am } Med Qual 11: 55-56, 1996. 66. Eisenberg, J.M., Doctors' Decisions and the Cost of Medical Care. Ann Arbor, MI: Health Administration Press, 1986. (See, in particular, Part II, on "Changing Physicians, Practice Patterns.") 67. Davis, D.D., et al. Changing Physician Performance: A Systematic Review of the Effect of Continuing Medical Education Strategies. Health Affairs Spring 1988, pages 34-48. 68. See note 46. 69. Donabedian, A. The Effectiveness of Quality Assurance. Int / Qual Health Care 8: 401-407, August 1996. 70. Donabedian, A., "Reflections on the Effectiveness of Quality Assurance," in R.H. Palmer, A. Donabedian, and G.J. Povar, Striving for Quality in Health Care: An Inquiry into Policy and Practice. Ann Arbor, MI: Health Administration Press, 1991, pp. 59-128. 71. Becker, M.H., editor, The Health Belief Model and Personal Health Behavior. Health Education Monograph, Volume II: 324-508, 1974. 72. Donabedian, A. Models of Organizing the Delivery of Health Services and Criteria for Evaluating Them. Milbank Q 50: 103-154, October 1972. 73. Donabedian, A. Quality of Care—Who Is Responsible for It?" Am / Med Qual 8: 32-36, Summer 1993.


Appendix 1

Pediatric Appropriateness Evaluation Protecol (AEP)

Pediatric Admission Criteria A. Severity of Illness 1. Sudden onset of unconsciousness (coma or unresponsiveness or disorientation) 2. Acute or progressive sensory, motor, circulatory, or respiratory embarrassment sufficient to incapacitate the ability to move, feed, breathe, urinate, etc. 3. Acute loss of sight or hearing 4. Acute loss of ability to move body part 5. Persistent fever (>37,8°C (100°F) orally or >38.3°C (>101°F) rectally) for more than 10 days 6. Active bleeding 7. Wound dehiscence or evisceration 8. Severe electrolyte/acid base abnormality (any of the following values): a) Na < 123 mEq/L, Na > 156 mEq/L b) K < 2.5 mEq/L, K > 5.6 mEq/L c) CO2 combining power (unless chronically abnormal) < 20 mEq/L, CO2 combining power (unless chronically abnormal) > 36 mEq/L, d) Arterial pH < 7.30, arterial pH > 7.45

145

146

APPENDICES 9. Hematocrit < 30% 10. Pulse rate greater or less than the following ranges (optimally a sleeping pulse for 12 years of age, 90-200/60-120 mm Hg 12. Need for lumbar puncture, where this procedure is not done routinely on an outpatient basis 13. Any of the following conditions not responding to outpatient (including emergency room) management: a) Seizures b) Cardiac arrhythmia c) Bronchial asthma or croup d) Dehydration e) Encopresis (for cleanout) f) Other physiologic problem 14. Special pediatric problems a) Child abuse b) Noncompliance with necessary therapeutic regimen c) Need for special observation or close monitoring of behavior, including calorie intake, in cases of failure to thrive

B. Intensity of Service 1. Surgery or procedure scheduled within 24 hours necessitating a) general or regional anesthesia or b) use of equipment, facilities, or procedure available only in a hospital

Appendices

147

2. Treatment in an intensive care unit 3. Vital sign monitoring every two hours or more often (may include telemetry or bedside cardiac monitor) 4. IV medications and/or fluid replacement (does not include tube feedings) 5. Chemotherapeutic agents that require continuous observation for life-threatening toxic reaction 6. IM antibiotics at least every eight hours 7. Intermittent or continuous respirator use at least every eight hours

Pediatric Day of Care Criteria A. Medical Services 1. Procedure in operating room that day 2. Procedure scheduled in operating room the next day, necessitating preoperative consultation of evaluation 3. Cardiac catheterization that day 4. Angiography that day 5. Biopsy of internal organ that day 6. Thoracentesis or paracentesis that day 7. Invasive CNS diagnostic procedure that day (e.g., lumbar puncture, cysternal tap, ventricular tap, pneumoencephalography) 8. Gastrointestinal endoscopy that day 9. Any test requiring strict dietary control for the duration of the diet 10. New or experimental treatment requiring frequent dose adjustments with direct medical supervision 11. Close medical monitoring by a doctor at least three times daily (observations must be documented in record) 12. Postoperative day for any procedure described in numbers 1 or 3 to 8 above B. Nursing/Life Support Services 1. Respiratory care — intermittent or continuous respirator use and/or inhalation therapy (with chest physical therapy, intermittent posi-

APPENDICES

148

2. 3. 4. 5. 6. 7. 8.

tive pressure breathing) at least three times daily, isoetharine hydrochloride (Bronkosol) with oxygen, Oxyhoods, oxygen tents Parenteral therapy — intermittent or continuous IV fluid with any supplementation (electrolytes, protein, medications) Continuous vital sign monitoring, at least every 30 minutes for at least 4 hours IM and/or subcutaneous injections at least twice daily Intake and/or output measurement Major surgical wound and drainage care (e.g., chest tubes, t tubes, Hemovacs, Penrose drains) Traction for fractures, dislocations, or congenital deformities Close medical monitoring by nurse at least three times daily with doctor's orders

C. Patient Condition Being revised the day before for the day of care 1. Inability to void or move bowels, not attributable to neurologic disorder — usually a postoperative problem Being reviewed within two days before the day of care 2. Transfusion due to blood loss 3. Ventricular fibrillation or EGG evidence of acute ischemia, as started in progress note or in EGG report 4. Fever at least 38.3°C (10PF) rectally (at least 37.8°C (100°F) orally), if patient was admitted for reason other than fever 5. Coma — unresponsiveness for at least one hour 6. Acute confusion state, including withdrawal from drugs and alcohol 7. Acute hematologic disorders — significant neutropenia, anemia, thrombocytopenia, leukocytosis, erythrocytosis, or thrombocytosis — yielding signs or symptoms 8. Progressive acute neurologic difficulties

Appendices

149

Reasons List For Inappropriate Admission 1. Any needed diagnosis and/or treatment that can be done on an outpatient basis 2. Patient admitted for diagnostic testing and/or treatment because patient lives too great a distance from a hospital for it to be done on an outpatient basis 3. Patient admitted for diagnosis and/or treatment because it was not possible to be scheduled on an outpatient basis (although, aside from scheduling, testing and treatment could have been done on an outpatient basis) 4. Patient needs institutional care, but at a level less than an acutecare hospital — general (unspecified) 5. Patient needs care in a chronic disease hospital 6. Patient needs care in a skilled nursing facility 7. Patient needs care in a nonskilled nursing facility 8. Premature admission (e.g., on Friday for a procedure scheduled for the following Monday) 9. Other — specify

For Inappropriate Day of Care For patients who need continued hospital stay for medical reasons 1. 2. 3. 4. 5.

Problem in hospital scheduling of operative procedure Problem in hospital scheduling of test or nonoperative procedure Premature admission Patient "bumped" because of operating room problems Delay due to "40-hour week" problem (i.e., procedures not done on weekend) 6. Delay in receiving results of diagnostic test or consultation needed to direct further evaluation/treatment 7. Other — specify

APPENDICES

150

For patients who do not need continued hospital stay for medical reasons 1. Hospital or physician responsibility a) Failure to write discharge orders b) Failure to initiate timely hospital discharge planning c) Overly conservative medical management of patient by physician d) No documented plan for active treatment or evaluation of patient e) Other — specify 2. Patient or family responsibility a) Lack of family for home care b) Family unprepared for patient's home care c) Patient/family rejection of available space at appropriate alternate facility d) Other — specify 3. Environmental responsibilities a) Patient from unhealthy environment — patient kept until environment becomes acceptable or alternative facility found b) Patient is convalescing from an illness, and it is anticipated that his/her stay in an alternative facility would be less than 72 hours c) Unavailability of alternative facility d) Unavailability of alternative nonfacility-based treatment (e.g., home health care) e) Other — specify

Appendix 1 Note Kreger, B.E.; and Restuccia, J.D. Assessing the Need to Hospitalize Children: Pediatric Appropriateness Evaluation Protocol. Pediatrics Volume 84, number 2, August 1989, pages 242-247.

Appendix 2

Staging Criteria for Cancer of the Breast and Cholecystitis

Stage and Substage

Description: Cancer of the Breast

Description: Cholecystitis

Carcinoma with no skin edema, or ulceration or fixation to chest wall. The axillary nodes are not clinically involved.

1.0

Chronic cholecystitis.

Carcinoma with no skin edema or ulceration or fixation to chest wall. The axillary nodes are clinically involved but less than 2.5 cm in transverse diameter and not fixed to overlying skin or deeper structures of axilla.

2.0

2.1 Chronic cholecystitis with choledocholithiasis. 2.2 Acute cholecystitis. 2.3 Acute cholecystitis with choledocholithiasis. 2.4 Gangrene of gallbladder. 2.5 Acute cholecystitis with localized perforation. 2.6 Empyema of gallbladder.

Carcinoma with one or more of the following: edema of skin of limited extent, involving less than one-third of the skin over the breast; or skin ulceration; or solid fixation of tumor to chest wall; or massive involvement of axillary lymph nodes: (a) single node, or group of fused nodes, measuring 2.5 cm or more in transverse diameter, or (b) chain of nodes extending high into the axilla; or fixation of the axillary nodes to overlying skin or deeper structures of axilla.

3.0

3.1 Gallstone ileus. 3.2 Acute cholecystitis with acute supperative cholangitis. 3.3 Free (gross) perforation of gallbladder. 3.4 Acute cholecystitis and pancreatitis. 3.5 Septicemia (specify organism). 3.6 Shock.

(continued)

151

152

APPENDICES Appendix 2—Continued Description: Cancer of the Breast

Carcinoma with one or more of the following: a combination of any two or more of the five signs listed in Stage 3; or extensive edema of skin over the breast; or satellite skin nodules; or clinically involved supraclavicular lymph nodes; or internal mammary metastases as evidenced by a parasternal tumor; or edema of arm; or distant metastases.

Stage and Substage 4.0

Description: Cholecystitis Death.

Death. The breast cancer criteria were adapted from Donegan (1979) by Gonnella, Louis, and McCord, 1981. The cholecystitis criteria were adapted from Gonnella, Louis, and McCord, 1981. Hornbrook, M.C. "Hospital Case Mix: Its Definition, Measurement and Use. Part II: Review of Alternative Measures." Medical Care Rev. 1982 Summer;39(2): 73-123.

Appendix 3

Pneumonia: Severity Levels

153

Category

2

1

4

3

Cardiovascular

Pulse Rate 130

Life threatening arrhythmias and/or hypotension

Fever

< 100.4° and/or chills

100,5°-102,0°

102,1°- 103,9° and/or rigors

> 104,0°

Hematology

WBC 4,500-11,000/cu mm Bands < 10%

WBC 11,100-20,000/cumm Bands 10-20%

WBC 20,100-30,000/cu mm Bands 21 -40%

WBC >30,100/cu mm Bands > 40%

Chest X-ray or CT Scan

No infiltrate or consolidation visible

Infiltrate and/or consolidation in ^ lobe; pleural effusion

Infiltrate and/or consolidation in >1 lobe but ^3 lobes or cavitation or necrosis of lung

Infiltrate and/or consolidation in >3 lobes

Respiratory

Normal respiration

Dyspnea on exertion

Dyspnea at rest

Apnea

Courtesy of Susan Horn, Johns Hopkins University.

Appendix 4

MEDISGRPS Method for Classifying Cases by Severity and Assessing Progress Towards Improvement

155

Severity Groups KCF Category

2

3

Myocardial infarction (acute) (extension)

3rd degree heart block

1

EKG

Ischemia Atrial fibrillation

Check X-ray

Cardiomegaly

Physical exam

Rales

4

Congestive heart failure Coma Cardiomyopathy

Cardial catheterization Laboratory

p02 60-69 CPK 121-239

pO2 45-59 CPK 240+

Vital signs

Respirations 25-32

Respirations >32

p02 < 45

Q Severity Group I : patients with minimal findings, indicating a low potential for organ failure. Q Severity Croup 2: either acute findings connoting a short time course with an unclear potential for organ failure, or severe findings with high potential for future organ failure. Q Severity Group 3 : both acute and severe findings indicating a high potential for imminent organ failure. Ul Severity Group 4: critical findings indicating the presence of organ failure. *A registered trade name of MediQual Systems Inc. Source: Brewster A., et al., MEDISGRPS: A Clinically Based Approach to Classifying Hospital Patients at Admission. Inquiry 22: 377-387, Winter, 1985.

Figure A4.1. Example of MEDISGRPS severity review timetable: medical service admission with 11-day length of stay.

157


Appendix 5

The Apache II Severity of Disease Classification System

159

Low Abnormal Range

High Abnormal Range Physiologic Variable Temperature — rectal (°C)

Mean arterial pressure — mm Hg

+4

+3

O

O

>41°

39°-40°

O

0

+2

0

+1

+2

+3

O 38.5°-38.9°

O 36°-38.4°

O 34°-35.9°

O 32°-33.9°

O

O

30°-31.9°

180 130-159 110-129 Heart rate (ventricular response)

O

0

0

>180 140-179 110-139 Respiratory rate — (nonventilated or ventilated)

O >50

+4

+1

0

O

70-109

50-69

O

O

O

70-109

55-69

40-54

O

0

0

35-49

25-34

12-24

O O O Oxygenation: A-aDO2 or P,O2 (mm Hg) >500 350-490 200-349 a. FIO2 > 0.5 record A-a DO2 b. FIO2 < 0.5 record only P,O2

O 3.5

2-3.4

0

O

O

O

5.5-5.9

3.5-5.4

3-3.4

2.5-2.9

0

O

1.5-1.9

0.6-1.4

O 1000 cc, hemothoraces > 1000 cc,

Appendices

187

ruptured aorta, retro-peritoneal and pelvic hematoma, pericardial tamponade, and intracranial hematoma. 20. (GENERAL): Autopsies for trauma patients dying within 48 hours of ED arrival.

Cardiovascular Indicators Summary List 1. (CABG 1): Mortality of patients undergoing isolated CABG procedures, subcategorized by initial and subsequent CABG procedures and by emergent and non-emergent procedures. 2. (CABG 2): Patients undergoing isolated CABG procedures returning to OR for treatment of postoperative thoracic bleeding, subcategorized by presence or absence of thrombolytic therapy received within 48 hours prior to CABG. 3. (CABG 3): Intra- or postoperative cerebrovascular accident (CVA) in patients undergoing isolated CABG procedure. 4. (CABG 4): Patients undergoing isolated CABG procedure with intra- or postoperative insertion of intra-aortic balloon pump or other circulatory support device. 5. (CABG 5): Patients with prolonged postoperative length of stay for isolated CABG, subcategorized by emergent and non-emergent procedures. 6. (CABG 6): Patients undergoing non-emergent, initial, isolated CABG procedures receiving internal thoracic (internal mammary) artery graft/s. 7. (CABG 7): Patients undergoing isolated CABG procedures for bypass of only a single coronary artery, excluding CABG procedures only for the left main coronary artery. 8. (PTCA 1): Mortality of patients following PTCA subcategorized by emergent and non-emergent clinical status. 9. (PTCA 2): Patients undergoing non-emergent PTCA with occurrence within the same hospitalization of either an acute MI established post-PTCA or CABG post-PTCA.

188

APPENDICES

10. (PTCA 3): Patients with repeat of PTCA of same lesion occurring within 72 hours of most recent PTCA, subcategorized by emergent and non-emergent status of original PTCA. 11. (PTCA 4): Patients with post-PTCA complications at femoral or brachial artery insertion site, subcategorized by thrombolytic therapy within 48 hours prior to PTCA. 12. (PTCA 5): Patients undergoing attempted or completed PTCA in whom any lesion attempted is not dilated. 13. (MI 1): Mortality of patients with principal discharge diagnosis of acute MI, subcategorized by time of onset of the symptoms and time of death relative to time of presentation to the hospital. 14. (MI 2): Hemorrhagic complications in patients who received thrombolytic therapy for acute MI, subcategorized by complications occurring to patients prior to discharge from the institution initiating therapy and post-transfer complications occurring to patients who received therapy prior to transfer. 15. (MI 3): Patients admitted for acute MI, rule out acute MI, or unstable angina who do not have a discharge diagnosis of acute MI, subcategorized by admission to an ICU, monitored bed or unmonitored bed. 16. (MI 4): Use of thrombolytic therapy in patients with principal discharge diagnosis of acute MI, subcategorized by time of presentation relative to onset of symptoms and by time of initiation of therapy relative to time of onset of symptoms. 17. (CHF 1): Patients with diagnosis at discharge of CHF who did not have a chest X-ray, EGG, electrolytes, and BUN or creatinine performed within 24 hours preceding or following the time of admission. 18. (CHF 2): Patients with diagnosis at discharge of CHF without one of the following etiologies documented by a physician: hypertensive heart disease, cardiomyopathy, aortic or mitral valve disease, chronic ischemic heart disease. 19. (CHF 3): Patients with diagnosis at discharge of CHF and initial chest X-ray finding of CHF without a recorded weight loss of five pounds or more during hospitalization. 20. (CHF 4): Patients with diagnosis at discharge of CHF and chest X-ray finding of CHF, developing laboratory evidence of the hypo- or

Appendices

189

hyperkalemia during hospitalization or lacking two potassium determinations during hospitalization. 21. (CHF 5): Mortality of patients with diagnosis at discharge CHF, subcategorized by the presence or absence of acute florid pulmonary edema as evidenced on chest X-ray within 24 hours following admission and by deaths occurring during the first 24 hours after admission and during the remainder of the hospital stay.

Obstetrical Care Indicators Summary List 1. Patients with primary caesarean section for failure to progress.* 2. Patients with attempted vaginal birth after caesarean section (VBAC), subcategorized by success or failure.* 3. Patients with excessive maternal blood loss defined by either postdelivery red blood cell transfusion or a low post-delivery hematocrit or hemoglobin (Hct < 22%, Hgb < 7 gms), or a significant pre- to postdelivery decrease in hematocrit (decrease > 11%) or hemoglobin (decrease > 3.5 gms), excluding patients with abruptio placenta or placenta previa.* 4. Patients with diagnosis of eclampsia.* 5. The delivery of infants weighing less than 2500 grams, following either induction of labor or repeat caesarean section without medical indications.*,tt 6. Term infants admitted to an NICU within 24 hours of delivery and with NICU stay greater than 24 hours, excluding admissions for major congenital anomalies.* *The Task Force has recommended to the Joint Commission that indicators #1 through #12 be further tested for possible inclusion in the JCAHO performance monitoring system. In addition, the Obstetrical Care Task Force believes that all of the above indicators are of value for use in individual hospital quality assurance programs. 11 The lists of medical indications for induction and for repeat caesarean section are defined in the LCAHO Clinical Indicator Abstract for this indicator. Please note: The wording of each indicator may be subject to revision by the Task Force following the results of further testing.

190

APPENDICES

7. Neonates with an Apgar score of 3 or less at 5 minutes and a birthweight greater than 1500 grams.* 8. Neonates with a discharge diagnosis of massive aspiration syndrome requiring either NICU admission for greater than 24 hours or oxygen therapy.* 9. Neonates with a discharge diagnosis of significant birth trauma.* 10. Term infants with a diagnosis of hypoxic encephalitis or clinically apparent seizure prior to discharge from the hospital of birth.* 11. Deaths of infants weighing 500 grams or more, subcategorized by neonatal deaths, or prepartum stillborns and intrapartum stillborns.* 12. Neonatal deaths of infants with a birthweight of 799-999 grams born in a hospital with an NICU.* 13. Maternal readmissions within 14 days of delivery. 14. Intrahospital maternal deaths up to and including 42 days postpartum during the primary stay or during readmission stay. 15. Infants weighing less than 1800 grams delivered in a hospital without an NICU. 16. Neonatal transfers from non-NICU hospital transferred to a NICU hospital.

Anesthesia Care Indicators Summary List 1. Patients diagnosed with CNS complications occurring during or within one day following the administration of anesthesia. * 2. Patients developing peripheral neurologic deficit during or within two days following the administration of anesthesia.* 3. Patients with cardiac arrest during or within 24 hours following the administration of anesthesia, excluding patients with required intraoperative cardiac arrests.* 4. Patients with acute myocardial infarction during or within two days following the administration of anesthesia.* 5. Patients with unplanned respiratory arrest during or within 24 hours following the administration of anesthesia.*

Appendices

191

6. Unplanned admission of inpatients to the hospital following outpatient procedures involving anesthesia.* 7. Unplanned admission of patients to an intensive care unit following the administration of anesthesia.* 8. Death of patients during or within 48 hours following the administration of anesthesia.* 9. Patients with discharge diagnosis of fulminant pulmonary edema developed during or within one day following the administration of anesthesia. 10. Patients diagnosed with aspiration pneumonia's, occurring during or within two days following the administration of anesthesia. 11. Patients developing postural headache within four days following the administration of spinal or epidural anesthesia. 12. Patients experiencing dental injury during anesthesia care. 13. Patients experiencing ocular injury during anesthesia care.


Appendix 9

A More Modest List of Performance Measures for Hospitals Proposed by the Joint Commission on Accreditation of Health Care Organizations, February, 2000

Core Performance Measures in Five Focus Areas Acute Myocardial Infarction (AMI) • Smoking Cessation Advice/Counseling —AMI patients with a history of smoking, who are given smoking cessation advice or counseling during hospitalization. • Aspirin at Arrival — AMI patients who are given aspirin within 24 hours of arrival or within 24 hours prior to arrival at the hospital. • Reperfusion Therapy: Time from Arrival to Initiation — Timely reperfusion (opening blocked arteries) of eligible AMI patients; time from arrival to initiation of thrombolysis medication administration or primary percutaneous transluminal coronary angioplasty procedure (PTCA). • Aspirin at Discharge — AMI patients who are prescribed aspirin at discharge from the hospital. • Beta Blocker at Arrival — AMI patients who receive beta blocker medication within the first 24 hours of arrival to the hospital. 793

APPENDICES

194

• LVEF < 40% Prescribed ACEI at Discharge — AMI patients with low left ventricular ejection fraction (index of how well the heart functions) who are prescribed an angiotensin converting enzyme inhibitor (ACEI) medication at discharge from the hospital. • Beta Blocker at Discharge — AMI patients who are ideal candidates for beta blocker medication who are given a prescription for beta blockers at discharge. • Intrahospital Mortality-Patients with a primary diagnosis of AMI who expire during hospitalization.

Heart Failure • Patients with Atrial Fibrillation Prescribed Warfarin at Discharge — Heart failure patients with atrial fibrillation (irregular heartbeat) who are given a prescription for oral anticoagulation therapy (warfarin) at discharge from the hospital. • Diet/Weight/Medication Management Instructions at Discharge — Heart failure patients who receive patient education (as documented on their written discharge instructions) regarding all of the following: all discharge medications, weight monitoring, diet, activity level, follow-up appointment, what to do if symptoms worsen. • Assessment of Left Ventricular Function — Heart failure patients not admitted on angiotensin converting enzyme inhibitors (ACEIs) or angiotensin receptor blocking agent (ARBs) medications who have left ventricular ejection fraction (LVEF) evaluated before or during admission. • LVEF < 40% Prescribed ACEI at Discharge — Patients with low left ventricular ejection fraction (LVEF) who are prescribed an angiotensin converting enzyme inhibitor (ACEI) medication at discharge. • Smoking Cessation Advice/Counseling — Heart failure patients with a history of smoking, who are given smoking cessation advice or counseling during hospitalization.

Appendices

195

Community Acquired Pneumonia • Pneumonia Screen or Pneumococcal Vaccination — Patients age 65 or older who are screened for or given pneumococcal vaccination during hospitalization. • Smoking Cessation Advice/Counseling — Pneumonia patients with a history of smoking, who are given smoking cessation advice or counseling during hospitalization, or advice or counseling is given to pediatric caregiver about effects of secondhand smoke. • Oxygenation Assessment — Patients who receive oxygenation assessment (determine amount of oxygen in blood) within 24 hours of hospital arrival. • Blood Cultures — Patients who have blood cultures collected, had them drawn prior to first dose of antibiotic administration in the hospital. • Antibiotic Timing —Time in hours from initial presentation at hospital, to first dose of antibiotics. • Empiric Antibiotic Regimen Non-ICU — For pneumonia patients not admitted to an Intensive Care Unit (ICU), the antibiotic given is consistent with current consensus guidelines (e.g., the American Thoracic Society, Infectious Disease Society of America, and the Centers for Disease Control and Prevention). • Empiric Antibiotic Regimen ICU —For pneumonia patients admitted to an Intensive Care Unit (ICU), the antibiotic given is consistent with current consensus guidelines (e.g., the American Thoracic Society, Infectious Disease Society of America, and the Centers for Disease Control and Prevention).

Surgical Procedures and Complications • Surgical Site Infection within 30 Days (for Selected Surgical Procedures) — Patients undergoing selected surgical procedures who develop a surgical site infection (SSI) within 30 days of the procedure.

APPENDICES

196

• Timing of Prophylactic Administration of Antibiotic — Timing of when patients were given prophylactic (preventive) intravenous antibiotic administration for selected surgical procedures.

Pregnancy and Related Conditions • VBAC Rate — Patients who have had a caesarean section who have a vaginal delivery. • Third- or Fourth-Degree Laceration — Patients who have vaginal deliveries with third- or fourth-degree laceration (tear).

Index

Aberrance monitoring, 101, 115 anecdotal or clinical, 102 statistical or epidemiological, 103 Acceptability, 4, 18, 22, 24, 61, 63, 68, 73, 76 Accessibility, 18, 101 Accreditation, 65 Adaptability to variation, 62, 69 Additional institutional interest, 40 fairness, 42 institutional enhancement, 41 meeting externally imposed requirements, 41 representativeness, 42 risk reduction, 41 Adherence ratio, 69 Adverse events, 31, 101-103, 105, 115 Algorithmic presentation of a given clinical situation, 70 Amenities of care, 18, 20-21, 26 Analysis, 15, 102, 107-108, 127 by person, 108 by place, 108 by time, 108 small area, 108 statistical, 87 Approaches to assessing performance, 46-47 Behavior change, 27, 47, 111, 124, 129

Benchmarking, 64, 112 Branching criteria, 70 Case study, 114 primary, 114 secondary, 115 Case variation, 67, 69-70 Case-mix adjustment, 53, 56, 100 Certification and recertification, of admissions, 93—94 of stays, 95-96 Challenge monitoring, 89, 113-114 Check lists, 83 Circumvention, 125-126 Claim form, 87 Clinical audits, 33 indicators (performance indicators), 109 performance epidemiologist 118, 120 Cohort trajectory monitoring, 112 Common-cause variation, 103, 106 Contextual factors, 134—135 culture, beliefs, values, assumptions, 134-135 leadership, 135 organization, 135 sponsorship, 135 Control limit, 105, 107-108 Controls over the process of care, 93

797

198 Criteria and standards, 60-64, 66-76, 78, 80, 82, 95. See also Criteriastandards explicit, 65-67, 69-70, 72, 111, 122, 137 guided, 64 implicit, 64-67, 69, 72, 111, 114 inferred, 64-65 Criteria maps, 70 Criteria-standards, 61, 64-76, 78, 80, 82, 95 attributes, 61-62 comparison, 66 definition of, 61 degree of specification and explicitness, 62,64 importance of, 61 selecting the Referent of, 74 specification of, 74 steps in formulating, 72 Delineating the field of inquiry, 72 Delphi method, 73 Derivation, 62-63 empirical, 63 normative, 62 Diagnosis-related groups (DRGs), 80 Diagnostic outcomes, 110-111 Direct intervention, 125—126 Education, 22, 46, 120, 127, 129-130 of consumers, 127 continuing, 120, 129-130 professional, 126-127 Educational and motivational activities, 124, 126-127, 129, 131 long-term, 126-129 short-term, 129-132 Effectiveness, 4-5, 7-9, 20, 24, 26, 108, 112, 118-119, 124, 130, 132, 134 Efficacy, 4 Efficiency, 4, 9-10, 17, 23, 26, 62, 71-72, 118 Equity, 4, 22, 24, 26, 61, 86 Facilitation, 125

INDEX Factors influencing the conduct of health care, 128-129 interpersonal exchanges, 129 market mechanisms, 128 organizational mechanisms, 128 political action, 129 Feasibility, 40, 42-43, 76 operational requirements, 42-43 organizational readiness, 42 Feedback, 100, 129-130 Financial records, 78, 86 Formal organization, 135 Functional scale measuring therapeutic outcomes, 111 Goal-attainment monitoring, 110-111 accountant, 111 accounting, 110 Improving the record, 83 Incentives, 96, 98, 125, 130-131 Inspection team, 87 Internal customers, 127 Legitimacy, 4, 16, 22, 73, 119 Made-to-order recording, 84 Maximally effective care, 13, 16 Mean, 61, 63, 73, 103, 104 Median, 61, 63, 73, 103-104 Medical records, 36, 42, 52, 64, 66, 71, 76, 78-80, 82, 84-85, 87-88, 93, 113-114, 122, 126 consequences of deficiencies, 81 deficiencies, 78, 80-81 difficulty of interpretation, 78, 80 doubtful veracity, 79 incompleteness, 78-79 justifying responses, 81 responses and remedies for deficiencies, 81 Method for reaching consensus, 73—74 Methods of classifying patients, 99-100 Computerized Severity Index (CSI), 100 MEDISGRPS, 100

199

Index Methods of classifying patients (continued) Acute Physiology and Chronic Health Evaluation (APACHE), 100 Simplified Acute Physiology Scale (SAPS II), 100 Mode, 8-9, 16, 32, 33, 47, 49, 61, 63, 73-74 Modifying the Assessment Procedure, 82 Monitoring and improving clinical performance, 26 Nominal-group process, 73, 110 Observation, 36, 52, 78, 80, 82, 87-89, 97, 100, 103-105, 107, 113-114, 132, 134 direct, 36, 52, 78, 82, 87-88 outcome, 87-88 process, 47, 49 structure, 87 Observation of practice, 88 Observer error, 79 Obtaining information, 82, 84 Operational factors, 136 Opinion survey, 33 Optimality, 4, 11, 13, 16-17, 23, 26 Optimally effective care, 13, 16 Outcome, 11, 33, 46, 47, 49, 51-57, 6062, 64, 68, 71, 75, 78, 87-88, 96, 100, 102, 108, 110-112 Panel, 13, 15,72,73-75,96 Patient preference, 18-19, 21, 81 Patient—practitioner relation, 18—19, 20— 22, 26, 52, 63, 79, 81 Pattern of periodicity, 104 Penalties, 41, 97, 129, 131 Performance monitoring, 33, 130, 136-137 clinical and anectodal, 33 and readjustment, 136, 137 statistical or epidemiological, 33 Planned reconnaissance, 31-33 Preformulated check list, 83 Problem identification by group discussion, 32

reporting, 31 unsolicited, 31 status index, 112 Problem-oriented record, 83 Process, 22, 34, 46-47, 49, 51-54, 56, 57, 60, 62-65, 68, 70-71, 73, 75, 7880, 87-89, 93, 96, 102, 106, 108, 110, 114-115, 118, 124, 126-129, 137 Quality, 4, 8, 10, 15-18, 20-22, 24-26, 30, 32-34, 40-43, 46, 47, 49-57, 60-61, 63, 65, 67-72, 74, 75, 78, 80-83, 85-89, 92-93, 99-102, 108-111, 113-114, 118-120, 122, 124-125, 127-132, 134-138 assurance, 32, 46, 50, 92, 110, 118-120, 122, 124, 127, 131, 134 components, 4, 7, 18, 24-25, 131 choosing a definition of, 24 circles, 32, 120 components of, 4, 25 monitoring, 32, 41-42, 68, 120, 122, 124, 129-130, 132, 134-136 choosing how, 27 choosing when, 27, 91 concurrent, 92-93 constructing a monitoring system, 27 effectiveness of, 134 externally required, 30 leadership, 118 priorities in, 27, 40 prospective or anticipatory, 92, 93 retrospective, 92, 93 Quality of life, 15, 22, 54, 111 Quality-adjusted year of life, 15 Quartile, 63 Rating, 38, 89 qualitative, 65, 89 quantitative, 89 Readjustments in the system, 124-126 long-term, 125 short-term, 125 Record linkage, 82

200 Recordability, 62, 70-71, 75 Reminders, 98-99, 129-130 Report card, 128 Reward, 129-131, 135 Routine surveillance, 32-33 Routinization, 125—126 Screening efficiency, 62, 71—72 Second surgical opinion, 93, 96-97 Selection of panels of experts, 72 Sentinel effect, 97 event, 101 Special-cause variation, 103, 106 Specified period of time, 112 Standardized patients, 114 specimen, 113 Statistical, control, 104, 106, 108 control chart, 104 reports, 78, 87 Status progress monitoring, 99, 126 Stringency, 62, 71 Structure, 33, 46-47, 49-52, 56-57, 60, 62, 86-88, 108, 118, 124

INDEX Structure-process-outcome model, 49 relative usefulness, 49 selecting approaches, 56-57 Supplementing the record, 82 Surveys, 33, 78, 84-86 of patients and family members, 85 of population, 86 of practitioners, 85 Team of evaluators, 87 Test situations, 78, 89 Therapeutic outcomes, 110-111 Time window, 75 Total quality management, 74 Tracer matrix, 36—37 Tracer method, 33-34, 42 Tracers, 36 Trouble-shooting, 30—31 Unsolicited problem-reporting, 31 Validity, 62-63, 67-69, 81, 87 Verification of information, 82 Welfare of patient, 40-41

An Introduction to Quality Assurance in Health Care

Quality Assurance

Improving Mental Health Care: Commitment to Quality

Competition and Quality in Health Care Markets

Quality, Risk and Control in Health Care

Challenges in Analytical Quality Assurance

Challenges in Analytical Quality Assurance

An Introduction to Community Health

An Introduction to Community Health

An Introduction to Health Psychology

An Introduction to Health Psychology

Power Transformers Quality Assurance

Understanding Managed Care: An Introduction for Health Care Professionals

Envisioning the National Health Care Quality Report

Envisioning the National Health Care Quality Report

Quality Assurance in Continuing Professional Education: An Analysis

Introduction to survey quality

Introduction to Survey Quality

Introduction to Survey Quality

Quality Assurance in Environmental Monitoring: Instrumental Methods

Quality Assurance in Dialysis, Second Edition

An Introduction to Hospitals and Inpatient Care

Quality Assurance for Environmental Analysis

Dairy Processing and Quality Assurance

Agile Software Development Quality Assurance

Dairy Processing and Quality Assurance

Quality Assurance in the Analytical Chemistry Laboratory

Computational Intelligence in Software Quality Assurance

Handbook of Software Quality Assurance

An Introduction to Quality Assurance in Health Care

Quality Assurance

Improving Mental Health Care: Commitment to Quality

Competition and Quality in Health Care Markets

Quality, Risk and Control in Health Care

Challenges in Analytical Quality Assurance

Challenges in Analytical Quality Assurance

An Introduction to Community Health

An Introduction to Community Health

An Introduction to Health Psychology

An Introduction to Health Psychology

Power Transformers Quality Assurance

Understanding Managed Care: An Introduction for Health Care Professionals

Envisioning the National Health Care Quality Report

Envisioning the National Health Care Quality Report

Quality Assurance in Continuing Professional Education: An Analysis

Introduction to survey quality

Introduction to Survey Quality

Introduction to Survey Quality

Quality Assurance in Environmental Monitoring: Instrumental Methods

Quality Assurance in Dialysis, Second Edition

An Introduction to Hospitals and Inpatient Care

Quality Assurance for Environmental Analysis

Dairy Processing and Quality Assurance

Agile Software Development Quality Assurance

Dairy Processing and Quality Assurance

Quality Assurance in the Analytical Chemistry Laboratory

Computational Intelligence in Software Quality Assurance

Handbook of Software Quality Assurance

Recommend Documents