E-Health Systems Diffusion and Use: The Innovation, the User and the UseIT Model

E-Health Systems Diffusion and Use: The Innovation, the User and the USE IT Model Ton A. M. Spil University of Twente, ...

Author: Ton A.M. Spil

44 downloads 577 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

E-Health Systems Diffusion and Use: The Innovation, the User and the USE IT Model Ton A. M. Spil University of Twente, The Netherlands Roel W. Schuring University of Twente, The Netherlands

IDEA GROUP PUBLISHING Hershey • London • Melbourne • Singapore

Acquisitions Editor: Development Editor: Senior Managing Editor: Managing Editor: Copy Editor: Typesetter: Cover Design: Printed at:

Renée Davies Kristin Roth Amanda Appicello Jennifer Neidig Becky Shore Larissa Zearfoss Lisa Tosheff Integrated Book Technology

Published in the United States of America by Idea Group Publishing (an imprint of Idea Group Inc.) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.idea-group.com and in the United Kingdom by Idea Group Publishing (an imprint of Idea Group Inc.) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 3313 Web site: http://www.eurospan.co.uk Copyright © 2006 by Idea Group Inc. All rights reserved. No part of this book may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this book are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Spil, Ton A. M. E-health systems diffusion and use : the innovation, the user and the UseIT model / Ton A.M. Spil and Roel W. Schuring. p. cm. Summary: "This book gives an overview of models on the use and diffusion of information systems in the healthcare sector with particular attention to the role of the user"--Provided by publisher. Includes bibliographical references and index. ISBN 1-59140-423-1 (hardcover) -- ISBN 1-59140-424-X (soft cover) -- ISBN 1-59140-425-8 (ebook) 1. Information storage and retrieval systems--Medicine--Evaluation. 2. Medical informatics--Evaluation. 3. Health services administration--Information technology--Evaluation. 4. Medical care--Information services--Evaluation. 5. Medical records--Data processing--Evaluation. 6. Management information systems--Evaluation. 7. Information technology--Psychological aspects. 8. Diffusion of innovations. 9. Physicians--Attitudes. 10. Computer literacy. I. Schuring, Roel. II. Title. R858.S65 2005 610'.285--dc22 2005004544 British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

E-Health Systems Diffusion and Use: The Innovation, the User and the USE IT Model

Table of Contents

Preface ................................................................................................. vii Introduction ........................................................................................... ix Ton A. M. Spil, University Twente, The Netherlands Roel W. Schuring, University of Twente, The Netherlands Section I: Quantitative Evaluation Chapter I User Acceptance and Diffusion of Innovations Summarized ............... 1 Ton A. M. Spil, University of Twente, The Netherlands Chapter II Predicting Internet Use: Applying the Extended Technology Acceptance Model to the Healthcare Environment .......................... 13 William R. Chismar, University of Hawaii, USA Sonja Wiley-Patton, Louisiana State University, USA Chapter III Understanding Physician Use of Online Systems: An Empirical Assessment of an Electronic Disability Evaluation System .............. 30 Thomas A. Horan, Claremont Graduate University, USA Bengisu Tulu, Claremont Graduate University, USA Brian N. Hilton, Claremont Graduate University, USA

Chapter IV Computer and Stress in Social and Healthcare Industries ................ 61 Reima Suomi, Turku School of Economics and Business Administration, Turku Reetta Raitoharju, Turku School of Economics and Business Administration, Turku Chapter V The UTAUT Questionnaire Items ...................................................... 93

Section II: Qualitative Evaluation Chapter VI Information Systems Success Theoretical Framework ..................... 99 Margreet B. Michel-Verkerke, University of Twente, The Netherlands Ton A. M. Spil, University of Twente, The Netherlands Chapter VII The Dynamics of IT Adoption in a Major Change Process in Healthcare Delivery .......................................................................... 107 Liette Lapointe, McGill University, Canada Lisa Lamothe, Université de Montréal, Canada Jean-Paul Fortin, Université Laval, Canada Chapter VIII Introducing Electronic Patient Records to Hospitals: Innovation Adoption Paths ................................................................................... 128 Reima Suomi, Turku School of Economics and Business Administration, Turku Chapter IX USE IT: The Theoretical Framework Tested on an Electronic Prescription System for General Practitioners ................................ 147 Ton A. M. Spil, University of Twente, The Netherlands Roel W. Schuring, University of Twente, The Netherlands Margreet B. Michel-Verkerke, University of Twente, The Netherlands

Chapter X The USE IT Model Case Studies: IT Perceptions in the Multiple Sclerosis, Rheumatism and Stroke Healthcare Chains ................... 177 Margreet B. Michel-Verkerke, University of Twente, The Netherlands Roel W. Schuring, University of Twente, The Netherlands Ton A. M. Spil, University of Twente, The Netherlands Chapter XI USE IT Interview-Protocol ................................................................ 192

Section III: Qualitative and Quantitative Evaluation Chapter XII Combining Qualitative and Quantitative Methods in IS in Healthcare Revisited ......................................................................... 198 Bonnie Kaplan, Yale University, USA Chapter XIII Evaluation of Health Information Systems: Challenges and Approaches ........................................................................................ 212 Elske Ammenwerth, University of Health Sciences, Medical Informatics & Technology, Austria Stefan Gräber, University Hospital of Saarland, Germany Thomas Bürkle, University of Erlangen, Germany Carola Iller, University of Heidelberg, Germany Chapter XIV A Cross-Cultural Framework for Evaluation ................................... 237 Pekka Turunen, Shiftec, Finland Chapter XV Quantitative and Qualitative Methods: Added Value in Evaluating Electronic Patient Records ............................................. 250 Mirjan van der Meijden, Elkerliek Hospital, The Netherlands

Chapter XVI E-Health Systems: Their Use and Visions for the Future ............... 281 Pirkko Nykänen, Tampere University, Finland Chapter XVII Assessing E-Health ........................................................................... 294 Nilmini Wickramasinghe, Illinois Institute of Technology, USA Elie Geisler, Illinois Institute of Technology, USA Jonathan Schaffer, The Cleveland Clinic, USA Chapter XVIII The Evaluation Roadmap .................................................................. 324 Elske Ammenwerth, University for Health Sciences, Medical Informatics and Technology (UMIT), Austria Jytte Brender, University of Aalborg, Denmark Pirkko Nykänen, Tampere University Hans-Ulrich Prokosch, University of Erlangen, Germany Michael Rigby, Keele University, UK Jan Talmon, Maastricht University, The Netherlands About the Authors .............................................................................. 330 Index ................................................................................................... 339

vii

Preface

At the turn of the century we stumbled into the world of diffusion of information systems in healthcare. It seemed to be more confusion than diffusion when we studied the use of an electronic prescription system for general practitioners. The subject turned out to be much broader and deeper then we imaged at that time. This book gives an overview of efforts throughout the world to understand the innovation, the diffusion, and last but not least the actual USE of information systems. We are proud to present both young, excellent researchers that are on top of the subject at this moment and senior researchers who can guide us on the path of generalization of our ideas. Information technology (IT) is seen as an enabler of change in healthcare organizations but (information) technology adoption decisions in healthcare are complex because of the uncertainty of benefits and the rate of change of technology. Relevance and micro-relevance (sometimes referred to as job relevance) are seen as important determinants for IS success. The (lack of) adoption of IT traditionally found its reasons in resistance of the professionals in healthcare organizations, but this is clearly changing in healthcare today. Or, as one of the attendants in our numerous conference sessions said, “There is no resistance to good change!” The attitude of end users is important as both the technology acceptance model and the theory of planned behaviour show, but the research questions that attack this bastion comes from two sides; first, by making attitude a dependent variable in the quantitative UTAUT model and second, by stating that resistance is the result of the other determinants failing in the qualitative USE IT model.

viii

The beginning of each section is about innovation, diffusion and actual USE. The end of each section is about e-health and how to measure the success of e-health. The middle of each section is about applying qualitative models (Section 2) and quantitative models (Section 1). The fourth and last section wraps it up and combines the best of both worlds. We enjoyed working together with so many high-class researchers. We would like to thank them for their contributions and hope you enjoy the fruits. This book is dedicated to Mart.

Ton A. M. Spil Roel W. Schuring Buurse, 2005

ix

Introduction

Toward a Better Evaluation of the USE of E-Health Systems: Comparing USE IT and UTAUT Ton A. M. Spil University of Twente, The Netherlands Roel W. Schuring University of Twente, The Netherlands

Abstract Developing, implementing, and using information technology in organizations is a complex social activity. It is often characterized by illdefined problems or vague goals, conflicts and disruptions that result from organizational change. Successfully implementing information systems in healthcare organizations appears to be a difficult task. Information technology (IT) is seen as an enabler of change in healthcare organizations, but (information) technology adoption decisions in healthcare are complex because of the uncertainty of benefits and the rate of change of technology. In this book we compare the quantitative

x

Table 1. USE IT compared to UTAUT (Unified Theory of Acceptance and Use of Technology; Venkatesh, Morris, Davis, & Davis, 2003) compared to IS success model (Delone & McLean, 2002) USE IT vs. UTAUT vs. IS Success model

User Domain

IT Domain

Product

Relevance / Performance expectancy/ Net benefits

Requirements/ Effort expectancy / Information quality

Process

Resistance/ Attitude & social influence User satisfaction

Resources/ Facilitating conditions Systems quality

analysis of evaluating the use of information systems of healthcare professionals with the qualitative analysis. The final section shows a combined quantitative and qualitative approach.

Introduction This book can be read in three different ways. If you only want to know some basic notions of technology acceptance and e-health evaluation, it is sufficient to read the first chapter of every section, in which well known authors show us the way. If you only want to learn how these basic notions are applied in healthcare, then you can read the middle chapters of each section. If you want to know the state of the art on e-health evaluation, then you can read the final chapters of each section and the final chapters of each section show the newest models available. The continuous thread through this book constitutes of four dimensions: 1. 2. 3. 4.

The process of introducing an innovation The innovation product The user domain The IT domain

xi

The process in the innovation dimension refers to the innovation process, similar to the process defined by Saarinen and Sääksjärvi (1992) and the innovation process structure of Larsen (1998). The product is the result of this innovation process. This corresponds with the definition of the product by Saarinen and Sääksjärvi and the artifact structure in the framework of Larsen. Also, the IT domain is part of the artifact structure; the user domain represents the organizational structure in Larsen’s framework.

The UTAUT Analysis The first section starts with the basic constructs of quantitative evaluation studies on user acceptance; the editor summarized the work of Davis (1989), Ajzen (1991) and Rogers (1995). Then the quantitative models are applied in two chapters on different e-health systems. The main result of the work of Chismar and Wiley-Patton (Chapter II) is that job relevance is the most important determinant of e-health success. They used an extended technology acceptance model (TAM). Horan and his colleagues combined the TAM and theory of planned behavior models and add contextual determinants (Chapter III). They found that IT infrastructure, organizational processes related to IT, and the physician’s experience with computer use in a clinical setting were more significant for behavioral intention than general attitudes. In Chapter IV, Suomi and Raitorharju used a quantitative analysis to show the relationship between stress and IT. The section closes with the questionnaire ( Venkatesh et al., 2003) in Chapter V. A short summary of this major work is given below. Venkatesh et al. (2003) reviewed eight prominent models of user acceptance and managed to create a unified view with his UTAUT model. Seven constructs appeared to be significant determinants of usage in one or more of the individual underlying models: 1. 2. 3. 4. 5. 6. 7.

Performance expectancy Effort expectancy Social influence Facilitating conditions Attitude toward technology Self efficacy Anxiety

xii

Performance expectancy is the degree to which an individual believes that using the system will help him or her to attain gains in job performance. It relates to perceived usefulness, extrinsic motivation, job fit, relative advantage, and outcome expectations. Effort expectancy is defined as the degree of ease associated with the use of the systems. Related constructs from the underlying models are perceived ease of use, complexity, and ease of use. The social influence is the degree to which an individual perceives that important others believe he or she should use the new system. Related constructs are subjective norm, social factors, and image. Facilitating conditions are defined as the degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system. The constructs related to this determinant are perceived behavioral control, facilitating conditions, and compatibility. The last three determinants are theorized not to be direct determinants of intention. Self-efficacy and anxiety have been modeled as indirect determinants of intention fully mediated by perceived ease of use. But attitude is a different determinant. Venkatesh et al. (2003) defined it as an individual’s overall affective reaction to using a system. Four constructs of existing models are related to this determinant, namely attitude toward behavior, intrinsic motivation, and affect toward use and affect. In three cases the relation between attitude and behavioral intention is significant, so we cannot agree with the choice of Venkatesh to make it an indirect determinant. In Chapter VII we conclude that attitude is the cumulative consequence of effects from the other determinants. Still, the initial attitude is something personal and will influence behavioral intention. To measure the UTAUT determinants, a set of 31 questions are proposed for survey purposes. The questionnaire items are given in Chapter V.

USE IT Analysis The Second section starts out with the basic notions of IS success. DeLone and McLean, and Garrity and Saunders are summarized in Chapter VI. In Chapter VII, Lapointe and her colleagues show a qualitative, in-depth case study with the dynamics of IT adoption. Resistance and adoption factors lead to a better understanding of the challenges to be faced in redesigning work

xiii

processes in an interorganizational setting in healthcare. In Chapter VIII, Suomi studies innovation adoption paths for the electronic patient record. The section ends with the USE IT model, summarized in the following section. We can use a wide range of sources that discuss user perspectives in IT introduction. This section gives a short overview of intriguing literature. First, we present the dimensions of the USE IT-model to predict and evaluate innovation and diffusion of information systems: the innovation dimension and the domain dimension, which results in four determinants for success: relevance, requirements, resistance, and resources. The relevance determinant is defined by Spil, Schuring and Michel-Verkerke (Chapter IX) as “the degree to which the user expects that the IT system will solve his problems or help to realize his actually relevant goals.” The word expects expresses that relevance is a factor that is important in the course of the adoption process, not only in evaluation. The word actually is crucial in their view of relevance. Relevance is not to be confused with the degree to which the user considers outcomes as being positive. The set of outcome-dimensions that someone considers “positive” is larger than the set of outcome-dimensions that are relevant. Imagine a physician, who basically considers IT outcomes of a computer decision support system, such as assistance in diagnosis, disease prevention, or more appropriate dosing of drugs, as positive. This does not automatically imply that the IT adoption is relevant to him; it is only relevant if these dimensions are high on his or her goal agenda. Relevance defined in this way comprises relative advantage (Rogers, 1995), net benefits (DeLone & McLean, as cited in Chapter VI), perceived usefulness (Davis, 1989) and job relevance (Chismar & Wiley-Patton, Chapter II) and results in task support satisfaction, which is a criterion for user satisfaction (Garrity & Sanders, 1998). Within the framework of Venkatesh et al. (2003), it would overlap performance expectancy. Resistance is the personal attitude of all stakeholder groups towards the introduction of an information system (Spil, 2003). The main IS quality aspect of resistance is the attitude and the willingness to change. Pare and Elam (1999) also focus on the attitude of the professional when they assess clinical information systems. The end users have an important role because their norms and values determine the effectiveness of the information system. Resistance was found to be the cumulative effect of the other three determinants (Spil, Schuring, & Michel-Verkerke, Chapter VIII). Expectance of reduced quality of work-life satisfaction, high complexity, and the lack of trialability can result in resistance (Garrity & Sanders, 1998; Rogers,

xiv

1995). Observability reduces resistance (Rogers, 1995). Ajzen (1991) and Venkatesh et al. (2003) relate to this determinant as (behavioral) attitude. Resources are defined as the degree to which material and immaterial goods are available to design, operate and maintain the information system (Spil, Schuring, & Michel-Verkerke, Chapter IX). The main focus of the determinant resources will be on the people and on the costs these people cause. Next to that, the reliability of the information technology and the information systems are considered. Resources defined in this way refer to service and system quality (DeLone & McLean, as cited in Chapter VI), management support, and mature IS function (Saarinen & Sääksjärvi, 1992). Resources (human, physical and monetary components; Ansoff, 1965) are needed to implement the new information system into the organization. The human resources can both be insufficient in time and in experience (risk of technology). Insufficient material resources (Offenbeek, 1996) will have a limiting influence on the other three risk domains. Resources is closely related to the construct of facilitating conditions (Venkatesh et al., 2003) and therefore also related to compatibility and perceived behavioral control. The requirements determinant evaluates the meaning of the information system. Requirements are defined as the degree to which the user needs are satisfied with the product quality of the innovation (Spil, Schuring, & Michel-Verkerke, Chapter IX). This includes such aspects as the functional capability, the ease of start up, and the ease of use. Meeting the end user’s requirements results in high information quality (DeLone & McLean, as cited in Chapter VI), high interface satisfaction (Garrity & Sanders, 1998), and high compatibility (Rogers, 1995). To measure the determinants, the USE IT tool consists of structured interviews. In this way, a more precise insight can be obtained in the nature and relevance of problems and solutions, before implementation and this insight can be tested with the same tool during the evaluation of the implementation. The interview protocol is given in Chapter XI.

E-Health Evaluation It is evident that the use of modern information and communication technology offers tremendous opportunities to improve healthcare. However, there are also hazards associated with information technology in healthcare. Evaluation

xv

is a means to assess the quality, value, effects and impacts of information technology and applications in the healthcare environment, to improve health information applications and to enable the emergence of an evidence-based health informatics profession and practice (Ammenwerth et al., 2004). In the final section, a plea is made for a hybrid evaluation method. In Chapter XII, Kaplan shows that a quantitative evaluation can show what is wrong but not why. Therefore, we need a qualitative method to go into depth. Ammenwerth and her colleagues successfully apply this for a nursing information and communication system (Chapter XIII). Both the validation of the results and the completeness of the results can be supported by triangulation. In Chapter XIV, Turunen proposes a new model for evaluation that encompasses both quantitative and qualitative models. Van der Meijden uses both quantitative and qualitative methods to evaluate electronic patient records in Chapter XV. Nykänen shows the vision and evaluation of e-health in Chapter XVI. In Chapter XVII, Wickramasinghe, Geisler, and Schaffer give a possible approach to assessing e-health. This section is closed with the Declaration of Innsbruck, a roadmap of evaluation in healthcare, in Chapter XVII.

References Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211. Ammenwerth, E., Brender, J., Nykänen, P., Prokosch, H., Rigby, M., & Talmon, J. (2004). Visions and strategies to improve evaluation of health information systems. International Journal of Medical Informatics, 73, 479-491. Ansoff, I. (1965). Corporate strategy. New York: McGraw-Hill. Davis, F. D. (1989, September). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 319340. DeLone, W. H., & McLean, E. R. (2002). Information systems success revisited. Hawaii. Garrity, E. J., & Sanders, G. L. (1998). Dimensions of information success. In E. J. Garrity & G. L. Sanders (Eds.), Information systems success measurement (pp. 13-45). Hershey, PA: Idea Group Publishing.

xvi

Larsen, T. J. (1998). Information systems innovation: A framework for research and practice. In Information systems innovation and diffusion: Issues and directions (pp. 411-434). Hershey, PA: Idea Group Publishing. Offenbeek, M. A. G. v. (1996). Risk Analysis Based on Social and Organisational Dynamics. Groningen: Rijksuniversiteit Groningen. Pare, G., & Elam, J. (1999). Physicians’ acceptance of clinical information systems: An empirical look at attitudes expectations and skills. International journal of Healthcare Technology and Management, 1(1), 4661. Rogers, E. M. (1995). Diffusions of innovations. New York: The Free Press. Saarinen, T., & Sääksjärvi, M. (1992). Process and product success in information systems development. Journal of Strategic Information Systems, 1(5), 266-277. Schuring, R. W., & Spil, T. A. M. (2001). Relevance as a major driver of innovation diffusion of ICT in healthcare organisations. Proceedings of the Hospital of the Future: 1st International Conference on Management of Healthcare and Medical Technology, Enschede, University of Twente. Schuring, R. W., & Spil, T. A. M. (2003). Relevance and micro-relevance for the professionals as determinants of IT diffusion and IT-use in healthcare. In G. Grant (Ed.), ERP & Data warehousing in organizations: Issues and challenges. Hershey, PA: IRM Press. Spil, T. A. M. (2003, August 6). Assessing resistance of professional users as a determinant of IT-diffusion and IT-use in healthcare. Proceedings of ECITE 2002, Paris. Spil, T. A. M., & Schuring, R. W. (2003). Assessing requirements and resources of information systems as determinants of IT-diffusion and ITuse in healthcare. Proceedings of IFIP. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

Section I Quantitative Evaluation

User Acceptance and Diffusion of Innovations Summarized 1

Chapter I

User Acceptance and Diffusion of Innovations Summarized Ton A. M. Spil University of Twente, The Netherlands

Abstract There are many quantitative studies on the behavior of consumers of new products or services. This chapter shows the starting point of three major lines of quantitative research. The next four chapters will lean heavily on these three subjects: diffusion of innovations, technology acceptance, and theory of planned behavior. The first line of research is the diffusion of innovations research that can be traced back to Rogers. One of the most important features of the book is the characterization of the persuasion step with relative advantage as a major driver for acceptation of innovations. The second line is the many applications of the technology acceptance model (TAM) that was introduced by Davis in the late 1980s. The most important result of this article is that perceived usefulness was significantly more strongly linked to usage than was ease of use, both defined in this chapter.

Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

2 Spil

Last but not least, the school of planned behavior is introduced with a summary of the early work of Ajzen, in which he states that at the most basic level behavior is a function of salient information relevant to the behavior. Relevance will be a common thread in this book.

Introduction Predicting user acceptance of innovations used to be a process of looking into a crystal ball before Everett Rogers introduced his book Diffusion of Innovations in 1983 (Rogers, 1983). In 1989, Fred Davis introduced the TAM in the MISQ, which was adapted by a mass of researchers (Davis, 1989). Icek Ajzen, at the same time, approached the same problem from another perspective with the Theory of Planned Behavior (Ajzen, 1991). But still, the diffusion of innovations is very difficult to predict. In the second section of this book, Ajzen argued that a quantitative method of measurement can indicate a diffusion problem but can not easily explain the reason behind the problem. In this section the quantitative methods are shown and todays applications are surveyed. This ends up in the Unified Theory of Acceptance and Use of Technology from a recent questionnaire of Visnawath Venkatesh, Fred Davis, Gordon Davis and Michael Morris (2003), which is shown in Chapter 4 and summarized in the Preface.

Innovation Diffusion Theory The first ingredient to our theoretical framework is based on the innovation diffusion theory by Rogers (1983, 1995). This theory builds on a wide range of empirical studies, including studies in the healthcare sector (e.g. Rogers & Scott, 1997). Diffusion is the process by which 1) an innovation 2) is communicated through certain channels 3) over time 4) among the members of a social system. Diffusion is a special type of communication concerned with the spread of messages that are perceived as new ideas. An innovation is an idea, practice, or object that is perceived as new by an individual or other unit of adoption. The characteristics of an innovation, as perceived by the members of a social system, determine its rate of Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Figure 1. The diffusion curve of three innovations (Rogers, 1995) 100%

Late Adopters

90%

Percent of Adoption

80% 70%

Innovation I

Innovation II

Innovation III

60% 50% 40% 30%

Take-Off

20% 10% 0%

Earlier Adopters Time

adoption. Figure 1 shows the relatively slower and faster, rates of adoption for three different innovations. Innovativeness is the degree to which an individual or other unit of adoption is relatively earlier in adopting new ideas than other members of a social system. Rogers (1995) distinguishes five adopter categories, or classifications, of the members of a social system on the basis on their innovativeness: 1) innovators, 2) early adopters, 3) early majority, 4) late majority, and 5) laggards. •

Innovators are the first 2.5% of the individuals in a system to adopt an innovation. Venturesomeness is almost an obsession with innovators. This interest in new ideas leads them out of a local circle of peer networks and into more cosmopolite social relationships. Communication patterns and friendships among a clique of innovators are common, even though the geographical distance between the innovators may be considerable. Being an innovator has several prerequi-


4 Spil

•

•

•

•

sites. Control of substantial financial resources is helpful to absorb the possible loss from an unprofitable innovation. The ability to understand and apply complex technical knowledge is also needed. The innovator must be able to cope with a high degree of uncertainty about an innovation at the time of adoption. While an innovator may not be respected by the other members of a social system, the innovator plays an important role in the diffusion process: that of launching the new idea in the system by importing the innovation from outside of the system’s boundaries. Thus, the innovator plays a gatekeeping role in the flow of new ideas into a system. Early adopters are the next 13.5% of the individuals in a system to adopt an innovation. Early adopters are a more integrated part of the local system than are innovators. Whereas innovators are cosmopolites, early adopters are localites. This adopter category, more than any other, has the greatest degree of opinion leadership in most systems. Potential adopters look to early adopters for advice and information about the innovation Early majority is the next 34% of the individuals in a system to adopt an innovation. The early majority adopts new ideas just before the average member of a system. The early majority may deliberate for some time before completely adopting a new idea. “Be not the first by which the new is tried, nor the last to lay the old aside,” fits the thinking of the early majority. They follow with deliberate willingness in adopting innovations, but seldom lead. Late majority is the next 34% of the individuals in a system to adopt an innovation. The late majority adopts new ideas just after the average member of a system. Like the early majority, the late majority makes up one third of the members of a system. Adoption may be the result of increasing network pressures from peers. Innovations are approached with a skeptical and cautious air, and the late majority do not adopt until most others in their system have done so. Their relatively scarce resources mean that most of the uncertainty about a new idea must be removed before the late majority feel that it is safe to adopt. Laggards are the last 16% of the individuals in a system to adopt an innovation. They possess almost no opinion leadership. Laggards are the most localite in their outlook of all adopter categories; many are



near isolates in the social networks of their system. The point of reference for the laggard is the past. Laggards tend to be suspicious of innovations and change agents. Resistance to innovations on the part of laggards may be entirely rational from the laggard’s viewpoint, as their resources are limited and they must be certain that a new idea will not fail before they can adopt. Active information seeking by professionals in any field usually is a characteristic of individuals (and organizations) who are elite, research oriented, and endowed with adequate resources. The healthcare professionals who most need medical information resources are often least likely to receive them, as Rogers and Scott (1997) stated. This problem is an example of the general tendency for widening information. The relationship between the different structures is described by Rogers (1995) in his book on diffusion of innovations. Rogers studied the adoption of existing innovations (and much less the development of innovations) and the decision process that is involved. Given that decisions are not authoritative or collective, each member of the social system faces his or her own innovation decision that follows a five step process (see Figure 2): 1. 2. 3. 4. 5.

Knowledge: A person becomes aware of an innovation and has some idea of how it functions. Persuasion: A person forms a favorable or unfavorable attitude toward the innovation. Decision: A person engages in activities that lead to a choice to adopt or reject the innovation. Implementation: A person puts an innovation into use. Confirmation: A person evaluates the results of an innovation decision already made.

The persuasion stage (Stage II) is characterized by relative advantage, compatibility, complexity, trialability and observability. Relative advantage is the degree to which an innovation is perceived as being better than


6 Spil

the idea it supersedes. The innovation decision is made through a cost-benefit analysis in which the major obstacle is uncertainty. People will adopt an innovation if they believe that it will, all things considered, enhance their utility. Compatibility is the degree to which an innovation is perceived as consistent with existing values, past experiences, and needs of potential adopters. Complexity is the degree to which an innovation is perceived as relatively difficult to understand and use. Trialability is the degree to which an innovation may be experimented with on a limited basis. Observability is the degree to which the results of an innovation are visible to others. Relative advantage, compatibility, trialability, and observability are positively related to the rate of adoption, complexity, negatively. Reinvention, which is defined as the degree to which an innovation is changed or modified by a user, is an important issue in the implementation stage (see Figure 2, Stage IV; Rogers, 1995). The most striking feature of diffusion theory (Orr, 2003) is that, for most members of a social system, the innovation decision depends heavily on the innovation decisions of the other members of the system. This influence of peers is not confirmed by Chismar and Wiley-Patton (2003) for physicians. Ease of use is the opposite of the complexity, as mentioned by Rogers (1995). Davis (1989) investigated the relationship between perceived ease of use, perceived usefulness, and self-reported usage. Perceived usefulness is defined as “the degree to which a person believes that using a particular system would enhance his or her job performance.” Perceived usefulness can be regarded as an element of relative advantage, as defined by Rogers (1995). Davis found that the perception of ease of use and usefulness by the users, are more important than objective measures of these factors. His conclusion is that the actual use of a system highly correlates with perceived usefulness and hardly with ease of use. Users are often willing to cope with some difficulty of use in a system that provides critically needed functionality. Although difficulty of use can discourage adoption of an otherwise useful system, no amount of ease of use can compensate for a system that does not perform a useful function. (Davis 1989) Ease of use, on the other hand, seems a prerequisite for considering the usefulness. Davis’ results “are consistent with an ease of use à usefulness à usage chain of causality” (Davis, 1989). This will be further explained in the next section.



Discontinuance Continued Rejection

Continued Adoption Later Adoption Adoption

Rejection 2. Perceived Characteristics of th Innovation

1. 2. 3. 4. 5.

Characteristics of the Decision-Making Unit

1.

3.

2.

Socioeconomic characteristics Personality variables Communication behavior

Relative advantage Compatibility Complexity Trialability Observability

II PERSUASION I KNOWLEDGE Previous practice Felt needs/problems Innovativeness Norms of the social systems 1. 2. 3. 4.

PRIOR CONDITIONS

COMMUNICATION CHANNELS

III DECISION

1.

IV. IMPLEMENTATION

V CONFIRMATION

Figure 2. A Model of stages in the innovation decision process, adapted from Rogers, 1995. The innovation decision process is the process through which an individual (or other decision making unit) passes from first knowledge of an innovation, to forming an attitude toward the innovation, to a decision to adopt or reject, to implementation of the new idea, and to confirmation of this decision.


8 Spil

The Technology Acceptance Model (TAM) People tend to use or not use an innovation to the extent they believe it will help them perform their job better. Davis (1989) referred to this variable as perceived usefulness. It is defined as “the degree to which a person believes that using a particular person believes that using a particular system would enhance his or her job performance”. A second variable, in contrast, is “ the degree to which a person believes that using a particular system would be free of effort”. This is called perceived ease of use. The regression analysis suggested that perceived ease of use may actually be causal antecedent to perceived usefulness (see Figure 3). From multiple disciplinary vantage points, perceived usefulness and perceived ease of use are indicated as fundamental, namely • •

• • •

self-efficacy as “judgements of how well one can execute courses of action required to deal with prospective situations” (Bandura, 1986), cost-benefit paradigm as “cognitive trade-off between the effort required and the quality of the resulting decision” (Beach & Mitchell, 1978), diffusion of innovation, as described in the previous paragraph (Rogers, 1995), evaluation of information reports echoes the distinction between usefulness and ease of use (Larcker & Lessig, 1980), and channel of disposition as a psychological trade-off between information quality and access quality (Swanson, 1982).

Swanson and other information scientists emphasize the conceptual similarity between the usefulness and relevance notion (Saracevic, 1975).

Results A first study at IBM’s development laboratory was used to refine the measurement scales, resulting in the scale items used for Study 2, as given in Table 1. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Figure 3. The causal relation between ease of use and usefulness Perceived ease of use

Perceived usefulness

Usage

Table 1. Scale of items for the TAM model Scale items Usefulness 1. Work more quickly 2. Job performance 3. Increase productivity 4. Effectiveness 5. Make job easier 6. Useful Ease of Use 1. Easy to learn 2. Controllable 3. Clear and understandable 4. Flexible 5. Easy to become skillful 6. Easy to use

The new scales were found to have strong psychometric properties and to exhibit significant empirical relationships with self-reported measures of usage behavior. Usefulness was significantly more strongly linked to usage than was ease of use.

The Theory of Planned Behavior The theory of planned behavior, by Ajzen (1991), is found to be well supported by empirical evidence. The intentions, together with the perceptions of behavioral control, account for considerable variance in actual behavior. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

10 Spil

Intentions are assumed to capture the motivational factors that influence a behavior; they are indications of how hard people are willing to try, how much an effort they are planning to exert, in order to perform the behavior. The word intention is not new and derived from motivational studies. The perceived behavioral control refers to people’s perception of the ease and difficulty of performing the behavior of interest. The view is compatible with the concept of perceived self-efficacy (Bandura, 1986), also studied by Davis (1989). From empirical findings the TPB postulates three conceptually independent determinants of intention: 1. 2. 3.

Attitude toward the behavior Subjective norm Perceived behavioral control

The attitude toward behavior refers to the degree to which a person has a favorable or unfavorable evaluation or appraisal of the behavior in question. The subjective norm refers to the perceived social pressure to perform or not to perform the behavior. The perceived behavioral control refers to the perceived ease or difficulty of performing the behavior and it is assumed to reflect past experience as well as anticipated impediments and obstacles (see Figure 4). Ajzen (1991) extended his theory with the role of beliefs in human behavior. At the most basic level behavior is a function of salient information relevant to the behavior. I would like to emphasize relevance at this point in the summary because it has a central role in this book. Three kinds of salient information (later called beliefs) are: 1. 2. 3.

Behavioral beliefs Normative beliefs Control beliefs

Together they provide the basis for perceptions of behavioral control.



Figure 4. Theory of planned behavior 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Attitude

Subjective norm

Intention

Behavior

Control

Attitudes develop reasonably from behavioral beliefs people hold about the object of the attitude. Normative beliefs are concerned with the likelihood that important referent individuals or groups approve or disapprove of performing a given behavior. The control beliefs deals with the presence or absence of requisite resources and opportunities.

References Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211. Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall. Beach, L. R., & Mitchell, T. R. (1978). A contingency model for the selection of decision strategies. Academy of Management Review, 3(3), 439-449. Chismar, W. G., & Wiley-Patton, S. (2003). Does the extended technology acceptance model apply to physicians. Paper presented at the 36th Hawaii International Conference on System Sciences, Hawaii. Davis, F. D. (1989, September). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 319340. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

12 Spil

Larcker, D. F., & Lessig, V. P. (1980). Perceived usefulness of information: A psychometric examination. Decision Sciences, 11(1), 121-134. Orr, G. (2003). Diffusion of innovations [Book review]. Retrieved March 15, 2003, from http://www.stanford.edu/class/symbsys205/ Diffusion%20of%20Innovations.htm Rogers, E. M. (1983). Diffusions of innovations. New York: The Free Press. Rogers, E. M. (1995). Diffusions of innovations (2nd ed.). New York: The Free Press. Rogers, E. M., & Scott, K. L. (1997). Saracevic, T. (1975). Relevance: A review of and framework for the thinking on the notion in information science. Journal of the American Society for Information Science, 26(6), 321-343. Swanson, E. B. (1982). Measuring user attitudes in MIS research: A review. OMEGA, 10(2), 131-145. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: toward a unified view. MIS Quarterly, 27(3), 425-478.


Predicting Internet Use 13

Chapter II

Predicting Internet Use: Applying the Extended Technology Acceptance Model to the Healthcare Environment William R. Chismar University of Hawaii, USA Sonja Wiley-Patton Louisiana State University, USA

Abstract The technology acceptance model (TAM; Davis, 1989) has been widely used by information technology researchers to gain a better understanding of information technology (IT) adoption and its use in organizations. While TAM has been applied and tested in academic and corporate settings, involving students, business managers, and clerical and administrative types as participants, few studies have evaluated TAM in the healthcare environment. This chapter examines the applicability of the extended technology acceptance model (TAM2; Ventkatesh & Davis, 2000) in the context of physicians’ intention to adopt Internet-based health applications. Data was collected in a survey of pediatricians to see how well the extended model fits in the medical sector. This chapter Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

14 Chismar & Wiley-Patton

discusses the implications, limitations, and possible explanations for the inconsistent results found within the model when applied to such professional users as physicians.

Introduction In an executive order issued on April 27, 2004, United States President George W. Bush called for widespread deployment of health information technology within 10 years. This progressive action by the federal government to advocate and support the use of health information technologies may be the result of reports similar to the Institute of Medicine’s (IOM), To Err is Human: Building a Safer Health System (IOM, 1990), that medical errors kill between 44,000 and 98,000 people in U.S. hospitals each year. According to the National Vital Statistics Report, more people die in a given year from medical errors than from motor vehicle accidents (43,548), breast cancer (42,297) or AIDS (16,516). Also, more than 8.8 million adverse drug events occur in ambulatory care each year (Center for Information Technology Leadership, 2004). To err is human; however, medical errors may be decreased and prevented by integrating clinical processes with information technology (Bates, Leape, & Cullen, 1998). According to a statement issued by the IOM of the National Academies, “To significantly reduce the tens of thousands of deaths and injuries caused by medical errors every year, healthcare organizations must adopt information technology systems that are capable of collecting and sharing essential health information on patients and their care” (IOM). Such systems should be national and integrative in nature and conform to a national health data standard in order to improve the quality and reduce the cost of health care for Americans (IOM, 2003). The Medical Records Institute suggests that the Internet and Internet-based health applications (IHA), for example, electronic health records, eprescribing, and mobile health, are the goals of most healthcare organizations. Others contend that the use of the Internet for electronic medical records, e-billing, and patient scheduling can enable the healthcare industry to reduce its inefficiencies and errors in the care-delivery processes (HIMSS/IBM Leadership Survey, 2000). While the use of IT in healthcare



has increased tremendously, key players, particularly physicians, still have not fully embraced the valuable resource of the Internet. Despite the purported advantages of IT investments in healthcare, many doctors do not widely use IHAs in their clinical practices. Physicians often misunderstand the functions and full potential of the Internet (Wang & Song, 1997). To address this issue, we utilized TAM2 to examine physicians’ intention toward the adoption of the Internet and IHAs. TAM has been well tested and proven to be quite reliable and robust in predicting user acceptance in business related studies. TAM and TAM2 both posit that an individual’s intention to use a system is determined by two primary belief factors: perceived usefulness and perceived ease of use. Davis (1989) asserted that perceived usefulness is the degree to which a person believes that a particular information technology would enhance his or her job performance. Perceived ease of use is the degree to which a person believes that using a particular innovation would be free of effort. TAM2, however, incorporates two additional theoretical constructs: cognitive instrumental processes and social influence processes. Four cognitive factors influence perceived usefulness: job relevance, output quality, result demonstrability, and perceived ease of use. Three social forces influence perceived usefulness: subjective norm, image, and voluntariness (see Figure1).

Figure 1. Extended TAM model (TAM2) Experience

Voluntariness

Subjective Norm

Image

Perceived Usefulness

Job Relevance Intention to Use

Output Quality Result Demonstrabilit y

Perceived Ease of Use

Usage Behavior

Original TAM model



As described by Venkatesh and Davis (2000), job relevance is an individual’s perception of the degree to which the technology is applicable to his or her job. Output quality is an individual’s perception of how well a system performs tasks necessary to his or her job. Result demonstrability is the tangibility of the results of using the technology. Perceived ease of use both directly and indirectly impacts on perceived usefulness. Subjective norm is defined as a person’s perception that people who are important to her think she should or should not use the technology. Image is the degree to which one perceives the use of the technology as a means of enhancing one’s status within a social group. Finally, voluntariness is the extent to which one perceives the adoption decision as nonmandatory (Venkatesh & Davis, 2000). TAM has been tested across a number of industries and technologies, but only four published studies relate to healthcare. Hu, Chau and Tam (1999) investigated physician acceptance of telemedicine technology and found evidence that TAM does not fit well with physicians. A significantly modified version of the original TAM model has been tested for IT adoption by family physicians (Dixon & Stewart, 2000), but did not include the enhancements from TAM2. Chau and Hu (2002) examined physicians’ acceptance of telemedicine technology, following a theory comparison approach. The study evaluated the extent to which the technology acceptance model, the theory of planned behavior (TPB) and an integrated model using both TAM and TPB could explain individual physicians’ technology acceptance decisions. The findings suggested that TAM was more appropriate than TPB and the integrated model for examining technology acceptance by individual professionals. To date, TAM2 has not been tested in the healthcare arena, with the exception of this research. In this study, we tested the applicability of the TAM2 model in the healthcare setting, specifically within pediatrics. The growing importance of the Internet for physicians (Medical Records Institute), lead us to investigate the adoption of the Internet and IHAs within the field of pediatric. Our goal was to help address the needs of the pediatric and medical community as a whole in applying IT (Shiffman, Spooner, Kwiatkowski, & Brennan, 2001).



Method To evaluate TAM2, modifications were made to the original questionnaire to better fit the physician community. The construct “voluntariness” was omitted because neither the use of the Internet and IHA was being mandated, nor was there any expectation that it would be mandated in the foreseeable future. Similarly, the construct “experience” was intended to be used for studies after participants have worked with a system. TAM2 hypothesized a decrease in the strength with which social influence processes affect perceived usefulness and intention to use with increasing experience with a technology over time (Venkatesh & Davis, 1989). Since we were not testing a particular system or IT, we omitted the experience construct as well. The TAM2 questionnaire was adapted and tailored to be more specific to pediatricians. The modifications were based on the findings of a physiciancentered focus group and pretest procedures. Changes made to the original questionnaire to assure face and content validity for the context of this study consisted of the following: 1.

2.

3.

4. 5.

Exclusion of such variables as voluntariness, experience, and usage behavior because we were not testing an actual technology or system; therefore, voluntariness versus mandatory, experience, and self-reported usage were not applicable to this study. The inclusion of two additional questions to the output quality variable based on information ascertained from the physician-based focus group. Concerns of output quality regarding IHAs were pressing among pediatricians in the focus group. Rewording of all sentences to incorporate the nomenclature of and the word pediatrician(s) to questions, where applicable. This change was based on feedback from the pretest and pilot study, in order to increase interest by providing personal and professional appeal and thus enhance response rate. Substituting the word system with IHA for Internet-based health applications or the Internet. Changing the verb tense of some sentences to future tense rather than present tense. For example, the perceived usefulness and perceived



6.

7.

ease of use variables make assumptions that subjects have some experience with the technology in question. Discussions with physicians and the author of TAM assured us that it was appropriate to change the verb tense for the pretest because usefulness and ease of use is speculative (F. D. Davis, personal communication, January 25, 2001). Reduction of the Likert-type scale from a 7 point to a 5 point Likert-type scale, where 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. This reduction was made based on suggestions during the focus group and pilot test. Items were randomly sequenced on the questionnaire to reduce potential ceiling or floor effect that induces monotonous responses to items designed to measure a particular construct.

The full list of questions and their associated constructs is in the appendix. The TAM2 questions were included in an eight-page survey with sections on the current use of computer and Internet-based technologies, anticipated future use of the Internet, barriers to the use of IHAs, attitudes toward the technologies, and open-ended questions allowing respondents to express their views. As this chapter is part of a larger study designed to understand physicians’ attitudes toward and intentions to adopt IHA, we discuss here only the applicability of TAM2 in predicting physicians’ intention. (A complete copy of the survey is available from the authors.) Although the TAM scales have been validated by much prior research, the modified instrument was examined for reliability within the context of pediatricians. Reliability was examined using Cronbach’s alpha values. As summarized

Table 1. Reliability of scale measures Construct Perceived Usefulness Perceived Ease of Use Intention to Use Subjective Norm Image Job Relevance Output Quality Result Demonstrability

Cronbach’s Alpha .86 .85 .83 .86 .92 .75 .86 .72



Table 2. Component extraction and total variance Initial Eigenvalues Total Component

% of Cumulative Total Variance % 8.319

1

10.067

43.768

43.768

2

2.503

10.883

54.651

3

2.067

8.985

65.636

4

1.558

6.774

70.409

5

1.001

4.350

74.760 4.066

7.014 4.974 5.114

in Table 1, all of the values were above .70 the acceptable range recommended by the literature and most were above the .80, which is considered “very good” (Nunnally, 1978). Factor analysis was then conducted to examine convergent and discriminant validity. We performed the maximum likelihood method analysis with a promax rotation, because of the probable correlation between or among the factors examined. As shown in Table 2, five factors were extracted with an eigenvalue > 1.0. Convergent validity is considered to be satisfactory when items load high on their respective construct or factor. Most of the items exhibited a loading higher than 0.50 on their respective factors, with the exceptions of result demonstrability (0.30), perceived ease of use (0.40), and job relevance (0.45). In spite of the loading of these items, desirable convergent validity was achieved. Discriminant validity was evaluated by examining whether each item loaded higher on the construct it measured than on any other construct. Table 3 summarizes the results suggesting that the measurement exhibited somewhat reasonable discriminant validity for the context of this study. However, the factor loadings are not considered as clean or consistent with factor analysis results reported in previous studies (Davis, 1989; Hu et al., 1999; Shiffman et al., 2000; Venkatesh & Davis, 2000). Because our constructs had been chosen a priori (based on theory instead of experience) we Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


did not need to a) interpret the meaning of the variable or b) give a descriptive name to the factors. The identification of the variables and labeling of factors had been completed by the investigators in prior studies (Davis, 1989; Venkatesh & Davis, 2000). Understanding the composition of the TAM2 theory, it is not difficult to see why job relevance and intention to use loaded on the perceived usefulness construct. Result demonstrability, which is the tangibility of the results of using a technology loaded on the perceived ease of use construct. Notice also

Table 3. Component factor loading (PU = Perceived Usefulness; ITU = Intention to Use; JR = Job Relevance; IMG = Image; SN = Subjective Norm; OPQ = Output Quality; PEOU = Perceived Ease of Use; RD = Result Demonstrability.)

PU1 PU2 PU3 PU4 ITU1 ITU2 JR1 JR2 IMG1 IMG2 IMG3 SN1 SN2 OPQ1 OPQ2 OPQ3 OPQ4 PEOU1 PEOU2 PEOU3 PEOU4 RD1 RD2

1 .833 .749 .917 .775 .728 .929 .574 .459

.494*

2

.461*

.404 .756 .896 .890 .589 .300

3

4

.838 1.012 .876 .798 .987 .641 .519

5

.693 .950



that two variables-job relevance (2) and perceived ease of use (1) are listed on factors 1 and 2 due to their moderate loadings on both. Again, because the constructs were chosen a priori, we applied them according to the literature (Venkatesh & Davis, 2000). To address our research question regarding the applicability of TAM2 for pediatricians, we performed separate regression analyses with intention to use (ITU) and perceived usefulness (PU) as the criterion variables all other variables were entered as predictor variables. The results are presented in the following section.

Results The study targeted practicing pediatricians in Hawaii. Of the 205 questionnaire packets mailed, 89 were returned completed and deemed usable. This yielded an effective response rate of 43%. Among the respondents, 58% were males and 42% were females. Almost half of the respondents work in a solo practice, and 71% work in a practice with 10 or fewer physicians. These figures are comparable to U.S. figures as reported by the American Academy of Pediatrics in its survey of members with the exception of practice size. The national average of solo pediatric practices is only 10.6%.

TAM2 Regression Results Regression analyses were used to explain intention toward usage (ITU) also referred to as behavioral intention. Consistent with prior research (Chau & Hu, 2002; Davis, 1989; Hu et al., 1999; Venkatesh & Davis, 2000), perceived usefulness (B = .660, p < .000) was a strong determinant of intention to use. The effects of perceived usefulness and output quality explained 59% of the variance of usage intentions by pediatricians. While perceived usefulness had a significant effect on intention to use, perceived ease of use and the social processes of subjective norm and image did not. Stepwise regression analyses were performed for perceived usefulness, as it is the determinant variable posited by the literature. To explain perceived



Figure 2. Results of regression analyses (significant relationships at p < .05) Subjective Norm

Perceived Usefulness R2 = 0.59

Image Job Relevance

Result Demonstrability

B = .660 (p < .000)

B = .775 (p < .000) Intention to Use R2 = 0.59

Perceived Ease of Use

Output Quality

usefulness, we performed multiple regression analysis with perceived usefulness as the criterion, or dependent variable, and all of the remaining variables, with the exception of intention to use, as possible predictor or independent variables. TAM2 explained up to 59% of variance in perceived usefulness. The factor determining perceived usefulness was job relevance (B = .775, p < .000). Again, perceived ease of use and the social influence processes were not significant at the 0.05 level in this model. Figure 2 shows these results.

Discussion The understanding of factors that influence physicians’ intention to adopt IHAs is important as the Internet becomes increasingly more useful and commonplace in the healthcare setting. This study examined the applicability of TAM2 for predicting physicians, specifically, pediatricians’ intention to adopt IHA. The results suggest that TAM2 was partially adequate and applicable in the professional context of physicians. As theorized, perceived usefulness was found to have a significant and strong influence on physicians’ usage intention. However, perceived ease of use, one of TAM’s core constructs, was not significant. These findings are not consistent with prior studies that have not used physicians as the targeted subjects (Davis, Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


1989; Davis, Bagozzi, & Warshaw, 1989; Venkatesh & Davis, 2000); however, our results are similar to those studies that did use professionals as their test bed (Chau & Hu, 2002; Hu et al., 1999; Jayasuriya, 1998). Consistent with Hu et al. (1999), who evaluated TAM examining physicians’ acceptance of telemedicine, ease of use was found to have no significant influence on attitude and perceived usefulness. Chau and Hu (2002) also examining physicians’ acceptance of telemedicine, reported that perceived usefulness was the most significant factor affecting physicians’ acceptance of telemedicine technology, while perceived ease of use had no significant effect on either perceived usefulness or attitude. Jayasuriya (1998) evaluated TAM in the professional context of nurses. The results showed that computer acceptance and use by nurses would be affected more by their perception of the computer’s usefulness than by the training provided. They concluded that the most important factor that determines the use of computer technology for work in the case of healthcare professionals (especially nurses) would be their perceived usefulness and the level of skill they acquire of the computer applications (Jayasuriya, 1998). Hu et al. (1999) explained that physicians on the average have a higher level of competence, intellectual and cognitive capacity, adaptability to new technologies, and reliable access to assistance in operating technology. Physicians are considerably different from the students, administrative staff, knowledge workers, and system developers typically examined in previous TAM studies. For these reasons, the variables of perceived ease of use may not be sufficient or perceived as critical with this professional user group. Chau and Hu (2002) explained that physicians tended to be more pragmatic in their acceptance of telemedicine technology. Physicians, unlike nonprofessionals, focused on the usefulness of the technology rather than on the ease of using it. Keil, Beranek, and Konsynski (1995) reported that, in determining whether or not to use a technology, usefulness is more important than ease of use. In our context, an important factor for pediatricians’ intention to adopt IHA was whether the technology was useful, relevant, and the output quality sufficient for their completion of daily tasks. Chau and Hu suggested that physicians have relatively strong staff support for operating medical equipment and related technologies as well as a relatively high competence capacity, which allows them to comprehend new technologies quicker than the average population. Also, physicians are willing to adopt Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


beneficial applications of information technology, even if that technology is not easy to use. These factors could account for the lack of weight placed on perceived ease of use. Keil et al. (1995) explained that “no amount of ease of use will compensate for low usefulness”. Perceived ease of use was measured in this study and, contrary to several previous studies, did not have a significant effect on behavioral intention or perceived usefulness. With this in mind, our study suggested that pediatricians are willing to adopt and use IHAs if those applications are perceived as beneficial in helping physicians in the performance their daily jobs. Usefulness is operationalized as increasing pediatricians’ productivity, improving their quality of care, enhancing their effectiveness, and providing overall practical service. Two of the three cognitive instrumental determinants of perceived usefulness, job relevance and output quality, theorized in TAM2 were significant in this study. Again, the results emphasized the pragmatic perceptions that physicians have toward the adoption of information technology. The insignificant effect of social influence processes (subjective norm and image) is interesting as it relates to other studies of TAM2. Venkatesh and Davis (2000) reported that subjective norm had a direct effect on intention to adopt that weakened over time. Our results suggest that, as a whole, pediatricians’ decisions to adopt IHA are not influenced by peer pressures or how they will be perceived if they adopt the technology. Similar to the findings of Chau and Hu (2002), the physicians in our study seemed to be rather independent in making technology adoption decisions, by not putting much weight on subjective norm or image, therefore not having concern about others’ opinions on whether they should or should not adopt IHA. A recent study concluded that the insignificant effects of subjective norms on intention suggest that a physician, when making the technology acceptance decision, may value his or her own assessment more than that of others Chau & Hu, 2002). Our analysis revealed the importance of the perception usefulness and pragmatism by physicians for their adoption and use of information technologies in their clinical practices. This study had several limitations. First, our sample size was small because we targeted a particular subject group, pediatricians in Hawaii. Second, we emphasized IHAs in general as our targeted technology. Thus, caution needed to be taken when generalizing the findings and discussion to other technologies and professional groups. Despite the limitations, results from this study have interesting implications and recommendations for future research. First, TAM2 was able to explain Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


over half of the behavioral intention variance (R2 = 0.59), which previous studies involving physicians were not able to explain (HIMSS/IBM Leadership Survey, 2000; IOM, 2003). Second, the results showing an insignificant effect for perceived ease of use combined with similar findings from previous TAM studies implied that this construct within TAM2 is not applicable in the professional context, specifically, physicians. Third, our findings suggested that physicians operate as individuals rather than as a monolithic group when deciding to adopt IHA, based on the lack of significance placed on subjective norm and image in this study and prior research (Chau & Hu, 2002; Hu et al., 1999). The study presents implications for healthcare IT management. To encourage the individual physician to adopt and use of IHA, the organizational management needs to a) emphasize the usefulness of the technology to the physician, and b) de-emphasize the ease of use of the particular information technology, while focusing on the importance and utility of the technology in performing daily tasks. Recommendations for future research for understanding and thus enhancing physicians’ adoption and use of IHA are as follows: 1.

2.

3.

4.

A modified version of TAM2 would be very useful in assessing physicians’ attitudes toward acceptance of IHAs. In developing this model, new constructs should be tested for the abstract notion of easy to use or “without effort”. Physician-centered variables with regard to ease of use should be defined. Additional research is needed to examine the effects of physicians’ characteristics on IT adoption, evaluation, and comparison of physicians across specialties, disciplines, geographic boundaries, and cultures would be valuable. In addition, future studies should look at the effects of “hands-on” educational interventions on basic TAM2 relationships in a professional context. Perhaps the constructs “ease of use” and “subjective norm” would fair differently in a study if physicians had hands-on experience with an actual technology and if a referent other, as suggested by Rogers (1983), introduced or encouraged use of a particular information technology. Finally, upon examining the literature of social networks and observing the interactions and social integration among healthcare professionals, we



believe that social network theory and analysis will have a particular relevance in understanding the IT adoption and diffusion process of healthcare professionals nationwide. Social network theory is a branch of social science that applies to a wide range of human organizations, from small groups of people to entire nations. The term network refers to a set of objects, or nodes, and the mapping or description of the relationship between the objects (Freeman, 1983). Social network theory is studied as a means of understanding the connection of one individual (e.g., an ITsavvy physician) to others (e.g., other physicians, nurses or support staff). We recommend applying social network theory and analysis to the study of IHAs’ adoption in the healthcare environment for a richer understanding of factors that encourage and possibly inhibit the adoption and ultimate use of the Internet by physicians.

Appendix: Survey Questions Intention to Use • •

Assuming that significant barriers to the use of IHAs are overcome, I intend to use IHAs. If significant barriers did not exist, I predict I would use IHA.

Perceived Usefulness • • • •

IHAs could increase my productivity. IHAs could improve the quality of care that I deliver. IHAs could enhance my effectiveness. IHAs could be useful in my job.

Perceived Ease of Use • •

My interaction with IHAs will be clear and understandable. IHA will be easy to use.



• •

Interacting with IHAs will not require a lot of mental effort. It will be easy to get IHAs to do what I want them to do.

Subjective Norm • •

Pediatricians who influence my behavior think I should use IHAs. Pediatricians who are important to me think I should use IHAs.

Image • • •

Having IHAs will be a status symbol. Pediatricians who use IHAs have more prestige than those who do not. Pediatricians who use IHAs have a high profile.

Job Relevance • •

Usage of IHAs is relevant to the delivery of pediatric care. Usage of IHAs is important to the delivery of pediatric care.

Output Quality • • • •

The quality of consumer health information on the Internet is high. The quality of pediatric information currently on the Internet is high. The quality of professional information on the Internet is high. I expect the quality of future IHAs to be high.

Result Demonstrability • •

IHAs could reduce the cost of my care delivery. I believe I could communicate to others the consequences of using IHAs.



• •

The results of using IHAs will be apparent to me. I would have difficulty explaining why using IHAs may or may not be beneficial.

References American Academy of Pediatrics. Characteristics of pediatricians and their practices: the socioeconomic survey. Periodic Survey #43-Part 1. Retrieved February 15, 2002, from http://www.aap.org/research/ ps43aexs.htm Bates, D. W., Leape, L. L., & Cullen, (1998). Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA, 280(15), (pp. 1311-1316). Center for Information Technology Leadership. (2004). Retrieved April 18, 2004, from http://www.citl.org/research/ACPOE.htm Chau, P. Y. K., & Hu, P. J-H. (2002). Investigating healthcare professionals’ decision to accept telemedicine technology: An empirical test of competing theories. Information and Management, 39, 297-311. Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), 319340. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003. Dixon, D. R., & Stewart, M. (2000). Exploring information technology adoption by family physicians: survey instrument valuation. Proceedings of the AMIA Symposium. Freeman, L., White, D., & Romney, A. (Eds.). (1989). Research methods in social network analysis. Fairfax, VA: George Mason University Press. HIMSS/IBM Leadership Survey. (2000). Healthcare information and management systems society (HIMSS). The 11th Annual HIMSS Leadership Survey Sponsored by IBM: Trends in Healthcare Information and Technology. Retrieved May 6, 2002, from http://www2.himss.org/ survey/2000/survey2000.html



Hu, P, Chau, P., & Tam, K. (1999). Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of Management Information Systems, 16(2), 91-112. Institute of Medicine. (1999). To err is human: Building a safer health system. Washington, DC: National Academy Press. Institute of Medicine. (2003). Patient safety: Achieving a new standard for care. Washington, DC: National Academy Press. Jayasuriya, R. (1998). Determinants of microcomputer technology use: Implications for education and training of health staff. International Journal of Medical Informatics, 50, 187-194. Keil, M., Beranek, P. M., & Konsynski, B. R. (1995). Usefulness and ease of use: Filed study evidence regarding task considerations. Decision Support Systems, 13(1), 75-91. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGrawHill. Rogers, E. M. (1983). Diffusion of innovations (3rd ed.). New York: Free Press. Shiffman, R. N., Spooner, S. A., Kwiatkowski, K., & Brennan, P. F. (2001, September 21-22). Information technology for children’s health and healthcare: Report on the information technology in children’s healthcare expert meeting. J Am Med Inform Assoc., 8, 546-551. Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204. Medical Records Institute. Retrieved May 6, 2002, from http:// www.medrecinst.com/ Wang, K., & Song, K. (1997). The physician and the Internet. Mayo Clinic Procedures, 72(1), (pp. 66-71).


30 Horan, Tulu & Hilton

Chapter III

Understanding Physician Use of Online Systems:

An Empirical Assessment of an Electronic Disability Evaluation System Thomas A. Horan Claremont Graduate University, USA Bengisu Tulu Claremont Graduate University, USA Brian N. Hilton Claremont Graduate University, USA

Abstract This chapter develops a conceptual model for physician acceptance and tests this socio-work structure model using a nationwide survey of physicians (n = 141). The domain focus of this study is physician acceptance of online disability evaluation systems for generating and


Understanding Physician Use of Online Systems

31

managing medical examination reports. The survey measured whether behavioral intention to use the new system varied as a function of IT infrastructure, organizational readiness relating to IT, physician experience with computer use in clinical settings, and both specific and general attitudes toward online system use in clinical settings. Survey findings suggest that each of these factors affects behavioral intent to use online disability evaluation systems and that these factors are more important than generalized attitudes toward online systems or socio-demographic predictors. Findings suggest that work-practice variables are important when considering physicians use of online systems. The chapter concludes with a discussion of implications for devising advanced testing systems that can be used to ensure active physician participation in medical informatics systems

Introduction The use of computers for information management and decision support in the medical field dates back to early mainframe use in large hospital settings (e.g., PROMIS; Westberg & Miller, 1999). In today’s world, managed care has created a flood of information for hospitals and medical practices in which patients see many different medical practitioners during the course of medical treatment. As a result, storage of information regarding a patient becomes distributed and requires access by many different stakeholders at any given time. In the disability evaluation environment, there are myriad histories, assessments, treatments, and correspondences, the management of which places the physician in the role of “information manager.” A major source of this complexity is that these activities often employ different terminology to describe the same functional characteristics of the disability in question (Demeter, Andersson, & Smith, 1996). For instance, a treatment-oriented assessment may use one terminology while a financially-oriented legal assessment uses a very different terminology. Additional complexities arise when physicians need to spend valuable time attempting to clarify these translations as a precursor to their own assessment. As a result, disparate and often insufficient data are collected along with attendant but unnecessary paperwork. These inefficiencies occur especially in the medical evidence collection phase. Indeed, the process of gathering the necessary medical evidence to render a legal judgment can



easily create a recursive loop that significantly lengthens the decision process by several years. This chapter reports on a research study that examined technology acceptance of an online disability evaluation system, which is used by physicians as an integral part of the disability assessment process1. The first part of the chapter provides a conceptual overview of the inquiry, beginning with a brief review of literature pertinent to physician acceptance of technological systems, and then progresses to outline a conceptual model for assessing a physician’s behavioral intent to adopt a new online system. The chapter then analyzes data using both descriptive and multivariate analyses, specifically, path analysis, from a nationwide survey of physicians. The final section attends to the research and practice implications of the survey findings.

Conceptual Overview In their role as information manager, physicians need to recognize when it is necessary to seek additional information sources rather than rely primarily on past experience, stored cognitive knowledge, or heuristics (Westberg & Miller, 1999). Information systems (IS) are designed specifically to handle this type of organizational environment, where computer systems and software applications can synthesize information, provide diagnostic support, and create knowledge repositories that aid physicians. Therefore, it is important to understand the physician decision-making process in order to design an IS that can support this process. This section starts with the physician decision-making literature and continues with the related literature regarding the use of IS to support decision making, including studies relating to physician resistance to use of IS, mediating factors for resistance and use, and the latest studies that report no resistance to “good change”. A consideration of how medical informatics systems can be viewed in terms of the task-technology fit (TTF) literature review concludes the overview section.

Work-Practice Considerations of Physicians Patel, Arocha, and Kaufman (2001) provided a framework for analyzing medical cognition—a subfield of cognitive and decision science devoted to the



33

study of cognitive processes inherent in medical tasks. This framework provides constructs regarding expert decision making at various levels. Their work asserts that “medical knowledge can be constructed as a hierarchically ordered conceptual system that serves to partition problems into manageable clusters of information”. Physicians have the task of making many types of decisions in their daily work routine. These decisions are often complex decisions that require large amounts of information requiring tools and aids to assist in synthesizing the information. Physician decision-making tasks can be located on a continuum ranging from structured, semistructured, to unstructured decision making. The structured decision-making process involves decisions that have been previously evaluated by experts in the field (McKinlay, Feldman, & Potter, 1996). Physicians merely follow a series of steps to arrive at the appropriate decision outcome (Ryan, 1998). The semistructured process involves a combination of processes that have been previously evaluated and implemented and those that are of an unstructured nature. According to McKinlay et al., unstructured decision making is exponentially more complex than the structured decision-making task (McKinlay et al., 1996). Many variables contribute to the complex nature of unstructured decision making, such as, large numbers of decision alternatives, highly variable outcomes or consequences of the decision alternatives, multiple decision makers involved in the decision-making process, and high stakes for decision consequences. In the case of disability evaluation, decision making is a relatively structured process in that the requisite steps required to complete the process is predefined, leaving physicians with fewer unstructured decisions to consider.

Factors Affecting Physicians’ Acceptance and Use of Decision Support Systems in the Clinical Setting There have been patterns of resistance to use of IS in the medical community (Johnston, Leung, Wong, Ho, & Fielding, 2001; Mikulich, Liu, Steinfeldt, & Schriger, 2001), with considerable attention to factors which inhibit or facilitate acceptance by physicians. Current literature suggests that physicians’ resistance is most acute when considering clinical decision support systems (CDSS) in clinical practice (Johnston et al., 2001; Ridderikhoff & Van Herk, 2000). To identify the prevailing attitudes associated with the adoption of computers in clinical practice among physicians in Hong Kong, Johnston et al.



(2001) performed an empirical investigation, surveying 4,850 randomly selected physicians. The survey focused on the details of the physicians’ practice, actual computerization of or intention to computerize clinical functions, attitudes towards computerization, self-perceived computer ability, self-perceived knowledge, and demographic information. Research indicated that many potential psycho-social variables continue to be obstacles for physicians’ acceptance and use of decision support systems in the clinical setting (Johnston et al., 2001). Perceived high implementation costs, added requirements for staff training (Bomba, 1998; Mitchell & Sullivan, 2001), relevance (Schuring & Spil, 2002), managing change (Nielsen, 1998), the disruption to practice (Ridsdale & Hudd, 1994), perceived negative impact on doctor-patient communication (Ridsdale & Hudd, 1994; Thakurdas, Coster, Curr, & Arroll, 1996), and increased pressure on physicians to function in an ever-changing, increasingly complex world of medicine (Greatbatch, Heath, Campion, & Luff, 1995; Mitchell & Sullivan, 2001), are some issues that continue to contribute to physician resistance to the acceptance and use of CDSS. It is clear that a gap exists between the intended use of CDSS in the clinical setting and the actual usage of the system by the physician. Research has shown that there are unexplored variables that mediate and moderate physicians’ resistance to the use of CDSS in the clinical setting, even when the use of the system creates a more accurate assessment than does the physician without the use of the CDSS (Johnston et al., 2001; Kaplan, 1997; Kaplan, Brennan, Dowling, Friedman, & Peel, 2001; Leung, Johnston, Ho, Wong, & Cameo, 2001; Ridderikhoff & Van Herk, 2000). For example, an empirical study was conducted in which physicians were required to use a computer system to solve a number of clinical patient problems with the help of a diagnostic decision support system (Ridderikhoff & Van Herk, 2000). Although 75% of the participants believed that computers were useful in daily clinical work, only two thirds of the group could imagine a theoretical possibility of computers supporting physicians in their diagnosis. Of greater interest was that, in those cases where the physicians’ diagnosis was incorrect and the system’s differential diagnosis was correct, the physician did not reconsider his or her own diagnostic opinion. When asked for their opinion about the system’s differential diagnosis, the participants indicated they found it very useful when the system confirmed their diagnosis but marginally relevant to their judgment when the system refuted their diagnosis.



35

In order to gain a greater understanding of the factors that contribute to physician resistance, it must be recognized that resistance to change is a result of technical and social impact (Spil, Schuring, & Katsma, 2002). While there is considerable evidence regarding the hesitancy that a physician may have toward the use of IT, recent research suggests that such hesitancy is a function of specific system expectations. Spil et al. reported that, although resistance is claimed to be the determinant of IT use by researchers, it is merely the cumulative consequence of other effects that prevents physicians from using IT (Spil et al., 2002). Previous research also found that physicians have positive attitudes about using information systems to access up-to-date knowledge, for continual medical education, for access to healthcare in rural and remote areas, for improving quality of patient care, and for interaction within a healthcare team (Pare & Elam, 1999). It is reported (Pare & Elam, 1999; Spil et al., 2002) that there should be no resistance to a change that can be perceived as “good change.” Therefore, it is important to understand the factors that contribute to resistance in order to increase the use of IT among user groups. This is especially true in terms of understanding structured versus unstructured decision making and the tasks that can be supported through the use of IT.

Task-Technology Fit There is often a gap between the requirements of physicians’ tasks and the functionality of decision support systems that can aid in these tasks (Schuring & Spil, 2002). Early identification of the discrepancies between user requirements and system functionality (requirements-functionality gap) is often the key factor to be addressed in the design of information systems. Current work in the area of task-technology fit (TTF) underscores the need for more research into the requirements-functionality gap. Lucas, Walton, and Ginzberg (1988) suggested that research should be conducted to explore more fully the role of discrepancies and to identify variables that reflect the differences between system features and the user’s needs. Goodhue and Thompson (Goodhue & Thompson, 1995) suggest that TTF is an excellent focus for developing a diagnostic tool for analyzing how information systems can support various tasks in an organization. Specifically, they recommend that it be detailed enough to more specifically identify gaps



between systems capabilities and user needs. Goodhue and Thompson (1995) showed that users can successfully evaluate TTF and that carefully developed user evaluations can be crafted to measure TTF. This study also concluded that the value of a technology is dependent upon the tasks of the user. Zigurs, Buckland, Connolly, and Wilson (1999) researched the linkage between TTF and group support systems (GSS). As part of this research, they developed profiles of the relationship between group tasks and GSS technology and demonstrated the relative impact the TTF model had on group performance. They found that the higher the TTF, the better the groups performed. Dishaw (1999) extended the TTF model further by showing a relationship between the TTF model and variables from the constructs in the technology acceptance model (TAM). The TAM model states that, for technology to be used, the user must perceive that it is easy to use (perceived ease of use) and that it will be useful (perceived usefulness). Dishaw’s research shows that an increase TTF has a direct impact on perceived ease of use and an indirect impact on perceived usefulness. There is little existing research that focuses on the TTF of CDSS for physician work-system practices and work-practice compatibility (Kaplan et al., 2001). Consequently, this is an area of research that must be addressed in order to gain a broader understanding of physician acceptance of IT in medical practice. In short, this literature review underscores the value in empirical investigations of the acceptance and use of clinical decision support systems in a manner that refines and extends traditional theories of technology acceptance and diffusion such as the TAM (Davis, 1989), the theory of planned behavior (Ajzen & Fishbein, 1980), and the theory of reasoned action (Godin & Kok, 1996). However, this empirical study goes beyond the usual measures of systems performance to focus more on the social and behavioral patterns physicians’ employ, such as work-practice compatibility or organizational and technical readiness, when deciding to accept or use a CDSS in practice. Further, this study aims to bring this notion of “fit” together with concepts from technology acceptance and reasoned action and to do so in a manner that explains the specific behavioral intentions for a specific type of relatively structured decision making regarding online disability evaluations.



37

Research Design Technology Acceptance Models A number of technology acceptance models, such as Roger’s diffusion of innovations model, Kwon and Zmud’s diffusion/implementation model, and Davis’s TAM have been developed (Kwon & Chidambaram, 2000). Of these, the TAM has been the focus of many IS researchers and practitioners over the past 20 years and has been used as tool regarding the adoption of IS by organizations and individuals (see the related chapter of this volume). The premise of the TAM is that users’ attitudes towards new technologies are shaped by two related factors: perceived usefulness (PU), and perceived ease of use (PEOU). That is, PU (the degree to which a person believes that using a particular system would enhance their job performance) exerts an influence on users’ PEOU (the degree to which a person believes that using a particular system would be free of effort). This model has been applied in a number of organizational environments and has been a reliable predictor of users’ actual actions, especially among university students and business executives (Chau & Hu, 2001; Kwon & Chidambaram, 2000). However, Succi and Walter (Succi & Walter, 1999) raise some questions about whether TAM can predict the attitudes of physicians towards new IT because of the unique knowledge intensive circumstances surrounding medical decision making. The theory of planned behavior (TPB) attempts to overcome some of the limitations of the TAM by incorporating additional factors that may influence an end-users’ decision to use the technology over and above their initial perceptions. In particular, Chau and Hu (2001) introduced an important new construct, compatibility (the degree to which the use of the new technology is perceived by a person to be consistent with their work practices). Chau and Hu (2001) argued that the physicians in their study would be more likely to consider technology useful if they perceived it to be compatible with their existing work practices. In addition, physicians would consider technology easy to use if they did not need to change their work practices significantly and, hence, compatibility can be seen to favorably affect a physician’s attitude toward accepting new technology.



Another extension to TAM was proposed by Mathieson, Peacock, and Chin (2001) and examined the influence of perceived user resources. They examined perceptions of whether adequate resources can facilitate or inhibit how well individuals perceive that they can execute specific courses of actions. Their study was motivated by the fact that TAM assumes usage of an information system is voluntary and achievable. However, barriers may exist and may inhibit an individual from using a system, even if they are inclined to do so. In relation to this study, the technology needs to be considered within the context of the physicians’ practice. Moreover, given the fact that a physician’s main focus is not technology; it is reasonable to expect that his or her office may not be technically or organizationally equipped to utilize new systems. As a result, the introduction of a new technology in this environment can be disruptive to their current workflow. Therefore, it is important to consider these two constructs in addition to TAM model.

Research Model An analysis of physicians’ use of the online system employing the same constructs that are traditionally used in TAM and TPB was performed. However, a new dimension to the model was added by including variables that capture work system practices such as computer related activities within the current work practice system and environment. Whereas the TAM and TPB models focus primarily on individual use of technology in a medical practice, this research explored the components of organizational and work systems practices and their influence on users’ behavioral intent to use systems. Moreover, this research examined whether these components exert a stronger influence on a physicians’ willingness to use an online system compared to the individual user constructs traditionally studied using TAM and TPB models. Figure 1 illustrates a modified version of TAM to include constructs that capture the organizational and work-practice variables that will aid these models in better explaining user acceptance and identifying possible change in management strategies. The independent variables are grouped into four categories: a) Social-Demographic, b) Organizational/Technical Readiness, c) Attitude, and d) Work Practice Compatibility.



39

Figure 1. Proposed research model

Research Questions Drawing upon this research model, the study was guided by the following research questions: 1.

2.

To what extent is physician behavioral intent to use online disability evaluation systems influenced by social demographic, readiness, attitudinal, and work practice compatibility factors? To what extent is physician behavioral intent to use online disability evaluation systems influenced by the overall “fit” between new innovation and the technical readiness of current practices?



Research Methodology To address these research questions, this study analyzed data from a nationwide survey of physicians. This survey was conducted by the authors drawing upon a nationwide network of physician providers made available by a private sector company that was developing a new online disability evaluation system. From an overall population of 10,000 physicians, a survey was sent to 278 representing a stratified sample of active disability providers chosen from a population of 500 disability evaluation providers who met a minimum threshold in terms of their activity in disability evaluations (at least 25 evaluations per year). It is important to note that the 500 physician population did not include the top 40 providers in terms of total evaluation conducted. This exclusion was made in consultation with the private sector company; the rationale being that a) the top providers had a very strong economic reason to report favorably on the system, as their level of economic participation with the company was substantial, and b) several of these top providers had been already exposed to the online system in a testing environment. In this sense, the sample represented the most likely next tier of online system users (the implications of this sample selection are addressed in the discussion section). A total of 144 surveys were received, with 141 valid questionnaires, representing a return rate of 52%. The survey instrument contained 36 items on a three-page survey. This instrument contained various question types such as check-off, fill-in, and a 5 point Likert scale, with anchors at strongly agree and strongly disagree or at extremely important and not important. For the constructs work process compatibility and social-demographic factors, open-ended or multiple-choice questions were used. The questions were selected for their theoretical importance as well as their potential relevance to practice after a series of meetings with the company.

Findings Descriptive Statistics Among the 141 responses, 60% of the respondents were between 40 to 60 years old; 77% were male, and 23% were female; both genders represented



41

various specialties. A majority (82.5%) of the physicians said that they use computers more than several times a week. Among the respondents of the survey, 69% of the practices have less than 10 staff members. Only 17% reported that they have more than 10 staff members working in the practice. In their practice, 67.4% of the physicians use an Internet browser, 75.4% use office applications, and 11% did not have an Internet connection at their practice. The general pattern of descriptive findings suggests a group that is relatively accepting of technology (see Figure 2). While 78.5% of the physicians believe that computers make their job easier, 60.3% of the physicians believed that it takes a lot of time to find information on the Internet. In terms of the new disability online system, 67.4% believed that they would have enough technology resources and skills to use the new online system. Physicians said that they would use the new online system to file all reports (59.3%), and 19.5% reported that they would partially use it. A similar majority (58.9%) of the physicians said that they would use the new online system directly (i.e., they would personally prepare the online report either during or after the evaluation), and 25.6% said that they would use the new system indirectly (i.e., they would prepare the report on paper and the administrative staff will enter the data online). This compares to a reported current (offline) practice where 57.4% of the physicians often dictate reports to transcription services; 14.2% of the physicians often prepare reports by themselves, and the administrative staff types and enters them; and 24% of the physicians often type their own reports. In terms of multivariate analyses, there were three dependent variables of interest. The first was the measure of how often they would use the new online system. The second variable was the measure of their intention to use the new system directly, and the third was a measure of their intention to use

Figure 2. Selected descriptive results



Figure 3. Results of correlations

the new system indirectly. Pearson product moment correlations for the key variables are presented in Figure 3. At the bivariate level, there are significant correlations with several of the key attitudinal, readiness, and compatibility factors (i.e., all classes except socialdemographics have significant bivariate correlations). These relationships were often quite strong; for example, there was a strong correlation (r = .522; sig. = .01) between the belief that the new system would make report reviewing easier and intention to use the new system frequently. For the behavioral intention measurement of physicians entering data directly, significant positive correlations exist for frequency of computer use and the belief that it takes a lot of time to find information on the Internet. This finding is somewhat counterintuitive; there is a positive correlation between agreement with this statement and behavior intention to use the new online system. While the attitudinal item “It takes me a lot of time to find information I need on the Internet” was originally construed to be a measure of general technology attitude (agreeing with the statement suggested a negative attitude), these findings raise questions as to construct validity. It might be as much a measure of experience for example, with more experienced users having a realistic assessment of the time it can take to find items. Use of office applications and ease of report generation were significantly correlated with two behavioral intent measurements, namely, Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


43

direct physician data input and how often physician will use the system. Significant positive correlations exist between physician typing own report and physician entering the data directly, and conversely, significantly negative correlations exist between physician using staff to type the report and the physician entering the data directly.

Multivariate Data Analysis In order to analyze the variables that contributed significantly to physician intentions to use the new online system, three stepwise multiple regressions were performed. The variables were organized into three categories that traditional technology acceptance theories indicate are highly predictive of behavioral intent, namely, a) Social-Demographic, b) Organizational/ Technical Readiness, c) Attitude (see Figure 1). A fourth category was added, d) Work Practice Compatibility, to analyze the predictive nature of physicians’ technologically ready work environment and the nature of the task-technology fit on physicians’ intention to use the system. As noted above, behavioral intent was measured by assessing how often physicians would be willing to use the new system and if they were willing to use it, whether that would use it directly or indirectly. Figures 4, 5, and 6 illustrate these relationships. The stepwise regression for predicting how often physicians would be willing to use the new online system produced a model with an R2 = 0.447, which demonstrates that the five independent variables accounted for 44.7% of the dependent variable (see Figure 4). Of the five variables, two were members of the Organizational/Technical readiness category and three were part of the Attitude category. The most significant predictor for this variable

Figure 4. Stepwise multiple regressions



was the physicians’ belief that the new online disability evaluation system was a good idea (Beta = .226, sig t = .004). Additionally, variables that measured physicians’ belief that additional feature are useful (Beta = .214, sig t = .007) and physicians’ frequency of computer use (Beta = .202, sig t = .005) were also significant predictors of the behavioral intent to use the new online system. In terms of a physician’s intention to directly use the new online system (Direct), seven variables entered the regression equation (see Figure 5). Of these seven variables, one was in the Organizational/Technical Readiness category, four were in the Attitude category and two were in the Work Practice Compatibility category. The R 2 was 0.439, which demonstrates that the independent variables accounted for 43.9% of the dependent variable. The most significant predictor of Behavioral Intent (Direct) was the “belief that preparing for the exam is easy” (Beta = .284, sig. t < .001). This was followed by, “physician uses staff to type report” (Beta = -.270, sig. t < .001). Other significant predictors include the “belief that computers make their job easier” (Beta = .265, sig. t = .001), frequency of computer use (Beta = .250, sig. t = .001), and the “belief that it takes a lot of time to find information on the Internet” (Beta = .243, sig. t = .001). For physicians who indicated that they would not enter the data themselves (Indirect), five variables entered the regression equation (see Figure 6). Of these five variables, two were members of the Attitude category and three were in the Work Practice Compatibility category. The R2 was 0.243, which demonstrates that the independent variables accounted for 24.3 % of the dependent variable. The most significant predictor of Behavioral Intent (Indirect) was “physician uses staff to type report” (Beta = .324, sig. t < .001). Two other significant predictors included “physician types own




45


report” (Beta = -.284, sig. t = .001) and the “belief that it takes a lot of time to find information on the Internet” (Beta = -.239, sig. t = .002).

Path Analysis The results of path analysis with dependent variable Behavioral Intent for Direct/Indirect Use and Behavioral Intent of Use Frequency are reported next. Upon completion of stepwise multiple regressions and bivariate correlation analysis, the estimated model indicates that work practice compatibility has significant direct effects on predicting the way physicians will use the new system. While predicting Use Frequency, Attitude and Perceived Usefulness are the only two variables that have significant direct effects. This signifies that physicians’ frequency-of-use predictions are based on the usefulness of the system and their attitude towards the system. However, the decision regarding how they will integrate the new system into their practice can be predicted by the work practice compatibility variable. Social Demographics and Perceived Readiness variables did not have any direct effect on behavioral intent. Behavioral Intent for Direct/Indirect Use The analysis of the data revealed that the only significant variable for predicting INDIRECT use was work practice compatibility. Figure 7 presents the estimated path model. Table 1 illustrates significant predictors of INDIRECT use with the corresponding error vector. The results of the estimated model indicate that work practice compatibility variables (WPC13c



Table 1. Prediction of Physicians indirect use of online system Independent Variables

Beta

t

Sig. t

WPC13c – Physician prepares reports and administrative staff types and enters.

.358

4.584

< .001

WPC13b – Physician types their own report

-.253

-3.249

.001

R = .414 R2 = .171 Adj. R2 = .159

E=

1 - R 2 = .896

and WPC13b) were good predictors of physician acceptance as measured by intent to use the new online system indirectly. The single error vector was between .85 and .95, indicating a moderate causal relationship. The path coefficients and error vectors for the estimated model are given in Figure 7. The decomposition table for bivariate covariation is given in Table 2. Analysis of the decomposition table reveals that all non causal values (< 0.05) showed a good relationship between the model prediction and correlation. Thus, it appears that the expected and observed outcomes are virtually the same for 100% of the relationships. Behavioral Intent of Use Frequency The analysis of the data revealed that the significant variables for predicting the frequency of use were perceived usefulness (PU29) and attitude (A24, A25) where as work practice compatibility variable did not stay in the regression equation. This indicates that the frequency of use is driven by the usefulness of the system and user attitude towards the system rather than the work practice

Table 2. Decomposition table for BI33 indirect use

Original Covariation Direct Effects Indirect Effects Total Effects Noncausal

33I/13c 0.33 0.36 0.00 0.36 -0.03

33I/13b -0.21 -0.25 0.00 -0.25 0.04



47

Figure 7. Estimated path model for BI33 indirect use

compatibility variables. Table 3 illustrates significant predictors of frequency of use. Tables 4 through 7 illustrate the remaining significant predictors of the other variables that remained in the regression equation. The calculations of the error vectors are presented in Table 8. The path coefficients and error vectors for the estimated model are given in Figure 8. The decomposition table for bivariate covariation is given in Table 9. The results of the estimated model indicate that eight variables are moderate to good predictors of physician acceptance as measured by the frequency of intended use of the new online system. Two of the eight error vectors (PU29



Table 3. Prediction of Physician’s frequency of use of online system Independent Variables

Beta

t

Sig. t

PU29 – The online system would allow you to generate a narrative report with minimal keystrokes.

.303

4.269

< .001

A24 – This online evaluation system is a good idea.

.305

4.042

.001

A25 – I would find the actual experience of using this online system difficult.

.252

3.520

< .001

R = .640 R2 = .409 Adj. R2 = .396

Table 4. Prediction of PEOU and PU for A25 Independent Variables

Beta

t

Sig. t

PU32 – The online system would allow you to customize a report.

.201

2.615

.010

PEOU17 – I find it easy to learn how to use new software applications.

.209

2.731

.007

PU15 – Computer systems do not help me to save time.

.359

4.861

< .001

R = .556 R2 = .309 Adj. R2 = .294

Table 5. Prediction of PEOU and PU for A24 Independent Variables

Beta

t

Sig. t


.209

2.600

.010

PU31 – The online system would allow you to electronically submit the completed report in a secure manner instead of faxing.

.378

5.414

< .001

PU16 – Computer systems make it easier to do my job.

.256

3.056

.003

R = .647 R2 = .419 Adj. R2 = .406



49

Table 6. Prediction of PEUO for PU15 Independent Variables

Beta

t

Sig. t

PEOU18 – It takes me a lot of time to find information I need on the Internet.

.429

5.597

< .001

Independent Variables

Beta

t

Sig. t

PEOU18 – It takes me a lot of time to find information I need on the Internet.

.154

1.996

.048


.513

6.644

< .001

R = .429 R2 = .184 Adj. R2 = .178

Table 7. Prediction of PEUO for PU16

R = .601 R2 = .362 Adj. R2 = .352

Table 8. Calculation of error vectors e=

Endogenous Variable

R

R2

BI34 – Frequency of use

.640

.409

.769

A24 – This online evaluation system is a good idea.

.647

.419

.762

A25 – I would find the actual experience of using this online system difficult.

.556

.309

.831

PU15 – Computer systems do not help me to save time.

.429

.184

.903

PU16 – Computer systems make it easier to do my job.

.601

.362

.799

PU29 – The online system would allow you to generate a narrative report with minimal keystrokes.

.260

.068

.965

PU31 – The online system would allow you to electronically submit the completed report in a secure manner instead of faxing.

.224

.050

.975

PU32 – The online system would allow you to customize a report.

.351

.123

.936

1- R 2



Table 9. Decomposition table for BI34 frequency of use


34/29 0.47 0.30 0.00 0.30 0.17

34/24 0.52 0.31 0.00 0.31 0.21

34/25 0.43 0.25 0.00 0.25 0.18

29/18 0.26 0.26 0.00 0.26 0.00

24/16 0.51 0.26 0.00 0.26 0.26

24/31 0.52 0.38 0.00 0.38 0.14

24/17 0.44 0.21 0.22 0.43 0.02


25/32 0.36 0.20 0.00 0.20 0.16

25/17 0.36 0.21 0.07 0.28 0.08

16/18 0.40 0.15 0.00 0.15 0.24

16/17 0.59 0.51 0.00 0.51 0.07

31/17 0.22 0.22 0.00 0.22 0.00

15/18 0.43 0.43 0.00 0.43 0.00

32/17 0.35 0.35 0.00 0.35 0.00

25/15 0.45 0.36 0.00 0.36 0.09

and PU31) were higher than .95, indicating a weak causal relation, and two of the remaining six error vectors (PU32 and PU15) were between 0.85 and 0.95, indicating a moderate causal relationship. The error vectors for the variables of BI34 (.769), A24 (0.762), A25 (.831), and PU16 (0.799) were all less than .85, indicating a good relationship. The R2 value of 0.409 (Q34), 0.419 (A24), 0.309 (A25), and 0.362 (PU16) indicate a strong relationship, accounting for 40.9%, 41.9%, 30.9%, and 36.2% respectively of the variation in the dependent variable. The other R2 values of 0.068 (PU29), 0.050 (PU31), 0.123 (PU32), and 0.184 (PU15) indicate a weak to moderate relationship. Analysis of the decomposition table reveals that seven of the noncausal values out of 15 were more than 0.10, indicating a difference between what the model predicts and what the correlation describes. Five noncausal values (< 0.05) showed a good relationship between the model prediction and correlation. Three noncausal values (between 0.05 and 0.10) showed a moderate relationship between the model prediction and correlation. Thus it appears that the expected and observed outcomes are virtually the same for about 53% of the relationships.



51

Figure 8. Estimated path model for BI34 frequency of use



Discussion All levels of analyses reveal a pattern of support for the notion that behavioral intent of the physician to use the new online system is a function of several attitudinal, organizational, technical, and work system factors. When behavioral intent was measured on a general basis (how often the physician would be willing to use the new online system), the physicians’ organizational and technical readiness (Construct B) as well as specific attitudinal measures toward the online system (Construct C) were more significant in predicting physicians’ behavioral intent to use the system than social-demographic or work practice compatibility measures (Constructs A and D respectively). When physicians’ behavioral intent to use the system was measured in terms of interaction (whether they would directly input the information in the system or whether they would delegate this responsibility to a staff member), specific attitudinal measures toward the online system (Construct C) and work practice compatibility measures (Construct D) were significant in predicting their behavioral intent. Organizational and technical readiness of the physicians’ work environment (Construct B) again was significant, but only for physicians directly inputting information into the new online system. Thus, it appears that as the physician considered discrete activities pertaining to the new system, these responses were less associated with more global attitudes regarding technology. This demonstrates the fact that when individual behavioral intent is assessed (i.e., physicians are queried about interacting with the system either directly or indirectly), work practice compatibility issues play a more significant role in predicting behavioral intent than do the measurements associated with traditional theories of technology acceptance. Moreover, the more technologically ready a physician’s current practice was (work practice compatibility) the more likely they were to use the system. In two of the three behavioral intent measurements, the constructs from traditional theories of technology acceptance were not as predictive of behavioral intent as were the constructs for technical readiness and work practice compatibility. The results of the path analysis also suggest that work system compatibility is important for a detailed understanding of online system usage2. These observations present an interesting challenge to the TAM and TPB models, which state that general attitudes toward a system’s perceived usefulness, ease of use, behavioral control, and subjective norms were the primary predictors of behavioral intent. Whereas these two models identify Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


53

variables contributing to individual variables that effect behavioral intent, they do not consider organizational work system readiness as having a high level of predictive power of behavioral intent. This supports the notion that overall “fit” is important; that is, technology acceptance can be seen not as an isolated event, but as something that represents one step in the continuum of physicians’ work processes and decision-making processes. Looking more broadly, it is recognized that these patterns are suggestive of a very complex socio-technical phenomena, and that there is much still to be done in terms of a comprehensive predictive model of physician adoption of technology (Spil et al., 2002). Analysis of the data from this research provides promising empirical support for Spil (2002) and Pare and Elam’s (1999) claim that physician resistance to technology is not merely a factor of behavioral attitudes towards technology based simply on how useful and easy the system is to use. Rather, it is based on complex social attitudes and norms that are challenged when new technology is introduced into the workplace (Spil et al., 2002). Thus, future research is needed to provide a deeper understanding of the nature of physician use of technology by analyzing these complex social attitudes as they relate to technology acceptance and use.

Implications in Research The field of medical informatics is moving away from the simplistic notion of physician acceptance or resistance, toward a more nuanced understanding of the factors surrounding relative acceptance or resistance (Pare & Elam, 1999; Schuring & Spil, 2002; Spil et al., 2002; Succi & Walter, 1999). This study examined one particular form of physician online use: disability assessment. The results provided previously are promising for several reasons. First, they suggest that physician acceptance is a function of several factors, which include the organizational and technical readiness of the current work system environment within which physicians operate. A next step for research would be to conduct confirmatory analysis in related, relatively unstructured clinical disability treatment settings to better understand how attitudinal, organizational, and work system factors operate in these various milieus. A second path for future research emanates from the design limitation of current research models. Most significantly, the survey assessed behavioral intent to engage in online disability evaluation. The phenomenon of behavior intent can



be useful in determining inclinations toward a new technology. However, in order to truly understand the relative influence of the factors of interest in physician behavior with online systems, one must ultimately focus on adoption behavior. Secondly, the sample for the current research focused on an active stratum of users of these systems (see the Method section). The next step in this analysis would be to conduct a comparative assessment of this group, vis-à-vis other strata. It would be interesting to uncover the extent to which the various factors apply up and down the spectrum. For example, for very frequent disability evaluators, it would be interesting to understand whether there is any predictive power attributable to traditional TAM factors, given the strong economic ties to the system. On the other end of the spectrum, it would be interesting to understand which factors (or combinations thereof) explain why certain physician practices do not cross the threshold to become active users of the system. In the area of work practice considerations, future research could explore further how the inherent nature of physicians’ decision-making tasks, such as disease diagnosis and treatment, could be supported with technology. As evidenced by this research, physicians who already function in a highly technologically ready work environment are more likely to use technology than those who do not. Further exploration into the effects of technologically ready work practices on physician decision making would advance knowledge in the quest for a better understanding of physician acceptance of technology in the clinical setting. Moreover, continued research in the area of TTF could explore how different levels of technology can provide greater support for various medical tasks that would be necessary to further understand physician acceptance of technology in clinical practice settings.

Implications in Practice While the principle objective of this study has been to explore and understand those factors associated with physician acceptance of online systems, the findings do suggest a number of practical considerations. First, there is a need to take a holistic approach to deploying new online systems, in contrast to an approach that would focus, for example on affecting the overall attitude of physicians about new systems. Part of this holistic approach would include a) insuring technological and organizational readiness, b)increasing staff readiness and training for implementing and using the system, and c) providing physician orientation to the benefits of information systems to their practice. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


55

An additional implication of the findings deals with the type of outreach and training that might be effective on creating online use among physicians. The findings from this research suggest, for example, that general marketing efforts to change attitudes toward online use would be ineffective at best. Rather, outreach should focus holistically on providing an inventory of the “readiness” of the candidate office to conduct online examinations. “Readiness assessment” could include a technical “work-systems” assessment as well as a physician and (key) staff orientation to the specific dimensions of the system. This is but one example of how research from medical informatics can inform the migration of practices from offline to online.

Possibilities for Advanced Practices Laboratory An underlying theme of the findings is that medical informatics systems need to be assessed within the context (e.g., work system) of their use. This theme has its roots in the foundation papers included in this volume (Davis, 1989; DeLone & McLean, 1992; Kaplan et al., 2001), and by the approaches that have driven the design of this research (e.g., TAM, TPB, TTF). However, the findings of this research and related research serve to underscore the importance of assessing the use of medical informatics systems within the precise context of their use, including how this use evolves over time. In other words, research needs to address how the technologies are situated in the organization and dynamic learning that may occur within the organization (Brown & Duguid, 1991). One method would be to establish an advanced practices partnership with medical informatics organizations. Such a “living lab” would facilitate a longitudinal context-rich approach to assessing adoption, use and organizational responses to online medical informatics systems. The objectives of this laboratory would be to establish a network of physicians to test innovative technological solutions within their work context, assess the use of technological solutions within specific medical informatics domains, facilitate the deployment of technological solutions among physicians, and provide research and training opportunities for researchers and practitioners. Consequently, this living lab would provide a nationwide network and laboratory environment for the testing, training and deployment of innovative technological solutions to improve various processes in the medical informatics field.



Conclusion Medical informatics continues to grow as an important source of productivity improvements in the medical arena. Yet, physician acceptance remains an oft cited barrier to new clinical informatics systems (Patel & Kaufman, 1998). This research has helped to establish a context-based approach to understanding factors that influence physicians’ behavioral intent, as well as ultimate behavior. Such factors include the setting in which the physician works, the type of work practice in which they are engaged, as well as the perceptions regarding the value of specific informatics systems. This research effort has shown that physicians who currently operate in technologically ready medical practices are more likely to use new information systems in their practices than physicians who rely primarily on nonautomated work systems. Multivariate data analysis of variables affecting physicians intention to use new systems assisted in understanding the value of the integrated research model, but the greater value of this research is the notion that technology adoption efforts need to attend to other factors surrounding the physician work system practices and work behavior, not just the general attitude that a physician has regarding information systems. This includes possible efforts to better equip medical settings, to introduce intermediate computer-related work systems, and to emphasize potential benefits of specific systems. Thus, the individual variables of the current technology acceptance models such as TAM, TPB, and TTF need to be expanded to include constructs that capture work system practices and work system compatibility in order to provide a more precise picture of the factors that affect a user’s intention to use a new technology.

Acknowledgments This study was conducted pursuant to a cooperative research agreement between Claremont Graduate University and the QTC Management. The authors gratefully acknowledge the support of QTC Management (especially Drs. Shahani and Kay, & Ms. Kay) in terms of providing access to the physician network used in this study as well as for financial, technical, and administrative support in the conduct of the study. The authors also appreciate the technical and statistical assistance provided by June Hilton throughout the



57

course of the project, and the assistance of Jackie Burton in the writing of an earlier draft of this chapter. Preliminary findings from this research have previously been reported at AMCIS 2003 (Tulu et al., 2003) and HICSS 2004 (Horan et al., 2004, January 5-8), and a technical paper on the path analysis and implications for TAM/TPB theory extension is also available on request (Horan, Tulu, & Hilton, 2004).

References Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice Hall. Bomba, D. (1998). A comparative study of computerized medical records usage among general practitioners in Australia and Sweden. MedInformation, 9, 55-59. Brown, J. S., & Duguid, P. (1991). Organizational learning and communitiesof-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40-57. Chau, P. Y. K., & Hu, P. J. H. (2001). Information technology acceptance by individual professionals: A model comparison approach. Decision Sciences, 32(4). Davis, F. D. (1989, September). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95. Demeter, S. L., Andersson, G. B. J., & Smith, G. N. (1996). Disability evaluation. St. Louis, MO: Mosby Year Book. Dishaw, M. T. (1999). Extending the technology acceptance model with tasktechnology fit constructs. Information & Management, 36(1), 9-22. Godin, G., & Kok, G. (1996). The theory of planned behavior: A review of its applications to health-related behaviors. American Journal of Health Promotion, 11(2), 87-98.



Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19(2). Greatbatch, F., Heath, C., Campion, P., & Luff, P. (1995). How do desktop computers affect the doctor-patient interaction? Family Practice, 12, 32-35. Horan, T. A., Tulu, B., & Hilton, B. (2004). Physician technology acceptance of online disability evaluation systems. Manuscript submitted for publication. Horan, T. A., Tulu, B., Hilton, B., & Burton, J. (2004, January 5-8). Use of online systems in clinical medical assessments: An analysis of physician acceptance of online disability evaluation systems. Paper presented at the 37th Annual Hawaii International Conference on System Sciences, Hawaii. Johnston, J. M., Leung, G. M., Wong, J. F. K., Ho, L. M., & Fielding, R. (2001). Physicians’ attitudes towards the computerization of clinical practice in Hong Kong: A population study. International Journal of Medical Informatics, 65, 41-49. Kaplan, B. (1997). Addressing organizational issues into the evaluation of medical systems. Journal of the American Medical Informatics Association, 4(2), 94-101. Kaplan, B., Brennan, P. F., Dowling, A. F., Friedman, C. P., & Peel, V. (2001). Toward an informatics research agenda. Journal of the American Medical Informatics Association, 8(3), 235-241. Kwon, S. K., & Chidambaram, L. (2000). A test of the technology acceptance model: The case of cellular telephone adoption. Paper presented at the 33rd Hawaii International Conference on Systems Sciences, Hawaii. Leung, G. M., Johnston, J. M., Ho L., Wong, F., & Cameo, S. C. (2001). Computerization of clinical practice in Hong Kong. International Journal of Medical Informatics, 62, 143-154. Lucas, H. J., Walton, E. J., & Ginzberg, M. J. (1988). Implementing packaged software. MIS Quarterly, 12(4), 537-549. Mathieson, K., Peacock, E., & Chin, W. W. (2001). Extending the technology acceptance model: The influence of perceives user resources. The DATA BASE for Advances in Information Systems, 32(3), 86-112.



59

McKinlay, J. B., Potter, D. A., & Feldman, H. A. (1996). Non-medical influences on medical decision-making. Social Science Medicine, 42(5), 769-776. Mikulich, V. J., Liu, Y. A., Steinfeldt, J., & Schriger, D. L. (2001). Implementation of clinical guidelines through an electronic medical record: Physician usage, satisfaction and assessment. International Journal of Medical Informatics, 63(3), 169-178. Mitchell, E., & Sullivan, F. (2001). A descriptive feast but an evaluative famine: Systematic review of published articles on primary care computing during 1980-1997. British Medical Journal, 322, 279-282. Nielsen, A. R. P. L. (1998). A study into levels of, and attitudes towards information technology in general practice (Vols. 1 & 2). Sydney, Australia Pare, G., & Elam, J. J. (1999). Physicians’ acceptance of clinical information systems: An empirical look at attitudes expectations and skills. International journal of Healthcare Technology and Management, 1(1), 4661. Patel, V. L., Arocha, J. F., & Kaufman, D. R. (2001). A primer on aspects of cognition for medical informatics. Journal of the American Medical Informatics Association, 8(4), 324-343. Patel, V. L., & Kaufman, D. R. (1998). Medical informatics and the science of cognition. Journal of the American Medical Informatics Association, 5, 493-502. Ridderikhoff, J., & Van Herk, B. (2000). Who is afraid of the system? Doctors’ attitude towards diagnostic systems. International Journal of Medical Informatics, 53, 91-100. Ridsdale, L., & Hudd, S. (1994). Computers in the consultation: The patient’s View. British Journal of General Practice, 44, 367-369. Ryan, G. W. (1998). What do sequential behavior patterns suggest about the medical decision-making process? Social Science Medicine, 46(2), 209-225. Schuring, R. W., & Spil, T. A. M. (2002). Assessing Relevance and microrelevance to the professional’s working process as a determinant of ITdiffusion and IT-use in healthcare. Paper presented at the International Information Resource Management Association Conference, Seattle, WA.



Spil, T. A. M., Schuring, R. W., & Katsma, C. (2002, July 15-16). Assessing resistance of professionals as a determinant of IT-diffusion and IT-use in healthcare. Paper presented at the 9 th European Conference on IT Evaluation (ECITE 2002), Paris, France. Succi, M. J., & Walter, Z. D. (1999). Theory of User Acceptance of Information Technologies: An Examination of Healthcare Professionals. Paper presented at the 32nd Hawaii International Conference on System Sciences, Hawaii. Thakurdas, P., Coster, G., Curr, E., & Arroll, B. (1996). New Zealand general practice computerization: Attitudes and reported behavior. New Zealand Medical Journal, 106, 419-422. Tulu, B., Hilton, B., & Horan, T. (2003). Physicians Acceptance of Webbased Medical Assessment Systems: Findings From a National Survey. Paper presented at the Americas Conference on Information Systems, Tampa, FL. Westberg, E. E., & Miller, R. A. (1999). The basis for using the internet to support information needs of primary care. Journal of the American Medical informatics Association, 6(1), 6-25. Zigurs, I., Buckland, B. K., Connolly, J. R., & Wilson, E. V. (1999). A test of task-technology fit theory for group support systems, database for advances in information systems. Summer, 30(3/4), 34-51.


Computer and Stress in Social and Healthcare Industries

61

Chapter IV

Computer and Stress in Social and Healthcare Industries Reima Suomi Turku School of Economics and Business Administration, Turku Reetta Raitoharju Turku School of Economics and Business Administration, Turku

Abstract Social and healthcare industries offer demanding occupations, as they are very human-contact intensive workplaces and, moreover, the customers are usually met in critical and not-wished-for situations. Possible actions are many, and seldom are there clear procedures on how to continue: Each customer contact is a place for genuine decisions. Add to this deliberate service situation a computer, and you can count on difficulties. Our focus is on how information systems affect the stress levels of health and social-care workers. Our empirical study shows–among many other factors–strong correlation between the use of computers and stress levels in the healthcare professions: The more computer use, the more stress. We


62 Suomi & Raitoharju

discuss what could be done to manage stress levels in relationship to computer use in health and social-care industries. In conclusion, we wrap the research findings together and propose our extensions to the current knowledge on the relationship between stress and information systems in health care. Our most important finding is that when users understand the total collaborative work setting, computer work obtains meaning, and stress levels reduce.

Introduction Knowledge or computer work and healthcare work are professional areas that spread out in modern society. Unfortunately, workers in both sectors are prone to suffer from stress. As we combine healthcare work and computers in the same environment, the situation becomes even more problematic. That is why we want to study the way in which computers cause stress in healthcare professions. Some of the factors that increase stress levels in both industries are summarized in Table 1. Research on stress and related phenomena has deep roots both in computer and information research as well as in healthcare research. In the field of computing, dominant topics seem to be those of information overload (Edmunds & Morris, 2000; Laskin, 1994; Levinson, 1970; Maes, 1994), coping with constant change (Thong & Yap, 2000; van Rooyen, 2000; Wastell & Newman, 1996) and problems of working in groups and virtual organizations (Anony-

Table 1. Work stressors in healthcare and computer industries Healthcare Intensive human contact Customer interaction, often in a crisis situation Decision-making pressure under insufficient information Area’s total professional knowledge growing fast, information overload Computer industry Work often performed by virtual teams Area’s total professional knowledge growing fast, information overload Constant change of the discipline Work with abstract entities Need for networking



63

mous, 1996; Armour, 1995; Kokko, Vartiainen, et al., 2004; Schill, Toves, et al., 1980; Trent, Smith, et al., 1994). Our research questions include the following: 1. 2. 3.

How do computers affect workers’ stress levels in health and social care industries,? What kinds of effects does information technology have on work? What can be done to manage stress levels in the healthcare industries?

Methodologically, our research was very classical. First, we ran through a literature study on stress in general and specifically in the healthcare and computer industries. Second, 5,000 Finnish healthcare professionals were given a comprehensive survey on their working conditions; the relationship between information technology (IT) and work was one of the topics discussed. This survey was originally administered by Statistics Finland on the order of the Finnish Institute for Occupational Health. The chapter unfolds as follows: In the first section, we define the terms used (among others, stress), and briefly discuss why stress is very much present in health and social-care industries. In next section, we review the current literature on the relationship between stress, computers, and information systems. As stress is a usual phenomenon, in the following section we review how to manage stress levels in organizational work settings. Section 4 reviews results from a survey that got responses from 3,072 social and healthcare professionals in Finland. Finally, in the next section we discuss what could be done to reduce computer-based stress in the social- and healthcare industries. In the final section we discuss conclusions.

Computers as a Source for Stress Defining Stress Stress is a meaningful concept in many disciplines and has hundreds of definitions. For example, in physics stress is defined as “the force per unit



area resulting from the application of a load” (SGIA, 2004). In fonetics and music, stress is defined as follows: “the relative prominence of a syllable or musical note (especially with regard to stress or pitch)... he put the stress on the wrong syllable” (WordNet, 2004). Stress can also be seen as just a biological phenomenon: “Stress is the sum of the biological reactions to any adverse stimulus, physical, mental or emotional, internal or external, that tends to disturb a person’s normal state of well-being” (MedSearch, 2004). A very general and widely usable definition is “mental or physical tension that results from physical, emotional, or chemical causes” (ViaHealth, 2004). For our purposes in this chapter, the definition for stress describing “difficulty that causes worry or emotional tension” (WordNet, 2004) is very suitable. We can see that even though the basic prequisities differ, the definitions are related. Scanning through the definitions, we can see that stress is a neutral term: It is per se neither bad nor good; everything depends on the context. Especially, we want to stress that stress is not solely a negative phenomenon but can have positive effects, even in the field of psychology. As the World Federation for Mental Health (WFMH) has defined, “Stress is not necessarily negative. Some stress keeps us motivated and alert, while too little stress can create problems. However, too much stress can trigger problems with mental and physical health, particularly over a prolonged period of time” (WFMH, 2004). The National Institute for Occupational Safety (NIOSH) defined positive stress as challenge: The concept of job stress is often confused with challenge, but these concepts are not the same. Challenge energizes us psychologically and physically, and it motivates us to learn new skills and master our jobs. When a challenge is met, we feel relaxed and satisfied. Thus, challenge is an important ingredient for healthy and productive work. The importance of challenge in our work lives is probably what people are referring to when they say “a little bit of stress is good for you.” (NIOSH, 2004) In biology, stress is often connected to the following phenomena: Stress is defined as a nonspecific response of the body to any demand made upon it which results in symptoms such as rise in the



65

blood pressure, release of hormones, quickness of breath, tightening of muscles, perspiration, and increased cardiac activity. (WFMH, 2004) Similarly, we often define that computer systems can be stressed. For example, a computer, a database management system, or a network connection can be stressed. Similarly, as in the case of the human body, we want to tell that the systems are overloaded by an external force. Biologically, job stress can lead to loss of health or even to death. Typical biological disorders based on job stress follow (NIOSH, 2004): •

•

•

•

•

Cardiovascular Disease: Many studies suggest that psychologically demanding jobs that allow employees little control over the work process increase the risk of cardiovascular disease. Musculoskeletal Disorders: It is widely believed that job stress increases the risk for development of back and upper extremity musculoskeletal disorders. Psychological Disorders: Several studies suggest that differences in rates of mental health problems (such as depression and burnout) for various occupations are due partly to differences in job stress levels. (Economic and lifestyle differences between occupations may also contribute to some of these problems.) Workplace Injury: Although more study is needed, there is a growing concern that stressful working conditions interfere with safe work practices and set the stage for injuries at work. Suicide, Cancer, Ulcers, and Impaired Immune Function: Some studies suggest a relationship between stressful working conditions and these health problems.

There is too a specific literature on job or occupational stress. WFMH (2004) defined job stress and discussed its main outcomes as follows: Job stress can be defined as the harmful physical and emotional response that occurs when the requirements of the job do not match the capabilities, resources, or needs of the worker. Job



stress can lead to poor health and even injury. Long-term exposure to job stress has been linked to an increased risk of musculoskeletal disorders, depression, and job burnout, and may contribute to a range of debilitating diseases, ranging from cardiovascular disease to cancer. Stressful working conditions also may interfere with an employee’s ability to work safely, contributing to work injuries and illnesses. In the workplace of the 1990s, the most highly ranked and frequently reported organizational stressors are potential job loss, technological advances, and ineffective top management. At the work unit level, work overload, poor supervision, and inadequate training are the top-ranking stressors.

Computers and Stress Thong and Yap (2000) defined what a theoretical framework on occupational stress should include (their framework is depicted in Figure 1). We structure our discussion on computers or information systems as related to stress, according to their framework. When Thong and Yap (2000) talk about the importance of focus on work environment, they mean that occupational stress should always be understood and studied in a defined environment. Occupational stress, for Figure 1. Key points for a theoretical framework on computers and stress (adapted from Thong & Yap, 2000) Focus on work environment

Consider effects of individual differences

Consider effects of social support

Focus on IS profession specifically

Consider both intra- and extra-organizational factors

Consider cognitive appraisal process

IS PROFESSION OCCUPATIONAL STRESS

Consider effects of coping strategies

Consider both individual and organizational stress outcomes

Identify relevant variables in each category of the stress process



67

example, in the computer industry, is different from occupational stress in the healthcare industry. Focus on IS profession specifically means that a occupational health model for the IS profession should be specific to that profession. Similarly, we can infer that an occupational health model for health professionals should be specific to them. Intra- and extra-organizational factors refer to the stress-causing factors, stressors. They are born both within and outside organizations. Intraorganizational factors can be affected by the organization’s management, but extra-organizational factors are beyond their control. We discuss four major stressors supported by computerization, namely those of: 1. 2. 3. 4.

information overload, the need to maintain different virtual organizations and social networks, constant change, and the built-in conflict in formal organizations and in virtual organizations.

A major stressor in work related to information systems, as already defined in Table 1, is that of information overload. Information overload is a major burden for information workers (Edmunds & Morris, 2000). Actually, knowledge workers tend to collect more data and information than what is actually needed. Butcher (1998) cited some reasons why managers and information workers tend to collect and use more information than what would actually be needed: • • • • • •

They collect information to indicate a commitment to rationalism and competence that they believe improves decision making. They receive enormous amounts of unsolicited information. They seek more information to check out the information already acquired. They need to be able to demonstrate justification of decisions. They collect information just in case it may be useful. They play safe and get all information possible.



•

They like to use information as a currency and not get left behind their colleagues.

Information overload can be limited in several ways. One revolution happened when reading habits shifted from intensive reading to extensive reading: “One historian has claimed that a ‘reading revolution’ took place in the later eighteenth century, and the sense of a shift from intensive to extensive reading” (Burke, 2000). In a similar way, we can say that a lot of work in organizations has turned from “intensive” to “extensive.” Maintaining social networks is a hard task needing a lot of effort. Especially, failures in establishing a social network can result in increased stress. (Cross, Nohria, et al., 2002) identify following myths as it comes to the building and maintaining of networks. • • • • • •

The more communication the better. Everyone should be connected to everyone. We can not do much to aid informal networks. How people fit into networks is a matter of personality (which can not be changed). Central people who have become bottlenecks should make themselves more accessible. I already know what is going on in my network.

Change means that leaving something that is safe and known also causes stress. To the computer profession is tied the concept of seeking for the new. For example, the business process redesign metaphor supports this kind of thinking. The same is true for the search for competitive advantage. Levinson (1970) nicely described the connection between stress and seeking for new innovations, in this example of scientists: Unlike most other people, who much prefer to accept what they find around them and who therefore are uneasy with people who question the status quo, the scientist built up a tolerance for the anxiety which comes from seeking out the new and unfamiliar. Once a man takes on a scientific career, he has, in essence, identified himself with a group whose whole



69

rationale is tied up with intellectual rebellion, embodied in the rejection of old knowledge and a courageous search for the new. (Levinson, 1970) Working in formal organizations always builds in conflict and stress. Pondy (1967) defined the following sources of conflict in formal organizations: •

Bargaining conflict occurs among parties who have an interest in maintaining and encouraging a shared relationship.

•

Bureaucratic conflict occurs between two or more parties where some form of power relationship (superior/subordinate) exists Sytematic conflict occurs among parties as a part of lateral or working relationships

•

This source of stress might be a reason why less formal virtual organizations gain ground, even though they surely have their sources of stress. Recent studies have shown that even virtual organizations can be stressful (Kokko et al., 2004; Kurland & Egan, 1999; Markus, Manville, et al., 2000; Suomi 2000; Walsham, 1994). Virtual organizations have the following characteristics that easily cause stress for workers (Gristock, 1997): • • • • • •

Mediated interaction Geographical dispersion Mobility Diversity of actors Asynchronous work time Temporary structure

Cognitive appraisal processes refer to the ways on how individuals develop their understanding and awareness of their and their co-workers’ stress. As with other diseases, this is connected to the phenomenon of health deficit. Individuals live through three phases when struggling with their health problems: In the first phase, the disease is not identified. In the second phase, the disease is diagnosed but not taken care of. In the third and last phase, the diseases is diagnosed and taken care of as well as possible (Bonaccorso & Sturchio, 2002). Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Individual differences materialize when identifying stress, reacting to work stressors, and coping with them. Most likely in any profession a part of the professional development is the capacity to handle stress. As professionals develop, they most likely develop better skills to manage stress. Dreyfus (1992) defined the following categories of professionalism: • • • • • • • • •

Blind person Jerk Novice Advanced beginner Professional Proficient professional Expert Master Legend

There is also some discussion hinting that women and men might feel stress in a different way (Meyerson, 1998). Social support can help in handling stress. Social support can come from supervisors, colleagues, friends, relatives, and spouses (Thong & Yap, 2000). Unfortunately, work with information systems, especially if innovation is looked for, necessitates some conflict, as is documented in recent literature. Working in groups connected to the search for innovation fosters built-in conflict and stress situations: There is a diminished benefit for working together in teams if this does not create some level of conflict. Group work provides one means of bringing multiple perspectives to be a on a common problem. A lack of conflict among team members is often called “group-think” and the literature provides several examples of poor decisions from groups with too little conflict. (Sawyer, 2001) Robey (1984) visualized the factors that lead to conflict and stress in computer-focused group work, as in Figure 2.



71

Figure 2. Conflict factors in teamwork (Adopted from Robey, 1984) Characteristics of team members

Existing team conflict

Software development team performance

Conflict management

Characteristics of the team

Figure 3. The components of good management for IT-professionals (Adopted by Agarwal & Ferratt, 2002)

Recruiting Posture • Sourcing • Skills Sought • Competitive Differentation Elements • One-time Inducements

Career Development and Security • Longer Term Career Development • Organizational Stability and Employment Security

Concern for Individual • Opportunities for Advancement • Opportunities for Recognition • Quality of Leadership • Sense of Community • Lifestyle Accommodations

Compensation • Compensation and Benefits

Concern for Productivity • Performance Measurement • Work Arrangements • Employability Training and Development

Social support from the colleagues is important, but a lot can be done by the formal management. Good human resource management practices play a key role (Holm, Lähteenmäki, et al., 2002). Agarwal and Ferratt (2002) define good management of IT-professionals as having five components: • • •

Recruiting Posture Career Development and Security Compensation



• •

Concern for Individual Concern for Productivity

Their star-model of good management of IT-professionals is summarized in Figure 3. Coping strategies refer to the ways in which individuals manage stress. We would like to take a closer study of two major universal coping strategies: 1. 2.

avoidance of role conflicts building of trust in the workplace

In formal organizations, individuals suffer from stress if they suffer from imbalance between their behavior and their expected roles. According to Kallinikos (2004), bureaucratic order, also the basis of modern organizations, rests on the following principles: •

•

•

Selectivity: Individuals taking organizational roles are expected to suspend nonrole demands and act on the basis of a well-specified and delimited set that constitutes the role. Mobility: A role, being and abstract set of functional requirements, can be unleashed from the particular circumstances into which it is embedded, and be transferred across various organizational contexts. Reversibility: Jobs can be altered or redesigned and the organizational sanctioning of job positions modified or even withdrawn.

Especially, selectivity is a root cause for stress. Individuals are not allowed to act naturally, but they are expected to adopt unnatural roles, where they have to suspend their normal personal characteristics. Working in workplace where you can trust others and where you feel trusted is a major benefit that reduces stress. Lack of mutual trust is a major stress factor in organizations. Luo and Najdawi (2004) identified five measures through which trust can be built:



1. 2. 3. 4. 5.

73

Calculative process: A trustor develops trust based on a calculation of the costs and rewards a trustee to cheat or cooperate in a relationship. Prediction process: A trustor develops trust by predicting a trustee’s future actions based on his or her past behavior. Intentionality process: A trustor develops trust based on his or her prediction of the intentions of the trustee. Capability process: A trustor develops trust based on an evaluation of the trustee’s ability to perform his promises. Transference process: A trustor develops trust based on transferring trust from a known entity to an unknown entity.

The characteristics an individual can develop in order to maintain trust are the following (Luo & Najdawi, 2004): • • •

Ability refers to whether and individual has a set of skills and competencies that would enable him or her to perform the promises. Benevolence is the extent to which an individual is perceived to have good intention toward others without profit motive. Integrity refers to whether an individual adheres to a set of principles that are acceptable to those who may trust him.

Bashein and Markus (1997) deliver a list of characteristics of a person that instill trustworthniness: • • • •

Similarity and likability Prolonged interaction Appropriate behavior Consistent behavior

Individual and organizational stress outcomes vary a lot. Usually, the focus on stress studies is on individuals, but the organizational dimension should not be forgotten. An interesting stream of literature is that compares organizational malfunctions with those of traditional diseases of humans. We have identified four entities in this discussion: Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


• • • •

Paranoia Autism Attention Deficit Disorder Web Addiction

Paranoia is a psychological disorder characterized by delusions of persecution or grandeur (WordNet, 2004). Both organizations and humans can manifest paranoia. Uneasiness and stress in a workplace can have its roots in suspicion about the behavior of others. Again, however, a sensible amount of suspicion can be a good characteristics of a human. Kramer (2002) calls this type of suspicion prudent paranoia: Prudent paranoia is a form of constructive suspicion regarding the intentions and actions of people and organizations. An interesting theory is that the modern information society has characteristics of autism, a mental health disorder connected to behavioral difficulties. According to Beruch (2001), an autistic society and a person with autism both have the following characteristics: • • • • • • •

Profound lack of affective emotional contact Intense insistence on sameness Bizarre and elaborate repetitive routines Being mute or having a marked abnormality of speech Fascination with and dexterity in manipulating objective, high levels of visuo-spacial skills or rote memory Learning difficulties in other areas Attractive, alert, and intellectual appearance

Attention deficit/hyperactivity disorder (ADD/ADHD) is a disease in which individuals find it difficult to concentrate. It is defined as follows: Attention Deficit Hyperactivity Disorder (ADHD) is a condition that becomes apparent in some children in the preschool and early school years. It is hard for these children to control their behavior and/or pay attention. It is estimated that between 3 and 5 percent of children have ADHD, or approximately 2 million children in the



75

United States. This means that in a classroom of 25 to 30 children, it is likely that at least one will have ADHD. (NIMH, 2004) The importance of attention was brought to the knowledge of the IS community by Davenport, in his classic article (Davenport & Beck, 2000). The message was that modern society is rich in information; the scarce resource is human attention. So we can speak of an attention economy. Again, both individuals and organizations can have difficulties in concentrating effort and attention on important things. Computers can cause stress, even in leisure time. One difficult condition is that of Web addiction. Young (1996) defined the symptoms of Web addiction as follows: • • • • • •

Staying online for 38 hours a week or more in leisure time Lying to family or colleagues about time spent on the Internet Restlessness Irritability and anxiety when not engaged in computer activities Neglect of social obligations Consistent failure to quit computer activities

Finally, Thong and Yap (2000) underscored that in order to perform quantitative research on stress, there is a need of identifying relevant variables. Without distinct and clear variables, testable and predictable hypotheses can not be built and tested.

Eliminating Stress: Usual Tools Because stress is mostly conceived as negative, action should be taken to eliminate it. As stress is cause by a mismatch between the individual and his or her environment, activities on both sides can be beneficial. As discussed earlier, humans feel stress in very different ways, and their stress management capabilities greatly vary. Here we concentrate more on activities organizations and working environments can take to reduce risks.



According to Price et al. (1988), three types of health prevention can be identified: primary, secondary, and tertiary. 1.

2.

3.

Primary prevention aims to eliminate causal factors in the development of problems. Theoretically, opportunities to improve psychological well-being could include reducing workloads or increasing employees‘ control over their work. Secondary prevention aims to reduce the severity or duration of disorders and thus avoid the development of more serious, chronic, or disabling conditions. Occupational stress interventions of this form could include stress management training programs, where the employees can be taught how to handle stress in their work environment. Tertiary prevention deals directly with existing disorders or problems. The aim of tertiary activities may be either to cure or restrict the disorder to the extent whereby the employee is restricted by the disability. This can include services such as counselling and psychotherapy.

Also, Cooper and Cartwright (1997) use a three class classification: 1.

2.

3.

Stressor reduction: Taking action to modify or eliminate sources of stress at work and thus reduce the negative impact on the individual. The focus is in adapting the environment to fit the individual. Stress management: Prompt detection and management of stress by increasing awareness and improving management skills of the individual through training and education. This prevention can help individuals to recognize the symptoms of the stress and improve relaxation techniques and work modification skills. Employee assistance programs: Concerned with the treatment, rehabilitation, and recovery process to individuals who are suffering from ill health as a result of stress. This level activities typically involve the provision of counseling services.



77

Stressor Reduction Prevention tools to eliminate stress consist of both organizational changes and individual targeted operations. Sauter, Murphy, et al. (1990) defined the following general guidelines for preventing negative stress from occurring: 1. 2. 3. 4. 5. 6. 7.

Ensure that the workload is in line with workers’ capabilities and resources. Design jobs to provide meaning, stimulation, and opportunities for workers to use their skills. Clearly define workers’ roles and responsibilities. Give workers opportunities to participate in decisions and actions affecting their jobs. Improve communications, reduce uncertainty about career development and future employment prospects. Provide opportunities for social interaction among workers. Establish work schedules that are compatible with demands and responsibilities outside the job.

Elkin and Rosch (1990) stated a slightly different list concerning the same subject: 1. 2. 3. 4. 5. 6. 7. 8.

Redesign the task. Redesign the work environment. Establish flexible work schedules. Encourage participative management. Include the employee in career development. Analyze work roles and establish goals. Provide social support and feedback. Build cohesive teams.



9. Establish fair employment policies. 10. Share the reward. Tsutsumi and Kawakami (2005) presented effort–reward-based guidelines to prevent occupational stress: Extrinsic Effort • •

Even distribution of workload Reduction of long overtime work; secure rests or holidays

Extrinsic Rewards •

•

•

•

Monetary and nonmonetary compensatory reward • Encouraging praise for good work • Introduction of additional reward system such as welfare facilities and retirement benefits Esteem reward • Skill up of supervisors–managers for interpersonal relationship and social skill Career opportunity • Clear distinctions of the stages of promotion • Appropriate training for career development Other organizational system relevant to rewards (target dimensions) • Better information (sense of fairness) • Assessment of an employee’s performance with her or his consent (sense of fairness) • Developing social support at or beyond the workplace (esteem and buffering of effort–reward imbalance) • Mentoring system (esteem and career opportunity)



79

Primary intervention strategies to reduce stress demand often cultural change. To find out the crucial stressors an organization should define them by some prior diagnosis (Cooper & Cartwright, 1997). This can be done by using risk assessment to identify the organizational stressors responsible for employee stress.

Stress Management Stress management training aiming to educate employees about the impact of stress and teaching skills to cope with work demands is one of the most evaluated training programs (Reynolds, 1997). Psychological well-being appears to depend on a vast number of related factors, some of which work or employment cannot have any impact on, such as temperament and early experiences (Reynolds, 1997). On a study of job stress management and its effects on stress symptoms (Feuerstein et al. 2004) found significant decreases in pain, symptoms, and functional limitations. The role of these secondary preventive actions is essentially damage limitation addressing the consequences rather than the sources of stress (Cooper & Cartwright, 1997). These kind of actions implicitly assume that the organization will not change but continues to be stressful, and therefore the individual has to develop resistance and coping skills with stress.

Employee Assistance Program Evidence has been found that counseling and other employee assistance programs are effective in improving the psychological well-being of employees (Cooper & Cartwright, 1994). These kinds of actions have also been suggested to have considerable cost benefits. For example, a counseling program in the U.K. Post Office resulted in a reduction in absenteeism in one year by approximately 60% (Cooper & Sadri, 1991). In this study, some significant improvements in mental health and self-esteem were found. According to the findings of Cooper and Cartwright (1997), the reasons that stress management and employee assistance programs are more popular in organizations than stress prevention actions can be listed as follows:



• • • •

There is relatively more published data available on the cost-benefit analysis of such programs The counselors, physicians, and clinicians responsible for healthcare feel more comfortable with changing individuals than with organizations It is considered easier and less disruptive for a business to change the individual The organization looks like it is taking an action to safeguard employee’s health

However, reducing stressors before stress even occurs can not only be more cost effective than curing actions but also reduce absenteeism and improve organization performance, not to mention the human benefits.

Empirical Findings In order to empirically examine the influence of IT to stress, a data analysis was made. The purpose was to test if the use of IT had some kind of relation to different factors causing stress in the healthcare environment.

Table 2. Sample demographics Ge n d e r Male Female Age -25 26-35 36-45 46-55 55W o r k ex p er -5 6-15 16-25 26-

% 10 90 2 35 36 17 10 ien ce 11 30 34 19



81

Figure 4a. Use of IT Amount of IT use at work

31 % 37 %

Not at all Seldom Sometimes A lot

20 % 22 %

Figure 4b. IT skills IT skills evaluations

19 %

39 % Insufficient Reasonable Good

42 %

Sample Questionnaires were mailed to 5,000 social and healthcare employees in Finland. A total 3,072 returned questionnaires were accepted. The response rate was 72.2%. The sample did not reach new or temporary social- or healthcare employees because the sample was taken from a register that was two years old. A summary of the demographic characteristics of the sample is



presented in Table 2. The data were collected by the Finnish Institute of Occupational Health. Thirty-one percent of respondents did not use IT at their work at all. 20% used just a little IT, 22% used IT sometimes, and 27% used a lot. 39% of respondents considered their IT skills to be insufficient, 42% thought they had reasonable IT skills, and 19% answered they had good IT skills.

Table 3. Results of the study (+) = the direction of correlation is positive; (-) = the direction of correlation is negative; X= no statistically significant correlation.) Estimate how How much Personal Way the Estimate the much use of IT increase of IT IT used at skills to use use of IT is will increase in influence at work work? IT experienced the future Amount of extra 0, 203 (+) work without compensation Lack of time to accomplish the 0, 143 (+) work tasks properly Continuous stress and 0, 184 (+) pressure of unfinished work Several interruptions 0, 263 (+) and disturbances at work Ability to manage current 0, 174 (+) work tasks Possibilities to influence the 0, 158 (-) planning of changes at the workplace Other education during the last 0, 250 (+) 12 months

0, 176 (+)

0, 151 (-)

0, 179 (+)

0, 155 (-)

X

0, 138 (+)

0, 224 (+)

0, 177 (+)

X

0, 147 (-)

0, 217(+)

0, 155 (+)

0, 170 (+)

0, 159 (-)

0, 258 (+)

0, 153 (+)

0, 208 (+)

0, 199 (-)

0, 153(-)

0, 137(-)

0, 148 (-)

0, 160 (+)

0, 160 (-)

0, 210 (+)

0, 212(+)

0, 211(-)

0, 236(+)

0, 141 (-)



83

Analysis of Data The data were analyzed using the SPSS program, Version 5.0. Cross tabulation was used in order to find relations between different factors. Significance rate was set to 0, 05. For those job satisfaction factors that had statistically significant IT usage, the contingence coefficient was calculated in order to find out the intensity of correlation. For ordinal scaled factors, gamma-value was tested to find out the direction of correlation. The result showed correlation between the use of IT and different jobsatisfaction factors. In Table 3 are the contingence coefficients and the direction of the correlation.

Results A relation between stress and the use of information technology was found. There was statistical significance between enough time to do work tasks properly and the use of IT. Correlation was positive, which means that the more the respondents used IT in their work the more they felt they did not have enough time to carry out their work tasks properly. There was also a correlation with the amount of use and how much interruption the respondents had in their work. This correlation was also positive: The more the respondents used IT in their work the more interruptions they felt they had. Also, the expectations of increasing the use of IT in work correlated with the amount of interruptions. Respondents that had most interruption in their work were the ones that most expected the use of IT to increase in their work. Possibilities to have influence over work tasks had a correlation with the extent of use and the expectations of IT. Employees that were using more IT thought they had fewer possibilities to influence their work. Also, the employees that expected the IT to increase more were those who thought they had less power over their work. This can result from the routines IT brings to work. The ways the tasks are made have to be standardized, and therefore employees cannot work as freely in their own way as they used to. Employees that used more IT in their work seemed to make more work without compensation. Also, the more skilled the employees were in using IT, the more they answered they work extra hours without compensation.



Table 4. Impacts of IT The more IT usage, the more 1. extra work without compensation 2. pressure of unfinished work 3. disturbance and interruptions at work 4. capability to manage current tasks 5. possibilities to influence work

Even though the purpose of IT is to improve work processes and therefore decrease work stress, it seems that it may not be the reality in the social and healthcare areas. According to the results, the most stressed workers were those using the most IT in their work. This can be due to lack of IT skills; not knowing how to use IT properly can cause additional work. Another explanation could be that the tasks requiring the use of IT have been given to those people already most stressed. It can also be that there is more stress in the sectors that use IT the most. Education seems to have a positive correlation with the use of IT and IT skills. Employees with the most education in work managed IT better in work. Respondents that used more IT in their work think they have more possibilities to participate in changes in their work. The ability to manage current work tasks correlated with the use of IT and IT skills. Employees that used more IT thought they could manage well with their current work tasks or even more challenging tasks in the future. Some factors that had a correlation with IT use are summarized in Table 4. The results indicate that IT use in health and social-care environment can have more of an impact on work than purely positive or negative. IT use can cause stress by disturbing other work tasks by taking extra time. Also, disturbance and interruptions at work relate to IT use, which can indicate that IT skills of personnel are not sufficient or that the applications used are not sophisticated enough. Also, poor IT skills of colleagues can lead to interruptions if assistance is often needed to solve other’s computer-related problems. However, the effects of IT need not only be negative. Capability to manage current tasks and possibilities to influence one’s work relate to higher rates of IT use.



85

What Can Be Done in the Health and Social-Care Industries to Eliminate IT-Based Stress? As factors affect stress in the healthcare sector, these factors are also used to discuss the ways of eliminating direct negative effects of IT in health and socialcare industries. Most approaches to stress reduction in the workplace focus on individuals (van der Hek & Plopm, 1997) still more permanent and efficient effects are anticipated from organization focused interventions (Karasek, 1997). Using parts of the guidelines presented previously (Sauter, Murphy, et al., 1990), we define the following general guidelines for preventing negative IT stress from occurring in healthcare. •

•

•

Ensure that the workload is in line with workers’ capabilities and resources: It is highly important to take into consideration that learning to use new IT demands time and effort. Employees should also have the possibility to be trained properly in order to be able to get the capabilities to use the new technology. The workload should be adjusted according to estimations of the effort needed to use the new IT. This is especially important in a hectic healthcare environment, where the employees already are under a large workload. Clearly define workers’ roles and responsibilities: It is important to clarify the importance of IT for the whole organization. Proper information should be filed in the systems from the beginning in order to avoid mistakes and misinterpretations at a later stage. Healthcare tasks such as filling in patient records are often delegated from doctors to secretaries. This can lead to mistakes and to the disappearance of important information. Give workers opportunities to participate in decisions and actions affecting their jobs: When designing a new information system, the role of the end users in health and social care should be more diverse. Participation in designing phase can make the launching and use easier and at the same time motivate the users. Although employees in the healthcare are not experts in IT they are experts in their own field. This knowledge should be moved into the IT as well in order to create better systems and to get users involved with IT.



•

Provide opportunities for social interaction among workers: IT use should not alienate colleagues from each other. Even if more IT is used for communication and distance working, personal meetings are vital in the health and social-care environment. Also, effort reward-based guidelines (Tsutsumi & Kawakami, 2004) can be used to find way to reduce IT-based stress.

Extrinsic Effort Use of IT should be distributed evenly. No group of professionals should allocate the use of IT to other groups. Information should be placed in the information systems at the point it is born.

Extrinsic Rewards •

•

Monetary and nonmonetary compensatory reward • Encouraging praise for good work could also include rewards for training to use new IT. Some monetary reward system concerning the use of IT could be presented to motivate the users. Esteem reward • The managers should be trained to value and encourage the employees to use IT.

Table 5. Eliminating stress in healthcare Problem:

Cure:

Extra work without compensation

IT training Reward system Adjustment of workload IT training IT support Even distribution of IT tasks IT training IT support Participative design

Pressure of unfinished work Disturbance and interruptions at work



•

87

Other organizational system relevant to rewards • Better information about new IT projects and plans should be distributed. • Mentoring system for using IT for new employees.

The most important indirect effects of IT use in the healthcare sector that we found in our empirical study were listed in Table 2. Eliminating these indirect effects is both an organizational- and individual-level task. To reduce stress caused by these factors, some guidelines of usual stress elimination tools were used to create Table 5.

Guidelines to Prevention of IT-Based Stress in the Healthcare Environment As a summary, we use the classification of Price. Health prevention among healthcare workers in computer-use related stress could go as follows: Primary Prevention • • • •

Cooperation on designing and launching new IT Training to use IT; not only one application Showing the whole chain of IT use, not only one part, without understanding where to go Sufficient IT support

Secondary and Third Prevention • •

Job stress-management programs should include a discussion about IT. Discussion and training can be helpful in this situation. Meetings can change negative attitudes by showing the whole chain of information.



Conclusions Occupational health is increasingly more threatened by mental problems than by physical accidents. Inability to work is often a result of mental problems, and a factor of mental problems is stress. Stress is born out of haste, overburden with work, and missing or insufficient resources to handle the workload. Fragmentation of working time is one important cause of stress. We conducted a literature study about the connections between IT use and stress at work. The reasons computers and applications cause stress were discussed, and a list of approaches that can alleviate stress in computer work was presented. However, we did not conclude that stress is bad: A managed level of stress can increase productivity and in total boost work performance at the individual and organizational levels. Stress can be manifested in several ways: It can be healthy and productivity-enhancing, it can be harmful, or it can develop into a disease or increase the development of other diseases. We found that stress is a very meaningful and usual concept in almost any scientific discipline. At a very general level, it means external pressure towards an entity. One can clearly see that human stress can be included into this definition. Occupational stress is one subcategory of stress. Stressors from work and other factors have an complex interplay, and the total load on the individual as related to his or her stress carrying capacity is the important issue. We used the framework of Thong and Yap (2000) to discuss the relationship between computers and stress. It can be interpreted that the connection between computers and stress is multilevel. Computers and information systems as such cause stress. More important, however, is that the governance structures, work processes, and working habits they enable can be more stressful than earlier, noncomputerized processes. On the other hand, computers could be implemented in a way that reduces risk. We would like to conclude that computers at the very least are risk-neutral devices: Stress outcomes materialize because of their unskilled implementation and wrong working habits developed by individuals and organizations. An interesting genre of research is that in which problems and malfunctions in organizations are compared to known human diseases. The cases of paranoia, autism, ADHD, and Web addiction are interesting examples of this type of discussion. Three types of health prevention can be identified: primary, secondary, and tertiary. The earlier that stress is identified and treated, the easier and Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


89

cheaper the task is. This is very similar to the tracking of mistakes and malfunctions in information systems: The earlier they can be tracked down and eliminated in the system life cycle, the better. Our empirical study found relationships between computer usage and the following issues in the healthcare industry: 1. 2.

extra work without compensation pressure of unfinished work

3. 4. 5.

disturbance and interruptions at work capability to manage current tasks possibilities to influence work

Even here, we see that computer usage had positive effects (points 4 and 5). One of our conclusions is that computer application is not always meaningful for healthcare staff, as they do not understand the total work process they are a part of and to which the computer system should give support. Educating them and giving a stronger motivation to use computers would be one thing eliminating stress in healthcare professions.

References Agarwal, R., & Ferratt, T. W. (2002). Enduring practices for managing IT professionals. Communications of the ACM, 45(9), 73-79. Anonymous. (1996). Aluminum labor pact stresses teamwork. Purchasing, 120(11), 48B22. Armour, N. L. (1995). The beginning of stress reduction: Creating a code of conduct for how team members treat each other. Public Personnel Management, 24(2), 127-132. Bashein, B. J., & Markus, M. L. (1997). A credibility equation for IT specialists. Sloan Management Review, 38(4), 35-44. Beruch, Y. (2001). The autistic society. Information & Management, 38, 129-136.



Bonaccorso, S. N., & Sturchio, J. L. (2002, April). Direct to consumer advertising in medicalising normal human experience. British Medical Journal, 324, 910-911. Burke, P. (2000). A social history of knowledge: From Gutenberg to Diderot. Polity Press. Butcher, H. (1998). Meeting managers’ information needs. London: Aslib. Cross, R., N., Nohria, et al. (2002, Spring). Six myths about informal networks and how to overcome them. Sloan Management Review, 67-75. Davenport, T., & Beck, J. C. (2000, September-October). Getting the attention you need. Harvard Business Review, 119-126. Dreyfus, H. L. (1992). What machines still can’t do. Cambridge, MA: MIT Press. Edmunds, A., & Morris, A. (2000). The problem of information overload in business organizations: A review of literature. International Journal of Information Management, 20(1), 17-28. Gristock, J. (1997). Communications and organizational virtuality. Electronic Journal of Organizational Virtualness. Holm, J., Lähteenmäki, S., et al. (2002). Best practices of ICT workforce management: A comparatible research initiative in Finland. The Journal of European Industrial Training, 26(7), 333-341. Kallinikos, J. (2004). The social foundations of the bureaucratic order. Organization, 11(1), 13-36. Kokko, N., Vartiainen, M., et al. (2004). Work stressors in virtual organizations. International Congress New Work 2004. Heraklios, Greece. Kramer, R. M. (2002, July). When paranoia makes sense. Harvard Business Review, 62-69. Kurland, N. B., & Egan, T. D. (1999). Telecommuting: Justice and control in the virtual organization. Organization Science, 10(4), 500-513. Laskin, D. (1994). Dealing with information overload. Journal of Oral Maxillofacial Surgery, 47(7), 661. Levinson, H. (1970). Executive stress. London: Harper & Row. Luo, W., & Najdawi, M. (2004). Trust-building measures: A review of consumer health portals. Communications of the ACM, 47(1), 109113.



91

Maes, P. (1994). Agents that reduce work and information overload. Communications of the ACM, 37(7), 31-146. Markus, M. L., Manville, B., et al. (2000, Fall). What makes a virtual organization work. Sloan Management Review, 13-26. MedSearch. (2004). MedSearch: Medical reference for Gulf-War related research. From http://www.gulflink.osd.mil/medsearch/glossary/ glossary_s.shtml Meyerson, D. E. (1998). Feeling stressed and burned out: A feminist reading and re-visioning of stress-based emotions within medicine and organization science. Organization Science, 9(1), 103-118. National Institute for Occupational Safety and Health. (2004). Stress at work. Washington, DC: Author. NIMH. (2004). National Institute of Mental Health: Attention Deficit Hyperactivity Disorder. From www.nimh.nih.gov/publicat/adhd.cfm Pondy, L. R. (1967). Organizational conflict: Concepts and models. Administrative Science Quarterly, 12, 296-320. Robey, D. (1984). Conflict models for implementation research. Applications of Management Science (Suppl. 1), 89-105. Sauter, S. L., Murphy, L. R., et al. (1990). Prevention of work-related psychological disorders. American Psychologist, 45(10), 1146-1158. Sawyer, S. (2001). Effects of intra-group conflict on packaged software development team performance. Information Systems Journal, (11), 155-178. Schill, T., Toves, C., et al. (1980). Interpersonal trust and coping with stress (Pt. 2). Psychological-Reports, 47(3), 1192. SGIA. (2004). SGIA’s (specialty graphic imaging association) glossary of terms. From http://www.sgia.org/glossary/Ss.cfm Suomi, R. (2000). Different conceptual approaches to virtual organization. The 5th International Telework Workshop, Stockholm, Sweden. Thong, J. Y. L., & Yap, C. S. (2000). Information systems and occupational stress: A theoretical framework. Omega, 28, 681-692. Trent, J. T., Smith, A. L., et al. (1994). Telecommuting: Stress and social support. Psychological Reports, 74, 1312-1314. Walsham, G. (1994). Virtual organization: An alternative view. Information Society, 10(4), 289-292. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Van Rooyen, E. (2000). A comprehensive change management framework for information technology-driven change in organisations. Proceedings of the 8 th European Conference on Information Systems, Vienna, Austria. Wastell, D., & Newman, M. (1996). Information system design, stress and organizational change in the ambulance services: A tale of two cities. Accounting, Management and Information Technologies, 6(4). WFMH. (2004). World Federation for Mental Health: The world mental health day project: glossary. From http://www.wfmh.org/wmhday/ sec3_pt3_4_glossary.html ViaHealth. (2004). ViaHealth disease and wellness information. From http://www.viahealth.org/disease/cardiac/glossary.htm#Glossary %20of%20Cardiac%20Terminology WordNet. (2004). WordNet: A lexical database for the English language. Young, K. S. (1996). Caught in the net. New York: Wiley.


The UTAUT Questionnaire Items 93

Chapter V

The UTAUT Questionnaire Items1

Performance Expectancy PU = Perceived Usefulness (Davis, 1989) RA = Relative Advantage (Rogers, 1983) OE = Outcome expectations (Compeau & Higgins, 1995) • • • •

P1. I would find the system useful in my job (PU). P2. Using the system enables me to accomplish tasks more quickly (RA). P3. Using the system increases my productivity (RA). P4. If I use the system, I will increase my chances of getting a raise (OE).


94 Spil & Schuring

Effort Expectancy PEU = Perceived ease of use (Davis, 1989) EU = Ease of use (Moore & Benbassat, 1991) • • • •

E1. My interaction with the system would be clear and understandable (PEU). E2. It would be easy for me to become skillful at using the system (PEU). E3. I would find the system easy to use (PEU). E4. Learning to operate the system is easy for me (EU).

Attitude Toward Using Technology AtB = Attitude toward Behavior (Fishbein & Ajzen, 1975) AtU = Attitude toward Use (Thomspon, Higgins, & Howell, 1991) A = Affect (Compeau & Higgins, 1995) • • • •

A1. Using the system is a bad/good idea (AtB). A2. The system makes work more interesting (AtU). A3. Working with the system is fun (AtU). A4. I like working with the system (A)

Social Influence SN = Subjective Norm (Ajzen, 1991) SF = Social Factors (Thompson, Higgins, & Howell, 1991) •

S1. People who influence my behavior think that I should use the system (SN).



• • •

S2. People who are important to me think that I should use the system (SN). S3. The senior management of this business has been helpful in the use of the system (SF). S4. In general, the organization has supported the use of the system (SF).

Facilitating Conditions PBC = Perceived Behavioral Control (Ajzen, 1991) FC = Facilitating Conditions (Thompson, Higgins, & Howell, 1991) • • • •

F1. I have the resources necessary to use the system (PBC). F2. I have the knowledge necessary to use the system (PBC). F3. The system is not compatible with other systems I use (PBC). F4. A specific person (or group) is available for assistance with system difficulties (FC).

Self-Efficacy SE = Self Efficacy (Compeau & Higgins, 1995) I could complete a job or task using the system… • S1. If there was no one around to tell me what to do as I go. • S2. If I could call someone for help if I got stuck. • S3. If I had a lot of time to complete the job for which the software was provided. • S4. If I had just the built-in help facility for assistance.


96 Spil & Schuring

Anxiety AN = Anxiety (Compeau & Higgins, 1995) • • • •

AN1. I feel apprehensive about using the system. AN2. It scares me to think that I could lose a lot of information using the system by hitting the wrong key. AN3. I hesitate to use the system for fear of making mistakes I cannot correct. AN4. The system is somewhat intimidating to me.

Behavioral Intention to Use the System B = Behavioral Intention (consistent with the underlying theory for all the intention models. • • •

B1. I intend to use the system in the next months. B2. I predict I would use the system in the next months. B3. I plan to use the system in the next months.

References Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211. Compeau, D. R., & Higgins, C. A. (1995). Computer Self-Efficacy: Development of a measure and initial test. MIS Quarterly, 13(3), 189-211. Davis, F. D. (1989, September). Perceived usefulness, perceived ease of use, and user acceptance of Information technology. MIS Quarterly. Fishbein, M. & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. New York: Addison Wesley.



Moore, G. C., & Benbassat, I. (1991), Development of an instrument to measure the perception of adopting an IT innovation. IS Research, 2(3), 192-222. Rogers, E. M. (1983). Diffusion of innovations. New York: Free Press. Thompson, R. L., Higgins, C. A., & Howell, J. M. (1991). Personal computing, toward a conceptual model of utilization. MIS Quarterly, 15(1), 124143. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

Endnote 1

Adapted from Venkatesh, Morris, David and Davis (2003)


98 Spil & Schuring

Section II Qualitative Evaluation


Information Systems Success Theoretical Framework

99

Chapter VI

Information Systems Success Theoretical Framework Margreet B. Michel-Verkerke University of Twente, The Netherlands Ton A. M. Spil University of Twente, The Netherlands

Abstract This chapter deals with the qualitative theory of information systems success. First, the definition of success used in this research will be given. Then, the theory on diffusion of innovations and success of information systems will be discussed to reveal what factors determine the successful implementation of information systems as far as the product and the user are involved. The outcome of this part of the literature study will be used in the construction of the USE IT model, in Chapter IX.


100 Michel-Verkerke & Spil

Definition of Success According to the Collins Paperpack Dictionary (1995) and Geerts, Heestermans, and Kruyskamp (1984), success can be defined as the achievement of an aim or the attainment of wealth, fame, or position. As will be argued in this chapter, the main aim of implementing an information system (IS) is to satisfy users, who use the IS when performing their daily tasks. Berg (2001) offered different definitions of success: Success could mean the actual use of a system, but also the appreciation of this use by the users or their managers. When discussing success, it must be clear what criteria or parameters are used to measure success. He regarded success as a dynamic concept, since the view– and by that, the criteria–on what success is, might change in time. The literature study that is presented in the next sections will lead to a list of factors that are considered to be essential in accomplishing a successful e-health implementation.

The Innovation Process and the Product Different ways to look at the innovation process exist. The first way presented here is the view of Larsen (1998), because it gives a good overview of what factors and elements are involved. Larsen stated that “elements of IS innovations include technical issues, human concerns, managerial actions and knowledge, interactions among line employees and information technology (IT) experts, strategic, tactical and operational requirements, organizational elements, and vision.” This means that a holistic vision is needed to analyze IS success. “The quality of the IS/IT product is a necessary but not sufficient prerequisite for IS innovation success. The people within the organizations determine the outcome.” The framework for IS innovation that Larsen (1998) presents is meant as a “checklist to map the elements actors include in their innovation undertakings and increase the awareness of IS innovation aspects yet not considered” (see Figure 1). The elements are categorized in five structures: the Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


101

Figure 1. The IS innovation framework: Key issues structure Idea phase Idea percolation

Idea molding

Change process definition

Usage phase

Change creation

Change anchoring

Change termination

Time horizon structure

Formal groups Organizational structure Informal groups

Change refinement

Innovation process structure

Units and networks

G R O U P S

Creation phase

Long term (strategic) Intermediate (planning and control) Short term (daily operations)

Human activity system view

Individual actors Artifact structure

Knowledge structure

Indepth expertise Pincipal understanding Rudimentary insight

Core business activity level Information needs and requirements level IS/IT expert level IS level Database level

Note. Innovation process structure comprises the project management, because IS innovation changes over time. Knowledge structure concerns the IS knowledge of managers and the business knowledge of the IS experts. Organizational structure has units and networks that make the formal organization. The formal group is the power elite who use innovations to realize their own objectives; informal groups often introduce changes. Individuals often initiate change. Artifact structure is not only the IS, but also the people’s visions and objectives connected with the IS. The marked boxes position the present research. Reprinted from Larsen (1998).

innovation process structure, the organizational structure, the time horizon structure, the knowledge structure, and the artifact structure. The framework is used to position this research and the discussed IS-success literature. The boxes marked in Figure 1 show that this research deals with individual actors on the information needs and requirements level. Larsen does not say which elements or structures have a greater or lesser influence on human activities, nor does he say which aspects enhance success. McGowan and Madey’s (1998) research on EDI implementation1 showed that factors that influence the adoption decision differ from the factors that influence the success of the implementation. The decision to adopt EDI was highly influenced by the customer’s demands to do so. After the adoption decision was made, the organization changed its processes to make the EDI implementation successful. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Larsen (1998) pictured the elements that should be considered in the innovation process and Rogers described what factors influence the decision to adopt an innovation. But what factors determine the success of this adoption? One model to explain success is the reformulated model of IS success of DeLone and McLean (2002), which gives a central role to the user by the variables “user satisfaction” and “use” (see Figure 2). To use the model to measure success, it is necessary to define from which point of view the measurement will take place. Like Berg (2001), DeLone and McLean state that this view is defined by the stakeholder and by the type of system involved. But whatever stakeholder or system is chosen, the net benefits depend on use and user satisfaction. These two interdependent factors play a key role in IS success: Net benefits will not be established without use and user satisfaction which factors are reinforced by achieved net benefits. Considering the innovation decision process of Rogers, one could say that the expected relative advantage in the persuasion stage must become true to lead to continued adoption in the confirmation stage (Chapter I, Figure 2). In the model of DeLone and McLean (2003), use and user satisfaction both depend on three other factors: information quality, system quality, and service quality. The operationalization to e-commerce metrics is shown in Table 1 as an example. These three qualities can be seen as aspects of the IS system or strongly related to it. In this way, these factors can be characterized as product quality. When placed in the framework of Larsen (1998), DeLone and McLean consider only the artifact and innovation process structure (see Figure 1). Figure 2. The reformulated IS success (DeLone & McLean, 2002)

INFORMATION QUALITY

USE (INTENTION TO USE) SYSTEM QUALITY

NET BENEFITS USER SATISFACTION

SERVICE QUALITY



103

Complying with DeLone and McLean (2003), Garrity and Sanders (1998) considered user satisfaction as the main criterion for IS success. From their research, Garrity and Sanders concluded that user satisfaction can be measured by measuring the dimensions: task support satisfaction (including decision-making satisfaction), quality of work life satisfaction, and interface satisfaction. Another view on the success of ISs was demonstrated by Saarinen and Sääksjärvi (1992). They stated that the success of an IS depends on the success of both the process and the product (see Figure 3). Process success is determined by success of the development process and by success of the use process. Quality of the IS product and impact of the IS on the organization result in product success. The development and use process can be linked with the innovation process structure of Larsen’s (1998) framework. The quality of the IS product is influenced by the artifact structure, and the impact of the IS on the organization is affected by the organization structure (see Figure 1). Saarinen and Sääksjärvi (1992) also found that factors affecting process success differ from factors affecting product success and showed that factors explaining success differ from factors explaining failure. Projects succeeding well in the use process can be characterized by a mature IS function (matching with Table 1. Delone and Mc Lean (2003) e-commerce success metrics Systems Quality • Adaptability • Availability • Reliability Response time • • Usability Information Quality • Completeness Ease of understanding • Personalization • • Relevance Security • Service Quality Assurance • • Empathy • Responsiveness

Use • • • •

Nature of Use Navigation patterns Number of site visits Number of transactions executed

User Satisfaction Repeat purchases • • Repeat visits User Surveys • Net Benefits Cost savings • • Expanded markets • Incremental additional sales Reduced search costs • • Time savings



Figure 3. Framework of factors affecting different success criteria in an IS development project (Saarinen & Sääksjärvi, 1992) STATE OF THE IS FUNCTION Corporate level: coverage of applications perceived maturity Business unit: coverage of applications perceived maturity MANAGEMENT INVOLVEMENT IN PROJECT INITIATION steering committee management initiative strategic IS planning management support INVESTMENT TYPE specificity uncertainty complexity novelty QUALITY OF DEVELOPMENT RESOURCES users' abilities system analysts' abilities

SUCCESS of the development process PROCESS SUCCESS SUCCESS of the use process

QUALITY of the IS product PRODUCT SUCCESS IMPACT of the IS on the organization

USE OF DIFFERENT DEVELOPMENT METHODS implementation approaches procurement strategies

service quality mentioned by DeLone & McLean), high level of management support, high experimentation rate (matching trialability mentioned by Rogers) and high level of using outside resources. Features of failures in the use process are: project initiated in a strategic IS planning project, system with high specificity, high requirements uncertainty (both related with complexity mentioned by Rogers) and use of software packages. In summary, one could say that potential users of an IS decide to start using the system because they expect the system to be useful to them and to bring them some kind of advantage, and that they will continue to use the system when they are satisfied with the system and the benefits it brings. The success of the IS can be measured by the user’s satisfaction with the system. In this approach, a causal relationship is suggested between the quality of the system (in this book,



105

the e-health system) and the success of the system for a certain group of users (in this book, care professionals).

References Berg, M. (2001). Implementing information systems in healthcare organizations: Myths and challenges. International Journal of Medical Informatics, 64(2-3), 143-156. Collins Paperback Dictionary. (1995). Glasgow, Great Britain: HarperCollins. DeLone, W. H., & McLean, E. R. (2002). Information systems success revisited. Proceedings of the 35th Hawaii International Conference on System Sciences. DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems, 19(4), 9-30. Garrity, E. J., & Sanders, G. L. (1998). Dimensions of information success. In E. J. Garrity & G. L. Sanders (Eds.), Information Systems Success Measurement (pp. 13-45). Hershey, PA: Idea Group Publishing. Geerts, G., Heestermans, H., & Kruyskamp, C. (1984). van Dale groot woordenboek der Nederlandse taal (Elfde, herziene druk ed.). Utrecht: Van Dale Lexiciografie BV. Larsen, T. J. (1998). Information systems innovation: A framework for research and practice. In T. J. Larsen & E. McGuire (Eds.), Information systems innovation and diffusion: Issues and directions (pp. 411-434). Hershey, PA: Idea Group Publishing. McGowan, M., & Madey, G. R. (1998). Adoption and implementation of electronic data interchange. In T. J. Larsen & E. McGuire (Eds.), Information Systems Innovation and Diffusion: Issues and Directions (pp. 116-140). Hershey, PA: Idea Group Publishing. Saarinen, T., & Sääksjärvi, M. (1992). Process and product success in information systems development. Journal of Strategic Information Systems, 1(5), 266-277.



Endnote 1

EDI = Electronic Data Interchange, a method of electronic exchange of messages.


The Dynamics of IT Adoption 107

Chapter VII

The Dynamics of IT Adoption in a Major Change Process in Healthcare Delivery Liette Lapointe McGill University, Canada Lisa Lamothe Université de Montréal, Canada Jean-Paul Fortin Université Laval, Canada

Abstract The primary objective of the study presented in this chapter is to understand the dynamics of implementing an integrated information technology (IT) system designed to support the treatment of breast cancer. Preliminary findings permitted the identification of adoption and resistance factors at individual, professional, and organizational levels. These findings provided


108 Lapointe, Lamothe & Fortin

insight into the potential challenges to be faced in redesigning work processes in an inter-organizational setting. Significant adoption factors included individual participation in the project and a positive assessment of benefits of the system by both individuals and professional groups. Alternatively, the absence of a multidisciplinary committee, coupled with the complexity of the environment, the tasks, and the technology were among factors identified as troublesome. In terms of the dynamics of the process itself, our observations revealed that promoters adopted a contingent and political approach. While this strategy is usually considered appropriate in healthcare organizations, information generated by this study will allow for a better understanding of important factors and mechanisms to consider and address in subsequent phases of implementation.

Introduction Globally, new health challenges, coupled with scarce resources, are creating an overwhelming need to ensure that every dollar invested in healthcare is maximized to its fullest potential. It therefore becomes critical to evaluate the development, introduction, and use of clinical information systems. However, this task remains extremely challenging, in part because of the lack of information available to both health professionals and managers. As indicated in the chapter by Ammenwerth and Turunen, there exists no widely accepted framework for the evaluation of clinical information systems, neither in management nor in the medical field. As is the case with the experiences related in other chapters of this book, this study aims to better understand the dynamics of implementing medical information systems. More precisely, this chapter provides information regarding the factors and mechanisms that influence the outcome of integrated medical system implementations in an inter-organization context. Indeed, over the past 10 years, many healthcare organizations have responded to pressure to improve performance, quality, and patient satisfaction with an attempt to create integrated healthcare systems. Such systems can be defined as “a network of organizations that provides or arranges to provide a coordinated continuum of services to a defined population and is willing to be held clinically and fiscally accountable for the outcomes and the health status of the population served” (Shortell, Gillies, Anderson, Mitchell, & Morgan,



1993). The process of integration implies the introduction of information technology (IT). This, like any other change that generates a major transformation, challenges the way people think about workplace behavior. This chapter presents the preliminary findings of an empirical study that aimed to follow the change process as it occurred during the implementation of an integrated delivery system for treating women with breast cancer throughout all stages of the disease. This system, initiated in Quebec, focused on improving the care for, treatment of, and research on breast cancer. Inspired by guidelines issued by a government program (Battle against Cancer Program), promoters aimed to improve the continuity of care, the sharing of information among caregivers, and the diffusion of knowledge about the disease. Promoters assumed an adaptive approach for the design of the delivery of care rather than imposing a specific model. However, the project was guided by four principles: a network of services between local and regional institutions, multidisciplinary work, an emphasis on community services for patients, and the identification of a privileged caregiver at each stage of the disease. This would be achieved with the development of computerized medical records accessible by multiple caregivers in various care institutions, including general practitioners working in private clinics. The project (using the French acronym DRIOQ) was seen as a research endeavor that would pave the way for implementing similar networks of services for other cancer diseases. It was meant to serve as a model for networks to be implemented across Quebec and Canada. The project was granted funding from the Canadian government and it involved private systems designers. The project had a relatively short period of time within which to achieve its goals. There were only 2 years between the allocation of funds and the deadline by which the system had to be implemented. The principal objective of this study is to thoroughly understand the dynamics of interactions among partners involved in the process of a clinical system implementation and to identify factors that influence adoption and resistance to the system. The findings helped to identify variables and mechanisms that can facilitate the adoption of such systems and to suggest ways to better manage the implementation of clinical information systems. At the same time, they served to help explain the difficulties encountered in implementation. This chapter begins by explaining the theoretical framework used for the analysis. It then describes the methodology used, after which, it presents an analytic description of the findings in



terms of factors influencing adoption and resistance to the system and of the change management process. It concludes by presenting contributions of this work for research and practice.

Theoretical Approach Previous research (Anderson, 1997; Doolin, 1999; Greene, 2003; Lapointe, 1999) has shown that the implementation of clinical information systems is a challenging task. To be successful, it is essential to revisit the process of delivering care, to take advantage of the facilitating factors, and to overcome the barriers to implementation. To reach this goal, it is necessary to take into account individual attitudes as well as the attributes and relationships of professional groups. At the individual level, research in IT suggests different models to better understand the implementation process (Davis, Bagozzi, & Warsaw, 1989; Mathieson, 1991; Moore & Benbasat, 1995, Venkatesh, Morris, Davis, & Davis, 2003). Most individual-level models are based on the theory of reasoned action (Fishbein & Ajzen, 1975) in which individual behaviour is directly determined by one variable intention, which, in turn, is determined by two variables: attitude and subjective norms. At the group level, the models of political interaction (Markus, 1983) and system of professions (Abbott, 1988) provide valuable insight. In a professional organization, power dynamics are mainly related to the struggle to maintain or expand professional domains. As explained by Abbott’s system of professions (Abbott, 1988), domains are constantly redefined in the workplace through negotiations between professional groups. As shown in the political variant of the theory of interaction (Markus, 1983), resistance to IT can then be explained by the interaction between the characteristics of the system and the power distribution within the organization. In this model, five variables are used to explain resistance. Through the distribution of power in the organization, it is possible to understand the organizational balance of power before the implementation of a given system. This variable will influence the interests of the actors to modify this balance and the power shift implied by the use of the system. The interests of the actors are important because they have a direct impact on the actors’ political tactics and perception of the social significance of the



system. Resistance is therefore explained by the potential of the system to change the power distribution in the organisation. If a group of actors considers that the use of a system will support its position of power, then the group will be inclined to use that system. However, if they believe that the implementation will result in a loss of power, they will resist. Using this theoretical background, this study attempts to more thoroughly comprehend how systems are adopted or rejected. In this particular case, an additional layer of complexity had to be taken into account: the interactions among the various organizations (namely primary care organizations, private clinics, and hospitals). For this reason, the study also seeks to better understand the management of the change process. Difficulties encountered in the management of complex change processes are, for the most part, related to the ambiguity surrounding the object of change and therefore, the implementation process (Lamothe, 1999).

Data Sources and Analysis Two sites were chosen for the experiment. The networks were designed to revolve around two referral hospitals offering specialized care in breast cancer; one is located in an urban region and the other in a semiurban region. The differences in their modes of delivery were expected to generate the necessary learning to expand elsewhere. Other institutions that participated in the networks were chosen according to criteria such as volume of clientele, variety of professional practices, existing links with hospitals, and interest in the project. Data collection was based on observation, interviews, analysis of documents, and notes taken at committees. People interviewed (N = 17) were selected using a snowball approach (Lincoln & Guba, 1985), meaning that they were identified by the persons concerned, progressively, as the project evolved. Scheduling of interviews was adapted to the evolution of the project. From October 1999 to April 2000, observation and documentation was conducted on the evolution of the process during the initiation phase. Then, three series of interviews were conducted: May-July 2000, November-December 2000, and March 2001. Data collected were coded using a qualitative data analysis software, Nudist-N*Vivo. Document sources included minutes taken at meetings, publicity documents prepared by the Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


promoters, and other documents relevant to the context of the project. Meetings were attended and notes were taken. They permitted the identification of critical issues and how each was addressed. In order to understand the interactions among partners involved in the process and to identify factors influencing adoption or resistance to the system, the dynamics of the development and the implementation were studied at both the management and professional levels. The analysis of the change management process, with the identification of change agents and their actions, was aimed at understanding how these actions might influence adoption of, or resistance to, the system. The analysis of interactions among professionals concerned was aimed at identifying both facilitating and constraining factors to adoption at the individual, professional, and organizational levels. For the analysis of adoption/resistance factors, techniques for data reduction and presentation similar to those suggested by Miles and Huberman (1994) were used. Separate tables allowed the differentiation between promoters’ and users’ views on individual, professional, and organizational factors at the beginning of the project and at the experimentation phase. For the change management process, a reconstruction of the chronological events (committees formed, presentations to users, etc.) allowed the identification of both deliberate and emergent strategies. By the deadline, only one network had been tested. This was primarily due to delays in delivering the technology. However, interest had been piqued and authorities decided to extend the project deadline. Because the study needed to adapt to the course of events, the current analysis of the change process and of the factors influencing adoption and resistance focused on the initial phases of the process. Interviews were conducted both during the development phase and after the beta-version of the system had been implemented in the hospital. Other institutions are now being connected to the network. Consequently, the analysis presents a) an identification of adoption/resistance factors as observed and as expressed by users and partners (managers, professionals, in-institutions systems technicians) at the development stage and during the experimentation stage and b) a critical evaluation of the change management process in the early stages of the implementation.



Adoption and Resistance Factors The Individual Level Adoption and resistance are two ends of a continuum of reactions to the implementation of a system. In order to explain reactions toward the introduction of IT, it is important to take into account individual factors that lead to resistance or adoption behaviour (Moore & Benbasat, 1995). In this project, individual reactions were generally positive. The promoters were able to build upon the interest shown by future users and to focus this positive energy to convince them to participate in the development and implementation process. As was shown by Hartwick and Barki (1994), participation is a key factor for the success of the development of a system and the subsequent satisfaction of the users. Perceived benefits (Davis et al., 1989) were also seen in a positive light. Users were eager to use a system that would fit their clinical and research needs while simultaneously leverage to provide improved care and better answer to the needs of patients. Most users were also keen on the possibility of sharing information with their key partners. Being able to work in a complementary manner rather than in competition with one another along with the ability to improve communication and promote multidisciplinary teamwork were seen as important benefits for users. It was anticipated that the use of the system would improve the quality of data and save time in obtaining access to clinical information gathered by other caregivers. All users felt that better access to information would improve the quality of care for patients and were enthusiastic about this key benefit of the system. Moreover, users looked forward to a system that would allow the development of a database for breast cancer information. This database was considered to be a tremendously valuable tool for advancing cancer research. It was a driving force for participating physicians to encourage the development, adoption, and use of the system. The initial reactions to the system were, according to interviews and documentation, positive. Most users were satisfied with the beta-version of the system and the consequences of its adoption were perceived to be positive. From the very beginning of the process, however, some users expressed concerns and fears that needed to be addressed. The risk that the system would not meet the expectations of the users remained a salient concern. It was also important to ensure that all users would have a sense of equity when they



evaluated the implementation of the system and its consequences (Joshi, 1991). Some users feared that the system would be used as a control mechanism and that its use would lead to a loss of professional autonomy. Others, mostly those who were not familiar with the use of computers, feared that the investment (time and effort) would exceed the anticipated benefits. Additionally, this fear was augmented by concerns regarding the time required for data entry, as most healthcare workers were already overloaded. As shown by Davis et al. (1989), these factors can negatively impact the successful outcome of the implementation process. Many users, physicians, nurses, and others did not feel that they should be responsible for data entry. Research shows that this factor is an important source of resistance in clinical system implementation (Kaplan, 1987). These constraining factors must not be taken lightly by promoters and action must be taken in order to ensure that they do not negatively impact the outcome of the project.

The Group Level At the group level, resistance can be defined as a struggle to promote common interests and to avoid domination. In sociology and related fields, research shows that resistance will occur when a group perceives something as going against their interests (Markus 1983). In this case, there were numerous groups of actors involved in the project. Consequently, a wide variety of interests were promoted and, at times, the various points of view conflicted with one another. Not only was it important to take into account the various interest groups in the project (users, promoters, development team, private partners, etc.), but, within each group, it was important to identify the subgroups that would defend divergent interests (e.g., nurses vs. physicians; generalists vs. specialists). In this context, reaching equilibrium required special attention to be paid to each group’s interests and adapting management strategies accordingly. This point will be discussed further in the section dealing with the analysis of the change management process. •

Promoters: The promoters’ interests were, first and foremost, in the successful development and implementation of the system. One contributing factor toward the realization of the project was the initial



•

•

composition of the promoting team. Members of the group were considered to be leaders in their respective field. Moreover, in terms of clinical IT implementation, one of the members had previous experience, vision, and valuable input. It seemed like a winning team. On the other hand, the complexity of the project was often described as a constraining factor. According to some promoters, the level of complexity was such that it was difficult to encompass and manage the project. Finally, it should be noted that departures from the team, including that of a clinical leader, broke the rhythm of the implementation process and impaired the change management strategy. Private partners: Collaboration with private partners was promising. Their main interest was to ensure a successful outcome in the development of the technology, in order to promote their own reputations. Their expertise was considered to be a critical success factor. For example, one consulting firm was seen as being able to provide better structure in the development process. However, this firm evolved to play a more important role than was initially expected. At one point, the development process could have been labeled as techno-driven, which conflicted with the promotion of the interests of the users working in the field. This issue created dissatisfaction among users. Complaints were further enhanced by delays in the technical development and tensions arose. Professionals: All professional caregivers expected a system that would support their work and meet their needs. However, the needs of the individual groups often differed. In this regard, the fears of some users and future users were expressed in terms of complaints about the lack of clarity and precision in the objectives of the project. Expressing a need for more information, they articulated that the roles and responsibilities of various partners were not well defined. They expressed a desire for a multidisciplinary or a multi-organizational committee to whom they could voice their needs and interests. The absence of such a committee was seen as a detriment to the process. Professionals also voiced concerns in regard to some of the technical aspects and features of the system. Previous research indicates that it will be necessary for promoters to heed the concerns expressed in order to avoid further dissatisfaction and to ensure smooth implementation of the subsequent phases.



The Organizational and Environmental Levels Because of the inter-organizational nature of the project, it was essential to take into account each organization’s defining characteristics as they could potentially influence the implementation. For example, some organizations had experience with IT implementation projects, whereas others were newcomers to the field. Furthermore, each type of care-homecare, primary care, specialized care-had a specific mode of delivery. This created specific implementation needs, both in terms of system features and change strategies. In terms of links with the legal, political, and social environment, several factors were considered to be important. The will of the promoters to maintain good relationships with actors and groups from the external environment—no matter how difficult and complicated the process was—positively influenced the outcome. In addition, because the project was in line with the new vision of the ministry of health, it was possible to obtain additional resources. External support helped to reinforce the importance of the project among the various groups and individuals concerned. The legal environment, however, was identified to be a source of problems. A multitude of legal dispositions were complicating the expansion of the project and the development of the network (e.g., confidentiality laws). The scarcity of available resources in healthcare may have helped to provide the project with credibility, as it represented a basis to obtain new funding, to improve the level and quality of care, and to enhance the performance of the various organizations involved. As a result, the project became valuable because it provided new tools that were considered to be critical by caregivers. However, in the future, the need for additional resources to create and update the information may create a potential problem within the same context of scarce resources. This issue will have to be dealt with before the emergence of a crisis. Finally, competition between healthcare organizations—general hospital vs private clinics or private cabinets vs community-care organizations—was a factor that was, according to many users, worrisome because it represented problems that had the potential to be accentuated by the system implementation. Furthermore, the delivery of care in hospitals does not follow a particularly structured process due to the nature of the work. Because IT typically best supports very structured processes, the development and implementation of the system was particularly challenging. In addition, the traditional delivery of



care does not support multidisciplinary work in an inter-organizational context. This factor had the potential to impair the success of the implementation and required close monitoring. In this regard, some users expressed the concern that the organizational dynamics of the network was not sufficiently understood.

The Change Management Process The management of a complex change process deals with challenges that are largely related to the amount of ambiguity associated with the definition of the object of change and its implementation process. Challenges encountered by the project were identified and discussed. In general, the strategic approach followed was devised from a process that relied upon the following: •

• • • •

creation of an administrative structure that took into account the plurality of aspects of the project and internal and external environmental factors; constant management of the equilibrium between external and internal contexts through the creation of alliances at both levels; room for negotiations between partners involved with the service delivery model to be implemented; penetration strategy in an organization that relied on the identification of a champion; and prototype strategy for the development of the product.

Administrative Structure with a Pluralistic Strategic Apex The strategic apex of the project was pluralistic, as it was composed of members whose expertise and interests allowed them to cover all dimensions of the project (information systems experts, professionals, etc.). Also, the project’s composition allowed it to maintain the necessary links with actors from the external environment. This committee was efficient in stimulating a mandatory learning process through frequent interactions. Indeed, the service delivery model, as well as the computerized system, needed to emerge from this Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


learning process. Denis, Langley, and Pineault’s (1996) work on strategic change indicated that the emergence of an effective leadership team is a critical component of the process. Collective leadership builds, in part, upon complementary roles and equilibrium in relationships. However, the departure of a respected oncologist created an imbalance within the committee and resulted in a technology-driven period during the development. Systems designers seized the opportunity to influence the project. In this regard, because it allows for the representation of all parties involved, collective leadership can become a locus for the confrontation of diverse logic: The logic of constant negotiation on work processes (professionals) confronts the logic of rigor in the development process (systems designers). The goal of having private systems designers participate in the development of the system was to profit from their expertise; however, it was also necessary for them to invest in the project. Their interest was in participating in a research project that, even though it involved risks, could bring opportunities: developing a new technology within a closed budget, integrating multiple and complex activities, and reconciling the research logic with the usual logic used in development. Research objectives conflicted with the search for business solutions. The presence of more than one private firm raised some confusion as to each firm’s respective contribution; this subsequently resulted in some competition among the firms. The promotion of an immature technology was also responsible for delays in the project. Additionally, private firms were uncomfortable with a strategy that was based upon a collective leadership and a bottom-up approach. These challenges resulted in delays but were resolved with the development of an innovative system with evolving capabilities. Here, the project leader’s role is critical in the creation of tight and collective leadership. The project leader needs to channel negotiations and activities and is responsible for maintaining the equilibrium.

Management of the External Context Management of the external context was crucial to the legitimacy of the project. A committee, created from representatives of external stakeholders (professional groups, etc), was very influential. It ensured that the system developed adhered to the principles guiding the treatment of cancer patients and was consistent with other systems being developed. The committee’s involvement also influenced the permanence of the system within the organization. A long lifespan for the system was very important, considering the Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


amount of effort that users expended in transforming their work processes. The role of the committee also brought forth its share of challenges. Management of the project became very complex and highly political. Ironically, the committee that was seen as essential to making the system an integral part of the organization was also partially responsible for causing delays in the project and therefore required careful monitoring.

Creation of Alliances with Partners Many presentations were made to convince clinicians and institutions to participate. However, delays in the development of the computerized medical records created a time lag during which momentum and enthusiasm was lost. A planned inter-organizational committee was never created, much to the regret of the users. The service delivery mode created new interorganizational and inter-professional relationships. At this level, negotiations helped to reduce uncertainty. Lamarche, Lamothe, and Bégin (2001) have demonstrated that effective inter-organizational networks of services emerge from a learning process in which the autonomy of institutions and the existence of relationships built on confidence are critical issues at the negotiating table.

Definition of the Model for Delivery of Services The adopted strategy put an emphasis on the participation of stakeholders in the definition of the model to be implemented. This bottom-up approach was consistent with the internal characteristics of healthcare organizations, where professionals are in control of production (Lamothe, 1999). However, in the minds of the promoters four principles needed to be taken into account (a network of services between local and regional institutions, multidisciplinary work, an accent on community services for the patients and the identification of a privileged caregiver for each stage of the disease). These principles had not yet been the focus of any discussions, most likely due to the fact that the beta-system was not yet ready to connect multiple organizations. Consequently, this situation created some uncertainty. Indeed, the stability and efficiency of professional work is grounded in a negotiated order emerging from interactions between professionals in search of control over their domain and the domain(s) of others (Abbott, 1988; Strauss, Schatzman, Ehrlich, Bucher, & Sabshin, 1963; Strauss & Schatzman, 1964). Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Penetration Strategy: Local Championship Collaboration within an organization is hard to obtain. In this case, many people expressed their lack of time to dedicate to the project. The identification of a local champion helped to legitimize the project and was revealed to be very effective. Although this champion played an exceptional role strategically, a second champion proved necessary for implementation. In fact, past research has shown that the association of a champion with transformational leadership behaviours is essential (Howell & Higgins, 1990).

Prototype Strategy In a highly complex situation, a prototype strategy is often suggested, as it facilitates the learning process. In this instance, the participation of clinician users in the development of the prototype was crucial in order to adequately address their needs. Research shows that the use of prototyping can further the potential of IT to enhance consistency between strategy and operation, improve the quality of empowered employees’ decisions, and facilitate organizational learning (Fuglseth & Gronhaug, 1997).

Strategy Observation of the strategy that was used to manage the change process revealed that promoters had adopted a contingent and political approach. Their approach was contingent because they defined the major aspects of the project by analysing external influences (e.g., policies/strategies to combat cancer) and were preoccupied with the creation of structures and management systems most favourable to the achievement of objectives. This approach focuses on the potential influence of the external and internal environments on the content and process of organizational change. Their approach was also political because the product of change emerged from a complex process of negotiation among the partners involved. Important factors need to be taken into account in order to ensure the success of the proposed change: relationships between organizations that need to cooperate and coordinate their activities, complexity of the exchange processes, and external constraints. In this case, promoters also used political and symbolic language in order to legitimize change and reduce opposition. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


In healthcare organizations, this strategy is usually considered appropriate for the formulation, as well as the implementation of change for at least three reasons: the nature of the general and internal contexts of healthcare institutions, the attributes of the change process, and the characteristics of the changes themselves. Although separating these three sources of explanation may make them appear to be independent entities, they should be seen as three elements that are dynamically interrelated during a period of time (Dufour & Lamothe, 1999). The first source of explanation concerns the general operating context of healthcare organizations. Healthcare institutions are usually public organizations, or at least services for the public good, evolving within a system in which they are subject to public policies and programs established at the state level. However, they have a more diversified environment, thus making them accountable to a greater number of parties. This situation gives rise to a greater awareness of political behaviors (Paquin, 1992). Yet another explanation relates to the nature of the internal context of healthcare institutions. Within healthcare organizations, numerous individuals and groups with very different backgrounds, skills, and abilities interact on an ongoing basis, in a structure that demands a high level of functional interdependence. Production units depend on an arrangement of occupational spaces subjected to constant negotiations between the groups. This occupational structure, directly responsible for the control of production, explains the production system’s immunity from the administration (Lamothe, 1999). The very strong presence of political games is associated with an equally strong presence of emergent change. Certainly, actions undertaken at the level of both the general and internal contexts influence the structural form. Thus, the professional bureaucracy itself becomes an emergent product of political interactions between the system’s leaders and actors. A second reason why a combined political and contingency approach may be more appropriate relates to the particular features of the change processes in healthcare organizations. These processes are complex and are based upon the interactions, learning, and autonomous behaviour of the organization’s members rather than on the top-level leadership, vision, or mission. Under these conditions, planned, intentional change is difficult. Change is achieved through entrepreneurship and continuous dialogue between the stakeholders. Feedback and learning require the creation of interfunctional channels of communication as well as the establishment of new exchange networks among the key actors (Hart, 1992). The role of leaders is thus to encourage experimentation,



exploration, and risk, to facilitate transactions, to cultivate ideas, and to connect the results of these processes over time so as to provide a sense of direction in change (Denis et al., 1996). The third explanation relates to the features of the content of changes. Among the potentially important characteristics affecting the final outcome are: the high number of simultaneous and consecutive changes in healthcare organizations, the lack of consensus among participants on change and its necessity, the divisibility of its benefits, the scope and depth of the required change in behaviour, the rather long-term nature of, and confusion over, the goals being pursued; and the number and dispersion of decision points (Grindle, 1980; Hafsi & Fabi, 1997). In addition, considering the rather overwhelming and threatening nature of the changes, merely alluding to the possibility of change is enough to generate a certain amount of political energy.

Discussion and Conclusion This study highlights the fact that creating integrated delivery systems with the support of IT faces two major challenges related to two chief sources of uncertainty. First, although integrated delivery systems are generally considered to be an optimal response to changing modes of delivery, the best possible means to achieve them are still up for debate and various models find their way to the discussion table. In such a context, revisiting the process of care delivery to develop computerized clinical systems becomes a challenging task. Second, factors associated with the success of computerized clinical system implementation in healthcare organizations are numerous, complex, and often difficult to identify. The management of such a project therefore needs to take all those challenges into account. This study sheds some light on both of these aspects. Building on previous research, the findings of this study expose the diversity of interests related to such a project and illustrate the complexity of the context in which a computerized clinical system is to be implemented. Adoption/resistance factors may be individual, but also group related, as professionals struggle to secure or boost their relative position in a complex professional system. Intra-organizational factors can be identified as a determinant to the successful implementation of clinical systems in healthcare organizations. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Because of the nature of this project, it is also important to identify interorganizational factors and mechanisms that lead to success. More specifically, attention is drawn to the linkage of organizations with various cultures, having specific needs, and acting in competition with one another. This consideration increases the political activity around the project. Finally, environmental factors (ethical, legal, technological) need to be identified as they have a deterministic influence on the definition of the product developed. Such a complex context needs to be managed with precision. This study reveals how a contingent and political approach may be appropriate to address all dimensions of the context, to allow all stakeholders to take part in the development, and to leave room for innovation (integrated delivery system). It also demonstrates that such a strategy is conducive to the internal and external contexts of such organizations, even though it may seem awkward to apply. Such a pluralistic approach allows for all parties involved to participate in the creation of a collective leadership. Although difficult to maintain, its equilibrium is crucial if promoters wish to ensure successful achievement of their goals. A pluralistic approach also appears to be critical to the learning process that is required to allow professionals to adapt their practices. In healthcare organizations, as in other professional organizations, effective change-such as the implementation of a clinical information system-is an output of planned and emergent change. Computerizing clinical work is extremely complicated, partly because of its nature. Unquestionably, it is multifaceted and it involves numerous interdependent professionals. Moreover, clinical work allows for a bevy of informal relationships, which makes it problematic to structure with rigid rules. The analysis of this project highlights many factors that allow us to foresee a positive outcome for the implementation of this system. Indeed, data from the experimentation phase confirms that individuals are generally satisfied with the system. In general, professionals appreciate the system’s functionality and application. Furthermore, it seems that time may be saved by physicians, once the system is fully incorporated into their work routines. The new system allows for better access to medical records, easier follow-up of patients, closer monitoring of research projects, and improved exchange between caregivers. The objectives of the project are in line with those of the hospital where experimentation is being conducted. There is still a great deal of enthusiasm, especially on the part of the champion of the project, a physician. Additionally, there is a high probability that support will be available for the addition of other institutions to the network.



However, to really ensure a positive outcome of the implementation of the system, there are still challenges at all levels that must be addressed. Some of the resistance factors identified during the elaboration phase still trouble managers, clinicians, and other professionals. For example, at the individual level, unfulfilled expectations, lack of participation, interest, and/or experience with IT are still raising some opposition. At the group level, divergence of interests, power relations, and complexity still maintain the potential to cause resistance. Professional users want to be involved in negotiations. However, it is important to note that delays related to technical development appear to have a negative impact on their involvement. Moreover, it will be extremely important to better understand the inter-organizational relationships and their systemic compatibility as they retain the ability to cause difficulties, especially in light of the need for additional resources and the competition among organizations. In the next phases of the implementation, it will be crucial to appropriately manage resistance. At the same time, it will be essential to make the mose of the factors promoting a positive impact. The addition of other healthcare institutions to the network will widen the scope of experimentation and multiply the number of stakeholders. This additional level of complexity suggests that negotiations on the delivery model should be framed by guiding principles that will need to be clearly understood by each stakeholder. This would help to overcome the limitations encountered with a bottom-up approach. Without a doubt, the wide variety of existing clinical practices multiply the model options and it is clear that dead-end negotiations may be responsible for delays. In addition, although the information system needs to be adaptive to particular modes of practices, adaptation to a wide scope of models may overly increase the complexity of the technical development. With the creation of an inter-organizational network, the new mode of delivery needs to build upon organizational cultures emerging from embedded clinical practices. Very little is known about the development of clinical information systems in an inter-organizational context. This study attempts to address that lack of information while simultaneously bringing new insight to the optimal conditions necessary for the system to be transferred to other settings. Interorganizational relationships increase the political nature of the change process and therefore the complexity of the change management process.



References Abbott, A., (1988). The system of professions. Chicago: University of Chicago Press. Anderson, J. (1997). Clearing the way for physicians’ use of clinical information systems. Communications of the ACM, 40(8), 83-90. Davis, F. D., Bagozzi, R. P., & Warsaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 30(6), 983-1003. Denis, J-L,, Langley, A., & Cazale, C. (1996). Leadership and strategic change under ambiguity. Organization Studies, 17(4), 673-699. Denis, J-L., Langley, A., & Pineault, M. (1996). Devenir leader dans une organisation complexe. Gestion, 21(4), 15-24. Doolin, B. (1999). Casemix management in a New Zealand hospital: Rationalisation and Resistance. Financial Accountability & Management, 15(3/4), 397-418. Dufour, Y., & Lamothe, L. (1999). Les approches au changement dans les systèmes de santé. In C. Bégin, P. Bergeron, P-G. Forest, & V. Lemieux (Ed), Le système de santé québécois, un modèle en transformation. Montreal: Les Presses de l’Université de Montréal. Fishbein, M., & Azjen, I. (1975). Belief, attitude, intentions and behavior: An introduction to theory and research. Boston: Addison-Wesley. Fuglseth A., & Gronhaug, K. (1997). IT-enabled redesign of complex and dynamic business processes: The case of bank credit evaluation. Omega, 25(1), 93-106. Greene, J. (2003). Information technology. The $5 billion plan. Hospitals and Health Networks, 77(5), 16-18. Grindle, M. S. (1980). Policy content and context in implementation. Politics and the policy implementation in the third world. Princeton, NJ: Princeton University Press. Hafsi, T., & Fabi, B. (1997). Les fondements du changement stratégique. Montréal: Éditions Transcontinental. Hart, S. L.(1992). An integrative framework for strategy-making processes. Academy of Management Review, 17(2), 327-351.



Hartwick, J., & Barki, H. (1994). Explaining the role of user participation in information system use. Management Science, 40(4), 440-465. Howell, J., & Higgins, C. (1990). Champions of technological innovation. Administrative Science Quarterly, 35(2), 317-342. Joshi, K. (1991). A model of users’ perspective on change. MIS Quarterly, 15(2), 229-243. Kaplan, B. (1987). Initial impact of a clinical laboratory computer system: Themes common to expectations and actualities. Journal of Medical Systems, 11(2/3), 137-147. Lamarche, P., Lamothe, L., & Bégin, C. (2001). Effets des modes d’intégration des services en émergence dans la région sociosanitaire des Laurentides. Québec, Canada : Rapport de recherche. Lamothe, L., (1999). La reconfiguration des hôpitaux : un défi d’ordre professionnel. Ruptures, 6(2), 132-148. Lapointe, L. (1999). L’adoption de systèmes d’information cliniques par les médecins et les infirmières: Une étude des variables individuelles, socio-politiques et organisationnelles. Montreal: Université de Montréal. Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Newbury Park: Sage. Markus, M. L. (1983). Power, politics and MIS implementation. Communications of the ACM, 26(6), 430-444. Mathieson, K. (1991). Predicting user intentions: Comparing the technology acceptance model with the theory of planned behavior. Information Systems Research, 2(3), 173-191. Miles, M., & Huberman, M. (1994). Qualitative data analysis (2nd ed.). Sage. Moore, G., & Benbasat, I. (1995). Integrating diffusion of innovations and theory of reasoned action models to predict the utilization of information technology by end users. Proceedings of the IFIP Working Group 8.6 Conference, Oslo, Norway. Paquin, M. (1992). La planification stratégique dans le secteur public. chap.18. In R. Parenteau, et al., Management public–Comprendre et gérer les institution de l’etat. Ste-Foy: Presses de l’Université du Québec Shortell , S., Gillies, R., Anderson, D., Mitchell, J., & Morgan, K. (1993). Creating organized delivery systems: The barriers and facilitators. Hospital & Health Services Administration, 38(4), 447-467.



Strauss, A. H., Schatzman, L., Ehrlich, D., Bucher, R., & Sabshin, M. (1963). The hospital and its negotiated order. In E. Freidson (Ed.), The hospital in modern society (pp. 147-169). London: Collier-MacMillan. Strauss, A., & Schatzman, L. (1964). Psychiatric ideologies and institutions. New York: Free Press. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.


128 Suomi

Chapter VIII

Introducing Electronic Patient Records to Hospitals: Innovation Adoption Paths

Reima Suomi Turku School of Economics and Business Administration, Turku

Abstract Healthcare is on of the industries that is currently fast adopting information technology (IT) into use. Electronic patient records (EPRs) are at the hearth of healthcare information technology applications. However, patient data is seldom efficiently organized even within one organization, and when patient data is needed in applications covering several organizations, the situation becomes even more complicated. We draw some lessons on how EPR systems should look like from the customer relationship management literature point of view: After all, patients are the customers of healthcare institutions. As a guiding framework for this analysis we use the concepts developed by (Winter, Ammenwerth, et al,. 2001). Then we proceed to discuss how EPR systems diffuse in the healthcare industry and use the Internet standards adoption (ISA) model


Introducing Electronic Patient Records to Hospitals

129

presented by (Hovav, Patnayakuni, et al., 2004) as a starting point. We apply this model to the diffusion of EPR systems in the healthcare industry. We found big differences between customer relationship management and EPRs management. Customer relationship management aims at long-term relationships and customer profitability, which are not strong goals for EPR systems. Our analysis too resulted to the conclusion that the practical innovation adoption bath for EPRs over paper-based patient records is that of adoption through coexistence.

Introduction In this chapter we discuss the concept of electronic patient records (EPRs). Our basic question is, how can the innovation and concept of EPRs be diffused as efficiently as possible to the healthcare industry. This question is not without importance, as electronic healthcare records have remained an unsettled promise for over 40 years. Implementing an EPR is a demanding task as seen from the computer professionals point of view, too (Lincoln, 1993), not to speak of the challenges set for the medical profession. Implementing any information systems into healthcare organizations would be much easier than now if the organizations would have clear general and information system strategies that would tell them that they have to do this (Winter et al., 2001). Unfortunately, this is often not the case (Douglas & Ryman, 2003). Even national strategies supporting the establishment and introduction of EPR are scarce, and if they do exist, practical means, incentives and resources to implement the plans are often neglected (Linna, 2000). Therefore, introducing EPR is not a straightforward and easy task supported by strategic management and prospective users, but rather a risky and difficult process. We can well interpret that introducing EPR can be studied from the viewpoint of innovation diffusion in a complex system. Our chapter unfolds as follows. First we discuss the concept of EPRs, define the concept, and touch upon the history of the concept as well as the reasons for using EPRs. In the second section, we turn to the area of customer-relationship management and discuss how it could support the fast adoption of EPRs. As our framework for analysis, we use the concept analyses of knowledge management, cus-


130 Suomi

tomer-relationship management, and customer-knowledge management introduced by (Gibbert, Leibold, et al., 2002). Third, we introduce the Internet standards adoption (ISA) model presented by Hovav et al. (2004) and continue to discuss how it can be interpreted to shed light to the adoption of EPRs in the healthcare industry. Finally we draw conclusions on the whole story and summarize the most important findings. Methodologically our study is conceptual, with no original empirical research.

Electronic Patient Records EPRs are patient records in electronic form. Their advantages can be described as follows (Rogerson, 2000): The electronic patient record (EPR) is indicative of the advances in medical informatics and allows providers, patients and payers to interact more efficiently and in life-enhancing ways. It offers new methods of storing, manipulating and communicating medical information of all kinds, including text, images, sound, video and tactile senses, which are more powerful and flexible than paper based systems. A loved kid has many names. We use EPRs in this chapter, but other typical names include: • • • • • • •

patient-carried medical record computerized medical record electronic medical record digital medical record patient medical record information personal health record electronic health record.



131

In different cultures and environments, all these terms carry a little different meaning, and it will surely take a long time before the terminology is established. The history of EPRs dates back to the 1960s (Waegemann, 2004): Some 40 years ago, the vision for electronic health record (EHR) systems was born. At the time, health informatics experts thought that the obvious benefits of the EHR would lead to adoption within a few years, but in reality, little happened. In 1991, the Institute of Medicine’s report on computer-based patient records created great interest and encouraged many companies to invest in new products, only to discover that healthcare wasn’t ready for the electronic age. Now, EHR systems are again at the top of the list for many organizations. EPRs offer the following advantages over traditional, paper-based documents (Hadley & Hutchings, 2001): • • • • • • • •

available in several places at once 24 hour access; remote from home self sorting; different views of data avoid loss of records (dependant on resilience) audit trail of document use capture legible information completeness of requests link to evidence-based medicine.

Atwater (1989) documented how the American College of Surgeons, already at the beginning of the 20th century, defined the need for proper patient records (as cited in Berg & Winthereik, 2004): Accurate, accessible, and complete written records must be kept for all patients and should include patient identification, complaints, personal and family history, history of present illness,


132 Suomi

physical examination, record of special examination such as consultations, clinical laboratory and x-ray results, provisional or working diagnosis, proposed medical or surgical therapy, gross and microscopic findings, progress notes, final diagnosis, condition on discharge, follow up and in case of death autopsy findings. In addition to medical purposes, patient records are needed for other purposes. The British GPH GPH (2003) defined the uses of patient records as follows: •

• •

Clinical purposes • Facilitate the clinical care of individual patients • Assist in the clinical care of the practice population Nonclinical purposes • Assist in meeting administrative, legal and contractual obligations Additional purposes • Connected to decision making (Keselman, Patel, et al., 2003), education, governance, professional appraisal, and so forth

EPRs are principally tools for healthcare professionals. Usually, it is thought that the patient data belong to the patient, and EPRs should be studied only with the consent of the patient. Another idea is that patients can study EPRs themselves. Pyper, Amery, et al. (2004) reported good findings in a pilot study that allowed patients to view their own information in EPR archives: Most patients found the computer technology used acceptable. The majority found viewing their record useful and understood most of the content, although medical terms and abbreviations required explanation. Patients were concerned about security and confidentiality, including potential exploitation of records. They wanted the facility to give informed consent regarding access and use of data. Many found errors, although most were not medically significant. Many expected more detail and more information. Patients wanted to add personal information.



133

As with any data, EPRs can be stored in different database configurations. Because of the fragmented nature of healthcare provision, EPRs are usually in a distributed form. This means that in any consultation situation, it is almost impossible to get a comprehensive overview of the patients records. Luckily, this is not always needed, and it can alleviate the information overload (Laskin, 1994) experienced by healthcare professionals. For accessibility, clearness, and overview purposes, central databases would be of use (Kilman & Forslund, 1997). The ideal of an integrated patient record–one record with all information on a patient–will appear to be at least a century old (Berg & Winthereik, 2004). Building such structures is difficult, both technically as well as politically. One solution could be that of smart cards, which could contain all the EPR contents of the patient. Such a solution would be extremely difficult as it comes to data security. However, any storage solution containing such sensitive data as EPRs is not without its problems.

EPRs in the Light of Customer-Relationship Management EPRs store data about healthcare customers, the patients. So one could expect that they follow many of the principles that hold true for CRM systems. Both disciplines, as it comes both to practice and scientific research, have a lot of common regarding time and popularity. Both customer records and patients records have been around for about 100 years, also since the birth of modern organizations. Attempts to computerize these records date back to the 1960s, and both topics have experienced a boom during the last 10 years. CRM or customer resource management can be defined as follows (searchCRM.com 2004): CRM (customer relationship management) is an information industry term for methodologies, software, and usually Internet capabilities that help an enterprise manage customer relationships in an organized way. For example, an enterprise might build a database about its customers that described relationships in sufficient detail so that management, salespeople, people providing


134 Suomi

service, and perhaps the customer directly could access information, match customer needs with product plans and offerings, remind customers of service requirements, know what other products a customer had purchased, and so forth. Another definition of the terms is as follows (Wikipedia, 2004): The generally accepted purpose of customer relationship management (CRM) is to enable organizations to better serve its customers through the introduction of reliable processes and procedures for interacting with those customers. A successful CRM strategy is usually implemented through a software package designed to support these processes. Major areas of CRM focus on service automated processes, personal information gathering and processing, and self-service. It attempts to integrate and automate the various customer serving processes within a company. In this part of this chapter, we base our discussion on the concepts summarized by Gibbert et al. (2002) and presented in Table 1. We go through the characteristics of the discipline and discuss how the situation should be interpreted in the case of EPRs. EPRs work on the idea that knowledge about the patient is collected from the entire care-taking chain’s being in contact with the patients and also from a network of organizations. This data are then implemented to a database. EPRs clearly have both characteristics from knowledge management and CRM as well. The dimension introduced in customer knowledge management, collecting and using the customers/patients own experiences, especially about the customer contact situation, is still in its infancy in the healthcare industry. With axioms, EPRs work very much on the principle “if we knew what they know.” Usually, a lot of patient data are inaccessible in a patient-encountering situation. The rationale of EPRs is close to that of unlocking and integrating healthcare professionals’ view of the patient. Gaining information directly from the customer becomes increasingly appreciated. Data mining about individual patients, however, does not often come into the foreground in the



135

Table 1. Knowledge management, CRM, and customer knowledge management (adapted from Gibbert et al., 2002) Knowledge management CRM

Customer Knowledge management

Knowledge sought Employee, team, company, Customer in network of companies database

Customer experience, creativity and (dis)satisfaction with products/services

Axioms

If only we knew what our customers know

Rationale

Objectives

Metrics

Benefits Recipient of incentives Role of customer Corporate role

If only we knew what we know

Retention is cheaper than acquisition Unlock and integrate Mining employees’ knowledge knowledge about about customers, sales the customer in processes and R&D company’s databases Efficiency gains, cost Customer base saving, and avoidance of nurturing, re-inventing the wheel maintaining company’s customer base Performance against budget Performance in terms of customer satisfaction and loyalty Customer satisfaction Customer retention Employee Customer

Gaining knowledge directly from the customers, as well as sharing and expanding this knowledge Collaboration with customers for joint value creation

Performance against competitors in innovation and growth, contribution to customer success Customer success, innovation, organizational learning Customer

Passive, recipient of product

Captive, tied to Active, partner in value-creation product/service by process loyalty schemes Encourage employees to Building lasting Emancipating customers from share their knowledge with relationships with passive recipients of products to their colleagues customers active co-creators of value

service-encountering situation because of time limits and not in other situations because of privacy issues (Van der Haak, Wolff, et al., 2003). The objectives of EPR originated from better patient care, also from creating joint value (health) with the customer (Berg & Winthereik, 2004). Here, EPRs have a more modern view of the patients than most CRM systems. The traditional objectives of efficiency and cost savings are there, too (Mitev & Kerkham, 1998), but not so clearly defined by the healthcare organizations: Medicine seems to be more important than economy in healthcare organizations. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

136 Suomi

The metrics on EPRs, if any, concentrate on customer satisfaction and contribution to customer success. Actually, the object of measurement is very clear: the health of the patient. However, as we know, the total health of any patient is a very difficult concept to measure and quantify to any metrics. The benefits of EPRs are mainly customer satisfaction and customer success, not the retention of customers. One could say that demand for healthcare services is negative (Kotler, 1991), and all stakeholders are better off the less the demand for EPRs, and that healthier people are without any specific measures. So, in a sense, long-term relationships and customer retention are not sought. This is not to say that healthcare professionals do not know that long-term patient relationships are more efficient and effective than fragmented consultations with patients; private healthcare organizations depend on longterm, satisfied customers. EPRs are rather passive systems. They are not expected to create any incentives for action in a straight sense, even though some such functions already exist. We have seen how many other computer systems have turned from passive to active over time (Abecker, Bernardi, et al., 1999); take data repositories as an example. One could expect in the future that EPRs will turn from passive systems to more active systems. Role of the customer is perceived as rather passive in the case of EPRs. This might, however, be changing, and we see the empowerment of the patients ( Beun 2003; Young, Eckman, et al., 1995). Here we delve into the area of medical discipline and its valuations. The corporate role, also that of the healthcare organization, is to encourage employees to share their knowledge with their colleagues. Building lasting relationships with the customer, as discussed earlier, is secondary, as well as the idea of customer emancipation, but these are not totally foreign concepts, even in healthcare. To summarize our discussion in this section, our basic idea of putting CRM and EPR systems on the same line turned out to be partly misleading. CRM literature focuses on competition, keeping and retaining customers, and building long-term customer relationships. These are mainly unknown issues to the healthcare organizations, especially in the public sector. However, there is a closer similarity to knowledge management and customer knowledge management. These disciplines stress more the added value for customer and customer satisfaction, which can be interpreted as better health in the case of healthcare customers, the patients.



137

The Internet Standards Adoption (ISA) Model The models on how innovations spread within different systems are many (Abrahamson & Rosenkopf, 1997; Brancheau & Wetherbe, 1990; Brown, 1981; Mahajan & Muller, 1979; Rogers 1995; Wynekoop, Senn, et al., 1992; Zuscovitch, Héraud, et al., 1988). User acceptance of the systems is a crucial point widely acknowledged in the literature (Davis, 1989, 1993; Davis, Bagozzi, et al., 1989; Dillon & Morris, 1996; Karahanna, 1993). Most of the aforementioned studies, however, study the diffusion of innovation within one organization. Our focus here is on how innovations spread within a network of organizations. A recent contribution to this literature is that of Hovav et al. (2004), who based their work very much on the classics of Fichman and Kemerer (1993). Hovav et al. studied the adoption of the IPv6 Internet standard in the international context. We feel that this process has a lot of common with the process of diffusing EPRs: •

• •

The adaptor is not just one single organization, but a network of organizations. The construction and implementation of new patient record systems should be seen as the building of a network, a process in which both medical practice and the record system are significantly changed (Berg & Winthereik, 2004). There is a working earlier alternative. Different organizational units can adapt the innovation at the pace they feel appropriate.

The ISA model defines two major groups of factors that have an effect on the diffusion of innovations: • •

Usefulness of the features (UF). Environmental conductiveness (EC).

Also, the innovation interplays with its intended use environment. A lot depends on the match between these two factors.


138 Suomi

Usefulness of the features (UF) has the following subcomponents in the model (Hovav et al., 2004): • • • •

Relative advantage: The value of new features offered by the features of a new standard over existing standards. Compability: The ability of the standard to work with existing technologies or infrastructure. Complexity: Increased complexity of the standard’s features increases the effort required to implement it. Observable benefits: Includes trialability (the ability to verify and quantify the benefits of the new standard), and observability (referring to the adopters ability to observe benefits from the adoption of a given feature or set of features).

Environmental conductiveness (EC) has the following parts (Hovav et al., 2004): •

• •

• • •

Network externalities: Positive network externalities are needed for successful innovation diffusion, and they are defined as to the benefits created through the adoption of the new standard by other organizations in the community. The concept is often credited to Katz and Shapiro (1986). Related technologies: The base of technologies related to and compatible with an innovation. Installed base/drag: The current infrastructure can be characterized by its installed base and by the resulting inertia and sunk cost to that installed base. Communication channels: The channels that are available for adopters to facilitate accessibility to information regarding the new standard. Sponsorship: Governmental or private support of the new standard. Resources: The ratio between the needed and available resources to implement an innovation.



139

The framework of Hovav et al. (2004) further defined two ways through which adoption of innovations might happen. The first is called adoption through replacement, in which new technology replaces the old systems but uses them in the same way as the old one without implementing all features of the new innovation at once. The second option is that of adoption through coexistence, in which case the new system is implemented with some new features in use, but not all. The old and the new system coexist for best use. In the replacement scenario the conductiveness of the environment is initially high, but the features of the innovation not very good. In the coexistence scenario, the innovation characteristics are very good, but the conductiveness of the environment is not very high initially.

Applying the ISA Model to EPRs In this section, we analyze EPRs in light of the ISA model. The idea is to see whether the ISA model could increase understanding of how EPRs diffuse in healthcare organizations. The relative advantage of EPRs over paper-based or nonexistent records is clearly admitted. The major advantage is location-independent access to the patient data, if the right technology and networking solution is provided. EPRs lower transaction costs of patient contacts and information management, especially those caused by information site specificity (Suomi, 1991). This is not to say that paper-based documents would not be a competitive solution in some aspects. For example, discussions about the data privacy and security of EPRs, as well as the sustainability of patient data in electronic form, are frequent issues in healthcare settings. Compability of EPRs exists with earlier paper-based solutions. Never is an EPR integrated overnight, unless in the case of a new healthcare institution. In established healthcare organizations, old paper-based systems and new EPRs have to coexist, maybe even for decades. Rather than compatibility with the old system, the integration of the various computer-based systems for different purposes is a problem. Implementing EPR is an extremely complex issue necessitating skilled management ( Berg, 1999; Mitev & Kerkham, 1998; Toussaint & Berg, 2000). This might be one of the major reasons why the use of EPR is not spreading very fast.


140 Suomi

Observable benefits are not much discussed in the case of EPRs. User satisfaction surveys regarding EPRs are scarce (Howcroft & Mitev, 1999). Very few hospitals seem to run any cost-benefit analyses on the issue (Giokas, 2001). To be fair, there are very few cost-benefit analyses of CRM systems in general. This is mainly because the operational and strategic benefits are very hard to measure. The issue of trialability, the possibility of trying out the innovation before the final decision has been made to adopt it, is difficult in the case of EPRs. Their benefits first materialize after years, and doctors’ first encounters with the systems are usually confusing. Efficient usage of EPR needs routine. However, we have seen cases in our own research where individual doctors have visited private hospitals with good EPRs in place, and as they come to work in the public sector, they demand similar resources there. Strong network externalities are available in the case of EPRs. First, the prices of solutions get lower as they get more standard and widely adopted. All stakeholders, hospital management, healthcare professionals, information system managers, and patients get more accustomed to the systems as they spread. Exchange of information between different institutions gets easier as there is more material in electronic form, and gateway and other integration solutions are easier to find. Computer-based technologies in hospitals must be counted in the group of related technologies. However it would be an exaggeration to say that they lead to the introduction of EPRs. Rather, EPRs are the central solution. EPRs pressure related technologies to mature. There is a strong installed base of established manual and paper-based patient records. However, their existence does not hinder the diffusion of EPRs; rather, their usage must be seen as a precondition for successful adoption of EPRs. Perhaps in the future we will see different generations of EPRs that fight against each other in the innovation cycle and for the market share, but not so far. However, we can already see some market-share competition between classical proprietary EPRs and lighter versions that have user interfaces through Internet browsers. Information on EPRs is richly available, and the communication channels to get information about EPRs should also work very well. Sponsorship of EPRs is too strong, even though the official supporting plans and strategies too often are missing. Implementing EPR systems is very resource consuming. The most scarce resources are time, money, and professional attendance, especially for medical



141

Table 2. Usefulness of features and environmental conductiveness in the case of EPRs (* = some, ** = reasonable, *** = strong, (-) refers to a negative correlation to the amoung of innovation diffusion.) Usefulness of features Relative advantage Compability Complexity (-) Observable benefits Environmental conductiveness Network externalities Related technologies Installed base (-) Communication channels Sponsorship Resources

*** ** *** * *** * * *** ** *

staff and specialized information system professionals. Small hospitals may have trouble finding the needed finances, as integrated EPRs are an expensive and extensive investment (Gapenski, Vogel, et al., 1993; Heinzl & Güttler, 2000). In summary, it is clear that the strongest supporting factors for the fast adoption of EPRs are their relative advantage, the strong network externalities available, and the rich availability of information on them through different communication channels. The strongest factors inhibiting adoption are the complexity of EPRs and the lack of observable benefits, especially trialability and resources to implement the systems (see Table 2). From the discussion and the information in Table 2, it is not clear which of the dimensions, the usefulness of features or environmental conductiveness factors, is stronger. Accordingly, it is hard to say whether adoption through replacement or adoption through coexistence is more likely, based on this analysis. However, as is seen through practical evidence, both manual and EPRs will coexist for a long time. So the practical innovation adoption for EPRs over paper-based patient records is that of adoption through coexistence.


142 Suomi

Conclusion We have studied how EPRs are diffused as innovations and how to be successfully used they must be genuinely accepted by the potential adopters. Our discussion showed that the idea of standardized patient records is very old, and that even EPRs have been around for some 40 years. However, the technology is far from settled, and we can not speak of any coherent innovation looking for diffusion. We liked to equate patient record management with CRM. On a closer study we found big differences between the two disciplines. CRM aims at longterm relationships and customer profitability, which are not strong goals for EPR systems. Rather, EPRs are closer to the concept of customer knowledge management, where customer value (health, in our case) is a key goal. The dissimilarity of the concepts must be seen as a possibility, the disciplines still have a lot to learn from each other. We further studied the factors that might support or inhibit the adoption of EPRs. The strongest supporting factors for the fast adoption of EPRs are their big relative advantage, the strong network externalities available and the rich availability of information on them through different communication channels. The strongest factors inhibiting adoption are complexity of EPRs and the lack of observable benefits, especially trialability, and resources to implement the systems. It is hard to say whether adoption through replacement or adoption through coexistence is more likely based on this analysis. In practice both manual and EPRs will coexist for a long time. So the practical innovation adoption bath for EPRs over paper-based patient records is that of adoption through coexistence.

References Abecker, A., Bernardi, A., et al. (1999). Enterprise information infrastructure for active, context-sensitive knowledge delivery. Proceedings of the Seventh European Conference on Information Systems, Copenhagen, Copenhagen Business School.



143

Abrahamson, E., & Rosenkopf, L. (1997). Social network effects on the extent of innovation diffusion: A computer simulation. Organization Science, 8(3), 289-309. Atwater, E. C. (1989). Women, surgeons, and worthy enterprise: The general hospital comes to upper New York State. In D. E. Long & J. Golden (Eds.), The American general hospital. Ithaca, NY: Cornell University Press. Berg, M. (1999). Patient care information systems and healthcare work: A sociotechnical approach. International Journal of Medical Informatics, 55, 87-101. Berg, M., & Winthereik, B. R. (2004). Waiting for Godot: Episodes from the history of patient records. In M. Berg (Ed.), Health information management. London: Routledge. Beun, J. G. (2003). Electronic healthcare record; a way to empower the patient. International Journal of Medical Informatics, 69, 191-196. Brancheau, J. C., & Wetherbe, J. C. (1990). The adoption of spreadsheet software: Testing innovation diffusion theory in the context of end-user computing. Information Systems Research, 1, 115-143. Brown, L. A. (1981). Innovation diffusion: A new perspective. London: Methuen. Davis, F., Bagozzi, F., et al. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003. Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), 319340. Davis, F. D. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38, 475-487. Dillon, A., & Morris, M. G. (1996). User acceptance of information technology: Theories and models. Journal of the American Society for Information Science, 31, 3-32. Douglas, T. J., & Ryman, J. A. (2003). Understanding competitive advantage in the general hospital industry: Evaluating strategic competencies. Strategic Management Journal, 24, 333-347.


144 Suomi

Fichman, R. G., & Kemerer, C. F. (1993, Winter). Adoption of software engineering process innovations: The case of object orientation. Sloan Management Review, 7-22. Gapenski, L. C., Vogel, W. B., et al. (1993). The determinants of hospital profitability. Hospital & Health Services Administration, 38(1), 6380. Gibbert, M., Leibold, M., et al. (2002). Five styles of customer knowledge management, and how smart companies use them to create value. European Management Journal, 20(5), 459-469. Giokas, D. I. (2001). Greek hospitals: How well their resources are used. Omega—The International Journal of Management Sciences, 29, 73-83. GPH. (2003). Good practice guidelines for GP electronic patient records (Version 3). London: General Practitioner Committee, United Kingdom Department of Health. Hadley, A., & Hutchings, C. (2001). 1.25 million electronic patient records in XML at Poole. XML Europe 2001, Berlin, Germany. Heinzl, A., & Güttler, W. (2000). IT induced healthcare reconfiguration: German hospitals in transition. Proceedings of the Eighth European Conference on Information Systems, Vienna, Austria. Hovav, A., Patnayakuni, R., et al. (2004). A model of Internet standards adoption: The case of IPv6. Information Systems Journal, 14(3), 265294. Howcroft, D., & Mitev, N. N. (1999). Doctors on the Net: are GPS ready for the information superhighway? Proceedings of the Seventh European Conference on Information Systems, Copenhagen, Denmark. Karahanna, E. (1993). Evaluative criteria and user acceptance of enduser information technology: A study of end user cognitive and affective processes. Minneapolis, MN: University of Minnesota. Katz, M. L., & Shapiro, A. (1986). Technology adaption in the presence of network externalities. Journal of Political Economics, 94, 822-841. Keselman, A., Patel, V. L., et al. (2003). Institutional decision-making to select patient care devices: Identifying venues to promote patient safety. Journal of Biomedical Informatics, 36, 31-44.



145

Kilman, D. G., & Forslund, D. W. (1997). An international collaboratory based on virtual patient records. Communications of the ACM, 40(8), 111-117. Kotler, P. (1991). Marketing management analysis, planning, and contro. Englewood Cliffs, NJ: Prentice Hall. Laskin, D. (1994). Dealing with information overload. Journal of Oral Maxillofacial Surgery, 47(7), 661. Lincoln, T. L., E. D. J., et al. (1993). The electronic medical record: A challenge for computer science to develop clinically and socially relevant computer systems to coordinate information for patient care and analysis. The Information Society, 9. Linna, M. (2000). Healthcare financing reform and the productivity change in Finnish hospitals. Journal of Healthcare Finance, 26(3), 83-100. Mahajan, V., & Muller, E. (1979, Fall). Innovation diffusion and new product growth models in marketing. Journal of Marketing, 43, 55-68. Mitev, N. N., & Kerkham, S. (1998). Less haste more speed: Organisational and implementation issues of patient data management systems in an intensive care unit. Proceedings of the Sixth European Conference on Information Systems, Aix-en-Provence, France. Pyper, C., Amery, J.,et al. (2004). Patients’ experiences when accessing their on-line electronic patient records in primary care. British Journal of General Practitioners, 54(498), 38-43. Rogers, E. M. (1995). Diffusion of innovations. New York: The Free Press. Rogerson, S. (2000). Electronic patient records. IMIS Journal, 10(5). SearchCRM.com. (2004). SearchCRM.com definitions. Suomi, R. (1991). Removing transaction costs with inter-organizational information systems. Information and Software Technology, Special Issue on Information and Software Economics, 33(3), 205-211. Toussaint, P., & Berg, M. (2000). The Electronic patient record as an organisational artefact. Proceedings of the Eighth European Conference on Information Systems, Vienna, Austria. Waegemann, C. P. (2004). The year of the EHR? Health Management Technology, 20(4). Van der Haak, M., Wolff, A. C., et al. (2003). Data security and protection


146 Suomi

in cross-institutional electronic patient records. International Journal of Medical Informatics, 70, 117-130. Wikipedia. (2004). Wikipedia: The free encyclopedia. Retrieved from http:/ /en.wikipedia.org/wiki/Customer_relationship_management Winter, A. F., Ammenwerth, E., et al. (2001). Strategic information management plans: The basis for systematic information management in hospitals. International Journal of Medical Informatics, 64, 99-109. Wynekoop, J. L., Senn, J. A., et al. (1992). The implementation of CASE tools: An innovation diffusion approach. In K. E. Kendall (Ed.), The impact of computer supported technologies on information systems development. Elsevier Science. Young, M. J., Eckman, P., et al. (1995). Patients, physicians, and professional knowledge: Implications for CQI. Hospital & Health Services Administration, 40(1), 40-49. Zuscovitch, E., Héraud, J.-A., et al. (1988). Innovation diffusion from a qualitative standpoint. Futures, 20(3), 266-306.


USE IT: The Theoretical Framework 147

Chapter IX

USE IT:

The Theoretical Framework Tested on an Electronic Prescription System for General Practitioners Ton A. M. Spil University of Twente, The Netherlands Roel W. Schuring University of Twente, The Netherlands Margreet B. Michel-Verkerke University of Twente, The Netherlands

Abstract This chapter is the heart of the book and reflects five years of research on diffusion of e-health systems. The resulting USE IT model has four determinants that have to be balanced in assessing the diffusion and use of information systems. Resistance is defined as the degree to which the surroundings and locality negatively influences the users of IT and the degree to which IT-users themselves are opposing or postponing the IT change. The result of this study was that opposite to what was hypothesized,


148 Spil, Schuring & Michel-Verkerke

resistance was not the main determinant of success. It is a cumulative consequence of effects of the other three determinants with a little touch (initial attitude) of it’s own. Relevance is the degree to which the user expects that the IT-system will solve his problems or help to realize his actually relevant goals. Micro-relevance is the degree to which IT-use helps to solve the here-and-now problem of the user in his working process. The (job) relevance of the EPS to the working process of the professional was in all 56 cases the most important determinant. Requirements are defined as the degree to which the user needs are satisfied with the product quality of the innovation. Analysis of this determinant showed that a user-provider contract on requirements would help bridging the information gap between user and designer of IT in healthcare. Resources are defined as the degree to which material and immaterial goods are available to design, operate and maintain the information system. The presumption was made that this would be the least important determinant but the study did not confirm this. It showed that a thorough ex ante check on resources is still necessary.

Introduction In this contribution we will try to unravel the complexityof implementing information systems in healthcare (Berg, 2001; Southon et al., 1999; Walley & Davies, 2001). We look at successful change management from a user perspective. Determinants on other levels may also be important, such the level project, the level of the organization, or even the level of the system (Schuring & Spil, 2004). On the system level, reimbursement structures, regulations and the existence of standards may have an explanatory role. On the organization level, the previous strategic choices, strategic priorities, size, and location of the organization and many other factors may play a role. On the project level, resources, project management, and so forth will play a role. No explicit attention on these factors will be given in this contribution. Thornett (2001) described benefits as improved quality of care, disease prevention, and disease management of chronic physical illnesses. Why, then, do these systems not diffuse into the health organizations? The adoption of IT in healthcare has increased, which underscores the importance of user requirements (Beuscart-Zephir, Brender, Beuscart, & Menager-Sepriester, 1997). In



later work, Beuscart-Zephir linked the adoption to the activities of the healthcare professionals (Beuscart-Zephir, Anceax, Crinquette, & Renard, 2001). Fleisner and Hofkircher (1998) referred to the same problem when they concluded that relevant information will not be improved unless additional requirements are met. A multiple case study amongst 56 GPs on the influence of resistance, relevance, requirements, and resources on the introduction of an EPS demonstrates that the EPS is not used in at least 72% of the cases. First, a broad background of the model is given. To explain the nonuse of the system, we combine the notions of information usage of Delone and McLean (1992) and Davis (1989) and the notion of innovation from Rogers (1983, 1995). We use the semantic ladder from Stamper (1973) and information levels from Shannon and Weaver (1949) to straighten them out (social, pragmatic, semantic, and syntactic levels). To explain relevance, we use the notions of Saracevic (1975). This will be described in the definition and framework section. Together these concepts build up a framework for an interview model that we used in all cases, as described in the case study method just before the empirical results. Finally, we make conclusions for every determinant of the model.

Background We can use a wide range of sources that discuss user perspectives in IT introduction. This section gives a short overview of intriguing literature. The aim is to demonstrate that the four determinants explained separately in the next section should be handled as a whole, dynamic, interrelated set of quality criteria. One of the ultimate goals of our research project in this field is to propose a model that neatly balances the role of such factors. First, such factors may be looked for in general literature on change and on the introduction of new technologies. For example, in the balance model of organizational change risks, Leavitt (1965) introduced four domains in which these risks will occur: tasks, structure, technology, and people. Offenbeek and Koopman (1996) connect people with resistance potential because they can feel that the quality of their working life will be decreased. Mumford (1995) observed that user participation contributes to effective organizational change. Wissema (1987) defined resistance as willingness to change and the difference between results and expectations. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


When we focus on IT introduction more specifically, we again see a number of interesting literature sources. Thong and Yap (1995) discussed the usersatisfaction approach to IT effectiveness. They mentioned the debatable operationalization, poor theoretical construct, and misapplication as a result of the approach. On the basis of their review, they concluded that attitude is the construct that lies at the root of user satisfaction and suggest ways to improve operationalization and measurement of attitude. Paré and Elam (1999) studied attitudes, expectations, and skills in relation to physicians’ acceptance of IT systems. Physicians with formal training on computers were more knowledgeable about informatics concepts and reported that computers would be more beneficial to healthcare, although it is not clear whether the training causes this attitude. Also, it becomes clear that user priorities regarding IT innovations vary strongly. The functional uncertainty is often described in IS literature, and it occurred in the task domain of Leavitt (1965). In each situation, the interpretation and the meaning can be different. Therefore, it is necessary to establish a functional specification with user and providers of the information systems. Henry and Stone (1999) stated this to be information quality. Larsen (1998) noted, however, that “the quality of the IS/IT product is a necessary but not sufficient prerequisite for IS innovation success. The people within the organizations determine the outcome” (p. 413). Within the healthcare sector, Walley and Davies (2001) conducted a study to the internal barriers to IT advancement in the healthcare sector. The involvement of stakeholders is arguably one of the most distinctive characteristics of IT projects. There are instruments to identify user needs, but Walley and Davies (2001) questioned whether they are actually used. Van der Pijl (1994) showed that there is more to say about people than just resistance or user participation. Both users and providers of information systems have their own targets, which do not necessarily go hand in hand. A central question is whether the provider intention is the same as the user interpretation (Sperber & Wilson, 1986). Finally, resources (human, physical, and monetary; Ansoff, 1965) are needed to implement the new information system into the organization. The human resources can be insufficient in time and in experience (risk of technology). Insufficient material resources (Offenbeek & Koopman, 1996) will have a limiting influence on the other three risk domains. In this chapter we will focus on all four determinants of user adoption of IT in healthcare (i.e., resistance, relevance, requirements, and resources). It is most important to elaborate the construction of a framework that brings these factors



Table 1. USE IT model USE IT model Product Process

User Domain Relevance Resistance

Information Technology Domain Requirements Resources

together. Saarinen and Sääksjärvi (1995) pointed out that different factors act as critical success factors under different circumstances and make a distinction in (implementation) process factors and (innovation) product factors. This will also apply when a framework of success factors is limited to user-related factors. Table 1 gives an overview of the USE IT model.

USE IT: Definition and Framework Resistance The tendency of human beings to resist and fear new and unknown things and the willingness to stick to the familiar procedures has been studied widely (e.g., Goodstein & Burke 1991; Lanning, 2001). Attributing the rejection of innovations only to anxiety and fear of change, however, is an oversimplified view of the process of technology transfer (Raghavan & Chand, 1989). Carey (1988) found a correlation between acceptance of change and variables such as previous use (experience), education, and current usage of a new system. She also reported commitment, exposure to change, and preparation for change as being important for successful implementation of new technologies and systems. So, a much broader view on the subject of resistance is appropriate. We position it on the social level of the semantic ladder (Stamper, 1973). We start with the first known published reference to research on resistance to change in organizations by Coch and French (1947). They were early explorers in the world of resistance when they concluded, “by preventing or greatly modifying group resistance to change, this concomitant to change may well be greatly reduced”. Besides taking note of resistance influencing successful change, they stated that this resistance can be different on the Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


group and individual levels. Later change-management literature categorizes resistance into the individual, group, and organizational (structure) levels. On the group level, Lewin (1952) referred to “group standards” when looking for reasons for resistance to change. He concluded that the more individuals hold group standards to their environment, the greater will be the resistance to change of an individual group member. Lewin continued that group standards with social values are often referred to as “social habits.” Thus, the way to reduce the level of resistance may be either to diminish the strength of the value of the group standard or to change social habits. Both Lawrence (1954) and Zuboff (1982) concluded that resistance is not simply an irrational phenomenon to be overcome (Malinconico, 1983). Zuboff saw positive and negative aspects to resistance. More authors described the healing effect of resistance (Binney & Williams, 1995; Folger, Skarlicki, & Tesluk, 1999; Piderit, 2000). Insightful and wellintended debate, criticism, or disagreement do not necessarily equate to negative resistance, but rather may be intended to produce better understanding as well as additional options and solutions. Rogers (1995) also explained that it is perfectly rational to later adopter categories to be more hesitant about the introduction of an innovation; they do not have sufficient resources to overcome the consequences of an adoption failure. Kotter and Schlesinger (1979) diagnosed resistance from the negative viewpoint as follows: • •

Parochial self-interest (fear to loose something worthwhile; Wissema, 1987) Misunderstanding and lack of trust;

• •

Different assessments (believe that change is worthless; Wissema, 1987) Low tolerance for change

Fuller (1969) also discovered different levels of concern regarding resistance. Self-concern can be seen in awareness, personal commitment, and personal consequences. Task concern is related to controlling the change, and cooperation concern oversees both concerns in collaboration and reengineering. Schmidt, Lyytinen, Keil, and Cule (2001), whose study was a mirror of control



for our study, referred to Zmud (1979), who adds a fourth environmental category. Lapointe, Lamothe, and Fortin (2002) applied these categories in explaining the dynamics of IT adoption in healthcare. She based her theory on that of reasoned action, where “individual behavior is directly determined by one variable intention which, in turn, is determined by two variables: attitude and subjective norms.” In line with Scott (1985), Prasad and Prasad (2000) made the distinction between formal and informal (routine) resistance. Strebel (1996) already described this as personal compact, formally a job description and appraisal but informally psychological (mutual expectations) and socially (cultural values). This brings us back to the notion that change is intensely personal (Duck, 1993) and therefore our empirical material is gathered very close to professional coping with the change. Kotter and Schlesinger (1979) are mainly interested in the self concern. Mittelstaedt, Grossbart, Curtis, and Devere (1976) add the inability of either individual or group to cope with the change. Also the situation can call for postponement. The situational factors we see as mainly emerging on the organizational level: 1. 2. 3.

Not for them (reject) Unwilling or unable (accept) Postpone (time and situation; accept)

Gatignon and Robertson (1989) and Szmigin and Foxall (1998) sort like distinctions, the latter introduce opposition instead of unwilling or unable. Ram and Sheth (1987) call this habit resistance and also relate to Rogers (1983) when they stated that often, an initial resistance has to be overcome. (Please note that most of the literature in this paragraph is based on resistance of consumers. It might not always apply in a healthcare environment.) Offenbeek and Koopman (1996) introduced the resistance potential and made a distinction between changeability of the problem system and desired change. This potential of resistance would be people-determined resistance, according to Markus (1983). System determined resistance is handled in the technical determinant of IS success, but the interaction determined resistance, which is mainly political, inter-organizational resistance, can only be seen within the reasons to postpone of Mittelstaedt et al. (1976).



Other publications on the subject of resistance challenge enhance the “accepted” concepts of resistance (Dent & Goldberg, 1999). Piderit (2000) suggested that resistance to change is a complex, multidimensional response with emotional, cognitive, and intentional components. For example, no participation or not enough communication may result in an emotionally resistant attitude to the changes, even though the changes make good business sense (cognitive). Alternatively, initially enthusiastic (emotional) and clearly seeing the need for change (cognitive), people give up (i.e., our intentions change) because they are not given the support they expect and think they will need in order to make the changes happen. It is rare that employees are all negative or all positive across the three dimensions. It is important to remember that resistance to change is normal and frequently functional. “Moving too quickly toward congruent positive attitudes toward a proposed change might cut off the discussion and improvisation that may be necessary for revising the initial change proposal in an adaptive manner.” In other words, discussion, disagreement, and experimentation consistently can lead to more successful change, whereas effective communication and participation are powerful tools for overcoming and avoiding misunderstandings (Binney & Williams, 1995). Zaltman and Duncan’s (1977) resistance framework discusses four categories of barriers-cultural, social, organizational, and psychological-that can obstruct change. These categories are in turn broken down into a total of 18 resistance factors, which disrupt change efforts and distort adopter perceptions of innovations. This framework can be useful because it explores change from the opposite perspective to most other models. By focusing attention on factors that erect barriers to change, Zaltman and Duncan help to recognize such obstacles as they arise or even to identify and address their underlying issues before they arise. It is important to note that a given individual can harbor intense prochange and pro-resistance sentiments simultaneously. Though detailed in its 18 factors, this framework is not suitable for the goal of this study. More appropriate is a model with its roots in change management, educational and training literature, structuring resistance and affection into three categories: ability, attitude, and opportunity. Metselaar et al. (1996) described this as used in training against (negative) and with (positive) resistance based on a concept from the social psychology (Ajzen & Madden, 1986). Lanning (2001) came up with the same result in an empirical



study for a planned change approach. We adopt these findings into our “USE framework for resistance to IT change” where we should reckon that the main focus of these shifts from (inter) organizational, to group, to individual. In line with the USE IT model (Spil, Schuring, & Katsma, 2002), this subdivision is made into macro and micro resistance (see Figure 1). Attitude (Will) to Change (Micro Resistance) People who are expected to participate in the change project must have personal motivation and a sincere will to engage themselves in the development. Comprehension and acceptance of the basic idea in the project is an important condition. Will does not occur unless real effort at developing the organization can be perceived. Ability to Change (Macro Resistance) The level of knowledge and skills of those who are involved in a “change” project needs to be high enough to be able to contribute to the project. Jobspecific skills enabling people to use new tools and technology and to act according to new procedures and tasks must be adequate. But ability also means comprehension of project vision and understanding one’s own role in implementing the new technology. The user experience also adds up to his or her ability to change. Opportunity to Change (Macro Resistance) There is need for organizational systems (surrounding people and structures) to support the development process and implementation of the new technology. Sufficient resources, top management support, and commitment are essential to giving everybody the feeling that change and development can be achieved. These sub dimensions of resistance fit underneath the user satisfaction research model of Mahmood, Burn, Gemoets, and Jacquez (2000) as they call it user background and organizational support. The perceived benefits that complete the user satisfaction are in the USE IT model situated under the



relevance determinant (Schuring & Spil, 2003) and described in the next section.

Relevance Saracevic (1975) defined relevance as a measure of the effectiveness of a contact between a source and a destination in a communication process. This is a somewhat abstract wording of what we would call the degree to which the user expects that the IT system will solve his or her problems or help to realize the actually relevant goals. There are three dimensions that are kept implicit in Saracevic’ definition that we wish to stress. We use the word “expects” since we want to make more explicit that relevance is a factor that is important in the course of the adoption process, not only in evaluation. Second, instead of effectiveness we use “solve problems and goals.” By doing so, we imply that effectiveness has two dimensions: to take away existing negative consequences (problems) and to reward with positive consequences (reach goals). Third, the word actual is crucial in our view of relevance, as stated in the introduction. Relevance is not to be confused with the degree to which the user considers outcomes as being positive. The set of outcome dimensions that someone considers “positive” is larger bigger than the set of outcome dimensions that are relevant. Imagine a physician, who basically considers IT outcomes of a computer decision support system, such as assistance in diagnosis, disease prevention, or more appropriate dosing of drugs (Thornett, 2001), as “positive.” This does not automatically imply that the IT adoption is relevant. It is only relevant if these outcomes are high on the goal agenda. That is why we use the word actually. Again, this is a more explicit wording of a dimension that is implicitly included where Saracevic’ uses the word effectiveness in his definition. The actually relevant goals may be a mix of short-term goals and long-term goals. If, for example, smooth communication with hospitals or pharmacy is the prime actual problem or goal, he or she will only consider the IT innovation as relevant when it actually helps to improve that communication, notwithstanding the fact that he or she might have a positive attitude toward that innovation, as long as the innovation helps to solve other problems or other goals that are on the lower positions in the agenda ranking. We discovered in our case studies that it is not sufficient for an innovation to effectuate a positive attitude amongst users. The IT innovation should be relevant (Schuring & Spil, 2003).



Micro-relevance is a related concept that can be used to describe a similar phenomenon once the new IT is installed. Micro-relevance is defined as “the degree to which IT-use helps to solve the here-and-now problem of the user in his working process.” The use of new equipment or new IT procedures is a conscious activity. In every conscious activity that is oriented to a specific goal, there is a reason why that course of action is being chosen. Similar to what was discussed about “relevance,” not every course of action that a user basically considers as “positive” is “micro-relevant.” Again, let us illustrate this with an example. Imagine a patient with a virus infection visits a physician. The physician might notice the similarity to a number of other patients he has met that week and decide on diagnosis and treatment fairly quickly. To this doctor, the use of a decision support system to determine diagnosis is not micro-relevant. However, a colleague of his may not feel so confident and thus use the system. We discovered that micro-relevance is a key factor in explaining IT use in our case studies. Figure 1 gives an overview of relevance as we propose to use it. Relevance and micro-relevance are notable refinements of the way the role of the user is being discussed in the existing literature. Thornett (2001) implicitly referred to relevance and micro-relevance when he discusses limited adoption and use of DSS by primary physicians where “consultation time is lengthened by their use and there is no appreciable impact on patient satisfaction”. It is an example where other outcomes that are basically considered as positive (as mentioned previously: better diagnosis, more appropriate dosing of drugs, etc.) are overruled by limited relevance and micro-relevance. Saracevic (1975) provided an historic positioning of relevance. The roots lay in the 1930s and 1940s, where the distinction between information and relevant information is made by Bradford (Saracevic, 1975). In order to make the distinction between relevant and nonrelevant information, he discussed the nature of communication. By doing so, he recognized that relevance to a subject depends on specific dimensions, like, for example, the subject’s knowledge, representation, and values. He discussed a number of (philosophical) approaches to relevance. The elaboration we propose builds on the radical pragmatism-perspective or, more specifically, Cooper’s (1971) utility function: “Relevance is simply a cover term of whatever the user finds to be of value about the system output, whatever its usefulness, its entertainment, or aesthetic value, or anything else”. Wilson (1973) adds to this that relevance is situational. Ballantine et al. (1999) put it in the following way: Depending on the type of task, the information generated by the system may be more or less appropriate, which will affect its success or failure”. Saracevic (1975) distinguished various



other approaches to relevance, of which a number focuses on the basic source of relevance, like, logical relevance, the nature of interference and the pertinent view of relevance. We are very much aware of the fact that our elaboration of relevance does not in full retain the differences between those points of views. It is merely a practical elaboration that we use to predict user adoption. The pragmatic perspective (Iivari & Koskela, 1987) of relevance that we choose resembles the notion of “relative advantage” as discussed in the Innovation Diffusion literature by Rogers (1983, 1997). Rogers reserved a central role for “relative advantage,” which is the user’s view of “the degree to which an innovation is better than the idea it supersedes”. Relative advantage can be economic or social. “The nature of the innovation largely determines what specific type of relative advantage is important to adopters, although the characteristics of the potential adopter also affect which dimensions of relative advantage are most important” (Rogers). Based on a review of hundreds of empirical studies, Rogers concluded that relative advantage explains 49% of the rate of adoption of innovations. It is most notable that the organizational factors are not explicitly included in our user-relevance framework. It should be kept in mind that user’s agenda of problems and goals depend of his role in the society (Barnard, 1938) The influence of the organization on this agenda depends on many aspects, including the involvement with other organizations, on time and on place. As a consequence, our framework reflects the actual impact that organizational goals and preferences have on the user, and thus, on organizational behavior.

Requirements At the semantic level (DeLone & McLean, 1993; Shannon & Weaver, 1949; Stamper, 1973) we are concerned with how pattern types relate to what happens in the world. On this level we deal with the meaning of the system but this term brings along a lot of different meanings about its definition (Cohen, 1962). The meaning of a sign relates to the response the sign elicits in a given social setting (Liu, 1993). It is situational of nature since we have a range of pattern types that signify a certain meaning and a user (group) that interprets the expression (Spil, 1993). Therefore, it is necessary to establish requirements as thorough as possible. Wieringa (2001) defined requirements as desired properties needed to achieve the desired composite system properties. Pressman (1982) made a distinction between normal



Figure 1. USE Framework for resistance and relevance Macro Resistance

Macro Relevance

Opportunity

Macro Relevance

USE Framework for resistance to and relevance of IT change

Ability

Definition: the degree to which the surroundings and locality negatively influences the users of IT à (Co)determines: IT-diffusion Generic sub-dimensions: Opportunity to change is the degree in which the users are forced or allowed to change

+

Budget available, clear objectives, top management support, social improvement

-

Decrease of autonomy, local effort for general gain, remaining old structures

Ability to change is the change potential of the workers and the management

+

-

Training, education, experience and enough resources Constraints beyond the scope of the user that prevent him from using the IT

Definition: degree to which the user expects the that the IT-system will solve his problems or help to realize his actually relevant goals (Co)determines: IT-diffusion à

1 Economic improvements 2 Social improvements 3 Functional improvements 4 Saving of time and effort

Micro Resistance = Attitude

Micro relevance

1 Parochial self-interest

1 Absolute value of relevance

2 Misunderstanding or lack of trust

2 Here and now value

3 Different assessments

3 Low initial costs

4 Low tolerance of change

4 Immediacy of the reward

Definition: the degree to which IT-users themselves are opposing or postponing the IT change

Definition: the degree to which IT-use helps to solve the here-and-now problem of the user in his working process

requirements, expected requirements, and exiting requirements. Before defining requirements ourselves, we want to study the problem at a deeper level. “Many system designers do not appear to realize that with their present approach they are designing only partial systems” (Mumford, 1995). She argues that all needs of the end users should be identified. The notion of variance emerged from some early sociotechnical work design experiments in Norway (Mumford, 1983). A variance is defined as a tendency for a system or subsystem to deviate from some desired or expected norm or standard.



Key variances are the deviations on goals and functions, operational variances stem from the organizational problems. Together they get close to the main problem that we are addressing, the information gap between designer and user. Iivari and Koskela (1987) include three quality constructs on the semantic level that they call the input-output requirements: informativeness, accessibility, and adaptability. Informativeness describes the potentiality of the information systems, accessibility the quality of the user-IS interaction and adaptability points to the ability of the systems to change. DeLone and McLean (1992) enumerate the criteria for IS success from nine earlier studies. They declare themselves that there is not “one” measure of IS success but there are many dependent variables. They call their taxonomy on semantic level information quality. Usefulness or relevance is mentioned eight times in the nine studies. Schuring and Spil (2003) have studied the importance of relevance and made it a separate determinant on the pragmatic level. Timeliness is empirically used five times and adopted in our model. We keep using the term accessibility broadly, which includes convenience of access. Accuracy is studied four times and adopted under informativeness. We do not understand why there is no notion of adaptability or ability to integrate in the DeLone and McLean (1992) study. We adopt ability to integrate as the degree that the new system is imbedded in the organization. Brender and McNair (2001) used the ISO 900x structure and used the strategic, tactical, and operational level to perform their user requirements specification. Larsen (1998) also made this distinction. The strategic level is concerned with the problem definition, including objectives and global task description. The tactical level is interpreted as a preferred approach and the operational level includes a set of functional, performance and capacity criteria. The requirements determinant is defined as the degree to which the user needs are satisfied with the product quality of the innovation. We divide the requirements into macro and micro requirements (see Figure 2): • •

Strategic general requirements and tactical approach is the degree in which the users agree with the objectives and methods used. Functional requirements and performance requirements specify what the content of the innovation should be. In this study we chose timeliness



Figure 2. IT Framework for requirements and resources determinants IT Framework for requirements and resources

Micro

Macro

à

(Co)determines: IT diffusion Generic sub dimensions: Strategic general requirements and tactical approach is the degree in which the users agree with the objectives and methods used.

+

Clear objectives, iterative approach, users involved.

-

Unclear communication, no participation, education

Functional requirements and performance requirements specify what the content of the innovation should be.

+

-

Timeliness, Accurateness, Ability to integrate, Content Fuzziness, non contract

Resources

Definition: the degree to which material and immaterial goods are available to design, operate and maintain the system. à

Material

Definition: the degree to which the user needs are satisfied with the product quality of the innovation.

Immaterial

Requirements

(Co)determines: IT Use Generic sub dimensions:

1 Costs

2

Hardware and software

3

User and designers time

4

Adaptibility

5

Capabilities

6

Reliability

(accessibility), accurateness (informativenes), ability to integrate and content as main quality criteria but we acknowledge that this is specific for this study and not a complete list.

Resources Under the semantic level most researchers situate the syntactical level (Iivari & Koskela, 1987). They give efficiency criteria to measure the quality of the information system on this level (design costs, operations costs, and maintenance costs). Shannon and Weaver (1949) called it level A, the technical problem, and Stamper (1973) divided it into three levels (syntactic, empirical, and physical). Main quality criteria on these levels are formal specification, reliability and costs. The resources determinant is defined as the degree to which material and immaterial goods are available to design, operate and maintain the information system. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


The design costs can mainly be seen as time and capability of users and designers (Salmela, 1997) but also the size of the project and the complexity of the problem could be measured to assess the risk of the innovation design. Also hardware and software costs fall under this header. Formal specification on syntactic level can be checked on semantic level with the quality criterion accurateness (data, system, and information). The operations costs are mainly human resources but the abnormal costs can be derived from the reliability of the system (Iivari & Koskela, 1987). The maintenance costs can be shown with the quality criteria adaptability and portability that also link to the ability to integrate on a higher level.

Multiple Case Studies Results Case Study Method Nykänen (2000) distinguished four major evaluation perspectives: goal-oriented, standardized, effectiveness-based and stakeholder-based perspectives. In the goal-oriented evaluation, the emphasis is on rationality: measurement criteria and means to achieve the goal can be derived from the goal itself. This is possible if the criteria are clear and there are no conflicts of interests among the stakeholders. The downside of goal-oriented perspective is the inability to see other than the anticipated consequences of actions. In the standardized (or normative) evaluation, causes and consequences are not in the scope of interest, but compliance with rules, agreements, budgets, and principles are monitored (e.g., quality systems). In effectiveness-based view, the input–output ratio of actions is economically evaluated. The problem with this perspective is in expressing intangibles (e.g., health) in monetary terms. According to the stakeholder-based perspective, all actions are not always rational; aiming at one mutual goal, and therefore the criteria should be collected from several stakeholders’ view. The perspective has a lot of qualitative characteristics and it can be a quite laborious framework for a study design (Hakkinen, Turunen, & Spil, 2003). This study used the stakeholder-based perspective and was set up to both assess the situation regarding the EPS in the Netherlands and the theory that is described previously, that was set up to provide an instrument that could be



used to analyze the diffusion situation of the prescription system. This resulted in a case-study protocol that covers all the topics that are mentioned in the framework in open-ended questions. In line with the case-study approach by Yin (1984) we discerned different case situations on the basis of our theoretical framework. Particularly, the network situation (individual, group practice, healthcare center) of GPs and the degree of adoption of previous ideas (laggard; no computer) to innovator (using ICPC codes and electronic patient record) served as a basis to make categories of GPs. A total of 56 case studies were conducted. Each GP was visited in his/her own working situation and interviewed for over an hour. We agree with Brender (1999) that the kernel point of assessment is that of understanding the business processes.

Empirical Results GP Electronic Prescription System Empirical Resistance of GP The main problem formulation for this study was to find the obstacles of implementation of the EPS. Under the header resistance of the GP, questions were asked about problems or wishes that the GP experienced as important at the moment of asking, during implementation of the EPS. Figure 1 gives an overview of all the situational resistance factors mentioned. Here we will summarize the results of the main five factors: • • • • •

Time 55% User interface 33% Free choice 30% ICPC 27% Unwilling 20%

Fifty-five percent of the GPs said to be under immense time pressure. We think that this made the EPS less relevant to the GPs (Schuring & Spil, 2002), but it also levered the resistance because the GPs thought they would need more



time for a consult using the EPS instead of less time. These statements are confirmed by British research (Sullivan & Mitchell, 1995). Thirty-three percent of the GPs had problems with the quality of the software. Main problems were not related to the new EPS but more to the old GP IS that was not able to give a good user interface. The EPS we described in Lagendijk, Schuring, and Spil (2001) as “spoilers on a T-ford”. Thirty percent of the GPs say they do not want the law to be written by a new system and want to maintain free choice in prescribing drugs to their patients. Some say the advice are too conservative, others say they want to be able to try out new ideas. All of them say they want freedom of choice. Twenty-seven percent of the GPs think that ICPC is a problem when using the EPS. Some state that it is rather difficult to find a related ICPC to the diagnosed disease. Other state that it is not necessary to use ICPC for general diseases like flue because it costs time and it does not help the process. Twenty percent of the GPs are unwilling to use the EPS. That means that they have not looked at it and will not look at it just because they do not like the change. Here we see clearly a low tolerance of change.

Empirical Relevance to the GP Under the header relevance for the GP, questions were asked about problems or wishes that the GP experienced as important at the moment of asking, during implementation of the EPS. Figure 3 gives an overview of all the situational relevance factors mentioned. Here we will summarize the results of the main seven factors: 1. 2. 3. 4. 5. 6. 7.

Communication Time Money Software Free choice ICPC Formulary



Figure 3. Multiple case study results on relevance

In more than half of the case studies, the GPs said that improvements have to be made to communication with colleagues, pharmacists, and hospitals. They state that a standard way of working is very important to reach such a communication. The EPS does not deliver these features. Saarinen and Sääksjärvi (1992) measured the improved internal communication and improved inter-organizational communication under the header “impact of the IS on the organization”. None of these success factors was satisfied in our cases. In 55% of the case studies and independently of each other (the term was not mentioned by the interviewee) the GPs stated that there should be a diminishing of the time pressure. Both in the description of the EPS and in international literature, it is made assumable that EPS will not diminish the time of the consult (Mitchell & Sullivan, 2001; Thornett, 2001). Forty-five percent of the case studies reported that the GP expected a fee in return for going through the trouble of implementing and using EPS. At the moment of interviewing it was not clear what financial profit the new system would deliver for the GP. What was known was that it would save the government and insurance companies a large amount of money on costs of medicine. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


In about 20 case studies lack of trust in the existing software and in the software supplier were mentioned as a barrier for (wanting to) use the new EPS. They said that first things had to change in the GP-IS market and in the GP IS itself before EPS could be a success. About the same amount of GPs want to remain freedom of choice for medication of the patient. Although this seems to be a resistance matter it is also a relevance matter because the EPS does not comprehend new ideas and new treatments that are already known in the general practice. Although the use of ICPC seems useful to many GPs (in structuring and communicating), the time that it will cost to find the right code and the omissions of some codes form a barrier for EPS use. Twenty percent of the GPs make use of a personal or regional formulary. The EPS makes use of a formulary of the Dutch council for GPs and often does not have the possibility to keep the own formulary when an update of the software is installed. Finally, once the computer system was installed, use of the system was mostly sparse. The way of working was relatively complicated and added relatively little value in most patient–doctor contacts.

Empirical Requirements to the GP General Requirements The objectives for this innovation were mainly money driven. The system should decrease prescription costs with 150 million euro yearly. To the GPs we interviewed, the goals were not clear. Functional and Performance Requirements •

Content: The functionality of the GP information system can be divided into administrative functionality and medical functionality. We observed that the administrative use of the system has the overhand. Only 15 GP’s (27.3 %) made use of the SOAP (subjective, objective, assessment, plan) module in the systems, which is a prerequisite for the use of the EPS.



Communication with other GPs, hospitals, and pharmacists is a requirement that is high on the agenda of the GP (55%). Still, the new EPS does not support the communication at all. •

•

•

Timeliness: The time pressure is one of the most important problems of the GP today. Timeliness of the system is therefore an important performance criterion. Due to a bad user interface the GPs are not able to work several records parallel and therefore loose time in opening and closing the patient’s record. Accurateness: The accurateness of the system is good and might be too good. The system was rigidly designed to avoid failures and therefore has many signal functions. For instance, when prescribing medicines for influenza, the GP gets a lot of alternatives and warnings where he or she already exactly knows what to prescribe. Also, the accurateness of input is a problem, because 30% of the GPs think it is unnecessary and sometimes difficult to generate a code for all “vague” diseases like stomachache, headache, and so forth. Ability to Integrate: The electronic prescription system is delivered on CD-ROM as a stand-alone system. This means that it is not integrated in the GP information system and also not in the communication configuration of the GP. The GP therefore has to start the program for each patient and cannot work parallel because the system is not window-based.

Empirical Resources to the GP Costs For the GPs there are no costs involved in getting the system but they need to time to install and operate the system. In general, 30 million euro was spent in designing and implementing the system. Strangely, all system suppliers said they did not get money to change their GP IS. The operating and maintenance costs are not seen as a problem by the GP. The reward for using the system is seen a problem. Extra office support was promised by the government but in practice not given and not clear.



Hardware and Software The GPs have no faith in the suppliers of GP IS. Caused by fusions and takeovers, the suppliers and GPs are in a deadlocked situation where maintenance seems to be the only thing that happens. Thirty percent of the GPs call the quality of the GP information system an obstacle for using the new EPS. User’s and Designer’s Time From our questions to the GPs, it is very difficult to analyze the time spend in designing the system. In operating the system the GP loses time because he or she has to put more information in the computer and uses some time in consulting the system. With an average time of 6 minutes per consult this is a big problem. Adaptability From maintenance point of view, the system is very adaptable since a new version just has to be distributed without having to change the rest of the GP systems. Nevertheless we advised that the GP system itself had to be updated with the EPS as an integrative communicative subsystem within. Capabilities One of our final conclusions in the main report (Lagendijk et al., 2001) is that we think it is crucial in the continuance of the project that the average GP is addressed in stead of the innovative GP. In designing the system GP’s were involved but only voluntary GPs that are bound to be pro bias focused. We also found big differences in IT capabilities. Some GPs still used the paper record and no computer, and some GPs did all their activities on the computer. Different introduction scenario’s, therefore, are needed to diffuse the system into all GP practices.



Reliability According to the GPs, the system is reliable. Breakdown of the systems seldom occurs. The maintenance is reasonable, although one GP states: “We have to be at a patient’s house in 10 minutes and they can stay away for 10 days”.

Conclusions Resistance of GP’s is not the determinant of the use of the EPS (only 27%). Resistance is the cumulative consequence of effects of the other determinants and therefore it looks as if resistance is the most important determinant. This means that many studies work on the effect and not on the cause of the lack of IT use. Relevance has long since been a central notion to IT theory. The elaborated approach that we proposed in this paper was used in 56 case studies. These cases provided us with enough evidence that for this particular (electronic prescription) system in this particular (healthcare) branch, relevance was the most important determinant for failure of diffusion and use of the system. Although in many studies the social criteria of success are mentioned as more important than the technical criteria we cannot confirm this for these 56 cases. In most cases the resources were not sufficient to use the new EPS. On top of that the requirements of the users were not sufficiently met by the system. We like to draw the following conclusion for the healthcare organizations: Before starting a new project to build or buy a new information system in healthcare organizations it is necessary to explicitly measure the resources available. Next step is to make a contract containing functional and performance requirements both agreed upon by a broad (laggards and innovators alike) group of end users and the responsible designers of the system.



Finally, we can conclude that assessing the IT diffusion and IT use of the EPS with the USE IT model was a multiple case study balancing the sociotechnical determinants. The model has been used in two other healthcare situations and is now been used in a telecare project for stroke patients and a diffusion project of an electronic patient record in a hospital. We encourage other researchers to work with the model in other environments (also outside of healthcare) and encourage the use of the interview schedule.

References Ajzen, I., & Madden, T. J. (1986). The prediction of goal directed behavior: Attitudes, intentions and perceived behavioral control. Journal of Experimental Psychology, 22, 453-474. Ansoff, I. (1965). Corporate strategy. New York: McGraw Hill. Ballantine, J., Bonner, M., Levy, M., Martin, A., Munro, I., & Powell, P. L. (1998). Developing a 3D model of IS success. In Garrity & Sanders, Information systems success measurement. Hershey, PA: Idea Group Publishing. Barnard, C. I. (1938). The functions of the executive. Cambridge, MA: Harvard University Press. Berg, M. (2001). Implementing information systems in healthcare organizations: Myths and challenges. International Journal of Medical Informatics, 64, 143-156. Beuscart-Zephir, M. C., Anceaux, F., Crinquette, V., & Renard, J. M. (2001). Integrating users’ activity modeling in the design and assessment of hospital electronic patient records: The example of anesthesia. International Journal of Medical Informatics, 4, 157-171. Beuscart-Zephir, M. C., Brender, J., Beuscart, R., & Menager-Depriester, I. (1997). Cognitive evaluation: How to assess the usability of information technology in healthcare. Computer Methods and Programs in Biomedicine, 54, 19-28. Binney, G., & Williams, C. (1995). Leaning into the future. Changing the way people change organizations. London: Brealey.



Brender, J., & McNair, P. (2001). User requirements specifications: A hierarchical structure covering strategical, tactical and operational requirements. International journal of Medical Informatics, 64, 83-98. Brender, J. (1999). Methodology for constructive assessment of IT-based systems in an organisational context. International Journal of Medical Informatics, 56, 67-86. Carey, J. M. (Ed.). (1988). Human factors in management information systems. NJ: ABLEX. Coch, L., & French, J. R. P., Jr. (1947). Overcoming resistance to change. Human Relations: Studies Towards the Integration of Social Sciences, 1(2), 512-532. Cohen, L. J. (1962). The diversity of meaning. London: Methuen. Cooper, W. S. (1971). A definition of relevance for information retrieval. Information Storage and Retrieval, 7(1), 19-37. Davis, F. D. (1989, September). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. DeLone, W. H., & McLean. (1992). Information systems success: The quest for the dependent variable. Information Systems Research,3(1), 6095. Dent, E. B., & Goldberg, S. G. (1999). Challenging “resistance to change.” Journal of Applied Behavioral Science, 35(1), 25-41. Duck, J. D. (1993, November-December). Managing change: The art of balancing. Harvard Business Review, 109-118. Fleisner, P., & Hofkircher, W. (1998). The making of the information society: Driving forces, “‘Leitbilder’ and the imperative of survival.” Biosystems, 46, 201-207. Folger, R., Skarlicki, D. P., & Tesluk, P. (1999). Research notes–Personality as a moderator in the relationship between fairness and retaliation. Academy of Management Journal, 42(1), 100-110. Fuller, F. F. (1969). Concerns of teachers. American Educational Research Journal, 6, 207-226. Gatignon, H., & Robertson, T. S. (1989). Technology diffusion: An empirical test of competitive effects. Journal of Marketing, 53, 35-49. Goodstein, L. D., & Burke, W. W. (1991). Creating successful organizational change. Organizational Dynamics, 20 (4), 5.



Hakkinen, H., Turunen, P., & Spil, T. A. M. (2003). Information in healthcare process–Evaluation toolkit development. Proceedings of HICSS 36. Retrieved from http://www.hicss.hawaii.edu/diglib.htm Henry, J. W., & Stone, R. W. (1999). End user perception of the impacts of computer self-efficacy and outcome expectancy on job performance and patient care when using a medical information system. (IJHTM) [Special issue). Advances in the Management of Technology in Healthcare, 1(1/2), 103-124. Iivari, J., & Koskela, E. (1987, September). The PIOCO model for IS design. MIS Quarterly, 400-417. Kotter, J. P., & Schlesinger, L. A. (1979, March-April). Choosing strategies for change. Harvard Business Review, 106-113. Lagendijk, P. J. B., Schuring, R.W., & Spil, A. A. M. (2001). Het elektronisch voorschrijf Systeem, van kwaal to medicijn. Enschede, The Netherlands: University of Twente. Lanning, H. (2001). Planning and implementing change in organisations: A construct for managing change projects, Doctoral dissertation, Helsinki University of Technology, Espoo, Finland. Lapointe, L., Lamothe, L., & Fortin, J. P. (2002). The dynamics of IT adoption in a major change process in healthcare delivery. Proceedings of the 35th HICSS Conference, Hawaii. Larsen, T. J. (1998). Information systems innovation: A framework for research and practice. In T. J. Larsen & E. McGuire, Information Systems Innovation and Diffusion. Hershey, PA: Idea Group Publishing. Lawrence, P. R. (1954). How to deal with resistance to change. Harvard Business Review,32(3), 49. Leavitt, H. J. (1965). Applied organisational change in industry: Structural technological and humanistic approaches. In J. G. March (Ed.), Handbook of organisations (pp. 1144-1170). Chicago: Rand-McNally. Lewin, K. (1952). Field theory in social science, selected theoretical papers. In D. Cartwright (Ed.). London: Tavistock. Liu, K. (1993). Semiotics applied to information systems development, Doctoral thesis, University of Twente, Enschede, The Netherlands. Mahmood, M. A., Burn, J. M., Gemoets, L. A., & Jacquez, C. (2000). Variables affecting IT end-user satisfaction: A meta-analysis of the Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


empirical literature. International Journal of Human-Computer-Studies, 52, 751-771. Malinconico, S. M. (1983). Hearing the resistance. Library Journal, 108(2), 111-113. Markus, M. L. (1983). Power, politics and MIS implementation. Communications of the ACM, 26(6), 430-444. Metselaar, E. E., van der Kolk, J. M., & Wortelboer, F. Q. C. (1996). Werken aan veran-deringsbereidheid. Handboek effectief opleiden, 9, 135158. Miller, C. A. (1999). Bridging the information transfer gap: Measuring goodness of information fit. Journal of Visual Language and Computing, 10, 523-558. Mitchell, E., & Sullivan, F. (2001). A descriptive feast but an evaluative famine: Systematic review of published articles on primary care computing during 1980-97. British Medical Journal, 322, 279-282. Mittelstaedt, R. A., Grossbart, S. L., Curtis, W. W., & Devere, S. P. (1976). Optimal stimulation level and the adoption decision process Journal of Consumer Research, 3, 84-94. Mumford, E. (1995). Effective systems design and requirements analysis. London: MacMillan. Mumford, E. (1983). Designing human systems for new technology. Manchester Business School. Nykänen, P. (2000). Decision support systems from a health informatics perspective, Doctoral dissertation, University of Tampere, Tampere, Finland. Offenbeek, M. van, & Koopman, P. (1996). Interaction and decision making in project teams. In M. A. West (Ed.), Handbook of work group psychology. Wiley. Paré, G., & Elam, J. J. (1999). Physicians’ acceptance of clinical information systems: An empirical look at attitudes expectations and skills. International Journal of Healthcare Technology and Management,1(1), 4661. Piderit, S. K. (2000). Rethinking resistance and recognizing ambivalence: A multidimensional view of attitudes toward an organizational change. The Academy of Management Review, 25(4), 783-794.



Prasad, P., & Prasad, A. (2000, July-August). Stretching the iron cage: The constitution and implications of routine workplace resistance. Organizational Science, 11(4), 387-403. Pressman, R. S. (1982). Software engineering, a practitioner’s approach. McGraw-Hill. Raghavan, S. A., & Chand, D. R. (1989). Diffusing software engineering methods (IEEE Software, July: 81-90) [Computer software]. Ram, S., & Sheth, J. N. (1987). Bringing innovation to market, how to break corporate and customer barriers. New York: Wiley. Rogers, E. M. (1983). Diffusion of innovations. New York: The Free Press. Rogers, E. M., & Scott, K. L. (1997). The diffusion of innovations model and outreach from the national network of libraries of medicine to native american communities. Retrieved from http://www.nnlm.nlm.nih.gov/ pnr/eval/rogers.html Saarinen, T., & Sääksjärvi. (1992). Process and product success in information system development. Journal of Strategic Information Systems, 1(5), 266-277. Salmela, H. (1997). From information systems quality to sustainable business quality. Information and Software Technology, 39, 819-825. Saracevic, T. (1975). Relevance: A review of and framework for the thinking on the notion in information science. Journal of the American Society for Information Science, 26(6), 321-343. Schmidt, R., Lyytinen, K., Keil, M., & Cule, P. (2001, Spring). Identifying software project risks: An international delphi study. Journal of Management Information Systems, 17(4), 5-36. Schuring, R. W., & Spil, T. A. M. (2002). Explaining plateaued diffusion by combining the user-IT-success factors (USeIT) and adopter categories: The case of electronic prescription systems for general practitioners. International Journal of Healthcare Technology & Management, 44(3), 303-318. Schuring, R. W., & Spil, T. A. M. (2003). Relevance and micro-relevance for the professional as determinants of IT-diffusion and IT-use in healthcare. In G. Grant (Ed.), ERP & data warehousing in organizations: Issues and challenges (pp 219-232). Hershey, PA: IRM Press. Scott, J. (1985). Weapons of the weak: Everyday forms of peasant resistance. New Haven, CT: Yale University Press. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Chicago: University of Illinois Press. Southon, G., Sauer, C., & Dampney, K. (1999). Lessons from a failed initiative: Issues for complex organisations. Medical Informatics, 55, 33-46. Sperber, D., & Wilson, D. (1986). RELEVANCE, communication and cognition. Oxford, UK: Blackwell. Spil, T. A. M. (1993). The evaluation of SISP: From a quality undergrowth to a semiotic clearing. Proceedings of the First ECITE conference, Henley on Thames. Spil, T. A. M., Schuring, R. W., & Katsma, C. (2002). Assessing resistance of professionals as a determinant of IT-diffusion and IT-use in healthcare. Proceedings of the Ninth ECITE conference, Paris. Stamper, R. K. (1973). Information in business and administrative systems. New York: Wiley. Strebel, P. (1996, May-June). Why do employees resist change? Harvard Business Review, 86-92. Sullivan, F., & Mitchell, E. (1995). Has general practitioner computing made a difference to patient care? British Medical Journal, 311, 848852. Szmigin, I., & Foxal, G. (1998). Three forms of innovation resistance: The case of retail payment methods. Technovation, 18(6/7), 459-468. Thong, J. Y. L., & Yap, C. S. (1995). CEO characteristics, organizational characteristics and IT adoption in small businesses. Omega, 23(4), 429-442. Thornett, A. M. (2001). Computer decision support systems in general practice. International Journal of Information Management, 21, 3947. Van der Pijl, G. J. (1994). Measuring the strategic dimensions of the quality of information. Journal of Strategic Information Systems, 3(3), 179190. Walley, & Davies. (2001). Implementation IT in NHS Hospitals–Internal barriers to technological advancement. Paper presented on the First Hospital of the Future Conference, Enschede, The Netherlands. Wieringa, R. J. (2001). Requirements engineering: Frameworks for understanding. Chichester, UK: Wiley. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Wilson, P. (1973). Situational relevance. Information Storage and Retrieval, 9(8), 457-471. Wissema, J. G. (1987). Angst om te veranderen? Een mythe! Assen: Van Gorcum. Yin, R. K. (1984). Case study research, design and methods. London: Sage. Zaltman, G., & Duncan, R. (1977). Strategies for planned change. New York: Wiley. Zmud, R. W. (1979). Individual differences and MIS success: A review of empirical literature. Management Science, 25(10), 966-979. Zuboff, S. (1982, March-April). New worlds of computer-mediated work. Harvard Business Review, 142-153.


The USE IT Model Case Studies 177

Chapter X

The USE IT Model Case Studies:

IT Perceptions in the Multiple Sclerosis, Rheumatism and Stroke Healthcare Chains Margreet B. Michel-Verkerke University of Twente, The Netherlands Roel W. Schuring University of Twente, The Netherlands Ton A. M. Spil University of Twente, The Netherlands

Abstract In the previous two chapters, the determinants and theoretical background of the USE IT model is discussed. In this chapter, the application of the USE IT model in three cases are described to show the value and benefits of the USE IT model in practice. The USE IT model has four determinants: resistance, relevance, requirements, and resources. It can be used ex ante and ex post. The USE IT model is applied ex ante to find relevance and appropriate choices to overcome resistance for an ICT support of the multiple sclerosis (MS) healthcare chain and the rheumatism care guide, Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

178 Michel-Verkerke, Schuring & Spil

and as well ex ante as ex post in a local stroke service to measure the feasibility of a mobile device for general practitioners. The USE IT model proved to be very helpful not only in revealing the most urgent and relevant problems but also in discovering the crucial obstacles and prerequisites for implementing a solution to these problems. By that, the USE IT model served as a strong tool to decide whether healthcare processes should be supported by ICT and, if so, what processes should be used and how.

Introduction As was explained in the previous chapters, the USE IT model comprises four determinants: resistance, requirements, relevance, and resources to measure the factors in the social and technology domain that influence the diffusion and adoption of information systems and technology (see Table 1). The instrument to measure the determinants is a structured interview with the (intended) end users of the innovation. The interview questions are rubricated in five sections: process, requirements, relevance, resistance, and resources (see Chapter IX). A prerequisite to adoption of an innovation is that the innovation fits with the actual working process. We agree with Brender (1999) that the kernel point of assessment is that of understanding the business processes. For this reason, the section process is added to the interview list. Chapter VIII and IX showed how the USE IT model is able to explain afterwards (ex post) the lack of diffusion of an innovation in healthcare. In this chapter we will present another three case studies in which the USE IT model is used ex ante to reveal the factors that would influence the success of an intended innovation and to give an advice on the conditions and prerequisites the innovation had to meet to be accepted and used by the end users. The case study of the MS chain was set up to investigate the feasibility of ICT support and resulted in an advice to first organize the healthcare chain and then Table 1. USE IT model USE IT model Product Process

User Domain Relevance Resistance

Information Technology Domain Requirements Resources



support the care process with a modest and low-budget, Web-based information system, because the research showed that the relevance of MS care is very low for most care providers for the simple reason that the prevalence of MS is low. First, the cases will be discussed separately and then the results will be compared with each other to identify the value of the USE IT model.

Empirical Results: MS Healthcare Chain (ex ante) Case Research Method The special challenge of the research is to find a solution that is locally and practically applicable, and that can serve as a base for a more general, broader solution on the same time. This challenge is represented in the two research questions, which are formulated as follows: 1.

2.

In what way does an improvement of the information services in the healthcare chain contribute to the improvement of the quality of care for patients with MS? How should a solution look that solves the local problem but also aligns with knowledge and standards on EPR and serves as a first step or building block of an EPR?

To answer these questions, 17 care providers, which are part of the MS-care chain in Twente, a Dutch region, are interviewed with the preliminary version of the USE IT tool (Schuring & Spil, 2002). Also, 6 of the approximately 500 patients are interviewed to get an impression of how they experienced the given care (Michel-Verkerke, Schuring, Spil, & Hummel, 2003).

Empirical Resistance in the MS Healthcare Chain Little or no resistance towards the use of ICT as such is reported, but resistance exists towards a specific solution for the MS care. Care providers feared the



situation in which each patient group has its own computer-based record. They feared to be loaded with separate solutions for every separate chronic disease.

Empirical Relevance in the MS Healthcare Chain The USE IT tool made clear that MS care is not very relevant for most care providers. That is to say, care providers have high compassion for MS patients, but most care providers saw MS patients only now and then. MS care was relevant only for the four care providers in the region, who spend 15% to 40% of their time on those patients. Because of the infrequent contact with MS patients, the knowledge of most care providers about MS care seems to be insufficient. This lack of knowledge caused incidents of insufficient and inadequate care, as reported by the patients.

Empirical Requirements in the MS Healthcare Chain The main problem proved to be the lack of coordination in the healthcare chain. There hardly was a chain; we found handovers between care providers that were executed by the patient. Patient-flow and workflow was not organized for the specific patient group. Two coordination mechanisms could be found: the official referral system and informal communication (mutual adjustment). The first one was very insufficient, because it does not cover all the information needs and because the key role is destined for the GP, who is, in practice, too busy to fulfill his role as coordinator of care. The second is reasonably effective, but not very efficient and often slow. Not all healthcare providers are aware of the service that other care providers can provide.

Empirical Resources in the MS Healthcare Chain There is no regional electronic patient record or likewise ICT facility in the area studied that could serve as a basis for solutions. Some care providers use electronic records, but these systems are often used for administration purposes only and limited to the use in one institution. Almost all care providers have (or would have, with short notice) access to e-mail or the Internet.



Since the relevance of MS care is high to only a few care providers, most care providers could not spend much time and effort in implementing a specific solution for MS care.

Conclusions of the MS Healthcare Chain The main finding in this research was the low relevance of MS care for care providers. This resulted in resistance to a specific solution for MS care. Also the lack of up-to-date knowledge of what care should be provided can be explained by the low prevalence of MS. Another important finding was that the official referring system failed to serve as a routing system for patient flow. The compensating coordination mechanism, consisting of informal communication made the shape of the healthcare chain for this chronic progressive illness to be a complex network with many cross-relations, which did not match with the formal referral tree. It is not a great surprise that patients and care providers get lost or stuck in this spiderweb, although patients felt these problems to a lesser extent than care providers. From the research can be concluded that the research questions were too ambitious for this ill organized healthcare chain. By applying the USE IT model, both the failing coordination mechanism and the low relevance were revealed. Therefore, it is advisable to first start organizing the healthcare chain and then support this chain with an ICT solution that meets the following conditions: (a) No isolated solution for MS care: a specific solution must be expandable for other diseases, (b) implementation and maintenance must take very little effort and costs, and (c) the solution must adhere to the present ICT resources.

Empirical Results: Rheumatism Care Guide (ex ante) Case Method The Rheumatism Care Guide case was a small project in which we were asked to study the relevance of an electronic version of the care guide: a form to register the care needs of a rheumatism patient and the needs for support of his



or her partner. We had interviews with all future users of the proposed system and interviewed them with the USE IT model, with the emphasis on relevance.

Empirical Relevance in the Rheumatism Care Guide All interviewed users saw no usefulness in the (paper-based) Rheumatism Care Guide. It did not solve their general problems (i.e., capacity and time). The objectives of the users-indicating and lessening care needs of patients-are not supported by the system. An electronic form would not alter this. Some users indicated that others might use the system, but those other users denied that. A potential benefit of the Rheumatism Care Guide is the enhancement of uniformity of the working processes, but at the same time, the interviewees expressed their satisfaction with the forms in use now. The only contributed value of the Rheumatism Care Guide mentioned was the explicit attention it pays to the partner of the patient, but replacement of the home visits by a paper or electronic form was not accepted, because it was believed that this would reduce the quality of the information gathered and the social value of the working process of the users and the patients. All future users state that they do not want a system for rheumatism only. A new system should be much broader for all chronic patients and have more functionality than only intake information. The “here and now” problems are accessibility of the care providers, random character of the disease, waiting lists, and objective indication of the patient. All these problems are not addressed by the new system.

Conclusions: Rheumatism Care Guide The main conclusion of this study was that an electronic version of the care guide for rheumatism was not relevant to the users and patients and therefore we recommended to abandon the project.



Empirical Results: Stroke Service (ex ante and ex post) Case-Study Method: Stroke Service The care for stroke patients is organized in the Stroke Service, which comprises arrangements between general practitioners (GPs), the hospital, the rehabilitation center, a nursing home, and the home-care organization. When a patient feels ill, he or she calls the GP. The GP visits the patient and diagnoses the patient as probably having a stroke. The GP contacts the neurologist, who admits the patient to the hospital. When the patient arrives at the hospital a CT scan is made to decide whether the stroke is based on a brain infarct or bleeding. In case of a brain infarct, thrombolysis (i.e., resolving the thrombosis by medication) is applied to the patient. After this, or when bleeding is present, the patient is admitted to the stroke unit of the hospital. In 8 to 10 days, the patient’s situation is evaluated by a multidisciplinary team. When further treatment is considered to be useful, the patient is either discharged to the stroke reactivation unit of the rehabilitation center or the nursing home. When the patient finally returns to his or her home, the specialized nurse of the homecare organization will visit him or her several times. The goals of the Stroke Service are to admit all patients with a brain infarct to the hospital for thrombolysis within 3 hours and to start secondary prevention as soon as possible in order to minimize disability. To achieve these goals, a project is started to investigate whether a mobile solution can improve the processes in the Stroke Service and, if so, what this mobile solution should be (Vollenbroek-Hutten & Velthausz, 2002). In order to reveal the requirements and constraints, the USE IT method is used (Michel-Verkerke, Schuring, & Harten, 2003). Of the care providers taking part in the Stroke Service, 19 are interviewed. In Table 2, their professions related to the institutions are listed.

Empirical Resistance in the Stroke Service Almost all interviewed care providers expressed a positive attitude towards ICT in general and the use of ICT in the care processes. At the same time, most



Table 2. Professions of the interviewed care providers Institution General Practice Hospital Rehabilitation Center Nursing Home Home Care

Profession General Practitioner (6) Neurologist, Nurse (2), Physiotherapist Physician, Physiotherapist, Speech Therapist, Occupational Therapist, Psychologist Physician, Physiotherapist, Nurse Nurse

care providers foresee that the often limited and out-of-date ICT facilities will be a hindrance to the actual use of ICT in the care process. Another possible hindrance mentioned by several care providers is the expected and experienced resistance to the use of computers by their colleagues. They explain this resistance as feelings of uncertainty towards the use of computers. A resistance to change in general is not found. (Although, most of those interviewed probably can be categorized as “early adopters” or even “innovators.” The interviewed care providers often volunteered to join the Stroke Serviceproject.) The interviewed GPs all joined the pilot and have a positive attitude to the presented innovation. Some are members of the automation committee and are much in favor of the use of ICT. Others have a more critical attitude but are prepared to try out innovations. Before the pilot started, some resistance existed to the fact that the PDA could only be used for stroke patients. But most GPs participate in the TeleCare project, because they regard the “stroke case” as an example of other acute illnesses in which the PDA could be useful. Despite the possible hindrances, most care providers expect to make enough time and energy available for the introduction of ICT in the Stroke Service.

Empirical Relevance in the Stroke Service The care for stroke patients is the main job for 12 out of 19 care providers (40%-90% of their patients are stroke patients). Only the GPs and the homecare nurses see fewer stroke patients (fewer than 10% of their patients). Based Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


on the number of patients, it can be concluded that the care for stroke patients has very limited relevance for the GP. But when considering the possibility of intervention, the occurrence of a stroke is very relevant to a GP, because he or she is usually the care provider that is called first and the key person who can admit the patient to the hospital as soon as possible, which is crucial in case of an brain infarct. The interviews show that GPs are aware of this; the “time is brain” adage of the neurologist was frequently cited. Most important to all care providers is to provide good care. The main points of improvement, care providers report, are shortage of nurses, waiting lists for nursing homes, very limited budgets for investments, and the hampering of patient flow. When looking at micro-relevance, the lack of an electronic patient record as a tool to have all relevant patient data available in time, and at the point of care, is most mentioned. During shifts, the main problem experienced by GPs is the number of calls for nonurgent matters. These calls make up about 30% of all calls.

Empirical Requirements in the Stroke Service What care providers in the Stroke Service really need is information of and about their patients. This information does not only comprise the recent medical history, tests, and treatments concerning the discipline of the care provider, but also previous episodes of illness, coexisting diseases, and information of other, especially related disciplines. All care providers express the necessity of this information for their decision making. The most important information sources are the patient and his of her family, the physician and the written information sent by a care provider within the same discipline, working in the previous institution. The available information matches the expressed information need only partially. Especially the information about the episodes before the stroke occurred is missing. The incompleteness of information transferred from one institution to another within one discipline is the complaint most often mentioned. The absence of relevant information at the relevant time causes delays and the duplication of tests. During shifts, the relevant medical information in the GPs information system is not available and must be asked of the patient or his or her family. Information gathered this way is often incomplete.



Empirical Resources in the Stroke Service The ICT facilities for administrative and secretarial purposes are adequate, but they are not usually available to care providers. The number of computers available is too low and computers are not in the room where care is given. Also, most care providers consider the computers to slow and lacking e-mail or Internet access. GPs are an exception to this situation: All questioned GPs used their information systems to keep their patient records. Some GP surgeries had access to other’s patient records. Two of them also used a PDA with access to medical knowledge, but not to medical patient data. However, during shifts, GPs do not have access to any patient’s records except for the information of previous calls, which is available in the information system of the central doctor’s post (CHPE), where they reside during shifts. The stakeholder analysis conducted in this research revealed that updating and upgrading the ICT facilities heavily depend on the available budget, which is not easily allocated for this purpose. Also, time and money is scarce. With the shortage of personnel–especially nursing staff–it is hard to make time to introduce new technology. Despite this, most care providers are so enthusiastic that they manage to make time for those innovations they consider essential. This same enthusiasm helps them convince the management to allow and support training.

Conclusions: Stroke Service (ex ante) Good care for stroke patients is the main interest of care providers of the Stroke Service. The main problems they experience are shortages of nurses and shortages of beds in nursing homes. The lack of adequate and relevant patient information is the most micro-relevant problem to them.

The Pilot The pilot comprised the use of a PDA by GPs when seeing a possible stroke patient during shifts. The PDA displayed the information of patients of seven GP information systems at four GP surgeries, of which the GP information system could be disclosed. The PDA also contained forms to enter the physician’s findings, which were automatically transformed into a message that could be



sent to the emergency room of the hospital. To support the GP in decision making a checklist with the criteria for thrombolysis was incorporated in the PDA and available as text in the PDA.

Evaluation of Propositions Based on the USE IT Interviews This part of the evaluation research has several objectives. First, we want to know the user characteristics from the GPs who participated in the pilot. Second, we want to test whether the propositions we made about the relevance and usability of the innovation proved to be valid. Third, we want to test whether the USE IT interview method is suitable for predicting and evaluating in the same case. On basis of our ex ante analysis, we identified three CSFs. First, the solution should be micro-relevant for the general practitioners. Second, there need to be sufficient financial resources. Third, the opinion of the neurologist in the hospital about the solution will be critical (Michel-Verkerke, Schuring, & Harten, 2003).

USE IT Posttest Interviews After the pilot, two GPs had actually seen stroke patients during shifts, a third one tested the PDA during his shift. These GPs are interviewed about their experiences. Change in Process When using the PDA, it is not necessary for the GP to write a note to the neurologist and give the note to the patient before he or she is transported to the ambulance. The GP can enter his or her findings while talking with and examining the patient. After the information is entered, the GP can see–by pressing a button–whether the criteria for thrombolysis apply to this patient. The GP can also read the checklist. The SOAP (i.e., subjective, objective, assessment, plan) message composed this way can be sent to the emergency room, where the neurologist can print the information.



This change in process means that the GP–when calling the neurologist–knows what information the neurologist will ask for. It also means that the message about the patient is more complete and arrives before the patient arrives. This is confirmed by the GPs who actually used the PDA during the pilot study. The PDA Experience The GPs agreed on the usefulness of the PDA. Especially the availability of the checklist was appreciated. Because of the checklist the GPs felt more secure about the information they needed before calling the neurologist. They also used the checklist as a confirmation for their acting. One GP said that he learned form the checklist that the glucose level should be examined and he resolved to test glucose with every stroke patient from that moment on. In the two cases the medical history was not available on the PDA, because the two surgeries were not part of a disclosed GP information system. The GPs regretted this because they would have liked to have this information before or while visiting the patient in order to make a better decision on the medical policy. The PDA also contained forms to compose a SOAP message. These forms consisted of items with checkboxes and scroll menus. The forms were easy to use. Both the subjective and objective page contained a picture of a person on which the paralysis could be marked. Only the second one was considered to be useful. The use of the pen to click and scroll required some concentration but was not considered to be too difficult. The readability was sufficient. What was generally missed (and also mentioned during the training) was the item pulse (rate and quality, regular or irregular), because an irregular pulse diagnoses atrium fibrillation, an important cause of strokes. The size of the PDA was not a real problem, but a smaller (flatter) PDA without an antenna, was preferred, so it would fit better in a shirt pocket. Relevance The relevance of stroke or the relevance of problems experienced have not changed during the pilot. But actually seeing and using the PDA made the GPs much more aware of its capabilities. The desire to use the PDA for more patient



groups during shifts arose. Especially in possible life-threatening situations, such as heart attacks, GPs appreciate the “back-up” of a checklist, specific for that situation. It makes them feel more confident in making decisions and when negotiating with the medical specialist about hospital admission of the patient. The two functions appreciated most are the medical history of the patient and the checklist.

Conclusions Stroke Service (ex post) Objectives of the PDA (Focus) The benefit for the patient could not be tested, because the criteria for thrombolysis did not apply to the two stroke patients in the pilot study. The suitability of a wireless application was shown, since the GPs visited the patient at home and had no other means of connecting to the information systems of the CHPE. Too few patients were seen and the pilot study was too short to test whether the next phases in the care process would benefit from the better start of collecting and communicating information at the beginning of the care process, but the GPs agreed that their message was more complete than before. A PDA for stroke patients only is not micro-relevant for GPs. It does not solve their relevant problems, but a PDA that shows the medical history of all patients with acute illnesses and that contains all necessary checklists is micro-relevant for GPs during shifts. Evaluation Objectives The user characteristics are described in the previous sections. The pilot study was too small to validate our propositions, but no contradictory findings arose. The USE IT interview method seems suitable for predicting and evaluating in the same case.



Conclusions on the Value of the USE IT Model The EPS case in the previous two chapters showed that resistance of GPs is not the determinant of the use of the EPS. Resistance is the cumulative consequence of effects of the other determinants, and therefore it looks as if resistance is the most important determinant. This means that many studies work on the effect and not on the cause of the lack of IT use. In the three ex ante studies we presented in this chapter, we looked at the attitude of the users but with the relevance of the ICT in mind. To gain a positive attitude towards the suggested change, emphasized making the innovation as relevant to the end user as possible. In the MS case, this meant that the initial intention of introducing an EPR was canceled and organizational changes were suggested first, supported by a much more modest information system. In the Stroke Service relevance of stroke care was low for GPs, but the functions implemented in the PDA and the use during shifts were very relevant. The lack of relevance in the rheumatism case was discovered before implementation had started, so at a time that stopping the ICT project was still possible. In all three ex ante case studies, the resources were a big problem for project success. The level of ICT in healthcare in general is that low that innovative ICT projects will have major problems in the implementation stage. Even a bigger problem is the shortage of money and staff. This means that innovations have too use as little resources as possible: required investments have to be low, but also organizational changes and training have to cost as little time as possible. We learned from this research that the USE IT analysis of the characteristics of the end user helps to provide a more appropriate picture of the problem and the constraints and prerequisites for solving it. It is likely that we would have suggested a far more complicated ICT solution if we had only analyzed the MScare process as such, without specific consideration of the USE IT dimensions. The USE IT analysis helped us to balance the breadth of the proposed solution with the nature of the situation the future users of the system are in. We encourage other researchers to work with the model in other environments (also outside healthcare), and we would be happy to provide them with the interview schedule.



References Brender, J. (1999). Methodology for constructive assessment of IT-based systems in an organisational context. International Journal of Medical Informatics, 56, 67-86. Michel-Verkerke, M. B., Schuring, R. W., & Harten, W. H. V. (2003). Needs, requirements, interests and impact of integrated wireless ICT in care. Enschede, The Netherlands: University of Twente. Michel-Verkerke, M. B., Schuring, R. W., Spil, T. A. M., & Hummel, M. (2003). De MS-zorgketen in Twente: Eindverslag van het onderzoek naar de informatiestromen in de MS-zorgketen. Enschede, The Netherlands: University of Twente. Schuring, R. W., & Spil, T. A. M. (2002). Explaining plateaued diffusion by combining the user-IT-success factors (USIT) and adopter categories: The case of electronic prescription systems for general practitioners. International Journal of Healthcare Technology and Management, 4, 303-318. Vollenbroek-Hutten, M., & Velthausz, D. (2002). TELE-CARE: ICT Applications for tele-consultation to increase efficiency and effectiveness of care. Roessingh Research & Development, Telematica Instituut. Yin, R. K. (2003). Case study research: Design and methods. Applied Social Research Methods Series, 5. Thousand Oaks: Sage.


192 Spil & Schuring

Chapter XI

USE IT Interview Protocol1 Ton A. M. Spil University of Twente, The Netherlands Roel W. Schuring University of Twente, The Netherlands Date interview: Name interviewer: Name interviewee: Job interviewee: Organization:

P P1

Primary process What care do you provide? Most care providers contribute to different care processes. In our research we make the following distinction: • • • • • •

Diagnosis % Investigations outside the consulting room % Treatment % Nursing % Acute incidents occur: the whole day through / several times a day / several times a week Acute incidents dominate my work very much / somehow / a little / not

The categorization may be adjusted to the investigated care process as long as it is clear to what % of patients or tasks the innovations applies (see Rel. 7). How do you act at each of the above-mentioned tasks? • Do you follow a fixed pattern? • How long does a patient contact take? • Do you use equipment? • Do you use (human) support? If so, for whom else does this supporter work? • Where do you perform your tasks? Could they be performed elsewhere? • Do you always sit or stand in the same position towards the patient? (Make a sketch) • Do you have to look up or ask after things? • Do you have to prepare anything?

Continued on following page Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

USE IT Interview Protocol 193

P P2

P3 P4

P5 P6 P7

P8 P9

P 10

INF I1

Primary process What other tasks do you have apart from providing care? How much time or energy do these tasks take from you? time % energy % What exceptions or disturbances make that this kind of care or the coordination of this care fails? Do you use a care protocol or medical guideline for the care you provide? • Do you comply with this protocol entirely or partially? • What parts do you use, what parts don’t you use? • Does using the protocol fit with your way of working? Who refers patients to you? To whom do you refer patients? What other care providers or institutions are simultaneously involved with the care for your patients? • Do you work together? • Or do you work “in parallel”? How do you experience the cooperation with other care providers in respect to the providing of the care? With what care providers should you cooperate (more)? • Why? • With whom should you exchange more information? • What information? What do you find important in the contact with other care providers?

Information quality What information about the patient do you need to perform your job properly? (Distinguish according to the separate tasks, mentioned in P 1 and P 2) What information do you receive from • The patient? • The patient’s surrounding? • Other care providers? • With what purpose? • In what frequency?

I2

What form does this information have? • Letter (sent by post or handed over personally) • Fax • E-mail • In paper record • In electronic record Does this information suffice? • Do you experience problems? • Do you miss information?

Continued on following page


194 Spil & Schuring INF I3

Information quality What information do you generate yourself when providing care? What information do you give to: • The patient? • The patient’s surrounding? • Other care providers? • Managers? • External parties (e.g., insurance company, government)?

I4

I5

REL R1 R2

R3 R4

R5 R6 R7

What form does this information have? • Letter (sent by post or handed over personally) • Fax • E-mail • In paper record • In electronic record • Record only used for this patient group or this type of care • Record only used by your own discipline • Record only used in your institution How do you appreciate the quality of the proposed (or implemented) innovation? Regarding the: • Content • Objectives • Method • Possibility to integrate it in the present situation • Timeliness • Correctness Where the right end-users involved with making or selecting this innovation?

Relevance What do you experience, for you personally, as important in your daily work when you look at the care you provide? What aspects in the ability to provide care, do you experience as a bottleneck or problem? • Concerning the providing of care • Other aspects Are there any specific actions in the previously discussed processes that cause bottlenecks or problems? Do you know proposals for improvement, concerning these patients, for which you would do your utmost? How important are these proposed improvements in the chain of care in relation to other possibilities to improve aspects of your job? • Can you name other proposals for improvement, which are more important? • Can you name other proposals for improvement, which are less important? In what way could the use of ICT matter to you? • What application are you thinking of? • For what purpose or for what situation? What aspect of your job would you miss, if it would be removed? How important are your tasks for these patients, for you, in comparison with your tasks for other patients? • Why are these patients so important or of so little importance for you?


USE IT Interview Protocol 195 A A1

A2

A3 A4 A5 M M1

M2

M3

M4

Attitude To what extent are you convinced that the use of ICT is necessary to improve the providing of care? • What experience do you have with ICT? • How much time are you prepared to spend? • Do you use ICT to communicate? • How often do you use the Internet? • How often do you use specific systems yourself? Do you experience obstacles when implementing innovations? • Workload • Management support • ICT support • Money • Your skills How much time and energy do you think you can find to implement the changes that will occur when introducing innovations and ICT in this kind of care? Do your colleagues or managers stimulate you to participate in changes? Can you name other innovation-projects this organization is working on? • Are these projects equally important (or more or less important)? Means What ICT-facilities do you have at your disposal at your workplace? • Hardware • Software • For communication • Data What of these ICT-facilities do you use when providing care? • Hardware • Software • For communication • Data Is the technical support sufficient to guarantee the quality of the system? • Reliability • Availability • Security • Privacy Do you think you will have support to implement changes? • Time • Money • Training • Management support

1.1.1.1 C1.1.1.1.1 Concluding questions C1 Is there anything you would like to add? C2 May we contact you to think with us in the development of a ICT-application?


196 Spil & Schuring

References Spil, T. A. M., Schuring, R. W., & Michel-Verkerke, M. B. (2004). EPS: Do the professionals USE IT? International Journal of Healthcare, Technology & Management, 6(1), 32-55.

Endnote 1

Adapted from Spil, Schuring, and Michel-Verkerke (2004)


USE IT Interview Protocol 197

Section III Qualitative and Quantitative Evaluation


198 Kaplan

Chapter XII

Combining Qualitative and Quantitative Methods in IS in Healthcare Revisited Bonnie Kaplan Yale University, USA

Abstract Using a variety of research methods provides several advantages. Through multiple methods, different kinds of data are collected, each set of which might provide partial information needed for a complete picture, thereby strengthening the robustness of research results. Research that combines qualitative and quantitative methods, though rare, provides an example of the benefits of multimethod studies. In this chapter, I describe three evaluation research studies that used a combination of qualitative and quantitative methods. In these studies, researchers produced results they would not have achieved if they had not combined methods. The first study is of a pioneering computer-based patient record and clinical decision support system, PROMIS. The second study is of a clinical laboratory information system. The third, a more recent study, evaluated an automated Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Combining Qualitative and Quantitative Methods in IS in Healthcare Revisited 199

telephone health behavior advisory system. The PROMIS study compared different groups using PROMIS and compared PROMIS users with those using a manual patient record. The laboratory information system and telephone advisory system studies explored differences among ostensibly the same users of the same technology, only to find that these users divided into groups that differed in their responses to the technologies. Such studies could point to considerations other than technology per se that are important in how individuals react to and use technologies. All three studies are examples of how multimethod research can produce significant results.

Introduction Experimental designs and randomized controlled trials dominate in evaluating information systems in healthcare. Especially when combined with economic analyses, they are promoted as the way to conduct technology assessments and evaluations. Such approaches have an underlying positivistic epistemology (Kaplan, 2001). Similarly, positivism represents the predominant philosophical tradition in case research in information systems studies outside healthcare (Dubé & Paré, 2003). Increasingly, though, researchers are recognizing the limitations of relying on any one method as the gold standard for evaluation. For example, although excellent in pinpointing what changed, experimental approaches make it hard to assess why changes occurred. Longer term field studies and more qualitative or interpretive approaches are better for investigating processes, multiple dimensions and directions of causality, and relationships among system constituents and actors. Further, experimental designs prove difficult for following changes as they are developing, or in determining system design and implementation strategies that are well-suited to particular institutional settings and societal considerations. Consequently, numerous evaluation researchers call for taking advantage of the many approaches and theories, each with its own strengths, available to evaluation researchers. Reliance on experimental designs-or on any one approach, for that matter-unnecessarily limits the kinds of results and understanding evaluation can produce. In addition to calls for a broader selection from the repertoire of available


200 Kaplan

methods, there also is a growing literature on combining qualitative and quantitative methods. Because different methods have different strengths, multimethod research involves collecting dissimilar data regarding the same phenomena so as to complement each other and contribute to the robustness of a study through a process of triangulation (Ammenwerth, Iller, & Mansmann, 2003; Gallivan, 1977; Kaplan & Maxwell, 2004). Multimethod studies are not common. Fewer than 1/3 (30%) of the information systems case studies primarily in nonclinical settings used some form of data triangulation, while only 31% of 107 articles reported the use of a mix of qualitative and quantitative methods (Dubé & Paré, 2003), while a review of information systems research studies over roughly the same time period, using rather stringent criteria, reported mixing qualitative and quantitative methods in less than 2% of the studies (Gallivan, 1977). Such studies are rare in medical informatics as well. Only 4 of 27 studies of clinical decision support systems reported in 35 papers were multimethod, a strategy those researchers found valuable (Kaplan, 2001). Almost no studies directly address how qualitative and quantitative data were triangulated to produce results (Ammenwerth et al., 2003). In this chapter, I describe two early evaluation studies using a combination of qualitative and quantitative methods. One is of a pioneering computerbased patient record and clinical decision support system, ProblemOriented Medical Information System (PROMIS). It also is an exemplary and possibly the first evaluation to use ethnographic methods in medical informatics. The other is of a clinical laboratory information system. These studies are unusual in that researchers produced results they would not have achieved if they had not combined methods. One review cites only one other study that does this in the field of information systems (Gallivan, 1977). I end by describing a recent study that also combined qualitative and quantitative methods to evaluate an automated telephone advisory system. This study and the clinical laboratory information system study are unusual, too, in another respect. They each involve comparing groups using the same technology. Such studies could point to considerations other than technology per se that are important in how individuals react to and use those technologies, but there are few of them (Kaplan & Shaw, 2004). There are other multimethod studies, and other studies that combine qualitative and quantitative methods. I by no means wish to slight them but feel I cannot do them justice because I simply do not have the same depth of knowledge about them. Having worked with the lead author of the PROMIS study on another



project, and having conducted the two latter studies, means I can write about them in ways that I cannot do for excellent studies done by others.

Computer-Based Medical Records and Clinical Decision Support System: PROMIS PROMIS began in the 1960s by microbiologist, clinician, and educator Lawrence L. Weed, who soon was invited to develop it at the University of Vermont College of Medicine. It was a forward-looking and ambitious undertaking intended to replace medical records and guide clinical care in any medical practice. Weed‘s inspiring vision and evangelical promotion of his ideas influenced many others, and the concepts embedded in PROMIS spawned similar endeavors elsewhere. Weed intended the system as an active participant in medical practice as a way of overcoming problems of reliance on clinicians‘ memories and other difficulties of clinical practice. It embodied and extended the philosophy behind the Problem-Oriented Medical Record (POMR) that Weed already had developed. PROMIS enforced compliance with the POMR, volunteered information to guide clinical care, and required users to explain departures from programmed protocols. The system had built into it Weed‘s vision of team practice by requiring input and participation from members of the clinical team. The system also actively guided care; PROMIS volunteered information to instruct and guide clinical decisions and required users to justify deviations from programmed protocols. PROMIS was extensively tested and evaluated on a medical ward at the medical center at the University of Vermont (Fischer, Stratman, & Lundsgaarde, 1980; Lundsgaarde, Fischer, & Steele, 1981). This early evaluation study is exemplary both methodologically and for the insights on people, organization, and social issues it produced. The study described how the implementation of a new medical record and clinical guidance system was related to issues concerning professional roles and status; change management; user involvement; organizational communication; and relationships between the structure and organization of the medical record, philosophy of healthcare delivery, and clinical work-all still important in


202 Kaplan

electronic record and clinical decision support systems. Initially there was strong support for PROMIS, as evidenced by the University of Vermont’s switching to the POMR and desiring to have PROMIS developed and used there. Nevertheless, despite Weed‘s efforts and the potential that PROMIS held, it was removed from the ward on which it was tested and it has not been widely disseminated or used. The evaluation provides insight into why this happened. Ethnographic field methods were used to describe the local setting and to document the events surrounding the implementation. The research team deliberately mixed methods so they could check one data source against another, for example, to compare an individual‘s report of his behavior with behavioral observations. The study, therefore, used a variety of qualitative and quantitative methods to compare two medical wards, one using PROMIS and the other using manual record keeping. The main methods used were participant observation and informal interviewing, systematic behavioral observations, and structured interviews and questionnaires. On both wards, users were interviewed and observed, and time-activity sampling was performed. All 146 physicians affiliated with the Department of Medicine were surveyed and 100 responded to this fixed choice questionnaires. The evaluators reported that meaningful interpretation of the attitudinal survey relied on insights gained through interviews and observations. Respondents were enthusiastic about the POMR but not the computerized version. Although they wrote comments that tended to emphasize the negative aspects of PROMIS, they overwhelmingly supported the POMR, but there was a general attitude of ambivalence towards PROMIS during the entire demonstration project. It is significant that interview and observational data did not converge. Observation indicated that house staff circumvented using PROMIS. Interviews indicated that house staff felt that PROMIS adversely affected their relationships with patients, changed staff communication patterns, and increased the amount of time spent in record keeping activities. However, time-activity sampling showed that PROMIS did not substantially effect the amount of time house staff spent on these activities, thereby raising the question of why house staff thought that it did. If the evaluators had used only time activity study data, they would have



reported no differences between the PROMIS ward and the control ward. Had they used only observation, they would not have known why house staff was circumventing using the system. Had they used only interview data, they would, instead, have reported differences. Instead, the discrepancy in data to an explanation why house staff‘s perceptions differed from the time-activity measures. The evaluators explained that house staff‘s negative perceptions together with their short time on the experimental ward and the instability of a system under development may have led to their lack of facility with PROMIS, to their using the system in ways that were counter-productive, and to their not becoming comfortable with it. Because using multiple methods gave the evaluators rich and detailed data, they provide enough description to so that other explanations also may be developed from it. Weed’s intentions to revolutionize medicine through the use of PROMIS conflicted with dominant professional values and norms. This occurred in several ways. First, Weed built into PROMIS his vision of how medicine should be practiced and medical records kept. Part of that vision included more equal participation in healthcare by various members of the healthcare team. Increased emphasis on others’ participation contributed to house staff’s sense that their status was lowered with respect to other clinicians. Another aspect of the PROMIS vision included guiding clinical care according to built-in practice norms, which further violated physicians’ autonomous status. Further, PROMIS was developed and implemented with active involvement of nursing staff but with little involvement of medical staff. Nurses’ needs were focused on more than those of Weed’s physician colleagues. Weed also worked with nurses during the change process. It was nurses who were most enthusiastic about PROMIS. PROMIS enhanced the professional role of radiologists, pharmacists, and especially nurses, at what seemed to be the expense of medical house staff. Not surprisingly, then, PROMIS was more readily supported and accepted by these groups than by house staff. House staff, therefore, experienced using PROMIS as negatively affecting communication among physicians, record keeping, and relationships with patients, regardless of time-activity data to the contrary. Medical staff have primary decision-making power in a teaching hospital, and PROMIS eventually was removed.


204 Kaplan

Clinical Laboratory Information System The value of mixing qualitative and quantitative methods also is illustrated by a study of a clinical laboratory computer information system used by different laboratories within one department of an academic medical center. In this study, I worked with another researcher to collect data from multiple sources using several different methods. This provided us with rich data that led to puzzles and discrepancies that required resolution. Resolving these resulted in new theory development (Kaplan, 1986, 1987; Kaplan & Duchon, 1988, 1989). Our quantitative methods involved a survey questionnaire designed to assess the impact of the computer system on work in the laboratories. We gathered qualitative data from interviews, observations, and open-ended questionnaire questions. Statistical analysis of the survey data initially revealed no differences among laboratory technologists’ responses. Qualitative data analysis of their answers to open-ended questions indicated that laboratory technologists within each laboratory differed in their reactions to the system, as did laboratories as a whole. Some focused on work increases, whereas others emphasized improved laboratory results reporting and service. Although the quantitative survey data provided no apparent reason for these differences, the qualitative data did. As in the PROMIS study, discrepancies between the qualitative and quantitative data led us to further investigation, and, ultimately, to new theory. Qualitative and quantitative data at first seemed not to agree. The quantitative data initially indicated no differences among laboratories in their response to the computer system, yet differences were evident in the open-ended survey question data. Discrepancies also occurred within these data because technologists in the same laboratory disagreed over whether the computer system was a benefit. Rather than assuming that some technologists simply were wrong, or that either the qualitative or quantitative data were in error, we sought an explanation to allow for all these responses. Our initial clue to explaining these discrepancies arose when we explored the puzzle presented by interviewees repeatedly saying the computer system would not change laboratory technologists’ jobs but that it would change what technologists did. To address this puzzle, we developed hypothesizes and tentative theories to explain how the interviewees might not see a contra-



diction in their statements. We then reanalyzed quantitative survey data, with a focus on how the job of a laboratory technologist was conceptualized both by interviewees and respondents to open-ended survey questions. We used factor analysis to identify several ways in which both laboratories and laboratory technologists differed. We also developed a new construct, “job orientation,” as a measure of our finding from the qualitative data that different technologists had different views of their jobs, and that these different views affected their attitudes toward the computer system. According to the job orientation model, technologists (and also laboratories as a whole) who were results or product orientated saw the system enhanced their jobs by improving results reporting. Technologists who were “process oriented” thought the system interfered with their jobs, which, for them, was performing laboratory tests. This model was congruent with the qualitative data. Neither the researchers nor the laboratory personnel expected this finding. Ostensibly, all technologists had the same jobs. In the original quantitative analysis, few differences were discernable among technologists or among laboratories from the quantitative data, because standard quantitative measures of job characteristics assumed a uniformity of job situations and perceptions. However, this uniformity did not exist, as revealed in qualitative data that identified technologists’ own views of their jobs and of the system. This study illustrates several advantages to mixing methods. It was not possible to design, in advance, a quantitative study that would have tested the right hypotheses, because appropriate hypotheses could not be known in advance. Having multiple data sources and means of analysis enabled us to cross-validate our results by comparing their data. More than that, discrepant results from analyzing different data sources motivated us to find an explanation both of the data and the discrepancy. Because of the qualitative data, we could see how individuals construed the information technology, their jobs, and the interaction between the laboratory computer information system and their jobs. The qualitative data, therefore, enabled us to make sense of our quantitative findings and to reanalyze the quantitative data in a way that produced interesting results. The qualitative data helped us explain why the quantitative results were as they were, generating productive hypotheses and theory. Reconciling all the data contributed to a much richer final interpretation that resulted in a theory of how views of one’s job and views of a computer system are related.


206 Kaplan

Telephone Advisory System: Telephone-Linked Care A recent study also illustrated how combining qualitative and quantitative methods can produce results that otherwise would not have been obtained. This evaluation involved individuals’ reactions to using a voice response computer-based telecommunications system, Telephone-Linked Care (TLC), for individualized counseling about dietary and exercise behaviors. A twoarm randomized clinical trial compared two TLC health behavior change programs, one that promoted healthy eating and the other that promoted regular physical activity. To study individuals’ reactions to the system, we used a combination of quantitative and qualitative methods. After study participants completed a clinical outcomes trial of the system, they were given a structured exit survey questionnaire to obtain information about their attitudes towards the TLC system. In addition, we did ethnographic open-ended interviews with a selected participants (Glanz et al., 2002; Kaplan, Farzanfar, & Friedman, 2003). The survey data indicated that participants were fairly satisfied with the system and regarded the system as helpful. Participants also reported fairly high ratings of the system on a number of constructs formed by combining items on the questionnaire. In response to an open-ended survey question concerning benefits gained from using TLC, participants mentioned that TLC made them aware, showed them benefits, reminded them about healthy behaviors, changed their habits, and gave them confidence or motivated them. Interviewees mentioned similar benefits. However, interview data showed that some individuals were favorable to the system while others were not. When interviews were analyzed separately for interviewees, according to which version of the system they used, data indicated that, for the most part, participants tended to differ in their responses to TLC according to whether they used the diet or exercise change versions of the system. While those using the diet-change version generally said they found the system helpful and fun to use, those using the exercise-change version reported that the system did not help them, made them feel guilty, was inflexible, and talked down to them. The interviews provided the first indication that users of each version of the system differed in their responses to it. The interviews also suggested that



these differences had to do with the different relationships individuals formed with TLC. Originally, we had not planned to compare survey responses according to which version of the system participants used. We had thought simply of TLC as one system. In light of the interview data, we did this comparison. Individuals randomized to the diet-change arm of the trial consistently gave the system higher ratings than those in the exercise-change arm. Multivariate data analysis showed that the amount of contact with the TLC system, reflected in the number of calls, and the treatment group (diet vs. exercise), were significant predictors of reported satisfaction with and perceived helpfulness of the system, after controlling for individual characteristics. There is no obvious explanation for the finding that people who used the diet-change version of the system reacted more favorably than did users of the exercise-change version. Nevertheless, the interviews shed light on how their reactions differed, and the attitudinal surveys would not have been analyzed by comparing participants in each arm of the study, had the interview data not indicated a difference. Thus, we documented differences in reactions among individuals who differed only in the version of TLC they used. We explored reasons for these different reactions through a multimethod research design that combined ethnographic interviewing with a survey questionnaire. Although in this study data converged, the study illustrates how ethnographic interviewing can help in analyzing and interpreting survey results and provide additional dimensions to understanding why individuals react as they do.

Conclusions As these studies indicate, using a variety of evaluation research methods provides several advantages. A combination of evaluation research methods can address the diverse and diffuse nature of system effects and enable researchers to combine results in a way that maximizes understanding of causal links. Through multiple methods, a variety of data is collected, each set of which might provide partial information needed for a complete evaluation. In this way, putting together data collected by a variety of methods from a variety of sources strengthens the robustness of research results through a process known as “triangulation.” In addition, as illustrated by the laboratory Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

208 Kaplan

information system and telephone advisory system studies, the use of multiple methods allows for comparing differences among users of the same system. Lastly, a multiplicity of methods can help ensure that issues and concerns that were not included in the preliminary design can be integrated into an evaluation later on. This can happen in two ways. First, multiple methods increase the chances for such issues and concerns to emerge during the course of an evaluation. Second, study design then can be modified in light of evaluation findings. Combining qualitative with quantitative methods, therefore, makes it more possible to investigate the complex web of technological, economic, organizational, and behavioral issues surrounding a new information system (Kling & Scacchi, 1982). However, multimethod studies, particularly those involving both quantitative and qualitative methods, can be more difficult than studies with simpler designs. Expertise is needed in different kinds of data collection and analysis. Moreover, the volume, lack of structure, and richness of the data increase the difficulty and complexity of managing data collection and doing data analysis. For these reasons, research partnerships or teams have become more common. Of the information systems case studies mentioned previously in this chapter, 72% were conducted by a team of researchers (Dubé & Paré, 2003). In medical informatics, there is a growing tendency to form multidisciplinary evaluation research teams that incorporate social scientists and management specialists into implementation and evaluation efforts. People from different disciplinary backgrounds bring both methodological and content expertise. Multidisciplinary teams can produce robust and comprehensive evaluations (Kaplan & Shaw, 2004). Whether done by teams or individual researchers, multimethod research has flourished and produced significant results.

References Ammenwerth, E., Iller, C., & Mansmann, U. (2003). Can evaluation studies benefit from triangulation? A case study. International Journal of Medical Informatics 70, 237-248. Cook, T.D. & Reichardt, C.S. (1979). Qualitative and quantitative methods in evaluation research. Beverly Hills: Sage Publications.



Dubé, L, & Paré, G. (2003). Rigor in information systems positivist case research: Current practices, trends, and recommendations. MIS Quarterly 27(4), 597-635. Fischer P. J., Stratman, W. C., & Lundsgaarde, H. P. (1980). User reaction to PROMIS: Issues related to acceptability of medical innovations. In J. T. O’Neill (Ed.), Proceedings of the fourth annual Symposium on Computer Applications in Medical Care (pp. 1722-1730). Silver Spring, MD: IEEE Computer Society Press. (Reprinted from: Use and impact of computers in clinical medicine, pp. 284-301, by J. G. Anderson & S. J. Jay, Eds. 1987, New York: Springer) Gallivan, M. (1977). Value in triangulation: A comparison of two approaches for combining quantitative and qualitative methods. In A. S. Lee, J. Liebenau, & J. I. DeGross (Eds.), Qualitative method in information systems. London: Chapman & Hall. Glanz, K., Shigaki, D., Farzanfar, R., Pinto, B., Kaplan, B., & Friedman, R. H. (2002). Participant reactions to a computerized telephone system for nutrition and exercise counseling. Patient Education and Counseling, 49(2), 157-163. Greene, J. C., & Caracelli, V. J. (1997) (Eds.). Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms. New Directions for Evaluation, no. 74 (Summer 1997). San Francisco: Jossey-Bass. Jick, T.D. (1983). Mixing qualitative and quantitative methods: Triangulation in action. In J. Van Maanen (Ed.), Qualitative methodology (pp. 135-148). Beverly Hills, CA: Sage Publications. Kaplan, B. (1986). Impact of a clinical laboratory computer system: Users’ perceptions. In R. Salamon, B. I. Blum, & M. J. Jørgensen (Eds.), Medinfo 86: Fifth World Congress on Medical Informatics (pp. 1057-1061). Amsterdam: North-Holland. Kaplan, B. (1987). Initial impact of a clinical laboratory computer system: Themes common to expectations and actualities. Journal of Medical Systems, 11, 137-147. Kaplan, B. (1997). Addressing organizational issues into the evaluation of medical systems. Journal of the American Medical Informatics Association 4(2), 94-101.


210 Kaplan

Kaplan, B. (2001). Evaluating informatics applications-Clinical decision support systems literature review. International Journal of Medical Informatics 64(1), 15-37. Kaplan, B. (2004). Organizational evaluation of information resources in healthcare. In C. P. Friedman and J. C. Wyatt (Eds.). Evaluation methods in medical informatics, (2nd Ed.). New York: Springer, in press. Kaplan, B. & Duchon, D. (1989). A job orientation model of impact on work seven months post implementation. In B. Barber, D. Cao, D. Qin, and G. Wagner (Eds.), Medinfo 89: Sixth World Congress on Medical Informatics, (pp. 1051-1055). Amsterdam: North-Holland. Kaplan, B. & Duchon, D. (1988). Combining qualitative and quantitative methods in information systems research: A case study. MIS Quarterly, 12, 571-586. Kaplan, B. & Duchon, D. (1989). A job orientation model of impact on work seven months post implementation. In B. Barber, D. Cao, D. Qin, & G. Wagner (Eds.), Medinfo 89: Sixth World Congress on Medical Informatics (pp. 1051-1055). Amsterdam: North-Holland. Kaplan, B, Farzanfar, R, & Friedman, R. H. (2003). Personal relationships with an intelligent interactive telephone health behavior advisor system: A multimethod study using surveys and ethnographic interviews. International Journal of Medical Informatics 71(1), 33-41. Kaplan, B. & Maxwell, J. A. (2004). Qualitative research methods for evaluating computer information systems. In J. G. Anderson, C. E. Aydin, & S. J. Jay (Eds.), Evaluating healthcare information systems: Approaches and applications (2nd ed.). Newbury Park, CA: Sage. Kaplan, B., & Shaw, N. T. (2004). Future directions in evaluation research: People, organizational, and social issues. Methods of Information in Medicine, 43(3/4). Kidder, L.H. & Fine, M. (1987). Qualitative and quantitative methods: When stories converge. In M.M. Mark and R.L. Shotland (Eds.), Multiple methods in program evaluation (pp. 57-75). San Francisco: JosseyBass.



Kling, R., & Scacchi, W. (1982). The web of computing: Computer technology as social organization. In M. C. Yovitz (Ed.), Advances in computers (pp. 2-90). New York: Academic Press. Lundsgaarde, H. P., Fischer, P. J., & Steele, D. J. (1981). Human problems in computerized medicine. Lawrence, KS: The University of Kansas. Maxwell, J.A., Bashook, P.G., & Sandlow, L.J. (1986). Combining ethnographic and experimental methods in educational research: A case study. In D.M. Fetterman and M.A. Pitman (Eds.), Educational evaluation: Ethnography in theory, practice, and politics (pp. 121143). Beverly Hills, CA: Sage Publications. Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks CA: Sage Publications. Tashakkori, A. & Teddlie, C. (Eds.) (2002), Handbook of mixed methods in social and behavioral research, pp. 241-271. Thousand Oaks, CA: Sage Publications. Trend, M.G. (1979). On the reconciliation of qualitative and quantitative analyses: A case study. In T.D. Cook and C.S. Reichardt (Eds.), Qualitative and quantitative methods in evaluation research (pp. 68-86). Beverly Hills, CA: Sage Publications.


212 Ammenwerth, Gräber, Bürkle & Iller

Chapter XIII

Evaluation of Health Information Systems: Challenges and Approaches Elske Ammenwerth University for Health Sciences, Medical Informatics & Technology, Austria Stefan Gräber University Hospital of Saarland, Germany Thomas Bürkle University of Erlangen, Germany Carola Iller University of Heidelberg, Germany

Abstract This chapter summarizes the problems and challenges that occur when health information systems are evaluated. The main problem areas presented are the complexity of the evaluation object, the complexity of an evaluation project, and the motivation for evaluation. Based on the analysis of those problem areas, the chapter then presents recommendations of how to address them. In particular, it discusses in more detail what Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Evaluation of Health Information Systems

213

benefits can be obtained from applying triangulation in evaluation studies. Based on the example of the evaluation of a nursing documentation system, it shows how both the validation of results and the completeness of results can be supported by triangulation. The authors hope to contribute to a better understanding of the peculiarities of evaluation in healthcare, and to provide information how to overcome them.

Introduction It is hard to imagine healthcare without modern information and communication technology (ICT). It is evident that the use of modern information technology (IT) offers tremendous opportunities to reduce clinical errors, to support healthcare professionals, and to increase the efficiency of care, and even to improve the quality of patient care (Institute of Medicine, 2001). However, there are also hazards associated with ICT in healthcare: Modern information systems (ISs) are costly, their failures may cause negative effects on patients and staff, and possibly, when inappropriately designed, they may result in healthcare professional’s spending more time with the computer than with the patient. This all could have a negative impact on the efficiency of patient care. Therefore, a rigorous evaluation of IT in healthcare is recommended (Rigby, 2001) and is of great importance for decision makers and users (Kaplan & Shaw, 2002). Evaluation can be defined as the decisive assessment of defined objects, based on a set of criteria, to solve a given problem (Ammenwerth et al., 2004). The term ICT refers to technologies as such. Whether the use of these technologies is successful depends not only on the quality of the technological artifacts but also on the actors (i.e., the people involved in information processing and the organizational environment in which they are employed). ICT embedded in the environment, including the actors, is often referred to as an IS in a sociotechnical sense (Berg, Aarts, & van der Lei, 2003; Winter et al., 2001). Many different questions can lead the evaluation of IT. Within evaluation research, two main (and often rather distinct) traditions can be found: The objectivist (positivistic) and the subjectivistic tradition (Friedman &



Wyatt, 1997), which are related to the dominant use of either quantitative or qualitative methods (for details, see Chapter XII). Despite a large amount of published evaluation studies (e.g., van der Loo, 1995) found over 1,500 citations on evaluation of healthcare IT between 1967 and 1995, and Ammenwerth and de Keizer (2004) found 1,035 studies between 1982 and 2002; many authors report problems during evaluation. One of the main problems frequently discussed is the adequate choice of evaluation methods. While objectivistic researchers tend to concentrate on quantitative methods, subjectivistic researchers mainly rely on qualitative methods. Sometimes, a mixture of methods is applied. For example, qualitative methods are used to prepare quantitative studies, or quantitative measurements are used to support qualitative argumentation. However, there is still usually one tradition which dominates typical evaluation studies, leading to a focus either on quantitative or qualitative methods. Many researchers point to the fact that this domination of one method or tradition may not be useful, but that a real integration of various methods from both traditions can be much more helpful to get comprehensive answers to given research questions. The integration of the complementary methods (and even beyond this, of data sources, theories and investigators), is discussed under the term triangulation. In this chapter, we first want to review some of the underlying reasons that make evaluation of healthcare IT so difficult. We will structure the problems into three main problem areas: the complexity of the object of evaluation, the complexity of the evaluation project, and the motivation to perform evaluation. We will discuss means how to overcome the discussed problems. As one more detailed example, we then discuss what benefits can be obtained from applying triangulation in an evaluation study. Based on the example of the evaluation of a nursing documentation system, we show how both the validation of results and the completeness of results can be supported by triangulation.



215

Typical Problems in Evaluation of IT in Healthcare First Problem Area: Complexity of the Evaluation Object When understanding IT as part of the IS of an organization, it is clear that evaluation requires not only an understanding of computer technology, but also of the social and behavioral processes that affect and are affected by the technology. This complexity of the evaluation objects has some important consequences. First, the introduction of IT takes time. It is not enough to implement the technology and then to immediately measure the effects. Users and workflow need a lot of time to get used to new tools and to completely exploit the new possibilities (Palvia, Sharma, & Conrath, 2001). Thus, evaluation results can develop and change during this first period of use. Then, even after an introduction period, the evaluation object may steadily change (Moehr, 2002; moving evaluation target). For example, the use of IT may be affected by changes in work organization, or in staff. It is nearly impossible to reach a stable situation in a flexible healthcare environment which makes evaluation results dependant of the point in time where the evaluation took place. In addition, each IS in our definition is quite unique. While the IT may be similar in various departments, workflow, users and used functionality may be different. In addition, the organization of its introduction as well as the overall user motivation may differ. Thus, even when the same IT is introduced, its effects may be varying (Kaplan & Shaw, 2002). The influence of such factors on the results of an evaluation study is often hard to disentangle (Wyatt, 1994), posing the problem of external validity (Moehr, 2002): Many evaluation studies may be valid only for the particular institutions with their specific IS. The complexity of the evaluation object is an inherent attribute in healthcare IT evaluation and cannot be reduced. However, there are some ways to handle this problem in evaluation studies. To address the problem of external validity, the IT and its environment that is going to be evaluated should be defined in detail before the beginning of the study. Not only the software and hardware used should be described, but also the number of users and their experience and motivation, the way IT is introduced and used, the general technical infrastructure (e.g., networks) and any further aspects that may influence the usage of IT and its effects. The functionality and the way it is really used should also be of importance. Only this information may allow interpretation of the study results



and comparison of different locations. Then, to address the problem of the moving evaluation target, all changes in the IT and its interaction with the users should be carefully documented during the study. For example, changes in workflow, in staffing, or in hardware or software should be documented with reference to the ongoing evaluation. This permits the explanation of changes and differences in effects measured during the study period. Another approach to address the problem of the moving evaluation target may be to define smaller evaluation modules. This would allow the evaluation design or evaluation questions to be adapted to changes in the environment. Each module answered a question related to a defined phase of the introduction of the IT. In addition, an evaluation must be planned in a long-term perspective in order to allow the users and the environment to integrate the new IT. Hence enough resources for long-term evaluation (e.g., over several months or even years) should be available.

Second Problem Area: Complexity of the Evaluation Project Evaluation of IT is performed in the real and complex healthcare environment, with its different professional groups, and its high dependency on external influences such as legislation, economic constraints, or patient clientele. This poses problems to the evaluation projects, meaning the planning, executing and analyzing of an IT evaluation study. For example, the different stakeholders often have different conceptions and views of successful IT (Palvia et al., 2001). The different stakeholder requirements can serve as a frame of reference for evaluation during the early phases of the IT life cycle, but also guide evaluations during later phases. In each case, multiple-stakeholder views may lead to a multitude of (possibly conflicting) evaluation questions (Heathfield et al., 1999). Depending on the point of view adopted, the evaluation will require different study designs and evaluation methods. The evaluation researcher must decide, for example, on the evaluation approach, on the adequate evaluation methods (e.g., quantitative vs. qualitative), and on the study design (e.g., RCT vs. observational study). Each has its own advantages and drawbacks ( Frechtling, 1997; Moehr, 2002), making their selection a rather challenging endeavor. This multitude of possible evaluation questions and available evaluation methods makes the planning of an evaluation study quite complex. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


217

The complexity of the evaluation project has several consequences. First, the overall success of IT is elusive to define (Palvia et al., 2001), and it is therefore often difficult to establish clear-cut evaluation criteria to be addressed in a study (Wyatt, 1994). Each stakeholder group may have individual questions, and a universal evaluation in terms of absolute or relative benefits is usually not feasible (or, from a more subjectivistic point of view, not even possible). It is also unrealistic to expect that the IT itself will have a direct and easy to measure effect on the outcome quality of patient care, like in a drug trial (Wyatt, 1994). Thus, indirect measures are often used such as user satisfaction or changes of clinical processes, which, however, do not give a really complete picture of the benefits of IT. Often, changes in the evaluation questions may occur during the study (e.g., based on intermediate evaluation results, new insights, changes in stakeholders’ opinions, or changes of the IT [scope creep]; Dewan & Lorenzi, 2000). Changes in study questions, however, may be difficult to balance with study resources. Finally, the selection of adequate evaluation designs and evaluation methods is often regarded as a problem during evaluation studies. Evaluators may not be sufficiently aware of the broadness of available approaches, or be too deeply embedded in either the qualitative or the quantitative paradigm, neglecting the possible contributions of the complementary approach. Thus, inadequate methods or study designs may be chosen which may not be able to answer the original study questions. The following suggestions may be useful in order to deal with the complexity of the evaluation project. First, it is recommended that the general intention of the evaluation and the starting point should be agreed early on. In principle, evaluation should start before the new IT is implemented, in order to allow for early gathering of comparative data, and then continue during all phases of its life cycle (VATAM, 2000). Then, the areas of evaluation should be restricted to aspects which are of most importance to the involved stakeholders, and which can be measured with the available resources. A complete evaluation of all aspects of a system (such as economics, effectiveness, and acceptance) is usually not feasible. A balance between the resources of a study and the inclusion of the most relevant aspects has to be found. In addition, sufficient time should be invested into the definition of relevant study questions. All involved stakeholder groups should discuss and agree on the goals of evaluation (VATAM, 2000). The selected study questions should be relevant for decision-making with regard to introduction, operation or justification of IT (Ammenwerth et al., 2004). Conflicting goals should be discussed and solved, as they are not only problematic for an evaluation, but for the overall manage-



ment of new IT. Fourth, when new evaluation questions emerge during the study, they should only be included in the study design when it is possible without creating problems. Otherwise, they should be tackled in consecutive studies. Each shift in evaluation questions must thoroughly be documented. For each study question, adequate methods must be chosen. A triangulation of methods may be useful to best answer the study questions (Heathfield, Pitty, & Hanka, 1998). For example, to address the effects of a nursing documentation system, both quantitative methods (time measurement, user acceptance scales, documentation quality measurement) as well as qualitative methods (focus group interviews) were used. We will discuss this example later on in more detail.

Third Problem Area: Motivation for Evaluation An evaluation study can normally only be conducted when there is sufficient funding, and a sufficient number of participants (e.g., staff members, wards). Both these variables depend on the motivation of stakeholders (e.g., hospital management) to perform an evaluation. Sometimes, this motivation is not very high, because, for example, of fear for negative outcome, or of fear for revealing deficiencies of already implemented technology (Rigby, Forsström, Roberts, & Wyatt, 2001). In addition, the introduction of IT in an organization is a deep intervention that may have large consequences. It is thus often very difficult to organize IT evaluation in the form of an experiment, and to easily remove the system again at the end of the study in case the evaluation was too negative. Even with a motivated management, it may be difficult to find suitable participants. Participating in a study usually requires some effort from the involved staff. In addition, while the users have to make large efforts to learn and use a new, innovative system, the benefit of joining a pilot study is usually not obvious (the study is conducted in order to investigate possible effects), but participation may even include some risks for the involved staff such as disturbances in workflow. In summary, due to the given reasons, the hospital management as well as involved staff members is often reluctant to participate in IT evaluation studies. The described problem has consequences for the study. Without the support and motivation of the stakeholders to conduct an evaluation study, it will be difficult to get sufficient resources for an evaluation and sufficient participants



219

willing to participate. Second, due to the given problems, the study organizer tends to recruit any participant who volunteers to participate. However, those participants may be more motivated to participate than the “normal” user. This leads to the well-known volunteer effect, where results are better when participants are motivated. In addition, evaluation results are not only important for the involved units, but also for the overall organization or for similar units in other organizations. To allow transfer of results, the pilot wards or pilot users must be sufficiently representative for other wards or users. But, as each IT within its environment is quite unique (see Problem Area 1); it is difficult to find comparable or representative participants. To increase the number of participants, two approaches should be combined. First, the responsible management should be informed and motivated to support the study. The result of an evaluation study may be important to decide on new IT, and to support its continuous improvement. Then, the possible participants could be directly addressed. It should be made clear that the study provides the opportunity to influence not only the future development of IT in healthcare but also the own working environment. User feedback of study results may act as an important driving force for users to participate in the study. Offering financial compensation or additional staff for the study period may help to gain support from participants and from management. As in clinical trials, multicentric studies should be considered (Wyatt & Spiegelhalter, 1992). This would largely increase the number of available participants. This means however, that study management requires much more effort. A multicentric study design is difficult when the environment is completely different. In addition, the variation between study participants will be bigger in multicentric trials than in single-center ones. This may render interpretation and comparison of results even more difficult (cp. discussion in Problem Area 1).

Summary of General Recommendations The above discussed problems and approaches will now be summarized in a list of 12 general recommendations for IT evaluation in healthcare: 1. 2.

Evaluation takes time; thus, take your time for thorough planning and execution. Document all of your decisions and steps in a detailed study protocol. Adhere to this protocol; it is your main tool for a systematic evaluation.



3. 4.

Strive for management support, and try to organize long-term financial support. Clarify the goals of the evaluation. Take into account the different stakeholder groups. Dissolve conflicting goals.

5.

Reduce your evaluation questions to an appropriate number of the most important questions that you can handle within the available time and budget. If new questions emerge during the study, which cannot easily be integrated, postpone them for a new evaluation study. 6. Clarify and thoroughly describe the IT object of your evaluation and the environment. Take note of any changes of the IT and its environment during the study that may affect results. 7. Select an adequate study design. Think of a stepwise study design. 8. Select adequate methods to answer your study questions. Neither objectivist nor subjectivist approaches can answer all questions. Take into account the available methods. Consider being multimethodic and multidisciplinary, and consider triangulation of methods, data sources, investigators, and theories. Strive for methodical (e.g., biometrics) advice. 9. Motivate a sufficient number of users to participate. Consider multicentric trials and financial or other compensation. 10. Use validated evaluation instruments wherever possible. 11. Be open to unwanted and unexpected effects. 12. Publish your results and what you learned to allow others to learn from your work. One of the most discussed aspects is the selection of adequate methods and tools (Point 6) and, here especially, the adequate application of multimethodic and multidisciplinary approaches (Ammenwerth et al., 2004). The interdisciplinary nature of evaluation research in medical informatics includes that a broad choice of evaluation methods is available for various purposes. In Sections II and III of this book, several distinct quantitative and qualitative evaluation methods have been presented and discussed in detail. All of them have their particular application area. However, in many situations, the evaluator may want to combine the methods to best answer the evaluation questions at hand. Especially in more formative (constructive) studies, a combination of methods may seem necessary to get a more complete picture of a situation. To support



221

this, the method of triangulation has been developed and will now be presented in more detail.

The Theory of Triangulation The term triangulation comes from navigation and means a technique to find the exact location of a ship base on the use of various reference points. Based on this idea, triangulation in evaluation means the multiple employments of data sources, observers, methods, or theories, in investigations of the same phenomenon (Greene & McClintock, 1985). This approach has two main objectives: First, to support a finding with the help of the others (validation); second, to complement the data with new results, to find new information, to get additional pieces to the overall puzzle (completeness; Knafl & Breitmayer, 1991). Triangulation is, based on work by Denzin (1970), usually divided into the following four types, which can be applied at the same time: •

•

•

•

Data triangulation: Various data sources are used with regard to time, space, or persons. For example, nurses from different sites are interviewed, or questionnaires are applied at different times. Investigator triangulation: Various observers or interviewers with their own specific professional methodological background take part in the study, gathering and analyzing the data together. For example, a computer scientist and a social scientist analyze and interpret results from focus group interviews together. Theory triangulation: Data is analyzed based on various perspectives, hypotheses or theories. For example, organizational changes are analyzed using two different change theories. Methods triangulation: Various methods for data collection and analysis are applied. Here, two types are distinguished: within-method triangulation (combining approaches from the same research tradition), and between-method triangulation (combining approaches from both quantitative and qualitative research traditions, also called across-method triangulation). For example, two different quantitative questionnaires may



be applied to access user attitudes, or group interviews as well as questionnaires may be applied in parallel. It should be noticed that the term triangulation is only used when one phenomenon is investigated with regard to one research question. The term triangulation is often seen strongly related to the term multimethod evaluation; because methods triangulation is seen as the most often used triangulation approach. However, as we want to stress, it is not limited on the combination of methods, but also describes combination of data sources, investigators, or theories.

Example: Triangulation during the Evaluation of a Nursing Documentation System Background of the Study Nursing documentation is an important part of clinical documentation. There have been some attempts and discussions on how to support the nursing documentation using computer-based documentation systems. In 1997, Heidelberg University Medical Center started to introduce a computer-based nursing documentation system in order to systematically evaluate preconditions and consequences. Four different (psychiatric and somatic) wards were chosen for this study. In the following paragraphs, we will concentrate on those parts of the study that are relevant for the triangulation aspects of the study. Please refer to other publications for more details on methods and results, such as (Ammenwerth, Mansmann, Iller, & Eichstädter, 2003; Ammenwerth et al., 2001). Three of the four study wards had been selected by the nursing management for the study. On all three wards, the majority of nurses agreed to participate. Ward B volunteered to participate. The four study wards belonged to different departments. Wards A and B were psychiatric wards, with 21 resp. 28 beds; Ward C was a pediatric ward for children under two years of age, with 15 beds; Ward D was a dermatological ward, with 20 beds. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


223

Our study wards were quite different with regard to nursing documentation. In Wards A and B, a complete nursing documentation based on the principles of the nursing process–for details on nursing process, see, for example, Lindsey and Hartrick (1996)–had been established for several years. In contrast, in Wards C and D, only a reduced care plan was documented; documentation was mostly conducted in the ward office. Only in Ward C, major parts of documentation were also conducted in the patients’ rooms. The youngest staff member could be found in Ward D; the staff least experienced in computer use was in Ward C.

Study Design The software PIK (Pflegeinformations-und Kommunikationssystem, a German acronym for “nursing information and communication system”) was introduced on those four wards. The functionality covered the six phases of the nursing care process. The study period was between August 1998 and October 2001. Wards A and B started in 1998 with the introduction of the documentation system; Wards C and D joined in 2000. The study consisted of two main parts: The objective of the more quantitative study was to analyze the changes in the nurses’ attitudes with regard to nursing process, computers in nursing, and nursing documentation system, after the introduction of the computer-based system. Standardized, validated questionnaires were applied based on Bowman, Thompson, and Sutton (1983), for nurses’ attitudes on the nursing process; on Nickell and Pinto (1986), for computer attitudes; on Lowry (1994), for nurses’ attitudes on computers in nursing; and on Chin (1988) and Ohmann, Boy, and Yang (1997), for nurses’ satisfaction with the computer-based nursing documentation system. We carefully translated those questionnaires into German and checked the understandability in a prestudy. We used a prospective intervention study with three time measurements: approximately three months before introduction (“before”); approximately three months after introduction (“during”); and approximately nine months after introduction (“after”). The second part of the study was a more qualitative study. Here, the objective was to further analyze the reasons for the different attitudes on the wards. The quantitative study exactly described these attitudes, and the qualitative study was now intended to further explain those quantitative results. The qualitative study was conducted in February 2002, after the analysis of the quantitative



study was finished. In this qualitative study, open-ended focus group interviews were conducted with up to four staff members from each ward (most of them already have taken part in the quantitative study), with the three project managers from each department, and with the four ward managers from the wards. Open-ended means that the interviews were not guided by predefined questions. We used two general questions that started the interviews (e.g., “How are you doing with PIK?” “How was the introduction period”)? The rest of the interview was mostly guided by the participants themselves, with relatively little control exerted by the interviewers. All interviews were conducted by a team of two researchers. They took about one hour each. The interviews were audio taped and analyzed using inductive, iterative content analysis based on Mayring (1993). This means that the transcripts were carefully and stepwise analyzed, using the software WinMaxProf98. In the following paragraphs, only those results of the quantitative and qualitative study relevant for the triangulation aspects of the study will be presented. Please refer to the already mentioned study publications for more details.

Results of Quantitative Analysis of User Attitudes All in all, 119 questionnaires were returned: 23 nurses answered all three questionnaires, 17 nurses answered two, and 16 nurses answered one questionnaire. The return rates were 82% for the first questionnaire, 86.5% for the second questionnaire, and 90.2% for the third questionnaire. A quantitative analysis of the individual items of the questionnaires revealed unfavorable attitudes, especially in Ward C. In both Wards C and D, the nurses stated that the documentation system does not “save time” and does not “lead to a better overview on the course of patient care.” In addition, in Ward C, the nurses stated that they “felt burdened in their work” by the computer-based system and that the documentation system does not “make documentation easier.” In Wards A and B, the opinions with regard to those items were more positive. The self reported daily usage of the computer-based documentation system was quite similar among all wards: about 1 to 2 hours a day during the second and third questionnaires, with highest values in Ward B and lowest values in Ward A. The self-confidence with the system, as stated by the nurses, was



225

rather high on all wards during both the second and third questionnaire. The mean values were between 3 and 3.7 during the second questionnaire and between 3.4 and 3.8 during the third questionnaire (1=minimum, 4=maximum). Statistical analysis revealed that the overall attitude on the documentation system during the third questionnaire was positively correlated to the initial attitude on the nursing process, to the attitude on computers in general and to the attitude on computers in nursing. Both computer attitude scores were in turn positively correlated to the years of computer experience. For details, see (Ammenwerth, Mansmann, et al., 2003). Overall, the results of quantitative analysis pointed to a positive attitude on the computer-based nursing documentation already shortly after its introduction, which significant increase on three of the four wards later on. However, on ward C, the quantitative results revealed negative reactions, showing a heavy decline in the attitude scores during the second questionnaire. On ward C, the overall attitude of the computer-based system remained rather negative, even during the third questionnaire. What could be the reasons? In order to answer this question, a subsequent qualitative study was conducted.

Results of Qualitative Analysis of User Attitudes This part of the study was conducted as planned. Overall, about 100 pages of interview transcript were analyzed. Details of the interviews are published elsewhere (Ammenwerth, Iller, et al., 2003); we will summarize only the main points. In Ward C, some distinct features came up in the interviews that seem to have lead to low attitude scores at the beginning. For example, the nursing process had not been completely implemented before, thus the documentation efforts now were much higher. Documentation of nursing tasks covered a 24 hour day, due to the very young patients and their high need for care. Thus, the overall amount of documentation on Ward C was higher. Patient fluctuation was also highest in ward C. Nurses found it time-consuming to create a complete nursing anamnesis and nursing-care plan for each patient. The previous computer experience and number and availability of motivated key users was seen as rather low in Ward C. Then, during the introduction of the



nursing documentation system, the workload was rather high in Ward C due to staff shortage, which increased pressure on the nurses. Finally, and most important, nursing documentation had previously at least partly been carried out in the patients’ rooms. However, during our study, computers were installed only in the ward office. No mobile computers were available, which, according to the nurses, lead to time-consuming and inefficient double documentation. Interesting differences were found between the nurses and the project management. For example, the nurses stated in the interviews that they were not sufficiently informed on the new documentation system, while the project management stated to have offered information that had not been used. Another example is that the nurses felt that training was insufficient. In the opinion of the project management, sufficient opportunities had been offered. We will later see how this divergent information helps to complete the overall picture. In Ward D, the attitude on the documentation system was high in the interviews. The nurses saw benefits, especially in a more professional documentation, which would lead to a greater acknowledgment of nursing. Standardized care planning was seen to make care planning much easier, without reducing the individuality of the patient. Overall, the ward felt at ease while working with the new documentation system. In Wards A and B, the attitudes were also positive. The nurses stressed the better legibility of nursing documentation in the interviews. They said that time effort for nursing care planning was lower, but overall, time effort for nursing documentation was much higher than before. The interviews showed that the introduction period had been filled with anxiety and fear about new requirements for the nurses. Now, after some time, the nurses felt self-confident with computers. An interesting discussion arose on the topic of standardization. Most nurses felt that standardized care plans reduced the individuality of the care plans, and that they did not really reflect what is going on with the patient. Finally, those wards, too, mentioned insufficient teaching and support in the first weeks. These rather short summaries, from the interviews, should highlight some distinct features of the wards, showing similarities (e.g., on insufficient teaching and fears at the beginning), but also differences (e.g., on the question on standardized care plans or time effort).



227

Application of Triangulation in this Study After analysis of the quantitative study and the qualitative study, we now want to see how the different results can be put together to get a broader picture of the effects and preconditions of a nursing documentation system. We thus applied all four types of triangulation as described by Denzin (1970): •

•

•

•

Data triangulation: Various data sources were used: Within the quantitative study, data triangulation with regard to time was used as the questionnaires were submitted three times to the same users (data triangulation with regard to time). In addition, in the interviews, not only nurses but also project management and ward management were interviewed (data triangulation with regard to persons). Investigator triangulation: Within the qualitative study, the two interviewers had different backgrounds (one more quantitative coming from medical informatics, the other, more qualitative, coming from social science). Both acted together as interviewers, analyzed the transcript together, and discussed and agreed on results and conclusions. Theory triangulation: We learned from various complementing theories to better understand the results of our studies. For example, to explain the implementation phases, we took ideas both from the book of Lorenzi and Riley (1995; first-, middle-, and second-order change) as well as from the change theory of (Lewin, 1947; unfreezing, moving, refreezing phase). With regard to user evaluation, we used the technology acceptance model (TAM) of Davis (1993), and the task-technology-fit model (TTF) of Goodhue (1995). Methods triangulation: We applied between-methods triangulation by applying both quantitative questionnaires and qualitative focus-group interviews to investigate user’s attitudes.

As stated in the introduction, triangulation has two main objectives: To confirm results with data from other sources (validation of results) and to find new data to get a more complete picture (completeness of results). We will now briefly discuss whether triangulation helped to achieve those goals.



Validation of Results Validation of results is obtained when results from one part of the study are confirmed by congruent (not necessarily equal) results from other parts of the study. In our example, some parts of the study showed congruent results: First, both the questionnaire and the interviews focused on attitudes issues. In this area, both approaches lead to congruent results, showing, for example, favorable attitudes in three wards. In addition, both the questionnaires and the interviews showed problems with regard to the user satisfaction with the nursing documentation system in Ward C. However, as the interviews were conducted later, they could better show the long-term development in the wards. Hence, both data sources thus showed congruent results. Second, we found congruent results of the two scales attitudes on nursing process and attitude on the computer-based nursing documentation system within the standardized questionnaires. Both focus on different attitude items, both showed comparable low results in Ward C and higher results on the other wards, pointing to congruent measurements. Those two selected examples show how results of some parts of the study could be validated by congruent results from other parts of the studies.

Completeness of Results Besides validation, triangulation can increase completeness when one part of the study presents results which have not been found in other parts of the study. By this new information, the completeness of results is increased. The new information may be complementary to other results, or it may present divergent information. In our study, both questionnaires and interviews presented partly complementary results, which led to new insights. For example, impact of the computerbased documentation system on documentation processes and communication processes had not been detected by the questionnaire (this aspect had not been included in the questions). However, the documentation system seems to have influenced the way different healthcare professionals exchanged patientrelated information. This led to some discussion on this topic on all wards in the interviews and seems to have had an impact on the overall attitude. Those



229

effects only emerged in the group interviews (and not in the questionnaires); enlarging the picture of the effects of the nursing documentation system and helping to better understand the reactions of the different wards. Another example is the complementarity of the results in the interviews and questionnaires in Ward C. The interviews were done some time after the questionnaires. Thus, during this time, changes may have occurred. The change theory of Lewin (1947) stated that organizational changes occur in three phases: unfreezing (old patterns must be released, combined with insecurity and problems), moving (new patterns are tested), and refreezing (new patterns are internalized and seen as normal). The low attitude scores in Ward C, even at the last measurement point, indicate that the ward was in the moving phase during this time. During the interviews, the stress articulated by the nurses seems to be less severe. This can be interpreted as Ward C’s slowly changing from the moving into the refreezing phase. Triangulation can thus help to get a more complete picture of the object under investigation. Often, especially when applying various methods during the investigation, the results will not be congruent, but they may be divergent (e.g., contradictory). This is an important aspect of triangulation, as divergent results can especially highlight some points, present new information, and lead to further investigation. In our study, we found some divergent results. For example, during the group interviews, nurses from one ward stressed that they do not see a reduction in effort needed for documentation by the computer-based system. However, in the questionnaires, this ward indicated strong time reductions. This differences can lead to the questions of whether time efforts are judged with regard to the situation without the nursing documentation system (where the amount of documentation was much lower, and so was the time effort), or with regard to the tasks that have to be performed (the same amount of documentation can be done much quicker with the computer-based system). This discussion can help one better understand the answers. Interesting differences of point of view could also be found between the staff and the project management of one ward in the group interviews. While the nurses of this ward claimed in the interviews that training was suboptimal, the project management stated that sufficient offers had been made. Those apparent contradictions may point to different perceptions of the need for training by the different stakeholders. Those insights may help to better organize the teaching on other wards.



As those (selected) examples show, triangulation helped us to obtain a better picture of the reaction of the four wards. The evaluation results also led to some decision on how to improve the technical infrastructure as well as how to better organize the teaching and support in some wards. All wards are still working with the computer-based nursing documentation system.

Discussion Medical informatics is an academic discipline and, thus, evaluation is an important part of any system development and implementation activity (Shahar, 2002; Talmon & Hasmann, 2002). However, many problems with regard to healthcare IT evaluation have been reported. Wyatt and Spiegelhalter (1992) as well as Grémy and Degoulet (1993) already discussed the complexity of the field, the motivation issue, and methodological barriers to evaluation. Examples of meta-analysis of IT evaluation studies confirm those barriers (e.g. Brender, 2002; Johnston, Langton, Haynes, & Mathieu, 1994; Kaplan, 2001). In this chapter, we elaborated on a number of important problems and categorized them into three areas: the complexity of the evaluation object, the complexity of the evaluation project with its multitude of stakeholders, and the motivation for evaluation. A kind of framework to support evaluation studies of ISs may be useful to address the problem areas discussed in this chapter. In fact, many authors have formulated the necessity for such a framework (e.g., Grant, Plante, & Leblanc, 2002; Shaw, 2002). Chapter XIV will present a framework for evaluation in more detail. One important part of such a framework is the call for a multimethod evaluation. While triangulation has long been discussed and applied in research (one of the first being Campbell & Fiske, 1959), the idea of the possible advantages of multimethod approaches or triangulation in more general terms is not really reflected in medical informatics literature. The background of multimethod approaches has been more deeply discussed in Chapter XII. In general, both quantitative and qualitative methods have their areas and research questions where they can be successfully applied. By triangulating both approaches, their advantages can be combined. We found



231

that both complementary and divergent results from the different sources gave important new information and stimulation of further discussion. In the past, there has been a more basic discussion about whether intermethods triangulation is possible at all. It is discussed that the epistemological underpinnings between quantitative and qualitative research paradigms may be so different that a real combination may not be possible (Greene & McClintock, 1985; Sim & Sharp, 1998). However, this argumentation is not taking into account that a tradition of research has formed beyond subjectivistic and objectivistic paradigms. Evaluation methods are chosen accordingly to research questions and the research topic. Thus, the question of which methods to apply and how to combine them only can be answered with respect to the research topic and the research question and not on a general basis. Thus, as important as this discussion might be in the light of progress in research methods, evaluation researchers in medical informatics may be advised to start to select and combine methods based on their distinctive research question. This gives evaluation researchers a broad range of possibilities to increase both completeness and validity of results, independent of his or her research tradition.

Conclusions Evaluation studies in healthcare IT take a lot of time, resources, and know-how. Clearly defined methodological guidelines that take the difficulties of IS evaluation in healthcare into account may help to conduct better evaluation studies. This chapter has classified some of the problems encountered in healthcare IT evaluation under the three main problem areas of a) complexity of the evaluation object, b) complexity of the evaluation project, and c) limited motivation for evaluation. We suggested a list of 12 essential recommendations to support the evaluation of ISs. A broadly accepted framework for IT evaluation in healthcare that is more detailed seems desirable, supporting the evaluator during planning and executing of an evaluation study. Focusing on methodological aspects, we have presented some basics on triangulation and illustrated them in a case study. The correct application of triangulation requires–as other evaluation methods–training and methodological experience. Medical informatics evaluation research may profit from this well-established theory. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Acknowledgments This chapter is based on two earlier publications, published by Elsevier (Ammenwerth, Gräber, Herrmann, Bürkle, & König, 2003; Ammenwerth, Mansmann, & Iller, 2003).

References Ammenwerth, E., Brender, J., Nykänen, P., Prokosch, U., Rigby, M., & Talmon, J. (2004). Visions and strategies to improve evaluations of health information systems-Reflections and lessons based on the HISEVAL workshop in Innsbruck. International Journal of Medical Informatics, 73(6), 479-91. Ammenwerth, E., & de Keizer, N. (2004). An inventory of evaluation studies of information technology in healthcare: Trends in evaluation research 1982-2002. Methods of Information in Medicine, 44, 44-51. Ammenwerth, E., Eichstädter, R., Haux, R., Pohl, U., Rebel, S., & Ziegler, S. (2001). A randomized evaluation of a computer-based nursing documentation system. Methods of Information in Medicine, 40(2), 61-8. Ammenwerth, E., Gräber, S., Herrmann, G., Bürkle, T., & König, J. (2003). Evaluation of health ISs-Problems and challenges. International Journal of Medical Informatics, 71(2/3), 125-135. Ammenwerth, E., Iller, C., Mahler, C., Kandert, M., Luther, G., Hoppe, B., et al. (2003). Einflussfaktoren auf die Akzeptanz und Adoption eines Pflegedokumentationssystems-Studienbericht (2/2003). Innsbruck: Private Universität für Medizinische Informatik und Technik Tirol. Ammenwerth, E., Mansmann, U., & Iller, C. (2003). Can evaluation studies benefit from a triangulation of quantitative and qualitative methods? A case study. International Journal of Medical Informatics, 70(2/3), 237-248. Ammenwerth, E., Mansmann, U., Iller, C., & Eichstädter, R. (2003). Factors affecting and affected by user acceptance of computer-based nursing documentation: Results of a two-year study. Journal of the American Medical Informatics Association. 10(1), 69-84.



233

Berg, M., Aarts, J., & van der Lei, J. (2003). ICT in healthcare: Sociotechnical approaches. Methods of Information in Medicine, 42, 297-301. Bowman, G., Thompson, D., & Sutton, T. (1983). Nurses attitudes towards the nursing process. Journal of Advanced Nursing, 8(2), 125-129. Brender, J. (2002). Methodological and methodical perils and pitfalls within assessment studies performed on IT-based solutions in healthcare. Aalborg: Virtual Centre for Health Informatics. Campbell, D., & Fiske, D. (1959). Convergent and discriminant validity by the muli-trait, multi-method matrix. Psychological Bulletin, 56, 81-105. Chin, J. (1988). Development of a tool measuring user satisfaction of the human-computer interface. Proceedings: Human factors in Computing, Chi’88 Conf (pp. 213-218). New York: Association for Computing Machinery. Davis, F. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38, 475-487. Denzin, N. (1970). Strategies of multiple triangulation. In N. Denzin (Ed.), The research act (3rd ed., pp. 297-331). Chicago: Aldine. Dewan, N., & Lorenzi, N. (2000). Behavioral health information systems: evaluating readiness and user acceptance. MD Computing, 17(4), 5052. Frechtling, J. (1997). User-friendly handbook for mixed method evaluation [Ausdruck im Eval-Ordner]. Retrieved May 2005, from http:// www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/start.htm Friedman, C., & Wyatt, J. C. (1997). Evaluation methods in medical informatics. New York: Springer. Goodhue, D. (1995). Understanding user evaluations of information systems. Management Science, 41(12), 1827-44. Grant, A., Plante, I., & Leblanc, F. (2002). The TEAM methodology for the evaluation of information systems in biomedicine. Computers in Biology and Medicine, 32(3), 195-207. Greene, J., & McClintock, C. (1985). Triangulation in evaluation: Design and analysis issues. Evaluation Review, 9(5), 523-545. Grémy, F., & Degoulet, P. (1993). Assessment of health information technology: Which questions for which systems? Proposal for a taxonomy. Medical Informatics, 18(3), 185-193. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Heathfield, H., Hudson, P., Kay, S., Mackay, L., Marley, T., Nicholson, L., et al. (1999). Issues in the multi-disciplinary assessment of healthcare information systems. Assessment of Healthcare Information Technology & People, 12(3), 253-275. Heathfield, H., Pitty, D., & Hanka, R. (1998). Evaluating information technology in health care: Barriers and challenges. British Medical Journal, 316, 1959-1961. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21 st century. Washington: National Academy Press. Johnston, M., Langton, K., Haynes, R., & Mathieu, A. (1994). Effects of computer-based clinical decision support systems on clinician performance and patient outcome: A critical appraisal of research. Annuals of Internal Medicine, 120, 135-142. Kaplan, B. (2001). Evaluating informatics applications: Clinical decision support systems literature review. International Journal of Medical Informatics, 64, 15-37. Kaplan, B., & Shaw, N. (2002). People, organizational and social issues: Evaluation as an exemplar. In R. Haux & C. Kulikowski (Eds.), Yearbook of medical informatics 2002 (pp. 91-102). Stuttgart, Germany: Schattauer. Knafl, K., & Breitmayer, B. (1991). Triangulation in qualitative research: Issues of conceptual clarity and purpose. In J. Morse (Ed.), A contemporary dialogue (pp. 226-239). Newbury Park, CA: Sage. Lewin, K. (1947). Frontiers in group dynamics: Concepts, methods, and reality of social sciences: Social equalization and social change. Human Relations, 1, 5-14. Lindsey, E., & Hartrick, G. (1996). Health-promoting nursing practice: The demise of the nursing process? Journal of Advanced Nursing, 23(1), 106-112. Lorenzi, N., & Riley, R. (1995). Organizational aspects of health informatics - Managing technological change. New York: Springer. Lowry, C. (1994). Nurses’ attitudes toward computerised care plans in intensive care. Part 2. Intensive and Critical Care Nursing, 10, 2-11. Mayring, M. (1993). Einführung in die qualitative Sozialforschung. Weinheim: Psychologie-Verlag-Union.



235

Moehr, J. R. (2002). Evaluation: Salvation or nemesis of medical informatics? Computers in Biology and Medicine, 32(3), 113-125. Nickell, G., & Pinto, J. (1986). The computer attitude scale. Computers in Human Behaviour, 2, 301-306. Ohmann, C., Boy, O., & Yang, Q. (1997). A systematic approach to the assessment of user satisfaction with healthcare systems: Constructs, models and instruments. In C. Pappas (Ed.), Medical Informatics Europe ’97. Conference proceedings (Vol. 43, Pt. B, pp. 781-5). Amsterdam: IOS Press. Palvia, S., Sharma, R., & Conrath, D. (2001). A socio-technical framework for quality assessment of computer information systems. Industrial Management & Data Systems, 101(5), 237-251. Rigby, M. (2001). Evaluation: 16 powerful reasons why not to do it-And 6 over-riding imperatives. In V. Patel, R. Rogers, & R. Haux (Eds.), Proceedings of the 10th World Congress on Medical Informatics (Medinfo 2001,Vol. 84, pp. 1198-1202). Amsterdam: IOS Press. Rigby, M., Forsström, J., Roberts, R., & Wyatt, W. (2001, September 8). Verifying quality and safety in health informatics services. British Medical Journal, 323, 552-556. Shahar, Y. (2002). Medical informatics: Between science and engineering, between academia and industry. Methods of Information in Medicine, 41, 8-11. Shaw, N. (2002). CHEATS: a generic information communication technology (ICT) evaluation framework. Computers in Biology and Medicine, 32, 209-200. Sim, J., & Sharp, K. (1998). A critical appraisal of the role of triangulation in nursing research. International Journal of Nursing Studies, 35(1/2), 23-31. Talmon, J., & Hasmann, A. (2002). Medical informatics as a discipline at the beginning of the 21st century. Methods of Information and Medicine, 41, 4-7. van der Loo, R. (1995). Overview of published assessment and evaluation studies. In E. M. S. J. van Gennip & J. S. Talmon (Eds.), Assessment and evaluation of information technologies (pp. 261-82). Amsterdam: IOS Press.



VATAM. (2000). The VATAM Websites. Validation of health telematics applications (VATAM). Retrieved May 2005, from http://wwwvatam.unimaas.nl Winter, A., Ammenwerth, E., Bott, O., Brigl, B., Buchauer, A., Gräber, S., et al. (2001). Strategic information management plans: The basis for systematic information management in hospitals. International Journal of Medical Informatics, 64(2/3), 99-109. Wyatt, J. (1994). Clinical data systems, part 3: Development and evaluation. The Lancet, 344, 1682-1688. Wyatt, J., & Spiegelhalter, D. (1992). Field trials of medical decision-aids: Potential problems and solutions. In P. Clayton (Ed.), 15th Annual Symposium on Computer Applications in Medical Care (pp. 3-7). New York: McGraw-Hill.


A Cross-Cultural Framework for Evaluation

237

Chapter XIV

A Cross-Cultural Framework for Evaluation Pekka Turunen Shiftec, Finland

Abstract The evaluation of information systems (ISs), especially in the healthcare field, is a complex task. Evidently, there is a need for better understanding of the different aspects evaluation. While in the general IS science field there have been several attempts to build frameworks and models to better understand the evaluation of ISs, in healthcare similar frameworks have been almost nonexistent. Unfortunately, general frameworks cannot be exactly applied for the cross-cultural evaluation of healthcare ISs, because they do not recognise the specific nature of the medicine. Based on works in different areas, this chapter represents an attempt at to combine them to conceptual frameworks for the evaluation of healthcare ISs. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

238 Turunen

Introduction The evaluation of health information systems (ISs) has proved to be an especially difficult task. Evaluation projects are often interdisciplinary by nature and designed by both information technology people and medical professionals (Heathfield, Pitty, & Hanka, 1998; Turunen & Talmon, 2000). Different parties have difficulties in understanding each other because, for example, a lack of common tools (Nykänen, 2000). Among others, one problem is the lack of framework for conceptual understanding of IS impacts and their evaluation. Frameworks and models, used for that purpose, are mainly drawn from different research areas and, unfortunately, are inadequate for this specific field. The frameworks for the evaluation of information systems are not able to describe the specific nature of the area (e.g., golden standards). At the same time, the frameworks for evaluation in the healthcare field do not usually recognise the difficulties of measuring impacts of ISs, the need for external validity or they are too general for the evaluation of ISs. Yet, it has been suggested that a new evaluation paradigm is needed (Shaw, 2002). This chapter presents well-known models from those different areas and combines them into an appropriate framework for the evaluation of health ISs. The framework is based directly on the previous works and is a logical extension of the historical development of such models and frameworks. The purpose of the frameworks is to formulate a conceptual guide of evaluation and emphasise the connection (noncausal by nature) between the different impacts of healthcare ISs. Thus, the framework may be useful in identifying relationships among the success variables of ISs. It assists in taxonomising existing evaluation results of ISs and, therefore, in comparing different evaluations. Equally, the framework should aid in making an overall judgment based on different evaluation methods and results of the healthcare IS. Furthermore, the frameworks may give some hints for the measurement of different impacts. The article presents frameworks at two levels. A general framework describes healthcare IS evaluation at a general level. In addition, a specific framework has been developed for diagnostic ISs. A procedure to minimise evaluation is also presented.



239

Frameworks for the Evaluation of Information Systems The best known and most widely used framework for the evaluation of ISs is probably DeLone and McLean’s (1992) IS success model (Iivari & Ervasti, 1994; Jurison 1996). This model has brought about a good amount of fair and unfair criticism and, therefore, has encouraged further development in the area (Ballantine et al., 1996; Bonner, 1995; Grover, Jeong, & Segars, 1996; Kangas & Manwani, 1998; Manwani & Kangas, 1998; Pitt, & Watson, 1994; Seddon, 1997; Seddon, Staples, Patnayakuni, & Bowtell, 1998). One of the most important further developments is the 3-D model of IS’s success (Balantine et al., 1996). The 3-D model has been divided into three main elements: a) development, b) deployment, and c) delivery (see Figure 1). The development element includes such things as technology, system type, quality of data, ISprofessional skills, and so forth. The second element contains variables such as user satisfaction, task impact, personal impact, and so on (e.g., alignment of individual business objectives, resources, and use of the output are included in the delivery element). In addition to these three elements researchers also refer to a fourth element of political, social, and economic impacts. The meaning of these aspects will be increased while implement-

Figure 1. The simplified version of 3-D model for information systems success. (Source: Balantine et al, 1996) Success

Delivery Delivery Operationalise

Plan

Learning

Integration IntegrationFilter Filter

E xo

Deployment Deployment Implementation Implementationfilter filter Technical Development Technical Development

Ex

g en

og e

ous

nou

sF

Fa c

a ct

tor

s

o rs


240 Turunen

ing wide Internet-based ISs. However, private-sector focus of evaluation is still often at the organisational level. Filters among elements try to illustrate that an impact at one level does not automatically cause an impact at the next level. One of the most important new features that Ballantine et al. (1996) have added to the original idea is the learning loop. Learning addresses that the evaluation is always a twoway process. There are three main features common to almost all the models, guidelines, taxonomies, or frameworks, which try to grasp the principal idea of information flow and evaluation of that process as a whole. All significant aspects can be seen, for example, in the 3-D model (see Figure 1 and the previous text). First, the division of ISs into three basic constructs or elements are typically a) technology, b) user, and c) organisation (Balantine et al., 1996; Cronk, 2000; DeLone & McLean, 1992; Enning et al., 1997; Friedman & Cornford, 1989; Hamilton & Chervany, 1981; Rowley, 1998; Salmela, 1997; Scott, 1994). In different studies slightly different terms, such as individual instead of user, are used for the same elements. Some of the studies and schools have concentrated in making more accurate analyses of the main elements (e.g., Chismar & Wiley-Patton, 2003; Venkatesh & Morris, 2000). Second, the frameworks cover a wide range of different kinds of both subjective and objective evaluation methods (e.g., Ballantine et al., 1996; Cronk, 2000; DeLone & McLean 1992). Third, most frameworks are based on the premise that an impact on one level leads most likely, but not always, to an impact on the next level. According to Nykänen’s (2000) longitudinal research of several evaluation cases, ISs’ evaluation models do not take into account healthcare environmental variables. Medicine, for example, tends to focus on patients, medical evaluation methods considered “golden standards” and attention is also given for physicians.



241

Frameworks in the Field of Healthcare A Framework for Assessing Quality of Care Donabedian’s (1980) framework is aimed at assessing quality of care. Quality assessment is built on the concept of three elements: a structure followed by a process and an outcome (see Figure 2). In Donabedian’s (1980) framework, the structure can mean human, physical, and financial resources. For example, medical tools are a typical example of the structure. The process is a set of activities (e.g., treatment plans that go on within and between practitioners and patients). The outcome is the change in a patient’s health status that can be attributed to given care. The health status is defined broadly to include, for example, patient satisfaction, a patient’s knowledge of health-related issues, and health-related behavioural change. In such a chain, each element is, to some extent, a cause of the element that follows. The structure of the framework and the content of the three elements are very much the same as in frameworks that are used in the IS science (Hebert, 2001).

A Framework for Evaluation of Telehealth The framework for telehealth evaluation has been based on DeLone and McLean’s (1992) IS success model and Donabedian’s (1980) framework for assessing quality of care (Hebert, 2001; see also Figure 3). The framework for telehealth evaluation is one rare attempt to combine different aspects of the cross-cultural research area. In the framework, individual and organisational elements have been innovatively divided into structures and outcomes, a taxonomy that the other

Figure 2. The functional relationship among three elements in Donabedian’s (1980) framework for assessing the quality of care Structure

Process

Outcome


242 Turunen

Figure 3. A framework for evaluation of telehealth (Source: Hebert, 2001)

Individual Structure: *patient -access to services, acceptability *provider -training to use the equipment -change in practice

Organizational Structure -scheduling -equipment location suitability -culture -cost -equipment effectiveness

Process of Care -satisfaction with care process -effectiveness of interaction compared to face to face -management of care process (provider and

Individual Outcomes *Patient -satisfaction with outcome of care; quality of life, functional status *Provider -satisfaction with outcome of care; -number of readmissions, -frequency of adverse effects Organizational Outcomes -efficient use of resources -cost effectiveness -utilization of services

researchers have not used. According to Hebert (2001), Donabedian’s (1980) structure is equal DeLone and McLean’s (1992) first element, “information and system quality.” Nevertheless, in the telehealth framework the structure has been connected with DeLone and McLean’s “individual and organisational impact,” which actually should mean Donabedian’s outcome, not structure (Hebert, 2001). Probably as a result of these, we use only three basic elements (structure, process, and outcome) for categorising two case studies. However, the telehealth framework is a first proper framework to cover the whole picture of the evaluation of telehealth.

Models for Evaluation of Medical Technology Fineberg, Bauan, and Sosman (1977) first presented the idea of modeling efficacy of clinical diagnostic technology (see Figure 4). Their model



243

Figure 4. Clinical efficacy model (Source: Fineberg, et al., 1977) Technical output

Diagnostic and Prognostic Information

Therapeutic Plan

Patient Outcome

greatly influenced physicians’ thinking (e.g., Chizeck & Katona, 1985; Cook, Brun-Buisson, Guyatt, & Sibbald, 1994; Maisey, 1996; Sailly et al., 1985; Szczepura & Stilwell, 1988; Ure, Spangenberger, Lefering, Dietrich, & Troidl, 1992) and it has been used for the evaluation of telehealth systems (Ohinmaa & Reponen, 1997). This model emphasises the clinical-care process, medical technology, and the significance of physicians (making diagnoses). Even though the clinical efficacy model is narrower than models in the IS science, the principal idea of the model is very similar to that of IS models, because they all present a Western scientific way of thinking. Guyatt, Tugwell, Feeny, Drummond, and Haynes (1986) have developed the clinical efficacy model in two ways. They have extended the evaluation of patient outcome to include the economic evaluation. They have also added the therapeutic process as a new process parallel to the diagnostic process. Their framework also emphasises randomised controlled trial (RCT) in the evaluation of medical technology. The new features are welcome extensions in the direction of a framework that covers all the impacts and evaluation methods of medical technology. Normally, stakeholders with a strict medical background approach the evaluation of health ISs through Fineberg et al.’s (1977) or Guyatt et al.’s (1986) pattern. The problem is that patterns can address only a minority of effects that medical technologies are causing and only a small minority that healthcare ISs are causing. Evaluation methods, such as the RCT, can hardly evaluate complex ISs in the real-life situation in which effects of the system may be indirect, intangible, or nonquantified (Davey, 1994; Feinstein, 1983; Heathfield et al., 1998; Hebert, 2001; Kaplan, 2001; Shaw, 2002). Traditional medicine has also favoured validation of outcome. However, the development of ISs is usually also a main issue in the evaluation of ISs. Unfortunately, a summative evaluation does not incorporate a development perspective (Hamilton & Chervany, 1981).


244 Turunen

A Conclusion of Prior Research: Frameworks for Evaluation of Healthcare Information Systems The idea of evaluation frameworks is to combine the key elements of both research areas in such a way that frameworks are able to cover the healthcare information area and can possibly be accepted by the different stakeholders. Another significant issue is to present nonmedical evaluation methods to stakeholders with a medical background. It is important try to use a language that looks familiar to stakeholders (Nykänen, 2000). The frameworks may provide new measurements for the evaluation of healthcare ISs, when it is not possible or reasonable to use traditional clinical measurements. Three basic elements familiar from the general IS area can be combined equally with elements of medical technology. The main combined elements are a) technical quality, b) user impact, and c) organisational and patient impact (see Figure 5). One has to take into consideration the role of societal impacts while increasing the amount of Internet applications and integration process in nationwide ISs. However, evaluating the impacts is still impossible or at least extremely difficult.

Figure 5. A general framework for the evaluation of impacts of health information systems Impacts Impacts of evaluation

Technical Quality Accuracy Reliability Response time Flexibility EUCS etc.

User Impact Impact to: Diagnose Decision making Treatment Nursing Working time User satisfaction etc.

Organisational and Patient Impact Patient outcome and satisfaction RCT Quality of services Patient’s health knowledge Measurement of processes Cost-benefit (e.g cb, c-e, c-u) ROI, DRG etc.

Societal Impact Population’s ability to work National welfare Service export etc.



245

Figure 6. A framework for the evaluation of diagnostic information systems Impacts Impacts of Evaluation Technical Quality Technical Accuracy Quality Reliability Response time Flexibility EUCS etc.

User Impact

Diagnostic Information

Therapeutic Information

Impact to: Diagnose, decision making, treatment, nursing, working time User satisfaction, RTC etc.

Organisational and Patient Impact Patient Outcome Productivity Quality of services C-B, C-E,C-U Customer satisfaction, RTC, ROI etc.

In Figure 5, the arrows showing effects of the IS and evaluation (they could also be called a learning loop) are shown above the figure. The use of a broken line is designed to emphasise the noncausal nature of ISs and the evaluation of their impact. According to Nykänen (2000), IS people are not able to consider the environmental variables of healthcare. For them the specific subframework emphasises, and hopefully, crystallise, the core process of medical treatment and the importance of evaluating it (see Figure 6). The specific framework is especially designed for diagnostic ISs or clinical decision support systems and can also be seen as a model for physicians.

The Use of Frameworks to Minimise Evaluation Set Resources are rarely available to enable the launching of massive evaluation programs (Friedman & Wyatt, 1997). Following aspects may ease formulating evaluation plans in order to minimise the evaluation set: 1. 2.

What are the expected relationships between the different elements or variables (Fineberg et al., 1977; Hebert, 2001)? How does use of new IS compare to the process of traditional care or previous IS (Hebert, 2001)?


246 Turunen

This procedure may help to formulate a minimum amount of evaluation trials to prove an impact of a system or to find the potential cases that may affect the outcome (e.g., improved medical data to lead better decision making and better outcome). The framework may help to classify evaluation results and thus, to compare evaluation studies. Because classification of evaluation studies seems to be in some sense a subjective task (see Figure 5), this framework is offered rather as a conceptual guide for classifying, not as a straightjacket.

References Ballantine, J., Bonner, M., Levy, M., Martin, A., Munro, I., & Powell, P. L. (1996). The 3-D model of information systems success: The search for the dependent variable continues. Information Resources Management Journal, 9(4). Bonner, M. (1995) DeLone and McLean’s model for judging information system success: A retrospective application in manufacturing. Proceedings of the Second European Conference on IT Evaluation, Henley Management College, UK. Chismar, W. G., & Wiley-Patton, S. (2003). Does the technology acceptance model apply to physicians. Proceedings of the 36th Hawaii International Conference on System Sciences. Chizeck, H. J., & Katona, P. G. (1985). Closed loop control. In Carson & Cramp (Eds.), Computers and control in clinical medicine. New York: Plenum Press. Cook, D. H., Brun-Buisson, C., Guyatt, G. H., & Sibbald, W. J. (1994). Evaluation of new diagnostic technologies: Bronchoalveolar lavage and the diagnosis of ventilator-associated pneumonia. Critical Care Medicine, 22(8). Cronk, M. C. (2000). Understanding complex information system constructs through holistic construal. Proceedings of Seventh European Conference on the Evaluation of Information Technology, Dublin, Ireland. Davey, B. (1994). The nature of scientific research. In K. McConway (Ed.), Studying health and disease. Oxford, England: The Open University Press. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


247

DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 1, 60-95. Donabedian, A. (1980). Explorations in quality assessment and monitoring. Vol. I: The definition of quality and approaches to its assessment. MI: Health Administration Press. Enning, J., Fabianne, E., Talmon, J., Vissers, M., Nykänen, P., Roine, R., et al. (1997). Guidelines for evaluation of telematics applications in medicine, Deliverable D03.2b. Retreived December 1, 2001, from http://www-vatam.unimaas.nl Feinstein, A. R. (1983). An additional basic science for clinical medicine: II. The limitations of randomized trials. Annals of Internal Medicine, 99, 544-550. Fineberg, H. V., Bauan, R., & Sosman, M. (1977). Computerized cranial tomography: Effect on diagnostic and therapeutic plans. Journal of the American Medical Association, 3, 244-227. Friedman, A. L., Cornford, D. S. (1989). Computer systems development: History, organization and implementation. Chichester: Wiley. Friedman, C. P., & Wyatt, J. C. (1997). Evaluation methods in medical informatics. New York: Springer. Grover, V., Jeong, S. R., & Segars, A. H. (1996). Information systems effectiveness: The construct space and patterns of application. Information & Management, 4, 117-191. Guyatt, G. H., Tugwell, P. X., Feeny, D. H., Drummond, M. F., & Haynes, B. R. (1986). The role of before-after studies of therapeutic impact in the evaluation of diagnostic technologies. Journal of Chronic Diseases, 39(4), 295-304. Hamilton, S., & Chervany, N. L. (1981). Evaluating information system effectiveness. Part I: Comparing evaluation approaches. MIS Quarterly, 3, 55-69. Heathfield, H., Pitty, D., & Hanka, R. (1998). Evaluating information technology in healthcare: Barriers and challenge. British Medical Journal, 316, 1959-1961. Hebert, M. (2001, September 2-5). Telehealth success: Evaluation framework development. Proceedings of Medinfo 2001, London.


248 Turunen

Iivari, J., & Ervasti, I. (1994). User information satisfaction, IS implementability and effectiveness. Information & Management, 27(4), 205-220. Jurison, J. (1996). The temporal nature of IS benefits: A longitudinal study. Information & Management, 30, 75-79. Kangas, K., & Manwani, S. (1998). Package implementation within a European MNC: Case of a Finnish project serving as a pilot. Proceedings of the Sixth European Conference on Information Systems, Aixen-Provence, France. Kaplan, B. (2001). Evaluating informatics applications-some alternative approaches: Theory, social interactionism, and call for methodological pluralism. International Journal of Medical Informatics, 64, 3956. Maisey, M. (1996). Evaluating the benefits of nuclear cardiology. Quarterly Journal of Nuclear Medicine, 40(1), 47-54. Manwani, S., & Kangas, K. (1998). IT project evaluation in multinational companies: A multiple stakeholder viewpoint of a Finnish project within a European MNC. Proceedings of the 1998 Information Resources Management Association International Conference, Boston, MA. Nykänen, P. (2000). Decision support systems in health informatics perspective. A-2000-10 Acta Electronica Universitatis Tamperensis (Vol. 55). Tampere. Ohinmaa, A., & Reponen, J. (1997). A model for the assessment of telemedicine and a plan for testing of the model within five specialities. FinOHTAn Report, 5. Pitt, L. F., & Watson, R. T. (1994). Longitudinal measurement of service quality in information systems: A case study. Proceedings of the 15th International Conference on Information Systems, Vancouver, British Columbia, Canada. Rowley, J. (1998). Towards a framework for information management. International Journal of Information Management, 18(5). Sailly, J. C., Lebrun, T., Vincent, P., Eeckhoudt, L., Izard, D., Leclerc, H., et al. (1985) The medical and economic consequences of automation in bacteriology: A case study in a French university hospital. Social Science & Medicine, 10, 1163-1166. Salmela, H. (1997). From information systems quality to sustainable business quality. Information and Software Technology, 39(12), 819-826.



249

Scott, J.E. (1994) The Measurement of Information Systems Effectiveness: Evaluating A Measuring Instrument. In Proceedings of the 15th ICIS. Vancouver (pp.111-128). Seddon, P. B. (1997). A respection and extension of the DeLone and McLean model of IS success. Information Systems Research, 8(3), 240-253. Seddon, P. B., Staples, D. S., Patnayakuni, R., & Bowtell, M. (1998). The IS effectiveness matrix: The importance of stakeholder and system in measuring IS success. Proceedings of the 19th International Conference on Information Systems, Helsinki, Finland. Shaw, N. T. (2002). CHEATS: A generic information communication technology (ICT) evaluation framework. Computers in Biology and Medicine, 32, 209-220 Szczepura, A. K., & Stilwell, J. A. (1988). Information for decision makers at hospital laboratory level: An example of a graphical method of representing costs and effects for a replacement automated technology in a haematology laboratory. Social Science & Medicine, 7, 715-725. Turunen, P., & Talmon, J. (2000, September 28-29). Stakeholder groups in the evaluation of medical information systems. Proceedings of Seventh European Conference on the Evaluation of Information Technology, Dublin, Ireland. Ure B. M., Spangenberger W., Lefering R., Dietrich A., & Troidl H. (1992). Routine gastroscopy before laparoscopic cholecystectomy: Evaluation of the technology in 376 patients. Z Gastroenterol, 30(8), 529533. Venkatesh, V., & Morris, M. G. (2000). Why don’t men ever stop to ask for directions? Gender, social influence, and their role in technology acceptance and usage behaviour. MIS Quarterly, 24(1).


250 van der Meijden

Chapter XV

Quantitative and Qualitative Methods: Added Value in Evaluating Electronic Patient Records Mirjan van der Meijden Elkerliek Hospital, The Netherlands

Abstract Contrary to what would be expected, the increasing application of patient care information systems hardly resulted in well-defined, methodical purchase or development and implementation trajectories. Together with the lack of thorough evaluations in daily practice, this was an important motivation to the development, implementation and evaluation of the studied electronic patient record. It is a problem that both successful and failed electronic patient record projects are seldom evaluated. This is a problem, because failures and successes can provide relevant information about system qualities, system requirements, important aspects of implementations, and so forth. A second problem is that the methods to evaluate are not yet fully developed as well as the content of evaluations. Clearly, this impedes evaluation practices. To contribute to solutions for


Quantitative and Qualitative Methods 251

both problems we did a case study in the field of neurology. We developed, implemented and evaluated an electronic patient record for stroke with qualitative and quantitative methods. The evaluation started before the development phase and ended after a trial period in daily practice. It was based on an evaluation framework (van der Meijden, Tange, Troost, & Hasman, 2003). The results of the evaluation showed that the previously described framework was applicable in evaluating electronic patient records. It was possible to assess attributes that represented the quality of record keeping, the impact on daily work, the opinions of nurses and physicians about electronic record keeping and the experiences of management. The analysis of these attributes provides insight in why the electronic patient record was successful in some aspects and was less successful in others. They also showed that the combination of questionnaires, interviews, chart reviews and observations in an evaluation provided deeper insight in reasons for use or non-use of an electronic patient record than one single method would have. Results collected with one method can supplement result– regarding the same subject–collected with another method.

Introduction Documenting intake data, progress notes, and test results is an important aspect of the healthcare process. It is disliked by many healthcare professionals who regard it to be a burden. Collection and documentation of data start the moment a patient enters the hospital; often the same data are documented on a number of different forms and for different reasons. Financial purposes, managerial purposes, and most and for all, delivery of care require adequate and timely data about individual patients or patient populations. Particularly, developments like shared care require that collected data are easily available to all healthcare professionals involved. Paper patient records, although having many advantages, do not suffice, then. Electronic records can provide the essential functionality of (multisite) availability, of timeliness, and so forth, and can be of additional value provided they fit their users’ practice (Dick & Steen, 1991). In addition, electronic records can be beneficial for the quality of care (Tang, 1999).


252 van der Meijden

The subject of this chapter is the evaluation of an electronic patient record for stroke (stroke EPR), which was tested in daily practice. The objective of the project was to define the requirements of an EPR, and to explore the possibilities of an EPR to improve the quality of record keeping and to support the communication in multidisciplinary care. In the present evaluation study we tried to answer the following research questions. What consequences does an EPR have on the completeness of record keeping? What consequences does an EPR have on the daily work of nurses and physicians? What consequences does an EPR have on their opinions about electronic record keeping? How is their usage behaviour? Finally, what are management’s experiences?

Research Setting Context Research took place at the stroke unit of the neurology department of an academic hospital in the Netherlands, where in 1996 a project started to coordinate the care for stroke patients. This project aimed at improving the information exchange between all healthcare providers involved in treatment and care for stroke patients. The first two weeks after the event, the acute phase of stroke, are considered to be crucial for further prognosis. Therefore, time is critically important and an immediate admission in the hospital after the event is considered essential. The neurology department has a fully equipped stroke unit, consisting of a medium care unit and two normal care wards. Upon admission, patients are transferred to the stroke unit of the neurology department, where a multidisciplinary team takes responsibility for the stroke patient. In this team neurologists and nurses collaborate with physical therapists, social workers, occupational therapists, and speech therapists to optimise diagnosis, treatment, and care for each patient. Approximately 300 stroke patients are admitted yearly to the stroke unit, which is about 70% of all patients admitted at the ward. The stroke patients are cared for by 20 experienced nurses and 12 to 14 residents with their supervisors. One resident is responsible for the emergency care and, together with two nurses, also for the medium care unit. At each normal



care ward, one resident and teams of 4 to 5 nurses are responsible for the patients. Adequate and timely information exchange is of high importance. Before the present study, nurses and physicians documented their findings in separate records, structured to a certain extent. Additionally, the other care providers reported in the nursing or the medical record, while maintaining their own paper records as well. In this setting an EPR for stroke was developed in close cooperation with staff–nurses and physicians. The decision to develop an EPR was based on the belief that the multidisciplinary team can be effectively supported by electronic record keeping and that a combined medical and nursing EPR is better equipped to serve as a means of communication than the separate paper records. Moreover, a hospital-wide electronic record proceeded slowly and therefore, anticipating on hospitalwide developments, it was decided to develop a stroke EPR in a research project.

Intervention The stroke EPR was based on an experimental electronic medical record, developed in a previous research project conducted at our department (Tange, 1999; Tange, Dreessen, Hasman, & Donkers, 1997; Tange, Schouten, Kester, & Hasman, 1998). That experimental electronic medical record had only a viewer function. Based on existing, structured paper forms, the medical intake and discharge forms and all nursing forms were designed. Medical progress notes were subdivided into main headings: medication, diagnostic tests, complications, assessment notes, reports of family talks, and the summary. The medication, diagnostic tests, and complications were structured, whereas assessment notes, reports of family talks, and the summary remained free text. Nursing progress notes were free text as well. The medical and nursing record were integrated into the EPR that was meant for documenting data of inpatient stroke patients only. To minimise double data entry, demographic data (admission data, date of birth, address, general practitioner, etc.) of patients were downloaded from the hospital information system (HIS). Other data, like lab results and radiology reports were available in the HIS and could be consulted using ALT-TAB. This stroke EPR was described in detail previously in van der Meijden, Tange, Boiten, Troost, and Hasman (2000a).


254 van der Meijden

The implementation of the stroke EPR started at the medium care unit in July 2000. The residents of the normal care wards started using it shortly afterwards. For practical reasons, only the stroke patients that were admitted via the emergency department to the medium care unit were included in the EPR (no transfers and no admissions directly to the normal care ward). During the trial period every user could telephone the developer with questions or problems. Moreover, at a certain point in the implementation phase a booklet for comments, questions, and trouble reporting was installed. In July 2001, the trial period ended. As became apparent during the implementation, the ICT department could not realise backup facilities. This unanticipated inconvenience influenced the course of the EPR project, because users felt the need to document all data, both in the EPR and in their paper records, to create some sort of back up. However, this situation offered the opportunity to compare patient data that were documented electronically as well as on paper.

Methods In our evaluation study we included several aspects of electronic record keeping. We studied system quality, data quality, usage, user satisfaction, individual impact, and organisational impact. In addition, we evaluated the process of development and implementation. We collected data with qualitative and quantitative methods and combined the results through triangulation.

Study Design The evaluation was originally designed as a before–after study with a series of baseline measurements before the implementation and a similar series of measurements post implementation. As shown in Figure 1, the baseline measurements commenced during development. These measurements started with a questionnaire assessing the user attitude toward computers, followed by a questionnaire assessing user satisfaction with the clinical paper records, user expectations, and user requirements of information technology (IT) applications in healthcare. The



Figure 1. The schedule of research measurements and development, implementation and maintenance in the stroke EPR project. Q = questionnaire, I = interviews, OBS = observations, CR = chart review, L = logging module. Baseline measurements:

Post implementation

Q, I, OBS, CR

measurements: Q, I, CR, L

Development

Jan 1998

Implementation

July 2000

Oct 2000

EPR trial / Maintenance

July 2001

questionnaires were complemented with face-to-face interviews, observations and reviews of paper records. These results were presented previously by van der Meijden, Tange, Boiten, Troost, and Hasman (2000b, 2001). Post implementation measurements started in October 2000. Usage behaviour was logged in a special logging module. In addition, chart reviews of paper and corresponding electronic records were conducted during the trial. Completeness of data in these records was compared in a noncontrolled trial. Towards the end of the trial period questionnaires were sent to all eligible users. Shortly after the end of the trial six nurses, six residents and four departmental managers were interviewed. The data collection methods are described in the next paragraph.

Data Collection Data were collected through reviews of paper charts, questionnaires, interviews, and logging of users’ actions. Table 1 shows which aspects were evaluated and which type of data was collected in the baseline study, in the EPR trial, or in both. Through triangulation, data collected at different times and with different methods were combined to present a more complete picture.


256 van der Meijden

Table 1. Timing and data collection methods for the different aspect of the evaluation Data collection methods*

CR (Paper + EPR)

Aspects evaluated Quality of record keeping x Usage User evaluation User satisfaction Data quality System quality Use User needs User expectations Impact on daily work Organisational impact Management’s evaluation Phase in research project Baseline x Post x implementation

Q (Paper)

I (Paper)

Q (EPR)

I L (EPR) (EPR) x x

x x x

x

x

x x

x x x x x x

x x

x

x x x x x x x x

x

x x

x

x

Quality of Record Keeping To investigate the quality of record keeping during the EPR trial we analysed the paper and corresponding EPRs of all stroke patients admitted and discharged or deceased between October 1, 2000, and June 1, 2001. We compared these results with the baseline completeness analysis. The attributes rated were completeness of the medical intake form and discrepancies in information between paper records and electronic records. The completeness analysis concentrated on a set of medical intake items that had been selected in close consultation with a senior neurologist (van der Meijden et al., 2000b). This set of clinical items was the minimum necessary for diagnosing stroke. Usage The EPR was equipped with a logging module that registered different actions, like log on-log off, the name of the current screen, the type of information, like admission data or progress notes, and the user ID. These data were stored in a database, separate from the actual patient data. In



addition, entered patient data were categorised based on type of data, for instance, intake data or vital signs. User Evaluation User satisfaction, user needs, expectations and other opinions were assessed by means of questionnaires and in-depth interviews. The aim of the interviews was to explain and supplement the results of the questionnaires. In addition, in the interviews the impact on daily work and the organisational impact were investigated. Questionnaire Immediately after the end of the trial period, EPR users received a questionnaire to assess their satisfaction with the EPR, to assess their knowledge of computers, their expectations of computer applications in healthcare, and required functionality. Two weeks after the initial questionnaire a reminder was sent. The questionnaire was divided into four sections. Sections 1, 3, and 4 were identical to the questionnaire used to collect baseline data (van der Meijden et al., 2001). Section 2 contained items concerning the respondent’s satisfaction with the EPR. Eleven items concerning screen messages and the organisation of data on screens were added, as well as three statements concerning training and support. These items were scored on a five point Likert scale. Except for the last three, items were selected from previously developed questionnaires (Lewis, 1995; Tange, 1995). All items in this questionnaire were positively worded. Higher scores indicated that respondents were more satisfied, possessed more knowledge of computers, expected more positive effects, or considered features to be more necessary. Interviews After the trial period in-depth interviews were held with 12 users. Six residents and six nurses were selected on frequent or infrequent usage. An independent interviewer did the interviews. This interview was divided into three clusters. Cluster 1 contained questions about the experienced positive and negative aspects of the EPR and the actual usage of the EPR in daily practice. In cluster 2 the respondents spoke about the development and


258 van der Meijden

implementation of the EPR (e.g., “What did you expect from the EPR”). In cluster 3 of these interviews the expected impact of computer applications on several aspects of daily practice, like communication with colleagues within the department and within the hospital, the quality of record keeping, and expected changes in work procedures, were elaborated. Management’s Point of View In addition to the users, the head of the department, a senior who was involved in the project, the head nurse and the informatician of the neurology department were interviewed by the same interviewer. The interview concentrated on their opinions about the course of the whole project, a reflection on their own role in the project, and what requirements an EPR and its implementation should meet in the future. In these interviews the positive and negative aspects of electronic records were only addressed indirectly. The results of these interviews were compared with the baseline interviews in which management reflected on the problems they had encountered until then, and positive as well as negative impacts they expected to occur after implementation.

Data Analysis Completeness was analysed in all EPRs containing a medical intake and in their corresponding paper records. Missing items were counted per patient and the counts for paper and for electronic records were compared with a paired T test, showing the effect of the computer. To compare the completeness of paper records before and after the implementation (reflecting changes over time), as well as to compare the medical paper records before and the electronic records after the implementation, the independent t test was used. The overall average scores for satisfaction, knowledge, and expectations of healthcare applications were calculated. Statistical analysis was performed with SPSS version 10.0. All interviews were audiotaped and transcribed. The transcripts of the interviews were analysed with the aid of QSR NUD*ist, a software package to structure qualitative data. Usage behaviour was analysed by combining the logging file that registered all log-in and log-out actions, save actions, switches between screens, and the



data entered in the EPR. We concentrated on data entry actions, the type of data entered, intake data, and daily entered data, and the number of log-ins during the trial period. Usage was assessed with two attributes: the login frequency and the number of entries per patient record.

Results Results of the different sources of information are combined when possible and necessary. Furthermore, baseline measurements are shown along with the trial measurements where relevant.

Quality of Record Keeping The analysis of quality was limited to completeness. Discrepancies in information were hard to detect, because the number of completely filled in EPRs was too limited. In the baseline measurement the completeness of medical intake data were analysed in 42 paper records. After the implementation, data of 100 patients were entered in the EPR. Residents entered the intake data of 54 patients. Of the corresponding 54 paper records, 16 were excluded from the analysis because no proper stroke intake form was used (8), a printout from the EPR served as paper intake form (identical to the EPR data and therefore excluded; (3), the intake form was missing (1), the clinical neurological paper record was unavailable in the medical archive (2), or there were no intake data available in the EPR (2). Thirty-eight usable combinations of paper and EPRs remained. One item was excluded in the comparison with the electronic records, because it had not been recorded due to a software error. Paper records in the baseline had a mean of 2.38 (SD = 2.24; n = 40) missing data, paper records post implementation 2.58 (SD = 2.31; n = 38) and electronic records 0.89 (SD = 1.29; n = 38). A comparison of completeness of paper records before implementation and paper records after implementation showed no significant difference. However, a significant difference was found when the completeness of the EPRs was compared with the completeness of paper records before implementation (paper: M = 2.25; SD = 2.05; n = 40 vs. electronic: M = 0.89; SD =1.89; n = 38; t = 3.476, p = Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

260 van der Meijden

0.001). EPRs were also more complete than their corresponding paper records (t = 4.58; p = 0.000). Time Trends At three points in time the completeness of data in the EPR was analysed. After four months, 19 patient records were included; between 4 and 6 months, eight were; and between 6 and 8 months, 27 were. As shown in Table 2, the completeness of data tended to decrease during the trial period. The number of residents entering data increased from 4 to 10, the average number of missed items per patient increased from 0.90 at four months to 1.33 between 6 to 8 months. During the eight months of regular use of the EPR the recording of the items did not significantly deteriorate. In several patient records, nurses entered blood pressure and pulse in their intake form if the resident, who filled in the medical intake form, did not enter any values. After subtracting the nurses input from the data the average number of missed items per patient increased to 1.37 at 4 months and to 1.78 between 6 to 8 months. These changes were not significant.

Usage Figure 2 shows that the number of newly created EPRs gradually declined till February. Afterwards it rose until May. In correspondence with this trend, usage showed a gradual decline until February. In the first six months of the trial period few residents used the EPR. Nurses used the EPR much more frequently. After February, usage by nurses as well as physicians increased. Nurses entered more patient records than residents and they did

Table 2. Data completeness in EPRs (average number per patient record) No Records analysed No of residents using the EPR Average number of missing items* Average number of missing items without nurses’ input*

4 mo 19 5 0.90

4-6 mo 8 4 0.88

6-8 mo 27 10 1.33

1.37

1.38

1.78



Figure 2. The number of EPRs, the number of logins, the number of entries per patient record, and the number of admissions in each month of the trial. a = Nurses, b = Residents. Number of admissions of stroke patients of the whole ward. The exact number of stroke patients admitted directly to the medium care unit is unknown. (a) " # No of Logins $ No of EPRs % No of Admissions & ' ! " # $ % & No of Entries

225 200 175 150 125 100 75 50 25 0 Oct

Nov Dec

Jan

Feb Mar

Apr May

(b) No of No of No of No of

225 200 175

Entries Logins EPRs Admissions

150 125 100 75 50 25 0 Oct Nov Dec Jan Feb Mar Apr May


262 van der Meijden

Table 3. The number of patient records in three different data type categories. The categories are based on the number of entries. Month

Category 1 Nursing Intake Vital Signs Orders

Oct 2000 Nov 2000 Dec 2000 Jan 2001 Feb 2001 Mar 2001 Apr 2001 May 2001 total

2 2 6 1 1 1 1 11

Category 2

Category 3

Category 1+ Category 2+ Medical Intake All other Progress Notes data 3 2 1 1 2 9

8 11 10 7 4 7 12 20 80

more entries per patient record than residents. Although the number of logins showed a dip in February, the number of entries was rather high. Table 3 shows the number of patient records in three categories. Patient records with fewer than 10 entries were excluded from the analysis. Category 1 comprised basic admission parameters, the nursing intake, vital signs, and in some cases, a few orders. The second category included medical and nursing intake data, progress notes, vital signs, and orders. The majority of the patient records contained all types of data (in addition to category 2: infusions, reports of meetings with patient and family, test results, summaries, etc.). An analysis of data collected after a considerable learning period (3 months) showed that at admission, nurses entered intake data plus orders and vital signs. During the course of an admission, vital signs and daily progress notes were added to the patient record. The medical part of the patient record was filled only with intake data, and occasionally with progress notes and orders. The analysis of daily nursing data showed a data entry peak at the times of changes of shifts: shortly before 7:30 a.m., 14:30 p.m., and 22:30 p.m., and in the afternoon, data were entered at other times as well (see Figure 3).



number of patients

Figure 3. The number of patients whose data were collected at admission by nurses (AN) and by residents (AR) or daily by nurses (DN) at different times of day in the first 2 weeks of May 2001. Arrows indicate changes of shifts. NN NR DN

10 8 6 4 2 0 00-02 02-04 04-06 06-08 08-10 10-12 12-14 14-16 16-18 18-20 20-22 22-24 time of day (h)

Users’ Evaluation The users’ evaluation of the EPR consisted of a questionnaire and in-depth interviews. In the interviews aspects of the EPR itself, data quality and system quality, as well as aspects of the project as a whole were addressed. The main themes emphasised by the interviewees were a) process of development and implementation, b) user satisfaction, c) impact on daily work, d) users’ expectations of electronic record keeping, and e) requirements for future systems. Thirty-four users received the post implementation questionnaire. The total response rate was 59%; 10 nurses and 10 residents returned the questionnaire. One respondent returned the questionnaire blank with the remark that he or she had worked at another department for 12 months and had never used the EPR. Two other respondents had never used the EPR and only filled in the general section. Their responses were excluded from the analysis. Five respondents were male, nine were female and the gender of three was not filled in. Thirteen respondents were younger than 40 years old, two were older, and two did not fill in their age. Eleven respondents regarded


264 van der Meijden

themselves as experienced computer users, and six felt inexperienced. The number of respondents after implementation was half the number before implementation, and the number of respondents that answered both questionnaires was six. Therefore, data were descriptively analysed. Contact with Developers Users who had been involved in the development of the EPR were satisfied with their role. Questionnaire respondents were satisfied with the training they received prior to the implementation. All interviewees remembered their initial training session and stated that it had been sufficient. The questionnaire results showed that 14 respondents knew whom to contact in case of problems with the EPR. This was confirmed by interviewees who could mention names and phone numbers of contact persons. According to the interviewees the booklet served as an effective means of communication. Users jotted down remarks, questions and problems, the developer in turn wrote down answers to questions and tried to solve problems. Most users consulted the on-line help and the EPR’s manual rarely; they preferred to consult colleagues or to contact the developers directly. User Satisfaction User satisfaction with the EPR was assessed by means of a questionnaire and it was also an important aspect of the interviews. The mean overall satisfaction score was 3.23 (SD = 0.53; n = 18) out of 5. Data entry and data retrieval had an overall score of 3.29 (SD = 0.48; n = 18) and 3.20 (SD = 0.84; n = 17) respectively. A complete overview of the questionnaire results is shown in appendix. In the interviews many of these results were confirmed. While the questionnaire concentrated on specific functionality of the EPR and ease of use, the interviews addressed also other aspects relating to the EPR and electronic record keeping, like system quality, data quality, and usage. They also showed that users were rather satisfied with the EPR as such.



Perceived System Quality According to interviewees, the EPR was easy to learn to use and, once learned, it was easy to use. This is not supported by the questionnaire results where six respondents were neutral, three respondents thought the EPR easy to learn while eight disagreed with that statement. A similar discrepancy was observed for general user friendliness. Thirteen respondents were–generally speaking– dissatisfied with the user friendliness of the system, while one respondent was satisfied. As an aspect of ease of use, the interviewees mentioned the organisation of data on the screen. Some identified the tabbed structure of many forms as a shortcoming compared to the paper record, others expected that once they got used to the EPR, they could locate the data easily. These results agree with the questionnaire results that show that respondents are relatively satisfied with the user friendliness of the screens. The next quote illustrates the opinion of several interviewees: When you use it regularly, you’ll find your way [in the EPR] easily. In addition, interviewees had critical remarks regarding the quality of the system. Real software problems were identified and solved. These were for example a dysfunctional link with the database of the HIS and the inability to save data of the discharge form. Many perceived problems, however, could not be reproduced nor explained. Part of the criticism seemed due to lack of understanding of how the EPR functioned, such as this response from a nurse: Then I had some problems, because the progress report of the night shift appeared between the progress reports of the evening and day shifts. Usually this is put at the end of the progress notes [in the paper record]. This nurse did not realise that the electronic progress notes were in chronological order, like on paper, but that for reasons of usability only the last three reports were shown on the screen.


266 van der Meijden

An important complaint concerned the availability of the EPR, which comprised two aspects. One aspect was reduced use now and then due to real or perceived software problems. The other aspect was a too small number of workstations. This was cited by both residents and nurses, who indicated that the patient record was needed at the patient’s bedside, there where care was delivered: And it is important too that I can bring it [patient record] with me to the patient, since it does not work when I examine the patient at the bedside and have to return to the nurses’ station to document my findings.(a resident) As a last aspect of the system, interviewees cited the lack of integration or linkage with other systems in the hospital. Users had expected more data to be available in the EPR. As an example, they mentioned laboratory results and radiology reports. Perceived Data Quality The EPR was complete for the domain of stroke, because users could document all relevant data for stroke patients. Moreover, it was clear where to enter what data. However, some interviewees noticed that data presentation was sometimes not accurate and reliable. According to them, data appeared at wrong places in the EPR or was not stored at all. In addition, in agreement with the functional requirements saved data could not be changed anymore, which was felt to be impractical in daily work. The accessibility of data at multiple sites was not perceived as a great benefit. Users indicated that the patient records were consulted most at the patient’s bedside and not elsewhere in the hospital. Moreover, they stated that the paper records were always available, thus an EPR could not improve availability and accessibility of patient data. Use Five of the EPR users said to have used it less than 10 times and 11 of them more than 10 times during the whole trial period. When these results were



compared with the actual number of logins, nine approximated their real usage, while seven underestimated or overestimated their real usage. On the one hand, interviewees acknowledged that stroke patients were a good group to try an EPR. On the other hand, they were reluctant to use the EPR, since it was only for stroke patients, and not for the whole ward or hospital. In the interviews respondents stressed that they felt urged to use the EPR, and in parallel the paper records. Therefore, many indicated that they hardly consulted the EPR and only entered data. Responses to the questionnaire show that three nurses retrieved data, five residents entered data, and three residents retrieved and entered data in the medical part of the EPR. One nurse and three residents retrieved data, while three nurses entered and retrieved data from the nursing part of the EPR. Two of the interviewed residents had only used the EPR in night and weekend shifts, the other four had used it daily for some time. The interviewed residents indicated that few had used the EPR at the emergency department, which confirmed the questionnaire results. The latter results show that one resident used the EPR in every shift; two others used it in some shifts. Additionally, one resident used the EPR in every shift in the staff rooms at the ward, whereas three did this during some of their shifts. In the inpatient clinic the EPR was used most extensively with three users in every shift and 14 users in some of their shifts. The number of stroke patients at the ward varied each day with some days without any stroke patient, which explained that not in all shifts the EPR was used. Impact on Daily Work Interviewees were satisfied with the user friendliness of the screens, and they stated that the EPR was easy to use. Furthermore, they assessed positively that the EPR integrated medical and nursing information into one patient record, although they did not benefit a lot in this project. In the daily routines of ward rounds and shift handovers, the paper medical and nursing records remained the primary source of information. The EPR caused extra work for the nurses and residents, because they kept both paper and electronic records for stroke patients. As main reason respondents cited that backup facilities were not realised. The second reason was that users did not fully rely on the EPR, because of the (perceived) software problems. The third reason that respondents posed was that only part of the ward used the EPR which meant that at handovers paper records needed to be available. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

268 van der Meijden

According to the interviewees, consulting physicians refused to use the EPR, because it was unfamiliar to them. In addition, the EPR was too specific and lacked links with other relevant data sources. Some interviewees–who had used the EPR a few times–complained about the difficulty to get an overview of the progress and present state of a patient. Organisational Impact None of the interviewees had personally perceived any influence on the communication between nurses and physicians, not in the ward handovers, nor during ward rounds. Moreover, none had experienced a shift in responsibilities between residents and nurses. In addition, not one of the interviewees expected such to occur in the future. User Needs Part of the questionnaire concerned the desired functionality of IT applications in healthcare. The questionnaire results are shown in the appendix. In general, users’ opinions did not change drastically, the majority of the functions or features that were felt important in the baseline study, were so in the evaluation phase. One exception was that the percentage of respondents, who wished that implementing an information system could occur without changes to the established clinical routines, increased from 26% to 42% after the EPR implementation. None of the interviewees expressed additional wishes with respect to the content of the stroke EPR, because in their opinion it was complete for its purpose. Reliability and constant availability were indicated as a primary requirement in interviews and in the questionnaire results. A second basic requirement following from the questionnaires and interviews was accessibility of the EPR. Questionnaire results showed that, like in the baseline questionnaire, most respondents wanted access to the EPR where they needed it. Also in the interviews, respondents repeatedly stressed the necessity of availability at the patient’s bedside, illustrated with the next quote.



It is most convenient to do that at the patient’s bedside; to write down the answers of a patient immediately. If I have to go to a computer three doors further or two meters away, it does not work. (a resident) A third requirement was to have a single EPR for the whole hospital to prevent for example the problems with the consulting physicians who refused to use the EPR. A related request was a link between all availably patient data repositories, like the HIS and an EPR. Furthermore, several users advised to gradually implement a new information system by starting hospital wide with a limited number of functions and extend the information system function by function. User Expectations of IT in Healthcare Expectations users had of IT applications in healthcare hardly changed after the implementation of the EPR (see the Appendix). Both before and after the implementation, respondents did not expect many effects of IT applications on healthcare. When asked for their expectations of electronic record keeping in general most interviewed nurses stated to have had few expectations beforehand. The only expectation they had was a reduction of double work and copying of data, which was not realised. One resident regarded legibility and no loss of data as advantages of an EPR. The integration of a nursing and medical record could have an effect on patient care, according to some interviewees. General aspects as multisite availability, continuous availability, and timeliness of data, however, would not. Two residents expected that their record keeping time, in comparison with paper record keeping, would increase permanently with electronic record keeping. Most respondents supposed that daily routines like the frequency of documenting patient data, ward rounds, shift handover would not change. Additionally, nurses did not expect the quality of care to change, since the [patient] care must be organised and given, whether I write or type my information. (a nurse)


270 van der Meijden

Management’s Point of View In the baseline measurements and after the trial period of the EPR the head of the department, the head nurse, the department’s informatician, and a senior neurologist (only after implementation) were interviewed about the project and the EPR, and their expectations. Expectations of Electronic Record Keeping Both heads participated in this project because they strived for higher quality record keeping. Their expectations did not change during the project. In their view, paper records were often incomplete, and their expectation had been that an EPR was a possible means to achieve quality improvement. They expected the integration of the medical and the nursing record in the EPR to improve record keeping, to reduce the copying of data and to prevent the transfer of information by telephone or quickly scribbled notes. Particularly for nurses the last aspect would reduce interpretation errors and uncertainty about responsibilities. The informatician and the neurologist had been rather sceptical about the chances of success of this project, primarily because the EPR was for stroke only and because they had had negative experiences in the past with ICT projects in daily routines. Evaluation of the EPR Project The development approach, prototyping, was considered valuable. The manager who had been closely involved in this process was very satisfied with that way of incrementally building an EPR. Prior to the implementation, the stability of the database had been a source of concern. However, the database had never become unstable and it had never been restored during the trial. Nonetheless, most of the problems management had expected beforehand occurred after or during implementation. All agreed that, although all parties had done their best, the project could have been better prepared. Management noted that the project was too much a standalone project. The EPR did not really fit into or onto the existing HIS and that made consulting physicians reluctant to use the EPR. They also noted that collaboration with the hospital’s ICT department started rather



late. Moreover, contrary to initial agreements, backup facilities could not be realised. Management agreed with EPR users that they had to maintain a paper record to have a back up and to have a patient record for other disciplines, making it extremely difficult to fully exploit the EPR and abandon paper completely. In addition, the project budget was too tight. Afterwards all concluded that a next EPR project should be better embedded in the organisation, meaning that the ICT department and other disciplines should be involved in such a project in a very early stage. Own Role: This Time, Next Time All managers were asked for their role in the project and what they would do differently a next time. None of them was unsatisfied. However, they stressed the importance of embedding such a project in the existing organisational and information structures. A next time they would improve communication within the department and within the organisation to create support for such a project or system, support within the department, but also in the hospital. Management acknowledged that users had few expectations of the EPR, partly because management had insufficiently informed and encouraged them: We did not enough marketing, you could say. (a manager) Also, the organisation had not been involved until late in the project. In a next project, the ICT department and other disciplines will be involved earlier and more closely. They will also try to have such a project and such a system embedded in the organisation’s information systems and strategy.

Discussion In medical informatics, the focus of information systems evaluations is shifting from a purely technical focus to studying the interaction of users with the system and their surroundings. Social, organisational, political and other nontechnical factors are often insufficiently addressed in purely


272 van der Meijden

quantitative studies, while they may be very important for a successful implementation. Therefore, several authors advocated to alter the approach to healthcare information system evaluations and to include qualitative methods (Anderson & Aydin, 1997; Beuscart-Zephir, Brender, Beuscart, & MenagerDepriester, 1997; Forsythe & Buchanan, 1991; Kaplan, 2001). In agreement with these developments, we adopted an approach that combined quantitative and qualitative methods. As our results show, the different data collection methods yielded rich data, supplementing each other, and providing a rather complete picture. In particular, questionnaire and interview results complement each other well. The questionnaire results reflected the general feeling users had. In the interviews users were stimulated to reflect on specific aspects to refine their general judgements. Just that made our evaluation valuable, since it can, for example, explain discrepancies between questionnaire and interview results regarding user friendliness of the EPR. As suggested previously (Kaplan, 1997), we did not restrict to technical issues. We included a variety of concerns like quality of record keeping, user satisfaction, quality of the EPR, as well as usage behaviour. Not only the point of view of users-nurses and physicianswas included, but also management’s opinions were. In addition, the evaluation started before the development of the EPR. We described a framework to evaluate patient care information systems (van der Meijden et al., 2003). In our evaluation, we had the opportunity to test the usability of our framework. It was a useful framework to organise our evaluation. The attributes assessed, like completeness of record keeping, collaboration of healthcare workers, direct benefit, impact on patient care, and so forth, were systematised clearly and were, in our opinion, relevant aspects. Furthermore, in this evaluation study we tried to answer several research questions. Our first research question was, What consequences does this EPR have on the completeness of record keeping? As the results showed, the electronic records were more complete than their paper counterparts and than the paper records before the EPR implementation. This seems to be a result of the EPR, since there is no difference between the completeness of paper records before and after implementing the EPR. Given the situation, we had not expected any improvement. After all, the paper record remained the main data source and users were reluctant to use the EPR. Nonetheless, as was reported previously, this shows that an EPR can improve the completeness of record keeping (Tang, Fafchamps, & Shortliffe, 1999). It also confirms that inconsistencies occur when patient records are kept in parallel (Aasly, 2001).



A limitation of our study is that we selected only intake items for completeness analysis. It can be argued that this list could have been extended to several other, general, clinically relevant items. However, we focused on timely and adequate diagnosis upon admission and the neurologist selected the items accordingly. In addition, if the completeness of the already structured intake items improved, an improvement may be expected for other items that were unstructured on paper and structured in the EPR (Shiffman, Brandt, & Freeman, 1997). Further, besides completeness, correctness could be a valuable attribute of quality of documentation (Brennan & Stead, 2000; Hogan & Wagner, 1997). Our second research question was, What consequences does this EPR have on the daily work of nurses and physicians? The integration of medical and nursing record was acknowledged to have potential benefits in an EPR-only situation. However, the double work in documenting data on paper and in the EPR was experienced as a burden, making it very hard to discern any benefits. The extra work was the main impact on daily work. Nevertheless, we sustain our expectation of positive results on healthcare provider efficiency and quality of care in situations without double record keeping (Garrett, Hammond, & Stead, 1986; Tang, 1994; Tange, Schouten, Kesters, & Hasman, 1998). Additionally, it might be expected that an EPR proves useful with a lot of historical information of many (readmitted) patients. This benefit, however, is realised only after a considerable period of regular use. The third research question, What consequences does this EPR have on the opinions of nurses and physicians about electronic record keeping?, was somewhat difficult to answer. In general, users judged the EPR depending on their experience with it. In the interviews a difference existed between frequent and infrequent users regarding the user friendliness. Infrequent users were unfamiliar with the EPR, and therefore, they had to search for the location of items. Once found, also they found data easy to enter and easy to interpret. Possibly the software problems that could not be reproduced were difficulties encountered by users who were not very familiar with the EPR. It is plausible that their expectations of how the EPR would respond to specific actions did not correspond with the true responses. Hence, the structured data entry forms provided potential benefits, but only after users had become familiar with the EPR. The fourth research question was, How is the usage behaviour of users? Usage behaviour is likely to depend on the familiarity with the EPR. We had expected that at the start of the trial users would collect many data to enter them all at once and that they would enter more frequently and less data at once


274 van der Meijden

toward the end of the trial. We were unable to analyse this carefully, because the paper records remained the primary source of information. Our next research question was, What are management’s experiences? Management focussed on two themes. The first was that the contribution of others was underestimated. The second issue they addressed and would improve in a future project was the communication within the department and within the entire organisation. According to them, communication had been insufficient. Nevertheless, they were encouraged in their initial expectations that electronic record keeping had potential and was feasible. The objective of the current evaluation was to study what consequences an EPR would have in daily practice. The explorative character of qualitative methods made them valuable to realise this aim. Furthermore, the intention was to have an incremental and formative design of our EPR. This was realised by the many informal contacts between developers, researchers and users during the trial. These contacts were a source of continuous input to improve the EPR. The most suitable study design in this setting was a before–after study design: the intervention was on a very small group of users and there was no comparable ward available. Nevertheless, we recommend further research to best possible designs to evaluate healthcare information systems.

Was the Project Successful? In terms of success and failure, this project may be considered a relative success, although not all initial objectives were met. For example, not all healthcare providers involved in the care for stroke patients used the EPR and of only part of the stroke patients all data were entered in the EPR. Nevertheless, as a learning trajectory this project was successful. This was illustrated with the following: a) the EPR had been up and running and contained all essential information; b) the department had become aware of the problems and pitfalls when introducing an EPR; and c) management had seen that an integrated medical and nursing electronic record could be realised and had users seen actually using this EPR in daily practice, even users who were complete computer-illiterate. For several reasons the EPR never replaced the paper medical and nursing records. First, paper forms were maintained for data collection for those other than stroke patients. One choice that influenced the course of the project was to limit the scope of the EPR to one disease, albeit an important one



in the field of neurology. During the trial the number of stroke patients varied a lot, and the number of non-stroke patients increased. As a consequence, users had to maintain more paper records than anticipated. Second, without the back up facilities users had to maintain their paper records besides the EPR for legal reasons. In practice, whether or not the EPR was filled in depended on the time left. Third, insufficient communication with other departments in the hospital partly caused the lack of backup and support facilities. Also the refusal of consulting physicians to use the EPR originated in suboptimal communication. Fourth, a tight budget resulted in a reduction of the initial number of PCs, which, in turn, was perceived as an inadequate availability and accessibility of the EPR by the users. Accessibility of the EPR was limited to five workstations, while users wanted the patient record at the patient’s bedside. The consequence was that they had to scribble notes on a piece of paper and copy it into the EPR later. Hence, users lost in this respect. Fifth, a lack of experience with this kind of projects resulted in an underestimation of both costs and efforts that would have been necessary to create the conditions for success. A set of functional requirements for the EPR was identified prior to the project. More complete requirements resulted from the project. An important addition was the integration of different data sources. Initially, the coexistence of the HIS and the EPR at each PC had been expected to suffice. However, after implementation, management recognised a need for further integration an EPR in the information systems already in use in the hospital. Users even requested to have one single EPR for the whole hospital with the HIS data integrated. More likely, the hospital would prefer one user interface and common functionalities in all systems. Regardless the technical realisation, the users need to perceive different systems as one (Kuhn & Giuse, 2001). To achieve this, a gradual expansion of functionality–hospital wide–might be a better way to full electronic record keeping (Atkinson & Peel, 1998), although this might invite inpatient departments to develop own EPR initiatives.

Conclusions The main reason for this project had been the need to support information exchange in the process of care of stroke patients. The expectation was that an EPR could provide enough functionality to fulfil that need. This pilot was started because hospital developments toward electronic record keeping


276 van der Meijden

advanced slowly. Hospital wide electronic record keeping has not been realised five years after the start of the pilot. The results of our evaluation also show that the respondents who were familiar with computers and with the EPR were more positive in their judgments. In other words, it will take some time before users learn to appreciate an EPR and before they can identify possibilities instead of only limitations. Clear short term and longterm benefits that appeal to users can support this trajectory. This evaluation showed once more that communication is crucial for success (Lorenzi & Riley, 1994, 2000). Communication with all stakeholders, users and the information department is important; in fact, the whole organisation should be adequately and timely informed to create sufficient support. This pilot yielded requirements for an improved version of an EPR and showed that an integrated EPR can support information exchange. It also provided insight in the problems surrounding electronic record keeping and the opportunities it can create.

References Anderson, J. G., & Aydin, C. E. (1997). Evaluating the impact of healthcare information systems. Int J Technol Assess Healthcare, 13, 380-393. Atkinson, C. J., & Peel, V, J, (1998). Transforming a hospital through growing, not building, an electronic patient record system. Meth Inform Med, 37, 285-293. Beuscart-Zephir, M. C., Brender, J., Beuscart, T., & Menager-Depriester, I. (1997). Cognitive evaluation: How to assess the usability of information technology in healthcare. Comput Methods Programs Biomed, 54, 19-28. Brennan, P. F., & Stead, W. W. (2000). Assessing data quality: From concordance through correctness and completeness to valid manipulatable representations. JAMIA, 7(1), 106-107. Dick, R. S., & Steen, E. B. (Eds.). (1991). The computer-based patient record: An essential technology for healthcare. Washington, DC: National Academy press.



Forsythe, D. E., & Buchanan, B. G. (1991). Broadening our approach to evaluating medical information systems. Proc 15th Annu Symp Comput Appl Med Care, 8-12. Garrett, L. E., Hammond, W. E., & Stead, W. W. (1986). The effects of computerized medical records on provider efficiency and quality of care. Meth Inform Med, 25, 151-157. Hogan, W. R., & Wagner, M. M. (1997). Accuracy of data in computerbased patient records. JAMIA, 4, 342-355. Kaplan, B. (1997). Addressing organizational issues to the evaluation of medical systems. JAMIA, 4, 94-101. Kaplan, B. (2001). Evaluating informatics applications: Some alternative approaches: Theory, social interactionism, and call for methodological pluralism. Int J Med Inf, 64, 39-56. Kuhn, K. A., & Giuse, D. A. (2001). From hospital information systems to health information systems: Problems, challenges, perspectives. In R. Haux & C. A. Kulikowski (Eds.), Imia yearbook (pp. 63-76). Schattauer. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int J HumanComput Interact, 7, 57-78. Lorenzi, N. M., & Riley, R. T. (1994). Organizational aspects of health informatics. Managing technological change. New York: SpringerVerlag. Lorenzi, N. M. & Riley, R. T. (2000). Managing change: An overview. JAMIA, 7, 116-124. Mikkelsen, G., & Aasly, J. (2001). Concordance of information in parallel electronic and paper based patient records. Int J Med Inf, 63, 123-131. Shiffman, R. N., Brandt, C. A., & Freeman, B. G. (1997). Transition to a computer-based record using scannable, structured encounter forms. Archives of Pediatrics and Adolescent Medicine, 151, 247-1253. Tang, P. C., Fafchamps, D., & Shortliffe, E. H. (1994). Traditional medical records as a source of clinical data in the outpatient setting. Proc Annu Symp Comput Appl Med Care, 575-579. Tang, P. C., LaRosa, M. P., & Gorden, S. M. (1999). Use of computer-based records, completeness of documentation, and appropriateness of documented clinical decisions. JAMIA, 6(3), 245-251.


278 van der Meijden

Tange, H. J. (1995). The paper-based medical record: Is it really obsolete? Comput Methods Programs Biomed, 48, 127-131. Tange, H. J. (1999). Consultation of medical narratives in the electronic medical record. Meth Inform Med, 38, 289-293. Tange, H. J., Dreessen, V. A., Hasman, A., & Donkers, H. H. (1997). An experimental electronic medical-record system with multiple views on medical narratives. Comput Methods Programs Biomed, 54(3), 157-172. Tange, H. J., Schouten, H. C., Kester, A. D. M., & Hasman, A. (1998). The granularity of medical narratives and its effect on the speed and completeness of information retrieval. JAMIA, 5(6), 571-582. Van der Meijden, M. J., Tange, H. J., Boiten, J., Troost, J., & Hasman, A. (2000a). An experimental electronic patient record for stroke patients. Part 2: System description. Int J Med Inf, 58/59, 127-140. Van der Meijden, M. J., Tange, H. J., Boiten, J., Troost, J., & Hasman, A. (2000b). An experimental electronic patient record for stroke patients. Part 1: Situation analysis. Int J Med Inf, 58/59, 111-125. Van der Meijden, M. J., Tange, H. J., Troost, J., & Hasman, A. (2001). Development and implementation of an EPR: How to encourage the user. Int J Med Inf, 64, 173-185. Van der Meijden, M. J., Tange, H. J., Troost, J., & Hasman, A. (2003). Determinants of success of inpatient clinical information systems: A literature review. JAMIA, 10, 235-243.



Appendix: Results Baseline and Evaluative Questionnaires * satisfied (%) baseline EPR MR NR

neutral (%) baseline EPR MR NR

dissatisfied (%) baseline EPR MR NR

overall satisfaction easy to learn availability at any time availability on any location completeness of: diagnostic interventions+ therapeutic interventions+ decisions made orders made+ plans made+ Data entry

59 54 60 60

89 59 79 75

6 18 12 24

31 27 23 20

4 26 11 14

29 35 18 18

10 19 17 20

7 15 11 11

65 47 71 59

53 59 53 47 47

46 46 52 70 52

44 47 25 41 40

23 24 30 40 37

42 42 37 30 37

19 13 38 24 20

23 17 17 13 17

12 12 11 0 11

44 40 38 35 40

ease of data entry speed of data entry lay-out of the record is adequate to enter data flexibility in deciding which data to register where, and in which words Data retrieval

70 73 46

78 69 63

53 12 35

19 23 46

7 15 26

18 29 35

11 4 7

15 15 11

29 59 29

61

67

35

25

26

18

14

7

47

ease of information retrieval ease of retrieving own data ease of retrieving others’ data understanding data: own entries+ others’ entries+ speed of getting an overview of state/progress of treatment: first impression full impression lay-out of the record is adequate to retrieve data speed of finding specific data speed of identifying missing data+ information retrieval not impeded by: overload of irrelevant details+ overlooking relevant data User interface

53 70 21

68 70 46

13 6 13

20 23 34

21 15 27

25 56 38

27 7 45

11 15 27

63 38 50

85 39

81 63

80 38

7 36

8 22

20 56

7 25

12 15

0 6

41 18 36

37 33 52

6 0 25

24 36 43

33 30 37

25 19 50

34 46 21

30 37 11

69 81 25

17 17

26 22

0 31

14 17

22 37

25 38

69 66

52 41

75 31

17 38

33 52

33 29

41 45

26 15

40 47

41 17

41 33

27 24

Satisfaction General aspects

clear screen messages easy to undo mistakes quick to undo mistakes clear manual clear on-line help data are easy to understand § data support daily work organisation of data on screens § user friendly screens § all necessary and expected functionality general user friendliness

0 0 0 25 0 53 12 53 47 6 6

25 18 12 56 64 29 35 35 35 18 18

75 82 88 19 36 18 53 12 18 76 76

* MR = paper medical record, NR = paper nursing record; + EPR equals or surpasses the paper medical and/or paper nursing record; § positive aspects of user interface


280 van der Meijden Desired features/ functionality

I can enter information in my own words and not need to know any special codes I can learn to use the system in less than 2 hours I can access the system at any place in the clinical setting The system always responds to my queries in less than 5 seconds I can interact with the computer without a keyboard. The system can be implemented with no changes whatsoever to existing clinic routines The system is always functioning. There is never any "downtime" When a system provides medical advice on the care of specific patients, it always provides multiple alternative recommendations When a system provides medical advice on the care of specific patients, it can quantify the level of certainty inherent in its recommendations The system takes a patient's own preferences into account when giving advice to clinicians The system can clearly explain the rationale for advice it gives on the care of patients Users can browse the information in a system as well as asking it to provide advice about care of specific patients The system has been demonstrated in research studies to make diagnoses at least as accurate as human consultants The system has been demonstrated in research studies to provide treatment recommendations at least as accurate as human consultants Number of respondents

Expected Effects Number of respondents Costs of healthcare Clinician autonomy Quality of healthcare Interactions within the healthcare team Role of the government in healthcare Management of medical/ethical dilemmas Enjoyment of the practice of medicine Status of medicine as a profession Continuing medical education The self-image of clinicians Humaneness of the practice of medicine The rapport between clinicians and patients Personal and professional privacy Clinicians’ access to up-to-date knowledge Patients' satisfaction with the quality of care they receive

Absolutely necessary (%) Before After EPR EPR 33 26

Absolutely unnecess. (%) Before After EPR EPR 3 5.3

20 70 40 5 26

16 84 42 5.3 42

5 0 0 35 8

5.3 0 5,3 26 0

72 32

79 42

3 11

5.3 5.3

24

22

11

0

21

29

10

0

33

41

8

0

13

16

13

11

32

41

3

12

32

41

5

12

40

19

Before EPR 42 3.0 3.0 3.7 3.6 3.1 3.2 3.1 3.3 3.6 3.4 3.0 3.0 3.0 4.1 3.3

After EPR 19 3.2 2.9 3.4 3.5 3.1 2.8 2.9 3.1 3.5 3.1 2.6 2.7 2.9 3.7 3.1


E-Health Systems: Their Use and Visions for the Future 281

Chapter XVI

E-Health Systems: Their Use and Visions for the Future Pirkko Nykänen Tampere University, Finland

Abstract E-health refers to use of information and communication technologies to improve or enable health and healthcare. E-health broadens the scope of healthcare delivery; citizens are in the center of services and services are offered by information systems often via the Internet. In this chapter ehealth systems are classified on the basis of their use and their functionality and the use is discussed from the viewpoints of citizens and health professionals. Citizens are increasingly using Internet and e-health systems to search for medicine or health-related information, and they become better informed and may take more responsibility of their own health. Health professionals are more reluctant to use the Internet and e-health systems in physician-patient communication due to the power and responsibility problems of decisions. In the future the sociotechnical nature of e-health should be considered and future systems developed for real use and user environment with user acceptable technology. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

282 Nykänen

Introduction In the information society it is important to develop and apply technologies in such a way that we empower citizens to play a full role. An essential part of the information society, healthcare services are needed by citizens and should be provided efficiently and made accessible to all (Haglund, 2002). With the information society a new concept, e-health, has been introduced to refer to the use of emerging information technology to improve or enable health and healthcare. Silber (2003) defined e-health as “application of information and communication technologies (ICT) across the whole range of functions that affect health” (p. 3). Eysenbach (2001) gave a broader definition for e-health: An emerging field in the intersection of medical informatics, public health and business, referring to health services and information delivered or enhanced thorough the Internet and related technologies. Alvarez (2002) emphasised the consumer-viewpoint when he defined e-health as a consumer-centered model of healthcare where stakeholders collaborate, utilising ICT and Internet technologies to manage health; arrange, deliver, and account for care; and manage the healthcare system. All these definitions support the conception that e-health means application of information technologies to promote health, and to support healthcare services delivery and use. E-health covers all health strategies: Prevention, treatment, and rehabilitation. It is essential that e-health applications meet the needs of citizens, patients, healthcare professionals, and policy makers. Therefore, evaluation studies are needed to assess the benefits, effects, and impacts of e-health on citizens, professionals, healthcare systems, and healthcare outcomes. E-health conceptualization broadens the scope of healthcare delivery; citizens are placed at the centre of services, services are in many situations offered to be used through the Internet (e.g., at home) and citizens can have interaction with health professionals who look after their health needs (Silber, 2003; Wilson, Leitner, & Moussalli, 2004). E-health is expected to contribute to development of new ways of delivering health services and to impact on the organisation and structure of the healthcare delivery system. E-health is not only a technological improvement, it is a reengineering of healthcare processes and is of consideration of the sociotechnical aspects of design and development of applications.



E-Health Systems E-health applications should make citizens better informed. All citizens should have access to services, use of services should be economically affordable, and citizens should benefit from the use of services. On the other hand, e-health services should improve the quality, availability and effectiveness of healthcare (Grimson et al., 2000; Silber, 2003; Wilson, 2002).

Types of E-Health Systems Traditionally, three broad categories of e-health applications can be identified: Delivery of care to patients by healthcare professionals, education and dissemination of health-related information and knowledge, and trading health products (Ruotsalainen et al., 2003). The first category covers systems for delivery of care to patients by healthcare professionals, including a wide range of applications from pure administrative to those for care delivery: •

•

•

Hospital systems: Scheduling systems, logistics systems, management information systems, hospital and patient administration systems, laboratory information systems, radiology information systems, pharmacy systems, nursing systems, and networked services such as electronic messaging between the hospital and other healthcare actors for communication of clinical information and administrative data, including telemedical services such as telepathology and teleconsultation for remote areas. Primary care systems: Information systems for general practitioners, pharmacists, and dentists for patient management, medical records, electronic prescribing, and information exchange. Home care systems: Systems that are used to deliver care services via telecommunication or wireless to the patient at home. Examples of such systems are remote vital-signs monitoring systems that enable the patient to receive targeted treatment and medication without the need to visit an outpatient clinic or occupy a hospital bed. These kinds of systems are particularly well developed in diabetes medicine, hypertension management, asthma monitoring, and home dialysis.


284 Nykänen

The second category covers systems for education and dissemination of health related information and knowledge including web-portals and specific health-related Web sites, virtual hospitals, and Internet-based consultation services. These systems may be targeted for: • •

Medical consultation, search for the second opinion, search for health-, disease-, or treatment-related information, Medical education and dissemination of medical publications, preventive materials, and public health related information.

These dissemination systems may help citizens to become informed and empowered through information and knowledge they are able to retrieve and access themselves from the Internet sources. However, the quality and validity of information can be questioned with many Internet information sources (Wilson, 2002). To promote these kinds of Internet information sources and to ensure people that they can confidently and with full understanding of known risks access and use information from the Internet, initiatives like Health on the Net, HI Ethics Program, and Health Online Actions have been established to develop guidelines and quality criteria and to promote codes of ethics for health-related Web sites (Spink et al., 2004; Wilson, 2002). The third category covers systems that are developed to trade health related products. e-commerce or e-trading of medical products, healthrelated goods, pharmaceuticals, and medical devices is a growing e-health area, and current procedures enable citizens to enter Internet shopping in an easy and secure way. E-health applications can also be classified according to their functionality and, following this principle, we may currently find the following groups of applications (Ruotsalainen et al., 2003): •

Regional health information networks that deploy advanced healthcare services at various levels of healthcare delivery system, including primary care, prehospital health emergency management, and hospital care. These systems are networked and implemented using various technologies. A typical feature for these systems is the integration of existing legacy systems, imaging systems, departmental systems, and



•

•

•

•

•

administrative systems into one network and development of new, innovative interfaces and applications to provide comprehensive services regionally. Hospital systems, clinical systems, diagnostic systems, hospital management systems that cover hospital information systems (HIS), various departmental systems like radiology information systems (RIS), pathology information systems, laboratory information systems (LIS), diagnostic systems like decision support, and knowledge-based systems and hospital management systems like accounting, resource management and booking systems. Telemedicine or teleconsultation systems are used to access an expert opinion, a second opinion, or to monitor remote patient at home or in another healthcare organisation. These systems are especially often planned to support delivery of medical expertise for rural areas. Insurance, cards, or systems that present the payer’s view on health services. These applications are developed to support the use of cards as means to access health services, to get information on the health insurance status, and to register users that are entitled to special services or reimbursements based on their health, age, or employment status. Citizen-centred systems, patient-centred systems and health information portals. These applications provide health-related information for patients and health professionals, and additionally they may provide possibilities for consultation services or for buying pharmaceuticals or other health-related products. Home care systems and health-related fitness systems. Home care systems are meant to monitor chronic diseases at home, to monitor elderly patients, or to teleconsult professionals from home. These systems are often based on wireless technology, such as mobile phones or handheld computers. Home care systems are also used to help in the management of care, in preparing care plans and in coordination of actions and tasks taken by members of care teams. Health-related fitness systems are those meant for healthy people that want to monitor their well-being and fitness.

We see from these typologies that enablers for most e-health systems are the electronic patient records and Internet-based technologies. Electronic patient


286 Nykänen

records make it possible to share medical information between the care providers across the health strategies and medical disciplines and facilitate consultation between care providers on a given patient. The electronic patient records also give possibilities for further networked applications such as electronic prescribing and integrated regional health information networks (Ruotsalainen et al., 2003). However, current electronic patient records still lack a uniform infrastructure for data exchange and systems are heterogeneous and lack agreed and shared vocabularies (Safran & Goldberg, 2000). This restricts sharing of information. Today many e-health systems are building more and more on e-commerce and e-government strategies and experiences on how to use Internet technologies to redesign operation of public services.

E-Health Systems Use by Citizens and Health Professionals The use of the Internet by citizens and also by health professionals is increasing worldwide (Budtz & Witt, 2002; Fallis & Fricke, 2002; Gruen, 1999; Holliday & Tam, 2004; Rodrigues & Risk 2003; Silber, 2003; Wilson et al., 2004). The Internet is used to access health-related information by people who are ill but also by people who are healthy who look for advice on healthy lifestyles, diets, habits, and health-related products (Jones et al., 2001). Accessing health information is one of the most common reasons for using the Internet: 50% to 75% of Internet users have used it to search for health-related information (Powell & Clarke, 2002). In 2003 approximately 62% of American Internet users searched the Internet for healthrelated information (Spink et al., 2004). In some cases citizens do not use tailored health-related Web sites but general Internet search engines to access health information. In 2003 40% of Europeans used the Internet to access healthcare services and information (Wilson et al., 2004). Budtz and Witt (2002) showed that of 93 patients in a study, 20% had used the Internet to get health information. In another study, Larner (2002) followed patients for 6 months and found that more than 50% of them had Internet access, and 82% of them were interested in accessing Web sites with relevant medical information. People using the Internet for health information searches reported that they value the convenience, anonymity, and volume of online information (Powell & Clarke, 2002). The patients who search for specific information on their diseases, diagnoses, and treatments report that it is



beneficial to have information, advice, and social support from the Internet (Potts & Wyatt, 2002). In 2002, on average 78% of European general practitioners were online (EHealth, 2004) and 48% of them used electronic patient record systems and, to some extent, other information systems to receive laboratory results and to transfer patient data to other healthcare organisations. Even 36% of general practitioners used telemedicine systems for home monitoring via the Internet or e-mail (Wilson et al., 2004). When analysing the use of e-health services and health information sources through the Internet, the key findings are that citizens want to have more information and they want better information (Wilson et al., 2004). The use of the Internet for health purposes is rising and citizens would like to have guidance from the health professionals regarding quality Web sites (Silber, 2003). The major reason for the citizens to use Internet information resources is to know more, to be able to ask more precise questions from health professionals, and to understand better health and well-being. Internet health resources support citizens to become better informed and knowledgeable, and through this they also may take more responsibility of their health and well-being. However, the Internet and other new technologies will interfere with the communication between health professionals and between them and the patient. Impacts of the Internet use do not only come from the communication but from the operational availability of information resources. Holliday and Tam (2004) found that more than 90% of patients had expressed their wish to communicate with their physician via e-mail, but physicians were reluctant to do so. Physicians explained that the physician-patient confidentiality, time concerns, and increased exposure for malpractice were the major reasons for their reluctance. Physicians are somewhat opposing the use of the Internet, especially when it interferes with their decision-making activity (Gerber & Eiser, 2001; Kleiner, Akers, Burke, & Werner, 2002). There seems to be two reasons: A question of power and a question of danger. A question of power implies that physicians want to keep the control on the medical activities, and the question of danger implies that they want to keep control on actions and decisions that are their responsibility. Some physicians even fear that they become technical executors of decisions taken from third parties (e.g., Internet information sources).


288 Nykänen

Expected Benefits by E-Health Systems Many studies (Alvarez, 2002; Ruotsalainen et al., 2003; Iliakovidis, Wison, & Healy, 2004) identify the promises of e-health to build on the advances in ICT to support the development of the healthcare infrastructure. Healthcare services are expected to be better accessible and data available any place and any time, independent from where data are stored or created. Healthcare professionals expect that e-health systems have remarkable impacts on healthcare routines and practices (Eng 2001; Grimson et al., 2000; Ruotsalainen et al., 2003; Wilson et al., 2004). Reliable and accurate information would be available easily and rapidly at the time and place where it would be needed. It would be possible to view information on the prior history of the patient and on diagnostic investigations to avoid redundant testing. Communication between the healthcare providers would be independent of their physical location, and this results in time savings and increased accuracy of diagnosis and effectiveness of care. Health professionals would be able to update their knowledge and expertise though online training sessions and they would be able to consult international colleagues in dealing with particularly complex cases (Ruotsalainen et al., 2003). Patients and citizens would benefit through better quality health information and services available to them (Ruotsalainen et al., 2003; Silber, 2003; Wilson et al., 2004). The possibility for home monitoring and treatment or follow-up would reduce the need for hospitalisation or travel in order to receive professional care. Patients would regulate with the consent the provision of their data to various healthcare providers. The possibility to control one’s own health data would facilitate and increase the mobility of citizens and patients, particularly those suffering of chronic conditions. E-health systems are expected to harmonise healthcare systems, thus enabling provision of seamless and continuous care (Grimson et al., 2000; Ruotsalainen et al., 2003; Wilson et al., 2004). Interorganisational cooperation would allow sharing of data and information whenever necessary. E-health systems are also expected to result in improved quality and costeffectiveness of healthcare systems. The fact that healthcare processes would be better documented and thus accountable would provide more possibilities for evaluation and quality assessment, and information could be



made available to support decision making and interventions at the public health level. Eysenbach (2001) listed the 10 e’s for e-health systems: improved efficiency in health care and decrease of costs, enhanced quality of care (e.g., by comparisons of care providers), evidence-based in the sense that the effectiveness and efficiency of interventions are proved by scientific evaluations, 4. empowerment of consumers and patients, 5. encouragement of a new relationship between the patient and the physician, 6. education of physicians through online resources, 7. enabling information exchange and communication, 8. extending the scope of health, 9. ethics as new forms of patient-physician interaction become possible, and 10. equity to have health care more accessible to all.

1. 2. 3.

There is at the same time a threat that those people who do not have money, skills, computers, and Internet connections cannot access e-health services. Despite of the availability of e-health systems and their potential benefits, e-health systems are not yet widely used in healthcare practices (E-Health, 2004). Iliakovidis et al. (2004) reported some examples of e-health systems which are now entering the European market. However, many promises of e-health research and development have not been fulfilled yet (Ruotsalainen et al., 2003). The major barriers (E-Health, 2004) include lack of commitment by healthcare authorities, missing interoperability of health information systems (which is mostly due to lacking conceptual models), ontologies, and sharable vocabularies. Additionally, the e-health developments by far have been rather technology-oriented and thus the developed systems are not easy to use, technology is expensive and the systems are vulnerable to changes in the environment. Maybe the most important reason for failures is that the sociotechnical nature of e-health systems have not been fully considered. Development of e-health systems requires thorough understanding of the health care work practices where systems are to be


290 Nykänen

installed. The needs of the users and the contextual aspects of the systems use should be the starting point for the development (Berg, 1999; Nykänen & Karimaa, 2004).

E-Health Visions for the Future The challenges for e-health are technical, social, economic and political. Eng (2001) pointed out that there is need for strong leadership from health policy makers and need for practical solutions for interoperable systems that are secure, respect confidentiality, and promote the best possible access to health care for all citizens. In our survey (Ruotsalainen et al., 2003), several of the respondents looked for a comprehensive e-enabled healthcare system, standardised and interoperable, which meets the necessary legal and functional requirements for optimal, seamless, cross-border delivery of services. Implementation of such a system is expected to lead to more equal, accessible, holistic and human-centred healthcare. In the future we suppose that several Internet-related and other trends will have influence on the design, content, functionality, dissemination, and use of future e-health systems. With globalisation, increasing number of ehealth resources will be developed overseas and for global audience (Eng, 2001). Thus, standardisation and cross-cultural factors will become increasingly important. The digital generation will most likely demand immediate access to information and will rely on online resources to inform health and other decisions (Eng, 2001). Wireless technologies may contribute to the growth of mobile e-health applications for both providers and consumers. Digital television may serve in the future as a cheap and easy platform to offer ehealth systems and information for large audiences. Personalisation and tailoring of applications to specific users will put emphasis on privacy and data security issues. There are many important questions still unanswered with e-health systems: Ethical and legal issues? Who will pay for the use? Who has access to ehealth systems? What are the standards, guidelines and good practices for development and use of these systems and technologies?



The basic requirements for the future e-health applications can be derived from the current health informatics and e-health situation. First, e-health applications should serve the users needs, the needs of citizens, patients, healthcare professionals, and healthcare organisations. Second, the applied technology should be user acceptable, cheap and not too cumbersome to use, and vulnerable to changes. Third, e-health applications should be based on the health care information systems infrastructure, they should be integrated with the environment and preferably also be interoperable. Fourth, standards should be applied on design and development. Fifth, security and safety issues have to be solved within the practice and legal frameworks. And finally, all e-health systems and applications should be evaluated to assess the effects and impacts and to find reasons for success or failure.

References Alvarez, R. C. (2002). The promise of e-health: A Canadian perspective. eHealth International, 1(4), 1-8. Berg, M. (1999). Patient care information systems and health care work: A sociotechnical approach. International Journal of Medical Informatics, 55, 87-101. Budtz, S., & Witt, K. (2002). Consulting the Internet before visit to general practice. Patients’ use of the Internet and other sources of health information. Scan. Journal of Primary Health Care, 20(3), 174-176. E-Health. (2004). Making better for European citizens: An action plan for a European e-health area. Commission of the European Communities. COM (2004), 356, Brussels, Belgium. Eng, T. R. (2001). The eHealth Landscape: A terrain map of emerging information and communication technologies in health and health care. Princeton, NJ: The Robert Wood Johnson Foundation. Eysenbach, G. (2001). What is e-health? [editorial]. J Med Internet Res, 1(2), e20. Fallis, D., & Fricke, M. (2002). Indicators of accuracy of consumer health information on the Internet: A study of indicators relating to information for managing fever in children at home. JAMIA, 9(1), 73-79.


292 Nykänen

Gerber, B. S., & Eiser, A. R. (2001). The patient-physician relationship in the Internet age: Future prospects and the research agenda. Journal of Med. Internet Res., 3(2), e15. Grimson, J., Grimson, W., Flahive, M., Foley, C., O’Moore, R., Nolan, J., et al. (2000). Multimedia approach to raising awareness of information and communications technology amongst health care professionals. International Journal of Medical Informatics, 58/59, 297-305. Gruen, J. (1999). The physician and the Internet: Observer or participant? MD Comput, 16(6), 46-48. Haglund, H. (2002). The significance of welfare services and their electronic applications for the business activities in the future. Retrieved February 11, 2003, from www.etampere.fi Holliday, I., & Tam, W. (2004). eHealth in the East Asian tigers. International Journal of Medical Informatics. Jones, R., Balfour, F., Gillies, M., Stobo, D., Cawsey, A. J. & Donaldson, K. (2001), The accessibility of computer-based health information for patients: Kiosks and the Web. Medical Information, 10, 1469-73. Iliakovidis, I., Wilson, P., & Healy, J. C. (2004). eHealth. Current situation and examples of implemented and beneficial e-health applications. Amsterdam: IOS Press. Kleiner, K. D., Akers, R., Burke, B. L., & Werner, E. J. (2002). Parent and physician attitudes regarding electronic communication in pediatric practices. Pediatrics, 109(5), 740-744 Larner, A. J. (2002). Use of the Internet medical Websites and NHS Direct by neurology outpatients before consultation. International Journal Clin. Pract., 56(3), 219-221. Nykänen, P., & Karimaa, E. (2004). Success and failure factors during development of a regional health information system: Results from a constructive evaluation study. Methods of Information in Medicine (accepted for publication). Potts, H. W. W., & Wyatt, J. C. (2002). Survey of doctors’ experience of patients using the Internet. J Med Internet Res, 4(1), e5. Powell, J., & Clarke, A. (2002). The WWW of the World Wide Web: Who, what and why. Journal of Med. Internet Res., 18, 4(1), e4.



Rodrigues, R. J., & Risk, A. (2003). eHealth in Latin America and the Caribbean: Development and policy issues. Journal of Med. Internet Res., 5(1), e4. Ruotsalainen, P., Nykänen, P., Doupi, P., Cheshire, P., Pohjonen, H., Kinnunen, J., et al. (2003), The state of eHealth in Europe [Rep. No. D1]. Stakes, Helsinki: MEDITRAV-project, EU IST 1999-11490, Safran, C., & Goldberg, H. (2000). Electronic patient records and the impact of the Internet. International Journal of Medical Informatics, 60, 7783. Silber, D. (2003). The case for ehealth. European Commission, Information Society, eHealth Conference, Atlanta, Belgium. Spink, A., Yang, Y., Jansen, J., Nykänen, P., Lorence, D., Ozmutlu, S., et al. (2004), A study of medical and health queries to Web search engines. Health Information and Libraries Journal, 21, 44-51 Wilson, P. (2002). How to find the good and avoid the bad or ugly: A short guide to tools for rating quality and health information on the Internet. BMJ, 9(324), 598-602 Wilson, P., Leitner, C., & Moussalli, A. (2004). Mapping the potential of eHealth. Empowering the citizen through eHealth tools and services. eHealth Conference 2004. European Institute for Public Administration. Maastricht, The Netherlands.


294

Wickramasinghe, Geisler & Schaffer

Chapter XVII

Assessing E-Health1 Nilmini Wickramasinghe Illinois Institute of Technology, USA Elie Geisler Illinois Institute of Technology, USA Jonathan Schaffer The Cleveland Clinic, USA

Abstract While healthcare is the biggest service industry on the globe, it has yet to realize the full potential of the e-business revolution in the form of ehealth. This is due to many reasons, including the fact that the healthcare industry is faced with many complex challenges in trying to deliver costeffective, high-value, accessible healthcare and has traditionally been slow to embrace new business techniques and technologies. Given that ehealth to a great extent is a macro-level concern that has far reaching micro-level implications, this chapter first develops a framework to assess a country’s preparedness with respect to embracing e-health (i.e., the application of e-commerce to healthcare) and from this, an e-health preparedness grid to facilitate the assessment of any e-health initiative.


Assessing E-Health

295

Taken together, the integrative framework and preparedness grid provide useful and necessary tools to enable successful e-health initiatives to ensue by helping country and/or organization within a country to identify and thus address areas that require further attention in order for it to undertake a successful e-health initiative.

Introduction Technology has the potential to help solve many of the problems faced by developed and developing countries alike; from improving healthcare delivery to opening up commerce opportunities. The number of nations offering ecommerce solutions is increasing every year, and consequently the number of workplaces that have Internet connection for business activities has been rapidly growing worldwide. Table 1 provides three estimates of e-commerce forecasts. These projections clearly highlight the important role of e-commerce on the global economy. However, as the Secretary General of the United Nations notes in his forward to the e-commerce and development report (UNCTAD, 2002), “knowing that an instrument is powerful is not enough to ensure that it will be put to the best use. We need to understand how it works, and how and when it should be used … and maximize its power.” Within the umbrella of e-commerce, one area, e-health, has yet to reach its full potential in many developed countries, let alone developing countries. Each country is positioned differently and has varying potential and preparedness regarding embracing e-commerce technologies generally and e-health in particular. Given the macro-level nature of many issues pertaining to the development of e-health (Alvarez, 2002), in order to be more effective in their e-health initiatives, it is important for countries to assess their potential, identify their relative strengths and weaknesses, and thereby develop strategies and policies to address these issues to effectively formulate and implement appropriate ehealth initiatives. To do this effectively, it is valuable to have an integrative framework that enables the assessment of a country’s e-health preparedness. This chapter proposes to develop such a framework that can be applied to various countries throughout the globe and from this to generate an e-health preparedness grid. In so doing, we hope to facilitate better understanding of ehealth initiatives and thus maximize their power.


296


Table 1. Worldwide e-commerce estimates and forecasts (in billion $) (Source: From United Nations Conference on Trade and Development, 2002. Available online from http://r0.unctad.org) Source

2000

2001

Forester IDC Emarketer

354.90

615.30

278.19

474.32

2002

2003

2004

2005

2006

2,293.50

3,878.80

6,201.10

9,240.60

12,837.30

4,600.00 823.48

1,208.57

2,367.47

(B2B only)

E-Health Reducing healthcare expenditure as well as offering quality healthcare treatment is becoming a priority globally. Technology and automation have the potential to reduce these costs (Institute of Medicine, 2001; Wickramasinghe, 2000), thus, e-health, which essentially involves the adoption and adaptation of e-commerce technologies throughout the healthcare industry (Eysenbach, 2001), appears to be a powerful force of change for the healthcare industry worldwide. Healthcare has been shaped by each nation’s own set of cultures, traditions, payment mechanisms, and patient expectations. Therefore, when looking at health systems throughout the world, it is useful to position them on a continuum ranging from high government involvement (i.e., a public healthcare system) at one extreme, to little government involvement (i.e., private healthcare system) at the other extreme, with many variations of a mix of private–public in between. However, given the common problem of exponentially increasing costs facing healthcare globally, irrespective of the particular health system one examines, the future of the healthcare industry will be partially shaped by commonalties such as the universal issue of escalating costs and the common forces of change including a) empowered consumers, b) e-health adoption and adaptability, and c) shift to focus on the practice of preventative versus cure driven medicine, as well as four key implications, namely a) health insurance changes, b) workforce changes and changes in the roles of stakeholders within the health system, c)


Assessing E-Health

297

organizational changes and standardization, and d) the need for healthcare providers and administrators to make difficult, yet necessary choices regarding practice management. E-health is a very broad term that encompasses various activities related to the use of many e-commerce technologies and infrastructures, most notably the Internet, for facilitating healthcare practice. The World Health Organization (WHO, 2003), a major world health body, defines e-health as “being the leveraging of the information and communication technology (ICT) to connect provider and patients and governments; to educate and inform health care professionals, managers and consumers; to stimulate innovation in care delivery and health system management; and, to improve our healthcare system”. In contrast, a technologically oriented definition of e-health is offered by Intel, which refers to e-health as “a concerted effort undertaken by leaders in healthcare and hi-tech industries to fully harness the benefits available through convergence of the Internet and healthcare”. Health professionals are increasingly being drawn into evaluating the Internet as a source of consumer information for health and medicine. Practitioners report that a growing number of patients arrive at their offices either with questions related to appropriate Web sites to visit or a large variety of healthrelated content gathered from the Internet. Some of this content may prove extremely helpful to the health and/or recovery of a patient. Because the Internet has created new opportunities and challenges to the traditional healthcare information technology industry, the advent of e-health seems fitting to address both these opportunities and challenges. The new possibilities for facilitating effective healthcare delivery fall primarily into the following main categories: a) the capability of healthcare consumers to interact with their providers online (B2C = “business to consumer”), b) the possibility to improve healthcare institution-to-institution transmissions of data (B2B = “business to business”), and c) The new possibilities for peer-to-peer communication of healthcare consumers (C2C = “consumer to consumer”). A more comprehensive definition of e-health would need to incorporate the healthcare, business, and technological perspectives; hence we define e-health as an emerging field at the intersection of medical informatics, technology, public health, and business. Thereby, e-health entails the delivery of health services and health information through the Internet and other related ecommerce technologies. In a broader sense, the term characterizes not only a technical development, but also a state-of-mind, a paradigm shift, and a


298


commitment for networked global thinking to improve healthcare locally, regionally, and globally by using information and communication technologies (Geisler,1999).

Web of Players in Healthcare Figure 1 depicts the Web of healthcare players and the key elements of the ehealth architecture that serves to support the interactions between and within this Web of players. What is of particular importance when trying to assess ehealth initiatives is a) it is imperative to understand the implications of any ehealth initiative on all the players who make up this Web, and b) to realize the full potential of the Internet or Web-based technologies that make up the IT architecture so that the most effective and efficient e-health initiative is, in fact, designed and implemented. To get a better appreciation for this we must first understand the traditional competitive forces affecting any organization and then the role of the Internet in impacting these forces (Geisler, 2001).

Traditional Competitive Forces The strategy of an organization has two major components (Henderson & Venkatraman, 1993). These are a) formulation-making decisions regarding the mission, goals and objectives of the organization, and b) implementationmaking decisions regarding how the organization can structure itself to realize its goal and carryout specific activities. For today’s healthcare organizations, the goals, mission and objectives all focus around access, quality and value and realizing this value proposition for healthcare then becomes the key (Wickramasinghe et al., 2004). Essentially, the goal of strategic management is to find a “fit” between the organization and its environment that maximizes its performance (Hofer, 1975). This then describes the market-based view of the firm, developed by Michael Porter (1980). The first of Porter’s frameworks is the generic strategies. The use of technology must always enable or enhance the businesses objectives and strategies of the organization. This is particularly true for 21st century organizations, where many of their key operations and functions are so heavily Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Assessing E-Health

299

Drugs Dispensed

Prescriptio Prescriptions

DI Reports

Prescriptions ASP Site

Lab Tests

G Private Network

B

DI Reports

Lab Tests

Healthcare Organization Patient

Provider

Supplier

e-Health

Payor

Regulator

D

A

Prescriptions

C linic – L oc a P O S

Clinic – ASP POS

E

C

Internet

Pharmacy

Response

Web Site

Pharmaceutical Information Network

Drugs Dispensed

F

Response

Request WAN

Request

Client-Server Computing Paradigm based Architecture

Figure 1. Web of e-health players (Source: Wickramasinghe et al., 2004)

reliant on technology and the demand for information and knowledge is so critical. A firms’ relative competitive position (i.e., its ability to perform above or below the industry average) is determined by its competitive advantage. Porter (1980) identified three generic strategies that impact a firm’s competitive advantage. These include cost, focus, and differentiation. Furthermore,


300


Porter (1980) noted that two, and only two, basic forms of competitive advantage typically exist: cost leadership and differentiation. Firms can use these two forms of competitive advantage to either compete across a broad scope of an industry or to focus on competing in specific niches, thereby leading to three generic strategies. Porter (1980) noted that firms should be cautious about pursuing more than one generic strategy, namely cost, differentiation, and focus. For example, if a cost leadership strategy is adopted, it is unlikely that a firm can also maintain and sustain differentiation, because it would not be possible to simultaneously pursue the costly capital investment or maintain high operating costs required for differentiation and, thus, in the long run, the firm has a confused strategy which leads to failure. In order to design and develop one’s strategy, an organization should first perform an industry analysis. Porters five forces (or competitive forces) model is most useful (Porter, 1980, 1985). Figure 2 depicts this model. Essentially, Porter has taken concepts from microeconomics and modeled them in terms of five key forces that together outline the rules of competition and attractiveness of the industry. The forces are as follows: 1.

Threat of new entrant: a company new to the industry that could take away market share from the incumbent firms

Figure 2. Porter’s competitive (five) forces model EXISTING COMPETITION EXISTING

COMPETITON

THREAT OF SUBSTITUTES THREAT OF

SUBSTITUTES

BARGAINING BARGAINING POWER OFPOWER OF SUPPLIERS SUPPLIERS

FIRM

THREAT OF THREAT OF NEW ENTRANTS NEW ENTRANTS

BARGAINING BARGAINING POWER POWER OF OF BUYERS/ BUYERS/ CUSTOMERS CUSTOMERS


Assessing E-Health

2. 3. 4. 5.

301

Threat of substitute: an alternative means that could take market share from product/service offered by the firms in the industry Bargaining power of buyers: the strength of buyers or groups of buyers within the industry relative to the firms Bargaining power of suppliers: the strength of suppliers relative to the firms in the industry Rivalry of existing competition: relative position and market share of major competitors

The collective strength of these five forces determines the attractiveness of the industry and thus the potential for superior financial performance by influencing prices, costs, and the level of capital investment required (Porter, 1985). Once a thorough industry analysis has occurred, it is generally easier for a firm to determine which generic strategy makes most sense for it to pursue and enables the firm to exploit most of its core competencies in its existing environment. In this chapter we propose the application of Porter’s (1985) five forces model to the Web of e-health players shown in Figure 1. Healthcare delivery organizations and the industry that supports them are competing and cooperating for patients and resources. They compete as suppliers of services, technology, and other industrial resources. This competition is among the entities in the healthcare sector as well as with organizations in other industries. The Web of e-health serves as an instrument that mediates the relations among the five forces (Geisler, 1999). This in turn creates several opportunities for the organizations involved to improve and enhance their abilities to navigate the competitive environment of their industry.

Role of the Internet of the Competitive Forces Feeny (2001) presented a framework that highlights the strategic opportunities afforded to organizations by using the Internet. In particular, he highlights three e-opportunity domains. Table 2 details these domains and their respective components.


302


Table 2. The three e-opportunity domains and their components Domain e-operations

Components • • • • •

e-marketing

• • •

e-services

• • • • •

Automation of administrative processes Supply-chain reconfiguration Reengineering of primary infrastructure Intensive competitive procurement Increased parenting value Enhanced selling process Enhance customer usage experience Enhanced customer buying experience Understanding of customer needs Provision of customer service Knowledge of all relevant providers Negotiation of customer requirements Construction of customer options

E-Opportunities in Healthcare Given the three areas of e-opportunities discussed previously, Glaser (2002) identified several key e-opportunities for healthcare, detailed in Table 3.

The Goals of E-Health The preceding definition of e-health is broad enough to capture the dynamic environment of the Internet and at the same time acknowledge that e-health encompasses more than just “Internet and Medicine.” Thus, e-health strives to achieve many goals beyond the use of the Internet. These goals, taken together perhaps best characterize what e-health is all about (or what it should be about; JMIR, 2003). Some of these goals of e-health include the following:


Assessing E-Health

303

Table 3. The e-opportunities for healthcare organizations Domain

Components

e-operations

• •

Internet-based supply purchasing Prescription writing, formulary checking, and interaction checking using hand-held devices

e-marketing

•

Delivery of consumer health content and wellness management tools over the Internet Use of consumer health profiles to suggest disease management and wellness programs Patient-provider communication and transaction applications Web-based applications to support the clinical conversation between referring and consulting physicians Increasing the level of information content in the product Increasing the information intensity along the supply chain Increase in the dispersion of information

• e-services

• •

Crossing multiple domains

• • •

•

•

Efficiency: One of the promises of e-health is to increase efficiency in heathcare, thereby decreasing costs. One possible way of decreasing costs would be by avoiding duplicative or unnecessary diagnostic or therapeutic interventions, through enhanced communication possibilities between healthcare establishments and through patient involvement (Health Technology Center, 2000). The Internet will naturally serve as an enabler for achieving this goal in e-health. Quality of care: Increasing efficiency involves not only reducing costs, and thus is not an end in and of itself but rather should be considered in conjunction with improving quality one of the ultimate goals of e-health. A more educated consumer (as a result of the informational aspects of ehealth) would then communicate more effectively with their primary care provider which will, in turn, lead to better understanding and improved quality of care.


304


•

Evidence-based: E-health interventions should be evidence-based in the sense that their effectiveness and efficiency should not be assumed but proven by rigorous scientific evaluation and support from case histories. Web-accessible case repositories facilitate the timely accessibility of such evidence and thus help in achieving the necessary support of a diagnosis or treatment decision. The evidence-based medicine goal of e-health is currently one of the most active e-health research domains, yet much work still needs to be done in this area. Empowerment of consumers and patients: By making the knowledge bases of medicine and personal electronic records accessible to consumers over the Internet, e-health opens new avenues for patient-centered medicine, enables patient education and thus increases the likelihood of informed and more satisfactory patient choice (Umhoff & Winn, 1999). Education: The education of physicians through online sources (continuing medical education) and consumers (health education, tailored preventive information for consumers) makes it easier for physicians as well as consumers to keep up to date with the latest developments in the medical areas of their respective interests. This, in turn, is likely to have a positive impact on the quality of care vis-à-vis the use of the latest medical treatments and preventive protocols. Extending: Extending the scope of healthcare beyond its conventional boundaries, in both a geographical sense as well as in a conceptual sense, leads to enabling such techniques as telemedicine and virtual operating rooms, both of which are invaluable in providing healthcare services to places where it may otherwise be difficult or impossible to do. Ethics: E-health involves new forms of patient-physician interaction and poses new challenges and threats to ethical issues such as online professional practice, informed consent, privacy and security issues (Healthcare Advisory Board, 2002). However, this is not an intrinsic feature of ehealth but rather a feature of the Internet technology which is the foundation for all e-business initiatives, therefore, e-health along with egovernment, e-insurance, e-banking, e-finance, and e-retailing must all contend with these ethical issues. Given the nature of healthcare, these issues could be more magnified. Equity: To make healthcare more equitable is one of the aims of quality generally identified by the American Institute of Medicine (Institute of Medicine, 2001) and is one of the goals of e-health. However, at the same

•

•

•

•

•


Assessing E-Health

305

time there is a considerable threat that e-health, if improperly implemented and used, may deepen the gap between the haves and the have nots, hence the need for a robust framework to ensure the proper implementation of e-health initiatives. In particular, some of the key issues for equity revolve around broad access and familiarity with the technology. Today, a large number of patients and consumers already use the Internet to retrieve health-related information, to interact with health providers, and even to order pharmaceutical products (for example drugstore.com). For example, it has been noted that “the number of Medline searches performed by directly accessing the database at the National Library of Medicine increased from 7 million in 1996 to 120 million in 1997, when free public access was opened; the new searches are attributed primarily to non-physicians” (JMIR, 1999). Thus, the Internet will act as a catalyst for evidence based medicine in particular and e-health generally in two ways: First, it enables health professionals to access timely evidence. Second, it enables consumers to draw from the very same knowledge base, leading to increased pressure on health professionals to actually use the evidence (PWC, 2003).

Key Challenges of E-Health By 2005, 88.5 million adults will use the Internet to find health information, shop for health products, and communicate with affiliated payers and providers through online channels, according to Cyber Dialogue (PWC, 2003). Today, the e-health consumer demand includes the need for specific health services, such as obtaining information when faced with a newly established diagnosis. Many key challenges must be addressed to develop optimal partnerships between consumers and other groups of healthcare stakeholders. Some of these include the need for: • • • •

meaningful collaboration with healthcare recipients, efficient strategies and techniques to monitor patterns of Internet use among consumers, preparation for upcoming technological developments, balancing between connectivity and privacy,


306


•

better understanding of the balance between face-to-face and virtual interactions, and equitable access to technology and information across the globe.

•

When we consider the domain of e-health at the macro level, three important issues must be carefully considered (Cyber Dialogue, 2001); namely, Procurement, Connectivity, and Benefits. We briefly discuss each in turn. E-Procurement Health systems must begin to contemplate how their organizations will adapt and leverage Internet-based tools to manage their medical supply chains. Procurement in healthcare supplies must move toward an e-business platform for data interchange because of the ubiquity and cost effectiveness of the Internet, the primary e-business platform. The Internet decreases many of the restrictions placed by geographic and time barriers and also facilitates the use of incorporating artificial intelligence solutions, such as intelligent agents that can be deployed to hunt for best buys. Buyers and suppliers also must work together toward standardization, including the development of a universal product numbering system to facilitate e-procurement. As shown in Figure 3, e-procurement of medical supplies is estimated to grow to 15% of medical supply spending in the U.S. by 2003, according to a Deutsche Bankreport (PWC, 2003) E-Connectivity Healthcare has traditionally been locally delivered because a patient’s usual first port of call is their local primary care physician. To reorient such a tradition, connectivity companies, which on the other hand have a global rather than local focus, must be pragmatic and move in incremental steps when connecting healthcare organizations. Technology is the integral tool, but it will not achieve its full potential or live up to its promise unless healthcare organizations successfully deploy it and then keep track of their clinicians’ and administrative workers’ effective use of it. To do so, managers must design processes and metrics for productivity; otherwise, it is like expecting someone to drive a car when his or her experience is limited to a 10 speed bicycle. However,


Assessing E-Health

307

Figure 3. Estimated U. S. medical e-procurement market sales (Source: Dialog Cybercitizen Health Trend Report [Re. No. 49], by Odyssey Research, 2003. Available online from www.odysseyresearch.org)

healthcare organizations will find that achieving Web-enabled connectivity offers the most opportunity initially, and that other functions such as disease management, outcomes management, and demand management can be Webenabled later. Health plans and hospitals are beginning to migrate to the Internet for claimsrelated transactions as the first step of a broader Internet strategy. This is illustrated in Figure 4. Because many organizations continue to use EDI for claims submissions, transactions surrounding claims (e.g., eligibility, referrals) will thus logically be the first to be targeted for e-health connectivity. Those health plans that are adopting Internet connectivity for these functions view them as the foundation on which to build other Internet-enabled partnerships with patients and providers. The number of healthcare transactions is outpacing the growth of health spending, creating a critical need to automate the handling of such transactions through the adoption of e-health. The number of claims submitted increased by about 7% during the past five years, according to the Health Data Directory.


308


Figure 4. Increase in the percentage of electronically filed claims (Source: Dialog Cybercitizen Health Trend Report [Re. No.. 49], by Odyssey Research, 2003. Available online from www.odysseyresearch.org)

90% 80% 70% 60% 50% 40%

Physician Claims

30%

All Health Claims

Hospital Claims

20% 10% 0%

1995

1996

1997

1998

1999

In contrast, healthcare spending has increased at between 5% and 6% during the same period (ibid). Many of the functions associated with claims submissions and payment are repetitive tasks that are more efficiently done by computers. The most expensive processes are not the claims submissions themselves, but the tasks surrounding the claims processing, such as eligibility checks and referrals. Coupled with increasing labor shortages, there is an urgent need for healthcare organizations to reengineer their processes. Health plans must understand physicians’ needs when designing Internet-based solutions and that first-mover advantage is not as important as a system that works. Many non-healthcare organizations have benefited from first-mover advantage in adopting e-commerce initiatives, which means they had access to the most capital, the best partners and could establish solid brand recognition. However, this is not necessarily an appropriate strategy for healthcare as healthcare is more pragmatic and a “show-me” industry where successful models replicate specialty by specialty.


Assessing E-Health

309

E–Benefits American industry has discovered the benefits of e-business. Like the conundrum of the chicken and the egg, many employers and health plans are awaiting the development and implementation of e-benefits and e-insurance products. Health plans do not want to deliver Web-based products if employers are not ready to use them. Employers cannot deliver e-benefits products until health plans develop them. However, starting with online benefits enrollment, this aspect of e-health is evolving in stages. A by-product of this evolution is the fusion of employee responsibility and empowerment; hence, employers will gradually cede more control for health benefits to employees themselves. One of the primary drivers of e-benefits is delivery of self-service capabilities in which employees can customize their own insurance plans and have ready access to them, just as they do with their brokerage accounts. By putting this information at their fingertips, employees may become more fiscally responsible about those benefits. Ultimately, they may want complete control over more and more aspects of their benefits. To empower employees toward that end, some employers will have to embrace a defined contribution model, also called self-directed or consumer-directed health plans. This builds on the findings of the report “Defined Contribution Healthcare,” which specifically discusses the various models of defined contribution health plans (ibid). The national research conducted by PricewaterhouseCoopers (PWC, 2003) indicated that few employers are willing to adopt self-directed, otherwise known as defined contribution, health plans today, but will more likely move incrementally toward Web-enabling benefits processing and hence shifting more responsibility to employees. As employers adopt more e-health initiatives and Web-enabled functions, they will move more responsibility for choices to employees. As that balance of responsibility tips towards the employees, employers will need to assess how ready their workforce is to accept increasing levels of responsibility (Geisler, 2000). If a significant portion of the healthcare insurance market moves to self-directed and Internet health benefit accounts, physicians may be most directly and adversely affected. Some contribution health plans have medical savings accounts as a centerpiece. In these accounts, employees pay out of a medical savings account for routine expenses up to $1,500 or $2,000. Physicians will need to contend with patients who are paying cash for their visits and who may shop around for the best value. Figure 5 shows the statistics of the probability of customers using the Internet when they have accessibility to PCs. Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

310


Figure 5. Internet usage among customers (Source: Dialog Cybercitizen Health Trend Report (Re. No.. 49), by Odyssey Research, 2003. Available online from www.odysseyresearch.org)

60% 50% 40% % of Households with PC

30%

Internet Access

20% 10% 0%

7/94 1/95 7/95 1/96 7/96 1/97 7/97 1/98 7/98 1/99 7/99 1/00 7/00

As the utility function of benefits moves toward the Internet, insurers must then develop products that serve new and existing customers, such as e-quote products that assist brokers and other intermediaries. Further, they also must follow the changing needs of consumers whose response to defined contribution health products is to date largely untested. In summary then, the effective addressing of the key e-health issues of procurement, connectivity, and benefits requires standardization, a pivotal prerequisite for the implementation of successful e-health initiatives. Without such standardization the exchange of documents and other procurement information, connectivity, and e-commerce enabled benefits clearly become more problematic. Unfortunately, standardization is woefully lacking in too many areas of healthcare, let alone e-health. Given the opportunities for ehealth to benefit various aspects of healthcare and the far reaching impact of any such e-health initiative, it becomes imperative then to have frameworks and models that not only bring to the forefront the key e-health issues but also provide guidelines for how to effectively bring to bear successful e-health initiatives for all healthcare organizations.


Assessing E-Health

311

A Framework for Assessing E-Health Potential We propose the framework shown in Figure 6 to assess the e-health potential and preparedness of countries. Healthcare polices are generally developed to a large extent at a macro, country level and thus we believe it is also necessary when looking at e-health to first take a macro perspective and analyze the level of the country in terms of embracing e-health. The framework highlights the key elements that are required for successful e-health initiatives and therefore provides a tool that allows analysis beyond the quantifiable data into a systematic synthesis of the major impacts and prerequisites. The framework contains four main prerequisites, four main impacts, and the implications of these prerequisites and impacts to the goals of e-health. By examining both the prerequisites and the impacts, we can assess the potential of a country and its preparedness for e-health as well as its ability to maximize the goals of e-health. We discuss the prerequisites next.

Prerequisites for E-Health As can be seen in Figure 6, the four critical prerequisites for any successful ehealth initiative include information communication technology (ICT) architecture/infrastructure, standardized policies, protocols and procedures, user access and accessibility policies and infrastructure, and, finally, government regulation and control. These will now be briefly discussed in turn. ICT Architecture/Infrastructure The generic architecture for most e-health initiatives was depicted in Figure 1. To support such a client-server architecture special attention must be paid to the ICT infrastructure. The ICT infrastructure includes phone lines, fiber trunks, and submarine cables, T1, T3 and OC, ISDN, DSL and other high-speed services used by businesses as well as satellites, earth stations, and teleports. A sound technical infrastruc-


312


Figure 6. A framework for assessing a country’s/region’s e-health potential

ture is an essential ingredient to the undertaking of e-health initiatives by any nation. Such infrastructures should also include telecommunications, electricity, access to computers, number of Internet hosts, number of Internet Service Providers (ISPs), and available bandwidth and broadband access. To offer a good multimedia content and thus provide a rich e-health experience, one would require a high bandwidth. ICT considerations are undoubtedly one of the most fundamental infrastructure requirements. Networks are now a critical component of the business strategies for organizations to compete globally. Having a fast microprocessor-based computer at home has no meaning unless you have high bandwidth-based communication infrastructure available to connect computers with the ISP. With the explosion of the Internet and the advent of e-commerce, global networks need to be accessible, reliable, and fast to participate effectively in the global business environment. Telecommunications is a vital infrastructure for Internet access and hence for e-commerce. One of the pioneering countries in establishing a complete and robust e-health infrastructure is Singapore which is in the process Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Assessing E-Health

313

of wiring every home, office and factory up to a broadband cable network which will cover 98% of Singaporean homes and offices (ibid). Standardization Policies, Protocols and Procedures E-health by definition spans many parties and geographic dimensions. To enable such a far-reaching coverage, significant amounts of document exchange and information flows must be accommodated. Standardization is the key for this. Once a country decides to undertake e-health initiatives, standardization polices, protocols and procedures must be developed at the outset to ensure the full realization of the goals of e-health. Fortunately, the main infrastructure of e-health is the Internet that imposes the most widely and universally accepted standard protocols such as TCP/IP and hypertext transfer protocol. It is the existence of these standard protocols that has led to the widespread adoption of the Internet for e-commerce applications. The transformation to e-health by any country cannot be successfully attained without the deliberate establishment of standardization policies, protocols, and procedures that play a significant role in the adoption of e-health and the reduction of many structural impediments (Samiee, 1998). User Access and Accessibility Policies and Infrastructure Access to e-commerce is defined by the World Trade Organization (WTO) as consisting of two critical components: a) access to Internet services and b) access to e-services (Panagariya, 2000); the former deals with the user infrastructure, while the latter pertains to specific commitments to electronically accessible services. The user infrastructure includes number of Internet hosts and number of Web sites, Web users as a percent of the population as well as ISP availability and costs for consumers, PC penetration level, and so forth. Integral to user infrastructure is the diffusion rate of PCs and Internet usage. The United States and the United Kingdom have experienced the greatest penetration of home computers (Samiee, 1998). For developing countries such as India and China there is, however, a very low PC penetration and teledensity. In such a setting it is a considerable challenge then to offer e-health, since a large part of the population is not able to afford to join the e-commerce bandwagon. Countries thus have to balance local call charges, rentals, subscription charges, and so forth, otherwise the majority of citizens will find these costs a disincenCopyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

314


tive. This is particularly significant for developing and emerging nations where access prices tend to be out of reach for most of the population. Upcoming new technologies hold the promise to increase the connectivity as well as affordability level, and developing countries will need to seriously consider these technologies. In addition to access to PCs and the Internet, computer literacy is important and users must be familiar not only with the use of computers and pertinent software products but also the benefits and potential uses of the Internet and World Wide Web (ibid). Governmental Regulation and Control The key challenges regarding e-health use include a) cost effectiveness (i.e., less costly than traditional healthcare delivery), b) functionality and ease of use (i.e., they should enable and facilitate many uses for physicians and other healthcare users by combining various types and forms of data as well as be easy to use), and c) they must be secure. One of the most significant legislative regulations in the United States is the Health Insurance Portability and Accountability Act (HIPAA; Protegrity, 2001). Given the nature of healthcare and the sensitivity of healthcare data and information, it is incumbent on governments not only to mandate regulations that will facilitate the exchange of healthcare documents between the various healthcare stakeholders but also to provide protection of privacy and the rights of patients. Some countries, such as China and Singapore, even control access to certain sites for moral, social, and political reasons while elsewhere transnational data flows are hindered by a plethora of regulations aimed at protecting domestic technology and related human resource markets (Goff, 1992; Gupta, 1992; Samiee, 1998). Irrespective of the type of healthcare system (i.e., whether 100% government driven, 100% private, or a combination thereof), it is clear that some governmental role is required to facilitate successful e-health initiatives. The significance of the preceding four prerequisites on e-health initiatives will be modified by the impacts of IT education, morbidity, cultural-social dimensions, and world economic standing, as elaborated upon in the following sections.


Assessing E-Health

315

Key Impact of E-Health Figure six highlights four key impacts of e-health, which we now discuss in turn. Impact of IT Education A sophisticated, well-educated population boosts competition and hastens innovation. According to Michael Porter, one of the key factors to a country’s strength in an industry is strong customer support (Porter, 1990). Thus, a strong domestic market leads to the growth of competition, which leads to innovation and the adoption of technology-enabled solutions to provide more effective and efficient services, such as e-health and telemedicine. As identified previously, the health consumer is the key driving force in pushing ehealth initiatives. We conjecture that a more IT-educated healthcare consumer would then provide stronger impetus for e-health adoption. Impact of Morbidity Rate There is a direct relationship between health education and awareness and the overall health standing of a country. Therefore, a more health conscious society, which tends to coincide with a society that has a lower morbidity rate, is more likely to embrace e-health initiatives. Furthermore, higher morbidity rates tend to indicate the existence of more basic health needs (WHO, 2003) and hence treatment is more urgent than the practice of preventative medicine and thus e-health could be considered an unrealistic luxury and in some instances such as when a significant percentage of a population is suffering from malnutrition related diseases is even likely to be irrelevant at least in the short term. Thus, we conjecture that the modifying impact of morbidity rate is to prioritize the level of spending on e-health versus other basic healthcare needs. Impact of Cultural-Social Dimensions Healthcare has been shaped by each nation’s own set of cultures, traditions, payment mechanisms, and patient expectations. While the adoption of e-health, to a great extent, dilutes this cultural impact, social and cultural dimensions will still be a moderating influence on any country’s e-health initiatives. Another


316


aspect of the cultural-social dimension relates to the presentation language of the content of the e-health repositories. The entire world does not speak English, so the e-health solutions have to be offered in many other languages. The e-health supporting content in Web servers and Web sites must be offered in local languages, supported by pictures and universal icons. This becomes a particularly important consideration when we look at the adoption and diffusion of evidence-based medicine, as it will mean that much of the available evidence and case study data will not be easily accessible globally due to language barriers. Therefore, for successful e-health initiatives it is important to consider cultural dimensions. For instance, an international e-commerce study by International Data Corporation (Wilson, 1999) indicated that Web surfing and buying habits differ substantially from country to country and this would then have a direct impact on their comfort to use e-commerce generally and e-health in particular, specially as e-health addresses a more fundamental need. Hence, the adoption of e-health is directly related to ones’ comfort with using the technology, and this in turn is greatly influenced by cultural dimensions. Also connected with cultural aspects is the relative entrepreneurial spirit of a country. For example, a study by Hofstede (1980) indicated that in a cultural context, Indians score high on “uncertainty avoidance” criteria when compared to their Western counterparts. As a result, Indians do not accept change very easily and are hostile towards innovation. This, then, would potentially pose a challenge to the starting up of e-health initiatives whose success depends on widespread adoption for their technological innovations. Thus, we conjecture that fear of risk and absence of an entrepreneurial mindset, as well as other cultural-social dimensions, can also impact the success of e-health initiatives in a given country. Impact of World Economic Standing Economies of the future will be built around the Internet. All governments are very aware of the importance and critical role that the Internet will play in a country’s economy. This makes it critical that appropriate funding levels and budgetary allocations become a key component of governmental fiscal policies so that such initiatives will form the bridge between a traditional healthcare present and a promising e-health future. Thus, this would determine the success of effective e-health implementations and, consequently, have the potential to enhance a country’s economy and future growth.


Assessing E-Health

317

The World Economic Forum’s global competitiveness ranking measures the relative global competitiveness of a country. This ranking takes into account factors such as physical infrastructure, bureaucracy, and corruption. Thus we conjecture that weak physical infrastructure combined with high levels of bureaucracy and corruption will lead to significant impediments to the establishment of successful e-health initiatives. In developing its e-health initiative, a good first step for any country is to assess its standing with respect to the four prerequisites and four impacts discussed previously in this chapter. In this way it will be possible for a country to evaluate its preparedness with respect to these parameters and consequently devise appropriate policies and strategies for an effective and successful e-health initiative. In the following section, we will attempt to provide a guideline that will facilitate such an evaluation.

E-Health Preparedness Grid By taking the four main prerequisites as well as the four major impacts, identified in our framework in Figure 6 (i.e., information communication technology infrastructure; standardization policies, protocols and procedures; user access and accessibility policies; and infrastructures, governmental regulations and role) as well as the impact of IT education, the impact of morbidity rate, the impact of world economic standing, and the impact of cultural-social dimensions, we developed a grid for assessing e-health preparedness (see Figure 7) in which we can plot various countries with respect to these key parameters. The grid consists of four quadrants that represent the possible states of preparedness with respect to the key parameters for e-health success. The low preparedness quadrant identifies situations that are low with respect to all four prerequisites for e-health potential. The medium preparedness quadrant identifies two symmetric situations; namely, a combination of high and low positioning with respect to the four prerequisites for e-health potential. Finally, the high preparedness quadrant identifies situations that are high with respect to all four prerequisites for e-health potential. This grid not only shows the possible positioning of a given country with respect to its e-health preparedness (i.e., low, medium, or high) but also the path it must take and, more specifically, the prerequisite factors it must focus on to migrate to the ideal state of preparedness; namely, being high with respect to all four prerequisites. The


318


grid also underscores the moderating role of the four impacts irrespective of the relative positioning on the state of preparedness of a given country.

Discussion From the e-health preparedness grid in Figure 7 we can see several implications. For countries that are low with respect to all four of the e-health prerequisites, much preparatory work is required to be e-health ready and thereby fully realize the goals of e-health (i.e., efficiency, evidence-based and preventive medicine, educated stakeholders, ethical awareness, enhanced quality care, empowered patients, extended reach and equity). For countries that map to the medium preparedness quadrant, more emphasis is needed on upgrading the deficiencies that cause them to score low on some prerequisites while they continue to maintain a high status on other prerequisites so that the full benefits of their e-health initiatives can be realized. Countries that are noted Figure 7. E-health preparedness grid

User Access and Accessibility Policies

Medium e-health preparedness

High e-health preparedness

Standardization Policies, Protocols, Procedures

Governmental Regulations and Role

Low e-health preparedness

Medium e-health preparedness ICT Architecture/ Infrastructure

Impacts of: IT Education World Economic Standing

Morbidity Rate Culturall/social dimensions


Assessing E-Health

319

for being pioneers and leaders in the area of e-commerce in general as well as e-health in particular would be expected to map on the high quadrant. The challenge for these countries would be to maintain their high status with respect to all the prerequisites. Furthermore, organizations within these countries can also begin to structure their respective e-health initiatives (i.e., micro-level issues) based on the key macro-level issues that both the integrative framework and e-health preparedness grid identify. It is important to note that irrespective of their positioning on the preparedness grid (i.e., low, medium, or high), all countries must take into account the moderating effect of the four impacts on their e-health initiatives. If countries ignore the moderating effect of these four impacts, it will not be possible to fully realize a successful e-health initiative (and thus would not be able to fully realize the benefits of e-health), even if a country maps into the high-preparedness quadrant. Finally, we conjecture that the type of health system vis-à-vis the public–private continuum would not significantly impact the positioning of a country into a given quadrant. The successful experience of Singapore, albeit an isolated case, somewhat lends credence to our grid, as it would mostly fit the profile of a country in the high-preparedness quadrant and thus would be expected to successfully undertake an e-health initiative (PWC, 2003). Clearly, additional empirical validation of our e-health preparedness framework is required which we leave to future studies.

Conclusions E-commerce, as noted by the U.N. Secretary General’s address, is an important aspect of business in today’s 21st century. No longer is it a luxury for nations; rather it is a strategic necessity in order for countries to achieve economic and business prosperity as well as social viability. One of the major areas within e-commerce that has yet to reach its full potential is e-health. This is because healthcare generally has been slow in adopting information technologies. Furthermore, there is a shortage of robust frameworks that may be used as guidelines for assessing countries’ e-health preparedness and identifying the key areas and deficiencies that need to be addressed in order for successful eheath initiatives to ensue. In addition, e-health is more than a technological initiative; it also requires a major paradigm shift in healthcare delivery, practice, and thinking. We have attempted to address this gap by developing a frame-


320


work that identifies the major factors involved in assessing the e-health preparedness of countries and thereby, facilitating countries to focus their efforts on the relevant issues that must be addressed in order that successful ehealth initiatives follow (i.e., the goals of e-health are realized). An outcome from our analysis indicates that the relative healthcare system (i.e., whether government driven, public, or two tier) would appear to have less significance in establishing successful e-health initiatives. The first step in the development of any viable e-health strategy is to make an assessment of the current state of e-health preparedness and then how to either move to a state of higher preparedness (i.e., the high quadrant) or focus on maintaining a current high quadrant status–both of these are identified through the use of our framework and thus its value. Finally, we note that with respect to our framework other parameters also exist and could also be considered important, perhaps even as important as the ones we used. However, we believe that the framework will still function the same way (i.e., provide a useful tool for any country trying to determine and develop a successful e-health initiative), irrespective of the number of parameters; in this regard the preference should be simplicity over complexity.

References Al-Qirim, N. (2004). Telemedicine: Building knowledge-based tele-health capability in New Zealand. In N. Wickramasinghe, J. Gupta & S. Sharma (Eds.), Creating knowledge-based healthcare organizations. Hershey, PA: Idea Group Publishing (forthcoming). Alvarez, R. C. (2002). The promise of e-Health: A Canadian perspective. eHealth International, 1(1), 4. Cyber Dialogue. (2000, May 23). Online health information seekers growing twice as fast as online population. Cyber Dialogue. Cyber Dialogue. (2001). The higher the connection speed, the higher the value: Broadband use as an indication of value amongst e-health users. Cyber. Eysenbach, G. (2001). Journal of Medical Internet Research, 3(2), e20. Feeny, D. (2001, Winter). Making business sense of the e-opportunity. MIT Sloan Management Review, 42(2), 42.


Assessing E-Health

321

Gargeya, V. & Sorrell. D. (2004). Moving toward an e-hospital. In N. Wickramasinghe, J. Gupta & S. Sharma (Eds.), Creating knowledgebased healthcare organizations (pp. 50-64). Hershey, PA: Idea Group Publishing. Geisler, E. (1999). Mapping the knowledge base of management of medical technology. International Journal of Healthcare Technology and Management, 1(1), 3-10 Geisler, E. (2000). The metrics of science and technology. Westport, CT: Greenwood. Geisler, E. (2001). Organizing for e-Business: The implementation of management principles in electronic commerce. Journal of e-Commerce and Psychology. Glaser, J. (2002). The strategic application of information technology in health care organizations (2nd ed.). San Francisco: Jossey-Bass. Goff, L. (1992). Patchwork of laws slows EC data flow. Computerworld, 26(15), 80. Gupta, U. (1992). Global networks: Promises and challenges. Information Systems Management 9(4), 28-32. Health Technology Center. (2000). A survey conducted for the Health Technology Center (HealthTech) by Harris Interactive in cooperation with Pricewaterhouse Coopers and the Institute for the Future (IFTF). Retrieved from http://www.ncddr.org/cgi-bin/good-bye.cgi?url=http:/ /www.healthtechcenter.org Healthcare Advisory Board. (2002, January). Use of hospital Web sites to engender community loyalty. Hofer, C. (1975). Toward a contingency theory of business strategy. Academy of Management, 18(4), 784-810. Hofstede, G. (1980). Culture’s consequences, international work related values. Beverly Hills, CA: Sage. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: Committee on Quality of Health Care in America Institute of Medicine, National Academy Press. JMIR. Retrieved from http://www.jmir.org Odyssey Research. Dialogue cybercitizen health trend report (Yr. 2001, No. 49). Retrieved from www.odysseyresearch.org


322


Panagariya, A. (2000). E-commerce, WTO and developing countries. The World Economy, 23(8), 959-978. Porter, M. (1990). The competitive advantage of nations. New York: Free Press. Porter, M., & Teisberg, E. (2004). Redefining competition in healthcare. Harvard Business Review, pp. 65-76. Porter, M. (1980). Competitive strategy. New York: Free Press. Porter, M. (1985). Competitive advantage. New York: Free Press. Porter, M. (1990). The competitive advantage of nations. New York: Free Press. Protegrity. (2001, May). Health Insurance Portability and Accountability Act (HIPPA) privacy compliance executive summary. Author. PWC. (2003). PricewaterhouseCoopers healthcare practice. Retrieved from www.pwchealth.com Samiee, S. (1998). The Internet and international marketing: Is there a fit? Journal of Interactive Marketing, 12(4), 5-20 Sharma, S. & Wickramasinghe, N. (2004). e-Health with knowledge management: Areas for tomorrow. In N. Wickramasinghe, J. Gupta & S. Sharma (Eds.), Creating knowledge-based healthcare organizations (pp. 110-124). Hershey, PA: Idea Group Publishing. Sieving PC Factors Driving the Increase in Medical Information on the Web: One American Perspective. J. Med. Internet Res., 1. Retrieved from www.jmir.org/1999/1/e3 Umhoff, B., & Winn, J. (1999, May 1). The healthcare profit pool: Who stands to gain and lose in the digital economy. Health Forum Journal. UNCTAD. (2002). United Nations Conference on Trade and Development. Retrieved from http://r0.unctad.org/ecommerce/ecommerce_en Wickramasinghe, N. (2000). IS/IT as a tool to achieve goal alignment: A theoretical framework. International Journal Healthcare Technology Management, 2(1-4), 163-180. Wickramasinghe, N., et al. (2004). A framework for assessing e-health preparedness. International Journal of E-Health. Wilson, T. (1999). Not a global village after all? Consumer behavior varies widely by country. Internetweek, 792, 13.


Assessing E-Health

323

World Health Organization. (2003). Retrieved from http://www.emro.who.int/ ehealth/

Endnote 1

An earlier version of this chapter is to appear in a volume of the Internation J. e-Health, 2005.


324 Ammenwerth et al.

Chapter XVIII

The Evaluation Roadmap Elske Ammenwerth, University for Health Sciences, Medical Informatics and Technology (UMIT), Austria Jytte Brender, University of Aalborg, Denmark Pirkko Nykänen, Tampere University Hans-Ulrich Prokosch, University of Erlangen, Germany Michael Rigby, Keele University, UK Jan Talmon, Maastricht University, The Netherlands

Declaration of Innsbruck (Extracted from Ammenwerth. E., Brender, J., Nykänen, P., Prokosch, H., Rigby, M., & Talmon, J. [2004]). Visions and strategies to improve evaluation of health information systems Reflections and lessons based on the HIS-EVAL workshop in Innsbruck. International Journal of Medical Informatics, 73, 479-491.) Copyright 2004, Elsevier Ireland Ltd. Used with permission. Results from the European Science Foundation sponsored Workshop on Systematic Evaluation of Health information Systems (HIS-EVAL), April 46th, 2003. Participants: Jos Aarts, Elske Ammenwerth, Andrea Berghold, Marie-Catherine Beuscart-Zéphir, Jytte Brender, Thomas Bürkle, Martina Deibl, Martin Denz, Nina Eminovic, Rolf Holle, Mathew Jones, Joe Liu, Christian Nøhr, Pirkko Nykänen, Ulrich Prokosch, Michael Rigby, Cornelia Ruland, Heike Sander, Nikki Shaw, Arjen Stoop, Jan Talmon, Vivian Vimarlund, Jeremy Wyatt. Initiator and contact point: Elske Ammenwerth, MIT-University for Health Sciences, Medical Informatics and Technology, [email protected] Copyright 2004, Elsevier Ireland Ltd

The Evaluation Roadmap

325

Declaration Health information systems are intended to improve the functioning of health professionals and organisations in managing health and delivering healthcare. Given the significance of this type of intervention, and the intended beneficial effect on patients and professionals, it is morally imperative to ensure that the optimum results are achieved, and any unanticipated outcomes identified. The necessary process is evaluation, and this should be considered an essential adjunct to design and implementation of information systems.

Definitions 1.

2.

A system is a set of components (e.g., actors, artifacts), together with their attributes and relationships which, as a whole, is needed to accomplish an objective. A health information system (HIS) comprises actors (e.g., healthcare providers) and artifacts (e.g., the information and communication technology (ICT) as well as the implemented algorithms and procedures) that together process health-related information in a healthcare organization. It operates in an organizational environment made up of people (e.g., system developers, politicians, managers, patients) and procedures, which influence its development and operation. Evaluation is the act of measuring or exploring properties of an HIS (in planning, in development, in implementation, or in operation), the result of which informs a decision to be made concerning that system in a specific context. Evaluation of HISs has to deal with the actors, the artifacts, and their interaction to best support the decisions to be made.


Observations 1.

2.

3.

4.

Evaluation generates information to improve knowledge and to generate insight. By doing this, evaluation of HISs will ensure effective current health information systems, and contribute to better future ones. Since practicing medicine is an information intensive activity, a better health information system may also lead to an improved quality of care. This also implies that evaluation of ICT in healthcare has a value only when there is a purpose, that is, there has to be a question to be answered (e.g., improvement of knowledge and generation of insight from a scientific perspective, or making informed decisions about design, procurement, development or routine operation of an HIS). Evaluation supports reflective practice. Every successful organisation and conscientious practitioner evaluates the outcome of their decisions to see whether the intended goals are obtained. Evaluation of HISs supports the continuous monitoring, review, and adjustment of their planning, development, implementation, and/or operation. Evaluations also support reflective practice in health informatics in general, enabling the emergence of an evidence-based health informatics profession. Evaluation is a challenging endeavour. Many actors in a health organisation are affected by ICT. Hence, various viewpoints and aspects can be considered in an evaluation. In addition, the organisational, economic, and legal environment in which the HIS has to operate is in a constant change. To complicate issues further, technological developments enable functionalities that could not have been foreseen when the information system was planned. Evaluation of HISs therefore has the challenging task of selecting an adequate methodology to derive valid and timely answers to given questions despite the continuously changing conditions. Evaluation is not free. Proper evaluations require skills and resources, which need to be planned, procured, and applied; in return the results of evaluation should be anticipated to have added value. This added value need not to be of monetary nature but can be of a functional, psychological, or social type as well as in terms of the value of the knowledge gained.


327

Recommendations 1.

2.

3.

4.

5.

6.

Evaluation should be seen as an ethical imperative. Information systems and their applications are complex, commit scarce resources, directly affect patients’ records and the delivery of their care, and apply developing solutions to changing needs. From an ethical perspective, evaluation of HISs has the same role in medical informatics as evidence and audit has in clinical care practice. Furthermore, and in a similar way, it should be seen as desirable to share findings as learning points and knowledge development for the common good across the health informatics and healthcare communities. Evaluation should be sufficiently funded. The funding authorities of health information systems (e.g., hospital management) should require– and fund–explicit and continuous evaluation activities during the planning, development, implementation, and operation of the HIS. Evaluators should be free from pressure. Planning and execution of an evaluation should be based on professional expertise and be free from any political, managerial, or other pressure with regard to the conclusions of the evaluation. The main concern of the evaluator should be to perform an independent, objective, and proficient study that provides the answer to the questions asked. Evaluation studies should be grounded on scientific theory and rigorous approaches. Only rigorous scientific grounding will increase their credibility and ease the interpretation of their results. On the one hand, they need to respect the need to comply with the practical information needs and constraints of the actual situation; on the other hand, they should objectively provide evidence for the derived conclusions. Evaluation methods should be selected with an open mind. The choice of methods should not be restricted by a single research paradigm, but take into consideration the variety of information needs, available approaches and methods from different professional and research domains. Reports on methodological and methodical studies1 should be encouraged. Scientific journals and conferences should promote further development of evaluation methodologies. Flexible and transdisciplinary approaches are needed that allow for the complexity


of the evaluation to be managed whilst also accommodating changes in the environment during the evaluation study. 7. Guidelines for good evaluation practice should be made available. To strengthen future evaluation studies recommendations for best practice should be prepared through a consensus-making approach based on existing experience and sources of knowledge. The recommendations should be widely published, not only in journals for the medical informatics community, but also in journals directed towards other stakeholders in HISs and healthcare delivery. 8. Terms, concepts, and guidelines for reporting on results of ICT assessment studies should be developed. These should include a set of well-defined common evaluation terms and concepts; they should be agreed and widely published, then adopted by scientific journals and beyond, as presentation standards, and as the criteria for acceptance for publication. 9. Evaluation should be promoted by centres of excellence. Such centres should operate on a not-for-profit basis, not having other commercial interests in health informatics. They can provide consultancy for healthcare organisations and promote both theory and good practice of evaluation. 10. Evaluation networks should be established. These will support the exchange of experience. They should include participants from different professional and theoretical backgrounds as to encourage the transdisciplinary synergy of evaluation approaches and theories from different domains. 11. An open access repository about evaluation studies should be established. This repository should contain information on planned, active, and finalized (and also terminated) evaluation studies. Whether or not the outcomes of such studies are (to be) published through the traditional channels, it is important for evaluators to have access to templates or paradigmatic approaches with contact information as well as lessons learned on methodological and practical issues. 12. Appreciation of methods of evaluation should be part of health informatics curricula. Each medical informatician should have knowledge of methodologies and methods necessary to accomplish evaluation of HISs. A firm theoretical foundation is needed. Health Informatics curricula can provide such a foundation, preferably with practical


329

exercises included. It should be noted that evaluation is such a complex endeavour that only extensive experience in the real world will make a healthcare or health informatics professional a professionally qualified expert in evaluation.

Endnote 1

Methodology means the “knowledge (logos) of methods,” that is, how to deal with methods. Methodical means that stringency is applied throughout, which is not necessarily implied by the methodology or methods applied.

330 About the Authors

About the Authors Ton A. M. Spil (1964) is teaching in the area of business information systems at the University of Twente, The Netherlands. In 1988 he finished his master’s degree in computer science and started his own company consulting big firms on strategic information systems planning. In 1996 he finished his PhD thesis on the effectiveness of these plans, and after that he specialized in the application area of healthcare and professional organizations. In 2000 he was project manager on a big e-health research project on electronic prescription systems for general practitioners. He edited a book titled Strategies for Healthcare Information Systems published by Idea Group Publishing in 2001 and chaired the healthcare track on several major conferences on information systems. He sits in the organizing committee of the European Conference on Information Systems and in the program committee of the American Conference of IS. The main subject of research in 2003 is the governance and interorganizational aspects of information strategy and user satisfaction of IT in healthcare (USE IT). Roel W. Schuring’s entire academic career is focused around the effective organization and change of primary processes. He started in Geneva for the International Labour Office (1987), where he studied the effects of assembly automation on the occupational structure of assembly work. Later, he broadened his scope and included other dimensions of optimal organization of assembly work. During the 1990s he mainly worked on the effective organization and introduction of programs for continuous improvement in organizations. Since 1998 he has worked for the healthcare management department. The two areas of interest of his group are the effective


About the Authors 331

organization of healthcare processes and the effective introduction of new technologies in healthcare processes. These technologies include both information technologies and medical technologies. Dr. Schuring works as a part-time consultant for major industrial companies in the field of operations strategy. Since October 1, 2004, he has worked as a manager in a hospital in The Netherlands. * * * Elske Ammenwerth is assistant professor and head of the research group assessment of health information systems at the University for Health Sciences, Medical Informatics and Technology (UMIT) in Innsbruck, Austria. Between 1997 and 2001, she worked as a research assistant at the Institute for Medical Biometry and Informatics at the University of Heidelberg, Germany. Her research comprises methodological and practical issues in the evaluation of various components of hospital information systems. She is head of the working group Assessment of Health Information Systems of the European Federation for Medical Informatics (EFMI). Thomas Bürkle is deputy head of medical informatics at the department of medical informatics and biomathematics of Muenster University, Germany. He is head of clinical information systems and member of the IT directorate of the University Hospital. Thomas Bürkle graduated in 1988 in medicine at Tuebingen University and received his PhD in 1989. After several years in general surgery and orthopaedics, he earned an additional informatics degree and started his medical informatics career at Giessen University. There he received the venia legendi in 2001. Thomas Bürkle is member of IMIA nursing informatics SIG and secretary of EFMI WG 5. He served in various working groups of the German Medical Informatics Association. William R. Chismar is the acting associate dean for the College of Business Administration at the University of Hawaii, USA, where he is also a professor of information technology management and an adjunct professor in the John A. Burns School of Medicine. He served six years as the chair of the Department of Information Technology Management. His research focuses on medical informatics and the organizational impacts of information technology, with current projects on the economic evaluation of hospital information systems, Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


the adoption of electronic medical record systems, and the use of the Internet in pediatrics. He has taught courses in the management of information systems, medical informatics, electronic commerce, database management, and telecommunications. He served on a research review committee for the National Institutes of Health’s National Library of Medicine. He founded and chairs the Information Technology in Healthcare track at the Hawaii International Conference on System Sciences. Jean-Paul Fortin, for more than 15 years has leads a research/evaluation team on health information and communication technologies adoption and diffusion. Medical specialist in community health, full professor at the Department of Social and Preventive Medicine at Laval University, Canada, senior medical advisor at the Quebec National Institute of Public Health and the Regional Office of Public Health, researcher/evaluator at the Quebec University Hospital Research Center, Dr. Fortin is also interim medical coordinator of the Quebec Telehealth Network. He has held direction functions at the Ministry of Social Affairs and at the Quebec Regional Board of Health and Social Services. He was senior technical advisor at the European Regional Bureau of the World Health Organization, special advisor at the Quebec Inquiry Commission on Health and Social Services, and expert-member at the Federal, Provincial and Territorial Advisory Committee on Population Health. Elie Geisler is professor and associate dean for research at the Stuart Graduate School of Business, Illinois Institute of Technology, USA. He holds a doctorate from Northwestern University. Dr. Geisler is the author of over 90 papers in the areas of technology and innovation management, the evaluation of R&D, science and technology, and the management of healthcare and medical technology. He is the author of eight books, including The Metrics of Science and Technology (2000), and Creating Value with Science and Technology (2001). He consulted for major corporations and for many U.S. federal departments, such as Defense, Agriculture, Commerce, EPA, Energy, and NASA. His forthcoming book is The Structure and Progress of Knowledge. Stefan Gräber, after studying mathematics, physics, informatics, and medicine, and doctorate in medicine Stefan Gräber was the head of a university computer centre for several years. Since 1992 he has been working at the



department of medical biometry, epidemiology and medical informatics of the University of Saarland, Germany. He is the head of the department for clinical information services. His research activities concern hospital information systems, especially the information management, the evaluation of health information systems, and computer-based training in medicine. Brian N. Hilton is a research fellow at the Claremont Information and Technology Institute, School of Information Science, Claremont Graduate University, USA. His research interests include geographic information systems, spatial decision support systems, open source software, and information system development. He received a PhD and a MS in management information systems from Claremont Graduate University and a BA in economics from the Richard Stockton College of New Jersey. Thomas A. Horan is associate professor in the School of Information Science and Director of the Claremont Information and Technology Institute (CITI) at the Claremont Graduate University (CGU), USA. Dr. Horan’s research addresses the planning and assessment of information technology systems, focusing on three substantive areas: healthcare informatics, community informatics, and digital infrastructures. This research has been reported in a wide variety of journals, including the Information Systems Frontiers, Communications of the ACM; Knowledge, Technology and Policy, and Journal of Urban Technology. His most recent book, Digital Infrastructures (edited with Rae Zimmerman) was published by Routledge Press in Fall 2004. Dr. Horan has both his master’s and doctorate degrees from CGU. Carola Iller studied educational and political science at the universities of Heidelberg and Bremen, Germany. From 1992 to 2000, she was research assistant at the University of Bremen, where she got her PhD in 1999. Since 2000, she has been assistant professor in the institute for educational science at the University of Heidelberg. Bonnie Kaplan is on the of the faculty of Yale University, USA. She is at the Yale Center for Medical Informatics and a faculty affiliate of the Information Society Project at the Yale Law School. She also is a senior scientist at Boston University’s Medical Information Systems Unit, and President of Kaplan



Associates. She specializes in evaluation, change management, benefits realization, and clinician and patient use of information technology. Author of numerous publications, she consults internationally for medical centers, government agencies, healthcare organizations, firms in the healthcare industry, and university advisory boards. She chairs the International Medical Informatics Association Working Group 13 on Social and Organizational Issues. In 2001, she co-founded and now chairs the Yale University Interdisciplinary Bioethics Project Research Working Group on Technology and Ethics and the Whitney Humanities Center Working Group on Science, Technology, and Utopian Visions. She holds a social science PhD from the University of Chicago and an interdisciplinary BA degree in mathematics and computer science from Cornell University. She is a Fellow of the American College of Medical Informatics and the recipient of the AMIA President’s Award. She taught a variety of information systems courses in business administration and hospital administration programs at several universities. She currently teaches in the Medical Informatics postdoctoral program and in undergraduates program at Yale University. Lise Lamothe is an associate professor at the health administration department of Montreal University, Canada. She received her PhD from McGill University in Montreal. Her research focuses on the governance and transformation of healthcare organizations. More specifically her research focuses on the professional dynamics prevailing to the transformation of healthcare organizations, the formation of integrated delivery systems, including the influence of IT on their formation. She is also interested on the policy implications of her research findings. Liette Lapointe is currently an assistant professor at the Faculty of Management at McGill University. She holds an MSc in healthcare administration from the Faculty of Medicine at Université de Montréal, Canada, and a PhD in administration (information systems) from HEC Montréal. Dr. Lapointe’s research interests concern resistance to information technology and the implementation of information systems in the healthcare industry. Mirjan (MJ) van der Meijden. The main field of interest of Mirjan van der Meijden is effective support of healthcare work, including communication, with ICT solutions for electronic record keeping. She is currently working at the ICT



department of a general hospital in The Netherlands. From 1996 till 2002 she worked at the department of medical informatics at the Maastricht University, where she finished her PhD thesis in 2002. The subject of her thesis was the development, implementation and evaluation of an electronic patient record. Margreet B. Michel-Verkerke. After her graduation as a physician, Margreet Michel-Verkerke (1957) spent 13 years on teaching nurses and doctors assistants. During this period she developed an interest in the application of information and communication technology in education. She started to study computer science at the Open University. In 2000 she continued her studies at the University of Twente, which resulted in her graduation as an MSc in business information technology in November 2003. From 1998 she worked as a project manager in the area of e-learning in vocational education. She decided to switch to the application of ICT in healthcare in 2001, when she became a research assistant at the University of Twente, The Netherlands. Her main fields of interest are the improvement of care processes by the enhancement of cooperation and coordination by use of ICT, and the electronic patient record. Pirkko Nykänen is currently employed as acting professor in information systems in Tampere University Department of Computer Sciences, Finland. She has a long history in R&D in health informatics domain: From 1975 to 2002 she worked as senior researcher at the Technical research Centre of Finland Medical Engineering Laboratory. Since then she has been working as a development manager in the National Research and Development Centre for Welfare and Health. She has been working as visiting researcher in Lille University, France, and in the Pennsylvania State University, USA. She has been involved in many EU R&D projects, has worked as expert for IST ehealth projects, and as reviewer for scientific journals and conferences. She has 120 publications in the medical and health informatics domain. Reetta Raitoharju is a researcher in Turku School of Economics and Business Administration, Turku. She received her master’s degree in 2003 and made her thesis work about the connections between the use of information systems and job satisfaction in Finnish social and healthcare. After graduating, she started her doctoral studies in Turku Centre for Computer Science. Her field of interest in research is the use of healthcare information systems from the



perspective of personnel. Currently she is working on a research project about IT-related stress in social and healthcare sectors in Finland. Jonathan L. Schaffer is managing director of e-Cleveland Clinic at The Cleveland Clinic, USA, where he is also a staff member in the Adult Reconstruction Section and the Orthopaedic Research Center in the Department of Orthopaedic Surgery. Dr. Schaffer graduated from Case Western Reserve University School of Medicine, completed his orthopaedic surgery residency at the University of Pennsylvania, and then completed a fellowship in adult reconstruction at Brigham and Women’s Hospital and a research fellowship at Boston Children’s Hospital, both at Harvard Medical School. Dr. Schaffer also holds a MBA from the Kellogg School of Management at Northwestern University. Reima Suomi has been a professor of information systems science at Turku School of Economics and Business Administration, Finland, since 1994. He is a docent for the universities of Turku and Oulu, Finland. Years 1992-93 he spent as a Vollamtlicher Dozent in the University of St. Gallen, Switzerland, where he led a research project on business process reengineering. Currently he concentrates on topics around management of telecommunications, including issues such as management of networks, electronic and mobile commerce, virtual organizations, telework, and competitive advantage through telecommunication-based information systems. Different governance structures applied to the management of IS and are enabled by IS also belong to his research agenda, as well as application of information systems in healthcare. Reima Suomi has over 300 publications, and has published in journals such as Information & Management, Information Services & Use, Technology Analysis & Strategic Management, The Journal of Strategic Information Systems, and Behaviour & Information Technology. For the academic year 2001-2002 he was a senior researcher varttunut tutkija for the academy of Finland. With Paul Jackson, he has published the book Virtual Organization and Workplace Development, with Routhledge, London. Bengisu Tulu is currently a doctoral student in management information systems at the School of Information Science at Claremont Graduate University, USA, where she also presently works as a research associate in the



Network Convergence Laboratory. Her research interests include voice/video over IP, security, and medical informatics. She is currently working on speech and video quality assessment for telemedicine applications and digital signatures in the healthcare domain. Ms. Tulu received her master’s degree in management information systems from Claremont Graduate University. Earlier she received a master’s degree in information systems and a bachelors degree in mathematics from Middle East Technical University, Turkey. Pekka Turunen holds the research director position of the Social and Health Information Technology Research Unit (Shiftec) at the University of Kuopio in Finland. He received his PhD from the Turku School of Economics and Business Administration in the Evaluation of Healthcare Information Systems, which has been his primary research interest in addition of management of ICT. He has been an honorary visiting research fellow in several research groups around Europe, including MIG in the University of Manchester and Department of Medical Informatics of University Maastricht. Sonja Wiley-Patton is an assistant professor of information systems and decision sciences at Louisiana State University in Baton Rouge, Louisiana, USA. Dr. Wiley-Patton is a KPMG Doctoral Fellow and received her PhD in communications and information sciences at the University of Hawaii in Honolulu, Hawaii. Her research interests include the adoption and diffusion of integrated clinical information systems, medical, clinical and bio-informatics, information technology acceptance, and the social impact of information technology diffusion among underrepresented populations and within nonindustrialized countries. Dr. Wiley-Patton has taught courses in the management of information systems, the management of information resources, and the organizational and social impacts of IT adoption. Dr. Wiley-Patton received a 2004 National Science Foundation Information Technology Resource (ITR) grant to study information technology adoption behavior of healthcare professionals. Nilmini Wickramasinghe is an assistant professor in the Computer and Information Science Department at the James J. Nance College of Business Administration at Cleveland State University, Ohio, USA. She teaches IS at the undergraduate and graduate levels in the areas of knowledge management as well as e-commerce and m-commerce, IT for competitive advantage organi-



zational impacts of technology and health care issues. In addition, Dr. Wickramasinghe teaches and presents regularly in many universities in Europe and Australia. She is currently carrying out research and is published in the areas of management of technology, in the field of health care as well as focusing on IS issues especially as they relate to knowledge work and e-business. Dr. Wickramasinghe is honored to be able to represent the US for the Healthcare Technology Management (HCTM) Association (URLhttp://www.hctm.net/ Conferences/2003/Conference_2003.html).


Index 339

Index

Symbols A accessibility 160 actual 156 actual usage 34 adaptability 160 adoption decision 101 through coexistence 139 through replacement 139 anxiety 95 attention deficit/hyperactivity disorder 74 attitude 10, 110, 155 axioms 134

B bargaining 69 benefits 136 bivariate level 42 breast cancer 111 bureaucratic 69

C calculative process 73 capability process 73 cardiovascular disease 65 clinical decision support systems (CDSS) 33 clinical efficacy 243 clinical laboratory information system 200 cognitive appraisal processes 69 cognitive instrumental processes 15 compability 6, 139 competitive forces 302 complex 139 confirmation 5 coping strategies 72 CRM 133 Cronbach’s alpha values 18

D data collection 111 data triangulation 227 decision 5 design costs 162 diagnostic systems 285


340 Index

Diffusion 2 distribution of power 110

E e-connectivity 307 e-Health 282, 296 early adopters 4 early majority 4 education 289 efficiency 289, 303 electronic health record (EHR) 131 electronic patient records (EPRs) 128 empirical relevance 180 requirements 180 resistance 179 resources 180 employee assistance programs 76 empowered consumers 296 empowerment 289 enabling 289 encouragement 289 enhanced quality of care 289 environmental conductiveness (EC) 137 EPR implementation 268 EPS 163 equity 289 ethics 289 evaluation 213, 242 evidence-based 289 ex ante 181 ex post 183 extensive 68 external context 118 validity 215 extrinsic effort 78 rewards 78

F factor analysis 19 formulation-making decisions 298 fourth element 239 frequency of use 46

G general practitioners (GPs) 183 group support systems (GSS 36

H home care systems 283 hospital information system (HIS) 253, 285 hospital systems 283

I image 16 implementation 5, 298 incentives 136 individual differences 69 information quality 102 systems 32, 239 informativeness 160 innovation diffusion theory 2 installed base 140 intensive 68 intentionality process 73 intentions 10 Internet standards adoption (ISA) 128, 137 Internet-based health applications (IHA) 14 intervention 253 investigator triangulation 227 IS innovation 100 IT education 315

J job orientation 205 job stress 66

K knowledge 5, 134

L laggards 4 late majority 4 learning loop 240


Index 341

M maintenance costs 162 medical cognition 32 medical informatics systems 32 methodology 329 methods triangulation 227 metrics 136 micro-relevance 157 mobility 72 motivation for evaluation 218 moving evaluation target 215 musculoskeletal disorders 65

N Net benefits 102 network externalities 140 nursing documentation 222

O objectives 135 observability 6 observable benefits 140 online disability evaluation 39 operations costs 162 organizational 52 outcome 241 output quality 16

P PDA experience 188 pediatricians 21 penetration strategy 120 people 100 perceived behavioral control 10 ease of of use 8, 15 system quality 265 usefulness 8, 15 persuasion 5 pioneering computer-based patient record 200 pluralistic strategic apex 117 political interaction 110

tactics 111 prediction process 73 prevention 282 primary care systems 283 private partners 115 problem-oriented medical information system 200 process 103, 241 product 103 professionals 115 promoters 114 prototype strategy 120 prudent paranoia 74 psychological disorders 65 pulse 188

Q qualitative methods 200 quality of care 241 quantitative methods 200

R rationale 134 readiness 52 record keeping 256 regional health information networks 284 regression analyses 21 rehabilitation 282 reinvention 6 relative advantage 5, 139 resistance 33 framework 154 potential 153 result demonstrability 16 reversibility 72 rheumatism 181

S selectivity 72 self concern 153 -efficacy 8 service quality 102 social


342 Index

influence processes 15 support 70 -care industries 63 sponsorship 140 stepwise regression analyses 21 stress management 76 stressor reduction 76 stroke service 183 structure 241 subjective norm 10 suicide 65 system quality 102 sytematic 69

W willingness to change 149 work stressors 62 workplace injury 65 World Federation for Mental Health (WFMH) 64 World Trade Organization (WTO) 314

T task-technology fit (TTF) 32 technical 52 technology acceptance model (TAM) 1, 36 teleconsultation systems 285 telehealth 241 telemedicine 285 telephone-linked care (TLC) 206 theoretical approach 110 theory of planned behavior (TPB) 9, 37 theory triangulation 227 time 215 time trends 260 transference process 73 treatment 282 trialability 6, 140 triangulation 207, 214, 221 typologies 285

U usage 256 usefulness of the features (UF) 137 user evaluation 257 user needs 160 user-satisfaction 150 users’ evaluation 263

V variance 160 voluntariness 16


Experience the latest full-text research in the fields of Information Science, Technology & Management

InfoSci-Online InfoSci-Online is available to libraries to help keep students, faculty and researchers up-to-date with the latest research in the ever-growing field of information science, technology, and management. The InfoSci-Online collection includes: Scholarly and scientific book chapters Peer-reviewed journal articles Comprehensive teaching cases Conference proceeding papers All entries have abstracts and citation information The full text of every entry is downloadable in .pdf format

InfoSci-Online features: Easy-to-use 6,000+ full-text entries Aggregated Multi-user access

Some topics covered: Business Management Computer Science Education Technologies Electronic Commerce Environmental IS Healthcare Information Systems Information Systems Library Science Multimedia Information Systems Public Information Systems Social Science and Technologies

“…The theoretical bent of many of the titles covered, and the ease of adding chapters to reading lists, makes it particularly good for institutions with strong information science curricula.” — Issues in Science and Technology Librarianship

To receive your free 30-day trial access subscription contact: Andrew Bundy Email: [email protected] • Phone: 717/533-8845 x29 Web Address: www.infosci-online.com

A PRODUCT OF Publishers of Idea Group Publishing, Information Science Publishing, CyberTech Publishing, and IRM Press

infosci-online.com

Books on Healthcare Information Systems Effective Healthcare Information Systems Adi Armoni, Ph.D., Tel Aviv University, Israel Health and medical informatics encompass a very broad field that is rapidly developing in both its research and operational aspects. The discipline has many dimensions, including social, legal, ethical and economic. This book, Effective Healthcare Information Systems puts a special emphasis on issues dealing with the most recent innovations such as telemedicine, Web-based medical information and consulting systems, expert systems and artificial intelligence. ISBN: 1-931777-01-2; eISBN: 1-931777-20-9; Copyright: 2002 Pages: 295 (s/c); Price: US $59.95

Strategies for Healthcare Information Systems Robert Stegwee, Ph.D., University of Twente, The Netherlands Ton Spil, University of Twente, The Netherlands Information technologies of the past two decades have created significant fundamental changes in the delivery of healthcare services by healthcare provider organizations. Strategies for Healthcare Information Systems provides an overall coverage of different aspects of healthcare information systems strategies and challenges facing these organizations. ISBN: 1-878289-89-6; Copyright: 2001; Pages: 232 (s/c); Price: US $74.95

Knowledge Media in Healthcare: Opportunities and Challenges Edited by Rolf Grutter, Ph.D., University of St. Gallen, Switzerland

Because the field of healthcare reflects forms of both explicit and tacit knowledge such as evidence-based knowledge, clinical guidelines and the physician s experience, knowledge media have significant potential in this area. Knowledge Media in Healthcare: Opportunities and Challenges is an innovative new book which strives to show the positive impact that Knowledge Media and communication technology can have on human communication within the field of healthcare. ISBN 1-930708-13-0; eISBN: 1-59140-006-6; Copyright: 2002; Pages: 296; Price: US$74.95

Excellent additions to your library– Please recommend to your librarian.

Managing Healthcare Information Systems with Web-Enabled Technologies Lauren Eder, Ph.D., Rider University, USA Healthcare organizations are undergoing major adjustments to meet the increasing demands of improved healthcare access and quality. Managing Healthcare Information Systems with Web-Enabled Technologies presents studies from leading researchers focusing on the current challenges, directions and opportunities associated with healthcare organizations and their strategic use of Web-enabled technologies. ISBN: 1-878289-65-9 ; eISBN:1-930708-67-X; Copyright: 2000 Pages: 288 (s/c); Price: US $69.95

Healthcare Information Systems: Challenges of the New Millennium Adi Armoni, Ph.D., Tel Aviv University, Israel

Healthcare information sysstems are crucial to the effective and efficient delivery of healthcare. Healthcare Information Systems: Challenges of the New Millennium reports on the implementation of medical information systems. ISBN: 1-878289-62-4; eISBN: 1-930708-55-6; Copyright: 2000 Pages: 256 (s/c); Price: US $69.95

Its Easy to Order! Order online at www.idea-group.com or call 1-717-533-8845 ext.10!

Mon-Fri 8:30 am-5:00 pm (est) or fax 24 hours a day 717/533-8661

Idea Group Inc. Hershey • London • Melbourne • Singapore • Beijing

New Releases from Idea Group Reference

Idea Group REFERENCE

The Premier Reference Source for Information Science and Technology Research ENCYCLOPEDIA OF

ENCYCLOPEDIA OF

DATA WAREHOUSING AND MINING

INFORMATION SCIENCE AND TECHNOLOGY AVAILABLE NOW!

Edited by: John Wang, Montclair State University, USA Two-Volume Set • April 2005 • 1700 pp ISBN: 1-59140-557-2; US $495.00 h/c Pre-Publication Price: US $425.00* *Pre-pub price is good through one month after the publication date

Provides a comprehensive, critical and descriptive examination of concepts, issues, trends, and challenges in this rapidly expanding field of data warehousing and mining A single source of knowledge and latest discoveries in the field, consisting of more than 350 contributors from 32 countries

Five-Volume Set • January 2005 • 3807 pp ISBN: 1-59140-553-X; US $1125.00 h/c

ENCYCLOPEDIA OF

DATABASE TECHNOLOGIES AND APPLICATIONS

Offers in-depth coverage of evolutions, theories, methodologies, functionalities, and applications of DWM in such interdisciplinary industries as healthcare informatics, artificial intelligence, financial modeling, and applied statistics Supplies over 1,300 terms and definitions, and more than 3,200 references

DISTANCE LEARNING

April 2005 • 650 pp ISBN: 1-59140-560-2; US $275.00 h/c Pre-Publication Price: US $235.00* *Pre-publication price good through one month after publication date

Four-Volume Set • April 2005 • 2500+ pp ISBN: 1-59140-555-6; US $995.00 h/c Pre-Pub Price: US $850.00* *Pre-pub price is good through one month after the publication date

MULTIMEDIA TECHNOLOGY AND NETWORKING

ENCYCLOPEDIA OF

ENCYCLOPEDIA OF

More than 450 international contributors provide extensive coverage of topics such as workforce training, accessing education, digital divide, and the evolution of distance and online education into a multibillion dollar enterprise Offers over 3,000 terms and definitions and more than 6,000 references in the field of distance learning Excellent source of comprehensive knowledge and literature on the topic of distance learning programs Provides the most comprehensive coverage of the issues, concepts, trends, and technologies of distance learning

April 2005 • 650 pp ISBN: 1-59140-561-0; US $275.00 h/c Pre-Publication Price: US $235.00* *Pre-pub price is good through one month after publication date

www.idea-group-ref.com

Idea Group Reference is pleased to offer complimentary access to the electronic version for the life of edition when your library purchases a print copy of an encyclopedia For a complete catalog of our new & upcoming encyclopedias, please contact: 701 E. Chocolate Ave., Suite 200 • Hershey PA 17033, USA • 1-866-342-6657 (toll free) • [email protected]

E-Health Systems Diffusion and Use: The Innovation, the User and the UseIT Model

Simulation: The Practice of Model Development and Use

Innovation and the Forest Sector

Noise and Diffusion in Bistable Nonequlibrium Systems

Taxation, Innovation and the Environment

Innovation and the Communications Revolution

The Overlapping Generations Model and the Pension

The Global Diffusion of Markets and Democracy

Relaxation and Diffusion in Complex Systems

The Global Diffusion of Markets and Democracy

Innovation Diffusion in the New Economy: Tacit Component

User Interface Management Systems: Models and Algorithms

Models for Innovation Diffusion (Quantitative Applications in the Social Sciences)

The Mathematics of Diffusion

Relaxation and Diffusion in Complex Systems (Partially Ordered Systems)

The Model

The mathematics of diffusion

The Mathematics of Diffusion

The Mathematics of Diffusion

The nonlinear diffusion equation

The Model

The Model

Molecular Model Systems in the Lepidoptera

Knowledge Diffusion and Innovation: Modelling Complex Entrepreneurial Behaviours

The Model

Engineering The User Interface

Diffusion and Reactions in Fractals and Disordered Systems

User Experience Innovation: User Centered Design that Works

Innovation And Diffusion Of Software Technology: Mapping Strategies

Indicator Systems for Sustainable Innovation (Sustainability and Innovation)

Diffusion and reactions in fractals and desordered systems

E-Health Systems Diffusion and Use: The Innovation, the User and the UseIT Model

Simulation: The Practice of Model Development and Use

Innovation and the Forest Sector

Noise and Diffusion in Bistable Nonequlibrium Systems

Taxation, Innovation and the Environment

Innovation and the Communications Revolution

The Overlapping Generations Model and the Pension

The Global Diffusion of Markets and Democracy

Relaxation and Diffusion in Complex Systems

The Global Diffusion of Markets and Democracy

Innovation Diffusion in the New Economy: Tacit Component

User Interface Management Systems: Models and Algorithms

Models for Innovation Diffusion (Quantitative Applications in the Social Sciences)

The Mathematics of Diffusion

Relaxation and Diffusion in Complex Systems (Partially Ordered Systems)

The Model

The mathematics of diffusion

The Mathematics of Diffusion

The Mathematics of Diffusion

The nonlinear diffusion equation

The Model

The Model

Molecular Model Systems in the Lepidoptera

Knowledge Diffusion and Innovation: Modelling Complex Entrepreneurial Behaviours

The Model

Engineering The User Interface

Diffusion and Reactions in Fractals and Disordered Systems

User Experience Innovation: User Centered Design that Works

Innovation And Diffusion Of Software Technology: Mapping Strategies

Indicator Systems for Sustainable Innovation (Sustainability and Innovation)

Diffusion and reactions in fractals and desordered systems

Recommend Documents