MEDINFO 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics (Studies in Health Technology and Informatics)

MEDINFO 2007

Studies in Health Technology and Informatics This book series was started in 1990 to promote research conducted under the auspices of the EC programmes’ Advanced Informatics in Medicine (AIM) and Biomedical and Health Research (BHR) bioengineering branch. A driving aspect of international health informatics is that telecommunication technology, rehabilitative technology, intelligent home technology and many other components are moving together and form one integrated world of information and communication media. The complete series has been accepted in Medline. Volumes from 2005 onwards are available online. Series Editors: Dr. J.P. Christensen, Prof. G. de Moor, Prof. A. Famili, Prof. A. Hasman, Prof. L. Hunter, Dr. I. Iakovidis, Dr. Z. Kolitsi, Mr. O. Le Dour, Dr. A. Lymberis, Prof. P.F. Niederer, Prof. A. Pedotti, Prof. O. Rienhoff, Prof. F.H. Roger France, Dr. N. Rossing, Prof. N. Saranummi, Dr. E.R. Siegel, Dr. P. Wilson, Prof. E.J.S. Hovenga, Prof. M.A. Musen and Prof. J. Mantas

Volume 129 Recently published in this series Vol. 128. P.J. Murray, H.-A. Park, W.S. Erdley and J. Kim (Eds.), Nursing Informatics 2020: Towards Defining Our Own Future – Proceedings of NI2006 Post Congress Conference Vol. 127. L. Bos and B. Blobel (Eds.), Medical and Care Compunetics 4 Vol. 126. N. Jacq, H. Müller, I. Blanquer, Y. Legré, V. Breton, D. Hausser, V. Hernández, T. Solomonides and M. Hofmann-Apitius (Eds.), From Genes to Personalized HealthCare: Grid Solutions for the Life Sciences – Proceedings of HealthGrid 2007 Vol. 125. J.D. Westwood, R.S. Haluck, H.M. Hoffman, G.T. Mogel, R. Phillips, R.A. Robb and K.G. Vosburgh (Eds.), Medicine Meets Virtual Reality 15 – in vivo, in vitro, in silico: Designing the Next in Medicine Vol. 124. A. Hasman, R. Haux, J. van der Lei, E. De Clercq and F.H. Roger France (Eds.), Ubiquity: Technologies for Better Health in Aging Societies – Proceedings of MIE2006 Vol. 123. D. Uyttendaele and P.H. Dangerfield (Eds.), Research into Spinal Deformities 5 Vol. 122. H.-A. Park, P. Murray and C. Delaney (Eds.), Consumer-Centered Computer-Supported Care for Healthy People – Proceedings of NI2006 – The 9th International Congress on Nursing Informatics Vol. 121. L. Bos, L. Roa, K. Yogesan, B. O’Connell, A. Marsh and B. Blobel (Eds.), Medical and Care Compunetics 3 Vol. 120. V. Hernández, I. Blanquer, T. Solomonides, V. Breton and Y. Legré (Eds.), Challenges and Opportunities of HealthGrids – Proceedings of Healthgrid 2006 Vol. 119. J.D. Westwood, R.S. Haluck, H.M. Hoffman, G.T. Mogel, R. Phillips, R.A. Robb and K.G. Vosburgh (Eds.), Medicine Meets Virtual Reality 14 – Accelerating Change in Healthcare: Next Medical Toolkit Vol. 118. R.G. Bushko (Ed.), Future of Intelligent and Extelligent Health Environment Vol. 117. C.D. Nugent, P.J. McCullagh, E.T. McAdams and A. Lymberis (Eds.), Personalised Health Management Systems – The Integration of Innovative Sensing, Textile, Information and Communication Technologies Vol. 116. R. Engelbrecht, A. Geissbuhler, C. Lovis and G. Mihalas (Eds.), Connecting Medical Informatics and BioInformatics – Proceedings of MIE2005 Vol. 115. N. Saranummi, D. Piggott, D.G. Katehakis, M. Tsiknakis and K. Bernstein (Eds.), Regional Health Economies and ICT Services ISSN 0926-9630

MEDINFO 2007 Proceedings of the 12th World Congress on Health (Medical) Informatics Building Sustainable Health Systems

Part 1

Edited by

Klaus A. Kuhn University Medical Center, Technische Universität München, Germany

James R. Warren Department of Computer Science, University of Auckland, New Zealand

and

Tze-Yun Leong School of Computing, National University of Singapore, Singapore

Amsterdam • Berlin • Oxford • Tokyo • Washington, DC

© 2007 The authors. All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without prior written permission from the publisher. ISBN 978-1-58603-774-1

Publisher IOS Press Nieuwe Hemweg 6B 1013 BG Amsterdam Netherlands fax: +31 20 687 0019 e-mail: [email protected] Distributor in the UK and Ireland Gazelle Books Services Ltd. White Cross Mills Hightown Lancaster LA1 4XS United Kingdom fax: +44 1524 63232 e-mail: [email protected]

Distributor in the USA and Canada IOS Press, Inc. 4502 Rachael Manor Drive Fairfax, VA 22032 USA fax: +1 703 323 3668 e-mail: [email protected]

LEGAL NOTICE The publisher is not responsible for the use which might be made of the following information. PRINTED IN THE NETHERLANDS

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Table of Contents Editorial Preface from the Scientific Program Co-Chairs Medinfo 2007 Scientific Program Committee In Memory of Dr Branko Cesnik

xxv xxvii xxviii

Part 1 Chapter 1. eHealth

1

eHealth Solutions to the Continuity of Care Linkcare – Enabling Continuity of Care for the Chronically Ill across Levels and Profession Marius Mikalsen, Ståle Walderhaug, Per Håkon Meland and Ole Martin Winnem

3

Sustainable Ubiquitous Home Health Care – Architectural Considerations and First Practical Experiences Michael Marschollek, Klaus-H. Wolf, Oliver-J. Bott, Mirko Geisler, Maik Plischke, Wolfram Ludwig, Andreas Hornberger and Reinhold Haux

8

Do Physicians Take Action on High Risk Family History Information Provided by Patients Outside of a Clinic Visit? Lynn A. Volk, Maria Staroselsky, Lisa P. Newmark, Hannah Pham, Alexis Tumolo, Deborah H. Williams, Ruslana Tsurikova, Jeffrey Schnipper, Jonathan Wald and David W. Bates Sharing Electronic Laboratory Results in a Patient Portal – A Feasibility Pilot Jonathan S. Wald, Karen Burk, Kate Gardner, Raisa Feygin, Elizabeth Nelson, Marianna Epstein, Eric G. Poon and Blackford Middleton

13

18

eHealth Systems Modeling An Evolving Systems-Based Methodology for Healthcare Planning Jon Warwick and Gary Bell

23

Model-Centric Approaches for the Development of Health Information Systems Mika Tuomainen, Juha Mykkänen, Heli Luostarinen, Assi Pöyhölä and Esa Paakkanen

28

Software Engineering Principles Applied to Large Healthcare Information Systems – A Case Report Fabiane Bizinella Nardon and Lincoln de A. Moura Jr.

33

Towards Modeling and Simulation of Integrated Social and Health Care Services for Elderly Alexander Horsch and Daryoush Khoshsima

38

Healthcare Delivery Systems: Designing Quality into Health Information Systems Phil Joyce, Rosamund Green and Graham Winch

43

A Mobile Data Collection Tool for Workflow Analysis Jacqueline Moss, Eta S. Berner and Kathy Savell

48

Towards Sustainable e-Health Networks: Does Modeling Support Efficient Management and Operation? Martin Staemmler

53

The openEHR Java Reference Implementation Project Rong Chen and Gunnar Klein

58

v

A Conceptual Model of Computerised Hospital Information System (CHIS) use in South Africa Lyn A. Hanmer, Sedick Isaacs and J. Dewald Roode

63

Are Problem-Oriented Medial Records (POMR) Suitable for Use in GPs’ Daily Practice? Etienne De Clercq, Viviane Van Casteren, Pascale Jonckheer, Peter Burggraeve, Marie France Lafontaine, Karen Degroote and Francis Roger France

68

Can an EPR Support the Concept of Family-Centred, Individualized Developmental Care of Premature Infants and Newborns? Christian D. Kohl, Claus Schott, Doris Verveur, Otwin Linderkamp and Petra Knaup-Gregori Multiple Detection Modalities and Disease Natural History of Breast Cancer Tony Hsiu-Hsi Chen, Amy Ming-Fang Yen, Grace Hui-Min Wu, Li-Sheng Chen and Yueh-Hsia Chiu

73 78

Care at a Distance Challenges in Telemedicine and eHealth: Lessons Learned from 20 Years with Telemedicine in Tromsø Gunnar Hartvigsen, Monika A. Johansen, Per Hasvold, Johan Gustav Bellika, Eirik Arsand, Eli Arild, Deede Gammon, Sture Pettersen and Steinar Pedersen

82

A Satellite Infrastructure for Health Early Warning in Post-Disaster Health Management C.E. Chronaki, A. Berthier, M.M. Lleo, L. Esterle, A. Lenglet, F. Simon, L. Josseran, M. Lafaye, Y. Matsakis, A. Tabasco and L. Braak

87

Remote Support for Stroke Rehabilitation: MyHeart’s “Neurological Rehabilitation” Concept Toni Giorgino, Paolo Tormene, Barbara Cattani, Caterina Pistarini and Silvana Quaglini

92

Effects of a Computerized Cardiac Teletriage Decision Support System on Nurse Performance: Results of a Controlled Human Factors Experiment Using a Mid-Fidelity Prototype Kirsten Carroll Somoza, Kathryn Momtahan and Gitte Lindgaard

97

Mobile Care Opportunities and Barriers for Mobile Health in New Zealand Farhaan Mirza and Tony Norris

102

Applying Mobile and Pervasive Computer Technology to Enhance Coordination of Work in a Surgical Ward Thomas Riisgaard Hansen and Jakob E. Bardram

107

Feasibility and Usability of a Home Monitoring Concept based on Mobile Phones and Near Field Communication (NFC) Technology Jürgen Morak, Alexander Kollmann and Günter Schreier

112

Using Personal Digital Assistants and Patient Care Algorithms to Improve Access to Cardiac Care Best Practices Kathryn L. Momtahan, Catherine M. Burns, Heather Sherrard, Thierry Mesana and Marino Labinaz

117

Interoperability Issues in eHealth Information Systems Improving Systems Interoperability with Model-Driven Software Development for HealthCare Ståle Walderhaug, Marius Mikalsen, Gunnar Hartvigsen, Erlend Stav and Jan Aagedal

122

Conformance Testing of Interoperability in Health Information Systems in Finland Tanja Toroi, Anne Eerola and Juha Mykkänen

127

Connecting Public Health and Clinical Information Systems by Using a Standardized Methodology Diego M. Lopez and Bernd G.M.E. Blobel

132

Clinical and Translational Science Sustainability: Overcoming Integration Issues Between Electronic Health Records (EHR) and Clinical Research Data Management Systems “Separate but Equal” Robert P. DiLaura

137

Searching for Answers The FA4CT Algorithm: A New Model and Tool for Consumers to Assess and Filter Health Information on the Internet Gunther Eysenbach and Maria Thomson

vi

142

WRAPIN: A Tool for Patient Empowerment Within EHR Michel Joubert, Arnaud Gaudinat, Célia Boyer, Antoine Geissbuhler, Marius Fieschi, HON Foundation Council members

147

How do Clinicians Search For and Access Biomedical Literature to Answer Clinical Questions? Annie Y.S. Lau and Enrico Coiera

152

Emerging Technologies Blogs, Wikis, and Discussion Forums: Attributes and Implications for Clinical Information Systems Jacob B. Weiss and Thomas R. Campion Jr. From Intermediation to Disintermediation and Apomediation: New Models for Consumers to Access and Assess the Credibility of Health Information in the Age of Web2.0 Gunther Eysenbach A Mobile Phone Based Remote Patient Monitoring System For Chronic Disease Management Mathieu Trudel, Joseph A Cafazzo, Melinda Hamill, Walter Igharas, Kevin Tallevi, Peter Picton, Jack Lam, Peter G. Rossos, Anthony C. Easty and Alexander Logan

157

162 167

Consumer Health Technologies How to Enhance Integrated Care Towards the Personal Health Paradigm? Bernd G.M.E. Blobel, Peter Pharow and Thomas Norgall

172

Developing Information Technology Attitude Scales for Health (ITASH) Rod Ward, Katherine Pollard, Margaret Glogowska and Pam Moule

177

Development of Patient Centric Virtual Organizations (PCVOs) in Clinical Environment for Patient Information Management Mohyuddin, W.A. Gray, Hazel Bailey, Wendy Jones and David Morrey SPIRS: A Framework for Content-Based Image Retrieval from Large Biomedical Databases William Hsu, L. Rodney Long and Sameer Antani

182 188

The Impact of Clinical Information Systems The Quality of Reporting of Health Informatics Evaluation Studies: A Pilot Study Jan Talmon, Elske Ammenwerth and Thom Geven

193

Exploring the Unintended Consequences of Computerized Physician Order Entry Joan S. Ash, Dean F. Sittig, Richard Dykstra, Emily Campbell and Kenneth Guappone

198

Impact of Health Care Information Technology on Hospital Productivity Growth: A Survey in 17 Acute University Hospitals Rodolphe Meyer, Patrice Degoulet and Louis Omnes Risk Management and Measuring Productivity with POAS – Point of Act System Masanori Akiyama and Tatsuya Kondo

203 208

eHealth at-Large Connecting the Dots: Creation of an Electronic Regional Infection Control Network Abel N. Kho, Paul Dexter, Larry Lemmon, Dauna Carey, Heather Woodward-Hagg, Steve Hare and Brad Doebbeling Dealing with Ecological Fallacy in Preparations for Influenza Pandemics: Use of a Flexible Environment for Adaptation of Simulations to Household Structures in Local Contexts Toomas Timpka, Magus Morin, Johan Jenvald, Elin Gursky and Henrik Eriksson

213

218

Effectiveness of a Chronic Disease Surveillance Systems for Blood Pressure Monitoring Damian Borbolla, Diego Giunta, Silvana Figar, Mercedes Soriano, Adriana Dawidowski and Fernan Gonzalez Bernaldo de Quiros

223

Nurses and Computers. An International Perspective on Nurses’ Requirements Carol S. Bond

228

vii

Issues in Clinical Care and Research The Need for Global Certification in the Field of Health Informatics: Some Ethical Issues Eike-Henner W. Kluge

233

Using Fieldwork in Analyzing Ethical Issues Related to IT in Health Care Ellen Balka, Christine Reidl and Ina Wagner

237

Handling Consent to Patient Data Access in a Hospital Setting Gunnar René Øie, Herbjørn Andresen and Inger Anne Tøndel

242

A Day in the Life of a Clinical Research Coordinator: Observations from Community Practice Settings Sharib A. Khan, Rita Kukafka, Philip R.O. Payne, J. Thomas Bigger and Stephen B. Johnson

247

eHealth System Adoption A Survey of U.S.A. Acute Care Hospitals’ Computer-Based Provider Order Entry System Infusion Levels Dean F. Sittig, Ken Guappone, Emily M. Campbell, Richard H. Dykstra and Joan S. Ash

252

E-Healthcare in India: Critical Success Factors for Sustainable Health Systems Udita Taneja, Sushil

257

Health Information Systems Adoption: Findings from a Systematic Review Maryati Mohd. Yusof, Lampros Stergioulas and Jasmina Zugic

262

Telemedicine Portal and Standardized Clinical Documents Ivan V. Emelin and Radik A. Eltchiyan

267

eHealth and Risk Management The Development of an Information System and Installation of an Internet Web Database for the Purposes of the Occupational Health and Safety Management System I. Mavrikakis, J. Mantas and M. Diomidous Reliability Assessment of Home Health Care Services Stergiani Spyrou, Panagiotis Bamidis, Vassilis Kilintzis, Irini Lekka, Nicos Maglaveras and Costas Pappas From the Description of Activities to the Identification of Risks for Clinical Management: A Proposal of Building, Merging and Sharing Knowledge Representations of Care Processes Pascal Staccini, Michel Joubert, Rémy Collomp, Jean-François Quaranta and Marius Fieschi

270 275

280

Near-Miss and Hazard Reporting: Promoting Mindfulness in Patient Safety Education Leanne M. Currie, Karen S. Desjardins, Patricia W. Stone, Tsai-ya Lai, Eric Schwartz, Rebecca Schnall and Suzanne Bakken

285

Chapter 2. Data Repositories and Information Retrieval

291

Wide Electronic Health Records A Territory-Wide Electronic Health Record – From Concept to Practicality: The Hong Kong Experience Antonio C.H. Sek, N.T. Cheung, K.M. Choy, W.N. Wong, Anna Y.H. Tong, Vicky H. Fung, Michael Fung and Eric Ho Northern Territory HealthConnect: Shared Electronic Health Record Service Implementation Experiences and Benefits Realised in Indigenous Health Stephen Moo and John Fletcher A Simulation-Based Performance Analysis of a National Electronic Health Record System Leonidas Orfanidis, Panagiotis D. Bamidis and Barry Eaglestone Principles-Based Medical Informatics for Success – How Hong Kong Built One of the World’s Largest Integrated Longitudinal Electronic Patient Records Ngai-Tseung Cheung, Vicky Fung, Wing Nam Wong, Anna Tong, Antonio Sek, Andre Greyling, Nancy Tse and Hong Fung

viii

293

297 302

307

Healthcare and Research Data Repositories Knowledge-Level Querying of Temporal Patterns in Clinical Research Systems Martin J. O’Connor, Ravi D. Shankar, David B. Parrish and Amar K. Das A Discussion About the Importance of Laws and Policies for Data Sharing for Public Health in the People’s Republic of China Xiue Fan and Ping Yu Confidentiality Preserving Audits of Electronic Medical Record Access Bradley Malin and Edoardo Airoldi Design of a Decentralized Reusable Research Database Architecture to Support Data Acquisition in Large Research Projects Jimison Iavindrasana, Adrien Depeursinge, Patrick Ruch, Stéphane Spahni, Antoine Geissbuhler and Henning Müller

311

316 320

325

Registers and Research The Cancer Biomedical Informatics Grid (caBIG™): Infrastructure and Applications for a Worldwide Research Community The caBIG Strategic Planning Workspace

330

The Integration of Grid Resources into a Portal for Research Collaboratories Yassene Mohammed, Sabine Rey, Fabian Rakebrandt and Ulrich Sax

335

Biomedical Data Mining in Clinical Routine: Expanding the Impact of Hospital Information Systems Marcel Müller, Kornel Markó, Philipp Daumke, Jan Paetzold, Arnold Roesner and Rüdiger Klar

340

The Swedish National Pharmacy Register Bengt Astrand, Bo Hovstadius, Karolina Antonov and Göran Petersson

345

Patient Record Documentation The Use of Existing Low-Cost Technologies to Enhance the Medical Record Documentation Using a Summary Patient Record [SPR] S. Bart and T. Hannan A New Approach in Nursing Documentation: Community Nursing Case Uros Rajkovic, Olga Sustersic, Vladislav Rajkovic and Vesna Prijatelj A Survey of the Effects of the full Computerized Nursing Records System on Sharing Nursing Records Among Health Professionals Yukio Kurihara, Naho Asai, Eri Ishimoto, Shigeyuki Kawamata and Satsuki Nakamura New Method of Realization of Nursing Diagnosis Based on 3N in an Electronic Medical Record System Young ah Kim, Mijung An, Jungyoen Park, Hyensun Jung, Yongoock Kim and Byungchul Chang

350 354

360 364

Integrated Health Records Integration of Longitudinal Electronic Records in a Large Healthcare Enterprise: The U.S. Veterans Health Administration Experience Arthur C. Curtis, Joseph Gillon and Dale C. Malmrose The AMPATH Medical Record System: Creating, Implementing, and Sustaining an Electronic Medical Record System to Support HIV/AIDS Care in Western Kenya William M. Tierney, Joseph K. Rotich, Terry J. Hannan, Abraham M. Siika, Paul G. Biondich, Burke W. Mamlin, Winstone M. Nyandiko, Sylvester Kimaiyo, Kara Wools-Kaloustian, John E. Sidle, Chrispinus Simiyu, Erika Kigotho, Beverly Musick, Joseph J. Mamlin and Robert M. Einterz

367

372

São Paulo City Health Information System – A Case Report Cláudio G.A. Costa, Beatriz F. Leão and Lincoln A. Moura Jr.

377

Experience in Implementing the OpenMRS Medical Record System to Support HIV Treatment in Rwanda Christian Allen, Darius Jazayeri, Justin Miranda, Paul G. Biondich, Burke W. Mamlin, Ben A. Wolfe, Chris Seebregts, Neal Lesh, William M. Tierney and Hamish S.F. Fraser

382

ix

Information Retrieval Multilingual Information Retrieval in Thoracic Radiology: Feasibility Study André Coutinho Castilla, Sérgio Shiguemi Furuie and Eneida A. Mendonça

387

Large-Scale Evaluation of a Medical Cross-Language Information Retrieval System Kornél Markó, Philipp Daumke, Stefan Schulz, Rüdiger Klar and Udo Hahn

392

EHR Query Language (EQL) – A Query Language for Archetype-Based Health Records Chunlan Ma, Heath Frankel, Thomas Beale and Sam Heard

397

Design and Evaluation of a Temporal, Graph-Based Language for Querying Collections of Patient Histories Ole Edsberg, Stein Jakob Nordbø, Erik Vinnes and Øystein Nytrø

402

Evaluation of a Simple Method for the Automatic Assignment of MeSH Descriptors to Health Resources in a French Online Catalogue Aurélie Névéol, Suzanne Pereira, Gaetan Kerdelhué, Badisse Dahamna, Michel Joubert and Stéfan J. Darmoni Template-based Data Entry for General Description in Medical Records and Data Transfer to Data Warehouse for Analysis Yasushi Matsumura, Shigeki Kuwata, Yuichiro Yamamoto, Kazunori Izumi, Yasushi Okada, Michihiro Hazumi, Sachiko Yoshimoto, Takahiro Mineno, Munetoshi Nagahama, Ayumi Fujii and Hiroshi Takeda

407

412

Challenges and Methodology for Indexing the Computerized Patient Record Frédéric Ehrler, Patrick Ruch, Antoine Geissbuhler and Christian Lovis

417

Experiments with Hierarchical Concept-Based Search Robert Moskovitch, Roee Sa'adon, Eytan Behiri, Susana Martins, Aviram Weiss and Yuval Shahar

422

Chapter 3. Sharing Data

427

Data: Standards and Sharing StructConsult: Structured Real-Time Wet Read Consultation Infrastructure to Support Patient Care Craig Morioka, John David N. Dionisio, Alex Bui, Suzie El-Saden and Hooshang Kangarloo

429

Refining DICOM for Pathology – Progress from the IHE and DICOM Pathology Working Groups 434 Christel Le Bozec, Dominique Henin, Bettina Fabiani, Thomas Schrader, Marcial Garcia-Rojo and Bruce Beckwith A Generic, Web-Based Clinical Information System Architecture Using HL7 CDA: Successful Implementation in Dermatological Routine Care 439 Thilo Schuler, Martin Boeker, Rüdiger Klar and Marcel Müller Analyzing the Key Variables in the Adoption Process of HL7 Alejandro E. Flores and Khin Than Win An XML Model of an Enhanced Data Dictionary to Facilitate the Exchange of Pre-Existing Clinical Research Data in International Studies Stephany N. Duda, Clint Cushman and Daniel R. Masys

444

449

Framework for Clinical Data Standardization Based on Archetypes Jose A. Maldonado, David Moner, Diego Tomás, Carlos Ángulo, Montserrat Robles and Jesualdo T. Fernández

454

Reaching Standards for Dissemination: A Case Study Helen Christensen and Kathleen Griffiths

459

A Discrete Time-Space Geography for Epidemiology: From Mixing Groups to Pockets of Local Order in Pandemic Simulations Einar Holm and Toomas Timpka

464

Data: Transfer and Technical Standards A National Study of eHealth Standardization in Finland – Goals and Recommendations Juha Mykkänen, Maritta Korhonen, Jari Porrasmaa, Tuula Tuomainen and Antero Ensio

x

469

Building a Womb-to-Tomb Health Record in Hong Kong – An Application of Information Architecture Vicky Fung, N.T. Cheung, Eric Ho, Clara Cheung, Hudson Chan, Kitty Tsang, Joycelyne Cheung, William Ho, Maggie Lau, Veronica Hung, Austen Wong, Anna Tong, W.N. Wong and Antonio Sek

474

Another HISA – The New Standard: Health Informatics – Service Architecture Gunnar O. Klein, Pier Angelo Sottile and Frederik Endsleff

478

Sharing Data Protecting Privacy While Sharing Medical Data Between Regional Healthcare Entities Tyrone Grandison, Srivatsava Ranjit Ganta, Uri Braun and James Kaufman

483

Pulling Back the Covers: Technical Lessons of a Real-World Health Information Exchange Atif Zafar and Brian E. Dixon

488

Geographically Distributed Complementary Content-Based Image Retrieval Systems for Biomedical Image Informatics Sameer K. Antani, Thomas M. Deserno, L. Rodney Long and George R. Thoma Utilizing SELinux to Mandate Ultra-Secure Access Control of Medical Records Peter R. Croll, Matt Henricksen, Bill Caelli and Vicky Liu

493 498

Secure Data Transmission Proposal of a French Health Identification Number Interoperable at the European Level Catherine Quantin, François-André Allaert, Paul Avillach, Benoît Riandeye Marius Fieschi, Maniane Fassa and Olivier Cohen

503

Clearinghouse: A Teleradiology Platform Emphasizing Security of Data and Communication Michael Spitzer, Lars Brinkmann and Frank Ueckert

508

K-Box: Automatic Structuring and Exchange of Medical Documents Based on the Clinical Documentation Architecture (CDA) Minh H. Doan, Paul-Ludwig Lott, Marek Václavík and Frank Ueckert

513

Chapter 4. Medical Knowledge, Ontologies and Terminologies

517

Understanding Natural Language Using Distributional Analysis to Semantically Classify UMLS Concepts Jung-Wei Fan, Hua Xu and Carol Friedman

519

A Reappraisal of Sentence and Token Splitting for Life Sciences Documents Katrin Tomanek, Joachim Wermter and Udo Hahn

524

Corpus-Based Error Detection in a Multilingual Medical Thesaurus Roosewelt L. Andrade, Edson Pacheco, Pindaro S. Cancian, Percy Nohama and Stefan Schulz

529

Defining Medical Words: Transposing Morphosemantic Analysis from French to English Louise Deléger, Fiammetta Namer and Pierre Zweigenbaum

535

Finding Malignant Findings from Radiological Reports Using Medical Attributes and Syntactic Information Takeshi Imai, Eiji Aramaki, Masayuki Kajino, Kengo Miyo, Yuzo Onogi and Kazuhiko Ohe

540

A Normalized Lexical Lookup Approach to Identifying UMLS Concepts in Free Text Vijayaraghavan Bashyam, Guy Divita, David B. Bennett, Allen C. Browne and Ricky K. Taira

545

Extracting Subject Demographic Information from Abstracts of Randomized Clinical Trial Reports Rong Xu, Yael Garten, Kaustubh S. Supekar, Amar K. Das, Russ B. Altman and Alan M. Garber

550

Coupling Ontology Driven Semantic Representation with Multilingual Natural Language Generation for Tuning International Terminologies Anne-Marie Rassinoux, Robert H. Baud, Jean-Marie Rodrigues, Christian Lovis and Antoine Geissbühler

xi

555

Discovering New Biomedical Knowledge: Machine Learning and Data Mining Biomedical Knowledge Discovery with Topological Constraints Modeling in Bayesian Networks: A Preliminary Report Guoliang Li and Tze-Yun Leong

560

Automatic Infection Detection System Ove Granberg, Johan Gustav Bellika, Eirik Årsand and Gunnar Hartvigsen

566

Risk Stratification for LDL Cholesterol Using Induction Technique Seung Hee Ho, Hyun Young Park, Yang Soo Jang and Sun Ha Jee

571

Results from Data Mining in a Radiology Department: The Relevance of Data Quality Martin Lang, Nanda Kirpekar, Thomas Bürkle, Susanne Laumann and Hans-Ulrich Prokosch

576

Evaluating Learning Models with Transitions of Human Interests Based on Objective Rule Evaluation Indices Hidenao Abe, Hideto Yokoi, Shusaku Tsumoto, Miho Ohsaki and Takahira Yamaguchi

581

An Automated Data Pattern Translation Process for Medical Data Mining Anna Shillabeer

586

A Data Mining Approach to Analyze Non-Compliance with a Guideline for the Treatment of Breast Cancer Amir R. Razavi, Hans Gill, Hans Åhlfeldt and Nosrat Shahsavar

591

Bayesian Networks for Multivariate Data Analysis and Prognostic Modelling in Cardiac Surgery Niels Peek, Marion Verduijn, Peter M.J. Rosseel, Evert de Jonge and Bas A. de Mol

596

Role of Syndromic Management Using Dynamic Machine Learning in Future of e-Health in Pakistan Aijaz Qadir Patoli

601

Classification Systems Combining Lexical and Semantic Methods of Inter-Terminology Mapping Using the UMLS Kin Wah Fung, Olivier Bodenreider, Alan R. Aronson, William T. Hole and Suresh Srinivasan

605

Biomedical Vocabularies – The Demand for Differentiation Josef Ingenerf and Siegfried J. Pöppl

610

Development of a Taxonomy for Health Information Technology Brian E. Dixon, Atif Zafar and Julie J. McGowan

616

A Practical Approach to Advanced Terminology Services in Health Information Systems Maria Laura Gambarte, Alejandro Lopez Osornio, Marcela Martinez, Guillermo Reynoso, Daniel Luna and Fernan Gonzalez Bernaldo de Quiros

621

SNOMED Terminology Toward the Interoperability of HL7 v3 and SNOMED CT: A Case Study Modeling Mobile Clinical Treatment Amanda Ryan, Peter Eklund and Brett Esler

626

A Web-Based SNOMED CT Browser: Distributed and Real-Time Use of SNOMED CT During the Clinical Research Process Rachel Richesson, Asif Syed, Heather Guillette, Mark S. Tuttle and Jeffrey Krischer

631

Using SNOMED CT® as a Reference Terminology to Cross Map Two Highly Pre-Coordinated Classification Systems Steven H. Brown, Casey S. Husser, Dietlind Wahner-Roedler, Sandra Bailey, Linda Nugent, Karla Porter, Brent A. Bauer and Peter L. Elkin

636

Practical Issues in Using SNOMED CT as a Reference Terminology Senthil K. Nachimuthu and Lee Min Lau

640

Issues in Terminology Management Statistical Selector of the Best Multiple ICD-coding Method Eiji Aramaki, Takeshi Imai, Masayuki Kajino, Kengo Miyo and Kazuhiko Ohe

xii

645

Creation and Evaluation of a Terminology Server for the Interactive Coding of Discharge Summaries Hernán Navas, Alejandro Lopez Osornio, Analía Baum, Adrian Gomez, Daniel Luna and Fernan Gonzalez Bernaldo de Quiros

650

Thesaurus Anomaly Detection by User Action Monitoring Jeferson L. Bitencourt, Píndaro S. Cancian, Edson J. Pacheco, Percy Nohama and Stefan Schulz

655

Harmonizing Clinical Terminologies: Driving Interoperability in Healthcare Russell A. Hamm, Sarah E. Knoop, Peter Schwarz, Aaron D. Block and Warren L. Davis IV

660

Semantic Issues in Healthdata Classification A New Machine Learning Classifier for High Dimensional Healthcare Data Rema Padman, Xue Bai and Edoardo M. Airoldi

664

Structuring of Free-Text Diagnostic Report Hirofumi Fujii, Hiromasa Yamagishi, Yutaka Ando, Nobuhiro Tsukamoto, Osamu Kawaguchi, Tomotaka Kasamatsu, Kaoru Kurosaki, Masakazu Osada, Hiroshi Kaneko and Atsushi Kubo

669

Semantic Issues in Integrating Data from Different Models to Achieve Data Interoperability Rahil Qamar and Alan Rector

674

Medication Reconciliation Using Natural Language Processing and Controlled Terminologies James J. Cimino, Tiffani J. Bright and Jianhua Li

679

Biomedical Text Mining and Standard Terminology Comparing Medical Code Usage with the Compression-Based Dissimilarity Measure Thomas Brox Røst, Ole Edsberg, Anders Grimsmo and Øystein Nytrø

684

A Scale-Free Network View of the UMLS to Learn Terminology Translations Chintan O. Patel and James J. Cimino

689

Assigning Categorical Information to Japanese Medical Terms Using MeSH and MEDLINE Yuzo Onogi

694

PharmARTS: Terminology Web Services for Drug Safety Data Coding and Retrieval Iulian Alecu, Cédric Bousquet, Patrice Degoulet and Marie-Christine Jaulent

699

Machine Learning Approach for Automatic Quality Criteria Detection of Health Web Pages Arnaud Gaudinat, Natalia Grabar and Célia Boyer

705

Using Discourse Analysis to Improve Text Categorization in MEDLINE Patrick Ruch, Antoine Geissbühler, Julien Gobeill, Frederic Lisacek, Imad Tbahriti, Anne-Lise Veuthey and Alan R. Aronson

710

A Comparison of Impact Factor, Clinical Query Filters, and Pattern Recognition Query Filters in Terms of Sensitivity to Topic Lawrence D. Fu, Lily Wang, Yindalon Aphinyanagphongs and Constantin F. Aliferis

716

A Method for Defining a Journal Subset for a Clinical Discipline Using the Bibliographies of Systematic Reviews Nancy L. Wilczynski, Amit X. Garg, R. Brian Haynes, for the Nephrology Hedges Team

721

Ontologies Aequus Communis Sententia: Defining Levels of Interoperability Peter L. Elkin, David Froehling, Brent A. Bauer, Dietlind Wahner-Roedler, S. Trent Rosenbloom, Kent Bailey and Steven H. Brown

725

What’s in a code? Towards a Formal Account of the Relation of Ontologies and Coding Systems Alan L. Rector

730

A Road from Health Care Classifications and Coding Systems to Biomedical Ontology: The CEN Categorial Structure for Terminologies of Human Anatomy: Catanat J.M. Rodrigues, C. Rosse, M. Fogelberg, A. Kumar and B. Trombert Paviot

xiii

735

The Nodes Focusing Tool for Clinical Course Data of Hypergraph Structure in the Ontological Framework CSX Output from POMR-Based EMR system Yasuyuki Hirose, Ryuichi Yamamoto and Shinichiro Ueda

741

Ontology Based Modeling Integrating Descriptive, Functional and Cooperative Aspects in a Domain Ontology to Computerize Home Care Charts N. Bricon-Souf, S. Ngom and S. Hamek

746

The ICNP-BaT – A Multilingual Web-Based Tool to Support the Collaborative Translation of the International Classification for Nursing Practice (ICNP) Ulrich Schrader, Peter Tackenberg, Rudolf Widmer, Lucien Portenier and Peter König

751

Ontology Based Modeling of Pandemic Simulation Scenarios Henrik Eriksson, Magnus Morin, Johan Jenvald, Elin Gursky, Einar Holm and Toomas Timpka

755

An Ontology-Based Model of Clinical Information Thomas Beale and Sam Heard

760

Terminologies Creation of a Local Interface Terminology to SNOMED CT Alejandro Lopez Osornio, Daniel Luna, Maria Laura Gambarte, Adrian Gomez, Guillermo Reynoso and Fernán González Bernaldo de Quirós

765

A Feasibility Study on Clinical Templates for the National Health Service in Scotland Derek Hoy, Nicholas R. Hardiker, Ian T. McNicoll and Phil Westwell

770

Keeping Up with Changing Source System Terms in a Local Health Information Infrastructure: Running to Stand Still Daniel J. Vreeman The Role of Local Terminologies in Electronic Health Records. The HEGP Experience Christel Daniel-Le Bozec, Olivier Steichen, Thierry Dart and Marie-Christine Jaulent

775 780

Representing Biomedical Knowledge: Ontologies Ontology-Based Knowledge Base Model Construction-OntoKBCF Xia Jing, Stephen Kay, Nicholas Hardiker and Tom Marley

785

Integrating Biological Pathways in Disease Ontologies Julie Chabalier, Jean Mosser and Anita Burgun

791

Reconciliation of Ontology and Terminology to Cope with Linguistics Robert H. Baud, Werner Ceusters, Patrick Ruch, Anne-Marie Rassinoux, Christian Lovis and Antoine Geissbühler

796

SNOMED CT’s Problem List: Ontologists’ and Logicians’ Therapy Suggestions Stefan Schulz, Boontawee Suntisrivaraporn and Franz Baader

802

Automatic Checking of the Correctness of Clinical Guidelines in GLARE Paolo Terenziani, Luca Anselma, Alessio Bottrighi, Laura Giordano and Stefania Montani

807

Knowledge Zone: A Public Repository of Peer-Reviewed Biomedical Ontologies Kaustubh Supekar, Daniel Rubin, Natasha Noy and Mark Musen

812

PICO Linguist and BabelMeSH: Development and Partial Evaluation of Evidence-Based Multilanguage Search Tools for MEDLINE/PubMed Paul Fontelo, Fang Liu, Sergio Leon, Abrahamane Anne and Michael Ackerman Lessons Learned from Cross-Validating Alignments Between Large Anatomical Ontologies Songmao Zhang and Olivier Bodenreider

817 822

Subject Index

I

Author Index

IX

xiv

Part 2 Chapter 5. Decision Support and Workflow

827

Decision Support at the Point of Care: Clinical Guidelines How Updating Textual Clinical Practice Guidelines Impacts Clinical Decision Support Systems: A Case Study with Bladder Cancer Management Jacques Bouaud, Brigitte Séroussi, Ambre Brizon, Thibault Culty, France Mentré and Vincent Ravery Improving Compliance to Guidelines Through Workflow Technology: Implementation and Results in a Stroke Unit Silvia Panzarasa, Silvana Quaglini, Giuseppe Micieli, Simona Marcheselli, Mauro Pessina, Corrado Pernice, Anna Cavallini and Mario Stefanelli

829

834

Towards a Decision Support System for Optimising Clinical Pathways of Elderly Patients in an Emergency Department Marc Cuggia, Delphine Rossille, Aude Arnault, Jacques Bouget and Pierre Le Beux

840

Ontology-Based Modeling of Clinical Practice Guidelines: A Clinical Decision Support System for Breast Cancer Follow-Up Interventions at Primary Care Settings Samina R. Abidi, Syed S.R. Abidi, Sajjad Hussain and Mike Shepherd

845

Learning Causal and Predictive Clinical Practice Guidelines from Data Subramani Mani, Constantin Aliferis, Shanthi Krishnaswami and Theodore Kotchen

850

Supporting Therapy Selection in Computerized Clinical Guidelines by Means of Decision Theory Stefania Montani, Paolo Terenziani and Alessio Bottrighi

855

Development, Deployment and Usability of a Point-of-Care Decision Support System for Chronic Disease Management Using the Recently-Approved HL7 Decision Support Service Standard David F. Lobach, Kensaku Kawamoto, Kevin J. Anstrom, Michael L. Russell, Peter Woods and Dwight Smith

861

The TAR Model: Use of Therapeutic State Transitions for Quality Assurance Reporting in Chronic Disease Management R. Gaikwad, J. Warren and T. Kenealy

866

Decision Support in Health Information Systems Development of Case-Based Medication Alerting and Recommender System: A New Approach to Prevention for Medication Error Kengo Miyo, Yuki S. Nittami, Yoichiro Kitagawa and Kazuhiko Ohe

871

Is the Future Evidence-Based? Ben W.I. Sissons, W. Alex Gray, Tony Bater and Dave Morrey

875

Comparing Decision Support Methodologies for Identifying Asthma Exacerbations Judith W. Dexheimer, Laura E Brown, Jeffrey Leegon and Dominik Aronsky

880

Decision Support at the Point of Care: Enhancing Patient Safety Analysis and Redesign of a Knowledge Database for a Drug-Drug Interactions Alert System Daniel Luna, Victoria Otero, Daniela Canosa, Sergio Montenegro, Paula Otero and Fernan Gonzalez Bernaldo de Quirós

885

Closing the Loop: Bringing Decision Support Clinical Data at the Clinician Desktop Claudine Bréant, Francois Borst, René Nkoulou, Olivier Irion and Antoine Geissbuhler

890

Are Clinicians’ Information Needs and Decision Support Affected by Different Models of Care? Experimental Study Vitali Sintchenko, Tim Usherwood and Enrico Coiera Modeling and Acquisition of Drug-Drug Interaction Knowledge Frédéric Mille, Patrice Degoulet and Marie-Christine Jaulent

xv

895 900

Activity Modeling A Systems Development Life Cycle Approach to Patient Journey Modeling Projects Joanne M. Curry, Carolyn McGregor and Sally Tracy

905

The Nurse—Patient Trajectory Framework Gregory L. Alexander

910

System Analysis and Improvement in the Process of Transplant Patient Care Catherine J. Staes, R. Scott Evans, Scott P. Narus, Stanley M. Huff and John B. Sorensen

915

St Elsewhere’s or St Everywhere’s: Improving Patient Throughput in the Private Hospital Sector Jennifer A. Laffey and Moran Wasson

920

Clinical Guidelines and Protocols A Meta Schema for Evidence Information in Clinical Practice Guidelines as a Basis for Decision-Making Katharina Kaiser, Patrick Martini, Silvia Miksch and Alime Öztürk

925

Creating Interoperable Guidelines: Requirements of Vocabulary Standards in Immunization Decision Support 930 Karen M. Hrabak, James R. Campbell, Samson W. Tu, Robert McClure and Robert (Tony) Weida Automatic Treatment of Temporal Issues in Clinical Guidelines in the GLARE System Luca Anselma, Paolo Terenziani, Stefania Montani and Alessio Bottrighi

935

Chapter 6. Improving Quality

941

Optimal Order Entry Information and Communication Processes in the Microbiology Laboratory – Implications for Computerised Provider Order Entry Andrew Georgiou, Joanne Callen, Johanna Westbrook, Mirela Prgomet and George Toouli

943

Using an Accident Model to Design Safe Electronic Medication Management Systems Farah Magrabi, Geoff McDonnell, Johanna I. Westbrook and Enrico Coiera

948

Securing Chemotherapies: Fabrication, Prescription, Administration and Complete Traceability Stéphane Spahni, Christian Lovis, Monique Ackermann, Nicolas Mach, Pascal Bonnabry and Antoine Geissbuhler

953

Multitasking by Clinicians in the Context of CPOE and CIS Use Sarah Collins, Leanne Currie, Vimla Patel, Suzanne Bakken and James J. Cimino

958

Supporting the Care of Patients with Cancer Diffusion of Electronic Health Records – Six Years of Empirical Data Christian Nøhr, Stig Kjær Andersen, Knut Bernstein, Morten Bruun-Rasmussen and Søren Vingtoft

963

Text Categorization Models for Identifying Unproven Cancer Treatments on the Web Yin Aphinyanaphongs and Constantin Aliferis

968

Currency of Online Breast Cancer Information Funda Meric-Bernstam, Muhammad Walji, Smitha Sagaram, Deepak Sagaram and Elmer Bernstam

973

Children’s Contributions to Designing a Communication Tool for Children with Cancer Cornelia M. Ruland, Laura Slaughter, Justin Starren, Torun M. Vatne and Elisabeth Y. Moe

977

Extending Care “It’s Your Game”: An Innovative Multimedia Virtual World to Prevent HIV/STI and Pregnancy in Middle School Youth Ross Shegog, Christine Markham, Melissa Peskin, Monica Dancel, Charlie Coton and Susan Tortolero HeartCareII: Home Care Support for Patients with Chronic Cardiac Disease Patricia Flatley Brennan, Gail Casper, Susan Kossman and Laura Burke

xvi

983 988

A Web-Based Communities of Practice Support System for Caregivers Shirley L. Fenton, H. Dominic Covvey, Douglas W. Mulholland, Donald D. Cowan, Judith Shamian and Bonnie Schroeder

993

Core Features of a Parent-Controlled Pediatric Medical Home Record Roberto A. Rocha, Alfred N. Romeo and Chuck Norlin

997

Improving Quality Empowering Patients to Improve the Quality of Their Care: Design and Implementation of a Shared Health Maintenance Module in a US Integrated Healthcare Delivery Network Eric G. Poon, Jonathan Wald, Jeffrey L. Schnipper, Richard Grant, Tejal K. Gandhi, Lynn A. Volk, Amy Bloom, Deborah H. Williams, Kate Gardner, Marianna Epstein, Lisa Nelson, Alex Businger, Qi Li, David W. Bates and Blackford Middleton Clinical Communication Ontology for Medical Errors Yang Gong, Min Zhu, Jun Li, James P. Turley and Jiajie Zhang u-SHARE: Web-Based Decision Support / Risk Communication Tool for Healthcare Consumers with Unruptured Intracranial Aneurysms N. Aoki, M. Sakai, T. Nakayama, S. Fukuhara, S. Ohta, N. Kikuchi, M. Oishi, T. Kiuchi, K. Nozaki and N. Hashimoto Health On the Net Foundation: Assessing the Quality of Health Web Pages All Over the World Célia Boyer, Arnaud Gaudinat, Vincent Baujard and Antoine Geissbühler

1002

1007

1012

1017

Accurate Drug Prescribing The Use of Electronic Medication Reconciliation to Establish the Predictors of Validity of Computerized Medication Records Alexander Turchin, Tejal K. Gandhi, Christopher M. Coley, Maria Shubina and Carol Broverman

1022

Evaluation of an Electronic Medication Reconciliation System in Inpatient Setting in an Acute Care Hospital Abha Agrawal, Winfred Wub and Israel Khachewatsky

1027

Computerized Management of Chronic Anticoagulation: Three Years of Experience Beatriz H. Rocha, Laura H. Langford and Steven Towner

1032

Physicians’ Response to Guided Geriatric Dosing: Initial Results from a Randomized Trial Josh F. Peterson, Benjamin P. Rosenbaum, Lemuel R. Waitman, Ralf Habermann, James Powers, Debbie Harrell and Randolph A. Miller

1037

Chapter 7. Usability

1041

Optimising User Interfaces Graphical Overview and Navigation of Electronic Health Records in a Prototyping Environment Using Google Earth and openEHR Archetypes Erik Sundvall, Mikael Nyström, Mattias Forss, Rong Chen, Håkan Petersson and Hans Åhlfeldt Which Parts of a Clinical Process EPR Needs Special Configuration Anders Barlach and Jesper Simonsen User Driven, Evidence Based Experimental Design: A New Method for Interface Design Used to Develop an Interface for Clinical Overview of Patient Records Niels Boye, Frey Eberholst, Richard Farlie, Lene B. Sørensen and Karen Marie Lyng User Interface Optimization for an Electronic Medical Record System Kai Zheng, Rema Padman and Michael P. Johnson

1043 1048

1053 1058

Cognitive Systems AdaRTE: Adaptable Dialogue Architecture and Runtime Engine. A New Architecture for Health-Care Dialogue Systems L.M. Rojas-Barahona and T. Giorgino

xvii

1063

Multi-Channel Physiological Sensing of Human Emotion: Insights into Emotion-Aware Computing Using Affective Protocols, Avatars and Emotion Specifications Panagiotis D. Bamidis, Andrej Luneski, Ana Vivas, Christos Papadelis, Nicos Maglaveras and Costas Pappas

1068

A Framework for Cognitive Monitoring Using Computer Game Interactions Holly B. Jimison, Misha Pavel, Payton Bissell and James McKanna

1073

Mobile Phone Computing for In-situ Cognitive Behavioral Therapy Magnus Bang, Toomas Timpka, Henrik Eriksson, Einar Holm and Conny Nordin

1078

User Perception and Acceptance Methods for Measuring the Impact of Health Information Technologies on Clinicians’ Patterns of Work and Communication Johanna I. Westbrook, Amanda Ampt, Margaret Williamson, Ken Nguyen and Leanne Kearney

1083

Enhancing User Acceptance of Mandated Mobile Health Information Systems: The ePOC (electronic Point-Of-Care Project) Experience Lois Burgess and Jason Sargent

1088

When Usage and User Satisfaction Differ: The Case of an Electronic Discharge Summary Thomas Bürkle and Philip A. Engel

1093

Mapping Clinicians’ Perceptions About Computerized Protocol Use to an IT Implementation Framework Shobha Phansalkar, Katherine A. Sward, Charlene R. Weir and Alan H. Morris

1098

Usability e-Health in Scotland: Setting a Baseline for Stakeholder Alignment Sharon Levy, Anne Casey and Alison Wallis

1102

Usability of Institutional Cancer Web Sites: An Italian Case Study M. Cristina Mazzoleni, Raffaella Butera, Franco Corbella, Vittoria Balcet and Enrico Masenga

1106

Investigating Internet Use by Mental Health Service Users: Interview Study John Powell and Aileen Clarke

1112

Text Characteristics of Clinical Reports and Their Implications for the Readability of Personal Health Records Qing Zeng-Treitler, Hyeoneui Kim, Sergey Goryachev, Alla Keselman, Laura Slaughter and Catherine Arnott Smith

1117

Advances in User Interface Generic Screen Representations for Future Proof Systems – Is It Possible? Two-Model Approach to a Generic GUI Helma van der Linden, Thilo Schuler, Rong Chen and Jan Talmon

1122

Speech Recognition in Dental Software Systems: Features and Functionality Jeannie Yuhaniak Irwin, Shawn Fernando, Titus Schleyer and Heiko Spallek

1127

Assessing the Impact of Recording Quality Target Data on the GP Consultation Using Multi-Channel Video Maigaelle V. Moulene, Simon de Lusignan, George Freeman, Jeremy van Vlymen, Ian Sheeler, Andrew Singleton and Pushpa Kumarapeli

1132

Chapter 8. Sustainability

1137

Sustainability: Government Policy and Initiatives Sustainable Health Systems: Addressing Three Key Areas Prajesh N. Chhanabhai, Alec Holt and George Benwell

1139

Locating Nursing Classification Schemes Within Health Information Strategies for New Zealand Shona K. Wilson and Jan Duke

1144

xviii

SHARE, from Vision to Road Map: Technical Steps Mark Olive, Hanene Rahmouni, Tony Solomonides, Vincent Breton, Yannick Legré, Ignacio Blanquer and Vicente Hernandez Penetration and Adoption of Health Information Technology (IT) in Thailand’ s Community Health Centers (CHCs): A National Survey Boonchai Kijsanayotin, Supasit Pannarunothai and Stuart Speedie

1149

1154

Sustainability: Disaster Planning and Public Health The Health Informatics Center of Acadiana – Informing Health Policymaking in Post-Katrina/Rita Louisiana 1159 L. Philip Caillouet Informatics Solutions for Emergency Planning and Response Elizabeth E. Weiner and Patricia A. Trangenstein

1164

A Multidiscipline Conceptual Framework for Consumer Health Informatics Robert A. Logan and Tony Tse

1169

Sustainable Health Systems A Japanese Model of Disease Management Naoki Nakashima, Kunihisa Kobayashi, Toyoshi Inoguchi, Daisuke Nishida, Naomi Tanaka, Hiromi Nakazono, Akihiko Hoshino, Hidehisa Soejima, Ryoichi Takayanagi and Hajime Nawata

1174

Towards Sustainability of Health Information Systems: How Can We Define, Measure and Achieve It? Sebastian Garde, Carola M. Hullin, Rong Chen, Thilo Schuler, Jana Gränz, Petra Knaup and Evelyn J.S. Hovenga

1179

Bermuda Triangle or Three to Tango: Generation Y, e-Health and Knowledge Management Kwang Chien Yee

1184

Health Service Organisations and Professionals: An Information Systems Model for Transforming the Nexus Between Accreditation and Practice Jo-Anne Kelder Health Informatics: An Intercultural Perspective Quynh Lê

1189 1194

Sustainable Systems: Design Success Successful Systems Sustaining Change Sheila Bullas and John Bryant

1199

MUST – A Participatory Method for Designing Sustainable Health IT Finn Kensing, Hrönn Sigurdardottir and Arjen Stoop

1204

Change Management and the Sustainability of Health ICT Projects Karen Day and Tony Norris

1209

A Sustainability View on the EPR System of N.N. Burdenko Neurosurgical Institute Michael A. Shifrin, Elena E. Kalinina and Eugene D. Kalinin

1214

Chapter 9. Genomics

1217

Representing Biomedical Knowledge: Ontologies and Genomic Data Repositories Predicting Coronary Artery Disease with Medical Profile and Gene Polymorphisms Data Qiongyu Chen, Guoliang Li, Tze-Yun Leong and Chew-Kiat Heng

1219

Towards a Top-Domain Ontology for Linking Biomedical Ontologies Holger Stenzhorn, Elena Beißwanger and Stefan Schulz

1225

The Molecular Medicine Informatics Model (MMIM) Marienne Hibbert, Peter Gibbs, Terence O’Brien, Peter Colman, Robert Merriel, Naomi Rafael and Michael Georgeff

1230

xix

Cancer Genomics Object Model: An Object Model for Multiple Functional Genomics Data for Cancer Research Yu Rang Park, Hye Won Lee, Sung Bum Cho and Ju Han Kim

1235

Algorithms for Genomic and Genetic Data Analysis Automatic Pedigree Reconstruction for Genetic Studies in Isolated Populations C. Larizza, I. Buetti, G. Milani, A. Nuzzo, C. Sala, D. Toniolo and R. Bellazzi

1240

Enhancing the Quality of Phylogenetic Analysis Using Fuzzy Hidden Markov Model Alignments Chrysa Collyda, Sotiris Diplaris, Pericles Mitkas, Nicos Maglaveras and Costas Pappas

1245

Determining Transcription Factor Activity from Microarray Data Using Bayesian Markov Chain Monte Carlo Sampling Andrew V. Kossenkov, Aidan J. Peterson and Michael F. Ochs Ensemble Stump Classifiers and Gene Expression Signatures in Lung Cancer Lewis Frey, Mary Edgerton, Douglas Fisher and Shawn Levy

1250 1255

Algorithms and Research Frameworks for Genomics From “Glycosyltransferase” to “Congenital Muscular Dystrophy”: Integrating Knowledge from NCBI Entrez Gene and the Gene Ontology Satya S. Sahoo, Kelly Zeng, Olivier Bodenreider and Amit Sheth

1260

Building a Research Model for Human Genetic Variation Knowledge Management Yulong Gu and James Warren

1265

ECTracker – An Efficient Algorithm for Haplotype Analysis and Classification Li Lin, Limsoon Wong, Tze-Yun Leong and Pohsan Lai

1270

A Dynamic Query System for Supporting Phenotype Mining in Genetic Studies Angelo Nuzzo, Daniele Segagni, Giuseppe Milani, Carla Rognoni and Riccardo Bellazzi

1275

Chapter 10. Biomedical Image and Signal Processing

1281

Biomedical Signal Interpretation Identifying QT Prolongation from ECG Impressions Using Natural Language Processing and Negation Detection Joshua C. Denny and Josh F. Peterson

1283

A Comparison of Supervised Classification Methods for Auditory Brainstem Response Determination Paul McCullagh, Haiying Wang, Huiru Zheng, Gaye Lightbody and Gerry McAllister

1289

Non-Linear Analysis for the Sleepy Drivers Problem Ioanna Chouvarda, Christos Papadelis, Chrysoula Kourtidou-Papadeli, Panagiotis D. Bamidis, Dimitris Koufogiannis, Evaggelos Bekiaris and Nikos Maglaveras

1294

Identification and Genotype Related Classification of Children with Long QT-Syndrome Using 24h Holter Recordings Matthias Bauch, Annamaria Siegler, Markus Khalil, Jörg Zehelein, Herbert E. Ulmer and Hartmut Dickhaus

1299

Information Visualisation and Imaging Informatics Temporal Abstraction and Data Mining with Visualization of Laboratory Data Katsuhiko Takabayashi, Tu Bao Ho, Hideto Yokoi, Trong Dung Nguyen, Saori Kawasaki, Si Quang Le, Takahiro Suzuki and Osamu Yokosuka Automated Interpretation of Optic Nerve Images: A Data Mining Framework for Glaucoma Diagnostic Support Syed S.R. Abidi, Paul H. Artes, Sanjan Yun and Jin Yu Intelligent Querying and Exploration of Multiple Time-Oriented Medical Records Denis Klimov and Yuval Shahar

xx

1304

1309 1314

Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks for Image Retrieval Evaluation Henning Müller, Célia Boyer, Arnaud Gaudinat, William Hersh and Antoine Geissbuhler

1319

Biomedical Image Processing Improving Computer Aided Disease Detection Using Knowledge of Disease Appearance Tatjana Zrimec and James S. Wong

1324

MR Atlas for Articular Cartilage Morphology: Potential to Detect Shape Differences Hussain Z. Tameem and Usha S. Sinha

1329

Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval Jayashree Kalpathy-Cramer and William Hersh

1334

Quantification of Myocardial Perfusion for CAD Diagnosis Hartmut Dickhaus, Markus Erbacher and Helmut Kücherer

1339

Chapter 11. Education and Training

1345

Health Informatics Education and Training Enabling the Safe and Effective Implementation of Health Informatics Systems – Validating and Rolling Out the ECDL/ICDL Health Supplement Michael J. Rigby, Carol Hulm, Don Detmer and Luca Buccoliero A Multi-Method Approach to Assessing Health Information Systems End Users’ Training Needs Yiyu Qiu, Ping Yu and Peter Hyland Making Health Informatics Competencies Useful: An Applied Health Informatics Competency Self-Assessment System H.D. Covvey, S. Fenton, D. Mulholland and K. Young

1347 1352

1357

eLearning and Distance Education E-learning for Students in Their First Year: A French Experimentation at the Medical School of Grenoble Jean-Marie Renard, Daniel Pagonis, Jean-Philippe Vuillez, Jean-Paul Romanet and Bernard Sele

1362

E-learning at Porto Faculty of Medicine. A Case Study for the Subject ‘Introduction to Medicine’ Cristina Costa-Santos, Ana Coutinho, Ricardo Cruz-Correia, Ana Ferreira and Altamiro Costa-Pereira

1366

Designing m-Learning for Junior Registrars – Activation of a Theoretical Model of Clinical Knowledge Anne Marie Kanstrup, Niels Boye and Christian Nøhr

1372

Evaluation and Assessment of the Online Postgraduate Critical Care Nursing Course Reena Patel

1377

Professional Education – Nursing Informatics Development and Evaluation of a PDA-Based Decision Support System for Pediatric Depression Screening Ritamarie John, Penélope Buschman, Mosely Chaszar, Judy Honig, Eneida Mendonca and Suzanne Bakken

1382

Data Mining Results from an Electronic Clinical Log for Nurse Practitioner Students Patricia Trangenstein, Elizabeth Weiner, Jeffry Gordon and Ryan McNew

1387

Phase I Implementation of an Academic Medical Record for Integrating Information Management Competencies into a Nursing Curriculum Carole A. Gassert and Katherine A. Sward ROC van Twente: Nursing Education in Care and Technology William T.F. Goossen, Anneke T.M. Goossen-Baremans, Laura Hofte and Bert de Krey

xxi

1392 1396

Professional Education – Research Multiple Measures of Provider Participation in Internet Delivered Interventions Thomas K. Houston, Ellen Funkhouser, Jeroan J. Allison, Deborah A. Levine, O. Dale Williams and Catarina I. Kiefe

1401

Medical Students’ Knowledge and Perceptions of e-Health: Results of a Study in Sri Lanka Sisira Edirippulige, Rohana B. Marasinghe, Anthony C. Smith, Yoshikazu Fujisawa, Walisundara B. Herath, M.T.M. Jiffry and Richard Wootton

1406

Importance of Public Health Informatics: A Survey of Public Health Schools and Graduate Programs in the United States Janise Richards Educating Medical Students as Competent Users of Health Information Technologies: The MSOP Data Julie J. McGowan, Morgan Passiment and Helene M. Hoffman

1410 1414

Professional Education – Case Reports and Methods Studies Establishing a National Resource: A Health Informatics Collection To Maintain the Legacy of Health Informatics Development Beverley Ellis, Jean Roberts and Helen Cooper The EIPEN Project: Promoting Interprofessional Education in Health Professions Joseph Liaskos, Antonis Frigas, Kostantinos Antypas, Dimitrios Zikos, Marianna Diomidous and John Mantas Building ICT Capabilities for Clinical Work in a Sustainable Healthcare System: Approaches to Bridging the Higher Education Learning and Teaching Gap Kathleen Gray and Jenny Sim The Development of an Online Clinical Log for Advanced Practice Nursing Students: A Case Study Jeffry S. Gordon, Ryan McNew and Patricia Trangenstein

1419 1424

1428 1432

Computer-Assisted Medical Education Personalized Case Driven Parental Education Informatics in the NICU John Chuo, Pavel Sherman, Claire Drain and Casimir Kulikowski Conceptual Model of Health Information Ethics as a Basis for Computer-Based Instructions for Electronic Patient Record Systems Mihoko Okada, Kazuko Yamamoto and Kayo Watanabe PDA-Based Informatics Strategies for Tobacco Use Screening and Smoking Cessation Management: A Case Study Suzanne Bakken, W. Dan Roberts, Elizabeth Chen, Joann Dilone, Nam-Ju Lee, Eneida Mendonca and Marianthi Markatou

Chapter 12. Poster Contributions Selected for Best Poster Awards 2007

1437

1442

1447

1453

Consumer Informatics Needs Assessment for the Computer-Interpretable Hypertension Guideline at Public Health Centers in Korea 1455 EunJung Lee, SoYoung Kim, InSook Cho, JiHyun Kim, JaeHo Lee and Yoon Kim Education Development of a Personal Medical Recorder on a Cell Phone Akihiro Takeuchi, Katsura Kobayashi, Noritaka Mamorita and Noriaki Ikeda

1456

Emerging Technologies Implementing and Evaluating a Laboratory Information System to Optimize the Treatment of Tuberculosis Patients in Peru Joaquin A. Blaya, Sonya S. Shin, Martin J.A. Yagui, Luis Asencios, Javier Vargas, Carmen Suares, Gloria Yale and Hamish S.F. Fraser

xxii

1457

Health Information Systems Risk Analysis – A Tool for IT Development and Patient Safety A Comparative Study of Weaknesses Before and After Implementation of a Health Care System in the County Council of Ostergotland, Sweden Annica Öhrn and Gunilla Eriksson

1458

Multi-Label Text Classification of German Language Medical Documents Stephan Spat, Bruno Cadonna, Ivo Rakovac, Christian Gütl, Hubert Leitner, Günther Stark and Peter Beck

1460

What Health Influences Are Caused by EMR Working? – In Case of Japanese Nursing Situation Yukie Majima and Yasuko Maekawa

1462

Using PDA to Transform the Long MDS-HC Evaluation Form into a Favored System Chiao-Ling Hsu, Yu-Yin Kuo and Polun Chang

1463

A Sustainable, Multi-Organizational Model for Decision Support During Public Health Emergencies Julie J. McGowan, Margaret W. Richwine and J. Marc Overhage

1465

Customized Early Warning System Based on HTN for Home Healthcare Model Seung-Jin Jang, Jip-Min Jung, Sung-Oh Hwang and Young-Ro Yoon

1467

Open Source Patient Data Management System for Intensive Care J. Massaut, P. Reper, L. Hooghe and P. Gottignies

1468

Why Teach Computer Security to Medical Students? Ana M. Ferreira, Ricardo Cruz-Correia and Altamiro Costa-Pereira

1469

Application of Wireless and Mobile Computing Technologies to Improve the Efficiency of Patient Care and Education: The Role of Medical Engineering and Information Technology Lin Guo

1471

Knowledge Management Comparing Messages in an Online Communication Forum for Cancer Patients with Patients’ Messages to a Clinical Nurse Specialist Annette Jeneson, Trine Andersen and Cornelia Ruland

1473

Guideline-Based Visualization of Medication in Chronic Disease Ján Stanek and Michelle Joy Davy

1475

Efficiency and Safety of New Radiofrequency Identification System in Japanese Hospital Yuichiro Saito, Takashi Hasegawa and Tetsuo Sakamaki

1478

Development of Hypertension Management Ontology for Guideline-Based Clinical Decision Support System JiHyun Kim, InSook Cho, EunJung Lee, JaeHo Lee and Yoon Kim

1479

Secure Remote Access for Web Based Clinical Information System Using Policy Control of PCs and Healthcare PKI Authentication Katsuya Tanaka, Mayumi Yoshida and Ryuichi Yamamoto Infobuttons: A Study of Usability Lily Gutnik, Sarah Collins, Leanne M. Currie, James J. Cimino and Vimla L. Patel

1480 1481

Organisations The Application of a Clinical Data Warehouse to the Assessment of Drug-Warfarin Interactions Qiyan Zhang, Yasushi Matusmura and Hiroshi Takeda

1482

Standards Implementation of an Integrated Network for Health Research in Quebec Allen Huang, Jacques Lemieux, Jean-Claude Bouchard, Michel Bourque and Robyn Tamblyn

xxiii

1483

OpenECG: Promoting Interoperability Through the Consistent Implementation of the SCP-ECG Standard in Electrocardiography Catherine Chronaki, Franco Chiarugi and Ronald Fischer

1484

Subject Index

1485

Author Index

1493

xxiv

EDITORIAL Playing an active role in the 12th World Congress on Medical Informatics has been an honour and a privilege for the Editorial Committee. We closely cooperated with our colleagues and friends of the Scientific Program Committee (SPC) and the Organizing Committee (OC). Out of 638 submissions, 292 high quality papers were selected for publication in these proceedings by the SPC. Out of 218 poster submissions, 21 posters were nominated for a best poster award; they are included as 1 page contributions in these proceedings. While high quality contributions have been selected by the SPC, the Editorial Committee has focussed on providing proceedings with a high quality of format and presentation which made additional editing necessary. The proceedings are also appearing on CD-ROM. All accepted poster contributions are published on an additional CD-ROM. The MEDINFO 2007 Proceedings present an excellent overview of a dynamic and quickly growing field, demonstrating methodical and practical progress from around the world. Information science and, specifically, (Bio-) Medical Informatics have become core pillars of foundational and clinical research, of medical care, and of prevention. MEDINFO 2007 in Brisbane will be the leading conference of the year, bringing a worldwide community together. We have organized the proceedings into twelve chapters, covering topics such as eHealth, Decision Support, Improving Quality, Usability, Sustainability, Genomics, Biomedical Image and Signal Processing, and Education and Training. Within each chapter, the articles are organized according to the conference sessions; the session titles are shown in the table of contents. The assistance of HISA has been invaluable in all editorial steps, including communication with authors, language editing, and formatting of manuscripts. We owe specific thanks to Joan Edgecumbe, Dale Proposch and Tom Morgan.

The Editorial Committee Klaus A. Kuhn, Germany James R. Warren, New Zealand Tze Yun Leong, Singapore

xxv

Disclaimer While the Medinfo 2007 Editorial Committee has made every effort to ensure that all accepted contributions are published in these two volumes of the MEDINFO 2007 Proceedings, it reserves the right to: o Edit/alter one or more parts of an original contribution, including its title, author and affiliation listings, as it sees fit in order to comply with publications requirements. o

Withhold the publication of a contribution due to one or more of the following circumstances: • failure to meet the final deadline for submission of all original/revised contributions; • failure to revise the original contribution in accordance with the instructions of the Scientific Programme Committee, including the advice of the Editorial Committee.

The Editorial Committee is not responsible for the alteration and/or omission of part or parts of an original contribution in cases where a contribution has not fully complied with the submission guidelines and has to be partially or fully re-written and/or reformatted for the final publication.

xxvi

Preface from the Scientific Program Co-Chairs The theme of Medinfo2007 is “Building Sustainable Health Systems”. Particular foci are health challenges for the developing and developed world, the social and political context of health care, the knowledge that is needed for safe and effective health care, and the difficult task of building and maintaining complex health information systems. Sustainable health information systems are those that can meet today’s needs without compromising the needs of future generations. It is a challenge and an opportunity to develop and implement systems that can be improved – not replaced. To achieve this and to build a global knowledge society, we need to seek increased scientific and technological cooperation, and we need to facilitate access to and use of high-quality knowledge and information. We are pleased to report that more than 900 submissions were made to Medinfo2007. This includes full papers, panels, posters, and workshops. Submissions were made from every region of the world and from many countries, including, among others, Argentina, Australia, Belgium, Brazil, Canada, China, Finland, France, Germany, Great Britain, India, Israel, Italy, Japan, New Zealand, Portugal, Russia, Singapore, Switzerland, and the United States, making this a truly international conference. The majority of submissions (638) were made in the full paper category. Papers were refereed by members of the biomedical informatics community and final decisions were made by the SPC members at a three day meeting in Chamonix, France in late January 2007. Of the 638 papers submitted, 292 were accepted for presentation and publication (a 46% acceptance rate). 260 were not accepted for publication, but their authors were offered the possibility of presenting their work in the Medinfo2007 poster session, and they were also given the opportunity to create a set of slides that is planned to be part of a continuous slide show throughout the meeting. 86 paper submissions were rejected (13%). The contributions to Medinfo2007 reflect the breadth and depth of the field of biomedical and health informatics. Papers cover topics in health information systems, knowledge and data management, education, standards, consumer health and human factors, emerging technologies, sustainability, organizational and economic issues, genomics, and image and signal processing. To recognize the truly outstanding contributions to the conference, there will be “Best Paper” and “Best Poster” awards at the meeting. A student paper competition will also be held. All decisions will be made by an international jury at the meeting itself, and the winners will be announced during the closing session of the Congress. On behalf of the members of the Medinfo2007 program committee (listed below), we wish to thank all those who contributed to Medinfo2007 by sending in their paper, poster, panel, and workshop contributions. We give special thanks to all those who carefully reviewed the many excellent submissions. We expect that the papers included in this volume will be of great interest to anyone engaged in biomedical and health informatics research and application. Alexa T. McCray, PhD Harvard Medical School Boston, Massachusetts, USA

Heimar Marin, RN, PhD Federal University of São Paulo São Paulo, Brazil

xxvii

Medinfo 2007 Scientific Program Committee SPC Chairs Alexa McCray (Co-Chair, US) Heimar Marin (Co-Chair, Brazil) SPC Members Europe Elske Ammenwerth (Austria) Ricardo Bellazi (Italy) Jan van Bemmel (Netherlands) Patrice Degoulet (France) Nicos Maglaveras (Greece) George Mihalas (Romania) Americas Dominik Aronsky (US) Marion Ball (US) Alex Jadad (Canada) Alvaro Margolis (Uruguay) Charles Safran (US) Asia-Pacific John Bennett (Tutorial chair, Australia) HM Goh (Malaysia) Kazuhiko Ohe (Japan) Hyeoun-Ae Park (Korea) Africa Ben Bondo Monga (Congo) Middle East Yuval Shahar (Israel) Ahmed Al Barrak (Saudi Arabia)

xxviii

In memory of

Dr Branko Cesnik Australia

It is with sadness that as we complete this foreword we acknowledge the passing of Branko Cesnik Melbourne on Sunday, 10, June, 2007 after a courageous battle with cancer. Branko was one of the founders of the Health Informatics Society of Australia, the Asia Pacific Association of Medical Informatics and the Australian College of Health Informatics. Branko was also a member of the Board of the International Medical Informatics Association (IMIA) as the Vice-President of Membership from 1999 to 2003. He was the lead editor of the Medinfo '98: 9th World Congress Medical Informatics Proceedings (Korea). In addition he served IMIA through a variety of other responsibilities, such as being a member of a Medinfo Scientific Program Committee. Those who knew him as a colleague will remember his knowledge, level-headedness, spirit of cooperation and sharing, and above all getting things done—no matter what was required or the amount of time required. Those who knew him as a friend will remember his panache, his spirit and his zest for living life on the edge and to the fullest. That zest and spirit accompanied him throughout all of the latter days of his struggle. His great leadership and vision were instrumental in laying the foundation for health informatics developments “Down Under” that has this year occasioned MEDINFO 2007 to come to Australia.

This page intentionally left blank

Chapter 1. eHealth

1

This page intentionally left blank

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Linkcare – Enabling Continuity of Care for the Chronically Ill across Levels and Profession Marius Mikalsena, Ståle Walderhauga+b, Per Håkon Melanda, Ole Martin Winnema a

b

SINTEF ICT, Norway Medical Informatics and Telemedicine group, Dept for Computer Science, University of Tromsø, Tromsø, Norway

tems are expected to maintain the continuity of care, shared care, and the empowerment of patients in the management process [3].

Abstract Chronic diseases are increasing rapidly and this phenomenon is becoming a major burden to the health delivery system around the world. A new health care paradigm with focus on chronic treatment and care will actualize the need for interoperable standards based services due to the complexity of care where different health levels and professions are involved. Given the complexity of the domain, we argue the need for a systematic and formal approach to the development of interoperable information systems if there shall be any real support of the cooperating actors. We describe our work on technical interoperability done in the Linkcare project addressing new models of care and technology to support them in the domain of the chronically ill using concrete results from an architecture built using the MAFIIA architecture framework and the UML 2.0 profile for software services, and argue that building formal architectural descriptions on the basis of shared interface descriptions and profiles are an important part of achieving continuity of care based on sustainable health systems.

The new models of care, including increasingly cooperating stakeholders and empowered users, would benefit from interoperable technology to support continuity of care, e.g., shared access to electronic patient records (EPRs). The situation today is that most stakeholders in the care model have their own proprietary systems, with no or limited possibilities for exchanging information with the outside world in a standardized fashion. Development of interoperable healthcare services would make for more efficient work processes and constitute considerable savings. Walker et.al. [4] argue that the value of health care information exchange and interoperability (HIEI) in the United States alone can yield a net value of $77.8 billion per year once fully implemented. There are obstacles to interoperability though. Iakovidis lists reasons such as organizational and cultural matters, technological gap between healthcare professionals and information science experts, and legal requirements as to why integrating services in healthcare is a challenge [5].

Keywords:

This article presents Linkcare1, a European Union project, addressing new models of care for the chronically ill and the technology to support them. Three standalone systems in three different countries (Norway, Cyprus and Spain) supporting different aspects of care of the chronically ill provided the outset for the Linkcare project. This is a relevant scenario as cooperation between different stakeholders using different systems is what characterizes care for the chronically ill.

medical informatics, information systems, systems integration, continuity of patient care

Introduction The number of patients suffering from chronic diseases has increased rapidly over the recent decades. The disease burden is changing from acute to chronic care, 35.000.000 people died from chronic diseases in 2005, and 60% of all deaths are due to chronic diseases [1].

There is a need for a new way of designing information services in healthcare to ensure an interoperability level that supports coordinated management of care.

The new conditions require that we rethink traditional models of care. One prominent aspect of new care models used for chronic conditions is the cooperation of several different stakeholders in the care process [2]. Stakeholders include the patient himself, his relatives, and caregivers.

In this paper we describe our approach to achieve interoperability between the above mentioned systems. We define interoperability as [9]: “A system’s ability to exchange information with other systems and to mutually use the exchanged information”. This ability is fundamental in order to allow stakeholders in the continuity of care process to provide high quality care to the chronically ill. We

Additionally, patients are no longer passive consumers of health services, but are instead demanding more control over their own treatment, together with increased responsiveness and improved quality of care services provided by the involved healthcare institutions. Today, healthcare sys-

1

3

Linkcare project (eTen, Grant Agreement Number C517435)

M. Mikalsen et al. / Linkcare – Enabling Continuity of Care for the Chronically Ill across Levels and Profession

will show how we applied the MAFIIA architecture framework [9] together with the UML 2.0 profile for software services [13] to build an architecture enabling sustainable interoperability between systems in this domain.

architecture is a high-level, generic architecture which is used as the basis for development of concrete system architectures, and to compare architectures of existing systems to each other.

This article is organized as follows. First, in the Methods section, we describe the methods used to build the architecture, namely the MAFIIA architecture framework, the UML 2.0 profile for software services and the Linkcare process. Second, the Results section presents the results from applying the methodology as a set of MAFIIA viewpoints. In the Discussion section we present the experiences gathered in our work, before making our concluding remarks.

UML 2.0 Profile for software services The UML2.0 Profile for Software services is a profile for UML 2.0 which allows for the modeling of services and service oriented architecture solutions. A UML profile extends the expressiveness of UML with domain specific knowledge, in this case software service specification. The use of a service oriented architecture approach in Linkcare followed from the fact that the original system platforms already were build for the web and web services.

Methods

We present a subset of the profile, intended to explain the results and discussion sections. The complete profile is explained by Johnston [13], and the following are relevant excerpts. A Service Partition represents some logical or physical boundary of the system. Service partitions are used to represent the web, business, and date tiers of a traditional n-tier application. Any owned parts of a partition shall be a Service Provider. A Service Provider is a software element that provides one or more services. A service provider has a property that captures information about its location, but the meaning of this is implementation independent. A Service Gateway is only available for use on partitions and not directly on service providers. Gateways are used on partitions to expose services. A Service is the model element that provides the end-point for service interaction (in web-service terminology). A Service Consumer is any component (includes services) that consumes services. A Service Channel represents the communication path between two services. Interaction may occur over a channel, but the channel does not represent any particular interaction. In web services, each service denotes the bindings associated with it, and in the modeling profile, you denote binding either on the communication between services or between a service and a consumer.

This section details the background information relevant to the research presented in this paper. We cover the MAFIIA architecture framework, and the IBM UML 2.0 Profile for Software Services, and explain how these were applied by describing the Linkcare process. MAFIIA The service architecture that is presented in this document is created using MAFIIA/H (Modelbased Architecture description Framework for Information Integration Abstraction/Healthcare) [9]. MAFIIA is an architectural description framework for software intensive systems with special a focus on Information Integration Systems (IIS). An architectural description created using MAFIIA consists of a set of views. Viewpoints are used to create a view. The view consists of one or more models that describe and present different aspects related to structure and behaviour for a target system. Five different viewpoints are defined in MAFIIA. They are; i) The Context Viewpoint which describes all aspects of the Target System’s environment, which is of importance to be able to document all the interfaces between the Target System and its environment, and what the Target System is intended to do in its environment. ii) The Requirements Viewpoint, which document all specific requirements related to the Target System. iii) The Component Viewpoint which identifies and documents specific physical or logical components. iv) The Distribution Viewpoint which describe the logical distribution of software and hardware components. The distribution view shows if some components cannot be separated and if any must be separated. v) The Realisation Viewpoint which describes any constraints on how the target system’s components should be implemented and deployed into its environment.

The Linkcare process The guiding principles of the Linkcare architectural process was the following. First, it was essential to preserve the heritage application, that is the original systems that the Linkcare services were built on, and minimal change should be introduced upon these systems. Second, based on a market survey performed in the project, the services created should meet concrete needs in the market. Third, it should be easy to compose new services compliant to the architecture. We applied the MAFIIA workflow, starting with definition of system concerns and assets, before specifying the five architectural views. First, we used the Linkcare heritage systems as the foundation for capturing the core business processes (functionality) that must be described in the architecture and identified all relevant stakeholders in relation to the Linkcare services (described in the context viewpoint). Second, from the use cases, architectural requirements were defined (requirements model). Third, in the component viewpoint, the resulting system informa-

In addition to the viewpoints, a MAFIIA description includes description of concerns, system assets and reference architectures. Concerns are related to the documentation of the functional aspects of the target system and its environment. A concern is visible and treated in relation to any view. Concerns are related to functionality. System assets are sources of information that can be used when developing the architecture descriptions. The reference

4

M. Mikalsen et al. / Linkcare – Enabling Continuity of Care for the Chronically Ill across Levels and Profession

tion model was described, together with models that show how the Linkcare services are composed, the collaboration of services, and the interfaces of the services. Fourth, in the distribution viewpoint the logical distribution of Linkcare components were modelled, and in the realisation viewpoint we investigated the platforms on which the Linkcare systems were running to identity constraints influencing the realisation of the architecture (such as technical platforms and development processes).

problems the target system solves, or what functionality it implements (e.g. stakeholders). Incorporating the results of the Linkcare process (market analysis and studying the Linkcare systems), we identified 17 core stakeholders modeled in a UML use case model/actor hierarchy accompanied by text outlining each stakeholder. The Environment Systems model documents other technical systems (environment systems) that are involved in the implementation of the business models, or influences the operation of the target system. We identified seven services that constituted the Linkcare services (the target system). These services are modeled as a Service Partition and made available for other services (Service Consumers) through Service Gateways. This was done in order use the gateway construct to control the access to services as the partition in Norway differs from the Spanish partition as different services are provided.

To enable information sharing and version management in the process a CSCW tool named eRoom was used. The modelling was done using Rational Software Architect, and the complete architecture was written in Microsoft Word. Three project meetings, and one two day workshop were used to cooperate and decide on the architecture.

Results – viewpoints of the architecture In the following we introduce core results from the MAFIIA viewpoints modeled using the UML 2.0 profile for software services.

class LinkCare Serv ice Partition

«ServiceProvider» LinkCare Service Registry

consists of Depends on

Publish Service

Concerns

«ServiceConsumer,Busi... LinkCare Service Partition

One of the main concerns was identified to be interoperability as defined by MAFIIA. Another important concern was security, but it is outside the scope of this paper to address this.

1

«ServiceProvider» LinkCare Performance Module

1 1..

0..1

1

1

1

1

0..1

«ServiceProvider» LinkCare Educational Information Access Service

0..1 0..1

«ServiceProvider» LinkCare CSCW Service

System assets Relevant standards, a project dictionary, and software profiles and patterns were identified as important assets. We applied Chari et al’s [8] framework for weighing the scope and usability of a set of selected standards, and found that CEN ENV 13940 (Health Informatics System of Concepts to support continuity of Care) was the most appropriate standard. The dictionary, an agreed upon reference list of concepts, was created based upon scenario descriptions and story boards from the three heritage systems, and, as far as possible, mapped to the terminology of the chosen standard. The profile chosen was the UML 2.0 profile for software services, and web service patterns such as the publish-subscriber, asynchronous query, command façade, and message bus pattern were used to guide the design.

«ServiceProvider» LinkCare Professional Mobile Access Service 0..1

«ServiceProvider» LinkCare Call Centre Service

0..1

«ServiceProvider» LinkCare EHR Service

0..1

«ServiceProvider» LinkCare Patient Mobile Access Service

Figure 1 - The Linkcare service partition Applying the service profile in this way implies that all Linkcare services can be accessed through well-defined interfaces (the gateways). We designed a Linkcare Service Registry that controls the location and availability of the Linkcare services (using the concept of location in the Service provider). This structure enables new Linkcare services to be added, existing services to be modified and relocated without notifying its clients and communication (Service Channels) between Linkcare service partitions can be managed through the gateways.

Reference architecture The MAFIIA reference architecture divides the target system (in our case, the Linkcare services) into a set of logical tiers, and defines how the system interfaces with the environment. A tier is a logical partitioning of a system where each tier has a unique responsibility. Each tier is loosely coupled to the tiers directly above and below. All components in the Linkcare architecture were linked to the reference architecture in the component viewpoint.

We used UML component diagrams to stereotype environment systems either as Service Providers or Service Consumers according to the software services profile. UML Sequence diagrams were used to detail the interaction between the Linkcare services and the environment systems. Requirements viewpoint Having a clear understanding of what were to be solved following the context viewpoint, we gathered requirements for the target system We organized the requirements into generic requirements for all the services, requirements for each service, and generic interoperability requirements.

Context viewpoint The context viewpoint consists of the business aspects model, the environment systems model, and the business to system mapping model. The business aspect models documents any business related concern that increases the understanding of what

5

M. Mikalsen et al. / Linkcare – Enabling Continuity of Care for the Chronically Ill across Levels and Profession

The original Linkcare system instances are kept on separate application servers. This is because these are separate systems, maintained by separate organizations in separate countries. The services will cooperate through the Linkcare services, which will help to keep the core business functionality separated, and avoiding building monolithic systems.

Component viewpoint The component viewpoint consists of the system information model, the system decomposition model, the system collaboration model and the component and interface specification model. The system information model describes the most important information elements in the Linkcare system. As in the dictionary, the elements in this model were mapped to the CEN standard as far as possible. The model was built using a UML class diagram and the elements were additionally explained using text.

Some services, such as the Linkcare patient mobile access service and the Linkcare patient mobile access service are services that typically require rich clients and introduce special requirements on services. Examples of which are extended session logic to handle session interruptions (e.g. when transferring measurement data) and extended security mechanisms.

The system decomposition model describes how the target system is divided into different subsystems and components. The Linkcare services were modeled as being part of a Service Partition consisting of one to many Service Providers (the services). The Service Partition publishes services to the Linkcare Service Registry. Interaction with the service registry was detailed in a UML sequence diagram. A UML component diagram was used to stereotype the components to the MAFIIA reference architecture.

The Linkcare EHR service is also kept separated. This is because sensitive data typically is transferred using this service, and extra security mechanisms (such as firewalls, log on mechanisms etc) is needed. The Linkcare service registry service will receive a higher amount of service requests (service lookup) than other services. In order for this not to influence other services, this service is also kept separate. Realization viewpoint In the realization viewpoint we considered the technical platforms that the original Linkcare systems were running on. The Norwegian and Cypriot system ran on .NET technology, while the Spanish system is based on Oracle solutions. In the realization view we outlined how web services (one way of implementing services) was to be done on the different platforms. We were focusing on, service description, service implementation, service publishing, discovery and binding, and service invocation and execution.

Figure 2 - Mapping to MAFIIA Reference architecture

Discussion

A UML composite structure diagram were used to model the Service Gateways on a partition, and a UML collaboration diagram were used to model the Service Channels on the gateways.

The reason we chose to use MAFIIA in this domain is because it is a proven and documented framework for creating information integration architecture descriptions. An overview of other projects in which MAFIIA is used is provided by Walderhaug et al [9]. In Linkcare we use MAFIIA to ensure that the architectural description created for Linkcare is inline with requirements to architectural descriptions for interoperable systems, and contains the necessary viewpoints. Using the viewpoints and the guidelines that MAFIIA provides, allows us to “think right” when designing the system and ensures that all relevant information, such as important stakeholder, what standards are applied, decomposition of the system and the systems environment and how the system relates to the environment, is sufficiently documented. We believe that the lack of such documentation can considerably hamper the development of interoperable and sustainable systems for continuity of care.

In the system collaboration model we defined the main interactions between the components in the architecture using a component diagram showing use relations. This was elaborated with text explaining the collaborations, and detailed using UML communication diagrams showing the intra-service communication. In the component and interface specification model each interface has a specification that shows which methods are available and the input/output parameters of these methods. Distribution viewpoint In the distribution viewpoint we modeled the logical distribution of services focusing on what functionality could be distributed together and what should be kept separate (e.g. for business or security reasons). We used a UML deployment diagram and identified four nodes (a node is something that can host software).

In addition to enforce the presence of essential documentation, the use of architecture frameworks also ensure that the architecture is described on a sufficient level of abstraction, making it more resilient to change in the

6

M. Mikalsen et al. / Linkcare – Enabling Continuity of Care for the Chronically Ill across Levels and Profession

domain it is addressing. It is important to note that abstract does not imply informal. On the contrary, MAFIIA suggests using UML as the formalism when describing models. We are inline with Fowler 10], in that there are three ways of using UML; as sketch, as blueprint and as programming languages. We have used UML as blueprints to describe interfaces, and let the implementation of components be hidden in the architecture. This is inline with other current standardisation initiatives see e.g. the Healthcare Service Specification Project [11], a joint effort between the OMG and HL7.

prises. Consequently, we are currently continuing our work on profiles by creating domains specific healthcare profiles that incorporate information from standards (such as CEN) in support of developing interoperable healthcare systems (see e.g., [12]).

References [1] Beaglehole R. (ed.). Preventing chronic diseases: a vital investment: WHO global report. World Health Organisation, ISBN: 92 4 156300 1. 2005. [2] Winnem OM, Walderhaug S. Distributed, role based, guideline based decision support, Proceedings of E-he@lth in Common Europe, Springer pp. 101-109, ISBN 83915141-2-9. 2002. [3] Tattersall R. The expert patient: a new approach to chronic disease management for the twenty-first century. Clin Med 2002; 2(3): 227-229 [4] Walker J, Pan E, Johnston D, Adler-Milstein J,. Bates DW, Middleton B. The value of health care information exchange and interoperability. Health Affairs Web Exclusive 2005 (Jan.19): W5-10 W5-18. [5] Iakovidis I. Towards a Health Telematics Infrastructure in the European Union, In Information technology strategies from US and the European union: transferring research to practice for healthcare improvement, Amsterdam, IOS Press, 2000. [6] Patient Mobility in the European Union - Learning from experience: World Health Organization 2006, 2006. [7] MDA Guide Version 1.0.1 http://www.omg.org/cgi-bin/ doc?omg/03-06-01 [8] Chari K, Seshadri S. Demystifying Integration. Communications of the ACM, vol. 47, pp. 58-63, 2004. [9] Walderhaug S, Stav, E, Tomassen SL, Røstad L, Moe NB MAFIIA - an Architectural Description Framework: Experience from the Health Care Domain. In: Konstantas D, Bourrières JP, Léonard M, Boudjlida N. (eds.): Interoperability of Enterprise Software and Applications. Springer, 2005; 43-54 [10] Fowler M. UML Distilled, 3 ed: Addison Wesley, 2004. [11] The Healthcare Service Specification Project, http:// hssp.wikispaces.com/ [12] Walderhaug S, Mikalsen M, Hartvigsen G, Aagedal J. Improving Systems Interoperability with Model-driven Software Development for Healthcare. In Proceedings of Medinfo 2007. [13] Johnston S. UML 2.0 Profile for Software Services. http://www-128.ibm.com/developerworks/rational/library/ 05/419_soa/

We have used design patterns and the UML 2.0 profile for software services when modelling the architecture. Patterns describe a well proven solution to a recurring design problem. By applying agreed upon patterns (such as the publish-subscriber pattern) we are one step closer to having interoperable systems. A UML profile can be used to further specialize the architecture design. The UML profile defines stereotypes, tagged values and constraints that can be assigned to modelling elements in the design process [13]. We have designed our architecture using the UML 2.0 profile for software services, modelling our services as service partitions with stereotypes like service gateways, communication over service channels, and the notion of service providers and consumers. We experienced several benefits of using this approach. First, since the MAFIIA framework was already assuming the use of UML, applying a UML profile in the framework was straightforward. Second, the profile incorporated software service design terms, allowing us to unambiguously represent the system artefacts in a formal language. Third, the UML Profile assists in communicating concepts in the project. A Service Provider for instance, is a well defined concept and elements stereotyping this concept are bound by the constraints as defined in the profile. We used the profile in the tool it was created for, which is Rational Software architect, so a potential drawback of the profile is that it is not automatically applicable in other tools.

Conclusion and future work In this paper we have argued for a systematic and formal approach to the development of interoperable information systems using architecture frameworks such as MAFIIA that en-able formal software interface descriptions. We have shown how we applied a UML profile providing us support in design-ing our architecture inline with best practice. Healthcare enter-prises could impose vendors to apply the same design patterns in their system design, thus preparing the ground for interoper-able healthcare enter-

Address for correspondence Marius Mikalsen, SINTEF ICT, NO-7465 Trondheim, Norway. [email protected]

7

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Sustainable Ubiquitous Home Health Care – Architectural Considerations and First Practical Experiences Michael Marschollek, Klaus-H. Wolf, Oliver-J. Bott, Mirko Geisler, Maik Plischke, Wolfram Ludwig, Andreas Hornberger, Reinhold Haux Institute for Medical Informatics, Technical University Carolo-Wilhelmina of Braunschweig, Germany

lization of standards such as the HL7 Clinical Document Architecture (CDA) [4] or prEN 13606 [5] and the inadequate integration into regional health information system infrastructures, if existent, are regarded as key factors [6, 7]. Only few examples of the use of standards in home care can be found in literature. Van der Linden et al. present PropeR, a re-usable, modular EHR system based on open source components such as the openEHR standard, and report on its use for multidisciplinary care of stroke patients [8, 9], but make no statement on the representation of sensor data. In [10] the processes in home care are described in detail and a framework for home care cooperation is proposed. The authors use the XML standard for messaging between different health information systems.

Abstract Despite the abundance of past home care projects and the maturity of the technologies used, there is no widespread dissemination as yet. The absence of accepted standards and thus interoperability and the inadequate integration into transinstitutional health information systems (tHIS) are perceived as key factors. Based on the respective literature and previous experiences in home care projects we propose an architectural model for home care as part of a transinstitutional health information system using the HL7 clinical document architecture (CDA) as well as the HL7 Arden Syntax for Medical Logic Systems. In two short case studies we describe the practical realization of the architecture as well as first experiences. Our work can be regarded as a first step towards an interoperable – and in our view sustainable – home care architecture based on a prominent document standard from the health information system domain.

Concentrating on the use of standards from the health information system domain, we aim: •

to elucidate the requirements for sustainability in home care systems (aim 1), • to propose a functional, adaptive and modular (and in our view sustainable) architectural model (aim 2), and • to report on two current projects of the authors and their status of implementation (aim 3). The research questions addressed are the following:

Keywords: Home care, information system architecture, standards, sensors.

Introduction Telemedicine in general and telemedical systems for home care have been a key interest in medicine and medical informatics in the last two decades. Many research projects and funding programs have been conducted and some of the solutions developed have been successfully introduced in medical care and are commercially available. On the one hand there is a considerable advancement and maturity of the underlying technologies and methods, e.g. computer systems, ubiquitous computing, sensor systems, signal analysis, networks and health information systems, and on the other hand there is first proof that specific patient groups stand to benefit from tele home care [1]. Despite this there is no widespread use of these technologies as yet. The reasons for this lack of sustainability are manifold and it is often argued that, apart from technological barriers, evidence for cost benefit is sparse for tele home care systems [2], that profound changes in the work processes of care providers are required and that physicians lack the resources to interpret huge amounts of sensor data recorded in home settings [3]. In particular, the lack of uti-

Q1: What are the basic requirements for sustainable home care architectures with respect to the usage of standards? Q1.1: What are the categories of data processed in home care, what are the basic paths of information flow, and who are the participants? Q1.2: What are suitable forms of medical data representation for further use in transinstitutional health information systems? Q2: What is an architectural model suitable to meet these requirements and how can it be implemented?

Materials and methods Based on our previous experiences in building home care systems and with regard to the respective literature we summarize the basic architectural requirements for a sustainable, standards-based home care system (first section in Results). We then propose an architectural model along

8

M. Marschollek et al. / Sustainable Ubiquitous Home Health Care – Architectural Considerations and First Practical Experiences

with the strategy for its practical realization in the following section, employing the Three-layer Graph-based meta model (3LGM2)-methodology for statical modeling of health information systems [11, 12]. In the final two sections in Results we report on two case studies using our architectural model and present their current status of implementation. In the Discussion we balance advantages and disadvantages of our architecture and finally conclude with an outlook on our future work.

ent, configurable forms of data aggregation and presentation to fit user needs. In order to achieve semantic interoperability the heterogenous data gathered from multiple sources have to be represented in a consistent form, so that they can be used in transinstitutional health information systems. A prominent standard for clinical documents is the HL7 CDA, which is designed primarily for clinical documents such as discharge letters or diagnostic findings, and not for huge amounts of sensor readings.

Results

An architectural model for standards-based home care

Requirements for sustainable home care architectures

We propose an architectural model for a sensor-based home care system that makes use of the HL7 CDA (version 2), for data representation and includes a decision support infrastructure based on the HL7 Arden Syntax for Medical Logic Systems and its Medical Logic Modules (MLMs) [17].

In home care, mostly two different categories of data are gathered. Firstly, there are sensor data that are inherently a very heterogenous group, with sensors that measure just one parameter once a day, e.g. a sphygomanometer for blood pressure measurements, to multi-sensor devices that continuously record multiple channels and transfer a data stream to a computer system, e.g. a 12-lead ECG. Different persons with individually different diseases or functional disabilities will need customized sensor device compilations that fit their situation best. Therefore there is an urgent need for device interoperability standards that allow to build self-configurating ‘plug-and-play’ sensor sets with known features [13]. The ISO/IEEE 11073 medical device standard is a candidate [14], though at present hardly used outside intensive care units. Apart from medical sensors, context sensors that provide valuable information for the interpretation of the medical data also should be considered, e.g. accelerometers for the classification of activities of daily life (ADL) [15].

Figure 1 shows the general component model of our architecture. The patient or respectively the person is the primary source of data in the home care environment, but relatives and care givers may also provide valuable data on her or him, e.g. in the form of questionnaires or observations. Thus these are also part of the architecture. All data gathered are transformed into CDA documents and then transferred to a home server with a personal electronic health record (pEHR) system. Attached to this system is a decision support system that automatically retrieves stored CDA documents, analyzes them and in turn produces synopses or reports in CDA format or as messages, e.g. email alarms to a health care provider. The processing of the data takes place in two stages: firstly, the huge amounts of sensor data are preprocessed by employing temporal abstraction methods, intelligent filters and machinelearned classifiers to aggregated data units in CDA format. Secondly, the data are analyzed along with other CDA documents on the patient from ‘external EHR’ systems, i.e. other components of the transinstitutional health information system (cmp. Figure 1), with predefined MLMs. The MLMs contain medical knowledge coded in Arden Syntax to extract medically relevant information that in turn can be used e.g. to trigger alarms. The model architecture also contains two loops of feedback (cmp. Figure 1, dotted arrow) to the patient: an inner loop with machinegenerated reports made available via an interface, e.g. the home TV set, and an outer loop, where the feedback is posted by a health professional, e.g. a General Practitioner, on her or his institutional system and then transmitted via the transinstitutional communication infrastructure to the personal EHR at home and then sent to the interface.

A second category of data are data actively provided by the persons themselves, from relatives and care givers. These may either be structured, e.g. in a standardized questionnaire for well-being, and therefore can be represented with reference to code systems such as SNOMED CT [16], or unstructured such as e.g. in free text in emails or spoken comments. The semantic integration of the latter into existing health information systems remains a largely unsolved problem. The addressees of home care not only are a source but also a target of information, because feedback is a necessary precondition for promoting self-management and patient empowerment. When multiple sensors are used in home care, huge amounts of data will be recorded every day. There is an irrevocable need for intelligent processing of these data, which may be done already within the sensor device, on an additional mobile device worn by the patient, or on a computer at the patient’s home, probably all three in form of cascading preprocessing procedure. Data fusion is necessary both on technical and content level, and conclusions should be drawn considering all available data from a patient, including those stored in her or his electronic health record (EHR). As there are different groups of recipients of information – the patients, their care givers and their physicians – with different informational needs and levels of ‘health literacy’, there should also be differ-

Figure 2 shows the 3LGM2 model of the architecture described, with all entities, functions and relationships subdivided into three distinct architectural layers: the domain layer and the logical and physical tool layer.

9

M. Marschollek et al. / Sustainable Ubiquitous Home Health Care – Architectural Considerations and First Practical Experiences

Figure 1 - Component model of the proposed architecture based on the HL7 clinical document architecture standard (CDA). pEHR = personal electronic health record; MLM = Medical Logic Module

Figure 2 - Three-layer graph-based meta model (3LGM2) of the proposed home care information system. pEHR = personal electronic health record; CDA = clinical document architecture; MLM = Medical Logic Module

10

M. Marschollek et al. / Sustainable Ubiquitous Home Health Care – Architectural Considerations and First Practical Experiences

systems. For transinstitutional health information systems – and home care can be considered as part of these – they are vital in order to achieve interoperability and thus sustainability, in this context denoting that such a system meets the requirements for future use. Yet in the past often proprietary document ‘standards’ or formats were used in this field and still are. In addition to this, no generally approved form of representation for sensor data exists.

Case study 1: Laboratory Installation In our health-enabling technologies lab, a complete apartment equipped with different sensor technologies, we have installed a prototype system with a personal EHR system developed in our institute [18]. We experiment with different wireless as well as cabled sensors, e.g. triaxial accelerometers, that transfer real-time data to the system which then are analyzed and aggregated into CDA level 3 documents (cmp. Figure 1). In addition to the sensor data, data on the persons’ well-being are recorded with computerized questionnaires and stored subsequently, currently in CDA level 2 because we use a non-standardized questionnaire that is not listed in a coding system. We also use an Arden compiler that was constructed to retrieve CDA documents from the EHR system, to analyze them and to generate email messages, e.g. to a health care professional. This compiler was developed and tested in a home care project providing telemedical support for two patients having received a heart-and-lung transplantation [19]. A feedback component is included in the system in the form of a digital TV set with a media portal software. It features a “health” plugin currently showing a frequently updated graph summarizing a person’s activity level that is recorded with a wearable wireless accelerometer unit.

Our approach aims at bridging the gap between multiple data types encountered in home care by persistently using a prominent clinical document standard – the HL7 CDA. An obvious advantage of this approach is that such documents can easily be used within existing transinstitutional health information systems. Furthermore their semantics are clear and thus they can be used along with other clinical documents by decision support systems, as demonstrated in our architecture. The installation of a personal in-home EHR system with a decision support component has – apart from reducing data transfer costs – the benefit that decisions can be made in a timely manner and that the data recorded are kept in a private area within a secure system. So the patient may decide which information to pass on. A security solution based on a ticket-based electronic consent system for granting scalable access to the information stored in the personal EHR system has been developed by some of the authors and is part of the system used [18].

Case study 2: The LASS project The LASS project – a German acronym for self-sufficient and self-determined life in old age – aims at assessing the use of health-enabling technologies (e.g. sensor systems for fall detection) to improve both the quality of life and the efficiency of care for elderly people in their home environment. As part of this project, we currently establish an architectural infrastructure as decribed above in Figures 1 and 2.

Limitations The HL7 CDA standard is not primarily designed for the representation of sensor data, and the XML files tend to grow large. Therefore intelligent preprocessing is needed, bearing an inherent but unavoidable loss of information. Parts of our prototype architecture have only been tested in our health-enabling technologies lab so far. Further evaluation is necessary.

In a case study [20] we recorded annotated data from five elderly persons over a period of two weeks with a multisensor device (Sensewear Pro 2, Bodymedia, Pittburgh) and – employing methods of our previous research on this topic [15] – constructed machine-learned classifier algorithms for the classification of activities of daily life and body position that are used by our data preprocessing module. The data from the multi-sensor device are subsequently transformed into a CDA level 3 document that, once stored in the personal health record, can be analyzed by the Arden engine described in the previous section. The specific aim is to identify elderly peoples’ activity profiles that provide valuable information on the persons’ behavior as well as context information for other medical data recorded in home care. We also installed specially prepared, easy-to-use PC systems in four of the elderly persons’ homes and tested daily questionnaires on well-being, the acceptance of which was good. The answers were stored in CDA level 2 documents as stated above.

Apart from this, there is still a lack of analyses regarding the costs and – measurable or perceived – benefits of such a home care system, both for the patients respectively users and the health care professionals. Some studies imply a reduction of costs as the duration of hospital stays is reduced [1], other studies show an increased overall cost as the technology for monitoring and feedback is expensive [21]. Our future work will include a cost-benefit evaluation of the proposed system.

Conclusion We have proposed an architectural model for a sensorbased home care system that makes use of two prominent standards from the domain of health information systems, the HL7 CDA (version 2) for semantically accessible data representation and the HL7 Arden Syntax for Medical Logic Systems for medical knowledge representation and decision support. We believe that the use of standards is an important precondition for building an interoperable and therefore sustainable home care system [22].

Discussion The importance of using standards for information interchange in the health care domain has often been stated and has partly been realized in institutional health information

Our future work will be directed towards the evaluation of all parts of our prototype system in a field study outside lab

11

M. Marschollek et al. / Sustainable Ubiquitous Home Health Care – Architectural Considerations and First Practical Experiences

conditions. Furthermore we will develop and implement further preprocessing modules and transformation components for several sensor devices as well as MLMs with the help of our medical partners. Besides this, we will integrate care givers into our system, both laypersons and professionals.

[12] Wendt T, Haber A, Brigl B, Winter A. Modeling Hospital Information Systems (Part 2): using the 3LGM2 tool for modeling patient record management. Methods Inf Med 2004;43(3):256-67. [13] Zywietz C. Communication and interoperability for serial comparison in continuous health care--the new challenges. Stud Health Technol Inform 2004;108:172-80. [14] Yao J, Warren S. Applying the ISO/IEEE 11073 standards to wearable home health monitoring systems. J Clin Monit Comput 2005;19(6):427-36. [15] Marschollek M, Wolf KH, Plischke M, Haux R. Classification of activities of daily life from long-term realistic multi-sensor data. In: IEEE Health Pervasive Systems (HPS06); 2006 June 29th; Lyon: IEEE; 2006. p. 30-4. [16] College of American Pathologists. SNOMED CT. [Webpage] [cited 2006 Nov 16th]; Available from: http:// www.snomed.org/snomedct/index.html [17] Health Level Seven (HL7). HL7 Standards. [Webpage] [cited 2006 Nov 11th]; Available from: http://www.hl7.org/ [18] Bergmann J, Bott OJ, Pretschner DP, Haux R. An e-consentbased shared EHR system architecture for integrated healthcare networks. Int J Med Inform 2006;76(2-3):130-6. [19] Geisler M, Bott OJ, Tegtbur U, Bergmann J, Pretschner DP. Zur Integration einer auf der Arden-Syntax basierenden Entscheidungsunterstützungskomponente in eine Telematikplattform. Beispielhafte Umsetzung im Rahmen der telemedizinischen Nachbetreuung von Patienten mit Herz-/Kreislauferkrankungen. german medical science [Meeting Abstract] [cited 2006 Nov 11th]; Available from: http://www.egms.de/de/meetings/gmds2006/06gmds281. shtml [20] Marschollek M, Ludwig W, Schapiewski I, Schriever E, Schubert R, Dybowski H, et al. Multimodal home monitoring of elderly people – first results from the LASS study. In: IEEE First International Workshop on Smart Homes for Tele-Health (SmarTel07); 2007; Niagara Falls, Canada (May 21-23): IEEE; 2007. [21] Bott OJ, Hoffmann I, Bergmann J, Kosche P, von Ahn C, Mattfeld DC, et al. Simulation based cost-benefit analysis of a telemedical system for closed-loop insulin pump therapy of diabetes. Stud Health Technol Inform 2006;124:435-40. [22] Haux R. Individualization, globalization and health - about sustainable information technologies and the aim of medical informatics. Int J Med Inform 2006;75(12):795-808.

Acknowledgments The authors thank the following persons for their support in conducting case study 2: Ines Schapiewski and Elin Schriever (German Red Cross Braunschweig), Hartmut Dybowski and Rainer Schubert (City of Braunschweig) and Hubertus Meyer zu Schwabedissen (Medical Center Braunschweig).

References [1] Cleland JG, Louis AA, Rigby AS, Janssens U, Balk AH. Noninvasive home telemonitoring for patients with heart failure at high risk of recurrent admission and death: the Trans-European Network-Home-Care Management System (TEN-HMS) study. J Am Coll Cardiol 2005;45(10): 1654-64. [2] Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. Bmj 2002;324(7351):1434-7. [3] Lamothe L, Fortin JP, Labbe F, Gagnon MP, Messikh D. Impacts of telehomecare on patients, providers, and organizations. Telemed J E Health 2006;12(3):363-9. [4] Dolin RH, Alschuler L, Boyer S, Beebe C, Behlen FM, Biron PV, et al. HL7 Clinical Document Architecture, Release 2. J Am Med Inform Assoc 2006;13(1):30-9. [5] CEN/TC251. prEN 13606-1: Health informatics — Electronic health record communication — Part 1: Reference model (CEN FV and ISO DIS Draft). 2006 [cited 21.09.2006]; Available from: http:// www.centc251.org/WGI/N-documents/WG1_N0616a_votingreport_ENQ_13606-4dispositionv1.2.doc [6] Bott OJ. Health information systems: between shared care and body area networks. Methods Inf Med 2006;45 Suppl 1:53-4. [7] Koch S. Meeting the Challenges - the Role of Medical Informatics in an Ageing Society. Stud Health Technol Inform 2006;124:25-31. [8] van der Linden H, Grimson J, Tange H, Talmon J, Hasman A. Archetypes: the PropeR way. Medinfo 2004;11 (Pt 2):1110-4. [9] van der Linden H, Boers G, Tange H, Talmon J, Hasman A. PropeR: a multi disciplinary EPR system. Int J Med Inform 2003;70(2-3):149-60. [10] Bricon-Souf N, Anceaux F, Bennani N, Dufresne E, Watbled L. A distributed coordination platform for home care: analysis, framework and prototype. Int J Med Inform 2005;74(10):809-25. [11] Winter A, Brigl B, Wendt T. Modeling hospital information systems. Part 1: The revised three-layer graph-based meta model 3LGM2. Methods Inf Med 2003;42(5):544-51.

Address for correspondence Dr. Michael Marschollek, MD MSc Institute for Medical Informatics Technical University Carolo-Wilhelmina Muehlenpfordtstrasse 23 D-38106 Braunschweig, Germany +49 531 391 9504; [email protected]

12

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Do Physicians Take Action on High Risk Family History Information Provided by Patients Outside of a Clinic Visit? Lynn A. Volk, MHSa, Maria Staroselskya, Lisa P. Newmarka, Hannah Phama, Alexis Tumoloa,b, Deborah H. Williams, MHAa,b, Ruslana Tsurikovaa,b, Jeffrey Schnipper, MD, MPHb,d Jonathan Wald, MDc,d, David W. Bates, MD, MSca,b,d a

Clinical and Quality Analysis, Information Systems, Partners HealthCare System, Wellesley, MA, USA Division of General Medicine, Department of Medicine, Brigham & Women’s Hospital, Boston, MA, USA c Clinical Information Research & Development, Information Systems, Partners HealthCare System, Wellesley, MA, USA d Harvard Medical School, Boston, MA, USA b

family history and in determining appropriate follow-up and monitoring.[7,8]

Abstract Clinically relevant family history information is frequently missing or not readily available in electronic health records. Improving the availability of family history information is important for optimum care of many patients. Family history information on five conditions was collected in a survey from 163 primary care patients. Overall, 53% of patients had no family history information in the electronic health record (EHR) either on the patient’s problem list or within a templated family history note. New information provided by patients resulted in an increase in the patient’s risk level for 32% of patients with a positive family history of breast cancer, 40% for coronary artery disease, 50% for colon cancer, 74% for diabetes, and 95% each for osteoporosis and glaucoma. Informing physicians of new family history information outside of a clinic visit through an electronic clinical message and note in the EHR was not sufficient to achieve recommended follow-up care. Better tools need to be developed to facilitate the collection of family history information and to support clinical decision-making and action.

In this study, we assessed the availability of coded or structured family history information in EHRs and the extent to which solicited input from patients would identify those at higher risk as compared to risk assessed using information readily available in their EHR alone. We also assessed whether physicians took action to change care management based on new family history information provided by a patient outside of a clinic visit.

Methods Study setting Partners HealthCare System is an integrated healthcare delivery system in the Boston, Massachusetts, United States (U.S.) area that includes two major academic medical centers, several community hospitals, and an extended physician network. A comprehensive EHR is in use at the majority of ambulatory care practices. This EHR includes the ability to capture provider notes, results of lab and diagnostic tests, problem lists, medication lists, and automatically generates patient-specific reminders for health maintenance items. However, at the time of this study, input of structured or coded family history information was not supported.

Keywords: family history, electronic medical records, clinical decision-making

Introduction

Patients who choose to enroll are able to electronically communicate with their physician practice through a secure web-based patient portal called Patient Gateway. At the time of the study, features of Patient Gateway included patients being able to view their EHR medication and allergy lists, and request prescription renewals, appointments, and referrals.

Family history data often goes unrecorded or is available only as uncoded free text in the electronic health record (EHR).[1,2,3] Patient provided family history information may provide incremental value if one can learn of increased risks that may influence care management.[1,4] Although sophisticated family history assessment is most expertly done by those trained in this area, much can be learned about a patient´s risk from whether a relative with a condition is a primary or secondary relative, how many relatives have the condition, and the approximate age at which they were diagnosed.[5,6] Clinical decision support can aid physicians in identifying patients “at risk” due to

Recruitment Patients were recruited from a single Partners-affiliated ambulatory primary care practice in the suburbs of Boston. A packet was sent through postal mail with a cover letter, consent form, opt-out card, privacy notice, and a return envelope to send back the consent form.

13

L. A. Volk et al. / Do Physicians Take Action on High Risk Family History Information Provided by Patients Outside of a Clinic Visit?

Once consent was received, a paper-based survey that asked about medications, health maintenance status, and family history was populated with patient–specific data and sent to consenting patients. A two-step process was required to distribute the survey because of the specific medical information included that could not be generated without patient consent. The Partners Human Research Committee approved the study and the survey was administered from November 2003 – Feburary 2004.

available was defined as family history information recorded on a patient’s problem list or within a note template containing a family and social history section. To evaluate physician action in response to the family history note, electronic health records were reviewed for visit, medication, and procedure information consistent with recommendations made in these clinical areas by local experts, based on guidelines from the Risk Management Foundation of the Harvard Medical Institutions, Inc., the U.S. Preventive Services Task Force, American Heart Association, National Cholesterol Education Program, American Diabetes Association, and National Osteoporosis Foundation.[11-19]

Up to two reminders were sent to non-responders to solicit consent forms and also to complete the survey once they had consented. Survey data on medications and health maintenance are reported elsewhere.[9,10]

P-values were calculated for comparisons between respondents and non-respondents using a chi-square test for proportions and Wilcoxon rank-sum test for continuous variables.

Surveys Patients were asked to provide family history information in a structured format that required specifying the number of family members by degree of relationship and age category that had been diagnosed with a condition. In addition to requesting information on six specific conditions (glaucoma, osteoporosis, coronary artery disease, diabetes mellitus, breast cancer, and colon cancer), patients could complete the form for other conditions for which they had a positive family history.

Results Among 1098 patients solicited, 189 patients consented and a family history survey was completed by 163 patients. Respondents and non-respondents were similar in the proportion that were female (66% and 67%, respectively), and their average number of medical problems (4.8 and 4.4, respectively) and medications (3.5 for both) listed in the EHR. Respondents were significantly older than nonrespondents on average (50.6 years and 46.8 years, p=0.001).

Patients were then instructed to return the surveys to researchers who provided a copy of the survey back to patients. Patients were informed that information provided in their surveys would be shared with their primary care physicians and may become part of their medical record.

New information identifying higher risk profile

Surveys were tracked and analyzed using Microsoft Access. Once a survey was received, researchers created a note within the patient’s electronic health record with Partners Health Information Services approval. This note reported on the information provided by the patient and clearly identified the source of the information as obtained from the patient through a research survey. Physicians were sent a clinical message within 90 days through the EHR messaging system that informed them of the new note in the patient’s record.

Fifty-three percent (53%) of the respondents had no family history information in the electronic health record either on the patient´s problem list or within a templated family history note. For glaucoma, osteoporosis, and colon cancer over 90% of the patients determined to have a positive family history were identified based solely on new information provided by the patients in the survey. (Table 1) A family history of diabetes, breast cancer, or coronary artery disease was more frequently available within the EHR, although more than 80% of patients with a positive family history for these conditions were still identified through the survey alone.

Analysis Patient provided family history information was compared with information readily available in the EHR. Readily

Table 1 - Sources of positive family history information and changes in patient risk level Source of Positive Family History Condition Glaucoma Osteoporosis Diabetes Breast Cancer Colon Cancer Coronary Artery Disease

Change in Risk Level with Info from Survey Alone

EHR and/or Survey N(%)

Survey Alone N(%)

Reduced N(%)

Did Not Change N(%)

Increased N(%)

20 22 51 47 68 82

19 (95%) 20 (91%) 42 (82%) 40 (85%) 66 (97%) 67 (82%)

0 (0%) 0 (0%) 1 (2.4%) 0 (0%) 0 (0%) 0 (0%)

1 (5.3%) 1 (5.0%) 10 (23.8%) 27 (67.5%) 33 (50.0%) 40 (59.7%)

18 (94.7%) 19 (95.0%) 31 (73.8%) 13 (32.5%) 33 (50.0%) 27 (40.3%)

14

L. A. Volk et al. / Do Physicians Take Action on High Risk Family History Information Provided by Patients Outside of a Clinic Visit?

Table 2 - Recommended actions occurring following clinical message indicating new family history information Condition & Recommendations (N=total patients with increased risk level due to family history information from survey alone)

Patients Meeting Criteria for Physician Action N

Patients for Whom Recommended Action Occurred After Clinical Message was Sent N (%)

18 16

14 (78%) 1 (6%)

13 12

6 (46%) 5 (42%)

31

23 (74%)

Glaucoma (N=18) visit w/in specified time consider IOP Osteoporosis (N=19) visit w/in specified time DEXA scan Diabetes (N=31) visit w/in specified time

Consider fasting glucose or HbA1c 31 26 (84%) Breast Cancer (N=13) visit w/in specified time 13 5 (38%) clinical breast exam 13 3 (23%) mammogram 13 8 (62%) referral to genetic counselor* 13 0 (0%) Colon Cancer (N=33) visit w/in specified time 32 13 (41%) colonoscopy 9 2 (22%) genetics referral for HNPCC* 1 0 (0%) Coronary Artery Disease (N=27) visit w/in specified time 27 13 (48%) lipid profile 27 19 (70%) cholesterol lowering med 11 3 (27%) blood pressure lowering med 7 3 (43%) aspirin advice/use 27 5 (18%) smoking cessation counseling 0 n/a** exercise/nutrition counseling/referral 22 8 (36%) consider plasma homocysteine or Creactive protein 27 2 (7%) *genetic tests may not have been documented in accessible EHR data **n/a = not applicable New information provided by patients led to an increased risk level for 50% of patients with a positive family history of colon cancer, and the majority of patients with a positive family history for diabetes (74%), osteoporosis (95%), and glaucoma (95%). Increases in risk were found for 32% of patients with a positive family history of breast cancer and 40% with coronary artery disease.

Although patients at higher risk of glaucoma had visits with their physicians more than three quarters of the time after clinical messages were sent, only 1 out of 16 patients (6%) had a documented intraocular pressure measurement (IOP). For osteoporosis, fewer had visits (46%) than did the glaucoma patients, but 42% did receive a dual x-ray absorptiometry (DEXA) scan within the specified time after the clinical message.

Physician action in response to clinical messages

In follow-up to new information regarding a patient’s family history of colon cancer, only 2 out of 9 (22%) patients meeting the criteria to have a colonoscopy received one. However, 15 of the 33 patients at higher risk for colon cancer had already had a colonoscopy sufficiently recently that they did not meet criteria to have one within a year of

Overall, of the 21 actions physicians should have taken across all conditions in response to new family history information, 16 of the actions were carried out less than 50% of the time. (Table 2) However, for the smoking cessation counseling action, no patients met the criteria of being a current smoker so no action was required.

15

L. A. Volk et al. / Do Physicians Take Action on High Risk Family History Information Provided by Patients Outside of a Clinic Visit?

the clinical message. This finding may be reflective of compliance with general screening guidelines for patients regardless of risk level associated with family history. The same may be true of diabetes and breast cancer actions; 84% of the patients meeting the criteria for fasting glucose or HbA1c tests and 62% requiring mammograms had received them within the time specified by the guidelines.

for Disease Control (CDC) and U.S. Surgeon General’s Family History Initiative, have been encouraging patients to complete their family histories and discuss them with their physicians.[21] As personal health records gain momentum and are designed to include family history, the numbers of patients with more comprehensive electronic repositories of this important information will increase.

Although referrals for genetic testing related to breast or colon cancer were not documented for any of those patients meeting the criteria, these may have occurred but not been documented in the generally accessible EHR data.

To get the greatest value from this patient data, however, it will be important to create effective processes to facilitate the transfer and use of this information without undue burden on busy primary care physicians. Sending an electronic clinical message outside of a patient visit to alert physicians of new family history information was investigated in this study as a proxy for patients completing data online and then submitting it to their physicians through a patient portal, such as Patient Gateway. Our results show that simply sending a message outside of a visit is not sufficient for achieving appropriate screening rates for patients at increased risk due to family history.

For coronary artery disease (CAD), 70% of patients with a positive family history had a lipid profile done after the clinical message was sent which may also reflect general screening. However, only 27% of patients meeting the criteria for cholesterol lowering medications and 43% of those for blood pressure medications received these prescriptions after the clinical message. Only 36% of those patients with a positive family history for CAD and an LDL >130 and/or body mass index (BMI) of >25 had documented exercise and nutrition counseling or referrals.

The lack of physician follow-up may be due, in part, to the information being presented outside of a visit when a physician would not be able to discuss it with a patient and when action would be outside the normal physician workflow. In addition, no clinical decision support was provided so that physicians could evaluate the importance of the information and know what action was required. Tools that support the electronic capture of coded data by patients, deliver this information to physicians around visit time, support provider review and documentation in the electronic health record, and offer clinical decision support integrated with electronic ordering to guide appropriate follow-up are important for closing this gap in patient care.

Discussion Accurate and current family history information is becoming increasingly important with the advent of genomics and the promise of personalized medicine. Typically, patients are asked about their family history at their initial visits with their primary care physicians, but this information is generally recorded as free text in notes, and updating this information is done sporadically or not at all. When the information is captured in unstructured format, it is inaccessible to automated clinical decision support. As a result, physicians are missing opportunities for providing optimum screening and prevention for those patients at higher risk for certain conditions based on their family history.

Limitations The response rate was low, probably in part because a very large packet was sent and a two-step enrollment process was required. The responders were somewhat older than non-responders and may have differed in other ways not measured. In addition, the study was uncontrolled. Whether any recommended actions occurred between a patient’s receipt of the survey and when a clinical message was sent were not specifically assessed.

In this study, we found that significant family history information indicating that patients were at an increased risk for certain conditions was frequently missing from the EHR, at least as structured or coded information, or within a templated note. This was particularly true of glaucoma, osteoporosis, and diabetes. Coronary artery disease, breast and colon cancer family history information were more readily available, perhaps due to greater prevention emphasis, patient knowledge of risks, and focus of physicians on soliciting this information. More structured tools to support the electronic capture of this information could help to guide comprehensive data collection for all conditions in which there are known care implications when an individual is at risk due to family history. Decision support that uses this information would also ensure the physicians take appropriate actions when patients have positive family histories.

Conclusions Clinically relevant family history information is frequently missing or not readily available in electronic health records. For all diagnoses of interest, patients often provided family history data that might influence use of screening tests, especially for glaucoma, osteoporosis, and diabetes. With the increased emphasis on family history as the field of genomics becomes more sophisticated, these results demonstrate the potential benefits of soliciting information from patients in a structured format outside of visits. However, sending clinical messages to physicians informing them of new family history information outside of a visit is not sufficient to ensure appropriate screening occurs. Better tools need to be developed to facilitate the flow of family history information from patients to their

A recent study reported that, whereas 96% of patients understand the importance of knowing one’s family medical history, only 30% have collected this information.[20] Recent U.S. national efforts, such as those by the Centers

16

L. A. Volk et al. / Do Physicians Take Action on High Risk Family History Information Provided by Patients Outside of a Clinic Visit?

primary care physicians and to support clinical decisionmaking and action.

[11] Risk Management Foundation of the Harvard Medical Institutions. RMF Breast Care Management Algorithm. Cambridge, MA, 2003. http://www.rmf.harvard.edu/files/ documents/BCA2003.pdf [cited 2006 Dec 1]. [12] Risk Management Foundation of the Harvard Medical Institutions. RMF Colorectal Cancer Screening Algorithm. Cambridge, MA, 2004. [13] U.S. Preventive Services Task Force. Aspirin for the primary prevention of cardiovascular events: recommendations and rationale. Ann Intern Med 2002;136:157-60. [14] U.S. Preventive Services Task Force. Guide to Clinical Preventive Services. Report of the United States Preventive Services Task Force. 3rd ed. International Medical Publishing, 2002. http://www.ahrq.gov/clinic/gcpspu.htm [cited 2006 Dec 1]. [15] U.S. Preventive Services Task Force. Screening for Glaucoma. http://www.ahrq.gov/clinic/uspstf/uspsglau.htm [cited 2006 Dec 1] [16] National Osteoporosis Foundation. Physician’s Guide to Prevention and Treatment of Osteoporosis. http://www. nof.org/physguide/index.asp [cited 2006 Dec 1] [17] Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation. Dec 17 2002; 106(25):3143-3421. [18] Pearson TA, Blair SN, Daniels SR, Eckel RH, Fair JM, Fortmann, SP, Franklin BA, Goldstein LB, Greenland P, Grundy SM, Hong Y, Miller NH, Lauer RM, Ockene IS, Sacco RL, Sallis JF, Smith SC, Stone NJ, Taubert KA. AHA Guidelines for Primary Prevention of Cardiovascular Disease and Stroke: 2002 Update. American Heart Association Science Advisory and Coordinating Committee. Circulation. 2002; 106(3):338-391. [19] American Diabetes Association. Standards of Medical Care in Diabetes. Diabetes Care 2006; 29:S4-S42. [20] Yoon PW, Scheuner MT, Gwinn M, Khoury MJ. Awareness of family health history as a risk factor for disease-United States, 2004. MMWR Morb Mortal Wkly Rep 2004; 53:1044-7. [21] United States Department of Health & Human Services. U.S. Surgeon General’s Family History Initiative. Available from: http://www.hhs.gov/familyhistory. 2006 Nov 15 [cited 2006 Dec 1].

Acknowledgements The authors would like to thank Tiffany Wang, Margaret Lippincott, Virginia Carlson, and Tony Yu for their assistance with this study. This work was supported, in part, by the Markle Foundation, New York, USA which is not responsible for the contents of this paper.

References [1] Murff HJ, Byrne D, Syngal S. Cancer risk assessment: quality and impact of the family history interview. Am J Prev Med 2004; 27: 239-45. [2] Rich EC, Burke W, Heaton CJ, Haga S, Pinsky L, Short MP, Acheson L. Reconsidering the family history in primary care. JGIM 2004; 19: 273-80. [3] Guttmacher AE, Collins FS, Carmona, RH. The family history – more important than ever. NEJM 2004; 351: 233336. [4] Yoon P, Scheuner MT, Peterson-Oehlke K, Gwinn M, Faucett A, Khoury MJ. Can family history be used as a tool for public health and preventive medicine? Genet Med 2002; 4: 304-10. [5] Murff HJ, Spigel DR, Syngal S. Does this patient have a family history of cancer? An evidence-based analysis of the accuracy of family cancer history. JAMA 2004; 292: 1480-89. [6] Qureshi N, Bethea J, Modell B, Brennan P, Papageorgiou, A, Raeburn S, Hapgood R, Modell, M. Collecting genetic information in primary care: evaluating a new family history tool. Family Practice 2005; 22(6): 663-69. [7] Emery J, Walton R, Murphy M, Austoker J, Yudkin P, Chapman C, Coulson A, Glasspool D, Fox J. Computer support for interpreting family histories of breast and ovarian cancer in primary care: comparative study with simulated cases. BMJ 2000; 321: 28-32. [8] Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330(7494): 765 [9] Staroselsky, M, Volk LA, Tsurikova R, Pizziferri L, Lippincott M, Wald J, Bates DW. Improving electronic health record (EHR) accuracy and increasing compliance with health maintenance clinical guidelines through patient access and input. Int. J. Med. Inform. 2006; 75: 693-700. [10] Staroselsky M, Volk LA, Tsurikova R, Newmark LP, Lippincott M, Litvak I, Kittler A, Wang T, Wald J, Bates DW. An effort to improve electronic health record medication list accuracy between visits: Patients’ and physicians’ response. Int. J. Med. Inform. 2007 (in press)

Address for correspondence Lynn Volk, Partners HealthCare System – Information Systems, 93 Worcester Street, Wellesley, MA 02481 USA, 781-416-8561, [email protected]

17

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Sharing Electronic Laboratory Results in a Patient Portal – a Feasibility Pilot Jonathan S. Wald, MD MPHa,c, Karen Burk, MAa, Kate Gardnera, Raisa Feygina, Elizabeth Nelsona, Marianna Epsteina, Eric G. Poon, MD MPHa,b,c, Blackford Middleton, MD MPH MSca,b,c a

Partners Information Systems, Partners HealthCare, Boston, MA, USA b Brigham and Women’s Hospital, Boston, MA, USA c Harvard Medical School, Boston, MA, USA

this paper. Since September, 2006, the new features are being deployed among primary and specialty care clinics as a result of the success of the pilot.

Abstract Surveys of patients consistently demonstrate a very strong interest in having secure electronic access to their own laboratory test results. In recent years, a number of health care providers and lab service centers have offered this capability, which now extends to millions of patients in the United States. Yet, little has been published on the methods of making lab results available. This case report identifies the objectives, methods, and results of a feasibility pilot conducted at Partners Healthcare from May to September, 2006. A candidate set of results were identified, approved for release, programmed into Patient Gateway, Partners’ secure patient portal, and studied. Patient and practice feedback was positive. No noticeable rise in patient concerns was observed by practice staff or through patient surveys. One-half of patients who viewed results accessed reference information linked to a result. Organizational and practice-level issues necessary to support continued rollout are described.

Materials and methods Partners HealthCare, a large integrated delivery network in Eastern Massachusetts comprised of two teaching hospitals, three community hospitals, four specialty hospitals, and a large number of community physicians in primary and specialty care, offers providers an enterprise ambulatory electronic record, the LMR (Longitudinal Medical Record), a patient portal extension, Patient Gateway (PG) [1], the CDR (Clinical Data Repository) that stores laboratory results, and other related clinical systems components. LMR (used by over 3,500 physicians) is interfaced with PG (used by over 30,000 patients since its launch in 2002). PG enables authorized patients to view real-time chart information abstracted from the LMR – currently medications, allergies, and future/past appointments. Laboratory test results are available to providers and staff who use LMR through a simple viewer. Results pending physician action across multiple patients are displayed through an advanced viewer, the Results Manager (RM) [2].

Keywords: Personal health records, diagnostic tests, communication, continuity of patient care, computerized medical records systems.

Introduction

Pilot objectives and metrics

In May, 2006, Partners HealthCare (Boston, MA, USA) began a feasibility pilot to introduce online laboratory results to patients in two primary care practices, via an established, self-developed secure internet portal, Patient Gateway. This paper aims to describe the rationale, design, and results of the four-month pilot to share our learning.

The primary pilot objective was to offer patients secure online access to their own lab results, striking a balance between the goals of “clinical transparency” (avoid withholding information) and “clinical sensitivity” (avoid violating patient and physician communication preferences). A second objective was to understand what technical, workflow, and organizational challenges must be addressed in order to successfully scale the new capability to all practices and patients. Pilot success would be measured by usage, spontaneous and solicited feedback from patients and providers, and brief surveys of patients about their experience.

Partners’ approach was informed by other organizations who have shared online results, notably Group Health Cooperative (Seattle, WA, USA) and CareGroup (Boston, MA, USA). The project context included: a) the desire for a short development time frame, b) the need to coordinate with provider workflow, c) the need to align with practice and organizational policies, and d) the need to address clinical information systems environment dependencies. Releasing laboratory results non-electronically will continue. Practical ways to address these challenges, while somewhat unique for each organization, are the focus of

Results feature design The feasibility pilot introduced two new features – patient access to browse laboratory results in PG (“Patient Results”), and provider ability to take a Results Letter she

18

J. S. Wald et al. / Sharing Electronic Laboratory Results in a Patient Portal – a Feasibility Pilot

had created using RM in LMR, and instantly send it electronically to the patient, via PG (“Online Results Letter”).

included the test name, result value, “#” character if the value is out of reference range, units, reference range, and an “information button” with a hyperlink to general reference information about the result that opens in a separate window and is not maintained by PG.

Expert panel. An expert panel of 7 clinicians was formed to identify the laboratory results to release and their timing rules (immediate release, or two business day embargo). The expert panel identified approximately 50 different results (chemistry, hematology, drug levels, endocrine, etc.). Since each clinical result (e.g. potassium) may correspond to multiple CDR items (e.g. serum potassium, blood gas potassium, serum potassium from another lab, etc), a total of 170 unique CDR items were included in the pilot. CDR results from all dates (matching a CDR code) were displayed.

General reference information in context was a design requirement. Lab Tests Online® (LTO), www.labtestsonline.org, a public resource for patients and caregivers with professionally reviewed articles, was selected for use during the pilot because of the breadth and quality of its content, and the ability of an outside site (like PG) to directly address a search query for a specific item.

Release Framework. Each CDR result was assigned a release category of “release now” or “release in two business days”. The immediate release assignment meant that once the result was available in the CDR (to the physician) it was also available to the patient. The embargoed release meant that the result was “held” from display in PG for two business days once it was available for review by the physician. The embargo clock began when the result became available (not pending). Release timing (immediate or 2 day embargo) was not predicated on whether the result was in reference or out of reference range. Software changes in the CDR system would be required to accomplish this, and were out of scope for the pilot. PG Patient Results design. In the patient portal, a new menu item under the “Health Record” menu was introduced: “Results”. This opened a “Results Summary” page with a table of results (Figure 1). Each row in the table lists a single result “name” (e.g. CBC – Complete Blood Count) and most recent value, using the CDR display name familiar to physicians. The name is hyperlinked to a “history” page (e.g. CBC results from all dates). On the Results Summary page, the “date” is hyperlinked to a “Specimen” page (e.g. CBC, PLT – Platelets, etc. for a given date). Sorting by column is supported.

Figure 2 – Lab Tests Online® search results for Sodium To minimize maintenance and gain sensitivity of the information retrieved given a variety of user interests, the information hyperlink avoids tight coupling to a single LTO page – but rather – executes a search query using the name of the result, bringing the user to the “Search Results” page in LTO (Figure 2). Each link works this way. If LTO were to modify their syntax for search queries, every PG link would break, making it easier to detect the problem, and simple to correct and maintain across all 170 CDR results items. LMR Physician Results Letter design. The new LMR feature for sending a Results Letter was essentially a button called “Send to PG” placed on the RM screen to allow the provider to take action – to make a Results Letter available to the patient via PG. When this happens, the patient is notified that a message from their doctor is waiting for them in PG. Multiple patient Results Letters can be “sent” as a batch to individual patients, just as batch printing is supported. In addition, the Results Letter display in LMR was enhanced to indicate when the patient had “read” the document (the display includes “first opened” and “last opened”). Before and during this pilot, providers had the ability to type or copy/paste results from LMR into a PG message and send that message to the patient. Both methods were voluntarily used during the pilot by physicians to provide interpretation of results to the patient.

Figure 1 – Results Summary page in Patient Gateway There is no patient notification of a new result. A “pending” result was displayed as “pending” to indicate it was not yet available (to providers). An embargoed result displayed as “held” during the embargo period, to indicate it was available for review by the provider, but not available yet in PG to the patient. Once displayed, each result

Configuration tool. A setup tool was created to activate the feature for each practice (default: all patients belonging

19

J. S. Wald et al. / Sharing Electronic Laboratory Results in a Patient Portal – a Feasibility Pilot

to that practice have the new features). In addition, a tool was created for activating the feature for a discrete list of patients to support limited deployment during the pilot. Auditing tools for pilot analysis recorded whether a patient ever looked at the new Results page in PG, or whether a patient ever received a Results Letter from a provider. Patient web surveys were sent to participating patients using Perseus SurveySolutions® software. Survey results were submitted anonymously.

Results Demographics and usage for PG results The pilot users included a total of 10 physicians in 2 practices who offered the PG Results feature to 3,583 patients with a PG account (mean age 42 years, 49% female) as shown in Table 1. 2,417 (67%) of those accounts had been activated (included an external email address for patient notification). Ten percent of email notification messages were returned “undelivered”. Over the next 8 weeks, 842 patients signed in to PG, and 594 (71%) viewed the Results page. Patients in practice A who viewed PG Results also viewed LTO reference information in 50% of cases. Spontaneous feedback from patients was uniformly positive. Suggestions for enhancements included: showing more results; showing radiology and cardiology reports; adding a graphing feature; and some usability improvements.

Participant recruitment Two primary care practices volunteered to offer the new features to their patients and physicians. Practice A activated the features for 2 (of 8) physicians and all of those physician’s patients. Practice B activated the features for all 8 of their physicians, and all their patients. Patient and physician use of the new features was completely voluntary. Physicians attended a single brief training session to familiarize them with the results that patients could see, and with the new button in LMR to send a Results Letter online to PG.

The survey results (Table 2) were strongly positive. They were collected anonymously (an IRB waiver was obtained since a survey response was considered an implied consent), so demographics are not available. Among 128 respondents, 89% felt the feature was “easy to use”, 85% felt results were clearly presented, 68% felt they would recommend the site (with the new feature) to others, and 64% found the LTO reference information “very helpful”. Three-quarters of respondents said they viewed the reference information. One in six respondents (17%) said they had questions for their provider as a result of seeing their results information. 31% wished for additional results, and 23% felt improvements would make the feature even more valuable.

Patients were notified of the new feature and invited to try it in two ways. A PG message was sent to each eligible patient (within the secure PG portal), which also triggers a notification message to the patient’s preferred email address (e.g. gmail, yahoo, etc). Upon signing in to PG, a Home page message announced the new features. This page displayed each time a user signed in. Pilot physicians and staff were contacted about their experience 8 weeks after the pilot began; after twelve weeks, a brief 10-item web-based survey was administered to patients who had used the new features to assess usability and impact.

LMR results letter “sent to PG” In the first 6 weeks of the pilot, 5 providers sent 121 Results letters to 107 patients. A total of 65 patients (61% of 107) opened 70 letters (58% of 107). Fifty letters (41%)

Spontaneous “feedback button” information from users was also collected.

Table 1 – Results: Patient Demographics, Account Activation, Results Usage, and Survey Completion Rate

Table 2 – Results of Patient Survey: PG Results feature

20

J. S. Wald et al. / Sharing Electronic Laboratory Results in a Patient Portal – a Feasibility Pilot

were opened in the first week, 60 (50%) by the end of the second week, and 70 (58%) by the 26th day.

results electronically using a secure patient portal? Our pilot experience identified the following challenges:

Fifteen (19%) of 80 patients who viewed the Results Letter sent by their physician responded to a brief survey. 13 (87%) felt it was easy to use; 10 (67%) felt they would recommend it to others; 10 (67%) preferred receiving the letter “Online only” and 4 (27%) liked receiving it “both online and printed”; a few suggested offering a “tabular” format with export to a spreadsheet. Ad hoc physician feedback was positive; and one physician asked if he could be notified when an online letter was *not* opened by the patient, so clinical staff could escalate if needed.

Clinical sensitivity. Individual physicians resist changes that they feel will create anxiety among their patients, such as an abnormal test result without proper interpretation and delivered at an appropriate time in an appropriate way. When a patient views a result online for the first time, is it empowering? Is it an alarming indicator of a new, serious problem? Will it reinforce the confidence placed in the physician, or create friction? Secure messaging, Results Letters, LTO, and other tools that enhance interpretation of results are important. Any communication approach (e.g. telephone, face-to-face, etc) can lack sensitivity, not just electronic information sharing. Also, if a less permissive provider denies their patients electronic access to results, the patients can still access them from another.

Spontaneous and solicited feedback from pilot physicians was positive. Some physicians reported that they no longer send hard copy Results Letters via postal mail, while others continue to do so. Physicians reported *no* increase in messages from their patients about clinical inconsequential results, such as an abnormal MCHC (mean corpuscular hemoglobin concentration). They reported *no* extra time expenditures as a result of the new features.

Physician workflow. Results management practices by physicians are extremely varied, even for the same physician. Physicians are notified of results by phone, pager, paper, fax, or computer. The timing of their notification varies from immediate and predictable, to delayed and unpredictable. Some physicians have staff to help them manage results, while others do not. Those who practice part-time or cover for others may not have access to results that are ready for review.

Discussion What have we learned from this pilot? Patients and their providers have long communicated about laboratory results. Patient access to their medical chart has been legally mandated for decades. Why is patient electronic access via a secure portal so important?

Systems to improve results management must address the larger context challenges, not just the technology. An important reason to offer patients direct electronic access to their results is that it prevents the unintended isolation of information from the patient, particularly when overworked physicians and staff are unable to detect and correct notification errors.

Poor communication and disrupted follow-up of abnormal results are a problem in emergency rooms [3] and ambulatory care [4,5]. Furthermore, patients during emergency, primary or specialty care are all too often unaccompanied [6,7,8] by the clinical information from their previous care, leading to poor quality. This can also be costly, especially when errors in test follow-up and interpretation are contributors to malpractice claims [9], as they commonly are.

Individual fears. The fears of individual physicians and organizational leaders can cause inaction. Despite years of experience at Partners and elsewhere, uninitiated physicians are convinced an “avalanche” of electronic messages, worried patient telephone calls, malpractice risk, and un-reimbursed tasks will befall them with the activation of online lab results.

Physicians play a role in communication breakdowns – usually despite their best efforts and due in large part to a lack of adequate “systems” in place to support error-free practice [10]. When asked directly, many physicians freely acknowledge the need to improve their processes for results management, voicing worries about delays and results-tracking failures [11].

Education and leadership are needed to support medical professionals as secure patient portals are introduced. Vigorous efforts are needed to address the fears that are unsupported by data, since they are emotionally based and may be refractory to modification through data alone. Direct physician experience is often the most effective antidote for fear.

Can communication failures be mitigated or avoided entirely through process understanding and with the help of information transparency? Many patients think so – they want access to their own clinical information, including laboratory results [12,13]. Patient safety experts also believe changes will help, such as explicit criteria for communication of abnormal results, test-tracking systems for ordering providers, and the use of information technologies [14]. Many physicians, recognizing that their usual methods of “surveillance” for clinically important information can be fragile at times, believe that patients [15] and automated systems [16] can help.

When concerns *are* supported by data, they should not be ignored. An example from our pilot experience is illustrative. Two business days from having their tests, patients have access to up to 50 different results, whether their physician has reviewed them or not. Pilot physicians are comfortable with this since results are reviewed and interpreted quickly in their practices. Different practice physicians (beyond the pilot) have questioned whether more delay is needed than two business days, since parttime physicians may not see test results for a week or

So, if patients, providers, and safety experts support transparency, what makes it so challenging to share laboratory

21

J. S. Wald et al. / Sharing Electronic Laboratory Results in a Patient Portal – a Feasibility Pilot

longer. Many physicians “batch” their review of results – waiting 3 weeks or longer to write a results letter to a patient, once all results are back (e.g. pap smears take 3 weeks). Work to address these practice variations is underway.

[2] Poon EG, Wang SJ, Gandhi TK, Bates DW, Kuperman GJ. Design and Implementation of a Comprehensive Outpatient Results Manager. Journal of Biomedical Informatics 2003;36:80-91. [3] Meza JP, Webster DS. Patient preferences for laboratory test results notification. Am J Manag Care 2000; 6(12):1297-300. [4] Kern LM, Callahan MA, Brillon DJ, Vargas M, and Mushlin AI. Glucose testing and insufficient follow-up of abnormal results: a cohort study. BMC Health Serv Res, 2006; 6:87. [5] Rojas M, Mandelblat J, Cagney K, Kerner J, Freeman H. Barriers to follow-up of abnormal screening mammograms among low-income minority women. Cancer Control Center of Harlem. Ethn Health 1996; 1(3):221-8. [6] Burstin H. "Crossing the Quality Chasm" in emergency medicine. Acad Emerg Med, 2002; 9(11):1074-7. [7] Gandhi TK, Sittig DF, Franklin M, Sussman AJ, Fairchild DG, Bates DW. Communication breakdown in the outpatient referral process. J Gen Intern Med 2000; 15(9):626-31. [8] Lauteslager M, Brouwer HJ, Mohrs J, Bindels PJ, Grundmeijer HG. The patient as a source to improve the medical record. Fam Pract 2002; 19:167–71. [9] Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA, Studdert DM. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med 2006; 145(7):488-96. [10] Institute of Medicine. Crossing the Quality Chasm: a new health system for the 21st century. Washington, DC: National Academies Press, 2001. [11] Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson AS, Bates DW. I wish I had seen this test result earlier! Dissatisfaction with test result management systems in primary care. Arch Intern Med 2004; 164(20):2223-8. [12] Connecting for Health. The personal health working group final report. Markle Foundation; 2003 Jul 1. Available from: http://www.connectingforhealth.org/resources/ final_phwg_report1.pdf. Accessed December 2, 2006. [13] Harris Interactive. Two in Five Adults Keep Personal or Family Health Records and Almost Everybody Thinks This is a Good Idea. August 2004; 4(13). Available from: http:// www.harrisinteractive.com/news/newsletters/healthnews/ HI_HealthCareNews2004Vol4_Iss13.pdf [14] Gandhi TK. Fumbled handoffs: one dropped ball after another. Ann Intern Med 2005; 142(5):352-8. [15] Smith PC, Araya-Guerra R, Bublitz C, Parnes B, Dickinson LM, Van Vorst R, Westfall JM, and Pace WD. Missing clinical information during primary care visits. JAMA 2005; 293:565-71. [16] Murff HJ, Gandhi TK, Karson AK, Mort EA, Poon EG, Wang SJ, Fairchild DG, and Bates DW. Primary care physician attitudes concerning follow-up of abnormal test results and ambulatory decision support systems. Int J Med Inform 2003; 71(2-3):137-49.

Engaging the patient. In many ways, we are exquisitely attuned to our patients. Yet, in the jumble of priorities, time pressures, reimbursement challenges, risk management, regulations, privacy concerns, budget priorities, and strong practitioner autonomy, the voice of the patient can be weak. It is critically important to engage patients directly in change efforts and to value their perspective. Patients are invited on the expert panel that recommends additional chart information to share, and patient-employees are participating on another key project. The direct involvement of our patients is key.

Conclusion Partners’ feasibility pilot for sharing a limited set of laboratory results with patients in primary care was successful as measured by usage, patient satisfaction, and physicianreported experience. Now that pilot has ended, the features are being activated for additional practices beyond the pilot. Our work identified a number of issues and challenges – some with the technology, and others due to workflow and organizational issues. Having gained experience in sharing laboratory results electronically, we plan to leverage the expert panel, lessons learned in the pilot, and patient enthusiasm to accelerate our progress toward medical information transparency. We believe that electronic patient access to their own results and other medical chart information, if provided with appropriate privacy safeguards and clinical sensitivity, offers a powerful remedy to communication breakdowns. Confidentiality – keeping clinical information isolated from others who do not need or have authority to access it – is critically important. But clinical information isolated from the patient can be a critical mistake – unhelpful, inappropriate, and potentially quite dangerous. We look forward to the challenges ahead of accelerating information transparency for patients while at the same time strengthening the physician-patient relationship, built upon caring, communication, and trust. Acknowledgments

Address for correspondence

We are grateful to Ted Eytan MD, Group Health Cooperative, Seattle, WA, USA, who provided useful guidance throughout this pilot.

Jonathan Wald, MD, MPH, e-mail: [email protected]

References [1] Wald JS, Pedraza LA, Reilly CA, Murphy ME, Kuperman GJ. Requirements development for a patient computing system. Proc AMIA Symp, 2001:731-5.

22

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

An Evolving Systems-Based Methodology For Healthcare Planning Jon Warwick and Gary Bell London South Bank University, London, UK.

ronment understand the issues they are faced with and resolve problems. To date, these approaches seem to have been largely restricted, quite understandably, to applications of systems theory and in particular soft systems methodology (SSM) either in an extended form (see for example [3, 4]) or in combination with other techniques [5]. Other more traditional operational research methods have also been used [6].

Abstract Healthcare planning seems beset with problems at all hierarchical levels. These are caused by the ‘soft’ nature of many of the issues present in healthcare planning and the high levels of complexity inherent in healthcare services. There has, in recent years, been a move to utilize systems thinking ideas in an effort to gain a better understanding of the forces at work within the healthcare environment and these have had some success. This paper argues that systems-based methodologies can be further enhanced by metrication and modeling which assist in exploring the changed emergent behavior of a system resulting from management intervention. The paper describes the Holon Framework as an evolving systems-based approach that has been used to help clients understand complex systems (in the education domain) that would have application in the analysis of healthcare problems.

In this paper we describe the Holon Framework which is itself an interdisciplinary approach derived from the systems movement and software engineering. It has had some success in teasing out key issues within ‘soft’ domains and, we argue, could provide those engaged in healthcare planning with an approach for better understanding the issues they are faced with and a means for better controlling change processes in what is often a turbulent environment.

Keywords:

Organizational complexity and ‘messes’

Healthcare Systems, Systems Thinking

All planning processes have limitations. The extent to which these limitations restrict the effectiveness of the resultant plans will, to a certain extent, depend on the nature of the organization concerned and the organizational culture. In terms of healthcare planning, a number of authors have tried to produce a taxonomy of healthcare issues and one such is that of Braithwaite, Hindle, Iedema and Westbrook [3] who characterize four deep-seated problems in healthcare and describe their occurrence in terms of a quasi hierarchical domains. The first domain is that of high level policy formulation and its implementation. Policy formulation is often highly politicized and contested and even what seems to be relatively clear policy is sometimes interpreted and actioned inconsistently. The second domain is the organizational or institutional domain in which the complex nature of interacting, collaborating and co-operating healthcare organizations which make up a national healthcare service can lead to fragmentation of policy, conflicting interpretations and goals, poor communication and a perceived lack of resources. The third domain relates to the service level where clinicians and patients interact. Issues here relate to resources and workloads, unachievable patient expectations and lack of communication between clinical groups. The fourth domain is at the individual level of one-to-one interaction between health professional and patient. Here the issues relate often to the difficult choices that must be made, bal-

Introduction In recent years there has been a growing acceptance among those working in the field of operational research that effective planning, conflict resolution, management and the control of change have become more and more difficult in domains that display ‘soft’ characteristics. Indeed, the emphasis in such situations often changes from one of ‘achieving goals’, to one of ‘understanding and exploring’ the problem situation and a number of methods have evolved to help decision makers better understand the web of complex issues with which they are faced [1]. Within the context of health informatics, and indeed of healthcare planning in general, there seems to be agreement that the issues faced by practitioners, decision makers and those involved in establishing policy are in a domain that could be described as [2]: “… more information intense, more organizationally complex, more turbulent, more paradoxical, more subjective, and more emotionally charged than other industries and businesses.” The characteristics of this description are typically those of issues situated in the ‘soft’ domain (as described in [1]) and in response there has been some interest in using ‘soft’ approaches to help decision makers in the healthcare envi-

23

J. Warwick et al. / An Evolving Systems-Based Methodology For Healthcare Planning

ances between organizational and individual work duties, low morale, work loads etc. In terms of producing effective decision making processes we can add to this list the potential problems relating to achieving ‘buy-in’ both from clinical and non-clinical staff and from senior management, the difficulty of embracing creativity and intuition within the process and the desire for plans which are sufficiently detailed so as to give clear direction but also remain flexible in response to environmental changes. Classic texts such as Mintzberg [7] document many of these issues.

tionships and control processes. Note here that the phrase ‘myths and meanings’ encompasses a wide range of descriptors and is used to contrast with ‘facts and logic’ which make up the complementary stream of enquiry. SSM refers to the importance of enabling change that is both systematically desirable and culturally feasible. Three questions arise at this point. Firstly, how will we know if the planned changes are culturally feasible? In other words, how do we uncover and explore the organizational culture that we are working within? Secondly, it may be the case that the changes are not culturally feasible within the current dominant culture so how might we perhaps influence change in the organization’s culture? Thirdly, how can we ensure that the systematically desirable and culturally feasible change is enacted? In other words, how can we control the change process once changes have been identified?

We believe, as do others working in the field of systems theory, that resolving these issues centres around the ability to understand the nature and culture of the organization and to be able to capture some of the complex behavior and beliefs of actors within the organization. This belief is, of course, not new. Ackoff [8] contends that managers are not confronted with problems that are independent of each other, but with situations that consist of dynamic, transient and complex problems that interact with each other. He calls such situations ‘messes’. Furthermore, he states [8]:

The first two questions we have addressed in other work describes later in this paper. The third is key and we shall concentrate on this since the world of systems theory, information systems and, indeed, IT in general is littered with stories of system changes, organizational restructuring and IT interventions that have failed at the implementation stage [11]. UK healthcare services are no exception to this with new IT systems failing to produce the expected benefits and massive organizational change generating unexpected consequences [12]. Part of the explanation for this can be related, no doubt, to poor implementation procedures or IT system specification, but there are also issues related to complexity which need to be addressed. For example, in relation to the implementation of IT systems within healthcare, it has been noted that [5]:

“Messes are systems of problems, the sum of the optimal solutions to each component problem taken separately is not an optimal solution to the mess. The behavior of a mess depends more on how the solutions to its parts interact than on how they act independently of each other.” The taxonomy of healthcare issues of Braithwaite, Hindle, Iedema and Westbrook briefly described above, captures some of the key problems that must be addressed if complex healthcare issues are to be addressed and which encourage the use of systems based enquiry methods. Firstly, the four domains at which the issues have been identified illustrate the range of ‘voices‘ that must be heard if a full understanding of the problem situation is to be gained. Secondly, these ‘voices’ speak with very different volume in terms of their political and organizational power. Thirdly, the range of issues addressed through the four domains is huge and includes at one extreme high level policy development and at the other the one-to-one patient-clinician interaction. Fourthly, there is a key issue of complexity which is usually thought to be addressed through the application of systems thinking ideas but is often not clearly understood by stakeholders within the context of the problem. We shall return to the issue of complexity shortly.

“Given the complexity of the context, health informatics cannot simply focus on technical or information systems aspects alone. It has to take account of their relationship with clinical and managerial processes and practices as well as deal with multiple stakeholders and organizational cultures and accompanying politics.” We would argue that this statement does not, potentially, go far enough. It may well be the case that there are complex formal managerial and organizational relationships that need to be considered, but complex systems may well have a host of informal processes and relationships that are hidden from key stakeholders and organizational charts. The influence of informal processes and relationships may only become apparent once organizational changes are made.

Much work has been carried out in recent years in developing problem structuring methods that aim to provide tools for thinking about these messes [1]. Systems thinking has its advocates in healthcare applications [9] and one approach derived from systems thinking is Checkland’s Soft Systems Methodology [10]. Soft Systems Methodology advocates two streams of enquiry which explore the facts and logic of the situation from the perspectives of those involved (logic-based enquiry) and also the myths and meanings which constitute what we (as individuals) mean by an organization and organizational culture (cultural enquiry). Cultural enquiry will include roles, norms, and values as well as political and other power related rela-

Brookfield and Smith [13] argue that there is an inherent weakness in the management maxim that ‘if you can measure it, you can manage it’. Specifically the weakness is concerned with the measurement techniques used which often assume linearity of relationships and a reliance on a priori data as a predictor of future performance. If we couple with this a recognition that we may well only have a partial understanding of the effects of system intervention (what Simon referred to as bounded rationality [14]) then predicting how a system may react to structural or environ-

24

J. Warwick et al. / An Evolving Systems-Based Methodology For Healthcare Planning

mental change and controlling the change process itself becomes difficult.

ations. We consider a Holon to be an abstract representation of a social situation that captures all problems. It is used as a framework to discover relevant issues from stakeholders’ points of view; these are organized in a layered structure.

System complexity results in only a partial understanding of the true dynamics of the system [13]. Important here are macro and micro system properties and the notion of ‘downward causation’. Downward causation is the process through which a system’s micro components adapt to macro level intervention and this adaptation can, potentially, be very unpredictable. Thus the effect of macro level managerial intervention could be unpredicted micro level changes the emergent properties of which may subsequently influence the properties of the wider system. Brookfield and Smith argue that there is:

The Holon Framework combines soft elements (Framing, Enquiry, Visioning) and hard elements (Metrication and Modeling). It addresses ‘the who’, ‘the what’, and ‘the where’ type questions for the current state S0, and generates a vision of a desired state S1. Additionally, this produces a relevant metrics programme, and the collected metrics can be used as dynamic behavior patterns. It is then possible (using modeling techniques such as system dynamics) to tackle ‘the how’, ‘the why’ and ‘the when’ type questions. The metrication and modeling stages of the framework we see as crucial in application to complex systems.

“ … a degree of uncertainty associated with intervention outcomes from a managerial perspective because the performance metrics of models of intervention (their motives, logic, organizational scope, timescales, and implementation) cannot capture easily, if at all, emergent system responses.”

Table 1 - Aims of Holon Framework Stages

Many public bodies in the UK (and this applies particularly to health and education) are subject to high levels of government scrutiny which involves target setting and the measurement of ‘quality standards’. Clearly these issues relating to our ability to measure and predict system change are crucial in understanding how systems will respond to management intervention. It is difficult to predict system responses to change however systematically desirable and culturally feasible they might seem to be. Thus high level policy formulation in the earlier taxonomy may have unpredicted effects at the lower levels relating to clinical groups and individual personal transactions. Brookfield and Smith go on to contextualize their argument within the UK healthcare environment and use the introduction of Payment by Results [15] as part of the UK healthcare system as illustration.

Stages Framing

This stage has a number of objectives among which are that the stakeholders are identified and become familiar with the framework and that the investigators gain a broad understanding of the situation so that relevant holons (and sub-holons) can be identified and labelled.

Enquiry

This stage aims to identify the problems as perceived by the stakeholders.

Visioning

This stage attempts to collate various problems into themes to be addressed. These can be linked with a sub-holon hierarchical level.

Metrication

This stage analyses the themes and links the emergent problems with the appropriate hierarchical level. Metrics are generated to characterize specific problems.

Mathematical Modeling

This stage aims to analyse the data further using appropriate modeling techniques – for example a system dynamics model might be used to explain the situation of concern.

Action

This stage aims to facilitate change having achieved understanding of the area of concern

The Holon Framework The Holon Framework [16] takes from the systems movement the notions of systematic wholeness and systematic analysis and combines aspects of SSM with the Goal/ Question/Metric (GQM) method [17]. The GQM method is an integral part of a goal-oriented measurement initiative [18] which aims at the continuous improvement of software processes and products and which we now adapt for use with SSM in ‘soft’ contexts. The GQM method is a mechanism for defining and interpreting a set of operational goals using measurement which address the specific needs of a project and organization. Thus, within its process model the Holon Framework has two modes of working in that the ‘soft’ part relates to improvement by addressing questions associated with ‘the what’, ‘the where’ and ‘the who’ while the ‘hard’ part relates to control by addressing ‘the how’, ‘the why’ and ‘the when’. Checkland argues that researchers who apply systems concepts to investigate social situations face difficulties because these situations are never clearly defined. He prefers to use the word ‘Holon’ rather than ‘system’ as it highlights a distinctive approach to investigating such situ-

Stage Aims

While it is essential to generate a vision of future activity and to use the associated metrics to measure our progress towards the vision once action has been undertaken, it is

25

J. Warwick et al. / An Evolving Systems-Based Methodology For Healthcare Planning

also crucial to understand, through the modeling stage, how the system is evolving and the emerging dynamics that are being exhibited via the data collection process. Morecroft argues [19] that dysfunctional behavior can be prevented by investigating the systemic consequences of various decisions so that there is advantage to be gained by turning to traditional modeling techniques such as system dynamics. Thus as the system moves, perhaps, to a new equilibrium point as a result of the management intervention we can observe and understand this process better through the modeling and metrication stages.

culture and perhaps to influence changes in culture by using the framework [21]. We have also been able to demonstrate the organizational learning opportunities that are uncovered by use of the framework within the domain of higher education [22] a domain which has many of the same characteristics as the healthcare domain and, indeed, many other complex human activity systems.

Conclusions The healthcare domain is characterized by many of the same problems that afflict other complex systems in the management and business domains. What is different, however, is the critical nature of many of the decisions that need to be made whether these are part of the care process itself, or related to improving delivery, care and management of services. This emphasizes the need to monitor and control the effects any of intervention directly resulting in system change. There are many interested parties (representing the public, health professionals, politicians and commerce) and, as stated in [9], there needs to be a:

The most important traits of this framework are: • •

it provides an holistic view of a situation; the use of a soft methodology to enable the capture of the stakeholders’ point of view; • the researcher’s role is that of a facilitator; • the monitoring and controlling of the effects of bounded rationality and seeking to uncover new emergent system behavior; • an enhanced understanding of the problem situation by the client group and the development of a desirable and feasible vision; • the creation of a relevant metrics programme; • production of the ‘best solution’ to achieve the vision given cost and other environmental constraints; • the continuous use of system dynamics models to examine various ‘what-if’ scenarios and enhance understanding of the effects of macro management intervention. The Framing and Enquiry stages are means of exploring issues, drawing out themes, boundaries and experiences that the stakeholders feel are important within the problem context. These first two stages encourage a thorough examination of the current state, S0, resulting in its definition. Next we move to Visioning in which the client group explore a vision of the future that they feel is achievable and desirable. The vision will be expressed in terms of the holon structure used throughout the enquiry and may be expressed formally in terms of root definitions. It is important though that the discussion of S0 and the vision, S1, are linked through issues and problems. The stakeholder group should identify the critical issues and problems which require resolution if movement towards the vision is to be achieved. The issues and problems will generate goals, questions and metrics. The Metrication stage allows the stakeholders to learn more about the problems and issues in S0, and the subsequent Metrics Collection stage enables them to measure their progress towards S1. This is followed by the Action stage in which modeling is undertaken to clarify the processes which can effect movement from S0 to S1.

“… judicious consideration and balancing of cultural, managerial, clinical, technical, legal, political and economic issues.” Soft Systems Methodology has been used to some effect to help decision makers understand some of the issues, but we would argue that it has some shortcomings in application to healthcare services that the Holon Framework can help to overcome. In particular we feel that: •

The Holon Framework captures and structures different perspectives of the problem situation in much the same way as SSM would do. To that extent, it has the capacity to allow participant to explore both the systemic issues and the cultural (indeed personal) issues that reflect their actions within the system; • The framework can also allow exploration of organizational culture and provides a framework within which organizational learning can take place; • The framework allows the development of a vision and, furthermore, the identification of metrics that will measure progress from the current position towards the vision; • Metric data collection provides reference mode behavior which can be used as a basis for system dynamics modeling so that underlying dynamic effects within the system can be uncovered. This provides a means of measurement of progress towards the vision and also a window on the system through which unexpected properties emerging from the system may be observed and investigated. These unexpected changes bought about as result of management action will result in learning taking place as the client group seek to understand the nature of this difference between expected and observed outcomes. The modeling and metrication stages provide stakeholders with the means to uncover system changes at the micro level as they occur and begin to produce emergent effects. Stakeholders are not bound to traditional linear models or a

We have applied the emerging Holon Framework to situations characterized as ‘messes’ and have been able to demonstrate the value of the approach [20]. In addition, previous work with the Holon Framework with regard to the first two questions posed earlier has encouraged the view that it is possible to capture aspects of organizational

26

J. Warwick et al. / An Evolving Systems-Based Methodology For Healthcare Planning

priori data in trying to assess system response to any macro managerial intervention, but can use the emerging metrics to confirm beliefs or to uncover variance from expectation which might lead to significant learning opportunities regarding their own beliefs and actions, or perhaps those of others.

[9] Fahey D, Carson E, Cramp D, Gray J. Information communication technology in public health: the role of systems modeling. Health Informatics Journal. 2003; 9 (3): 163-181. [10] Checkland PB. Systems thinking, systems practice. England: John Wiley and Sons; 1981. [11] King J. Survey shows common IT woes. Computerworld. 2003, June 23, [Available on the internet, cited 2006 November 3]. Available from: http:// www.computerworld.com/managementtopics/management/ story/0,10801,82404,00.html [12] CBR Staff. Updated – NHS IT system crisis: the story so far. Online Computer Business Review. 2006 August 30. [cited 2006 November 3]. Available from: http:// www.cbronline.com/article_cbr.asp?guid=35AC0F096C33-4D0E-AC2C-D912E2AA6042 [13] Brookfield D, Smith D. Managerial intervention and instability in healthcare organizations: the role of complexity in explaining the scope of effective management. [paper on the internet], The University of Liverpool Management School, Research Paper Series No. 2006/27, [cited 2006 October 27]. Available from: http:// www.liv.ac.uk/managementschool/working%20papers/ wp200627.pdf [14] Simon HA. Rationality and decision making, In Models of Man. New York: John Wiley and Sons; 1957. [15] NHS. Delivering the NHS plan. [monograph on the internet]. UK; HMSO: 2002 [cited 2006 October 27]. Available from: http://www.nhshistory.net/ deliveringthenhsplan.pdf [16] Bell GA, Warwick JP. Towards establishing the use of holons as an enquiry method. International Transactions in Operational Research. In press 2007. [17] Basili VR, Caldiera G, Rombach DH. GQM paradigm. In: Marciniak JJ, editor. Encyclopedia of software engineering. New York: John Wiley and Sons; 1994. [18] Boehm BW, Rombach HD, Zelkowitz MV, editors. Foundations of empirical software engineering: the legacy of Victor R Basili. Berlin Heidelberg,: Springer; 2005. [19] Morecroft JD. System Dynamics: Portraying bounded rationality. Omega. 1983; 11 (2): 131-142. [20] Bell GA, Cooper MA, Warwick JP, Kennedy MS. The Holon Framework: process improvement and control for higher education. In: Groccia JE, Miller JE, editors. On becoming a productive university. Bolton: Anker Publishing Company; 2005. [21] Warwick J, Bell G, Kennedy M. Exploring myths and meanings in higher education planning. In: Williams D, Baryamureeba V, editors. Measuring computing research excellence and vitality, Kampala: Fountain; 2005. p. 143-158. [22] Warwick J, Bell G. Uncovering opportunities for learning in higher education planning. Proceedings of the World MultiConference on Systemics, Cybernetics and Informatics; 2006; Orlando, Florida, USA.

One strength of the Holon approach lies in discovering the complex interrelationships among the issues at play within the ‘mess’, and this may be explored using, for example, system dynamics. As an example from one education case study [20] admitting poorly qualified students increases the pressure on student support resources, decreases course quality, increases the burden on academics and yields an increased withdrawal rate among students. This results in the need to find additional students the following year. Dealing with poorer students reduces the academic productivity of staff, which reduces the reputation of the institution and the course. A spiraling negative loop structure begins to dominate and management intervention is required coupled with appropriate resource allocation to reduce the dominance of this loop. Such actions can be explored systematically and dynamically using appropriate modeling methods within the Holon Framework. We argue, therefore, that the Holon Framework would have considerable advantages over other systems approaches in application to the healthcare sector both in helping to understand and to deliver system change.

References [1] Rosenhead J, Mingers J, editors. Rational analysis for a problematic world revisited. England: John Wiley and Sons; 2001. [2] Plummer A. Information systems methodology for building theory in health informatics: the argument for a structured approach to case study research. Proceedings of the 34th Hawaii International Conference on System Sciences; 2001; Hawaii, USA. [3] Braithwaite J, Hindle D, Iedema R, Westbrook J. Introducing Soft Systems Methodology Plus (SSM+): why we need it and what it can contribute. Australian Health Review. 2002; 25 (2) : 191-198. [4] Kalim K, Carson E, Cramp D. A soft systems methodology analysis of the NSF for diabetes. Proceedings of the 6th International Conference of Strategic Issues in Health Care Management; 2004; University of St. Andrews, UK. [5] Atkinson C, Eldabi T, Paul R, Pouloudi A. Integrated approaches to health informatics research and development. Logistics Information Management. 2002; 15 (2): 138-152. [6] Davis R, Roderick P. Planning resources for renal services throughout UK using simulation. European Journal of Operational Research. 1998; 105 (2): 285-295. [7] Mintzberg H. The rise and fall of strategic planning. New York: The Free Press; 1994. [8] Ackoff RL. The future of operational research is past. Journal of the Operational Research Society. 1979; 30 (2): 93-104.

Address for correspondence Jon Warwick, Faculty of Business, Computing and Information Systems, London South Bank University, Borough Road, London SE1 0AA, United Kingdom.

27

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Model-Centric Approaches for the Development of Health Information Systems Mika Tuomainena, Juha Mykkänena, Heli Luostarinena, Assi Pöyhöläb, Esa Paakkanena b

a HIS R&D Unit, IT Services Centre, University of Kuopio, Finland Shiftec, Department of Health Policy and Management, University of Kuopio, Finland

ments, are economically implementable and promote information and knowledge longevity [3]. Models contain observations, requirements and assumptions about the current or prospective state and solutions of a specific domain. Numerous approaches, methods, notations and tools have been suggested and used in modeling. However, an analysis of different modeling approaches is needed for identifying a suitable approach for given development needs of HIS. It is also necessary to identify which aspects are covered by different modeling approaches and which features of different modeling approaches support the specific needs of healthcare.

Abstract Modeling is used increasingly in healthcare to increase shared knowledge, to improve the processes, and to document the requirements of the solutions related to health information systems (HIS). There are numerous modeling approaches which aim to support these aims, but a careful assessment of their strengths, weaknesses and deficiencies is needed. In this paper, we compare three model-centric approaches in the context of HIS development: the ModelDriven Architecture, Business Process Modeling with BPMN and BPEL and the HL7 Development Framework. The comparison reveals that all these approaches are viable candidates for the development of HIS. However, they have distinct strengths and abstraction levels, they require local and project-specific adaptation and offer varying levels of automation. In addition, illustration of the solutions to the end users must be improved. health information systems, modeling, information systems development

In this study, we compare three model-centric approaches in relation to the development and integration of health information systems. The aim is to identify the strengths and weaknesses of different types of approaches and to bridge the gap between the healthcare knowledge and software development. The comparison is based on the use of a conceptual framework, the literature, and our experience of applying these approaches in service-oriented HIS development and integration.

Introduction

Materials and methods

The understanding of healthcare activities is central for the development and integration of health information systems (HIS). The specification and improvement of healthcare processes is pursued in relation to complex activities which involve lots of communication in the changing healthcare environment. To support these needs, health care actors are increasingly applying methods and best practices of business process development and enterprise modeling for analyzing care activities and clinical routines, building patient-centric processes and improving the knowledge and understanding of health care organizations [1, 2]. Models are used to illustrate, identify improvements or simulate organizational processes and individual activities in healthcare. Modeling also supports the specification of requirements and creates a basis for the implementation or integration of health information systems.

In this section we introduce three model-centric development approaches and a framework for the analysis and comparison of these approaches. For this analysis, we have selected three very different model-centric approaches which have been applied varyingly in our projects and which can be applied in the development and integration of health information systems.

Keywords:

An analysis framework for modeling approaches To provide a systematic approach for this study, we are using a structured framework for assessing different modeling approaches. The main purpose of each approach is a viable starting point for such an assessment. In addition, we especially consider the support provided by each approach in different phases of an information systems development (ISD) value chain [4]. This chain initiates from the understanding of the target domain. It then proceeds to the specification of goals and requirements and to the design and development of solutions. The solutions are then used in the healthcare environment. To assess the completeness of the approach, we consider distinct aspects of information systems [5]: structure (which concepts and entities are included), function (which tasks are performed), and behavior (when and using what kind of

Modeling is used widely in software engineering and information systems development projects. Models are produced to aid the communication between users, healthcare managers, architects, designers and developers. Models add clarity to the application domain and the design, providing formalization and abstraction to the understanding of problems and solutions. Good models support complexity, cope with changes in the require-

28

M. Tuomainen et al. / Model-Centric Approaches for the Development of Health Information Systems

formations between PIMs and PSMs can also be repeated: each new platform provides new features to the implementation. Thus there can actually be several levels of PIM and PSM descriptions and several definitions of a platform even within one system.

interactions the tasks are performed). Each aspect has different features in different phases of the ISD chain. We argue that a modeling chain which supports the traceability of these aspects in different phases reduces the distance between healthcare knowledge and software development [4].

In an MDA specification of a system, the requirements in the CIM should be traceable to the PIM and PSM constructs that implement them, and vice versa. An MDA tool might transform a PIM directly to deployable code, without producing a PSM that would be visible to the user. This requires models on a very detailed level or many tool-specific assumptions. In practice, the application of MDA requires specific interpretations which are not provided in the MDA specifications. For this study, we have used the approach of an MDA toolkit [7] which defines the use of UML models on CIM, PIM and PSM levels and the phases of the software development lifecycle.

Other aspects which improve the traceability should also be considered in relation to HIS [4]. Due to the information-intensive nature of healthcare, special emphasis should be paid to semantic elements and entities in different phases to achieve a shared understanding of complex health information and processes. In addition, the illustration of solutions to the end users already in the early phase of the process is beneficial. Furthermore, the support for accurate, consistent, atomic and unambiguous documentation of the requirements promotes the utilization of tacit knowledge and shared understanding. In addition, we consider how definitive the approach is, or how many local and project-specific extensions are needed or allowed, and if there are guidelines to produce the results on a specific abstraction level. Furthermore, the visibility of end users or process participants in the models is considered. Traceability and productivity can also be improved by providing automatic transformations or generation of implementations from the models. Finally, the specific usage contexts and the dissemination situation of the approach are considered.

Process Modeling with BPMN and BPEL Business Process Modeling Notation (BPMN) [8] is a graphical notation for process modeling. It aims to be understandable by different users from business analysts to developers and people who manage and monitor business processes. Business Process Execution Language for Web Services (BPEL) [9] provides an XML-based language for the specification of business processes and interaction protocols. BPMN specification contains a mapping to BPEL. Hence, BPMN and BPEL can be used together for the modeling and web services-based implementation of processes.

Model-Driven Architecture (MDA) The Model-Driven Architecture (MDA) by the Object Management Group (OMG) supports software development through modeling techniques such as Unified Modeling Language (UML) [6]. Its abstraction levels separate logical and technological models: computation independent (CIM), platform independent (PIM) and platform specific (PSM). Three primary goals of the MDA are portability, interoperability and reusability through architectural separation of concerns.

There are three basic types of BPMN process models. The private processes are internal to one organization and often describe accurate workflows. Private processes can be transformed to executable BPEL descriptions. Public processes describe the interactions and messages between the private processes and other participants, displaying the communication activities and the flow control mechanisms. Collaboration processes in BPMN describe interactions between two or more business entities (public processes). Collaboration process can be potentially mapped to collaboration protocols of electronic business such as ebXML or RosettaNet. BPMN can be extended using BPMN artifacts that provide extra information in modeling tools. Such artifacts can be used to support the requirements of a vertical domain such as healthcare. Although the graphical diagram is the most notable part of BPMN, non-graphical attributes play an important role when BPMN models are mapped to execution languages.

A computation independent model (CIM) shows the system in its operation environment. The structure of the system, however, remains hidden or undetermined. The domain model of CIM helps in presenting exactly what the system is expected to do. A CIM often consists of several UML models, some providing more detail than others, or focusing on particular concerns. The primary user of the CIM is the domain practitioner, and it aims to bridge the gap between domain and IT experts. CIM is used as an aid to understanding the problem, but also as a source of a shared vocabulary for use in other models.

BPEL descriptions aim to support the automated execution of processes, defining the process from the standpoint of one participant only [9]. External partners of the process are defined as web services and contacted through interfaces described using WSDL (Web Services Definition Language). The external view of a BPEL description is also a web service. Public BPMN processes can also be transformed to abstract BPEL descriptions which describe interactions and hide other than communication-related aspects of the executable process. The BPEL specification does not include a graphical notation, but the notations have been mostly tool-specific. BPMN has been suggested as a generic graphical notation for BPEL [8].

A platform independent model (PIM) describes the system without showing aspects which are specific to the platform, or the technology that is used to realize the system. A PIM might consist of various enterprise, information and computational models. A PIM is suited to one or more architectural styles. Concepts such as technology-neutral virtual machines can be used for platform independence in PIMs [6]. A platform specific model (PSM) is produced by transforming a PIM. It combines the specifications in the PIM with details which specify how the system uses a particular type of platform to provide an implementation [6]. Trans-

29

M. Tuomainen et al. / Model-Centric Approaches for the Development of Health Information Systems

dards. Central information design models include domain information model (DIM), constrained information model (CIM), and serialized constrained information model. In addition, sequence and collaboration diagrams are used to describe the needed interactions and application roles and to link them to specific messages. The reuse of design model components and the harmonization of design models with the RIM are also considered. In the standard profiling phase, the specified standard can be profiled: its elements are annotated, constrained, extended, or left unchanged. This leads to a set of specification profiles and conformance statements. The specifics of the information exchanged, the dynamic model and the acknowledgement responsibilities are defined. In addition, this phase considers user documentation and Implementation Technology Specifications (ITS) such as XML schemas for the developers.

HL7 Development Framework (HDF) The Health Level 7 (HL7) Development Framework Methodology Specification (HDF) [10] is a framework for analyzing, designing, and documenting the processes, tools, actors, rules, and artifacts of the HL7 version 3 standards development. The approach is based on the use of a model-driven methodology and the derivation of specifications and other work products from a common set of reference models. It supersedes the earlier Message Development Framework [11]. An essential element in the HL7 Version 3.0 and the HDF is the HL7 Reference Information Model (RIM). All information models in the HL7 version 3 standards are based on the RIM and follow structural vocabulary specifications. The RIM is described using UML class diagram notation [11]. In addition, other modeling practices of the HDF apply some UML models directly. The HDF UML profile also defines extensions to the meta-model of the UML.

Results

The development process of the HDF has seven phases [10]: project initiation, requirements analysis and documentation, specification modeling, specification documentation, specification publication, specification approval, and standard profiling. The requirements analysis and documentation, specification modeling and standard profiling phases are within the scope of this work. Several diagram types, models and design and documentation tools are utilized in these phases. The requirements analysis and documentation phase produces artifacts which describe the healthcare business in a given domain using its terminology. In this phase, the business processes are described using storyboards, use cases and activity diagrams. In addition, spreadsheets and class models are used for message contents and domain analysis models (DAMs). The business rules are described by defining relationships, triggers and constraints to the exchange of data, and state diagrams. The domain experts usually develop these specifications and a glossary which are then used by technology experts for developing HL7 v3 messaging specifications. In the specification modeling phase, the reference models are constrained into design models, based on the artifacts from the previous phase. Some of the artifacts produced in this phase can be balloted as stan-

The analysis of the three model-centric approaches is presented in Figure 1 and Table 1. Figure 1 relates the suggested modeling artifacts of each approach to the ISD value chain. Table 1 organizes the other considerations of our framework. In addition to these considerations, some specific features of each approach were observed during the analysis. The MDA is an overarching approach. It aims to support all aspects of application development in a way which isolates technology changes from the logical solutions. The other approaches in this comparison could be included in an MDA-based approach. The MDA has been used in very different ways, and the meta-level foundations for the MDA are not required in various tools or projects. Hence, there is no "one MDA" but its application depends on situation-specific needs. Process modeling with BPMN excludes many aspects of the holistic systems development, such as organizational structures and resources, functional breakdowns, data and information models, strategies and business rules [8]. Users are not especially considered beyond "user tasks" in BPMN. On the other hand, BPEL engines provide valu-

Figure 1 - The modeling artifacts of the three approaches in relation to the information systems development value chain.

30

M. Tuomainen et al. / Model-Centric Approaches for the Development of Health Information Systems

Table 1 - Summary of the comparison of the three model-centric approaches. Consideration

Model-Driven Architecture

Business Process Modeling with BPMN and BPEL

HL7 Development Framework

Main purpose

model-based software development, insulation of business from the changing technology

process modeling notation and process execution

model-based production and management of interoperability standards

Structure & information

class and other UML diagrams on CIM, PIM and PSM levels

BPMN: little description, BPEL: through XML/WSDL interfaces

RIM, domain analysis and information models, constrained models, XML schemas for messages

Functionality

use case, class and other UML diagrams

on process level only, internal functionality in private BPMN

interaction names, receiver responsibilities for messages

Interactions

sequence, activity and other UML diagrams

abstract and collaboration BPMN: generic, BPEL: specific

triggers, application roles, interactions, UML diagrams

Main users of the method

CIM: domain practitioners, PIM, PSM: IT experts

business analysts, business process owners and designers

accurate user roles specified for different phases

Semantic specification details

CIM focuses on information viewpoint, including vocabulary, more specific in PIM and PSM

little emphasis on information or semantics other than parameters of interfaces

glossaries, data types and vocabularies specified for more rigorous information support

Illustration to the end users

CIM: vocabulary, environment, functionality

BPMN: visualized processes can be examined

storyboards, use cases and activity diagrams

Requirements quality

documentation identified but not specified for the requirements

many requirements are implicit in the process descriptions

on generalized level (for all standards), but clearly traceable

Definitiveness

very loosely defined, accurate methods and tools needed

not in notations and languages but in tools and methods

accurate phases and outcomes specified, messaging presumed

Abstraction levels

CIM, PIM and PSM, lots of freedom within levels

BPMN: private, public and abstract; BPEL: execution level

models and tools defined for all specification phases and steps

Visibility of users / health professionals

evident in use case models, CIM and PIM emphasize communication with users

BPMN: user interactions, lanes, pools, activities; BPEL: no distinction between user and system steps

storyboards, use cases, activity diagrams, indirectly through application roles and triggers

Automated implementation

transformations of models to executable code emphasized, various different possibilities

BPEL can be generated from private BPMN; BPEL is executed in workflow engines

implementation based on the specifications, little automation due to local variability

Dissemina-tion

promoted widely, solutions mostly tool-specific

promoted for business process healthcare-specific, mostly used management, increasing support for standardization only

currently under revision, new focuses on process modeling and management, relationships dynamic and static model approaches being refined to other aspects remain undefined standards is evident: the requirements are generalized, and able information about the activities related to the it is not always easy to find the origin of the solutions. processes. As the user interactions are typically required as HDF does not provide many guidelines or automated tools a part of the processes, the approach can be complemented to support the implementation. with models for user interface design. Other relevant aspects

requires more detailed methods, has been interpreted varyingly in different tools and methods

The HDF focuses on the information aspects, and the functionality is mostly implicit in the application roles and their responsibilities. The dependencies of the related interactions and triggers are illustrated in some domains, but they are not always specified. The HDF process has clear participation roles, but the requirements dilemma of

Discussion The three modeling approaches clearly support the documentation and communication purposes and shared understanding in the development of HIS. However, none

31

M. Tuomainen et al. / Model-Centric Approaches for the Development of Health Information Systems

of the studied approaches covers all the necessary aspects, or they do not provide a detailed support for different phases of the ISD chain. The description of processes, relationships and activities in healthcare is supported by clear notations and abstraction levels in the modeling approaches, but models do not generally suffice to provide atomic or unambiguous requirements. Furthermore, the reference from the solution models to the actual needs of the healthcare domain easily remains unclear.

specificity. They could be included in a comparison using the specified analysis framework.

In particular, improvements or local solutions are needed in the illustration of the solutions to the end users and stakeholders, the identification of aspects which are not covered by the approach and in the selection of graphical or textual notations for these aspects. The graphical models can always be introduced to the users, but this does not illustrate all the necessary aspects to them. In addition, the accurate specifications of the semantic aspects and the careful referencing to the needs and requirements are mostly left to the users of the approach.

[1] Staccini P, Joubert M, Quaranta J-F, Fieschi D, and Fieschi M. Modeling health care processes for eliciting user requirements: a way to link a quality paradigm and clinical information system design. Int J Med Inf 2001: 64 (2-3): 129-142. [2] Szirbik NB, Pelletier C, and Chaussalet T. Six methodological steps to build medical data warehouses for research. Int J Med Inf 2006: 75(9): 683–691. [3] Rubin KS, Beale T, and Blobel B. Modeling for Health Care. In: Demetriades JE, Kolodner RM, and Christopherson GA, eds. Person-Centered Health Records Towards HealthePeople. Springer, 2005; pp. 125-146. [4] Mykkänen J, Minkkinen I, Pöyhölä A, and Riekkinen A. Improving Traceability of Functional Requirements to Information Needs and Applications in Healthcare. Abstract. In: Doupi P, ed. Proceedings of the 6th Nordic Conference on eHealth and Telemedicine - NCeHT2006. Helsinki: Valopaino Oy, 2006, pp. 183-186. [5] Iivari J. Object-oriented information systems analysis: A framework for object identification. In: Shriver BD, ed. Proceedings of the 24th Annual Hawaii International Conference on Systems Sciences, Vol. II. IEEE Computer Society Press, 1994; pp. 205-218. [6] Miller J, Mukerji J, eds. MDA Guide version 1.0.1. OMG document omg/2003-06-01. OMG, 2003. [7] Fado DH. A review of the MDA toolkit. MDA Implementers' workshop, May 17-20, Orlando. OMG, 2004.

Acknowledgments This work is part of the SerAPI project, funded by the Finnish Funding Agency for Technology and Innovation TEKES grants no. 40251/06, 40353/05 and 40437/04, together with a consortium of companies and healthcare organizations.

References

The most notable differences in the studied approaches are related to the level of detail in the information models, functionality definitions, support for automated implementations and the definitiveness of the approach. These differences mainly stem from the different purposes of the approaches. All the approaches, however, have identified the distinction between the domain and IT expertise in the modeling efforts. The many applications of the MDA to various specific needs make it an attractive approach as a reference framework, but also require detailed refinements which are not commensurable. The process modeling with BPMN and BPEL provides a lightweight and clear approach for endto-end specification of processes, but leaves many aspects of the solutions unspecified. The HL7 Development Framework naturally provides the most advanced support for healthcare-specific requirements. It also emphasizes semantic aspects and provides most support for different phases of the ISD process. However, it focuses on standardization and messaging, and does not provide advanced automation or clear functional specifications.

[8] Business Process Modeling Notation Specification. OMG document dtc/06-02-01. OMG, 2006. [9] Thatte S, ed. Business Process Execution Language for Web Services version 1.1. Accessed 20 Nov 2006. http:// dev2dev.bea.com/webservices/BPEL4WS.html. [10] HL7 version 3, September 2006 ballot package, HL7 Development Framework. Health Level Seven, Inc, 2006. [11] Beeler GW. HL7 Version 3 - An object-oriented methodology for collaborative standards development. Int J Med Inf 1998: 48 (1-3): 151–161.

Conclusions The compared approaches are all viable modeling candidates for HIS. Their distinct scopes and strengths guide the selection according to different requirements. The approaches are extensible and non-exclusive. Besides the main scope, the selection depends on how definitive a guidance for the specification process and what kind of automation is desired.

Address for correspondence Mika Tuomainen University of Kuopio, IT Services Centre, HIS R & D Unit P.O.B. 1627, Fin-70211 Kuopio, Finland [email protected]

All the approaches, however, require additional semantic definition and the illustration of the solutions to the end users. The approaches should be locally combined or complemented with the documentation of these aspects. In the future, accurate ontology-based approaches, user interface illustrations or domain-specific modeling extensions should be included in them. Many additional approaches such as two-level information modeling and various domain-specific modeling approaches have also been suggested to support accurate semantics and domain-

32

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Software Engineering Principles Applied to Large Healthcare Information Systems A Case Report Fabiane Bizinella Nardona, Lincoln de A Moura Jr.a a

ZILICS – Health Information Systems, São Paulo, Brazil

Abstract

help us to decrease the rate of software failure in healthcare. In this spirit, this paper describes how the public healthcare information system of São Paulo city was built from a software engineering point of view, aiming to show the techniques adopted that allowed to deliver one of the largest healthcare information systems ever built on time and on-budget.

São Paulo is the largest city in Brazil and one of the largest cities in the world. In 2004, São Paulo City Department of Health decided to implement a Healthcare Information System to support managing healthcare services and provide an ambulatory health record. The resulting information system is one of the largest public healthcare information systems ever built, with more than 2 million lines of code. Although statistics shows that most software projects fail, and the risks for the São Paulo initiative were enormous, the information system was completed on-time and on-budget. In this paper, we discuss the software engineering principles adopted that allowed to accomplish that project’s goals, hoping that sharing the experience of this project will help other healthcare information systems initiatives to succeed.

The City of São Paulo healthcare information system São Paulo is one of the largest cities in the world, with 10.3 million people in the city and some 18 million in the Metropolitan Area.

Healthcare information systems, software engineering, electronic health record.

In June 2003, São Paulo Public Health came to operate as a “full managed-care” city, which means that resources from the National Health Fund are transferred directly to São Paulo City Department of Health (SPCDH) on a capitation basis. In exchange, the city agrees to meet certain production and quality goals, and has to send monthly reports that allow the Ministry of Health to assess if those criteria are being met.

Introduction

By becoming a “full managed-care” city, São Paulo Dept of Health came to manage the following yearly figures:

Keywords:

Several studies show that most software projects fail. An important survey in the past pointed out that only 16.2% of the software projects are completed on-time and on-budget [1]. If a large organization is involved, as it is the case of most public healthcare information systems, statistics shows that only 9% of their projects will finish on-time and on-budget. Due to its inherent complexity, healthcare projects face even higher risks and there are many reports of failed healthcare information systems. Recently, a report on the NHS software initiative, one of the most ambitious public healthcare information systems, described that the project has been failing on delivering what it was promised, with some of its contractors, formed by large and reputable companies, giving up and leaving the project [2]. In contrast, large amounts of money are spent on such projects every year, with a low success rate.

• 10 million primary care consultations • 8.5 million specialized consultations • 550 thousand hospital admissions São Paulo decided to invest in an Information System that would support Patient Flow Control and provide an Ambulatory Electronic Health Record System [4]. The core conditions that underlined the project can be summarized as: •

• • •

As suggested by Robert Grady in his book [3], as in medicine, great progress occurred in a relative short time when the rules against dissection were eased, we must experience a similar renaissance period in software development, making software autopsies to gather information that will

33

The information system would not only be fully compliant with all National Standards; but also help to promote them; Open standards and open source-code should be used at all levels, whenever feasible; Whenever possible the project should use the results of previous projects, existing technologies and concepts; São Paulo City would receive source-code and consulting support from the Health Ministry and, in return, would send back to the Ministry all deliverables from the project;

F. B. Nardon et al. / Software Engineering Principles Applied to Large Healthcare Information Systems A Case Report

•

Finally, and most importantly, the system should be fully integrated and provide a framework for continuously embodying new functions in an easy and natural way. The requirements and concepts summarized above were described at length in a Term of Reference prepared by São Paulo City Department of Health. The Term of Reference defined 4 major sub-projects that should result in one and only one Information System:

The actual numbers of SIGA deployment as of November 6 to 12, 2006 as informed by São Paulo City Health Department are described in Table 1, below. From a software engineering point of view, the project had almost all the risks that usually take projects to failure: the requirements were not well defined, the scope was large, the timeframe to deliver the first version of the project was very short (about 9 months), the project team was huge (about 80 people) and informaticians had different skill levels in the technology involved, ranging from beginners to experts. Most of the team members had never worked together before and the team was not all in the same geographic location.

Municipal health register, whose objective is to handle and process the identification data for health care users, workers and organizations, as well as the relationships among them. The Register is the prime data source for all other modules, as no operation can be carried out unless its actors are registered.

Table 1 - SIGA Saúde production, October 2006

This subsystem also stores and processes all standard vocabularies in use within SUS. All data within the Municipal Health Register are fully compliant with SUS.

Item Primary Care Units Using SIGA Saúde

Patient flow control handles all requests for health care services (consultations, procedures, inpatients admission and emergency) and finds the best possible match, based on criteria such as budget, distance, availability and waiting time. This module also processes authorization requests for high-cost high-complexity procedures. This is standard-procedure under SUS. It also handles exceptions, i.e. whenever resources use exceed predefined limits or are unavailable, an accredited doctor handles the exception, either by extending the budget, finding available resources or holding the request for some time, if that is the case.

Primary Care consultations scheduled Specialized Consultations scheduled High-Cost Procedures Requests Processed Patients Registered

Amount 372 1,657,023 35,250 223,225 8,357,863

Despite all odds, the team succeeded in delivering the system on time. As shown in the timeline of figure 1, the contract was signed in January of 2004, it took about two months to assemble the team and define the use cases that would be implemented. In March 2004 the first use cases started to be specified. The implementation of the software started in May 2004, and in September of that year the information system was deployed and in production with all the features planned completed. In about 4 months, more than 2 million lines of code were produced and about 300 use cases were implemented. Since then, new features and improvements have been added to the system.

This module also processes patient flow control within the health care unit and copes with waiting lists, either local or in other reference layer. Ambulatory electronic health record collects an essential dataset from the encounter and triggers related actions, such as notifying diseases or work-related diseases, when such conditions are met. Role-based access control system is a single sign-on system that identifies the user and its user profile, thus enabling or disabling access to system’s functions. Of course, all users have to be recorded in the Register, before being authorized to access the system. Initially, some 40 profiles were defined. Through system usage the Dept of Health decided to delete some and created other profiles, totaling more than 60 profiles currently. To develop the project, SPCDH hired several companies that, under its management, focused on specific subprojects or on threads such as hosting, communications, equipment, software development, training and support.

Figure 1 - The project timeline The following sections will describe the methods and techniques used for achieving these results, in the hope to give an insight on how software engineering principles applied to large health care information systems can be a key success factor for any large project.

The resulting information system came to be known as “SIGA Saúde”, which means “following health” and is an acronym for “Integrated System for Health Care Management”. As of this writing, SIGA Saúde is in use in 372 primary care units for a) registering patients, workers, health units and their services; b) scheduling local appointments.

34

F. B. Nardon et al. / Software Engineering Principles Applied to Large Healthcare Information Systems A Case Report

When a new component was developed, all the team was automatically informed of it, avoiding duplication by lack of knowledge on the components available in the component library.

Methods In a large project such as São Paulo City’s Health Information System, it is important to have the methods and the architecture of the system well defined before the first requirement is captured and the first line of code is written. Our strategy for that project was based on six pillars: • • • • • •

In the few situations where we identified a feature implemented twice, a “refactoring” process was made on the system to eliminate the duplicated component.

Well-defined software development process; Component-driven software development; Well-defined project guidelines and coding standards; Continuous integration; Project management; Knowledge sharing.

The component-driven strategy increased the productivity of the team, not only because it improved reuse of code, but also because it decreased the number of errors. Once an error was identified, the corresponding component was fixed and automatically all the modules of the system that used it were also fixed. Project guidelines and coding standards

Software development process

Since different teams, with different background and skill levels were working on the project, it was important to enforce that all the teams followed defined guidelines and coding standards. The main goal here was to have a homogeneous source code, making it easier to modify and reuse the software.

Our development process was based on the principles of the Unified Process [5]. The software was produced in an incremental process, in such a way that new parts of the information system were delivered at each iteration. The system specification was made using the use case methodology proposed by the Unified Process. However, our use case document templates were modified to allow non-technical stakeholders to better understand what was specified and how that would look like in the real system. Thus, we added to the template a draft of the screen prototypes with a clear description of the behavior expected for each screen visual element. These use case documents were discussed with the stakeholders and after they were formally approved by all, the same document was used by the implementation team to implement the use case.

These were the main guidelines defined for the project: •

A standard for directory structures, defining where each artifact, from specification to code should be stored; • A standard for compiling and packaging the system, defining standard scripts to build all the components using the same procedure; • A standard for code style, that was enforced automatically by tools that verified if the code produced was compliant to the standard; • A standard for documentation, creating templates either for software specification and for code documentation; • Guidelines for implementing new use cases, defining the technologies that would be used and how they should be used. The project was based on open-standards and on opensource paradigms. Java Technology was chosen from the first moment for its ability to generate systems that run on any platform; to make extensive use of object-oriented analysis, and to create reusable software components. Although the system should run on a variety of equipment, a basic platform was defined with Linux as the standard operating system, JBoss as the application server and Oracle as the database management system. The only proprietary piece of software chosen was Oracle, as for the foreseen volumes no free-software database management system was considered suitable for the task. The system was developed in three-layers as depicted in Figure 2.

The iterative process worked very well and allowed to diminish the risks of the project, since stakeholders were able to see the results early in the project and thus they understood the development process better and communication became easier as the project evolved. The first module of the system went into production by the end of May 2004 and by then the team and the stakeholders had a clear understanding of how the development process worked. Each iteration took about 6 weeks, with typically 4 weeks of coding, one week of testing and one week for the stakeholders testing and approval. At each iteration, while the development team was coding, the system analysts were specifying the use cases of the next iteration. Component-driven software development One of the principles we set out for this project was that no piece of functionality should be coded twice. To accomplish that, we adopted a component-driven software development. As each use case was ready to be implemented, a group responsible for the architecture of the project would identify which existing components could be reused, which ones should be modified and which new components should be created.

The application is web based, which lowered costs, since new versions are installed at only one place – the server. Also, users use simple diskless thin-clients to access the system, which also decreased maintenance costs.

There were well defined and well documented rules about how a component should be constructed, so we would guarantee that it could be reused by other parts of the system.

35

F. B. Nardon et al. / Software Engineering Principles Applied to Large Healthcare Information Systems A Case Report

defining the frameworks that should be used, it is important to create guidelines on how to use them.

The persistence and business layers of the system were implemented using Enterprise Java Beans 2.0 technology. The web layer was implemented using Struts and Tiles frameworks.

Continuous integration Continuous Integration [6] is a software development practice through which members of a team integrate their work frequently. The goal is to identify integration errors as soon as possible, reducing integration problems.

As happens with any complex technology, this architecture allows to use many different paths to implement the same functionality. If the developer is free to choose its own solution, the result is a heterogeneous code, with hard maintenance, and a potential increase in the number of errors. Also, when not well guided, beginners in the technology tend to make mistakes that well experienced developers would not.

This practice is very important when multiple teams are involved and when component-driven development is used. Our strategy was to integrate the software daily, using an automated tool. All the projects artifacts were stored in a central versioned source code repository. Each day, our continuous integration tool retrieved all the code produced and integrated it. If errors occurred, the developer responsible for it would receive an email alerting that his/her work had generated an integration error and this should be fixed.

To avoid this problem our strategy was to capture the knowledge of the most experienced developers and code this knowledge as metadata. Then, the developers would annotate the code with metadata stating the desired behavior of the use case and our framework would process these annotations and generate “expert code”. The code generation strategy eliminated boilerplate code, enforced the guidelines and standards, increased productivity and made beginners to produce code as an expert would.

As a result, in the last day of the iteration, we were certain that when joining the pieces together the software would not have integration problems. This contributed to decreasing the time spent solving integration problems and accelerated the development time. Project management

Differently from other coding generation tools available, however, our code generation strategy was conceived to allow extensions, making the system flexible.

Managing a large project as this one is always a challenge. Managing the human resources involved is certainly the most difficult task for project managers. The most subjective aspects of human resources management are out of the scope of this paper, but it is important to share some techniques that played an important role in the success of this project. The project had a well-defined issue tracking process. This process stated clearly how each issue (an error, an improvement, a new task) should be addressed on the project. It started with someone registering the issue in a web based project management software. This generated an email to the project manager. The project manager would analyze the issue and either reject it or schedule the issue to resolution, assigning it to a member of the team and registering a time estimate to the issue resolution. Once solved, the owner of the issue would register what had been done and how much time was spent. Since all the process was controlled by a web-based software tool, it was possible to verify in real time, how the project was evolving, who was late on the tasks, who had time to work on other tasks, what was scheduled to be done on each iteration, and so on. Since this project management tool was integrated with the version control repository, each time a issue was solved, it was possible to track back exactly what changes had been made on the project artifacts because of that issue, what helped to identify why things changed.

Figure 2 - System architecture After analyzing three years of statistics from the project, it is interesting to notice that the most modified files of the system were files that implemented the user interface. Not coincidently, the user interface implementation was the only part of the architecture that didn’t have a well-defined guideline or code standard. Although we had defined that the user interface should be coded using the Struts Framework and the Tiles technology, this was not enough to avoid beginners’ mistakes. This clearly shows that besides

This technique gave us important metrics that helped to keep the project on schedule and improved the communication among the team members.

36

F. B. Nardon et al. / Software Engineering Principles Applied to Large Healthcare Information Systems A Case Report

Knowledge sharing is very important on a large IT project. The best way to keep information about a project available for all is to enforce documentation.

There are many other aspects that could lead a project to failure, such as political forces, management changes, and others, but it is known that most IT projects failed because they can not deliver what was promised on time.

In the São Paulo City Health Information System project, all project artifacts were stored in a central versioned repository. This repository was available online and could be accessed by all the members of the team.

It is very important to choose the methods and the architecture of the information system before the first line of code is written. It is also important to communicate to all members of team clearly what the guidelines and methods are.

When a new requirement was identified and the software had to change, the documentation was changed and only then the software was changed.

Although the techniques described on this paper were very important to the success of the project, it is also clear that they require discipline from the team and strong management, what is not easily accomplished. In this sense, choosing the right team is also an important aspect of the project.

Knowledge sharing

This approach reduced the development time, by eliminating the need for fixing poorly documented requirements. It also improved system maintainability, since even when a member of the team left the project, the knowledge was well documented and the other members could continue the work on it.

The experience of the City of São Paulo Healthcare System should be used as an example to other public agencies that are striving to build a reliable information system. Our experience shows that applying the right software engineering principles help to achieve the goal of having an information system that will help to provide a better quality of care.

Results and conclusion Impact data related to using SIGA Saúde is still under collection. However, current assessment reveals that outpatient services productivity has increased about 35%.

References [1] Standish Group. The Chaos Report. Ed. The Standish Group. 1995. [2] McCartney L. U.K. Dept of Health: Prescription for Disaster. In: Baseline Magazine. November, 2006. Available at: http://www.baselinemag.com/article2/ 0,1540,2058194,00.asp. [3] Grady RB, Practical Metrics for Project Management And Process Improvement. ed. Prentice-Hall, 1992. [4] Moura LA, Vecina-Neto G, Lira ACO, Leão BF, Hirano EM, Troccoli FT, Pacheco R, Castanha LHA. A Construção do Sistema de Informação da Secretaria Municipal de Saúde de São Paulo. Proceedings of the IX Brazilian Conference on Health Informatics, Ribeirão Preto, SP, Brazil, November 2004. [5] Jacobson I, Booch G, Rumbaugh J. The Unified Software Development Process. Ed. Reading, MA: Addison-Wesley, 1999. [6] Fowler M. Continuous Integration. May, 2006. Available at http://www.martinfowler.com/articles/ continuousIntegration.html [7] Opinion, A Saúde na Era da TI (Health Care in the Age of IT), O Estado de São Paulo, 23 August 2006, São Paulo, SP, Brazil. [8] Duke’s Choices Award, www.sun.com/javaone/

Patients’ perception has changed for better, as patients have stated they can schedule appointments within the same month, against a three-month wait that was typical before SIGA Saúde. The Brazilian Press, usually very critical of public IT services published an opinion article stating their view that SIGA Saúde was a major contribution to the Bazilian Health System [7]. The project has also won a Duke’s Choice Awards, granted by JavaOne 2005 Conference Meeting [8]. SIGA Saúde architecure is very innovative. By being fully web-compliant and using public Internet for providing safe role-based access to the system, SIGA Saúde can easily be deployed in other cities or regions. Also, as SIGA Saúde is fully compliant with all Brazilian standards and SUS practices, it is a tool to be used to help SUS itself be implemented throughout Brazil. Finally, São Paulo City’s dimensions and complexity are so evident that taking SIGA Saúde to other cities and states requires simplification rather than new functionalities. The techniques described on this paper allowed to deliver a large healthcare information system on time and had a significant impact on decreasing the number of errors.

Address for correspondence Fabiane Bizinella Nardon, Av. Luis Carlos Berrini, 962 – Cj. 61, São Paulo – SP – Brazil. E-Mail: [email protected]

The results of this project show that using software engineering principles is a key factor to avoid project failure.

37

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Towards Modeling and Simulation of Integrated Social and Health Care Services for Elderly Alexander Horscha, Daryoush Khoshsimab a

Technische Universitt Mnchen, Germany, University of Troms, Norway b University of Applied Sciences Heilbronn, Germany

the personal involvement of elderly in the social life of a society as well as to deliver high-quality medical care.

Abstract In order to estimate the impact of an innovation on a segment of the health care system under certain assumptions such as different possible regulatory or financing schemes (scenarios) prior to its diffusion, one must understand the dynamic behavior of the entire system with its essential control loops. Aim of this feasibility study was to explore the potential of System Dynamics (SD) modeling for this purpose. First, a UML-based modeling of an Innovative Care for Elderly (ICE) system for provision of integrated social and health care services to elderly living at home was done. Then monetary and quality of life aspects of the social and health care system were described by two coarse SD models. On these models the impact of the introduction of the ICE system under certain assumption (scenarios) was studied, based on data from the German Health Expenditure and German Federal Statistics Office. The simulations show plausible behavior, however, are not yet detailed enough for a final conclusion. A major problem is missing data for setting model parameters: estimates had to be made. In conclusion, SD modeling might be a useful method for studying impacts of the diffusion of an innovation in the health for elderly sector, but more research is needed.

Figure 1 - The Innovative Care for Elderly (ICE) System At the Troms Telemedicine Conference 2004 we presented the system analysis for a comprehensive telematics support of social and health service chains for elderly [7] (Figure 1) including the analysis of requirements, business processes, business use cases and system use cases of a system which in its full range covers the following components: automated house; emergency call; virtual home for elderly; telecare service. There are pilot projects and accompanying studies such as [1][2] dealing with the integration of telecare services into service delivery. And there is a variety of efforts addressing care innovations such as assistive technology or telecare in different settings. But, to our knowledge, there is no deep understanding of the entire systems dynamic behavior and the impact of innovative services on the clients quality of life under the technological and organizational intervention of introducing health care at home. This might be due to the fact that sufficiently detailed models are fairly complex and need a lot of data to be parameterized in the right way (see e.g.[1][2]). Often, the necessary information is not available and has to be estimated.

Keywords: Health services; eHealth; telemedicine; elderly; system modeling; dynamic modeling; computer simulation.

Introduction The demographic trends in most countries show an overproportional increase of elderly in relation to the whole population, coming along with an increase of chronically ill or multi-morbid persons, and persons that cannot live without support from social and health care. The social and health care systems have serious problems with the increasing costs of services. The diversity of care services is an additional threat for geriatric clients. The current trends will lead to a situation where it will be mandatory to utilize information and communication technology to support an independent living of elderly at their own homes as long as possible, including not only health care services, but also social services with the overall goal to intensify

Being aware of these difficulties, we have tried to build two simulation models using the methodology of System Dynamics (SD) [10][11][9], with the goal to achieve, on a moderate level of granularity, the basis for understanding

38

A. Horsch et al. / Towards Modeling and Simulation of Integrated Social and Health Care Services for Elderly

the systems behavior and a starting point for further refinement and adjustment to relevant eHealth scenarios [8][5]. The first model is designed to investigate the economic issues of the system trying to answer the study question:

contacts to e.g. self-aid groups; support of relatives involved in the care process. 4. The Telecare Service. Through this service mainly the medical needs are covered. It comprises telemonitoring of the health status through e.g. vital parameters such as blood pressure, heart frequency or temperature, and all actions and services needed to handle medical or nursing tasks identified by the responsible professionals at the service center. In addition, a teleconsulation service is included for supporting a smooth professional to professional communication.

How should the costs be distributed among the different stakeholders, so that a win-win situation can be achieved? Is such a benefit-for-all situation achievable during the lifetime of the system? The general system behavior has been explored to observe the behavior it can produce and to observe the common mode of the system behavior. The second model was created to investigate the quality aspect of the system in order to answer the study question:

Systems thinking Considering a system “as a set of elements or components that work together in relationships for the overall good and objective (or vision) of the whole” [4] there are different viewpoints on SD (Figure 2). Descartes analytic thinking, based on the postulate that in separating the system in manageable pieces and understanding these pieces will lead to an understanding of the whole, when put together, is reflected by disjointed viewpoints on subsystems. This straightforward reductionist thinking usually fails to address complex systems because it is not able to capture the complexity adequately and deal with interrelationships and interactions between subsystems.

How may the system affect the clients quality of life? Main goal here was to identify the key factors influencing quality of life and to model their interrelationship in order to study the change of these entities over time, in dependence of the model parameters which describe the type of service implementation chosen in a region or country.

Material and methods The Innovative Care for Elderly (ICE) system As innovation to be studied in terms of diffusion into the health and social care sector for elderly, the Innovative Care for Elderly (ICE) System is used [7][8]. The system is fictitious, since this kind of overall systems does not exist, yet. It is a comprehensive telematics support of social and health service chains for elderly, which is customized to the needs of the individual client. A health manager or the ICE service provider together with the client chose the appropriate subset services, mandatory and optional ones. The Unified Modeling Language (UML) and the modeling software Together 6 (Borland) have been used for the analysis and description of requirements; business processes; business use cases and system use cases. The main idea is to have a system that in its full range comprises the following four components (see Figure 1):

Figure 2 - Viewpoints on System Dynamics (SD)[5]

1. The Automated House. Such a house can offer, through simple to use interfaces adapted to the degree of disability of the occupant, e.g. various control functions for operating parts of the house (doors, jalousies, etc) and devices in the household, as well as for device communication. It includes the connection to external data networks. 2. The Emergency Call. This component mainly addresses sick and/or disabled persons. In case of an emergency the occupant triggers an alarm and a connection to a central help desk is established for further action. This trigger event can for example be the push on a button or a signal from a fall detection device. 3. The Virtual Home for Elderly. This component provides a portfolio of social and health care services. Social services comprise, for example: videophone communication with the care personnel; social interaction; consulting in critical situations; mediation of

Another approach is the linear thinking, which tends to see simple sequences of cause and effect (Figure 2, linear control viewpoint). But in real systems, causes and effects are circular in their nature, not linear, forming positive or negative feedback loops (Figure 2, causal loops). Therefore, also linear thinking is inappropriate for modeling complex systems. This brings us to the point where interrelationships and control mechanisms come into the consideration, which the formal description of complex systems composed by subsystems which influence each other in a specific dynamic way.

39

A. Horsch et al. / Towards Modeling and Simulation of Integrated Social and Health Care Services for Elderly

complete Sector Frames is controlled by a total number of 85 hard variables. For details and a description of the complete model in iThink see .

Figure 3 - Sector Frame Demographics For our feasibility study, we used a method known as Systems Thinking [3]. This method provides: Figure 4 - Social Alarm (SA) System Expenditures

1. Stocks. Represent a certain amount of units-of-measure. Four types of stocks are used: Reservoir, Conveyor, Queue, and Oven. 2. Flows. Represent transmission of units-of-measure from one stock to another, and can be unidirectional, bidirectional, or unit converters 3. Converters. Change the unit-of-measure of a flow. 4. Connectors. Represent information or inputs that regulate flows. For our feasibility study we only used reservoirs, uni- and bidirectional flows, and connectors. As software tool, iThink Version 8.0 (High Performance Systems, Inc., Lebanon, NH, USA) was used. For the simulations, this software solves the finite difference equations determined by the model using iterative numerical methods (Euler, Runge-Kutta).

Figure 5 - Financial Sources of SA System

Results The economic model The unit-of-measure for the economic model is a monetary unit. Three so called Sector Frames these are model components which can be simulated separately have been modeled: 1. Demographics (here: Germany, see Figure 3) 2. Expenditure on system components (services) 3. Financial sources of system components (services) For Sector Frame 1, data from the 10th coordinated population prognosis for Germany (published 2003) have been used. Figure 3 shows the Flow clients controlled by the demographic variant chosen (elderly1, elderly5, or elderly9) and the fraction of elderly using the ICE system (clients frac). To give an example of the modeling of ICE components, Figures 4 and 5 show the Social Alarm (SA) system expenditures and the financial sources modeling, respectively, with the SA cost stock being the connecting model entity. The flow into the SA cost stock is controlled by initial costs, fixed costs, extra costs, etc., the flow out of the stock is controlled by the different financing sources such social security funds (SS) and general government excluding security funds (GG) on the public funds side (PF FS), and private social insurance (PSI), private insurance enterprise (PIE), social security funds (SS), private out-ofpocket payment (PP), non-profit institutions (NPI) on the private sector side (PS FS). The complete model with the

Figure 6 - Causal loops of the Quality of Life model The Quality of Life model The Quality of Life (QoL) model is constructed without sector frames and comprises solely soft variables. The clients QoL may be affected by many different factors, such as recognition, health, social relationships, richness of daily life, satisfaction with work. Influence on an individual changes over time. Influence on a population may be considered in a statistical way and described in terms of averages. In this feasibility study, we are interested on the large-scale influence on the QoL of a population. Our model is based on the three key factors driving the impact of the ICE System on the QoL: economical status, social status, and health status of the client. Figure 6 shows these factors which represent Stocks in the iThink and the essential causal loops QoL social status, and QoL health status. The model is controlled by a total number of 23 soft variables. For details and a description of the complete model in iThink see .

40

A. Horsch et al. / Towards Modeling and Simulation of Integrated Social and Health Care Services for Elderly

enterprises, and private out-of-pocket payment. Table 1 shows the setting of parameters for Social Alarm (SA), Virtual Home (VH), Automated House (AH), and TeleCare (TC) service, with frac indicating the fraction of the population receiving a service (e.g. SA frac = 0.1 indicating 10% receive SA service) or the fraction of financing source (e.g. AH PSI frac = AH PIE frac = AH PP frac = AH NPI frac = 0.25 indicating the automated house being financed 25% by the four sources PSI, PIE, PP, and NPI), with SS indicating social security funds, PP indicating private out-of-pocket payment, NPI indicating non-profit institutions, PSI indicating private social insurance, PIE indicating private insurance enterprise as source of financing. Initial costs and running costs, as well as further parameters are included as well, see [8] for details. For the second economic scenario (not described in more detail, here), public funds reimburse the highest share of expenditure on the ICE system.

Table 1 - Parameter settings for first economic scenario Parameter

Value

Parameter

Value

SA frac

0.1

VH NPI frac

0.25

VH frac

0.1

AH PSI frac

0.25

AH frac

0.1

AH PIE frac

0.25

TeC frac

0.1

AH PP frac

0.25

SA SS frac

0.95

AH NPI frac

0.25

SA PP frac

0.05

TeC PSI frac

0.25

VH PSI frac

0.25

TeC PIE frac

0.25

VH PIE frac

0.25

TeC PP frac

0.25

VH PP frac

0.25

TeC NPI frac

0.25

All other parameters are set to 0. The simulation runs Based on statistical data from the German Health Expenditure and German Federal Statistics Office, simulations have been run under varying circumstances and assumptions over different periods of time for different scenarios in order to demonstrate how a change of parameters (population progression, financing sources, time factors, fraction of clients assigned to the different services) changes the behavior of the whole system.

Figure 9 - Increasing QoL

Figure 10 - Oscillating QoL Two QoL scenarios have been considered (represented according to the scale shown in Figure 8):

Figure 7 - Simulation Run of First Economic Scenario

1. First scenario: Dominant impact of social aspects on QoL, with only 10% maximal change of QoL by economic status, but 30% by social status, and 50% by the health status. Assuming the first economic scenario (private financing dominant, see above), which has a negative impact on the economic status of the clients, parameters were set to: impact of ICE system on economic status = 20 (big negative impact); on social status = 55 (small positive impact); and on health status = 55. Simulation over 5 years shows increasing positive impact of the ICE system on the QoL (Figure 9).

Figure 8 - Scale for Impact on Quality For the first economic scenario, it was assumed: Germany with the middle variant of population progression, leading to middle number of elderly; 10% assigned to each service; the SA system 95% financed publicly, 5% privately; other services 100% financed privately, with a share of 25% for NPOs, private social insurance, private insurance

41

A. Horsch et al. / Towards Modeling and Simulation of Integrated Social and Health Care Services for Elderly

Assuming the second economic scenario (public funding dominant), the impact increases faster. 2. Second scenario: Dominant impact of economical aspects on QoL, with only 10% maximal change of QoL by social status, but 30% by economic status, and again 50% by the health status. Assuming the first economic scenario and the same impact of the ICE system on the three components of the clients status (economic, social, health), simulation over 5 years shows slight oscillation around the no-impact baseline (Figure 10). Assuming the second economic scenario (ICE system has no impact on economic status), QoL increases comparably to the first QoL scenario.

cesses have to be included, and more variables (e.g. market saturation, clients acceptance, quality of service) have to be included. The quality of life model requires more research. It should also be refined, for example by adding direct influence of the clients economic status on the social status. Future efforts must be put on the continuing development and validation of the models by empirical studies. Acknowledgments We thank the colleagues from Technische Universitt Mnchen, Ludwig-Maximilians-Universitt Mnchen, University of Troms, and Norwegian Center for Telemedicine, for the inspiring discussions. Furthermore, we thank the reviewers for their valuable comments on the first version of the paper.

Discussion

References

Total costs for the ICE system to the purchaser in both economic scenarios start by zero and grow very rapidly during the first year, and increase further during the next three years. After four years there is no significant increase. This dynamic behavior of the model is reasonable since costs and expenditures on the ICE system have been taken as constant, which is not realistic: production costs for devices will probably drop, while costs for human resource may rise. It is also desirable to extend the model by a sector frame for the current care delivery system for elderly, so that it would be possible to switch between both service delivery systems.

[1] Barlow J; Bayer S; Curry R: The design of pilot

telecare projects and their integration into mainstream service delivery. J Telemed Telecare 9 (Suppl 1) S1: 1-3, 2003. [2] Bayer S, Barlow J, Curry R: Assessing the impact of a care innovation: telecare, Imperial College, London, unpublished, 2004. [3] Carpa F: Ecology, Systems Thinking and Project-Based Learning. Procs Ann Conf Project-Based Learning, 1998. [4] Haines S: Systems Thinking and Learning From chaos and complexity to elegant simplicity, Center for Strategic Management, 1999. [5] Hitchins DK, http://sysdyn.clexchange.org [6] Horsch A, Khoshsima D: Towards a Dynamic Modeling of Integrated Social and Health Services for Elderly at Home. Abstract, TTeC 2006. [7] Khoshsima D, Horsch A, et al.: System Analysis for a Comprehensive Telematics Support of Social and Health Service Chains for Elderly. Abstract, TTeC 2004. [8] Khoshsima D: Assessing the Impact of Telematics and Care Innovations for Elderly at Home using System Dynamics Simulation. Diploma thesis, FH Heilbronn, 2005. [9] Milstein B, Homer J: Background on System Dynamics Simulation Modeling. With a Summary of Major Public Health Studies. Rev. April 8, 2005. Syndemics Prevention Network www.cdc.gov/syndemics. [10] Royston G, Dost A, et al.: Using system dynamics to help develop and implement policies and programmes in health care in England. Syst Dyn Rev 15: 293-313, 1999. [11] Wolstenholme EF: Towards the definition and use of a core set of archetypal structures in system dynamics. Syst Dyn Rev 19: 7-26, 2003.

The QoL simulations have shown exponential increase in three of four cases: if dominance of social aspects on QoL is assumed, under both financial schemes (primarily private vs. primarily public) QoL increases; if dominance of economic aspects on QoL is assumed, then QoL increases only in case of primarily public funding, but essentially stays constant in the other case. The problem of missing data for the economic model could be tackled by involving e.g. health insurances and health care service providers of a test region in further studies with refined models. Concrete innovations introduced in a test region could serve as framework for validation studies. An empirical study based on questionnaires could help in getting realistic parameters for the QoL model.

Conclusion

Address for correspondence

The current models presented in this paper reflect the systems core components and possible dynamical characteristics. The models need substantial enhancement to make them realistic enough to give answers to our research questions. Details on costs and expenditure pro-

Alexander HORSCH, Prof. II, Dr.rer.nat., Dr.med.habil. Email: [email protected], or [email protected] Phone: Tel +49-89-4140-4330, Fax +49-89-4140-4974

42

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Healthcare Delivery Systems: Designing Quality into Health Information Systems Phil Joyce a, Rosamund Green b, Graham Winchc a

Faculty of Information and Communication Technologies, Swinburne University of Technology, Melbourne, Australia b Barwon Health, Geelong, Australia c University of Plymouth Business School, Plymouth, England

effectiveness of actual patient care, the efficiency of the hospital operations, the appropriate usage of resources and the expectations of patients, physicians, other hospital staff, etc, to develop measures of quality management in healthcare provision. Information and Communication Technology (ICT) support and help management of complex processes and operations with Enterprise Systems (ES) and Enterprise Resource Planning System (ERP) providing organisational wide systems that capture the processes of the organisation (horizontal flows) [3]. Moreover, eBusiness (and eCommerce) systems support the provision of services and/or goods electronically, e-fulfilment and these systems provide management with not only an effective tool to deliver product, information and funds but provide management with a method to model the strategic possibilities and implications to support healthcare delivery.

Abstract To ensure that quality is ‘engineered in’ a holistic, integrated and quality approach is required, and Total Quality Management (TQM) principles are the obvious foundations for this. This paper describes a novel approach to viewing the operations of a healthcare provider where electronic means could be used to distribute information (including electronic fund settlements), building around the Full Service Provider core. Specifically, an approach called the “triple pair flow” model is used to provide a view of healthcare delivery that is integrated, yet detailed, and that combines the strategic enterprise view with a business process view. Keywords: Health Information Systems, TQM, triple pair flow construct

The need for strategic vision in the executive group to be communicable to and useable by the other stakeholders in their developments is essential. This allows all stakeholders to gain a mental picture or model of what is being achieved and focuses management on: envisioning, specification, design and implementation. This can be surfaced and articulated with other stakeholders in such a way that they can see within their area, business processes and technological infrastructures that are the basis of their thinking. This will ensure that quality is ‘engineered in’. A holistic, integrated and quality approach is required, and Total Quality Management (TQM) principles are the obvious foundations for this. TQM is a business philosophy that encourages an over-arching responsibility - both individual and collective - to achieve quality and customer satisfaction. Commitment must be at every stage of the delivery. Importantly, the successful implementation of TQM will require the alignment of the organisation’s information systems and other management systems with the new TQM environment.

Introduction Health care providers in western society now command and require increasingly large budgets to provide health care for their patients (or clients) and their communities. Indeed, government funding to health care has been steadily growing during the last ten years. This has required health care providers to clearly define their resource usage and provide information (e.g., patient level data) on their organisational and operational processes in the provision of these services. This has seen the development of funding models that are based on the development of “best practices in health care.” Organisations responsible for healthcare delivery are presently facing competing challenges. Health managers are seeking to satisfy the ever increasing number and size of stakeholder groups with quality healthcare delivery that meets the patient’s needs, whilst delivering quality healthcare data to the hospial [1]. Healthcare delivery quality, as defined in [2] “as that kind of care which is expected to maximise an inclusive measure of patient welfare, after one has taken into account the balance of expected gains and losses (variability) that attend the process of healthcare in all its parts.” Clearly, the complexity of this task cannot be under estimated. It requires the development of effective management and operational processes that are capable of capturing information to support the management and control of healthcare delivery. The captured information allows healthcare providers to evaluate the

Integration of process and information technology integration in quality healthcare delivery Many large integrated information systems, particularly in public service projects, have notoriously underperformed and disappointed [4]. Often the majority of healthcare organisations do not understand the impact and effect of operating an integrated information system that captures the core processes of the organisation. Moreover, systems of this nature dramatically and fundamentally change the

43

P. Joyce et al. / Healthcare Delivery Systems: Designing Quality into Health Information Systems

beneficially), there appears to have been relatively little improvement in the ability of organisations to incorporate these innovations in their everyday activities. Similarly, the authors observe that the ability to identify and learn about new improvement methods no longer presents a barrier to most managers, but rather the biggest challenge is successful implementation. Repenning and Sterman [8] also assert: “You can’t buy a turnkey six-sigma quality program. It must be developed from within.”

way the organisation operates and interacts with its primary organisational objective and outcome: patient care. Similarly, healthcare managers may often have different objectives in their roles to support the organisational objective [5]. In this sense: clinical practitioners will primarily focus on the processes concerning patient outcomes; senior management on trend analysis and successful long term strategies and strategic planning; middle level management and reviewers with monitoring, review, productivity and resources utilisation; and business services management with billing, budgeting and accounting. Similarly, each group will have specific key performance indicators of their area’s performance in meeting the organisational objectives and outcomes.

A modeling approach to integrate the enterprise view with a business process view Previous work by the authors has integrated thinking from strategic management, business process engineering, and resource-based view (RBV) of the firm and balanced scorecard (BSC) analysis to produce an integrating framework for e-business design. This framework reflects both a top-down, entrepreneurial/customer-facing view with a bottom-up, instrumental, capability-based view of what can be done [9]. The original framework, which itself essentially reflects aspirations, has then been integrated with the e-business models of Weill and Vitale [10] and the process flow modelling from system dynamics used to present a method for visualizing, communicating and then developing a shared view or consensus on the critical flow processes that can operationalise a business vision [9]. The visualisations support the critical examination of base ideas by different stakeholder groups and different domain experts; the prototype systems can then be expanded and refined of to best serve the precise needs of the organisation and its stakeholders. This, it is argued, supports powerfully the process of internal development of systems that are part of and support a quality environment within the organisation.

Healthcare managers are responsible for the envisioning, specification, design and implementation of new electronic processes and must therefore take a holistic, integrated and quality approach. TQM principles are the obvious foundations for this. Total Quality Management (TQM) is essentially a business philosophy. For TQM to be successful, management decisions must be aligned and integrated into a system of continuous quality improvement to meet the expectation of the customer. There is a large amount of literature on the topic, and there are at least two major models that are used for business excellence assessment based on TQM principles – the EFQM Excellence Model [6] and the Baldridge Award [7]. These two world benchmarks use very similar criteria and dimensions for assessment, and both include Leadership, Processes, Information, and People Involvement as critical elements. However, much of this is essentially diagnostic or aspirational – ‘do we think we are a quality organisation?’ or ‘what should a quality organisation look like?’. The appropriateness of a direct TQM approach in healthcare is long established (for example, [1, 2, 5]) though little is instrumental in the sense that it provides tools that can directly support the development of effective, coherent, and purposeful systems within an organisational quality framework. This is particularly the case when it comes to the design of integrated fulfilment, information and fund transaction systems in complex information systems in a healthcare setting.

Information Flows

The ‘Triple- Pair Flow’ Construct for Envisioning Fulfilment Systems All business transactions are in one way or another a supply chain fulfilment system, and healthcare delivery systems are no exception. A goods or services need is fulfilled (i.e. satisfied) by the good or service being delivered or provided with payment being received by the supplying organisation in exchange. Effectively, three flow processes comprising all such systems: • information flows; • money flows, payment for goods or service; and • delivery of goods or services. These are just the primary flows, in a healthcare system delivery of patient-care comprises a variety of service elements including medications and other consumables, and similarly money flows may be direct from patients, from insurance companies and government funding agencies. These may be seen as refinements, alternatives or extensions of the primary flows above. However, an important consideration is that each of these flows can be two-way:

$ $

Flow of Goods or Services

Flow of Money

Figure 1 - The ‘triple-pair’ process flow model of supply chain fulfilment Repenning and Sterman [8] correctly reported that there is a growing emphasis on business processes rather than business functions as the keystone to organisational improvement and quality enhancement. Process oriented improvement techniques such as Total Quality Management and Business Process Reengineering have proven to be powerful tools for improving the effectiveness of many organizations. However, despite the fact that it is easier than ever to learn about these performance improvement techniques and where they are being used (presumably

Reverse information flows might include order acknowledgements, delivery notices, invoices, out-of-stock notifications, etc. It might also include information not directly related to individual order fulfilments, for example, stock position advisories and so on;

44

P. Joyce et al. / Healthcare Delivery Systems: Designing Quality into Health Information Systems

this group. They discuss how this sector can be served by enterprises with stakeholder groups and interconnecting goods/services, information and money flows as well as characterise the FSP model by a simple schema linking these and characterise the actor and flow structure as in Figure 3. This business model reflects the situation where access to a range products or services is provided through a primary provider who might not only supply its own products or services, but also sources related products and services from partner organisations. The primary relationship in this system is between the provider and the patient, but there are additional relationships involving flows of money, product/services, and/or information between the provider and its second-level supply network partners – which Weill and Vitale consider could be suppliers or complementors (resellers and other suppliers of complementary products and services) – and between the second-level suppliers and customers.

Reverse money flows might be refunds, cash-back, commissions, etc; and Reverse goods flows might be returns, trade-ins, etc. Goods in transit

Goods recieved

Goods despatched Returns in transit

Returns depatched

Returns received

Inf Pipeline Forward

Info out UP

Payments in Pipeline

Info in UP

Payments received

Info Pipeline BACK

I f

i

BACK

Payments authorised

Refunds in Pipeline

Info out BACK

R f

d

th

i

d

R f

d

i

d

Figure 2 - The triple pair model with detail for each flow process Joyce and Winch (2004) have described a novel construct – the “triple-pair flow” construct for envisioning such systems based on six main flows: two-way information, twoway goods and services, two-way money flows, as shown in Figure 1. If all the relevant flows relating to a particular healthcare provider could be represented within this triple pair model, then the configuration of the six flows can be mapped onto any business model that the healthcare company has in place, or wishes to adopt. Using the stock-flow diagramming convention of System Dynamics extends their construct. We will investigate how business process models can be mapped onto business models. System Dynamics is well suited to this application. There are many excellent texts that describe the principles and processes in system dynamics modelling and [11] is a leading example text. From system dynamics earliest day of development [12], has always explicitly reflected industrial and business structures as a complex inter-related set of flows of money, materials and information, and has always been concerned with the structural relationships that make up business processes as well as the softer processes. This dynamic process model perspective leads to a representation of the enterprise as a set of six sub-systems comprising stocks (or levels or inventories) and flows, as in Figure 2. This basic enhancement simply establishes that extension with detail is possible, and that the substructures will comprise a set of stock-flow chains. The process of expanding this to a full representation of the supply chain(s), funds transaction chain(s) and information flows can then proceed as an iterative process involving all key stakeholders. The graphical processes involved in system dynamics are proven to be of significant effectiveness in supporting team building, alignment, communicating views and thoughts with different specialist groups, and consensus building (see for example, [1317]). It is also the basis for what [14] described as “operational thinking” within what he considered the amalgam of critical thinking skills.

$ S u pp lier P rovider

P, I $

$ Fu llS ervice P ro vid er

Cu stom er

$

P, I $

Reseller

P, I

Figure 3 - Weill and Vitale’s Characterisation of their Full Service Provider Model From an overview of the fulfilment processes in a health service - patient care delivery, fund transaction and information flows, including suppliers of medications and other consumables, private and public funding bodies, and regulatory bodies, it is easy to see how these map onto the FSP model in its simple form. This then is the starting point for using the triple-pair construct in the envisioning, specification and design of systems that will engineer in the customer focus, stakeholder alignment and system coherence necessary to achieve quality healthcare delivery. Dimensions of healthcare delivery within a full service provider perspective Through TQM management system managers are able to integrate the vision of quality into the structure of the organisation. This must been seen at the clinical level, in the direct support of suitable patient outcomes, and from a management level, in the direct support of financial and business services and to ultimately create an accountable strategic plan for long term quality achievements. If we consider the role of a hospital as a domain in its attempt to provide quality healthcare we are able to examine the implications of the Full Service Provider Model providing a clear picture of the process of healthcare provision and the implications of TQM. The development of information system architecture with an emphasis on TQM is possible by the development of new systems or the redevelopment of older bureaucratic systems. There is a tendency when reviewing information systems in the healthcare area to create separate decision systems and management systems. Information systems within healthcare domain often fall into three categories [2]. These are:

An integrative design process for quality healthcare delivery and stakeholder need fulfilment One Weill and Vitale [10] model is the Full Service Provider (FSP), which they define as “offering a full range of services in one domain … directly as well as via ‘complementors’ ”. Critically, they include health care provision in

45

P. Joyce et al. / Healthcare Delivery Systems: Designing Quality into Health Information Systems

achieved by meeting preordained patient related performance targets.

1. Clinical or medical information systems – designed to support the activities of patient support (i.e., hospital admissions, medical records, etc) 2. Operational administrative systems – designed to provide necessary non patient care activities for the organisation (i.e., financial, personnel, payroll, etc) 3. Decision support systems – designed to provide management with information for decision making (i.e., strategic planning, analysis and evaluation, etc)

A Triple Pair Flow View of the Processes in the Patient Management System and Government Reporting Using the construct described above, a first representation of the flow processes involved might appear as Figure 4. The sub-system is driven by the patient treatment process – this is shown at its most basic level of detail at the top as a simple admission / in-treatment / discharge process. (In a full model, this would involve a much greater level of detail as part of the core patient care delivery system.). Patient data is captured on admission, during treatment and on discharge, in the Admission Discharge Transfer (ADT) module of the clinical information system. After the data is accumulated, analysed, corrected and coded it is archived but also transmitted to the appropriate government body. This data may be checked against previous files and errors or anomalies returned to the hospital for review, after which it will be returned to the government body – this process is shown in an information recycling route. When the information is finally accepted it will be absorbed into the government body’s database and, where appropriate, will trigger related payments to the hospital.

Examples of key healthcare sub-systems If a complete healthcare organisation can be seen as a (FSP), within the Weill and Vitale classifications, then potentially new or reorganised structures can essentially be viewed. At the core will be a central service delivery subsystem, which, using the ‘triple pair’ flow interpretation. This reflects the technical details of the individual healthcare organisation, with the provider and the patient as the key players. In addition there would be other subsystems presenting the other inter-relationships between the FSP provider – the healthcare organisation itself – the patients and the other stakeholders. Patient management system and government reporting Patients receiving treament

Data transfered to archive

Data extraction

Data capture

This is a first simple diagram and is based on certain assumptions about how health care activities are undertaken and what triggers particular activities. For example, in this representation, data is accumulated and analysed in the ADT system at the hospital and transmitted subsequently, possibly on a weekly or monthly basis, to the government; it is conceivable that at the present time or in the future this data could be transmitted in real time, in which case the model representation would be amended slightly. The model reflects an integrating view of how this subsystem presently operates or could operate in the future. It links the three critical process flows – patients, information, and funding, and can form the basis for debate and discussion on the effectiveness of the system and how the various stakeholders’ needs and responsibilities inter-relate. While embedded within established business process engineering principles, the diagram is easily envisioned in terms of operations on the ground, enabling the focus of attention to be on the design of systems that will effectively deliver to all stakeholders. It can incorporate all-important data flow and identify trigger points which will impact on other stakeholders – for example when payments are to be triggered.

Discharges

Admissions

DTA archive

ADT information accumulation \ processing

Patient data

Error reconciliation

Error requests in process Revised data

Error requests Government archive

Data accepted

Information in review

Data transmission to Government

Payment authorisations

Payment releases

Government funds

Fund commitments

Payment approval process

Provider accounts Processing delay

Figure 4 – Integrated process view of the ADT sub-system The patient management system maintains the primary source of patient demographic information for the organization. It is a detailed, date related database that is an integral corner stone for health care providers as they move towards an electronic health record. Enhanced systems also maintain data encompassing patient admissions, ward transfers, discharges, appointments, operations and other clinical interactions. In one specific public health funding model the total procedures and diagnosis attributed to a patient during their inpatient stay are assigned a single code. The government health body responsible for funding and health quality uses this code in conjunction with the length of the patient’s stay (LOS), time in intensive care and other critical clinical issues. The LOS is compared to the statistical mean of other patients accorded the same code from other health services and the level of funding determined. Funding is assigned to the health service retrospectively to the patients discharged from hospital. In some clinical areas such as the emergency department and surgical waiting list bonus funding can be

Diagnostic testing and payments sub-system Diagnostic test order entry has ethical and security implications but in the appropriate system is a quality initiative for data entry into a diagnostic management system. It also has the advantages of reducing data entry clerk errors, save clinical time eliminating paper based sitting in a box waiting for collection and provides the clinicians ordering diagnostic tests immediate feedback on the cost of the tests being ordered and previous diagnostic interventions. Diagnostic test requests are delivered electronically to other systems both internal and external to the organization where diagnostic tests have been outsourced. Once the tests have been completed for the patient, information is returned to the diagnostic management system in the form of results. These results if in particular are of a pathology

46

P. Joyce et al. / Healthcare Delivery Systems: Designing Quality into Health Information Systems

nature can be in a cumulative format to provide instant access to trend analysis. Cost of the tests performed and billing requisition is also returned in a combination of electronic and hard copy means to the health care provider, government body, insurer or patient him or herself.

Conclusions The approach described here reflects that the highest quality levels in electronically based fulfillment systems in healthcare can indeed be best achieved where an integrative approach to system design is applied throughout the system visioning, processes specification, and detailed design process. There have been many documented failures of large ICT and electronic fulfillment systems especially in the public and health sectors where the perception was that the design processes were unable to cope with the system complexity, the diverse and sometimes competing objectives held by different stakeholders, and multiple political agendas surrounding the service.

An integrated view of the processes involved in diagnostic testing The second sub-system discussed above concerns diagnostic testing, and again a simple representation of the integrated set of process flows is shown in Figure 5. Patient accounts Cost recovery P Health service accounts Diagnostic vendor accounts

Provider accounts

Government

Cost recovery H

References

accounts

Payments

Cost recovery G

for tests

Cost recovery IC

Test completions

Test requests

[1] A. K. Aggarwal and M. Zairi, "The role of total quality management in enabling a primary health-care orientation," Quality Management, vol. 8, pp. 347 - 360, 1997. [2] P. A. Rivers and S. Bae, "Aligning information systems for effective total quality management implementation in health care organisations," Total Quality Management, vol. 10, pp. 281-289, 1999. [3] C. N. Madu and A. A. Madu, "E-Quality in an integrated enterprise," The TQM Magazine, vol. 15, pp. 127-136, 2003. [4] R. H. Lowson and N. F. Burgess, "The building of operations strategy for e-business," The TQM Magazine, vol. 15, pp. 152-163, 2003. [5] P. Adinolfi, "Total quality management in public health care a study of Italian and Irish hospitals," Total Quality Management, vol. 14, pp. 141-150, 2003. [6] EFQM, "EFQM," 2005. [7] NIST, "Baldridge National Quality Program - National Institute of Standards and Technology," 2001. [8] N. Repenning and J. Sterman, Getting Quality the Old Fashion: Self-Confirming Attributions in the Dynamics of Process Improvement. Thousand Oaks, CA: Sage, 2000. [9] P. Joyce and G. W. Winch, "A Framework for Codifying Business Models and process Models in e-Business design," in Value Creation from e-Business Models, W. Currie, Ed. Jordan Hill, Oxford: Elsevier, 2004. [10] P. Weill and M. Vitale, Place to Space. Boston: Harvard Business Press, 2001. [11] J. D. Sterman, Business Dynamics: Systems Thinking and Modelling for a Complex World: McGraw-Hill, 2000. [12] J. Forrester, Industrial Dynamics. Cambridge: MIT Press (Republished by Pegasus Communications, Waltham Mass.), 1961. [13] C. Chandra, S. Kumar, and A. Smirnov, "E-management of supply chain: general models taxonomy," Human Systems Management, vol. 22, pp. 95-113, 2002. [14] B. Richmond, "Systems thinking: Critical thinking skills for the 1990s and beyond," System Dynamics Review, vol. 9, pp. 113-134, 1993. [15] J. Vennix, Group Model Building: Facilitating team learning using system dynamics. Chichester: John Wiley & Sons, 1996. [16] G. W. Winch, "Consensus Building in the Planning Process: Benefits from a 'Hard' Modeling Approach," System Dynamics Review, vol. 9, 1993. [17] G. W. Winch, "Developing Consensus: Reflections on a Model Supported Decision Process," Management Decision, vol. 33, 1995.

Insurance company accounts

Payment liability Diagnostic tests in progress

Test results received Diagnostic test requests

Test order Diagnostic test

issues

order entry system

Test process Diagnostic test

completions

management system Third party information repository

Patients receiving

Data accumulation

treatment Test cost Test cost database

\ trend capture

data capture

Figure 5 - Service, information and money flows involved in diagnostic testing The central spine of the diagram reflects the process of requests for diagnostic tests being raised and then completed. Depending on hospital procedures, tests orders might be raised at the point of treatment or through a central function. When test orders are issued they trigger the request for the test at the test service providers – an internal department or external vendor – and are then logged into the diagnostic test management system. In terms of understanding the inter-related processes from the hospital’s perspective, the depiction here is possibly adequate, though the vendor itself will have its own interacting system of test backlogging, information management and so on. When the test results are received back by the management system, payments to the vendor can also be triggered. The management system will also analyse all tests completed and bill out as appropriate to the responsible institution, the patient him or herself or their insurer. As part of the system, represented as a small structure at the bottom left, a database can be maintained to provide feedback to clinicians and possibly to inform them for their future clinical decisions. Data may be accumulated, analysed for trends and so on as well as provide external agencies involved in disease containment and healthcare policy. As in the previous example, this is only a very simple representation capturing processes only at the coarsest of levels in terms of aggregation, detail, and alternate or parallel processes. Such an initial model as this will likely be the starting point for an integrated design processes for emerging systems.

47

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Mobile Data Collection Tool for Workflow Analysis Jacqueline Mossa, Eta S. Berner b, Kathy Savella b

a School of Nursing, University of Alabama at Birmingham, Birmingham, USA School of Health Professions, University of Alabama at Birmingham, Birmingham, USA

decreasing the error associated with medical care will depend heavily on better systems analysis and design.

Abstract Faulty exchange and impaired access to clinical information is a major contributing factor to the incidence of medical error and occurrence of adverse events. Traditional methods utilized for systems analysis and information technology design fail to capture the nature of information use in highly dynamic healthcare environments. This paper describes a study designed to identify information task components in a cardiovascular intensive care unit and the development of an observational data collection tool to characterize the use of information in this environment. Direct observation can be a time-consuming process and without easy to use, reliable and valid methods of documentation, may not be reproducible across observers or settings. The following attributes were found to be necessary components for the characterization of information tasks in this setting: purpose, action, role, target, mode, and duration. The identified information task components were incorporated into the design of an electronic data collection tool to allow coding of information tasks. The reliability and validity of this tool in practice is discussed and an illustration of observational data output is provided.

Traditional systems analysis techniques to determine information and communication technology requirements for healthcare were originally developed for the design of business information systems. Generally, business procedures are analyzed through the use of interviews, questionnaires, focus groups, and the examination of organizational documents such as job descriptions, organizational protocols, and organizational policies (4). There are several problems with using this approach to designing information and communication systems for healthcare. One problem is the extremely dynamic and mobile nature of healthcare work. The variability of healthcare work allows for only component segments of work to be standardized, and even these are subject to change from one instance to the next (5). Users jump from one activity to another, perform activities differently under stress, and with different levels of experience (6). In addition, healthcare practitioners are highly mobile and favor synchronous communication (7). However, healthcare team members are geographically distributed and not always in close enough proximity for face-to-face verbal communication. This is in contrast to the environment that most information systems were originally designed for; stationary workers at a desk using a desktop computer (8).

Keywords: information, systems analysis, systems design, observation.

Another problem is that the use of surveying, interviewing, and focus groups alone implies that the users can articulate their work processes in an unambiguous manner. Unfortunately, discrepancies arise between reported and actual work practices, making direct observation the most reliable method for data collection (9). Direct observation can be a time-consuming process and without easy to use, reliable and valid methods of documentation, may not be reproducible across observers or settings. Clearly there is a need for tools that can make the observation processes more reliable and standardized. The purpose of this paper is to describe the development and evaluation of a mobile electronic data collection tool designed to capture information tasks of healthcare workers. It was originally developed by direct observation in a cardiovascular intensive care unit (CICU), but has since been adapted for other settings.

Introduction The communication of clinical information is the basis for coordination of patient care (1). Healthcare professionals work in teams that require circulating information to maintain team reliability. Team members use each other and Information Communication Technology (ICT) as repositories and depositories of their collective group knowledge, maintaining their own situational awareness for team coordination. Not surprisingly, faulty clinical communication and information access is a major source of error that can result in adverse events and poor patient outcomes (2). In a study designed to analyze the causes of medical errors, Leape et. al.(3) found that all seven of the most frequent medical errors identified were due to impaired access to information. These errors were primarily the result of system design faults and accounted for 78% of the total errors uncovered. Progress toward

48

J. Moss et al. / A Mobile Data Collection Tool for Workflow Analysis

Methods

Tool automation The paper and pen tool was transferred into an electronic format utilizing Microsoft Access to build a forms-based tool on a tablet computer. The observer was able to follow healthcare team members during the course of their work entering data directly into the study database through the use of a stylus pen. The data collection tool allowed rapid coding of information tasks in a uniform and consistent manner. Collecting data in this manner insured that the data were in a format amenable to analysis of information tasks.

Development of mobile data collection tool The electronic observational data collection tool designed for use in this study builds on previous work to develop an observational data collection tool designed and pilot-tested by Moss, Xiao, and Zubaidah (10) to determine information needs through the observation of charge nurses in a six-room trauma center operating room (OR) suite. The tool was pen and paper based and allowed coding of communication between healthcare workers. In the present study the tool was not only automated, but was refined to allow the documentation of all clinical information use, not just verbal communication.

Reliability and validity Healthcare team members participated in every phase of the tool development. Categorizations of information tasks as well as the selections under these categories were reviewed by CICU healthcare team members for accuracy and comprehensiveness. The completed electronic data collection tool was also reviewed by CICU team members for face validity.

Tool development and refinement The original categories were expanded and new categories were added in order to characterize the totality of care in the CICU. Initially, observations were recorded on pencil and paper documenting the communication activities and information use of individual healthcare team members in their usual activities. These observations served to confirm the preliminary logical data collection codes and to identify additional codes. Observations were made during different periods of the day in 3-4 hour increments of time in order to capture various activities of each team member. During this initial stage, statements made by participants were written down as comments along with the corresponding activity.

To insure reliable coding of information tasks in the setting, extensive training of data collectors was completed. Data collectors practiced data collection in the CICU and met with the principal investigator (JM) after each session to discuss the proper coding of each information task situation. In addition, the forms-based design of the data collection tool guaranteed that data would be collected completely and in a standardized manner. In a study of operating room communication, an earlier version of this tool demonstrated that the tool’s categorical set was able to show differences between characteristics of communication in two different settings and similarities in characteristics of communication in similar settings (12). The sensitivity of the tool’s categorical set was demonstrated by comparing the original data with data obtained from a general OR suite in a comparison hospital (12). These data demonstrated that the identified categories were able to discriminate between information needs in different types of operating room suites. (i.e., demonstrated construct validity). The same technique was used in this study to determine if data would vary in an expected and explainable manner demonstrating construct validity. Observations were conducted on a medical nursing unit and compared to observations conducted in the CICU. The patient population, physical work environment, and information communication technology were all very different from the CICU.

In addition to the observations, team members were interviewed to obtain a description of unit activities (i.e., how patient transfer and family visitation is accomplished) and role responsibilities. The unit was toured to ensure the all modes of information exchange utilized were included as a tool category choice. Observations in the clinical setting were repeated until stable categories were established for each participant’s information use. Approximately 50 hours were spent in this initial paper and pen phase. The observations in this phase were important in identifying who regularly participates in the care of the critically ill patient, modes of information and communication utilized, and the nature of information tasks in this environment. The accumulated data were analyzed through the use of inductive content analysis. Content analysis involves the use of two interrelated processes: identifying specific characteristics of concepts to be measured, and employing explicit rules for identification, coding, and recording of concept characteristics (11). Categories can be identified inductively through an analysis of the content, or deductively wherein theory-based categorical schemes are defined prior to data analysis (11). First, two experienced critical care nurses reviewed and sorted the data into information categories and associated category selections individually. Then, each individual categorization was compared and a final set of categories with associated definitions was developed. Through an iterative process, selections under each category were further developed and validated in the clinical setting, until the categorical set was stable.

Results Information task attributes The following attributes were found to be necessary components for the characterization of information tasks in this setting: purpose, action, role, target, mode, and duration. These attributes are described in detail below. Purpose Information task purpose is the reason for initiating the information task and was determined to fall into one of

49

J. Moss et al. / A Mobile Data Collection Tool for Workflow Analysis

five categories: team status, patient status, task status, instruction, and equipment management •

•

•

•

•

Mode The mode of the information task is the method used by participants for the transmission or procurement of information. Seventeen separate modes of communication were identified in this context including interpersonal (e.g. .face-to-face, wireless phone), information communication systems (e.g. computerized information system), public display artifacts (e.g. whiteboard), printed material (e.g. printed reference), and personal notes.

Team status- information task that is essential for the coordination of team members (i.e. reporting patient transfers, admissions, staff assignments; overhead page of staff member; writing patient’s on-call time from OR on whiteboard, on-call assignment; patient assignment to nurse/team) Patient status- information task related to a particular patient’s physiologic status (i.e. reporting vital signs, blood gas results; viewing monitor displays; viewing x-rays; reading test results; documenting patient assessment) Task status- information task that results in the performance of work or an action (i.e. verbal request for a medication, treatment, lab, or x-ray order; scheduling/ ordering a medication, test, surgery, or consult in the computer; checking on/delegating a task) Instruction- information task related to the exchange of knowledge (i.e. directing an orientee; drug/diagnosis reference; studying/teaching/testing) Equipment management- information task related to the preparation, function, placement, or location of equipment (i.e. location of equipment in the unit, reporting of malfunctions; directing use of equipment; cleaning/sterilizing equipment)

Duration The duration of each information task was recorded on the tool by starting the tool timer at the beginning of the task and stopping the timer at the completion or abandonment of the information task. Reliability and validity Tests of inter-rater reliability of the observational coding in the CICU were conducted between the study data collector and one of the investigators (JM). Both study team members observed the same nurses at the same time, entering data into separate tablet computers. This process was completed on approximately 7% of the information task observations. Analysis of these data indicated 91% percent agreement between the observers in the CICU (13). Data analysis revealed that the tool was able to show variation and similarities in information tasks between the two types of units in an expected manner. For example, CICU nurses’ information tasks were more likely to be related to managing patient data (50%) than Medical unit nurses (38%) and Medial unit nurses were more likely to be involved in information tasks related to directing the work of others (14.6%) than CICU nurses (8%). Table 1 compares nursing information tasks by purpose. Table 2 provides a comparison of nursing information tasks by action.

Action The information task component ‘action’ is a verb used to modify the information task purpose. Four attributes were found to be necessary for the description of information task action in this environment: directing, receiving, reporting, and seeking. • • • •

Directing- information tasks related to delegating or coordinating the work of others Receiving- information tasks related to the reception of information Reporting- information tasks related to the presentation or output of data Seeking- information tasks related to the active search for information

Table 1 - Comparison of nursing information tasks by purpose Purpose mgt CICU

Role

Med Unit

Role describes the functional role of the central participant being observed. These functional roles included: nurses, pharmacists, patient care technicians, unit clerks, and laboratory technicians.

patient

team

task

instruc- equip tion

50%

21.7%

11%

10.9%

6.3%

38.3%

23.7%

24%

10.5%

4.9%

Table 2 - Comparison of nursing information tasks by action

Target Action

The target of the information task is the source or conduit of communication or information exchange. For example, information might be sought from a physician, nurse, or system database. Twenty two information task targets were identified in this study. These targets included both individuals such as physicians, nurses, and laboratory technicians as well as electronic sources such as information system databases and patient monitoring devices.

CICU Med Unit

50

Directing Receiving Reporting Seeking 8%

21.3%

36.2%

10.9%

14.6%

21.9%

37.9%

25.4%

J. Moss et al. / A Mobile Data Collection Tool for Workflow Analysis

expanded our conceptualization of communication to include any exchange of information whether between healthcare team members or team members and inanimate information sources.

Characterization of nursing information tasks Although data were collected on the information tasks of several types of healthcare team members, nursing information tasks are characterized in this paper to provide an example of potential data collection tool output. Twelve CICU nurses were observed for approximately 120 hours. From these observations data were recorded regarding a total of 1,631 information tasks. Duration of information tasks ranged from 1 second to 14.11 minutes with a mean duration of 33.41 seconds. Due to space limitations only a sample of these data are presented.

To increase the granularity and usefulness of data collected we have added the category ‘action’ to modify and further describe the purpose of information tasks. Addition of the action modifier has allowed us to analyze the data and describe the healthcare information tasks at a level that can better inform the design and evaluation of ICT in practice illustrated by the characterization of information tasks for patient status and task status in the CICU data example.

Patient status

Information task comparison

Reporting information was the most frequently performed information task (43%) related to patient status, followed by seeking information (39%), receiving information (17%), and directing ( 2 indicates a positive trend (p=0.08)

Figure 5 – Portable data acquisition electronics

95

T. Giorgino et al. / Remote Support for Stroke Rehabilitation: MyHeart’s “Neurological Rehabilitation” Concept

their cost makes it unlikely support from insurances and national health-care systems. IT technologies are a promising solution for providing long-term rehabilitation at lower costs. At the same time, early discharge is recommended for patients with less severe degrees of disability2. The system allows for early discharge while still being monitored remotely. Finally, the concept addresses a frequently-reported issue for rehabilitation professionals, i.e., lack of quantitative measures on the progress of rehabilitation. Standard functional scales, such as the FIM functional motor index, are in fact partially subjective. The diffusion of computerized monitored rehabilitation aids may increase the objectivity of practice and allow for quantitative progress evaluation and therapy comparisons. Acknowledgments The authors would like to thank F. Lorussi and Prof. D. De Rossi (University of Pisa); Dr. G. Maggioni and R. Mauri (Fondazione Salvatore Maugeri, Pavia); Prof. M. Stefanelli (University of Pavia); and D. Tietze and R. Setz (Dr. Hein GmbH) for essential contributions to the project. The work described in this paper was partly funded by project “MyHeart” no. IST-2002-507816 in the Sixth Framework of EU IST projects.

Figure 8 – Sensor layout affects recognition accuracy (exercise: sideways arm abduction) The recognition algorithm used for the classification and real-time feedback on the exercises has been validated on several garment prototypes. The recognition is based on a “dissimilarity score” which is output by the recognition algorithm after a per-session calibration. A volunteer executed the rehabilitation exercises in sequence (table 1). Each exercise was repeated 10 times correctly, then 5 times mimicking the most frequent compensatory movements. Two distributions of “dissimilarity scores” were therefore obtained. The amount of overlap between distributions is proportional to the probability of a correct/ incorrect misrecognition. Quantitative measures allowed redesigning the sensor layout in order to maximise the classifier margin (figure 8).

References [1] The Italian Guidelines for stroke prevention. The Stroke Prevention and Educational Awareness Diffusion (SPREAD), IV edition, 2005. [2] http://www.dr-hein.com (Accessed 25 March 2007) [3] Schoch KF; Saunders HE. Conducting polymers, Spectrum, IEEE, vol.29, no.6, pp. 52-55, Jun 1992. [4] Tognetti A, Lorussi F, Bartalesi R, Quaglini S, Tesconi M, Zupone G, De Rossi D. Wearable kinesthetic system for capturing and classifying upper limb gesture in post-stroke rehabilitation, J Neuroengineering Rehabil, vol. 2, no. 1, pp. 8-8, Mar 2005. [5] Johanni M, Tietze D, Setz R, Hein A. EvoCare: a new standard in tele-therapy. Stud Health Technol Inform. 2004;108:228-34. [6] Luprano J, Sola J, Dasen S, Koller JM, Chetelat O. Combination of body sensor networks and on-body signal processing algorithms: the practical case of MyHeart project. Wearable and Implantable Body Sensor Networks, 2006. BSN 2006. 3-5 April 2006. [7] Giorgino T; Quaglini S; Lorussi F; De Rossi D. Experiments in the detection of upper limb posture through kinestetic strain sensors. Wearable and Implantable Body Sensor Networks, 2006. BSN 2006. 3-5 April 2006.

Conclusion Development of the NR rehabilitation system followed an integrated approach which involved research on tissueembeddable strain sensors, portable wireless electronics and ergonomic feedback, in order to be easy to use for patients which are often cognitively-impaired. Its most innovative component is the exercise recognition algorithms, which are able to provide feedback in real time, insensitive to several sources of noise stemming from the inherently noisy, multi-sensor approach taken.

Address for correspondence

The integrated nature of the NR system is now intended to support a multidisciplinary, telemedicine-based approach to rehabilitation. The benefits of this (and similar) ITbased aids are supported by current beliefs on rehabilitation, e.g. those expressed in the SPREAD guidelines [1]. First of all, long term rehabilitation programmes are considered beneficial for the recovery process1. However, 1

Email address: [email protected]

“A long-term rehabilitation programme promoting independence in daily activities is recommended to reduce the deterioration of the independence status obtained by the intensive or extensive rehabilitation” (Recommendation 14-73).

2

96

“In patients with slight-to-moderate disability following stroke, early discharge from the rehabilitation hospital is recommended […] (R. 14-61).”

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Effects of a Computerized Cardiac Teletriage Decision Support System on Nurse Performance: Results of a Controlled Human Factors Experiment Using a Mid-Fidelity Prototype Kirsten Carroll Somozaa, Kathryn Momtahanb, Gitte Lindgaardc Human-Oriented Technology Lab, Carleton University, Canada b University of Ottawa Heart Institute, Canada c Human-Oriented Technology Lab, Carleton University, Canada

a

The purpose of this research was to design an effective Decision Support System (DSS) to help primary healthcare professionals field these calls. The final DSS will initially be used by the NCs, but aims to assist less experienced care providers in the future. The literature suggests that challenges lie both in the practice of teletriage [3, 4, 5, 6], and in the design and development of decision support systems [7, 8, 9, 10, 11]. To the extent that an inadequate design may rest on poorly defined user tasks, one would expect improved DSSs to result if that limitation were to be overcome. This research therefore explored the use of task analysis as part of a user centered design process aiming to produce an effective DSS.

Abstract: A gap exists in cardiac care between known best practices and the actual level of care administered. To help bridge this gap, a proof of concept interface for a PDA-based decision support system (DSS) was designed for cardiac care nurses engaged in teletriage. This interface was developed through a user-centered design process. Quality of assessment, quality of recommendations, and number of questions asked were measured. Cardiac floor nurses’ assessment quality performance, but not their recommendation quality performance, improved with the DSS. Nurses asked more questions with the DSS than without it, and these additional questions were predominantly classifiable as essential or beneficial to a good assessment. The average participant satisfaction score with the DSS was above neutral.

In this study, cardiac best practice algorithms are introduced via a handheld DSS. Access to these best practices is expected to improve the quality of teletriage performance [1], provided that a) the best practices are sound, and b) the DSS delivers them so that they are easily used by NCs while on the phone with a patient. This research encompassed the design and evaluation of the DSS, but did not evaluate the best practice algorithms.

Keywords: Teletriage, user-centered design, user interface design, mobile computing, decision support system

Introduction

The hypotheses were: (1a) that nurses’ performance in assessment of a patient’s problem will improve using the DSS, and (1b) that nurses’ performance in recommendations for the patients will improve using the DSS; and (2) that nurses will ask fewer questions with the DSS than without it.

As the number of Canadian cardiac patients increases and the length of hospital stay decreases, there is a need for expert care and evaluation once patients return home from the hospital. The current services of homecare agencies, community hospitals, and family physicians are inadequate to meet the needs of this growing patient base for two main reasons:

Since there were many aspects to this research, this article will focus primarily on the results comparing nurses’ performance with and without the DSS. The design process, and the design itself, will not be discussed at length here.

1. inadequate access to professionals (many patients may have to wait several weeks before seeing their family physician once they leave hospital) [1] and 2. limited professional access to up-to-date, best practice guidelines for cardiac care [2]. The Nursing Coordinators (NCs) at the University of Ottawa Heart Institute (UOHI) receive more than 2000 calls per year from patients requesting help and advice for cardiology and cardiac surgery concerns. Though teletriage is significant in the care of these patients, there is no standardized protocol in place to guide the advice-giving, nor is there any formal training for staff.

Materials and methods Development of pain scenarios and standardized patient profiles Two Subject Matter Experts (SMEs) developed four pain scenarios, based on best practice algorithms that were determined with a larger expert team. There was one scenario for each of the following types of pain complaint: non-urgent cardiac surgery, urgent cardiac surgery, nonurgent cardiology, and urgent cardiology. The following

97

K. C. Somoza et al. / Effects of a Computerized Cardiac Teletriage Decision Support System on Nurse Performance

specific ailments were chosen as common examples of each type of pain (see Table 1).

The presentation of the four pain scenarios was randomized to avoid serial order effects. In all scenarios, the SME in the role of the patient made an effort to answer the nurse’s questions in an appropriate way, neither volunteering more information than requested, nor being furtive and vague. For each pain scenario, the SME referred to the corresponding standardized patient profile during the call to ensure she was giving consistent answers. Each standardized patient profile listed, in logical order determined by the algorithm, questions that the NC would likely ask with corresponding answers for that scenario. The profiles also contained answers to questions not necessarily in the optimal flow of questioning. Throughout the baseline and DSS tests, if a participant asked a question that was not on the list, the SME gave an answer consistent with the cause of pain in that scenario. She then added the new question and answer to the list, ensuring that a consistent response would be given should the same question be asked by another participant. To preserve the SME’s anonymity and increase semblance to a real telephone call, the SME was in another room and communicated via telephone throughout the test.

Table 1- Pain scenarios Scenario

Cardiac care area

Urgent or non-urgent

1: Post Pericardiotomy Syndrome (PPS)

Cardiac surgery

Non-urgent

2: Ischemic Pain

Cardiology

Urgent

3: Cardiac Tamponade

Cardiac surgery

Urgent

4: Stent Pain

Cardiology

Non-urgent

Each pain scenario was developed with a standardized patient profile whose characteristics matched the algorithm for the pain complaint in question. These simulated “patients” were created by the two SMEs, based on the algorithms and their practical experience.

The researcher read an introductory script before the pain scenarios began, telling the nurse that they would be acting in the role of an NC, and that their task was to assess the patient’s condition and advise them accordingly. To more closely mimic real-life use, nurses were asked to hold the telephone handset to their ear during the call even though the speaker phone was in use to record the session, enabling the researcher also to hear the “patient’s” responses. Participants proceeded through the phone call using a pen and paper to record whatever they wished. The participant marked the end of the call by hanging up the telephone handset.

Baseline test – Nurses’ performance without the DSS Because some of the NCs participated in development of the algorithms and because there are very few NCs at the UOHI, the NCs did not participate in DSS prototype testing or the baseline test. Instead, participants were cardiac care nurses from the UOHI who work with hospitalized cardiac patients. Twelve nurses completed the baseline testing, in which their performance was assessed without the DSS. During the baseline test, nurses used only pen and paper while on the phone with the patient, as these are the tools normally available to them. Seven of these nurses subsequently completed the DSS test, which scored their performance with the DSS prototype and helped the researcher evaluate the DSS interface. During both the baseline and DSS tests, an SME acted as the patient on the other end of the phone. This SME also acted as the UOHI attendant who transfers incoming patient calls to the NCs. The study did not use real patients as participants.

DSS Test – Nurses’ performance with the DSS prototype Seven of the original 12 participants were available and completed the proof-of-concept DSS test, on a mid-fidelity prototype. These sessions were scheduled approximately six weeks following the baseline testing. The same SME who participated in the baseline test acted again in the role of the patients for the DSS test. The SME did not personally evaluate the interface during the DSS test. Apparatus was similar to the baseline test with the addition of a Palm Tungsten T3 PDA including stylus, and the DSS prototype running on a Toshiba Portégé M200 tablet PC. The T3 was presented with only the software that comes installed from the manufacturer. The DSS prototype incorporated best practice algorithms for cardiac pain into an interface that supported nurses in asking the correct questions while they were assessing the patient, and assisted the gathering of patient information. With the exception of the first participant in the trial, participants did not have access to a pen and paper for the DSS test. The pen and paper were removed to encourage participants to use the prototype rather than the paper. All interaction with the tablet occurred through the included stylus.

This baseline provided initial data for the number of questions asked by each participant in each scenario, and for performance scores on quality of assessment and quality of recommendations. “Quality of assessment” measured nurses’ ability to ask the right questions in assessing the patient’s condition. “Quality of recommendations” measured their ability to give the patient correct advice based on the presenting problem in each scenario. The scoring of the quality of assessment and quality of recommendations was done independently by the two SMEs after many practice sessions and iterations of the scoring sheet. For the four scenarios tested, the average inter-rater reliability (IRR) for quality of assessment was .94, ranging from .83 to .98. For quality of recommendations, average IRR was .97, ranging from .86 to 1.

98

K. C. Somoza et al. / Effects of a Computerized Cardiac Teletriage Decision Support System on Nurse Performance

Figure 1 shows the quality of assessment scores for each scenario. While this figure suggests that performance scores improved substantially from the baseline to using the DSS in the case of Stent Pain and Post Pericardiotomy Syndrome (PPS) – the non-urgent scenarios, this improvement was less evident in the case of Ischemic Pain and Cardiac Tamponade – the urgent scenarios. A Wilcoxon Signed Ranks Test indicated that, statistically, quality of assessment did improve for every scenario with the DSS (p < 0.05). These results support one of the experimental assumptions, namely that that nurses’ performance in assessment would improve using the DSS.

Tablet PC interaction exercise Each nurse completed a short interaction exercise on the Tablet PC. This was not recorded and served only to give each nurse equal opportunity to practice interacting with the device via the stylus. All participants were first-time Tablet PC users. Practice scenario After the first participant, it became apparent that the nurses would likely benefit from some practice on the system before being asked to use and evaluate it in a simulated working environment. Therefore all subsequent participants received the following practice scenario. The researcher walked the nurse through a sample scenario, which was based on a fictitious patient with a chest cold rather than a cardiac pain concern. Then the researcher played the role of that same patient, and allowed the nurse to click through the tool on their own while asking questions of the patient. This gave them extra practice with the stylus, and with the prototype interface elements.

Quality of recommendations

Pain scenarios The pain scenarios proceeded exactly as in the baseline test, but the participants used the prototype DSS to assist in assessing the patient. Scenario order was randomized, and participants received them in a different order than in the first session to avoid order effects. Patient and physician names were changed from the baseline test. Figure 2 - Quality of recommendations, by scenario

As participants progressed through a patient call using the DSS, they saw questions on each screen (for example, “What makes the pain worse?”) and could input patient responses through checkboxes, pick lists, and freeform fields. Participants had freedom to move about within the interface, and were not instructed to read each question in sequence. At the point just before the software would throw a “flag” based on the input so far, suggesting a possible presenting issue and a recommended course of action, the participant was asked to guess what the patient’s problem was and give their own recommendations. The automated assessment flag was then revealed, and the participant was asked whether they agreed or disagreed with the DSS’ suggestions.

Figure 2 suggests that recommendation performance declined at least slightly across all scenarios. This decline was most evident in the Cardiac Tamponade (urgent) and Post Pericardiotomy Syndrome (PPS) (non-urgent) scenarios. However, contrary to the predictions, the Wilcoxon Signed Ranks Test revealed no significant effect (p > 0.05). In seven cases out of 28, the nurse did not correctly guess the patient’s problem, but they agreed with the automated DSS assessment. In these cases, recommendation quality might have improved in real life, had they seen the automated DSS flag before advising the patient – this assessment flag offered a possible condition, as well as recommended course of action.

Results

Why was there no improvement in recommendation scores with the DSS? The nurses were provided with essential and beneficial questions to ask (contributing to the quality of assessment score), but they were not provided with the recommendations contained on the DSS. In clinical practice, telepractitioners would not be forced to advise the patient without seeing the DSS recommendations. Additionally, cardiac floor nurses would be expected to give good recommendations without assistance, so improvement may be observed more readily in a more novice user group. Nurses’ lack of experience with the technology (PDAs and the tablet test platform) may have interfered with their ability to synthesize information during the phone call because a lot of their attention was devoted to trying to use the technology. Also, since this was the first time the pain scenarios were used, it is possible that these could benefit from further iteration. Finally, the floor

Quality of assessment

Figure 1 - Quality of assessment, by scenario

99

K. C. Somoza et al. / Effects of a Computerized Cardiac Teletriage Decision Support System on Nurse Performance

nurses participating in the study were not used to performing teletriage, which could have contributed to low recommendation scores both with and without the DSS.

Figure 4 shows the proportion of assessment questions asked that were on the list compared to not on the list. It suggests that most of the additional questions asked with the DSS were marked in the SMEs’ list. Furthermore, the number of questions not on the list remained fairly constant across scenarios between the baseline and DSS.

Number of questions asked

A Wilcoxon Signed Ranks Test indicated that nurses asked significantly more questions on the list with the DSS than without, but there was no significant difference in the number of questions asked that were not on the list. Earlier, it was reported that assessment scores increased with DSS use. This supports the results we see here, since nurses asked more essential and beneficial assessment questions with the DSS than without it. So at the same time this figure refutes hypothesis (2), it helps confirm hypothesis (1) in that asking more essential and beneficial questions with the DSS contributed to participants’ higher assessment scores. On later discussion between the SME and an NC, it came to light that less experienced teletriage practitioners don’t ask enough questions – this supports the literature [6]. In this respect, hypothesis 2 turned out to be misguided; it should have predicted an increase rather than a decrease in the number of questions asked, provided that additional questions asked could be shown to be relevant and beneficial to the final recommendation.

Figure 3 - Average number of questions asked by nurses during assessment Hypothesis (2) predicted that nurses will ask fewer questions with the DSS than without it. A Wilcoxon Signed Ranks Test, comparing the number of questions asked in the baseline test and the DSS test, indicates that participants in fact asked significantly more questions with the DSS than without it (Z= -2.31, p respondent Strong

Wikis Many-to-many Topical Change Synthesized into a single document Author = respondent Weak

Discussion Forums Many-to-many Chronological within topics Append Fragmented across entries and replies Author = respondent Strong

Informal

Formal

Informal

We define "identity" as a user's real name/credentials or as a persistent online pseudonym.

subject lines together on one page, separating the dialog of each topic to individually accessible sub-pages. Forums also can include hierarchical categories and sub-forums akin to folders in a file system for organizing topics (attribute 2). Each topic page displays responses in chronological order (i.e. most recent at the bottom), such that new responses are appended to existing ones (attribute 3). Despite the topical and chronological arrangement, knowledge still can be fragmented across entries and responses over time (attribute 4). The typical discussion forum interface provides the same formatting for all responses in a topic so that all appear equal. Additionally, the creator of a topic and all respondents have the same abilities to include text, images, and other media when they contribute content (attribute 5). The discussion forum interface displays a user's identity alongside each topic or response he or she contributes (attribute 6). The tone of voice in forums is informal and conversational, often with the use of the firstperson speaking voice (attribute 7). Discussion forums are a mature technology, and healthcare researchers have studied them extensively in terms of online social support for cancer patients and other purposes [8].

Blogs A blog (an abbreviation for "web log") is a website that facilitates individual expression using a one-to-many publication model, in which a single author creates entries that any number of readers can access (attribute 1) [4]. Blog software displays entries in reverse chronological order (i.e. most recent at the top) (attribute 2). In a blog, a response is called a "comment" and consists of a text reply and respondent's name. Comments are appended to entries and do not replace the original content of the entry (attribute 3). Knowledge, the interpreted understanding of information, is fragmented across the various entries and responses; blogs lack a formal structure for the synthesis of information contributed over time (attribute 4). The blog interface places greater emphasis on the author's entry than the respondents' comments (attribute 5). For example, the interfaces for commenting often have fewer options compared to the interface for writing original entries. Comments frequently are displayed with a smaller font and sometimes are visible only by visiting a page separate from the main page of entries. Furthermore, the blog author has control over the privacy levels of the entries, and the author also has the ability to delete comments submitted by others. The identity of each individual contributor is visibly labeled for each entry and comment, except in the case where authors permit respondents to be anonymous (attribute 6). Related to the purpose of selfexpression, the tone of voice in blogs is informal and spontaneous. Examples of existing health care related blogs include physicians and medical students sharing experiences and discussing clinical topics [5, 6], as well as patients updating family, friends, and/or the general public about treatment, recovery, and healthcare experiences [7].

Wikis A wiki is a website that facilitates document creation using a many-to-many publication model; any user can be an author by creating entries that any number of users can view (attribute 1). In a wiki, an entry is the creation of a page about a specific topic (attribute 2), and a response is modification to an existing entry (attribute 3). Wiki entries can link to other entries, as well as to other web resources. Because an entry's content is updated and/or replaced with new information, knowledge pertaining to a given topic is continually synthesized and displayed in a single cohesive document (attribute 4). The wiki interface provides no easily visible connection between individuals and their contributions (attribute 6). The interface also de-emphasizes any distinction between the original author and the respondents in favor of stressing the entry's content (attribute 5). Because the purpose of a wiki is to create documents, authors and respondents tend to write in a formal tone (attribute 7). Examples of healthcare wikis include Wikicancer, a social support resource for cancer patients and their families [9], and Clinfowiki, a growing

Discussion forums A discussion forum is a website that facilitates group dialog using a many-to-many publication model; each user can author entries that any number of readers can access (attribute 1). When an author creates an entry, called a "topic," the intent is for dialog to occur on a subject specified by the author. Users freely respond to the topics of their interest. The discussion forum interface lists topic

158

J. B. Weiss et al. / Blogs, Wikis, and Discussion Forums: Attributes and Implications for Clinical Information Systems

encyclopedia of medical informatics concepts sponsored by the Informatics Review [10].

discussion forum or blog model would not be satisfactory because this shift might alter other attributes that reduce the effectiveness of the problem list. The developers therefore can focus on redesigning the application with respect to the visibility of a contributor's identity (attribute 6) rather than redesigning the entire interface.

Case studies The authors present two case studies that demonstrate a systematic approach to analyzing the design of clinical information systems in terms of the attributes defined in Table 1. The first case study evaluates a “problem list” in an electronic medical record, and the second assesses online social support for cancer survivors.

The attributes defined in this paper do not offer a specific design solution for each problem, but the interfaces of existing blog and discussion forum software may provide inspiration for different ways to identify the author of a specific entry. Informaticians also can study wiki-like software to learn how these applications have addressed similar challenges. The proposed attributes should be used as an objective guide to support the subjective judgment of the developers and researchers in making design choices. A possible resolution for this problem list implementation might be to display the role of the individual who entered each item by displaying each portion of the text with different background colors that represent "primary care physician," "specialist," “nurse,” and “care tech.”

“Problem list” in an electronic medical record StarPanel, a web-based electronic medical record (EMR) developed at Vanderbilt University, contains a "problem list" for quick viewing (Figure 1) and/or updating (Figure 2) of a patient's current diagnosis, medications, and related clinical information [11]. Many aspects of the problem list resemble a wiki. All providers can create and access the problem list content (attribute 1), as well as modify the information (attribute 3). The interface presents information by clinical topic—"significant medical conditions, significant procedures, allergies and drug reactions, current medications, health maintenance, social and family history" (attribute 2) [11]. An advantage of this system attribute is that information stored by topic in discrete fields can be reused in other parts of the EMR [11]. Only the most recent version of the list is displayed; the chronology of individual updates is not immediately visible, although the user can access the previous versions of the problem list to look for changes. The interface provides a single, patient-centered source for quick access to the most recent and most relevant clinical information (attribute 4). The original author's entries are not presented differently from or emphasized more or less than each respondents' changes, although the most recent respondent is highlighted at the top of the page (attribute 5). The interface does not clearly identify each provider's contributions to the problem list (attribute 6), which may interfere with the ability of providers' to coordinate care. This example of clinical documentation features a tone of voice that is straightforward, objective, and formal (attribute 7). However, the tone of voice in the text can also be considered informal due to sentence fragments and piecemeal thoughts, which make the entry somewhat more blog-like (attribute 7).

Figure 1 - Problem list view

This case study illustrates how the proposed attributes can be used explicitly to evaluate a clinical information system. For the most part, the StarPanel problem list has the attributes of a wiki. A disadvantage of the current implementation is that users cannot easily determine the individuals who contribute specific parts of the problem list. As personal health records and patient portal systems begin to give patients the ability to update portions of their own medical records, the system interface must clearly display the identity and role of the person responsible for each specific update. Rather than viewing this complication as a reason against using a wiki-based approach for the problem list, one can instead determine which of the wiki-like attributes is problematic. Simply switching to a

Figure 2 - Problem list edit Case study: Online social support for cancer survivorship An estimated 10 million individuals treated for cancer are living in the United States today [12]. Improving the qual-

159

J. B. Weiss et al. / Blogs, Wikis, and Discussion Forums: Attributes and Implications for Clinical Information Systems

ity of life of these individuals currently is a clinical challenge and priority research goal [13]. The literature includes a growing number of studies that address the use of online cancer support groups by patients and/or their informal caregivers [14, 15]. Social support is a complex concept and includes dimensions of informational support as well as emotional support [16].

all be emphasized equally in the interface design? Different sets of attributes may be appropriate for different medical or social contexts. These choices have implications for patient-provider relationships, information ownership, information filtering, social support, and other socio-technical aspects of clinical care. By breaking down each technology into its fundamental elements, developers can select the desired properties of blogs, wikis, and forums to create interfaces that better match the goals of certain information system components. Several authors have suggested that further research is needed on how communication tools can effectively work in parallel and how they can be combined to create new technologies [3, 21-23]. For example, a “bliki” is a term that describes the combination of blog and wiki functionality [21]. The attributes proposed in this paper represent a first step toward a formal method of generating new ideas for system design by combining specific properties of emerging web collaboration software. The method could apply to health care applications as well as webbased communication applications in other domains.

Online social support interventions in health care have focused on discussion forums, newsgroups, chat rooms, and email listservs [14]. Online support groups only recently are beginning to include online communication technologies with blog and wiki functionality [9, 17]. Cancer patients and caregivers have used discussion forum software to facilitate both informational and emotional social support. Messages in online cancer support groups often include the sharing of information, encouragement, humor, and prayer [18]. Researchers have not fully considered when discussion forums might or might not be the most appropriate asynchronous collaboration tool for online social support Independently addressing each aspect of social support might better address the advantages and disadvantages of discussion forums, blogs, and wikis. Blogs may be a powerful interface for informal, personal expression related to cancer diagnosis and survivorship (i.e. emotional support). Additionally, blogs may be a good format for providers to inform patients of timely announcements, such as new clinical trials or fundraiser events (i.e. informational support), as they present information in a reverse chronological fashion (attribute 2). A discussion forum might be appropriate for the process of discussing information that involves a variety of opinions, such as advice on certain types of alternative treatments. However, the information and knowledge created through these discussions will be fragmented in messages over time. A wiki might be an appropriate tool for creating an information resource that summarizes the collective knowledge of cancer survivors, family, friends, and providers on certain topics [17], such as what to expect during the initial period following chemotherapy and radiation treatment [19]. Adjusting the interface to clarify the identity or role of the individual who contributes each part of the document could improve the credibility of the information, as discussed previously in terms of a collaborative problem list. These examples indicate that interactive health collaboration systems may need to offer a variety of communication technologies to facilitate both the emotional and informational social support needs of cancer survivors.

This study has limitations that warrant discussion. As seen in the evaluation concerning tone of voice in the problem list case study, it is not always clear which attribute value is the most appropriate description for a given interface. Further refinements to the attribute definitions by the authors and the informatics community will be needed. The attributes are based on the literature and the authors' personal involvement in design and use of web technologies, and experimentally validating the effectiveness of these definitions on existing software applications is planned for future studies. We have excluded podcasts, newsgroups, instant messaging, and other popular technologies from the current discussion because they do not meet our inclusion criteria of primarily text-based, asynchronous web-based software. Additional analysis should extend the attributes defined in Table 1 to address these technologies as well.

Conclusion Formally analyzing the attributes of emerging web technologies will support the creation of improved design approaches for health communication. The authors define seven attributes for researchers and developers to use in the design and evaluation of clinical information systems. Two case studies illustrate the application of the proposed attributes to improve the coordination of clinical care and social support.

Discussion

Acknowledgements

Developers of clinical applications create interfaces that inherently have attributes representative of wikis, blogs, discussion forums, or some combination of the three. These choices will fundamentally affect the way in which providers and patients use health information systems [20]. For example, if a patient, her providers, and her family members can all update a personal health record, should entries by the patient be emphasized more than the others, or should the patient, doctors, and family members

The authors would like to thank Drs. Eta S. Berner, Joshua C. Denny, and Dario A. Giuse for their contributions to this paper. This work was supported by NLM T15 007450-03

References [1] Coiera E. When conversation is better than computation. J Am Med Inform Assoc. 2000 May-Jun;7(3):277-86. [2] Mongkolwat P, Kogan A, Koh J, Channin DS. Blogging your PACS. J Digit Imaging. 2005 Dec;18(4):326-32.

160

J. B. Weiss et al. / Blogs, Wikis, and Discussion Forums: Attributes and Implications for Clinical Information Systems [3] Boulos MN, Maramba I, Wheeler S. Wikis, blogs and podcasts: a new generation of Web-based tools for virtual collaborative clinical practice and education. BMC medical education [electronic resource]. 2006;6:41. [4] Mortensen T, Walker J. Blogging Thoughts: Personal Publication as an Online Research Tool. In: Morrison A, ed. Researching ICTs in Context. Oslo: InterMedia, University of Oslo 2002. [5] Centor R. DB's Medical Rants. [cited 2006 November 30]. Available from: http://www.medrants.com [6] Walker G. Over!My!Med!Body! [cited 2006 November 30]. Available from: http://www.grahamazon.com/ [7] CaringBridge: Free online service connecting family and friends. [cited 2006 November 30]. Available from: http://www.caringbridge.org/ [8] Gustafson DH, Hawkins R, Pingree S, McTavish F, Arora NK, Mendenhall J, et al. Effect of computer support on younger women with breast cancer. J Gen Intern Med. 2001 Jul;16(7):435-45. [9] The Cancer Survivors Network, supplying information, resources, chat, stories, discussion forums and support for cancer survivors and caregivers. [cited 2006 November 30]. Available from: http://www.acscsn.org [10] Clinfowiki. [cited 2006 November 30]. Available from: http://www.clinfowiki.org/wiki/index.php/Main_Page [11] Denny J, Guise DA, Jirjis JN. The Vanderbilt experience with electronic health records. Sem in Colon and Rectal Surg. 2005 2005;16(2):59-68. [12] Cancer survivorship--United States, 1971-2001. Morb Mortal Wkly Rep. 2004;53(24):526-9. [13] Hewitt M GS, Stovall E. From Cancer Patient to Cancer Survivor: Lost in Transition. Washington, DC: The National Academies Press 2005. [14] Eysenbach G, Powell J, Englesakis M, Rizo C, Stern A. Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions. BMJ (Clinical research ed. 2004 May 15;328(7449):1166.

[15] Klemm P, Bunnell D, Cullen M, Soneji R, Gibbons P, Holecek A. Online cancer support groups: a review of the research literature. Comput Inform Nurs. 2003 May-Jun;21(3):136-42. [16] Williams P, Barclay L, Schmied V. Defining social support in context: a necessary step in improving research, intervention, and practice. Qualitative health research. 2004 Sep;14(7):942-60. [17] Cancer: Stories, support and information. [cited 2006 November 30]. Available from: http://www.wikicancer.org/ [18] Klemm P, Reppert K, Visich L. A nontraditional cancer support group. The Internet. Computers in nursing. 1998 Jan-Feb;16(1):31-6. [19] Stanton AL, Ganz PA, Rowland JH, Meyerowitz BE, Krupnick JL, Sears SR. Promoting adjustment after treatment for cancer. Cancer. 2005 Dec 1;104(11 Suppl):2608-13. [20] Wyatt JC, Wright P. Design should help use of patients' data. Lancet. 1998 Oct 24;352(9137):1375-8. [21] Bliki. [cited 2006 November 30]. Available from: http://en.wikipedia.org/wiki/Bliki [22] Woods L. Blogs & Wikis: Technologies for Enterprise Applications? [cited 2006 November 30]. Available from: http://gilbane.com/gilbane_report.pl/104/ Blogs__Wikis_Technologies_for_Enterprise_Applications. html [23] Herring S, Scheidt L, Bonus S, Wright E. Bridging the Gap: A Genre Analysis of Weblogs. HICSS '04: Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS'04) - Track 4; 2004: IEEE; 2004.

Address for correspondence Jacob Weiss 400 Eskind Biomedical Library 2209 Garland Ave. Nashville TN 37232 USA [email protected]

161

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

From Intermediation to Disintermediation and Apomediation: New Models for Consumers to Access and Assess the Credibility of Health Information in the Age of Web2.0 Gunther Eysenbach a,b b

a Department. Of Health Policy, Management, and Evaluation, University of Toronto, Canada Centre for Global eHealth Innovation, University Health Network, Toronto General Hospital, Toronto, Canada

ers can now not only access an unprecedented amount of health information, but increasingly also personal information from their electronic health record [1]. With direct and convenient access to an abundant amount of health information on the Internet, consumers now bypass the expert intermediary and gain direct access to unfiltered information [2]. In this situation, consumers have to assume new responsibilities for assessing the credibility of the information, and intermediaries sometimes defend their role as “gatekeeper” using quality arguments.

Abstract This theoretical paper discusses the model that, as a result of the social process of disintermediation enabled by digital media, traditional intermediaries are replaced by what this author calls apomediaries, which are tools and peers standing by to guide consumers to trustworthy information, or adding credibility to information. For apomediation to be an attractive and successful model for consumers, the recipient has to reach a certain degree of maturity and autonomy. Different degrees of autonomy may explain differences in information seeking and credibility appraisal behaviours. It is hypothesized that in an apomediated environment, tools, influential peers and opinion leaders are the primary conveyors of trust and credibility. In this environment, apomediary credibility may become equally or more important than source credibility or even message credibility. It is suggested to use tools of network analysis to study the dynamics of apomediary credibility in a networked digital world. There are practical implications of the apomediation model for developers of consumer health websites which aspire to come across as “credible: Consumers need and want to be able to be co-creators of content, not merely be an audience who is broadcasted to. Web2.0 technology enables such sites. Engaging and credible Web sites are about building community and communities are built upon personal and social needs.

As the role of “human” intermediaries diminishes or changes, consumers and patients are finding new ways to arrive at relevant and credible information. This can be human beings (peers) and/or technology (e.g., collaborative filtering tools). In this paper, the author proposes to refer to these new intermediaries as “apomediaries”, because they mediate without standing “in between” consumer and services or information. Rather, they “stand by” and provide added value from the outside, steering consumers to relevant and high-quality information without being a necessary requirement to obtain the information or service (Fig. 1). While intermediaries provide “upstream filtering”, apomediaries enable and facilitate “downstream filtering” [3]. Apomediaries can help to navigate through the onslaught of information, give additional credibility cues, and provide metainformation. Examples for apomediaries are consumer ratings on Amazon or epinions, technologies like PICS or MedPICS labels and their semantic web successors [4;5] enabling the machine-processable dissemination of such ratings, collaborative filtering and recommender systems such as StumbleUpon.com, and other second generation (sometimes called Web 2.0) Internet-based services that let people collaborate and share information online in a new way - such as social networking sites, wikis, communication tools, and folksonomies.

Keywords: internet, consumer health informatics, information quality, credibility

From disintermediation to “apomediation” The debate on quality and credibility in the digital age is a result of a social process of disintermediation through digital technologies, and the health industry is no exception: Just as in many other areas of life (e.g., travel industry), information and communication technologies empower consumers and enable them to cut out the middleman or intermediary (travel agents, real estate agents, librarians, pharmacists, health professionals) to access pertinent information or services directly, whenever they need it and where they need it. For instance, on the Internet, consum-

Disintermediation not only takes place on a society level in health care and other industries, but there are also parallels to the individual emancipation process that takes place for example during puberty, when adolescents strive to become more autonomous and have the desire to reduce the influence of the intermediary (parents), with peers

162

G. Eysenbach et al. / From Intermediation to Disintermediation and Apomediation: New Models for Consumers

(apomediaries) partly taking over the role of the former intermediary.

General implications of disintermediation

The disintermediation/apomediation model is useful because it allows us to analyze and discuss the implications of the disintermediation process at the societal level, for example for consumers of entire industries (ehealth), and to draw analogies to what is happening at the individual level during adolescence due to the emancipation process from traditional authorities and the use of digital media. These observations are free of judgment – it is not implied that the disintermediated / apomediated model is always better than the intermediated model. Rather, which model is “better” depends on the individual and the respective situations. In the following, the author will first discuss general implications if disintermediation takes place, and will then consider credibility implications.

Disintermediation enabled or enhanced through technology changes the role of the recipient (consumer, patient, youth), who now has the choice to determine whether, when and how they choose to use the intermediary. The better informed the recipient is (or perceives to be), the better he knows what information or services he needs, the less likely he will need an intermediary. For example, a consumer with a chronic condition (e.g., diabetes) will have a greater knowledge and self-efficacy to critically appraise information found on the Internet than a consumer with an acute illness, and will not need an intermediary. Similarly, an older adolescent eager to learn about sexuality is less likely to rely on an intermediary such as a parent or teacher as filter than a younger child.

Table 1 - Dichotomies in the intermediation versus apomediation model.

With increased literacy (including the ability to distinguish different types of information) and knowledge, i.e., when the receiver is knowledgeable about message content, the effects of source expertise will be attenuated, i.e., the credibility of “experts” and other authorities decreases [6], leading to an interesting positive feedback loop, where consumers learn to rely less and less on experts or intermediaries, preferring apomediation instead (Fig. 2). Again, there are parallels to what is happening during adolescence when youths learn to emancipate themselves from traditional authority figures.

Intermediation Model Dependency–Paternalistic System– Acute Illness–Pre-adolescent Kids– Illiterate Consumers

Decreased reliance on the intermediary

Apomediation Model Autonomy–Net–Chronic Illness– Adolescents–Literate Consumers

Traditional

Digital

Centralized

Highly Networked

Managed environment

Autonomy, emancipation

Dependence on Intermediaries (physicians, parents)

Guidance by Apomediaries (peers, Web 2.0 technology)

Credibility of Authorities/Experts

Credibility of Peers

Power held by intermediaries

Empowerment of consumers/ youth

Source expertise = traditional credentials (seniority, professional degrees etc)

Source expertise = first-hand experience, peers

Message credibility: professional language, message “length is strength”, comprehensiveness signifies expert status

Message credibility: Understandable language, “street cred”

Top-down

Bottom-up

More formal learning

More informal learning

Static hubs

Dynamic hubs

Source credibility more important than message credibility

Message credibility and credibility of apomediaries more important than source credibility

Figure 2 - Dynamic Disintermediation/Apomediation Model: Information is initially mediated and filtered by an intermediary. Once a critical threshold of knowledge, self-efficacy and autonomy is exceeded, apomediation can replace the traditional intermediary, while the recipient still has the option to choose the intermediary in case of failure.

Power shift As a result of disintermediation, the power-relationship between recipient and intermediary changes. This may create conflicts. For instance, a significant minority of health care providers see their authority challenged, perceive a deterioration in the physician-patient relationship, and fear a negative impact on the quality of health care or health outcomes, although most embrace the shift from a paternalistic to a partnership model [2;7-12]. Parents may be equally irritated when the youths are searching for information on birth control on the Internet rather than discussing these issues with them or a physician. Figure 1 - Disintermediation and apomediation (circles = apomediaries assisting in “downstream filtering”)

Concerns have been expressed that more information does not necessarily translate into more knowledge or better

163

G. Eysenbach et al. / From Intermediation to Disintermediation and Apomediation: New Models for Consumers

quality (self-)care, in particular as some of the quality and credibility of health information on the Internet clearly does not live up to professional standards. This view fails to recognize apomediation as an alternate mechanism to filter information.

capacity to make an informed decision) play a critical role in information seeking and credibility assessment behaviour. In other words, autonomous individuals are more likely to choose an apomediation model, and apomediation in turn increases autonomy. Navigation in the digital world requires and at the same time enables a high degree of autonomy.

Interestingly, the model presented above (Figure 1) also helps to explain some of the perceptions and frustrations intermediaries (health professionals, parents) often have with the disintermediation process, as they tend to see predominantly the “failures”, for instance patients having found irrelevant information on the Internet.

As mentioned above, there are parallels between certain groups of consumers who prefer a more paternalistic patient-doctor relationship and pre-teen dependent children on one hand, and empowered consumers and teenagers striving for autonomy on the other hand. For example, a chronically ill patient is more autonomous than a traffic accident victim, and will be more likely to seek out information from peers. A pre-adolescence teenager is more dependent on parents and other authority figures than an older teenager, who deliberately seeks autonomy and questions authority – in this developmental stage, apomediaries such as peers and peer-to-peer technology gain in attractiveness and relevance.

Interpreting the message and filtering for relevance It is interesting to note that when physicians express discontent about patients bringing stockpiles of “low quality” Internet printouts in the doctor’s office, the primary complain is not so much the low quality or credibility of that information in an objective sense, but its irrelevance for the specific patient. In one survey 73.8% of General Practitioners said about this information that it was “accurate”, and 65% of physicians even said the information was new to them, but many thought that the information often does not apply to that particular patients’ condition – in the same survey, only 44.7% said the patient correctly interpreted information [13]. Another survey found that those providers who thought much of the information was often irrelevant also had a more negative view on how the patient-physician relationship was affected [11].

Teenagers, people with chronic illnesses, educated people all share a common desire and capacity for autonomy, hence their credibility assessment heuristics are more on the right side of Table 1 above. On the other hand, younger kids, seniors, incapacitated people, people with acute diseases etc. are more likely to willingly submit themselves to a system in which they are dependent on intermediaries. The model proposed here theorizes that the desire for autonomy or perceived autonomy are predictors for differences in the interpretation of credibility cues (e.g., formal professional credentials versus “street cred”) and differences in information seeking and verification behaviours, and explains differences between pre-Internet and Internet generation, information seeking strategies of chronic versus acute patients, pre-teens versus teens, and illiterate versus literate consumers.

To what degree apomediaries actually succeed in the same way as intermediaries such as physicians or parents to filter information is largely an unexplored area. While there have been data on the “self-corrective” nature of digital media, to the effect that for example inaccurate statements on mailing-lists are corrected by peers [14], relatively little is known to what degree “apomediation” helps to filter relevant information and to contextualize information.

What is argued here is that variables like autonomy, self-efficacy and knowledge (thought to be predictors for embracing disintermediation) are presumably correlated with motivation (issue involvement, including knowledge and personal relevance of some topic) and ability (e.g., cognitive abilities, literacy, time), which, according to the elaboration likelihood model (ELM) of persuasion [16], affect message processing and, thus, influence message effects and credibility judgments. The ELM theorizes that higher issue involvement (motivation) and ability (which together affect what is here called autonomy) will lead to more effortful processing of a message (central route to persuasion), while lower motivation and ability will favour a peripheral route, where environmental characteristics of the message, like the perceived credibility of the source, presentation, or the attractiveness of the source are the primary credibility cues. An extension of this model has been proposed (though not empirically tested) by Fogg and Teng, who hypothesize that people with lower motivation and ability (those who are persuaded through a peripheral route) are more likely to adopt a binary evaluation strategy (credible or not credible), whereas people with higher motivation and ability employ a spectral evaluation strategy [17]. Thus, if we accept that in

Credibility implications The shift from intermediaries to what the author proposes to call apomediaries has implications for credibility constructs. Some credibility implications and research questions arising from the disintermediation/apomediation model are discussed in the following. Explaining group and situational differences Disintermediation usually means that people can, but must not use the intermediary, leading to groups of people who (or situations where people) continue to use intermediaries, trusting a more central authority, and others where people use apomediaries, relying on more decentralized, “networked” mechanisms to infer credibility (reputed, tabulated etc.). Reviewing audience factors in Web credibility research, Metzger [15] reminds us that “credibility is highly situational” and that “demographics, Internet use and reliance, and issue involvement have been found to influence perceptions of the credibility of Web-based information”. The disintermediation/apomediation model hypothesizes that factors such as self-efficacy and perceived or desired autonomy (freedom from authority and

164

G. Eysenbach et al. / From Intermediation to Disintermediation and Apomediation: New Models for Consumers

most cases autonomy is highly correlated with general motivation and ability, and if we accept that these variables also predict whether or not somebody chooses to emancipate themselves from a gatekeeper, and if we believe the predictions made by the ELM and its extensions, then the line of argument presented above becomes clear.

based on research would automatically have more credibility for consumers than anecdotes. In a focus group analysis with patients using evidence-based health information, Glenton and colleagues found that “participants described how they often made treatment decisions in a context of great pain and despair. Under such circumstances, they often had little energy to seek out written information and were sometimes too desperate to care what the research might have to say. Instead, they often gathered information about treatments through the personal anecdotes of friends and neighbors, and, in most cases, this experience-based information was considered to be more relevant than the evidence-based information” [19]. Not only is experiential information from apomediaries and peers more relevant for patients, it is hypothesized here that it often is also more or at least equally credible as information based on research.

Boosting credibility through disintermediation “Direct”, unmediated information is often is perceived as more credible because with “greater apparent mediation comes greater opportunity to impute motives and intentions of the communicator” [18], which is one of the most cited reason for why (live) television is perceived as more credible than newspapers [15]. In other words, disintermediation has the potential to increase the credibility of information. This is particularly true the health care field, where many consumers have a “healthy” (and sometimes not completely unjustified) mistrust in a system where doctors are paid per service, and where payers are under considerable cost-pressures leading to a perceived rationing of publicly available services. In addition, many consumers view the traditional health care system as being biased against alternative medicines [19], as health care professionals are incentivized to offer expensive therapies for which they are reimbursed more generously, as opposed to therapies which are “natural” but for which they cannot charge much. Such mistrust creates the desire to bypass the intermediary, and boosts trust in information which can be received without intermediaries.

Similarly. the notion of “source expertise” as being communicated primarily “through the comprehensiveness of a web site’s information, its professionalism, and its sponsor’s credentials” [15] is questionable in the health care context and perhaps in many other apomediated environments used by “autonomous” individuals. Here, “expertise” is not only expressed by credentials such as professional degrees and qualifications, but also first-hand experience. Experience-based credibility can be seen as one additional dimension of source credibility. Past research has identified that similarity in attitudes with the speaker as well as liking positively influences credibility perceptions. What might be added is that similarity of experiences (in the health care context: similarity of symptoms, diagnoses etc.) adds to credibility perceptions. In the context of youth, this is expressed by the term “street cred”, which has been defined as “commanding a level of respect in an urban environment due to experience in or knowledge of issues affecting those environments” [20].

While youths will rarely rationalize their mistrust against traditional intermediaries in a similar way, questioning and mistrusting traditional authorities is a natural part of adolescence, and information mediated through traditional authorities is often perceived as biased. Reinstating trust in the intermediary

Applying network theory to apomediaries: credibility hubs

It can also be hypothesized that transparency confirms the trustworthiness of intermediaries who “step aside” allowing and perhaps even facilitating direct access to information and transparency. In other words, once disintermediation has taken place, disintermediation has the potential to reinstate trust back to the intermediary if information provided through more direct channels proofs to confirm the information the intermediary used to provide. For example, health professionals who allow and actively encourage patients to access their own electronic health records help to reinstall patients’ trust in the medical system. If however the information now obtained from other channels is perceived to contradict information from the former intermediary, then the trust-relationship will be undermined. For example, youth accessing information on issues of sexuality through the Internet will loose trust in parents and teachers if this information contradicts the information these intermediaries have provided.

Apomediaries can be seen as highly complex networks of individuals and tools guiding consumers to credible information. While “networked” tools are often seen as a more equitable, democratic structure (as opposed to a system with intermediaries, who hold most of the power), network theory [21] teaches us that credibility networks are scalefree networks, where a rich-gets-richer phenomenon leads to the emergence of highly influential hubs, which in our context could be called credibility hubs. That is, not all apomediaries are equal, there are some apomediaries which have more influence than others. In the networked, apomediated model, some “nodes” (players or tools) become (or cease to be) credibility hubs in a more dynamic and fluid fashion than in the traditional model, where there is usually one intermediary whose credibility is influential and relatively stable. In a “networked credibility” model with apomediaries as nodes, former intermediaries do not disappear completely, they are just one of many apomediaries, with a seemingly equal chance of becoming a “credibility hub”, but in reality, they are more connected and have a better chance in ending up as a credibility hub,

Experiential credibility While traditional wisdom from credibility research suggests that perceived “accuracy” is a hallmark for message credibility [15], it would be a mistake to assume that “accuracy” means evidence-based information based on scientific studies and that evidence-based information

165

G. Eysenbach et al. / From Intermediation to Disintermediation and Apomediation: New Models for Consumers

For instance, a professional medical organization has a pre-existing social network which leads to other organizations linking to their website, leading that website to appear on top of Google, leading to more people linking to it, etc.

[5]

An interesting psychological phenomenon is that people attribute statements they believe to credible sources. For example, participants of an experiment who were exposed to a statement many times (and hence believed it) were more likely to attribute it to Consumer Reports (a credible source) than to the National Enquirer (a not so credible source) [22]. Such mechanisms may further increase the trustworthiness of credible sources, leading to a further rich-gets-richer phenomenon.

[6]

[7]

[8]

Conclusion

[9]

This paper discusses the idea that as a result of disintermediation, traditional intermediaries are replaced by what this author calls apomediaries, which are tools and peers standing by to guide consumers to trustworthy information, or adding credibility to information. It is hypothesized that in such an environment, tools, influential peers and opinion leaders are the primary conveyors of trust and credibility. In this environment, apomediary credibility may become equally or more important than source credibility or even message credibility. It is suggested to use tools of network analysis to study the dynamics of apomediaries in a networked digital world.

[10]

[11]

[12]

There are practical implications of the apomediation model for developers of digital media such as websites for consumers. Governments and other “authorities” – while certainly having credibility due to brand name recognition - do not typically do a very good job of creating “credible” Web sites – they always look and sound like government Web sites, and they lack the “edge” and the “street cred” that consumers and in particular youths are looking for to keep them engaged.

[13] [14]

[15]

[16]

Good Web sites allow consumers to share their voices and connect with others in a safe, positive, supportive, possibly moderated, online community. Consumers need and want to be able to be co-creators of content, not merely be an audience who is broadcasted to. Engaging and credible Web sites are about building community and communities are built upon personal and social needs. Acknowledgement

[17]

[18] [19]

Partly funded by The John D. And Catherine T. Macarthur Foundation, full chapter to be published in the MacArthur Series on Digital Media and Learning

[20]

References [1] Sittig DF. Personal health records on the internet: a snapshot of the pioneers at the end of the 20th Century. Int J Med Inf 2002;65:1-6. [2] Eysenbach G,.Jadad AR. Evidence-based patient choice and consumer health informatics in the Internet age. J Med Internet Res 2001;3:E19. [3] Eysenbach G,.Diepgen TL. Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information. BMJ 1998;317:1496-500. [4] Eysenbach G, Yihune G, Lampe K, Cross P, Brickley D. Quality management, certification and rating of health

[21] [22]

information on the Net with MedCERTAIN: using a medPICS/RDF/XML metadata structure for implementing eHealth ethics and creating trust globally. J Med Internet Res 2000;2:2E1. Eysenbach G,.Diepgen TL. Labeling and filtering of medical information on the Internet. Methods Inf Med 1999;38:80-8. Eastin MS. Credibility Assessments of Online Health Information: The Effects of Source Expertise and Knowledge of Content. J Comput Mediat Communic 2001;6. Anderson JG, Rainey MR, Eysenbach G. The impact of CyberHealthcare on the physician-patient relationship. J Med Syst. 2003;27:67-84. Brotherton JM, Clarke SJ, Quine S. Use of the Internet by oncology patients: its effect on the doctor-patient relationship. Med J Aust 2002;177:395. Gerber BS,.Eiser AR. The patient physician relationship in the Internet age: future prospects and the research agenda. J Med Internet Res 2001;3:E15. Hart A, Henwood F, Wyatt S. The role of the Internet in patient-practitioner relationships: findings from a qualitative research study. J Med Internet Res 2004;6:e36. Murray E, Lo B, Pollack L, Donelan K, Catania J, Lee K et al. The Impact of Health Information on the Internet on Health Care and the Physician-Patient Relationship: National U.S. Survey among 1.050 U.S. Physicians. J Med Internet Res 2003;5:e17. Murray E, Lo B, Pollack L, Donelan K, Catania J, White M et al. The impact of health information on the internet on the physician-patient relationship: patient perceptions. Arch.Intern Med 2003;163:1727-34. Wilson SM. Impact of the Internet on Primary Care Staff in Glasgow. J Med Internet Res 1999;1:e7. Esquivel A, Meric-Bernstam F, Bernstam EV. Accuracy and self correction of information received from an internet breast cancer list: content analysis. BMJ 2006;332:939-42. Metzger MJ, Flanagin A, Eyal K, Lemus DR, McCann RM. Credibility for the 21st Century: Integrating Perspectives on Source, Message, and Media Credibility in the Contemporary Media Environment. Communication Yearbook 2003;27:293-335. Petty RE, Cacioppo JT. Communication and persuasion: Central and peripheral routes to attitude change. New York: Springer, 1986 Fogg, B. J. and Tseng, H. (1999). The elements of computer credibility. http://captology.stanford.edu/pdf/p80-fogg.pdf [On-line]. Archived at: http://www.webcitation.org/5Lqk5JrrA Gunther AC. Attitude extremety and trust in media. Journalism Quarterly 1988;65:279-87. Glenton C, Nilsen E, Carlsen B. Lay perceptions of evidence-based information - a qualitative evaluation of a website for back pain sufferers. BMC Health Services Research 2006;6:34. Urban Dictionary (2006). Street Cred. http:// www.urbandictionary.com/define.php?term=street+cred [On-line]. Archived at: http://www.webcitation.org/ 5KT164TW0 Barabasi AL. Linked. Cambridge, MA: Perseus, 2002 Fragale AR,.Heath C. Evolving informational credentials: the (mis)attribution of believable facts to credible sources. Pers Soc Psychol Bull 2004;30:225-36.

Address for correspondence Gunther Eysenbach MD MPH, Centre for Global eHealth Innovation, 190 Elizabeth Street, Toronto M5G2C4, Canada

166

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Mobile Phone Based Remote Patient Monitoring System For Chronic Disease Management Mathieu Trudela, Joseph A Cafazzoac, Melinda Hamilla, Walter Igharasa, Kevin Tallevia, Peter Pictona, Jack Lama, Peter G Rossosab, Anthony C Eastyad, Alexander Loganbe a

Centre for Global eHealth Innovation, Toronto General Hospital, Canada b Faculty of Medicine, University of Toronto, Canada c Health Policy, Management and Evaluation, University of Toronto, Canada d Institute of Biomaterials and Biomedical Engineering, University of Toronto, Canada e Department of Medicine, Mount Sinai Hospital, Canada

factor for both renal and cardiovascular disease (CVD). [35] Despite being treatable, less than 15% of these patients have reached the widely accepted goal of 130/80 mmHg. [4, 6]

Abstract Rising concern over the poor state of chronic disease management led to the user-informed design and development of a home tele-monitoring system. Focus groups with patients and primary care providers guided the research team towards a design that would accommodate the workflow and concerns of the healthcare providers and the low use and comfort with technology found among the patient population. The system was trialed in a before-and-after pilot study of 34 patients with diabetes and hypertension. Findings demonstrate a significant improvement in systolic and diastolic blood pressure. An RCT beginning in 2007 is being conducted to confirm these findings. It is hypothesized that this user-centred approach, utilizing focus groups, iterative design and human factors methods of evaluation, will lead to the next-generation of home tele-monitoring applications that are more intuitive, less cumbersome, and ultimately bring about greater patient compliance and better physician management.

This research discusses the development and testing of a novel approach to tele-monitoring for improved chronic disease management. The primary objective of this study was to determine if a hypertension tele-monitoring support system, designed to enhance patients’ self-management and provide reliable information to clinicians for decisionmaking, will markedly improve blood pressure control in a primary care setting. A user-centric design approach was taken to the development of a mobile phone based remote patient monitoring system for the control of hypertension in type II diabetics. The system was developed (Phase I) using affordable devices and iterative gathering of user feedback to ensure high usability and low disruption to lifestyle and workflow of patients and providers.

Keywords:

The system concept would start with the regular home monitoring of blood pressure (BP) by the patient. BP readings would then be automatically transmitted by a mobile phone to the central data repository. A clinical rules engine would then check the data and the patient and family physician would be notified if the readings were outside of the desired range for a period of time. Given this alerting, the system would theoretically facilitate timelier follow-up of the patient’s care. Additionally, the availability of the home monitored data at patient visits may provide more information for assessing the patient’s condition.

blood pressure monitoring, self, blood glucose self-Monitoring, self care, mobile phone, computers, handheld

Introduction Chronic disease presents a growing challenge to the health and social care systems around the world. In 2005, chronic diseases accounted for 72% of the total global burden of disease in the population 30 years and older. [1] Although many chronic conditions are preventable or can be controlled with proper treatment, evidence is mounting that current chronic disease management practices are falling short. The prevalence of type-II diabetes is expected to rise to 4.4% of the worldwide population by 2030, for a total of approximately 366 million people globally. [2] Additionally, 40-50% of type II diabetics also have hypertension, which confers a 2 to 3-fold increase in the risk of morbidity and mortality for diabetics, and is a major risk

A pilot demonstration (Phase II) was undertaken to determine whether the system lowers the risk of dangerous complications for diabetic patients with high blood pressure. The study outcome was measured by assessing change in the measured blood pressure of study subjects.

167

M. Trudel et al. / A Mobile Phone Based Remote Patient Monitoring System For Chronic Disease Management

• •

Data gathering will require no user-intervention. User interface will consider visual and physical impairments. • User interface will consider limited or no previous knowledge of the operation of a mobile device. • Messages to the patient will be in lay terms. • Communication is secured via SSL connection to signed client-server applications. • Messaging to the patient will not be done via SMS due to usability problems. Messaging will be done via the client application, or by automated voice messages sent to the patient’s home phone. The strict adherence to these principles in order to simplify the use of the system for the end-user made the development and implementation more complex. The description of the realized system follows.

Phase I - Requirements Gathering and Design Methods Before any development of the system began, a consultative phase was initiated to investigate the problems with the existing management of hypertension in diabetics and gather patient/physician impressions of the system concept. From these sessions, conclusions were drawn to inform iterations of the system development. Two sets of focus groups were held. The patient set consisted of 24 type-II diabetics with hypertension, interviewed in small groups in one of four focus group sessions. The physician set consisted of 18 family physicians, also interviewed in small groups in one of four focus group sessions.

Results

System overview

Patient reaction to the concept was generally favourable, with patients showing a high regard and interest for the self-care aspects of the proposed system. Although patients were interested and intrigued by the use of a mobile device for the transmission of their BP data, they appeared to be more comfortable and familiar with the use of a mobile phone than a PDA. Surprisingly, there was little concern towards the security of the proposed electronic transmission of personal medical data. An additional significant finding from this group was that their level of Internet use was below expected. Although many had used the Internet in the past, their current use would be described as infrequent and simplistic, such as the occasional email. Many accessed the Internet through a proxy, such as a family member. These findings resulted in significant changes in the planned design of the system architecture.

Bluetooth-enabled medical devices (BP monitor, weight scale and glucometer) are used to transmit physiological data automatically to a mobile phone. A custom application running on the mobile phone acts as a personal medical diary of the patient’s test history. The application also relays data securely back to a central data repository where clinical rules are applied and alerts generated. These alerts are sent to the patient’s physician by fax (in the absence of any other electronic means of communication) and to the patient by automated text and phone messages. The system is designed to remind patients by text and voice message to take their home readings if they fail to adhere to the prescribed schedule. From the patient’s point of view, the measurement of their BP is as before, with no additional steps needed for the transmission of data. The original design goal of the system was to be as unobtrusive as possible, and to require no manual intervention on the part of the patient. If desired, the patient can simply plug the mobile phone into an electric outlet to maintain battery charge and leave it unattended as it automatically relays data back to the central repository. More technically inclined patients can use the diary and reporting features of the device to review all previous measurements taken in the home, summary reports, or view a graph of historical trends in their data. Since all device interaction is optional (beyond the standard measurement procedure), no technical acumen is required of patients in order to benefit from this system.

The physician findings were in contrast to that of the patient group. Physicians were skeptical that patients would adhere to regular measurements of BP. Their most serious concerns were around compensation and disruption of workflow. There was concern that they would be accountable to address alerts in a timely fashion. As well, they would not be compensated for their work related to addressing alerts and reports, since it was not tied to a face-to-face patient visit. Physicians insisted that the reporting of data be tied to a patient visit. Like the patient group, physician use of computers and the Internet in the office setting was minimal and could not be seen as a means to communicate patient reports or alerts.

Technical architecture The monitoring system consists of three main components; the patient components, the back end data repository and decision support system, and the alerting and physician-

Design principles Based on the findings of the study groups, the following design principles were formed: • • •

Personal computers are not to be used for information input, retrieval, or review by patient or physician. Phone and fax are the only means available to disseminate alerts and reports to physicians. The mobile phone or PDA is the information hub for the patient with the following considerations:

168

M. Trudel et al. / A Mobile Phone Based Remote Patient Monitoring System For Chronic Disease Management

Figure 1 - Mobile Phone Based Remote Patient Monitoring System reporting component. The data flow between these compoThe back end of the system comprises a web based server nents is shown in Figure 1. application, which gathers results from client devices, a database to store the results, and a set of clinical rules, The patient component of the system comprises the mediwhich are applied to the data as it arrives. These rules trigcal devices themselves and the mobile phone used to relay ger events handled by the alerting and reporting results to the server. There is also an interface for patients component of the system. to review previous readings on the device itself, in tabular, graph, and summary formats (see Figure 2). Progress mesThe alerting and reporting system provides secure web, sages are sent from the central server onto the mobile email, voice, and fax gateways into the data stored for each device after every reading, allowing for reminders and patient. Certain conditions (such as non-adherence to other coaching messages to be delivered automatically to scheduled measurements) may trigger alerts to be sent to the patient. the patient via an automated telephone message to the patient’s home phone number. The thresholds for these alerts can be set by the physician, on a patient-by-patient basis. Physicians can also request a fax (via an interactive telephone system) of the patient’s readings to date displaying averaged readings over the past 30 days, as well as a chart of the patient’s historical data measurements. These are also available online for providers who prefer to use the web for reviewing patient information. As substantial portions of the monitoring system operate in an uncontrolled environment, measures to ensure the confidentiality of patient data were a paramount concern in the design of the system. The initial transmission of data from the measurement device to the mobile phone is done via Bluetooth, providing strong encryption of patient data during transmission, and also ensuring the authenticity of received data. Data stored on the mobile phone has no readily individually identifiable information stored with it,

Figure 2 - BP Client Application

169

M. Trudel et al. / A Mobile Phone Based Remote Patient Monitoring System For Chronic Disease Management

as results are only tied to a specific patient at the server. This mapping associates patients to the unique numeric Bluetooth ID of the patient’s medical device, and is known only to the system server. Nonetheless, in the event a device is lost or misplaced, a ‘kill’ message can be sent to the device from the server, causing all stored data to be permanently deleted from the device. Data is sent from the mobile device to the server via a secured SSL web connection, providing an industry standard level of confidentiality in addition to the security offered by the underlying cellular network.

Discussion The first outcome measure looked at pre and post trial ambulatory and 2-week home blood pressure averages. In both cases the results were significant. Ambulatory monitoring is currently the gold standard for measuring blood pressure, and the study patients experienced an average improvement of 10 mmHg systolic and 5 mmHg diastolic. These results show great promise as the health benefits of lowering BP in a diabetic population are well documented. [7-9] However, this was only a pilot study with 30 patients, a full clinical trial is required in order to confirm the generalizability of these results. (see “Current and Future work”)

Use of the interactive telephone system to request a fax report requires the physician or patient to input a valid numeric user id and password. The information is validated and authenticated against the information in the database before access is granted to the patient record. Fax-based transmission of patient results is only permitted to pre-programmed physician office fax numbers.

The second outcome measure was patient adherence. The success in adherence experienced in the pilot may be partially attributable to the automated phone calls to the patient’s home, reminding them to continue to monitor. These phone messages were generated by the system when it was detected that the patient was not regularly monitoring their BP. The upcoming clinical trial will investigate the affect of these alerts on adherence. A comparison of the intervention group with the control arm of the study will demonstrate if adherence drops off without the adherence reminders.

Phase II - Pilot study Methods A thirty-patient trial of diabetic patients with hypertension in the Toronto area began in December 2005 and concluded in November 2006. Patients monitored their blood pressure at home for four months using the aforementioned system. Patients had mandatory visits with their physician at baseline, two and four months. (Other visits may have taken place due to patient health factors and/or the alerting system requesting that the patient go see the doctor because of elevated hyper or hypo-tension.)

Finally, the study demonstrated the technical feasibility and readiness of available and affordable technology to be applied for the purpose of home tele-monitoring (outcome measure three). Patients with the technology encountered only minor problems. Certain features of the mobile phone proved to be a challenge for some users with low dexterity or visual impairments. The small joystick for navigating on-screen menus, and the narrow power button were features that caused frustration. Additionally some patients had trouble viewing the screen when the backlight was off. The fax-back system proved to be a simple and efficacious method of providing data to the family physician, without changing their workflows significantly.

Main outcome measures of this pilot study are changes in BP compared to pre-trial baseline measurements, patient adherence, and technical and clinical feasibility of the system.

Results

Current and future studies

Shown in Table 1 are the results. Significant results were found in all cases.

An additional thirty-patient trial in the northern Ontario community of Chapleau began in July 2006 and adds glucometry in addition to home blood pressure measurement. The care model also differs in Chapleau. A nurse practitioner is the main primary care provider for diabetics in the area. Based in the local hospital, staff members have easy access to computers in the clinic, and prefer a workflow model where records and alerts are retrieved through a PC, rather than fax. A web-based application to accommodates this workflow, displays all data, reports and alerts, as well as allows patient parameters to be adjusted, such as measuring frequency, BP and glucose goals, and alert thresholds.

Table 1- Pre and post trial, ambulatory (24hr) and 2-week home measurement averages Systolic24hr Diastolic24hr Systolic2wk Diastolic2wk

Pre (SD) 143.3 (14.05) 81.4 (10.74) 140.8 (16.50) 81.7 (11.70)

Post (SD) 133.3 (12.56) 76.2 (10.70) 131.8 (13.88) 78.3 (10.40)

N 27

P 0.001

27

0.000

32

0.001

32

0.006

A randomized controlled trial (RCT) will be conducted in the greater Toronto area beginning in 2007. The control group will perform regular home monitoring of blood pressure, and the intervention group will utilize the telemonitoring system.

Of the 34 patients that were enrolled in the study, 32 remained at the end of the four-month trial. One physician withdrew their patient, and another patient withdrew due to a confounding health problem that affected their ability to home monitor.

170

M. Trudel et al. / A Mobile Phone Based Remote Patient Monitoring System For Chronic Disease Management

Conclusions The intent of a user-centric approach was to lead to the development of a system that was more intuitive, less cumbersome, and would ultimately bring about greater patient compliance and better physician management. The intent is to demonstrate a technology that is applicable to a wide variety of chronic diseases and conditions such as congestive heart failure and asthma and builds on the demand for new and more effective ways to improve chronic disease management. A secondary benefit of this study is the validation of the use of commodity hardware, which promises to provide significant cost savings in the delivery of the management tool.

[4]

[5]

[6]

[7]

Results from the pilot study indicate the potential efficacy of the system at reducing hypertension, which we will attempt to confirm in an RCT.

[8]

Acknowledgments This research was funded through the Primary Care Health Transition Fund of the Ministry of Health and Long Term Care of Ontario. The clinical trial in Chapleau, Ontario is funded through Project Chapleau, a joint initiative of Bell Canada and Nortel to research the impact of advanced technology on rural and dispersed communities. The RCT is funded by the Heart and Stroke Foundation of Canada.

[9]

National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: the JNC 7 report. Jama. 2003;289(19):2560-72. Singer GM, Izhar M, Black HR. Goal-oriented hypertension management: translating clinical trials to practice. Hypertension. 2002;40(4):464-9. Sowers JR, Epstein M, Frohlich ED. Diabetes, hypertension, and cardiovascular disease: an update. Hypertension. 2001;37(4):1053-9. Singer GM, Izhar M, Black HR. Guidelines for hypertension: are quality-assurance measures on target? Hypertension. 2004;43(2):198-202. Efficacy of atenolol and captopril in reducing risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 39. UK Prospective Diabetes Study Group. Bmj. 1998;317(7160):713-20. Effects of ramipril on cardiovascular and microvascular outcomes in people with diabetes mellitus: results of the HOPE study and MICRO-HOPE substudy. Heart Outcomes Prevention Evaluation Study Investigators. Lancet. 2000;355(9200):253-9. Estacio RO, Jeffers BW, Hiatt WR, Biggerstaff SL, Gifford N, Schrier RW. The effect of nisoldipine as compared with enalapril on cardiovascular outcomes in patients with noninsulin-dependent diabetes and hypertension. N Engl J Med. 1998;338(10):645-52.

Address for correspondence Joseph A. Cafazzo, Centre for Global eHealth Innovation, Toronto General Hospital, University Health Network, 190 Elizabeth Street, R. Fraser Elliott Building 4S400, Toronto, Ontario, Canada M5G 2C4, [email protected]

References [1] Strong K, Mathers C, Leeder S, Beaglehole R. Preventing chronic diseases: how many lives can we save? Lancet. 2005;366(9496):1578-82. [2] Wild S, Roglic G, Green A, Sicree R, King H. Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. Diabetes Care. 2004;27(5):1047-53. [3] Chobanian AV, Bakris GL, Black HR, Cushman WC, Green LA, Izzo JL, Jr., Jones DW, Materson BJ, Oparil S, Wright JT, Jr., Roccella EJ. The Seventh Report of the Joint

171

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

How to Enhance Integrated Care towards the Personal Health Paradigm? Bernd G.M.E. Blobela, Peter Pharowa, Thomas Norgallb a

eHealth Competence Center, University of Regensburg Medical Center, Germany b Fraunhofer-Institute for Integrated Circuits, Erlangen, Germany

so moving the focus from healthcare to health) this process-controlled strategy is no longer sufficient. Health, nowadays provided by organizations such as hospitals, primary care offices, policlinics or medical centers as well as, has to move closer to the citizens’ environment.

Abstract For improving quality and efficiency of health delivery under the well-known burdens, the health service paradigm has to change from organization-centered over process-controlled to personal health. The growing complexity of highly distributed and fully integrated healthcare settings can only be managed through an advanced architectural approach, which has to include all dimensions of personal health. Here, ICT, medicine, biomedical engineering, bioinformatics and genomics, legal and administrative aspects, terminology and ontology have to be mentioned. The Generic Component Model allows for different domains’ concept representation and aggregation. Framework, requirements, methodology and process design possibilities for such a future-proof and meanwhile practically demonstrated approach are discussed in detail. The deployment of the Generic Component Model and the concept representation to biomedical engineering aspects of eHealth are touched upon as essential issues.

Observing the citizens’ health status, context and conditions for providing person-centered (personalized) and dedicated health services implies the need for a new health paradigm: personal care, which completely integrates all of the principals involved in the care process. According to the definition of the Object Management Group principals are any actors in the domain in question such as persons, organizations, systems, devices, applications, components or even single objects. This does not mean that there will no loner be acute care and ambulant service, but that such services will be tailored to relevant personal care needs,.

Materials and methods

Keywords:

To realize patient care at any location in an individualized way, three technological paradigms have to be managed: mobile computing, pervasive computing and autonomous computing (Figure 1). Mobile computing enables the permanent accessibility of the principals involved, providing, for example, teleconsultation services. Pervasive computing allows for location-independent service provision, established as telemedicine services. For providing personalized care, services have to be flexible and cannot be rigidly predefined. Such adaptive

Personal health; system architecture; Generic Component Model; biomedical devices

Introduction Healthcare systems in industrialized countries, and increasingly those in countries in transition, are faced with the challenge of ensuring efficient and high quality care independently of time, location and local resources, utilizing advanced knowledge and technologies. This challenge must be realized despite demographic developments, the growth of multi-morbidity, demands for health services and expenditures for diagnostic and therapeutic procedures, and decreasing contributions to health insurance funds. To meet this challenge, the systems have been changing from an organization-centered towards a process-controlled care paradigm, which is also called shared care, managed care or disease management. This development is combined with extended cross-organizational communication and cooperation between all of the healthcare establishments involved in patient care. This process has to be supported by deploying advanced information and communication technologies (ICT) in health, connecting primary and secondary care. Regarding the need for prevention and the integration of social care in an aging society (addressing citizens before becoming patients, and

Pervasive Computing Location -independent service provision Telematics, Telemedicine

Ubiquitous Computing

Autonomic Computing

Mobile Computing

Self -organisation Health information systems

Accessability Tele -consultation

Figure 1 – Computing paradigms deployed in the personal care context (after Kirn and Müller (2005), changed [1])

172

B. G. M. E. Blobel et al. / How to Enhance Integrated Care towards the Personal Health Paradigm?

health information system design, towards a self-organizing environment, draws on current challenges in the research and development for autonomous computing.

expressed by the Information View and the functional aggregation of algorithms and services is expressed by the Computational View. These are described through platform independent models of the system expressing the system’s logical content. Platform-specific implementation details are described by the Engineering View, and the Technology View represents technology (or implementation) aspects. The system’s architecture (i.e. the system’s components, their functions and relationships) is characterized through the components’ concepts and their aggregations. The representation of concepts and association rules is provided by constraint models, which are derived from reference models.

Another aspect which is characteristic for personal health (pHealth) concerns the distance between the physical and the informational world. In the traditional ICT environment, this gap is mediated through human users. Introducing advanced technologies, this gap is getting closer to the real integration of the health subject (patient) in the health system, and even becoming a part of the information system environment (Figure 2).

Domain n Domain 2 Domain 1 Inform atics w orld ( “Bits ”) • Intra - and extra -organisational inform ation s ystem s (e.g. ER P) • Local, regional and global com m unication netw ork s (e.g. Internet)

G ap betw een physical and inform atics world

Business Concepts

Component View

Relations Network

D ata entry costs

Basic Services/Functions

Physical w orld ( “Atom s ”) • H um an beings • M eans of production • Produc ts

Hum an intervention required

Basic Concepts

N o hum an intervention required

Figure 2 - Closing the gap between physical and informational world (after Kirn and Müller (2005), changed [1])

Figure 3 - The Generic Component Model [4] The resulting pHealth information system is characterized by openness, flexibility, scalability, portability, user friendliness and user acceptance, service orientation, distribution at Internet level, being based on standards, semantic interoperability, lawfulness and trustworthiness. Organizational, contextual, rule-related or other constraining aspects of the system and its components are expressed by binding policies to the components and by ruling the component aggregation by policies .

The process to be supported has to be properly described, abstracting from reality by using appropriate grammars for modeling it. A model is a partial representation of reality. It is restricted to attributes the modeler is interested in. Defining the pragmatic aspect of a model, the interest depends upon the intended audience and the reason and the purpose for modeling the reality. The resulting model is used for a certain purpose and time as a proxy for reality. Therefore, the model (which is the result of an interpretation) must be interpreted itself.

Following the Generic Component Model approach, the pHealth information system architecture needs to combine the following paradigms: distribution; component-orientation; separation of platform-independent and platformspecific modeling, i.e. separation of logical and technological views; specification of reference and domain models at meta-level; interoperability at service level (concepts, contexts, knowledge); enterprise view driven process-controlled design; multi-tier architecture; appropriate multimedia GUIs; common terminology and ontology; unified design, development and deployment process; trustworthiness provided by appropriate security and privacy services, all as an integrated part of the design.

The simplification of systems through formal models can be provided in three dimensions according to the Generic Component Model [2] (Figure 3). The first level of simplification concerns the restriction to the domain of interest. Examples for such domains are the medical domain, administrative domain, technical domain, legal domain, etc. Within this domain, the system considered can be decomposed or composed for analyzing or designing it. This results in different levels of granularity or complexity, respectively, using specialization or generalization relationships. In the Generic Component Model, the following granularity levels have been derived: business concepts, relations network, basic services/functions and basic concepts. The third dimension of a generic system architecture touches different aspects of the system according to the ISO Reference Model – Open Distributed Processing [3]. Here, the business process is expressed by the Enterprise View, the informational expression of this process is

Results The architectural approach of the Generic Component Model has to be applied for pHealth information systems covering health telematics aspects, telemedicine concepts, biomedical engineering solutions and bioinformatics problems in a harmonized way.

173

B. G. M. E. Blobel et al. / How to Enhance Integrated Care towards the Personal Health Paradigm?

The knowledge representation is performed using metalanguages. For ensuring semantic interoperability, reference terminologies and ontologies have to be defined, using the aforementioned methods for knowledge representation in the Generic Component Model’s context.

adaptive, self-organizing systems. In the IHE Patient Care Device Technical Framework, the eHealth technology domain is considered to have defined structural and functional requirements for semantic interoperability. Starting with an ISO OSI based layered model, the interoperability requirements have been defined by ENV 13735 Health informatics – Interoperability of patient connected medical devices. This specification, combined with others, has been moved up to ISO/IEEE 11073 “Health informatics – Point-of-care medical device communication”, e.g. defining the underlying medical package model (Figure 4). The containment tree consisting of Medical Device System, Virtual Medical Device, Channel, Metric and reflecting the granularity levels, as well as its combination with the process management aspect, are consistently in line with the Generic Component Model.

In this way, different systems deploying different modalities, belonging to different domains and different business areas, using different domain languages, can be harmonized and bound to policies. From this modeling, the specification of invocation calls and the development of XML messages can be derived as is shown in [5]. The connection of biomedical devices for patient monitoring and care is provided using the CEN ISO/IEEE 11073 standards set (which is based on ENV 13734/13735 “VITAL” and IEEE 1073-x) [6] as well as CLSI (formerly NCCLS) POCT-1A [7]. Thereby, biomedical devices can be aggregated and replaced like any other component. This is especially true for future mobile, modular, personal systems for individually caring patients. Such systems can be deployed in clinical settings and in homecare. In that way, a patient’s transfer between both settings can be facilitated. Typical system components, at different levels of development and accentuation, are:

VMO

1 0..*

M edical Device System

Session Test

1

0..1

0..* Virtual M edical Device 0..1 1

0..* Channel

1

0..1 0..*

0..*

0..*

M etric

•

Highly integrated sensor and human-machine interface components at the body or wearable in clothing (e.g. intelligent wireless sensors; wearable devices; PDAs); • Components and infrastructure enabling communication between the aforementioned systems and components on the one hand and stationary systems and services on the other (e.g. Body Area Network; mobile phone/mobile network; wireless LAN; workstation with gateway function in patient’s home; appropriate middleware); • Distributed functions for sensor signal processing, state recognition and state monitoring up to person- and situation-related information and interventions offered (e.g. qualified management of emergency calls; cumulated multi-parameter records, processing and presentation using PDA or workstation); • Information and expert systems for recognizing and managing emergency cases, for informing the patient as well as decision support for healthcare professionals (e.g. localization of principals; access to reference data; person-specific support for interpretation of data, secure access to patient’s personal information/Electronic Health Record). The system functions must realize proper escalation strategies, while the system components next to the patient must be relatively autonomous to minimize communication and maintenance effort, energy consumption and, on the other hand, communicating autonomously with external stationary system components in certain cases (exceeding of thresholds, recognition of exceptional situations, emergency cases, alerts, but also for routine communication).

0..*

0..*

0..* 1..* Numeric

Sample Array

Tim e-SA

Distribution-SA

Enum eration

Com plex Metric

0..*

PM -Store

1 0..* PM-Segm ent

RealTim e-SA

Figure 4- ISO/IEEE 11073 Medical package model Some of the standards specifying the integration of medical devices in health information systems for enabling personal health systems, are presented in some detail below. Standards for medical device interoperability Unlike other clinical communication standards, the CEN ISO/IEEE 11073 family of standards provides real-time “plug-and-play” interoperability to facilitate the efficient adding and swapping of acute and continuing care devices, such as patient monitors, ventilators, infusion pumps, ECG devices, etc in critical environment settings. To fulfill real-time requirements, a highly efficient “Medical Device Encoding” scheme is used. Conversion to alternative XML encoding reduces real-time capabilities, but enables use of XML-specific concepts and tools [8] promising dramatic cuts of development time and effort for interoperability component implementation. “Plugand-play” practically means that all a clinician has to do is making the connection – the systems automatically detect, configure and communicate without any subsequent human interaction, maintaining both functional and semantic interoperability. Health informatics and clinical communication standards are often generally interpreted as relating to ISO/OSI Level 7 based on the assumption of a generally available LAN-infrastructure to build upon. A prominent example for this approach is the HL7 communication standard

Advancing the modeling approach towards the Generic Component Model methodology, biomedical device components can be designed to provide the same structure and behavior as all other system components in the sense of

174

B. G. M. E. Blobel et al. / How to Enhance Integrated Care towards the Personal Health Paradigm?

(appropriately named with a seven) [9]. For medical devices, interoperability explicitly implies all ISO/OSI Levels. While 11073 defines/modifies standards in ISO/ OSI levels 7 – 5, it chiefly references other standards (such as 802.x, IrDA, Bluetooth, etc.) in levels 1 – 4.

enable multi-parameter monitoring, micro-sensors can also be integrated into BSU units. The German IMEX project [11] aimed at communication between micro-systems and with external device systems, analyzing possible interfaces and their communication requirements. A Micro System Data format (MSD) was defined to enable the use of standard health telematics coding schemes for semantic core elements on the microsystem-level, minimizing the processing overhead for preparation of micro-system-generated data for external standard-based communication [12]. Thus the “semantic interoperability chain” is extended to the micro-system level. Figure 5 shows smart micro-systems as the other end of the healthcare interoperability chain.

For systems in homecare, and even more for mobile systems providing personal health services in dynamic environments, equivalent considerations apply. In order to enable functional interoperability using different (wired, IR and RF wireless) network technologies, CEN ISO/ IEEE 11073 provides standards for internetworking in the 11073.5 branch of the 11073 standards family. In that context, the 11073 specification for “Agent Device”, e.g. an infusion pump, pulse oximeter, or ventilator, and the corresponding 11073 specification for “Manager System” – a patient monitor or device manager shall be mentioned. Both are situated in different sub-networks using different network technologies. Other typical applications are wired-to-wireless transport gateways or LAN/IR access points. Based on CEN preparatory work [10], CEN ISO/ IEEE 11073-60101 defines an 11073/HL7 “Observation Reporting interface” (ORI) enabling device-to-HIS-level interoperability. It is the first standard in the 11073.6 “Application Gateway” branch of the 11073 standards family which is intended to provide interoperability among different application protocols. The 11073 coding scheme has been a registered HL7 Coding Scheme since 2003, permitting its use in HL7 messages.

Figure 5 - Smart micro-systems as the other end of the healthcare interoperability chain

Body area networks

Specification and implementation process

The Body Area Network (BAN) concept specifies wireless communication between several miniaturized, intelligent Body Sensor (or actuator) Units (BSU) and a single Body Central Unit (BCU) worn on the human body. It is characterized by a maximum range typical for human body dimensions, e.g. 2 meters. The BCU concentrates the BSU-originated data streams, performs intermediate storage and processing as well as communication to the outside world using standard wireless technology like DECT, WLAN or Bluetooth. The Network Access Unit (NAU) can be implemented as a medical gateway hosting an embedded web-server.

To select the right components, aggregation rules and policies (in the broadest sense of the term policy: legal, social, organizational, ethical, functional and technical) in order to guarantee semantic interoperability, agreed constraints or profiles can be introduced similar to OMG’s approach and similar to the process IHE is performing with its integration profiles. For describing a unified process, the Rational Unified Process [13] or the HL7 Development Framework can be used. For domain-specific specializations, existing models such as HL7 Domain Message Information Models (DMIMs), HL7 Refined Message Information Models (RMIMs), HL7 Common Message Element Types (CMETs) [14], and also openEHR Archetypes [15] can be reused as demonstrated in [16]. Also other knowledge representation means can be deployed [17]. The future advanced eHealth architecture for individualized healthcare with regioncrossing or European characteristics has been defined in the eHealth Action Plan of the European Commission and the EU Member States. The Electronic Health Record (EHR) is the core application of any eHealth platform; different countries are approaching this differently. The variants cover a medication file as starting point for eHealth in The Netherlands and the UK, the Sharable Record approach in Finland, right up to a comprehensive record architecture modeled and implemented within the national programme.

Both NAU and BCU provide standard interfaces, particularly implementing ISO/IEEE 11073 “Agent” functionality. From a communication perspective, a BAN can thus be regarded equivalent to an 11073-compliant modular medical device, implying semantic interoperability between BAN and remote professional or clinical systems. IMEX – a micro-system perspective for interoperability As the development of micro-sensors and micro-systems, particularly for homecare and pHealth-related applications, is progressing, the acquisition of multiple biosignals (for instance blood pressure (BP), ECG, respiration, urine flow) can be performed by means of miniaturized patient-worn equipment. Utilizing the BAN concept for collection and communication of data to

175

B. G. M. E. Blobel et al. / How to Enhance Integrated Care towards the Personal Health Paradigm?

Discussion

References

Interoperability implies a number of different concepts, e.g. functional interoperability and internetworking, semantic interoperability and application gateways. Health information integration (eHealth) has established a demand for interoperability between clinical and healthcare-related stakeholders, systems and processes or workflows. Domain-specific communication and interoperability standards are well established, but have to be supplemented for trans-domain use. Interoperability concepts for medical devices and for personal or mobile systems need to involve all 7 ISO/OSI reference model layers, more properly advanced to the Generic Component Model, including terminology/coding aspects.

[1] Kirn S, Müller G. Ministudie zu den F&E-Perspektiven auf dem Gebiet der Gesundheitstelematik. Hohenheim, University of Hohenheim 2005. [2] Blobel B. Analysis, Design and Implementation of Secure and Interoperable Distributed Health Information Systems. Series “Studies in Health Technology and Informatics” Vol. 89. Amsterdam: IOS Press, 2002. [3] ISO/IEC 10746 “Information technology – Open Distributed Processing, Part 2 – Reference Model”. [4] Blobel B. Advanced EHR architectures – promises or reality. Methods Inf Med 2006; 45: pp 95-101. [5] Blobel B, Stassinopoulos G, Pharow P. Model-Based Design and Implementation of Secure, Interoperable EHR Systems. In: Musen MA, Friedman CP, Teich JM, eds. AMIA 2003 Symposium “Biomedical and Health Informatics: From Foundations to Applications”, American Medical Informatics Association 2003 Proceedings, Bethesda, pp. 96-100.. [6] http://www.ieee1073.org [7] http://www.nccls.org or http://www.clsi.org [8] http://www.w3.org [9] http://www.hl7.org [10] CEN SSS-HIDE (2001). Health Informatics-Strategies for harmonisation and integration of device level and enterprise-wide methodologies for communication as applied to HL7, LOINC and ENV. [11] Becks T, Dehm J. IMEX – A New Knowledge Platform for Microsystems in Medicine. http://www.vde-mikromedizin.de [12] http://www.iec.ch/ [13] IBM. Rational Unified Process. www.ibm.com/software/awdtools/rup/ [14] Blobel B, Engel K, Pharow P (2006) Semantic Interoperability – HL7 Version 3 Compared to Advanced Architecture Standards. Methods Inf Med 2006; 45: pp 343-353. [15] Beale T, Heard S. Archetype Definition Language. openEHR Foundation, Ocean Informatics Australia 2004. [16] Lopez DM, Blobel B. Connecting Public Health and Clinical Information Systems by Using a Standardized Methodology. (in this volume) [17] Blobel B. Concept Representation in Health Informatics for Enabling Intelligent Architectures. In: Hasman A, Haux R, van der Lei J, De Clercq E, Roger-France F, eds. Ubiquity: Technology for Better Health in Aging Societies,. Series Studies in Health Technology and Informatics, Vol. 124. Amsterdam: IOS Press, 2006; pp. 285-291.

The advanced concept of pHealth extends eHealth by the inclusion of smart sensors, body-worn mobile systems and situation-specific activation of applications and human health professionals, thus providing personalized ubiquitous health services. Body Area Networks and microsystems are building blocks of future personalized health telematics infrastructures, and extend existing interoperability concepts. Another important eHealth pillar is the field of bioinformatics and genomics. As personal health requires personalized process models for optimal care, the underlying diagnosis and therapy has also to be individualized. This can be achieved by developing and deploying advanced bioinformatics and genomics as mentioned earlier. The transfer to pHealth information systems with processcontrolled, service-oriented, context-sensitive, semantically-interoperable information and communication architectures requires open, highly flexible individually tailored application systems for the cared for and the caring parties. Such applications cannot be pre-manufactured any more, but must be dynamically created and adapted to the actual requirements and needs. In that way, besides the well-established technology paradigms of Mobile Computing for realizing accessibility (e.g. teleconsultation) and Pervasive Computing for realizing independency of location when providing services (e.g. telemedicine), the paradigm of Autonomous Computing for realizing selforganizing systems can be introduced. The combination of the aforementioned technology paradigms leads to Ubiquitous Computing, which is bound to other paradigms and trends such as health grids. Personal health also requires an adequate legal framework and the new orientation of traditional organizational patterns.

Address for correspondence Bernd Blobel, PhD, Associate Professor University of Regensburg Medical Center, eHealth Competence Center Franz-Josef-Strauss-Allee 11 93053 Regensburg, Germany Phone: +49-941-944 6769 Email: [email protected]

Acknowledgement The authors are indebted to the colleagues from HL7, ISO TC 215 and CEN TC 251 for kind support.

176

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Developing Information Technology Attitude Scales for Health (ITASH) Rod Warda, Katherine Pollarda, Dr Margaret Glogowskaa, Dr Pam Moulea a

Faculty of Health and Social Care, University of the West of England, Bristol. UK

explored by Lee [8] when examining nurses’ attitudes in Taiwan. All of the studies in this area show that attitude is an important predictor of the use of IT. However, none of the tools that have been employed in the UK have been validated, and those from elsewhere are not necessarily appropriate in the UK context.

Abstract This paper reports on the ongoing development and testing of a set of scales designed to elicit the attitudes of United Kingdom healthcare staff towards Information Technology. The scales were developed in the light of existing literature. Testing included a test-retest completion by over 100 staff from various disciplines in three National Health Service Trusts and a comparison with an existing scale. Exploratory principal components factor analysis identified three components, with a loading of > 3.1. This pattern of item grouping could be understood and interpreted as representing ‘efficiency of care’, ‘education, training and development’ and ‘control’. The three scales comprise the Information Technology Attitude Scales for Health (ITASH). The results suggest that the developed scales together comprise a measure that can be used to establish staff attitudes towards IT use in the healthcare sector. Those undertaking research in this area might consider employing the scales to measure the factors that influence attitudes towards IT. Additionally, employers might usefully draw on the scales as they aim to support staff in IT use and embed IT systems within the healthcare workplace.

Stronge and Brodt [9] over 20 years ago, studied this area in the United States of America (USA) with their Nurses’ Attitudes Towards Computers (NATC) questionnaire. Others have continued this work [10-12]; however none of the resulting measures were considered appropriate, valid and reliable for use in the NHS, because of the different healthcare systems and/or the use of American phrases. The primary aim of this study was to develop and test a scale which can be used to examine the attitudes of NHS staff towards Information Technology (IT), both in general and specifically in the workplace. The main objectives of the research were to: • • •

Keywords:

Review existing IT attitude measurement tools Develop a health care specific attitude measure Test reliability, uni-dimensionality and validity

Methods

attitude, computers, health personnel, United Kingdom

Development of scales

Introduction

The research team collaborated in the design of the ITASH scales. Statements on issues identified from the literature [13-16] and practice were generated. Specific items were included identifying factors known to influence staff IT attitudes. The scales were incorporated into a questionnaire which also asked for demographic data, which included age group, gender, professional specialty, qualifications and current level of computer use at home and work.

The introduction of Information Technology (IT) in the National Health Service (NHS) in the United Kingdom (UK) has a long history, although the use made of IT varies widely between different geographical and practice areas [1]. Since the publication of the Information for Health Strategy [2], and more recently the ‘National Programme for IT’ (NPfIT), the use of computer applications within the NHS has been increasing. One of the barriers to successful implementation and use of IT in the NHS is believed to be the attitude of staff to IT use, however there is limited research on this topic and no validated instruments for measuring NHS staff attitude to IT could be found in the literature.

The topic areas in the attitude scales included: • • • • • • •

There are several models and tools used elsewhere in the world and in other sectors. These include the Technology Acceptance Model (TAM) [3, 4] as extended by Dixon into the Information Technology Adoption Model (ITAM) [5, 6] and the Rogers’ Innovation-Diffusion Model [7] as

177

Quality of care/ Communication Benefit to the organisation Confidence/anxiety/interest Dependability issues Training issues Professional role Feelings about change/control

R. Ward et al. / Developing Information Technology Attitude Scales for Health (ITASH)

Ten to 15 questions were written for each topic and checked for duplication and relevance. The questions were re-ordered to reduce bias and question fatigue [17]. A pool of 79 items was assembled about IT use in health care, respondents were invited to agree/disagree, using a 4 point Likert scale, as it was assumed that respondents would have an opinion. Following completion the statement values were reversed where necessary. The item pool was then used in two different ways - with NHS staff in a testretest format to investigate reliability; and secondly with Faculty staff exploring validity.

Information Technology Attitude Scales for Health (ITASH). The cumulative value of the responses to the statements in each scale becomes the respondent’s score for that scale. For the first and second scales, the minimum score is 17, while the maximum score is 68. Scores from 17 to 34.99, 35 to 50.99 and 51 to 68 indicate negative, neutral and positive attitudes towards the use of computers in health care, respectively. For the third scale, the minimum score is 14, while the maximum score is 60. Scores from 14 to 28.99, 29 to 41.99 and 42 to 56 indicate negative, neutral and positive attitudes towards the use of computers in health care, respectively.

Ethical approval was obtained from appropriate Local Research Ethics Committees. Paper questionnaires were delivered, as online mechanisms would have biased the results by excluding those without Internet access and those with negative attitudes to IT.

Reliability of the scales To assess the stability of the scales, a test-retest administration of the scales was sent to 145 respondents, of whom 121 completed the scales a second time, over a period of 2-3 weeks. Pearson’s correlation coefficients for the scores on the three scales were found to be r=.88 (P < 0.01), r=.75 (P < 0.01) and r=.83 (P < 0.01) respectively. The internal consistency of each scale was assessed by means of Cronbach’s alpha coefficients. The coefficients obtained were =0.88 (n = 150), =0.70 (n = 150) and =0.77 (n = 150), indicating a satisfactory degree of internal consistency [17]. These results indicate that the ITASH scales are reliable.

Between November 2005 and January 2006, 816 questionnaires were distributed via managers/matrons/professional leads. Trust 1 (Mental Health) – 470, Trust 2 (Acute Hospital) – 200, Trust 3 (Primary Care Trust) – 146. Follow up questionnaires were sent to all respondents who included their contact details between December 2005 and March 2006. To test for concurrent validity, copies of the questionnaire and the Stronge and Brodt [9] scale were distributed to UWE Faculty members. The questionnaire data from NHS and UWE staff were entered into SPSS for Windows (Version 13.0) and exploratory factor analysis was carried out.

Validity of the scales A publicly available tool [9] appeared to measure similar constructs to one of the three scales – effectiveness of care. In total 34 members of staff of the Faculty of Health and Social Care at the University of the West of England, Bristol completed both the newly developed questionnaire and the existing scale from Stronge and Brodt [9]. The results were compared to examine the concurrent validity of the new scales.

Results Response rates Response rates are detailed in Table 1.

Pearson’s correlation coefficient was calculated for each of the 3 scales developed: ‘Efficiency of Care’, ‘Education, Training and Development’ and ‘Control’. Scale 1 (Efficiency of Care) correlated adequately with the Stronge and Brodt [9] scale which looked at similar issues (r = .73). Scales 2 (Education, Training and Development) and 3 (Control) looked at different areas to those covered in the Stronge and Brodt [9] scale and therefore did not demonstrate correlation. It has not been possible to find measures suitable for establishing concurrent validity of the second and third scales at this time.

Table 1 - numbers of completed questionnaires

Round 1 (test) Round 2 (re-test)

Trust 1 78 57

Trust 2 35 28

Trust 3 38 36

Total 151 121

NB – Six questionnaires were returned anonymously and therefore these respondents could not be sent a second round questionnaire. Establishment of the scales

Analysis of scale scores and demographic factors

The scales were constructed using exploratory factor analysis, a standard technique in attitude scale development [18] This process identified three components, with statement loadings >3.1, as recommended by Watson and Thompson [19]. This pattern of item grouping could be understood and interpreted as representing:

For each of the three scales, ANOVA was used to analyse raw scale scores on the basis of demographic variables and those relating to patterns of computer usage. The significance level for the study was set at p=0.05 [21]. 120 is accepted as a reasonable sample to provide a reliability coefficient of 0.8 with a confidence interval of +/- 0.1.[22]

• Scale 1 ‘Efficiency of Care’ • Scale 2 ‘Education, Training and Development’ • Scale 3 ‘Control’ 17 items were included in each of scales 1 and 2, while 14 were included in scale 3. The three scales comprise the

Responses to each of the three scales were compared on the basis of various demographic characteristics of the respondents. The type of trust (ie community, mental health and acute) showed differences on the ‘Education,

178

R. Ward et al. / Developing Information Technology Attitude Scales for Health (ITASH)

Training and Development’ and ‘Control’ scales (F=7.23, p = 0.001; F=3.11, p=0.05 respectively). This suggests that the organisation in which the respondents worked influenced the extent to which they felt prepared for computer use and to which they felt they were able to influence their own working environment. Age revealed no differences, suggesting that age did not significantly influence attitudes toward computer usage. Gender differences were found on ‘Efficiency of Care’ and ‘Control’ scales (t = 2.92, p = 0.004; t = 3.28, p = 0.001), suggesting that men showed a significantly more positive response than women in these areas

and details of individual staff were not available, which would have allowed a more targeted approach. Respondents were not randomised and were self selecting and there may be a difference between those who completed the questionnaire and those that did not, possibly reflecting their attitude towards the subject area. The respondents were from three NHS trusts, within one Strategic Health Authority therefore local initiatives and factors, such as IT training may have influenced the results Representativeness is not claimed but the study secured a range of participants from different NHS organisations. It is not known how closely the demographics of the respondents match the make-up of the wider NHS workforce. However, a wide range of staff from various settings within the three NHS trusts participated in the study.

No differences in responses to any of the scales were found based on the frequency of computer use at home. However, raw scores for each of the three scales according to frequency of PC use at work revealed significant differences (F=4.21, p=0.017; F=5.93, p=0.003; F=5.69, p=0.004 respectively). Those using the computer more frequently at work appeared to display more positive attitudes

During attitude scale construction, the use of exploratory factor analysis is designed to reduce multiple indicators of attitudes etc. by explaining the relationships between potential scale items. It must be remembered, however, that there is always a subjective element in this process [22].

Differences were observed for all three scales (t=2.03, p=0.044; t=2.70, p=0.008; t=2.93, p=0.004 respectively) when they were compared with the level of computer training respondents had received. Respondents with a formal computer qualification displayed more positive attitudes on each of the scales. Similar differences were found when examining experience of computer use. Respondents were asked to rate their experience from 0 (none) to 10 (extensive). Those who rated themselves as 0,1,2 and 3 were grouped as “low experience”, 4,5 and 6 grouped as “medium experience” and those who rated themselves 7,8,9 and 10 as “high experience”. A KruskalWallis H-test of responses to each of the three scales was conducted on the basis of degree of experience. Differences were found on all three scales. (Scale 1 K2(2)=13.52, p 37.5 – BMI < 20 • Laboratory data – WBC > 11,000 or WBC 15 – CRP > 0.8 – Serum Albumin level < 3.0 Figure 6 - Nursing activity and situational variables

For processing of above procedure, all units of inpatient gathered the top 20 disease items that the hospitalization is required. Fianlly, total 125 disease items were selected and analyzed.

Results The 46 nursing plans were postulated from 125 disease items as a prior step before decision of nursing diagnosis and the identification of items of nursing activities (Figure 4). In the new set of nursing intervention (nursing activity), a total of 14,296 items were involved. Among them, 13,548 items were involved in the substructure of the NIC structure through a new methodology (Figure 5). Situational variables were extracted from the new set of nursing inter-

• • •

365

Automatic nursing diagnosis; Successful Implementation of ENR system; Successful Construction of Nursing Terminology DB; – Nursing diagnosis ; 13 domains 92

Y. A. Kim et al. / New Method of Realization of Nursing Diagnosis Based on 3N in an Electronic Medical Record System

– – – –

Nursing interventions ; 13,909 Nursing activities with Order Treatment; 2,183 Situational variables; 63 Set of situational variables connected with nursing diagnosis; 118 1 year after adopted EMR system, we got those responses from the nurses of clinical field.

nursing diagnosis is not an easy procedure for all the acting nurses. So, if there is no automatic creation or selection of nursing diagnosis, the realization of a nursing process would be only happen in the far future. To overcome this obstacle, the set of situational variables in Severance hospital was created and it reveals its effectiveness in real field(Figure 7).

Usage of nursing diagnosis, Addition of nursing activities is more increased. Request of creation of situational variables, request of changes of preset nursing diagnosis is also increased.

Conclusion The study revealed that standardization of nursing activity based terminology can be established and can be evolved by itself. An automatic nursing diagnosis system based on situational variables is a solution for realization of a Nursing Process based on 3N(NANDA, NIC, NOC). Automatic nursing diagnosis systems should be based on an objective, evidence based standpoint. Clinical Decision Support is also possible through automatic nursing diagnosis system.

Figure 7 - New methodology of Nursing Process in Severance ENR 125 disease items may be a small percentage among all disease entities, however, we are sure these 125 high-frequency diseases can involve all the nursing intervention items, nursing diagnosis, and nursing plans. This hypothesis should be tested in terms of the rate of participation and reuse of nursing process in the field, and tested though 1 year’s usage in Severance hospital. As another advantage of this system, the evaluation and validation of the activity based cost of the nursing activity will be enabled in every clinical activity. This kind of advantages would demonstrate the effectiveness or return on investment of this methodology.

Discussion The nursing process is one of the most valuable concepts for effective and safe patient care [1-2]. However, its realization has been a questionable one, because of difficulty in adoption in the field for real-world use. Most nursing professionals think that the items of the NIC system can accomplish the thread of the nursing process [3]; we focused on whether the NIC system is able to express the interventional activities of the real practice in the field. If there is limitation of expression of real activities, it would indicate a disconnect between the conceptual items and their related activities.

For better nursing process, the contents of substructure of the intervention items should be reinforced with the concrete relationship with the nursing diagnosis and outcome systems. [2][3][4]

So the trial of a process was done for the new connection between academic concept and real activities that can be involved in the academic concept. [2][4] All the currently used interventional activities should be gathered from the specified units. [4]

Acknowledgment This study was supported by a grant from the Korea Health 21 R&D Project, Ministry of Health & Welfare, Republic of Korea (Grant No. A050909).

Reference

During these procedures we find that the meaning of the items of NIC is much broader than items from the real field. So, many items were found that can not be expressed by the items of NIC system in the real field, so we categorized additional items of nursing activities as a substructure of NIC items from the clinical disease base. This new structuring was the key point of establishing the successful nursing process.

[1] Yong Oock Kim, et al. Essential elements of EHR system, Seoul (ROK): Koonja, 2005. [2] Saba VK. Essentials of Nursing Informatics. 2005, McGraw-Hill [3] Handbook of Informatics for Nurses & Health Care Professionals (3rd Edition), Toni Lee Hebda [4] NANDA, NOC, and NIC Linkages: Nursing Diagnoses, Outcomes, and Interventions, Marion Johnson, 2001, Mosby: St.Louis.

From the set of situational variables, we decide the nursing diagnosis. This can be said as a kind of clinical decision support system in the real patient care field. The nursing process is impossible without first designation of nursing diagnosis in current concept. However the decision of

Address for correspondence e-mail [email protected]

office 82-2-2228-22218

366

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Integration of Longitudinal Electronic Records in a Large Healthcare Enterprise: The U.S. Veterans Health Administration Experience Arthur C. Curtisa, Joseph Gillonb, Dale C Malmrosec a

VHA Health Enterprise Strategy, Boston MA, USA, b VHA Medical Center, Ann Arbor MI, USA c VHA OI Field Office, Salt Lake City UT, USA

care delivery system that promotes primary and ambulatory care. Between 1996 and 2005, the annual number of ambulatory care visits increased by 84%, the number of acute care hospital beds dropped by 68%, inpatient admissions dropped by 27%, and total bed days of care fell by 47%.

Abstract The U.S. Veterans Health Administration (VHA) provides care to some 5.2 million patients spread across the continental United States, Alaska, Hawaii, Puerto Rico, and the Philippines. Sites of care include 157 medical centers, nearly 900 outpatient clinics, long-term facilities, and home care. Over the last 10 years, major changes in the nature of VHA healthcare have imposed a requirement for longitudinal electronic health records and integration of those records across the enterprise at the point of care. VHA has now evolved through three generations of applications that support such integration. This paper reports on the VHA experience, points out lessons learned, and outlines future directions for electronic health record integration in VHA.

Aging of the veteran population is a major issue confronting the VA. Today, 9.6 million veterans are age 65 or older, representing 38 percent of the total veteran population. By 2030, the proportion of older veterans will increase to 45 percent of the total. As in the general U.S. population, those aged 85 or older (the "old-old") are the fastest growing segment of the veteran population, representing 3 percent of current veterans. The number of veterans age 85 or older is expected to nearly double from 764,000 to a peak of 1.4 million between 2003 and 2012.

Keywords:

In comparison with the U.S. Medicare population, the VHA patient population is older and sicker. 38% are over age 65. In comparison to age-matched Americans, VHA patients carry one additional mental health diagnosis and three additional non-mental health diagnoses. They are also less affluent: approximately 70% with annual incomes of less than $26,000 and some 40% with annual incomes less than $16,000. Demographics of the patient population are changing as older veterans die and veterans of more recent conflicts become eligible for care.

Electronic Health Records, EHR, Longitudinal Records, Data Integration, VistA, Veterans Health Administration.

Introduction Veterans health administration The United States Department of Veterans Affairs (VA) provides benefits and services to the nation’s veterans and selected family members (“To care for him who shall have borne the battle and for his widow and his orphan…” Abraham Lincoln). The Veterans Health Administration (VHA) is the branch of VA that provides healthcare benefits.

A progressively older population with more chronic diseases and increasing complexity of illnesses has resulted in involvement of more clinical providers (both primary and specialty care), more types of health care programs (inpatient, outpatient, and home care), and more diagnostic procedures done at geographically dispersed sites. In addition, restructuring of VHA into health care networks has increased patient travel and the number of institutions at which a given patient can be expected to receive care, which has in turn increased the medical and financial value of sharing information on patient care across sites on an enterprise basis.

VHA includes approximately 1,400 sites of care, including 157 medical centers, nearly 900 outpatient clinics, 135 long-term care facilities, and 88 home-care programs. VHA now treats some 5.2 million patients, with approximately 7.6 million enrollees. In 2005, VA workload included 587,000 inpatient admissions and some 52 million outpatient visits. 60% of US health professionals (70% of physicians) have some training in VHA. VHA has some 193,000 employees (approximately 14,000 physicians, 50,000 nurses, and 33,000 allied health personnel). VHA’s budget in 2005 was $32.5B US. In the last ten years, VHA has transformed itself from a loose federation of independent medical centers focused primarily on inpatient care into a fully integrated health

367

A. C. Curtis et al. / Integration of Longitudinal Electronic Records in a Large Healthcare Enterprise

VHA clinical information systems & electronic records

Content category

Veterans health information systems and technology architecture (VistA)

Medications Administered - Bar Code Medication Administration

The Veterans Health Information Systems and Technology Architecture (VistA) supports day-to-day operations at VA health care facilities [1,2]. It incorporates both administrative and clinical applications; including a full suite of ancillary departmental systems as well as the Computerized Patient Record System (CPRS). A descendant of the Decentralized Hospital Computer Program (DHCP, circa 1982), VistA has been in continual evolution since 1996. It consists of more than 100 corporately developed applications as well as software developed at local medical facilities; it also includes support for integration of commercial off-the-shelf products. Historically, VistA has employed a decentralized architecture operating at the medical center level; applications and national databases derived from locally generated data that reside in VA's centralized corporate data center lie outside the scope of VistA. VistA employs a client-server architecture: applications are written in M (also known as MUMPS) on the server side and in Delphi on the workstation side; VHA is in the early stages of a major technology migration that will shift application development to Java and relational database systems. VistA has been used by a number of private and public health care facilities in the United States and around the world.

Total count 850 M

Average daily volume 607 K

VHA has gone through three approaches to enterprise integration: WebTop, CPRS Remote Data Views, and VistA Web.

Materials and methods WebTop Motivation: Development of WebTop began in 1996 as a centrally-funded research project to explore the feasibility of extracting and displaying data from a VistA system using browser-based technology, with an emphasis on use of XML, Java technologies, and linkage of patient data to external resources. A follow-on project focused on integration of data across sites was undertaken in 1997, and the resulting application was brought to operational status in VISN 1 (VA New England) in 1998. Other VISNs adopted the technology, and at its peak, WebTop was deployed in 16 of 22 VISNs. WebTop development preceded that of CPRS, and was thus unable to take advantage of some of the foundations of the latter application. Architecture: WebTop provided browser-based access to VistA clinical data using a combination of client-side and server-side software controlled and organized by a middleware server. (See Figure 1.) The middleware retrieved clinical data from multiple VA Vista systems and formatted it into a consolidated display. The middleware also supported retrieval of data from other intranet or internet information resources, such as knowledge bases, using patient data to construct a search query. Deployment was based on a regional model, with one WebTop instance per VISN; the architecture allowed for cross-VISN connectivity to produce integrated data sets covering multiple regions, but this approach was never generally adopted due to both technical and organizational obstacles at the time.

Computerized patient record system (CPRS) CPRS is the electronic “clinical practice environment” for VA clinicians and all members of the healthcare team; it is a single, integrated clinical information system used throughout the organization in all healthcare settings (Inpatient, Outpatient, Long-term care). CPRS provides a longitudinal medical record as well as active functionality covering all aspects of patient care and treatment, including: electronic order entry and processing; entry and retrieval of clinical documents such as narrative notes and discharge summaries; display of results for laboratory, imaging, and diagnostic procedures; consult management and reporting; alerts of abnormal results; clinical reminders; problem list management; inpatient and outpatient medication management. It also serves as an enterprise integration platform, giving clinicians the ability to see electronic records from any other VHA facility where the patient has received care. As shown in Table 1, CPRS is heavily used.

WebTop middleware was based on Java servlets, which handled log-on, patient look-up, clinical data retrieval, and traversal of links to other network resources. In keeping with the focus on web technology, HTTP was initially the sole choice for connectivity; when CPRS was implemented, the set of transport layer protocols was expanded to include Remote Procedure Calls (RPC).

Table 1 - CPRS usage as of December, 2006 Content category Documents (Progress Notes, Discharge Summaries, Reports) Orders Images Vital measurements

Total count 874 M

Average daily volume 638 K

1.6 B 591 M 1.0 B

912 K 884 K 729 K Figure 1 - WebTop architecture

368

A. C. Curtis et al. / Integration of Longitudinal Electronic Records in a Large Healthcare Enterprise

Operational model: Users connected via browser to the WebTop middleware server, which validated the user’s access codes against a selected VistA system. Once validated, the user was assumed to have access to data that resided on any Vista system participating in the WebTop domain (see discussion of business rules below). After selecting a patient, the user could browse the patient’s record, selecting individually the type of data desired (e.g., lab results, progress notes, etc.). The middleware server processed such user requests, retrieved data from the individual VistA databases and aggregated it for display by applets running on the user’s workstation.

date, CPRS iteratively asks the local VistA if the data is ready yet; when all the data has been returned from the remote VistA systems, the local VistA answers CPRS's next query with a YES, and CPRS issues the third and final RPC to get the data. The data is then displayed in spreadsheet form or tabs of narrative text, much like the approach taken with WebTop. At the remote sites, RDV uses "visitor" accounts that have no access or verify codes, no menu options, and are aliased as "VISITOR". However, the identity of the user originally requesting the data is passed to each remote VistA system and entered in an access log.

CPRS remote data views

Motivation: VistAWeb was initially developed within one of VHA’s regional networks as an exploration of alternatives to WebTop and CPRS Remote Data Views. It also explored user interface alternatives, including alternative ways to deliver graphical information such as ECG or DICOM images.

VistA web

Motivation: Development of CPRS Remote Data Views (CPRS/RDV) began in 2001. Its objective was to support on-the-fly remote data views of patient information within VHA’s mainstream clinical application. Because CPRS deployment was ubiquitous, CPRS/RDV offered the first opportunity for true enterprise-wide integration of the longitudinal electronic health record in VHA.

Architecture: VistAWeb uses Medical Domain Objects (MDO) as its data abstraction layer, and MDOs handles all connectivity to VistA. On one side an MDO communicates with various data sources such as VistA, HL7, DICOM, XML, and SQL. On the other side, it delivers to the client a uniform, well-defined suite of objects from the medical domain (e.g., patient, provider, progress note, lab result, etc.) MDO delivers the same object regardless of the data source. So, to the client software, a progress note from VistA looks and behaves exactly the same as a progress note from an arbitrary source via HL7 or SQL or XML. All functionality is read-only.

Architecture: Traditional CPRS architecture was based on communication between a client CPRS workstation and a VistA server using remote procedure calls (RPCs). Security concerns, desire to avoid additional servers, etc. led to implementation via server-to-server communication rather than client-to-servers or client-to-middleware. (See Figure 2.) To add this new capability to CPRS, changes were made to both the CPRS application and the underlying infrastructure; the RPC Broker was enhanced to provide the Server-to-Server RPC support required, and HL7 messages were encapsulated and exchanged within RPC messages.

Figure 3 - VistAWeb architecture Data Access Objects (DAO) are used to carry out communication. In the general case, DAOs would exist for different access protocols: VistA, HL7, SQL, DICOM, or XML. MDO is an ongoing project; the DAOs currently operational are for accessing VistA systems via RPC and the HDR via HL7.

Figure 2 - CPRS Remote Data Views architecture Operational model: Since CPRS RDV is part of CPRS, user logon, authentication to VistA system, and selection of a patient are all accomplished in the normal CPRS context. CPRS knows the sites at which a patient has data courtesy of the VHA enterprise Master Patient Index; information on the sites applicable to a specific patient is kept as part of the local patient record, and a user asking for remote data can limit the sites queried if desired. Once a type of data is chosen, the sequence is as follows: RDV asks the local VistA for remote data; the local VistA system adds a user identifier to the request and passes it on to the remote VistA systems; the local VistA returns a handle to CPRS; while the remote VistA systems assemble the

Operational model: Users authenticate to VistAWeb via standard VistA credentials and are then able to browse the clinical record. Connections are made to sites determined by MPI data stored with the patient. If necessary, a new visitor identity is created at remote sites to support auditing of remote data access. Users are warned if they select a deceased or sensitive patient (e.g., patient as employee, patient as high-profile patient, etc.); a check is also made to prevent users from accessing their own records. User

369

A. C. Curtis et al. / Integration of Longitudinal Electronic Records in a Large Healthcare Enterprise

requests for a specific type of data are sent to DAOs, one DAO per site, each of which communicates directly with its associated VistA system. MDO acquires data from VistA using the same RPCs that CPRS uses; the data returned is labeled with a site tag, collated, sorted, and returned to the user’s browser as a standard HTML page – data displayed is patient-centric instead of geo-centric.

release, providing a convenient integrated view of data from within the context of the CPRS interface.

Discussion The industry press is replete with accounts of both successful and unsuccessful attempts to introduce clinical information systems into clinical care. Consideration of the factors which have led to VHA’s successes in this arena over a quarter century is beyond the scope of this paper: suffice it to say that a major factor has been a tradition of large-scale involvement of clinical users in defining requirements and functionality of clinical systems. Another is reflection in spending patterns of belief in clinical systems. Until very recently, VHA never had a budget line item for IT – it was paid for out of health care dollars, thus assuring that investments made were perceived as having high value in clinical care.

Results WebTop: Deployed in 1998, WebTop employed a user interface that was different than that of CPRS and required an independent user authentication step and invocation of a separate application (i.e., a web browser) to view an integrated set of patient data. In addition, as a field-developed product that required local or regional implementation effort over and above mainstream systems, WebTop never achieved enterprise-wide deployment. As a result, some users were reluctant to invest in learning how to use it. However, WebTop provided a compelling proof-of-concept, and demonstrated that clinicians were willing to make compromises on data normalization across sites (e.g., variability in drug or lab test names) in return for the ability to see data integrated across sites. WebTop undoubtedly succeeded at the regional level in meeting its fundamental objective of providing clinicians with the ability to access the equivalent of a single clinical repository containing all data on a patient. It quickly become a part of the daily life of clinicians in the VISNs in which it was implemented, as evidenced by the rapidity with which clinicians were heard from when there was any problem with the system.

Beyond such broad considerations, however, VHA has learned a number of useful lessons from the series of projects described in this paper which should be of use to other organizations taking on the challenge of data integration across a healthcare enterprise. •

•

CPRS/RDV: Implemented in April, 2002, CPRS/RDV rapidly supplanted WebTop due to its availability within the CPRS user interface as well as its national scope. As a corporate product it had advantages over field-developed products (e.g., inserting a stub in the user file to support tracking of access to patient data). It is interesting to note, however, that until WebTop was finally decommissioned, there remained a core of users who preferred its speed and simplicity in spite of its constraints. Implementation of CPRS RDV was a watershed event in terms of making data available across an entire healthcare enterprise of major size and scope, and it continues as a core function of the clinician interface. While it is clear from observation and anecdotal reports that CPRS RDV is used extensively, the application is unfortunately not instrumented so as to allow reporting on actual usage volume and on the types of data being retrieved.

•

VistAWeb: Released to the field in March of 2005, VistAWeb combines aspects of WebTop and CPRS, and adds major innovations of its own. What began as a local project to explore alternatives to a mainstream corporate application became, in turn a corporate product itself. In the process, important foundations were established: use of web services, a simplified model for clinical data, and return to a model for access to data from VistA sites not based on VistA-to-VistA communication. VistAWeb is in production use in VHA an alternative to CPRS RDV; it is slated to replace the Reports tab of CPRS in an upcoming

•

•

370

It is essential to achieve a compromise between perfection, in terms of normalization of vocabularies and forms of representation, and practical utility. As the saying goes, “Don’t let the perfect be the enemy of the good”. VHA’s experience clearly demonstrates that the human brain remains capable of smoothing over discontinuities that could otherwise prevent a project from getting started at all. Clinicians favor working in a single application over multiple applications or multiple browser windows, and value incorporation of functionality such as data integration with applications that are part of their primary workflow. Day-to-day operation of an integration framework depends on changing the traditional attitude that access to patient data at one medical center by staff at another is suspect. Thus, an enterprise or consortium intending to implement such a framework must be willing to implement business rules such as the following: a provider who has access to a given patient's data at one medical center has access to that patient's data at all participating sites; authentication of a provider's credentials at one site is sufficient to authenticate that provider's access at all sites; a locally acceptable indication of right to access patient data is an acceptable surrogate for permission to access patient data at all sites. Regardless of whether an organization has an inclination toward research, it is important to instrument applications so that usage patterns can be studied and understood. This has not been adequately appreciated in our own organization, and we have regretted it a number of times. While long-term success requires that an enterprise eventually establish standards for what data each site must commit to collect electronically, sites just begin-

A. C. Curtis et al. / Integration of Longitudinal Electronic Records in a Large Healthcare Enterprise

•

ning the process must educate providers to have realistic expectations for what data will be found in the base systems, and thus the integrated data set. While corporate information technology programs are essential to the widespread deployment and sustained operation of sophisticated systems, investigative projects and local innovation continue to be the wellspring of watershed projects that lead to major advances in the use of information systems in clinical practice.

We close this paper with an anecdote that illustrates the value of the applications described here. VHA was severely affected in the coastal Southeastern states when Hurricane Katrina hit on August 29 of 2005. Multiple sites were affected: as an example, the New Orleans VA Medical Center was submerged to the second floor (where the computer room was) and was closed, with most inpatients going to Houston, Texas. Outpatient services in affected areas were dislocated. The “RecoverAll” emergency response team was dispatched on September 1, and the New Orleans computer system was replicated in Houston in twenty-four hours, with all records restored. It is estimated that it required about 100 hours to restore EHR capability for all affected facilities.

Future plans While VistA is a tightly integrated suite of applications, VHA’s current facility-level systems represent islands of data in an enterprise sense bridged by a series of integration applications. Desire for a patient-centered, “gold standard” legal electronic record and a more tractable approach to making computable data available for decision support have motivated VHA to undertake migration to a new environment, referred to as “HealtheVet”, a core component of which is a health data repository (HDR) at a national level. Although the transition from VistA to HealtheVet will of necessity be gradual (akin to building an airplane in flight), the HDR will eventually serve as the source of a patient’s electronic health record, providing a superset of the integration available today and obviating the need for applications such as CPRS/RDV and VistAWeb. As a corollary activity, VHA is embarking on a major data standardization and data modeling effort which will support application re-engineering. A major long-term implication of this undertaking is the positioning of VHA as a single national health information organization (NHIO) interacting through a National Health Information Network (NHIN) with other public- and private-sector entities delivering care to VHA patients.

In the days following Hurricane Katrina patients started to appear at medical centers outside the affected area, initially at other VA medical centers in the Southeast and later at hospitals as far away as Puget Sound, in the far Northwest. The high number of refugees precluded normal access through CPRS/RDV. The solution was to grant temporary special user privileges to all VistAWeb users which gave users the ability to do patient lookup at other sites, and thus get VHA-wide data on any patient from the affected VISNs regardless of whether they were registered in the provider’s local VistA system. Thus, electronic health records – and beyond that the integration of data across sites – played a major role in continuity of health care through a massive natural disaster.

References [1] Kolodner RM, editor. Computerizing large integrated health networks: The VA Success. New York: Springer-Verlag. 1997. [2] Meldrum K, Volpp B, Vertigan R. Department of Veterans Affairs' Computerized Patient Record System. Proc AMIA Symp 1999(1-2):1214. [3] Brown SH, Lincoln MJ, Groen P, Kolodner RM. VistA: The U.S. Department of Veterans Affairs National Scale Hospital Information System. International Journal of Medical Informatics 2003:69(2/3):135-156. [4] Richard S. Dick and Elaine B. Steen, Eds., The ComputerBased Patient Record: An Essential Technology for Health Care, Institute of Medicine, Washington, DC, 1991.

Conclusions VHA has demonstrated the feasibility of an enterprisewide integrated patient record in a very large healthcare organization of national scope. While others have reported on clinical data integration projects, most are still in a formative stage and few if any have the breadth or depth of VistA and its current integration platforms [3]. Trends in healthcare and efforts at all levels encouraging the development of partnerships for sharing electronic health records have had major impact on requirements for crosssite data integration: practical technical and economic models are still being developed. It is interesting to observe that the majority of the concepts first proposed in the initial paper from the Institute of Medicine on electronic health records [4] are only emerging now, some 15 years later.

Address for correspondence Arthur Clayton Curtis MD VA Medical Center (WHC), 150 South Huntington Avenue Boston, MA 02130. [email protected] The opinions expressed in this paper are those of the authors, and not necessarily those of the Veterans Administration or the United States Government.

371

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

The AMPATH Medical Record System: Creating, Implementing, and Sustaining an Electronic Medical Record System to Support HIV/AIDS Care in Western Kenya William M. Tierneya, Joseph K. Rotichb, Terry J. Hannanc, Abraham M. Siikad, Paul G. Biondicha, Burke W. Mamlina, Winstone M. Nyandikod, Sylvester Kimaiyod, Kara Wools-Kaloustiana, John E. Sidlea, Chrispinus Simiyud, Erika Kigothod, Beverly Musicka, Joseph J. Mamlina, Robert M. Einterza a

Indiana University School of Medicine and the Regenstrief Institute, Indianapolis, Indiana, USA b Moi University School of Public Health, Eldoret, Kenya c Launceston General Hospital, Launceston, Tasmania, Australia d Moi University School of Medicine and Moi Teaching and Referral Hospital, Eldoret, Kenya

Moi Teaching and Referral Hospital created AMPATH – an Academic Model for the Prevention And Treatment of HIV/AIDS [2,3] to leverage the power of universities to establish systems of care, train multidisciplinary providers of HIV/AIDS care, and research innovative methods for enhancing the quality and outcomes of care. As shown in Figure 1, initial grown in AMPATH’s care programs was slow until mid-2004 when major funding was obtained. By mid-2007, more than 45,000 patients have been enrolled, 1500-2000 new patients being enrolled per month.

Abstract Providing high-quality HIV/AIDS care requires high-quality, accessible data on individual patients and visits. These data can also drive strategic decision-making by health systems, national programs, and funding agencies. One major obstacle to HIV/AIDS care in developing countries is lack of electronic medical record systems (EMRs) to collect, manage, and report clinical data. In 2001, we implemented a simple primary care EMR at a rural health centre in western Kenya. This EMR evolved into a comprehensive, scalable system serving 19 urban and rural health centres. To date, the AMPATH Medical Record System contains 10 million observations from 400,000 visit records on 45,000 patients. Critical components include paper encounter forms for adults and children, technicians entering/managing data, and modules for patient registration, scheduling, encounters, clinical observations, setting user privileges, and a concept dictionary. Key outputs include patient summaries, care reminders, and reports for program management, operating ancillary services (e.g., tracing patients who fail to return for appointments), strategic planning (e.g., hiring health care providers and staff), reports to national AIDS programs and funding agencies, and research.

Figure 1 – Monthly and cumulative AMPATH enrollment Hindering the ramping-up HIV/AIDS treatment programs is the lack of timely and accurate data on (1) the number of HIV-infected patients receiving care, (2) the number eligible for ARVs, (3) the number receiving ARVs, and (4) the positive and negative outcomes of treatment. Funding agencies often deny major program funding unless programs can document the number of patients treated and their outcomes. Paper-based record systems can only meet such information needs for small numbers of patients. The lack of EMRs has slowed the delivery of HIV care despite availability of funds for treatment from the U.S. [4], World Health Organisation [5], and philanthropic foundations [6]. This “Digital Divide” [7] has likely resulted in thousands of unnecessary deaths.

Keywords: edical record systems, computerized; HIV; acquired immunodeficiency syndrome; developing countries; Africa south of the Sahara; Kenya.

Introduction More than 38 million persons are currently living with human immunodeficiency virus (HIV) worldwide, 25 million (63%) in sub-Saharan Africa where only 1 million (= 140 O R o/data[at0001]/ev ents[at0006]/data[at0003]/item s[at0005]/value/value >= 90

Figure 1 - A typical EQL statement •

• •

•

Supports arithmetic operations (such as count, addition, subtraction, multiplication, and division), relational operations (>, >=, =, !=, 7

Scenario two

Arithmetic functions

Scenario description

A set of arithmetic functions, such as addition, subtraction, multiplication, and division can also be used in EQL. The use of these functions is the same as SQL, and is not described here.

Get all HbA1c observations that have been done in the last 12 months for a specific patient.

TIMEWINDOW clause TIMEWINDOW is an addition query clause used in EQL to constrain the query to data that was available in the system within the specified time criteria. This supports a

400

C. Ma et al. / EHR Query Language (EQL) – A Query Language for Archetype-Based Health Records

EQL expression

Future directions The EQL continues to be developed based on requirements from additional clinical query scenarios. New EQL features may need to be provided, such as statistical, string pattern matching and user-defined functions. Existential () and universal () quantifiers may be also required.

SELECT o FROM EHR e[ehr_id=$ehrId] CONTAINS COMPOSITION c [openEHR-EHR-COMPOSITION.report.v1] CONTAINS OBSERVATION o [openEHR-EHR-OBSERVATION.laboratory-hba1c.v1] WHERE c/context/other_context[at0001]/items[at0006]/ items[at0013]/value > current-date()-1PY

Scenario three

Other research areas may include 1) exploring how the EQL supports clinical decision support technologies, e.g., clinical guidelines presentation; 2) investigating the integration of EQL with clinical terminology servers; and 3) conducting field trials using the EQL to represent common clinical queries to retrieve openEHR-based EHR data sets.

Scenario description

Conclusion

Get a patient’s current medication list

The use of a common RM and archetypes supports the sharing of EHR data, and with the addition of a companion query language, such as EQL, achieving semantic interoperability of EHR information is much closer.

EQL expression SELECT c FROMEHR e[ehr_id=$ehrId] CONTAINS COMPOSITION c [openEHR-EHR-COMPOSITION.medication_list.v1] WHERE c/name/value= ‘current medication list’

Acknowledgements We extend our thanks to Lisa Thurston for her support, including the development of the EhrView component – a generic EHR viewer – that uses EQL for retrieving EHR data.

References Discussions

[1] National E-Health Transition Authority (NEHTA). Acronyms, Abbreviations & Glossary of Terms in Publications. In: eds. National E-Health Transition Authority (NEHTA), 2005. [2] National E-Health Transition Authority (NEHTA). Review of Shared Electronic Health Records Standards in Publications. In: eds. National E-Health Transition Authority (NEHTA), 2006. [3] Amatayakul M. When EHRs are a-ok. Healthcare financial management, 2006: 60(2): 146-148. [4] Nadkarni PM, Brandt C. Data Extraction and Ad Hoc Query of an Entity-- Attribute-- Value Database. J Am Med Inform Assoc, 1998: 5(6): 511-527. [5] W3C.2006. XQuery 1.0: An XML Query Language.http:// www.w3.org/TR/xquery/, accessed on 17 Oct. 2006 [6] Johnson S, Chatziantoniou D. Extended SQL for manipulating clinical warehouse data. Proceedings / AMIA ... Annual Symposium, 1999: 819-23. [7] Beale T, Heard S. Archetype Definitions and Principles in The openEHR foundation release 1.0. In: Beale T and Heard S, eds. The openEHR foundation, 2005. [8] Beale T, Heard S. openEHR Architecture: Architecture Overview in The openEHR foundation release 1.0.1. In: Beale T and Heard S, eds. openEHR Foundation, 2006.

This paper has introduced EQL – a declarative query language developed for querying openEHR-based EHRs by utilising the openEHR path mechanisms and unique containment syntax. However, the EQL syntax is not specific to the openEHR RM and can be used for any archetypebased information system. It could be used as a common query language for disparate archetype-based applications. The EQL will be submitted to the openEHR foundation as a candidate openEHR query language. EQL implementation Ocean Informatics Ptd Lty has implemented the components to process the EQL within the OceanEHR suite of EHR tools. These components include an EQL parser, the EHR query object model and a query processor. The implementation does not currently support all features of the EQL. However it has demonstrated the power and flexibility of using a common RM, archetypes and EQL, independent from the underlying system implementation, to retrieve any data set from an EHR. The EQL is not easily understood by health professionals due to the computer-oriented openEHR path syntax used in EQL. An EQL query editor, which allows users to generate and edit EQL statements, has been developed to empower clinicians with fine-grained access to their EHR data. The query editor provides users with access to an archetype repository to build FROM containment constraints and a tree representation of RM attributes and archetype structures used to set WHERE criteria, SELECT data items and ORDER BY preferences. The tool can then execute the generated query and display the returned results without the user seeing or knowing how to write an EQL statement.

Address for correspondence Dr Chunlan Ma Clinical Informatics Consultant [email protected] Heath FrankelProduct Development Manager [email protected]

401

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Design and Evaluation of a Temporal, Graph-Based Language for Querying Collections of Patient Histories Ole Edsbergac, Stein Jakob Nordbøc, Erik Vinnesbc, Øystein Nytrøac a

Department of Computer and Information Science (IDI) b Faculty of Medicine (DMF) c Norwegian EHR Research Centre (NSEP) Norwegian University of Science and Technology (NTNU), Trondheim, Norway

ments collection put it, “(...) I lack basic search functionality, mostly because of my own aversion against learning a programming language for searching (...)”.

Abstract Giving clinicians and researchers the ability to easily retrieve and explore relevant fragments of patient histories would greatly facilitate quality assurance, patient followup and research on patient treatment processes. Established database query languages are inconvenient for such exploration, and may also be too complex for users with limited backgrounds in informatics. We believe that understandability can be increased in return for a sacrifice of some of the power of expression found in general query languages. In order to design a specialized query language, we have collected and synthesized a tentative list of requirements. Based on these requirements, we have designed and implemented Practice Explorer, a prototype for visual query of collections of patient histories, and evaluated the understandability of its query language by testing with medical students. The results indicate that parts of the language are intuitive enough for users to understand without demonstrations, examples, feedback or assistance. They also provide some lessons for future work in this area.

The research question this article addresses is: How can we design a system for formulating temporal queries against patient history databases that is easily understandable for users with little competence in informatics, but still satisfies most of their query needs? The contributions of this article are 1) a tentative list of requirements for the expressiveness of patient history query languages, 2) an outline of our design for a query system satisfying the requirements and 3) results, observations and lessons learned from understandability testing of a prototype implementing our design. Related work The system most closely resembling Practice Explorer is the very recent PatternFinder [2]. The most important differences are that Practice Explorer visualizes queries as hierarchical, directed acyclic graphs, whereas PatternFinder uses linear chains of forms, and that Practice Explorer's query language, prompted by the requirements described below and enabled by the more flexible query model, is also able to express queries with parallel and alternative branches. It would be interesting to compare a form-based and a graph-based visual query system to see which is more intuitive for users without a background in informatics. Also related is TVQL [3], a visual query language where binary interval relations are specified via sliders, and where multiple such relations can be combined via neighborhood and disjunction. Another related study [4] proposes three new notations (elastic bands, springs and paint strips) for interval relations and experimentally compares their understandability. It also provides a table comparing different approaches to visualizing temporal relations and specifying combinations of such relations with logical expressions. In our case, the query graph both specifies local temporal relations and conjunction and disjunction of sub-queries through the parallel and alternative branching constructs.

Keywords: information retrieval. Data display. Medical record

Introduction Clinicians and health researchers have a need for querying their patient records for relevant history fragments. Four tasks where this need may arise are: 1. Retrospective study of guideline compliance [1]. 2. Re-consideration of treatment plans for groups of patients possibly affected by the discovery of new medical knowledge, such as the connection between H. pylori and peptic ulcers. 3. Selection of patients for scientific studies. 4. Development of research hypotheses through explorative search of patient records. The text-based database query languages available in today's patient record systems are either too complex for users without informatics competence or has insufficient power of expression. As one of the sources in our require-

Our visual representation of branching and joining constructs is borrowed from UML activity diagrams [5], a flowchart notation for defining workflows. In fact, our

402

O. Edsberg et al. / Design and Evaluation of a Temporal, Graph-Based Language for Querying Collections of Patient Histories

query graphs can be viewed as flowcharts, extended with some new constructs and given an alternative semantics suitable for matching against histories.

between the history's start and the initiation of the ACEinhibitor medication. We have no illusions that our list of requirements is complete. In our experience, potential users are often not fully able to understand the possibilities offered by a temporal query system without having such a system available for use on their own data. Therefore, it will be necessary to iterate between developing prototypes and collecting more requirements.

Materials and methods Requirements collection Through discussions with two general practitioners, a rheumatologist and a health researcher interested in clinical processes, and through a pilot study applying an early prototype to a general practitioner's database and letting him verbally specify queries to be executed by the developers, we collected example queries and synthesized the following list of requirements for patient history query languages. Queries should be able to find patterns consisting of:

Data source and data model Our test case is a patient record database from a general practitioner's office. For the sake of query performance, Practice Explorer extracts relevant data from the database at startup and represents the patient histories in main memory as lists of events. Each event has a time stamp and is considered to last for 24 hours because the test case database does not contain accurate time information at smaller granularities than days. The types of events include patient encounters (with diagnosis codes), lab tests, prescriptions and correspondence. For prescriptions, cessation dates are, where possible, heuristically deduced from fields in the prescription. Events have various attributes, such as codes, values and text, depending on their type. Practice Explorer is currently only able to extract data from Profdoc Vision, a patient record system widely used by Norwegian general practitioners. In this system, encounter diagnoses are coded according to the International Classification of Primary Care (ICPC), and prescriptions are coded according to the Anatomical Therapeutic Chemical Classification (ATC).

1. A primitive history element, such as a patient encounter, lab test, prescription or correspondence event. 2. Results limited by the date or the age of the patient at a specific point in a pattern. 3. A time interval in which a medication has been prescribed, including overlapping prescriptions, or the start or end of such an interval. 4. Time periods of variable length, with the possibility of specifying that a specified pattern should, or should not, occur during the period. 5. Repetitive occurrence of a specified pattern. 6. Sequences of specified patterns. 7. Parallel occurrence of specified patterns. 8. Alternative occurrence of specified patterns. 9. The first occurrence of a pattern in the whole history. It should also be possible to:

Query language The main idea behind Practice Explorer's query language is to visualize queries as directed acyclic graphs, with each vertex describing a part of the history and with the edges always directed to the right, towards the present time. A query defines what a segment of a history must be like to constitute a match for that query. The informal interpretation of a query is that, for a query to match a segment of a history, it must be possible to simultaneously walk from left to right through the query and the history segment, encountering a matching history part for every query element passed. An edge between two elements indicates that the match for the element on the right hand side must begin at the exact same time that the match for the element on the left hand side ends. In other words, edges do not represent passage of time, but connects temporally juxtaposed events. Figure 1 shows two very simple queries. The bottom query specifies that a contact with diagnosis code K86 must occur, immediately followed by a period of medication with beta blockers. The top query specifies the same situation, except that an unlimited amount of time, represented by the spring-like middle element, is allowed to pass between the contact and the start of the medication period.

10. Specify encounter events and medication intervals at various points of abstraction in relevant coding hierarchies. 11. Perform union and intersection set operations on query results. 12. Save and re-use queries or their components. Table 1 shows a natural-language specification of a query need that exemplifies many of the requirements. Table 1 - Natural-language specification of a query need Find all patients who initiated medication with an ACE-inhibitor for the first time in their histories without having any encounters coded as angina, myocardial infarction or heart failure in the preceding two-year period, and who had an en-counter coded as hypertension some time between the history's start and the initiation of the ACE-inhibitor medication. Find all patients who initiated medication with an ACEinhibitor for the first time in their histories without having any encounters coded as angina, myocardial infarction or heart failure in the preceding two-year period, and who had an encounter coded as hypertension some time

403

O. Edsberg et al. / Design and Evaluation of a Temporal, Graph-Based Language for Querying Collections of Patient Histories

The user builds a query by dragging components from a menu to a panel containing the query graph under construction. Dialog boxes ask for necessary parameters as components are added. Figure 2 shows a moderately complex query graph. The query language has a textual syntax, the central structure of which is defined by the grammar in table 2. A query is any derivation from . The visual representation of many of the elements can be seen in figure 2. Figure 1 - Simple queries.

Space does not permit giving a full formal definition of the query language and its semantics. We will now informally describe the semantics of the query components given by the grammar.

Table 2 - The central structure of the query language

::= | | sequentialComposition(, ) | parallelComposition(, ) | alternativeComposition(, ) | firstOccurrence() ::= encounter() | prescription() | medicationStart() | medicationEnd() | test(, ) | correspondence() | dateControl() | ageControl() ::= medicationInterval() | timeWindow(, ) | timeWindowWith(, , ) | timeWindowWithout(, , ) | repetition(, , , , )

Point queries, except for date controls and age controls, match events of the corresponding types satisfying the criteria in the parentheses. Date controls and age controls match all points in histories where the date or patient age could be successfully verified to belong to the specified range. •

Of the interval queries, medication intervals match any continuous time period where the patient is deduced, from prescription events, to be taking medication of the given code. The three types of time windows match time periods of duration between a given minimum and a given maximum, with a possible additional requirement that a given query must, or must not, match within the period. Repetition queries require that a given query gives repeating matches with upper and lower limits for the count and the duration allowed between repeats.

Figure 2 - Query building window showing a query satisfying the query need from table 1. From left to right, we see the following: an element matching the beginning of the history, followed by a branching into two parallel threads, along the upper of which the last two-year-period must not contain a contact with any of three codes and along the lower of which a contact with a given code must occur somewhere, followed by a re-joining of the parallel threads, followed by an interval of medication with a given type of drug, which must be the first of its kind in the entire history. (The query builder was constructed using the prefuse toolkit [6].)

404

O. Edsberg et al. / Design and Evaluation of a Temporal, Graph-Based Language for Querying Collections of Patient Histories

•

•

• •

Sequentially composed queries require that the queries match, in such a way that the end of match of the first query coincides with the beginning of the match of the second query. This construction gives rise to the edges in the query graph. Parallelly composed queries require that the queries match, and in such a way that the start points of the matches coincide and that the end points of the matches coincide. Alternatively composed queries require that at least one of the queries matches. First-occurrence queries merely require that there is a match of the given query and that this is the first such match in the entire history. (This effect can be achieved with a combination of other elements, but the requirement was important enough to warrant support as a simpler formulation.)

Figure 3 - Visualization of a query result Tests 1-3 were informal, with demonstration and explanation provided during the test. In tests 4-6, the subjects were given a leaflet containing instructions and examples, as well as a number of query construction tasks to be performed. During tests 1-6 we made the observation that the learning process was greatly enhanced by the availability of examples, demonstration or interactive assistance. In particular, getting feedback on the correctness of one’s queries improved ability to accomplish further tasks. We suspected that these factors could obscure issues related to our stated goal, which was to investigate the intuitive understandability of the underlying principles of the query system. Therefore, we devised a testing framework with the following rules: 1) The user will receive written instructions and query construction tasks to solve within an allotted time, 2) There must be no demonstration or assistance and 3) The instructions must contain no example queries. We carried out tests 7-12 in compliance with these rules. The subject was left alone for 105 minutes to read the instructions and attempt to solve the tasks, occasionally prompted via a loudspeaker to explain his or her thinking. Of the 25 tasks, the first 5 were point queries, the next 8 also involved intervals, the next 4 added branching constructs, and the final 8 required complex combinations of different types of elements, In the final 15 minutes, a developer interviewed the subject about the tasks that the test subject had failed to solve, and explained how those tasks should have been solved. Screen, video and audio were captured for further investigation.

The grammar gives the query graph an underlying tree structure where the matches of a node depends on how the matches of its children cohere with the rules and parameters for the node itself. The matches of a query graph are the matches of its root node. Execution of a query is performed with the leaf nodes scanning the history sequentially and the internal nodes iterating through the matches from their children. The language elements described above straightforwardly satisfy requirements 1-10. Intersection and union of query results can easily be achieved by parallel and alternative composition of the queries, thus satisfying requirement 11. The recursive definition of the query language means that parts of queries, down to primitive components, are queries in their own right that can easily be collapsed, given names, saved and re-used, thus satisfying requirement 12. Result visualization Practice Explorer consists of two main windows. The query builder was described in the previous section. The history explorer, is shown in figure 3. It displays a number of vertically stacked horizontal bars, each providing a compact, very simple, LifeLines-like [7], explorable visualization of a history above a common time axis. The history explorer dynamically limits its view to the histories containing matches for the query given by the current state of the query builder, marks the hits of the query with red boxes and synchronizes the histories so that they are aligned on the first match.

Results

Understandability testing

Figure 4 summarizes the correctness scores of test subjects 7-12 on the 25 query construction tasks. For an query to be classified as correct, it had to give the exact intended result.

We performed 12 two-hour understandability tests, each followed by a brief questionnaire. The goal was not to test general usability, but to investigate the intuitive understandability of the underlying principles. We therefore simplified the query builder, keeping only the following query elements: encounter, history start, history end, date control, age control, time window, time window without ..., sequential, parallel and alternative composition. We applied the system to 2066 general practice patient histories of lengths up to 12 years. The test subjects were 4thand 5th-year medical students.

From studying the users' actions on the screen, listening to them thinking aloud, and interviewing them afterwards, we made the following observations: 1. When a time window was specified as having a duration between an upper and a lower limit, for example 0 and 12 months, test subjects would often think of this as a fixed-length window from relative time point 0 to

405

O. Edsberg et al. / Design and Evaluation of a Temporal, Graph-Based Language for Querying Collections of Patient Histories

2.

3.

4.

5.

relative time point 12 months, even though the instructions explicitly stated otherwise. The test subjects made many errors where they seemed to assume that the query described the whole history and not just a fragment of it. Test subjects frequently and successfully used the match-aligned result visualization to check if their queries were correct. The test subjects made many errors related to not understanding that matches of parallel queries must cover the same time period. On tasks requiring nested branching constructs, test subjects seemed to strain under the mental effort required. Some ceased serious efforts to find a solution.

4. Attention must be paid to make the visualization of branching constructs reflect their properties and reduce the effort required in reasoning about them. 5. Refraining from giving examples, demonstration, assistance or feedback may make testing more effective in uncovering problems. Another question, partly addressed by our requirements collection, is whether the language is sufficiently expressive. On this topic we also note that, out of Allen's 13 primitive interval relations [8], our query language does not support overlaps or overlapped-by. The other 11 can be constructed with parallel composition and time windows. Supporting overlaps/overlapped-by would probably make the language more complex. Since we can match the start or end of a medication interval occurring during the match of another interval, we have not yet seen any query need requiring overlaps/overlapped-by.

Most of these observations either did not occur before tests 7-12, or occurred much more strongly in tests 7-12.

Conclusion We have designed, implemented and evaluated a temporal, graph-based query language based on our collected tentative list of requirements. Our understandability tests indicate that domain users relatively easily can construct point and interval queries, but not branching queries. Based on observations done during testing, we arrived at a list of lessons of potential relevance for the design of similar systems. Acknowledgments The authors wish to thank Arild Faxvaag, Anders Grimsmo, Joe Siri Ekgren, Berit Brattheim, Dag Svanæs, Yngve Dahl and Terje Røsand for valuable assistance and comments. Part of the work was funded by The Norwegian Research Council.

Figure 4: Scores for test subjects 7-12 on the 25 tasks. On the questionnaire, which used a 5-point Lickert scale, 3 out of 12 marked “agree” or “fully agree” on the proposition “I think the visualization and building of the queries was easy to understand based on the instruction”. 9 out of 12 marked “agree” or “fully agree” on the proposition “I think the visualization and building of the queries was easy to understand based on the explanation given afterwards.”

References [1] Quaglini S, Ciccarese P, Micieli G, Cavallini A. Noncompliance with guidelines: motivations and consequences in a case study. Stud Health Technol Inform. 2004;101:75-87. [2] Fails JA, Karlson A, Shahamat L, Shneiderman B. A Visual Interface for Multivariate Temporal Data: Finding Patterns of Events across Multiple Histories. IEEE Symposium on Visual Analytics in Science and Technology 2006. [3] Hibino S, Rundensteiner EA. A visual multimedia query language for temporal analysis of video data. MultiMedia Database Systems, pages 123--159. Kluwer Aca, 1996. [4] Chittaro L, Combi C. Visualizing queries on databases of temporal histories: new metaphors and their evaluation. Data & Knowledge Engineering 2003;44:239-264. [5] Fowler M. UML Distilled, 3rd edition. Boston: Pearson, 2004. [6] Heer J, Card SK, Landay JA. prefuse: a toolkit for interactive information visualization. Proceedings of CHI’05, ACM Press, 2005. [7] Plaisant C, Milash B, Rose A, Widoff S, Shneiderman B. Lifelines: Visualizing Personal Histories. Proceedings of CHI’96. ACM Press, 1996. [8] Allen J. Maintaining knowledge about temporal intervals. Communications of the ACM 1983:26(11):832-843.

Discussion Studying figure 4, it appears that test subjects 7-12, even without examples, demonstration or assistance, arrived at a reasonably good understanding of the linear parts of the query language. The branching constructs appear to have been poorly understood. The survey answers indicate that explanation did help a lot on understanding. From the observations, we arrive at some lessons that may be helpful when designing this kind of system: 1. It seems much more natural for users to interpret a minimum and maximum number of time units as time points defining a constant-length interval rather than as bounds on the duration of a flexible interval. 2. Users may find it more natural to build a query describing the whole history rather than just a fragment. 3. Match-aligned result visualization can help users correct their own thinking and build correct queries.

Address for correspondence Ole Edsberg, [email protected]

406

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Evaluation of a Simple Method for the Automatic Assignment of MeSH Descriptors to Health Resources in a French Online Catalogue Aurélie Névéola, Suzanne Pereirab, c, d, Gaetan Kerdelhuéb, Badisse Dahamnab, Michel Joubert d, Stéfan J. Darmonib, c a

U.S. National Library of Medicine, National Institutes of Health, Bethesda, USA b CISMeF, Resarch Department, Rouen University Hospital, France c GCSIS, LITIS EA 4051, Institute of Biomedical Research, University of Rouen, France d LERTIM, Marseille Medical University, France

automatic indexing tools was deemed necessary to reduce the indexing back-log of about 7,000 resources. Considering the limitations of automatic indexing methods, it is necessary to distinguish clearly which resources in the catalogue are indexed manually and automatically [3]. In fact, the question arises at the time resources are considered for inclusion in the catalogue: the curation policy must define which resources should be indexed with the higher quality indexing produced manually and which resources may be given less attention and indexed automatically. At CISMeF, the decision was mainly based on two criteria: (a) the depth of indexing required and (b) the level of coverage of a given topic. High quality (manual) indexing must be available for all the topics covered in the catalogue, but when a reasonable level of coverage has been reached (about a dozen resources according to the curator) additional resources may be indexed automatically. Besides, we assume that automatic indexing is more suitable to cover only the central main concepts discussed in a resource whereas manual indexing will be necessary for in-depth indexing. In any case, in answer to an information query, manually indexed resources will always be displayed before automatically indexed resources. Moreover, the type of indexing (manual vs. automatic) will be shown to the user.

Abstract Background: The growing number of resources to be indexed in the catalogue of online health resources in French (CISMeF) calls for curating strategies involving automatic indexing tools while maintaining the catalogue’s high indexing quality standards. Objective: To develop a simple automatic tool that retrieves MeSH descriptors from documents titles. Methods: In parallel to research on advanced indexing methods, a bag-of-words tool was developed for timely inclusion in CISMeF’s maintenance system. An evaluation was carried out on a corpus of 99 documents. The indexing sets retrieved by the automatic tool were compared to manual indexing based on the title and on the full text of resources. Results: 58% of the major main headings were retrieved by the bag-of-words algorithm and the precision on main heading retrieval was 69%. Conclusion: Bag-of-words indexing has effectively been used on selected resources to be included in CISMeF since August 2006. Meanwhile, on going work aims at improving the current version of the tool. Keywords: abstracting and indexing/methods, algorithms, catalogs, library, information storage and retrieval/methods, evaluation study, France, medical subject headings, Natural Language Processing

Teaching material (N=3629) and clinical guidelines (N=2978) are two vast categories of resources indexed in CISMeF. The average number of descriptors used to index a teaching resource is 9.89 +/- 10.87 vs. 13.64 +/- 15.69 for a clinical guideline. The difference is significant according to a Student test (p

When the user selects “holosystolic” in “ murmur”, then the portion under “...holosystolic” appears. After data is entered by this template, the narrative form is generated. Storage of data into the EMR database In the EMR database, all kinds of patient data are stored. We call a cluster of information a ‘medical event’, which is stored in one record of the database. The describing object is one of the medical events. The data in XML and narrative text data are stored in each field, respectively. EMR database is not suitable for data analysis, because the procedure for such analysis affects the response of the daily online transaction procedure. Furthermore, the patient data is included in the XML. Thus, in order to

413

Y. Matsumura et al. / Template-based Data Entry for General Description in Medical Records and Data Transfer

templates (3242 concepts) had been produced and stored in the database. The numbers of template in each class are shown in Table 1. In the description of progress notes, the number of templates for “physical findings” is much greater than the numbers for “symptom”, “assessment” or “plan”. Not only doctors, but also nurses, pharmacists and nutritionists used templates. Templates are frequently used in examination reports, operation reports, and summaries.

search the objective data, all the records have to be checked. To overcome this problem, the data in EMR is transferred to the DWH. In this process, the XML data is parsed to extract patient data, which is stored in the database of DWH (Fig. 2). The structure of the database of DWH is quite simple. One record is made to correspond to one value. The main fields of this database file are as follows: patient ID, date, template name, xPath name, xPath code, value code, value name, suffix. The xPath code and the xPath name indicate the traced items and values in the tree structure. The xPath code (or the xPath name) is expressed by the traced items and values codes (or names) connected with “\”, e.g. \A01\V0101\A02.

Table 1 – The classification of templates Classification of Template

No. of Concepts

No. of Template

37

46

Physical finding

399

494

Assessment

157

168

24

26

Disease relating description

618

766

Patient history

184

211

62

64

267

313

Operation report

55

62

Medical checkup

79

92

134

157

23

23

956

1013

Description by pharmacists

12

12

Description by nutritionists

44

44

other

191

234

Total

3242

3725

Symptom

Plan

Summary Examination report

Figure 2 - Flow of template master and patient’s data Search assisting system We developed an application system for searching the objective data from this database of DWH. In this system, users easily designate the objective data using a GUI. When users select an objective template, the system shows the names of items and values in the template. Thus, users can easily set the combination of search conditions in one template. The character string entered in the text box can also be the object of a search. In the case of numerical values, the conditions “more than” or “less than” can be set. The application system makes a SQL request according to the user’s designation and searches the objective data from the DWH database. When the system finds the description that meets the condition, a progress note including the description is shown. Additionally, the system shows the progress notes of other days for the same patient.

Description for rehabilitation Description by social worker Description by nurses

Results

We examined 100 templates randomly selected from the database. When we count the same item linked with multiple values as 1, the average number of items in one template is 34.8. The average number of layers in one template is 8.1. We examined the maximum depth of layers in one template, and found 1 layer in 17%, 2 layers in 43%, 3 layers in 24%, 4 layers in 25%, and 6 layers in 1% of templates. The total number of items in the database is estimated as about 112,800.

Availability of template Preparation of input templates that are useful and convenient for every member of the hospital staff is essential before a hospital initiates an EMR system. The task is so large for each hospital that we formed an organization to support the preparation of template contents and their supply. The template contents that are developed for hospital use are then stored in the database, from which the staff of the hospital can select useful ones for their practice. The dynamic template module is used in 35 hospitals as of October 2006. Different styles of templates were sometimes produced for a concept. As of that time, 3725

Data warehouse and search assisting system The DWH and search assisting system were implemented in Osaka University Hospital, and their usefulness was

414

Y. Matsumura et al. / Template-based Data Entry for General Description in Medical Records and Data Transfer

For general descriptions in medical records, many types of template have to be prepared. To bring this about, trial and error is inevitable, even after active use in daily practice [21]. Thus, it is necessary to have a system that enables users to produce templates easily and revise them even after releasing. Generally a template is made based on its own database file [14,15], which must be set on each occasion to produce a new template. If the template is revised, the corresponding database file must also be revised. Because this method disperses a patient data in many database files, it is not suitable for EMR system, which must enable users to refer quickly to any patient data.

evaluated. Without the search assisting system, users have to examine the xPath code of the value they want to search beforehand. Furthermore, it is quite difficult to write SQL for a combination of search conditions in one template. Contrasted with this, users can now search the objective data easily by this searching system.

Discussion General descriptions in medical records, such as progress notes, examination reports, operation reports and summaries, are so diverse and complicated that these data are generally entered as free text in EMR. In order to use these data for research, clinical evaluation and so on, natural language processing is one of the possible methods [17,18]. However, to achieve good results by this method, all of the words in the entire medical field, including abbreviations and frequent typing errors, have to be entered into this system beforehand. This would entail tremendous amount of work.

In our method, the template content master that regulates the content of a template and the patient’s data are simultaneously expressed in XML. After entering patient data by the template, the XML data and the narrative form of patient data are stored in the EMR database. Although this schema is practical for EMR systems, data analysis is virtually impossible. Thus, we developed a parser system that retrieves the data in XML from the EMR database and extracts patient data from it. The parser system then stores the patient data in the DWH database, in which one record corresponds to one value. Because patient data is originally in tree structure, xPath code and name is used to designate each property. This database schema is independent from the template contents; therefore, the user can produce and revise templates freely without thinking of the database schema.

The strategy of template-based data entry is a practical method from the viewpoint of data analysis [12]. The simple templates that are generally adopted, however, limit what users wish to express, and users have no choice but to tolerate these limitations. Descriptions in medical records are the observation records of patients. When the observation object is normal, the user just enters “nothing particular”. However, when something abnormal is found, they describe the object in detail. For the observation records, a tree structure is suitable: an abnormal object is described by several features, which may be further described by other properties [19,20]. Thus, a tree structure is necessary for the template. If the structured data is expressed as it is in EMR, it is not acceptable for users because it is quite different from the expression in free text. A person prefers an expression in which obviously understandable words are omitted, rather than a strictly redundant expression. To overcome this problem, we convert the structured data into narrative form [16]. By using this template, users can enter what they want to record in shorter time without the risk of typing errors. This strategy is quite acceptable for users.

Using this DWH, data entered by the template can be used for analysis. It is, however, difficult for users to find the xPath code of the objective data. Furthermore, in this database schema, it is difficult to execute combined search conditions in one template. Accordingly, we developed a search assisting system which assists in finding the xPath code in the template, and thus enables the user to easily set combinations of search conditions. Although this database schema is not the best for data analysis, the user can search the objective data from the database with the support of the search assisting system. By using this method, general descriptions in medical records can be structured and analyzed. Although it has been said that structured data entry is essential for advanced functioning in EMR, there are a few reports that it succeeded when limited to a few a fields [14,15,22] or a few department [23,24]. We achieved active use of the structured data entry system in every department in many hospitals.

The dynamic template is implemented in the EMR system produced by NEC and 35 hospitals actively use this module. Before starting to use EMR system, hospital staff prepared templates useful for their daily practice. During operating the EMR, templates were revised and new ones were added according to the users’ requirement. More than 3725 templates have been produced and more than 100,000 items were entered in the template master. More templates about physical findings or examination reports have been produced than those about symptoms. Template-based data entry is suitable for actively acquired data. On the other hand, it is not suitable for passively acquired data such as symptoms. The concept, which has many options (e.g. portion of skin in dermatology), is difficult to handle with templates. Although the dynamic template is not effective for every type of description in medical records, its popularity and practical accomplishments show that the strategy of dynamic templates, i.e. tree-structured template and conversion of structured data into narrative form, is acceptable in many fields.

Acknowledgments This work was supported by a grant-in-aid for scientific research from the Ministry of Education, Culture, Sports, Science and Technology of Japan (7) to Y.M. (15500341)

References [1] Delpierre C, Cuzin L, Fillaux J, Alvarez M, Massip P, Lang T. A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? Int J Qual Health Care. 2004 Oct: 16(5):407-16. [2] Matsumura Y, Kuwata S, Kusuoka H, Takahashi Y, Onishi H, Kawamoto T, Takeda H. Dynamic viewer of medical events in electronic medical record. Medinfo2001: 648-52.

415

Y. Matsumura et al. / Template-based Data Entry for General Description in Medical Records and Data Transfer [3] Hunt DL, Hayness RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. Jama 1998: 280:1339-46. [4] Van Ginneken AM. The computerized patient record: balancing effort and benefit. Int J Med Info 2002: 65: 97-119. [5] Bates DW, Pappipus E, Kuperman GJ, et al. Using information systems to measure and improve quality. Int J Med Infom. 1999: 53(2-3): 115-24. [6] Wisniewski MF, Lieszkowski P, Zagorski BM et al. Development of a clinical data warehouse for hospital infection control. J Am Med Inform Assoc. 2003: 10(5): 454-62. [7] Grant A, Moshyk A, Diab H, et al. Integrating feedback from a clinical data warehouse into practice organization. Int J Med Inform. 2006: 75(3-4): 232-9. [8] Zhang Q, Matsumura Y, Teratani T, Yoshimoto S, Mineno T, Nakagawa K, Nagahama M, Kuwata S, Takeda H. Evaluation of the observed cisplatin nephrotoxicity in adult cancer inpatients: a historical cohort study by using clinical data warehouse. Journal of Health Science 2006: 52(2) 192-7. [9] Lee FC, Chong WF, Chong P, Ooi SB. The emergency medicine department system: a study of the effects of computerization on the quality of medical records. Eur J Emerg Med. 20018(2):107-15. [10] Henry SB, Douglas K, Galzagorry G, Lahey A, Holzemer WL. A template-based approach to support utilization of clinical practice guidelines within an electronic health record. J Am Med Inform Assoc. 1998: 5(3): 237-44. [11] Los RK, van Ginneken AM, van der Lei J. OpenSDE: a strategy for expressive and flexible structured data entry. Int J Med Inform. 2005: 74(6): 481-90. [12] Rose EA, Deshikachar AM, Schwartz KL, Severson RK. Use of a template to improve documentation and coding. Fam Med. 2001: 33(7): 516-21. [13] Matsumura Y, Takeda H, Okada T, Kuwata S, Nakazawa H, Hazumi N, Inoue M. Devices for structured data entry in electronic patient record. Medinfo. 1998: 9 Pt 1:85-8. [14] Green JD Jr, Postma DS, Giddings NA, Sapp KR, Skinner T. Computerized medical record in a private neurotology practice. Am J Otol. 2000: 21(4):589-94.

[15] McCullagh PJ, McGuigan J, Fegan M, Lowe-Strong A. Structure data entry using graphical input: recording symptoms for multiple sclerosis. Stud Health Technol Inform. 2003;95:673-8. [16] Matsumura Y, Kuwata S, Okada Y, Emichi H, Kawamoto T, Shanmei J, Chen Y, Takeda H. Method of transforming of structured data into narrative form. Medinfo 2004;1746. [17] Meystre S, Haug PJ. Natural language processing to extract medical problems from electronic clinicaldocuments: performance evaluation. J Biomed Inform. 2006: 39(6): 589-99. [18] Hyun S, Bakken S, Friedman C, Johnson SB. Natural language processing challenges in HIV/AIDS clinic notes. AMIA Annu Symp Proc. 2003: 872. [19] Hanzlicek P, Spidlen J, Nagy M. Universal electronic health record MUDR. Stud Health Technol Inform. 2004: 105: 190-201. [20] van Ginneken AM. Considerations for the representation of meta-data for the support of structured data entry. Methods Inf Med. 2003: 42(3): 226-35. [21] Hobson JC, Khemani S, Singh A. Prospective audit of the quality of ENT emergency clinic notes before and after introduction of a computerized template. J Laryngol Otol. 2005: 119(4): 264-6. [22] Sistrom CL, Honeyman JC, Mancuso A, Quisling RG. Managing predefined templates and macros for a departmental speech recognition system using common software. J Digit Imaging. 2001: 14(3):131-41. [23] Bleeker SE, Derksen-Lubsen G, van Ginneken AM, van der Lei J, Moll HA. Structured data entry for narrative data in a broad specialty: patient history and physical examination in pediatrics. BMC Med Inform Decis Mak. 2006: 136:29. [24] Los RK, van Ginneken AM, van der Lei J. OpenSDE: a strategy for expressive and flexible structured data entry. Int J Med Inform. 2005: 74(6): 481-90.

Address for correspondence Yasushi Matsumura MD, PhD. Department of Medical Informatics, Osaka University Graduate School of Medicine. 2-15 Yamada-oka, Suita, Osaka 565-0871, Japan Tel: (+81)6-6879-5900, Fax: (+81)6-6879-5903 E-mail: [email protected]

416

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Challenges and Methodology for Indexing the Computerized Patient Record Frédéric Ehrlera,b, Patrick Ruchb , Antoine Geissbuhlerb, Christian Lovisb b

a Artificial Intelligence laboratory, University of Geneva, Geneva Switzerland Service of Medical Informatics, University Hospitals of Geneva, Geneva Switzerland

the documents are stored in a unique and large corpus, we have to work with numerous small corpora that must be indexed independently.

Abstract Patient records contain most crucial documents for managing the treatments and healthcare of patients in the hospital. Retrieving information from these records in an easy, quick and safe way helps care providers to save time and find important facts about their patient’s health. This paper presents the scalability issues induced by the indexing and the retrieval of the information contained in the patient records. For this study, EasyIR, an information retrieval tool performing full text queries and retrieving the related documents has been used. An evaluation of the performance reveals that the indexing process suffers from overhead consequence of the particular structure of the patient records. Most IR tools are designed to manage very large numbers of documents in a single index whereas in our hypothesis, one index per record, which usually implies few documents, has been imposed. As the number of modifications and creations of patient records are significant in a day, using a specialized and efficient indexation tool is required

Two key measures have been used to evaluate performance, efficiency and effectiveness. As no gold standard was available, an ad-hoc method of automatic query generation has been used to generate the required data. The structure of the dataset has been modified to analyze the consequence of the patient records corpus structure on the indexing process.

Background Patient records in Geneva University Hospital The Geneva University Hospitals (HUG) is a consortium of hospitals in four campuses and more than 30 ambulatory facilities in the state. The HUG have about 2,000 beds, 5,000 care providers, over 45,000 admissions and 850,000 outpatients’ visits each year. Over than 20,000 computerized records are open daily. More than 4,000 care providers do use this system, including physicians, nurses, medical secretaries, social care providers, physiotherapists, nutritionists, music therapists, etc. Beside order entry, clinical documentation is an important pillar of the CPR. More than 50,000 images, 25,000 lab results, 8,500 documents are stored every day. The clinical database contains more than 130 millions patient facts. All data is available online, there are no archives.

Keywords: information retrieval, patient records

Introduction The ideal patient record should be unique and persistent over time, consolidated within a distributed computerized patient record (CPR). Whatever its technical, architectural and conceptual organization, accesses should only be constrained by access policies. However, not all the information contained in the records is relevant for each encounter. Care providers waste precious time at searching and browsing the patient record to collect all information pertinent to the actual situation. The problem is emphasized when specific information is scattered in numerous documents. Therefore, creating an efficient tool that allows retrieving relevant information from patient documents should improve the efficiency of care providers.

Information retrieval The purpose of a "document retrieval system" is to select, from a relatively large collection of documents, a manageable number of documents that is likely to satisfy an expressed need for information [1]. It deals with the representation, storage, organization and access to information items. To accomplish the retrieval task, the system needs to build, usually prior to queries, a representation of each document in the collection. This representation consists of a table containing links that allow knowing which terms occur in the documents with which frequency [8]. During the search process, the system typically computes the degree of match between the terms contained in the index and a corresponding set of terms derived from the query [6]. This degree of match based on the frequency of the words in the document itself (TF) and in the corpus

This paper presents the methodological approach for using an Information Retrieval (IR) tool in order to index the patient records and allowing fast retrieval. The specificity of our approach arises from the specific structure of the patient records. Contrarily to common IR task where all

417

F. Ehrler et al. / Challenges and Methodology for Indexing the Computerized Patient Record

(IDF) provides the basis for deciding whether a document should or not be retrieved. IR evaluation Two very different aspects of an IR system can be measured: efficiency and effectiveness [2]. Efficiency can be measured in terms of the resources required by the system for the whole IR process, including storage space and computing resources. Effectiveness attempts to measure the success of an IR system. It should provide a domain neutral measure of the ability of the system to retrieve all relevant documents [5]. The most widespread method of evaluating effectiveness of an IR system involves providing precision–recall values for a set of queries posed on a specific documents collection [4]. Recall measures the proportion of relevant documents retrieved and precision measures the proportion of retrieved documents which are relevant.

Figure 1 - Distribution of the records size in the patient records corpus In regard to this distribution, we can stress two issues. First, as most of the records contain few documents (50% of the records have less than 7 documents), there will be little interest in using a tool not returning the exact answer. The second concern is related to records containing numerous documents. In this situation, constraint applied on small record is relaxed as returning a small subset of documents containing the answer could be sufficient. Even if the first answer is not the good one, reducing the possible research space brings a significant gain of time for the care providers.

Test collection The final application of our system is dedicated to index the patient records; however, experiments done in this paper are performed on OHSUMED dataset to facilitate measures and reliability. In this paper, two different kinds of corpus are considered, the source and the experimental corpus. The source corpus denotes the collection from which all the documents supporting the experiments are extracted. The experimental corpus refers also a collection of documents but this collection possesses a special structure. Indeed, in the experimental corpus, the entire documents collection is split into several groups. One group can therefore be assimilated to a given patient record. Note that the total number of documents is independent from the number of groups.

OHSUMED The choice of an alternate dataset has been focused on OHSUMED. OHSUMED is a bibliographical document collection, developed by William Hersh and Colleagues at the Oregon Health Science University. It is a subset of the Medline database consisting of 348,566 references from 270 medical journals from the years 1887 to 1991. All of the references have titles, but only 233,445 have abstracts [7].

The choice of OHSUMED as the source corpus has been motivated by the easiness of manipulation of the documents for the creation of the different datasets required for our experiments. This corpus is homogenous, meaning that its documents share similar properties like length, word distribution, and word frequency. This homogeneity avoids biasing the experiments and allows focusing only on the quality of retrieval. Additionally, using a standardized corpus accessible to other groups fosters comparisons of alternative methodologies.

We refer to the title plus abstract as a document. Comparison OHSUMED - Patient record In order to study the similarity of the two corpuses, we selected 8,192 documents from each corpus and compared the frequency of the words of the subsets. Our comparison focused on the frequency distribution of the words contained in the two corpuses as it is the most influential feature on the IR process. Indeed, the observed variation on the inverted index size induced by the modification of the number of patient records is strongly influenced by the distribution of the words in the corpus. While increasing the number of patient records indexed independently, the low frequency words have a different effect on the inverted index size compared to the high frequency words. The former will not be retrieved in independent index whereas the latter will be part of several ones.

As the OHSUMED corpus and the patient records corpus share important common properties regarding the IR process, we believe that we will be able to predict the behavior of our system on the patient records by looking at the results obtained on the OHSUMED corpus. Patient records The documents used for the study covers all aspects of clinical documentation, such as admission notes, discharge letters, activity reports, nurses’ data entries, consultations and interventions reports. Over 1,500 different categories of documents are used.

We notice in Figure 2 that OHSUMED and the patient record corpora are mainly similar in number of words per document. However, patient record corpus contains less unique words than the OSUHUMED. It is certainly due to

418

F. Ehrler et al. / Challenges and Methodology for Indexing the Computerized Patient Record

real life situation where they are never exactly similar to the content of the text. Moreover using randomness prevents to match too easily the queries in the initial text.

Methods Indexing and retrieval strategies The indexing of the documents is performed with EasyIR, an existing tool [9-10]. This tool has shown good performance on indexing and retrieval in the genetic and biomedical field and should be suitable to be applied on medical terminology. The specificity of the application arises from the necessity of processing the generated data in real time. In our situation, numerous documents are daily added in the patient records and queries can be performed immediately after. Given these constraints various indexation triggering strategies have been suggested.

Figure 2 - Frequency of the words contained in the corpus for 8,192 documents the fact that an important part of the OHSUMED corpus is related to genetics. As there is little consensus on naming genes and proteins, different spelling and acronyms are used and artificially increase word diversity. Beside these the patient records cover mainly the same subjects as the ones addressed in OHSUMED.

a) Indexation launched when a query is performed b) Indexation performed at fixed, but short, intervals c) Indexation triggered by a notification when a document is saved Indexation at fixed time has been dismissed, as it would lead to a lot of useless processing without ensuring proper indexation when needed. Indexation at query time could be interesting, as it requires less indexation process. However, as it inducing a delayed answer when queries are performed, it has a severe impact on perceived performance. Consequently, the last strategy, notification-based triggering appears to be the best solution.

Test collection construction The test collection is composed of three parts, the collection of documents, the queries and the relevance assessments. The test documents selected from the OHSUMED corpus are structured in order to mimic the existing structure of the patient records corpus. We vary the number of groups while keeping a constant number of documents in order to study the impact of the corpus structure on the performance [3]. As the number of documents is fixed, each increase in groups is accompanied by a proportional decrease in the number of documents per group.

Evaluation metrics The most basic evaluation of a “known-item search” task, consists in looking at the rank at which the target documents are retrieved. This measure called the “meanreciprocal-rank” is the mean of the reciprocal of the rank at which the known item was found, averaged over all the queries, and using 0 as the reciprocal for queries that did not retrieve the known document.

Queries and relevance assessments are dependent of the field of the application. Usually, experts are required to construct relevant and interesting queries for specific domains. We avoid this requirement by automatically generating the queries and their related relevance assessments by considering “known-item-search”. The goal in a “known-item search” is to retrieve just a unique document that has been used to build the query. It simulates a user seeking for a particular, partially remembered document in the collection by using a part of a document as a query

Another evaluation measure used is the recall at first retrieved document. It consists in computing the recall of the system when the first returned answer of the system is returned. In our application, recall at first retrieved document is a very useful method to evaluate the utility of our tool. When care providers perform searches in patient records containing few documents, they expect to obtain directly the correct document; therefore the tool must have a high recall at first retrieved document to be really useful.

The care providers behave quite similarly as the way “known-item-search” works. They have a precise idea of what they search and are often interested in only one particular document that contains the relevant information. Given these similarities, the effectiveness obtained using “known-item-search” is well suited to estimate the usefulness of the tool in a real situation.

Trial Indexing and retrieval - efficiency The objective is to measure the time cost induced by an increasing number of groups given a fix number of documents. The number of groups grows according to an exponential scale with a parallel decrease in the number of documents in each group.

The queries are generated by selecting random parts of the documents. Building queries using randomness has the advantage of creating more complex queries than the ones built selecting simply a subpart of the text. Queries built through this process reflect quite properly the ones used in

419

F. Ehrler et al. / Challenges and Methodology for Indexing the Computerized Patient Record

Figure 3 – efficiency given an increasing number of groups for 8,192 documents

Figure 4 - Total number of entries in the database given the number of groups for 8,192 documents

The measures have been performed using the 8,192 documents with the number of groups varying from 1 to 2,048.

number of group will not bring any further increase in size of the total index.

The results show a linear increase in time required for indexation given the number of groups (Figure 3). As the total number of document does not change, this increase in time can be attributed to the indexation initialization overhead.

In order to analyze the consequences of the size variation of the total entry table on the program overhead, we decomposed the indexation process in three main tasks. The first step consists in extracting all the words from the documents using tokenization in order to build the vocabulary. This does not lead to significant overhead variation according to the number of groups, as the complexity of this task is only dependent from the total number of words in the whole corpus.

The time required for retrieval follow a totally inverted tendency in comparison to the indexing process. During retrieval process, as the numbers of queries are constant, it is faster to perform queries on small indexes (Numerous groups). Indeed, if we perform 1,024 queries on an index containing 8,192 documents, it will last longer than performing these 1,024 queries on 1,024 different indexes of 8 documents.

After word tokenization, extracting the term frequency and inverse document frequency values of every term is required to compute the weights. As the inverse document frequency is dependant of the documents contain in every group, this computation is not done once per word but once per entry. Therefore, this process brings clearly an overhead.

Explanations of the indexing overhead In this paper, the total index of an experimental corpus refers to the merge of all the individual entries of the index table of each group that compose the corpus. As during the indexing process every group of documents is indexed separately, a separated index table is generated for each group and therefore the total index will contain redundant entries. Indeed, if an entry belongs to two individual indexes it will be present twice in the total index. As the total index contains all the entries which have to be stored on the database, its size is a good estimator of the space required in memory.

The last step consists in storing the indexes in the database. Obviously, there is an overhead linked to the number of groups for this task. As the size of the total index is bigger with a large number of groups, the necessary time to store the total index in the database is bigger too. Retrieval - effectiveness Effectiveness evaluation has been done with the mean reciprocal rank and also with recall at first retrieved document.

As shown in the Figure 4 the number of created entries grows in a logarithmic manner regarding to the number of groups. This is the consequence of the distribution of the frequencies of the words in the corpus that follows a logarithmic decrease (there are many infrequent words and few very frequent words). A word can not occur in a larger number of groups than its document frequency. Therefore, the closer we approach this threshold, the lower the probability of this word occur in an additional separated group and increase the size of the total index. Once a word occurs in a number of groups equal to its frequency, increasing the

420

F. Ehrler et al. / Challenges and Methodology for Indexing the Computerized Patient Record

Table 1 - Recall at top returned documents for different size of experimental corpus given 1,024 queries Number of document per group

answers requirements for real-time and sensitivity. As performing information indexing in patient records is timeconsuming, the finest tuning possible should be done in order to increase the efficiency.

recall at first retrieved document

MRR

8

96%

0.964

16

95%

0.961

32

94%

0.956

64

92%

0.935

128

88%

0.901

256

86%

0.885

Experiments done are sufficient to identify weaknesses and strengths of traditional information retrieval tools and define the improvements needed.

512

82%

0.847

Acknowledgements

1,024

80%

0.819

This work has been funded by the Swiss National Science Foundation (SNF 632-066041).

2,048

74%

0.769

References

4,096

73%

0.758

8,192

69%

0.722

[1] R. Bazera-Yates and B. Ribeiro-Neto. Modern Information retireval. New York: McGraw-Hill, 1983. [2] D. C. Blair and M. E. Maron. An evaluation of retrieval effectiveness for a full-text document-retrieval system. Communications of the ACM 28 (3): 289-299, 1985. [3] S. Buttcher and C. Clarke. Efficiency vs. Effectiveness in Terabyte-Scale Information Retrieval. Text REtrieval Conference (TREC) 2005, 2005. [4] O. Frieder, D. A. Grossman, A. Chowdhury and G. Frieder. Efficiency considerations for scalable information retrieval servers. Journal of Digital information 1 (5), 2000. [5] D. Grossman and O. Frieder. Information Retrieval: Algorithms and Heuristics: Kluwer Academic Press, 1998. [6] W. Hersh, C. Buckley, T. Leone and D. Hickam. OHSUMED: an interactive retrieval evaluation and new large test collection for research. ACM SIGIR ’94: 192-201, 1994. [7] M. Martynov and B. Novikov. An Indexing Algorithm for Text Retrieval. Third International Workshop on Advances in Databases and Information Systems - {ADBIS'96} 171175, 1996. [8] P. Ruch. Automatic Assignment of Biomedical Categories: Toward a Generic Approach. Bioinformatics 22: 658-664, 2006. [9] P. Ruch, C. Chichester, G. Cohen, F. Ehrler, P. Fabry, M. J, H. Muller and A. Geissbuhler. Report on the TREC 2004 Experiment: Genomics Track. The Thirteenth Text Retrieval Conference, TREC-2004, 2004. [10] P. Ruch, A. Jimeno Yepes, F. Ehrler, J. Gobeill and I. Tbahriti. Report on the TREC 2006 Experiment: Genomics Track. TREC Notebook paper, 2006.

The increasing time required when the documents are split into several groups has been a surprising result. However by analyzing the process, we have identify the sources of this overhead and are now able to apply the best solution to solve it. On the retrieval side, we show that the tool offers satisfying time response and offer good retrieval effectiveness, at least for small patient records.

The Table 1 sho*ws that the fewer documents contained in groups, the better the recall. It is encouraging to see that the tool is effective with small groups. However, results are less impressive with larger groups. Even if the analysis of the data shows that it is extremely rare to have more than 1,000 documents in a patient record this case must be take in consideration. Mean reciprocal rank results confirm the tendency observed with recall at first returned document measure: the top returned documents of small groups contain most of the time relevant results whereas queries in larger groups must be improved. Situation with patient records Given the structure of the patient records corpus one can expect that a significant computational power will be required to offer acceptable efficiency. Indeed, experiments show that when the structure is similar to the one of the patient record the indexing time suffer of a consequent overhead. However given this structure there will be no problem for the retrieval as groups containing few documents allow very quick answer For effectiveness the 3% of the patient records containing more than 100 documents will be problematic, however this problem should be marginal, at least in the beginning.

Address for correspondence Frederic Ehrler, Centre Universitaire d’informatique, Rue du Général Dufour 24, 1211 Geneva 4, Switzerland. E-mail: [email protected], TN: +41223797643

Conclusion Indexing patient records is an unusual information retrieval task as it implies to work with numerous corpuses of relatively small size. Working with such data has revealed weaknesses of systems initially dedicated to manage large corpuses. Highly dedicated tools are needed to

421

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Experiments with Hierarchical Concept-Based Search Robert Moskovitcha, Roee Sa'adona, Eytan Behirib, Susana Martinsc, Aviram Weissd, and Yuval Shahara a

Medical Informatics Research Center, Ben Gurion University, Beer Sheva, Israel b E&C Medical Intelligence Inc., 41 Madison Ave, New York, NY 10010 c Medical Corps, Israel Defense Forces, Israel d Stanford University, Stanford, VA Palo Alto Health Care Center, Palo Alto, CA, USA

studies was made by Hersh, in which an attempt to adopt Salton's tfidf approach to the conceptual resulted unsuccessfully [4]. Recently we presented Vaidurya, a concept based and context sensitive search engine, developed originally, within the Digital Electronic GuidEline Library (DeGeL) [5], to search for textual and marked up clinical practice guidelines, however, we extended it recently to handle general clinical documents. A detailed description of Vaidurya is provided in [6], as well as an extensive and rigorous evaluation, in which a small portion of this study results appear, however, in this study a wider evaluation is provided, in which both concepts based logic operators are evaluated as we will elaborate later. Vaidurya enables the user to query explicitly for concepts given a logic relation between them. In this study we present a novel hierarchical concept based retrieval method including a wide and detailed evaluation of the approach.

Abstract Many digital libraries use hierarchical indexing schema, such as MeSH to enable concept based search in the retrieval phase. However, improving or outperforming the traditional full text search isn’t trivial. We present an extensive set of experiments using a hierarchical concept based search retrieval method, applied in addition to several baselines, within the Vaidruya search and retrieval framework. Concept Based Search applied in addition to a low baseline is outperforming significantly, especially when queried on concepts in the third level and using disjunction within the hierarchical trees. Keywords: medical text retrieval, concept based search

Introduction

We start with a background review of concept based search (CBS) and MeSH. Then we describe briefly the search methods implemented in Vaidurya. We describe our research hypotheses, the experimental plan, and the results. Eventually we discuss the results and conclude.

Many digital libraries are indexed using a hierarchical conceptual structure; examples include PUBMED, in which documents are classified along the Medical Subject Headings (MeSH) concepts, and the National Guideline Clearinghouse (NGC1) library, each of whose documents is classified using multiple concepts from MeSH and the Unified Medical Language System (UMLS) [1]. Several sites allow browsing through the concepts using the hierarchical structure from the root to the most specific concepts (leaves), which forces the user to navigate the hierarchy. Others enable to query for concepts from MeSH relying on the pre-indexing of the documents along the concepts. Some studies proposed limiting the search to a specific concept (category) [2, 3] and its subconcept contents. In the medical domain, unlike the web, documents are often classified by a multitude of concepts, often as many as a dozen or even tens of concepts, a property which can be further exploited for better retrieval.

Background Concept based search In the medical domain, CBS refers to a text retrieval approach, in which documents are mapped to concepts, representing a meaningful abstract subject, based on its contents. Hersh’s SAPHIRE system [4] uses an approach, in which concepts used for indexing, are automatically extracted from the document. Commonly both documents and queries are mapped, in the case of the biomedical domain into vocabularies such as MeSH and UMLS. However, users are not always familiar with the concepts in these vocabularies, rendering it somewhat limited. It has been previously noted, that the particular implementation of concept-based search described above does not necessarily improve the retrieval performance [7] compared to traditional text retrieval methods. Other studies have tried to exploit the UMLS meta-thesaurus to expand queries, thus extracting the concepts from the query terms [7]. The authors found that query expansion degraded aggregate retrieval performance, but some specific instances of syn-

The NIH had invested huge amount of money during the past decades in building a set of controlled vocabularies and accessory tools to enable the implementation of concept indexing and retrieval. However, no study had shown that using the conceptual structures outperforms or improves the traditional full text search. One of the famous 1

www.ngc.org

422

R. Moskovitch et al. / Experiments with Hierarchical Concept-Based Search

onyms and hierarchy-based query expansion improved individual query performance. Aronson [8] compared his methods to Srinivasan's [9] and showed an improvement by expanding text-based queries with both phrases and concepts from the UMLS Metathesaurus. Neither used the hierarchical relationships of the Metathesaurus, yet reported an improvement over a non-query-expanded baseline. Rada [10] developed an algorithm to estimate the conceptual distance between documents and queries using MeSH and suggested that MeSH can be utilized to improve retrieval performance.

Full text search in Vaidurya Documents are represented and indexed using the vector space model introduced by Salton [12], commonly used in free text search, in which each document is represented by a bag-of-words. After the document terms are extracted, stop-words (e.g., "and", "the", "are") are removed and the left terms are stemmed to their root using porter algorithm,, a vector of terms is created, such that each index in the vector represents a term frequency in the document, known as term-frequency-inverse-document-frequency (TF*IDF). Term frequency (tf) represents the term appearances in the specific document, normalized by the most appeared term (tfmax), while the idf = log(N/n) represents the appearance of the term in the entire documents collection, where N is the size of the collection, and n is the number of documents in which the term appears. The free text retrieval is based on the cosine similarity [12] which measures the distance between a query and a document within the Euclidean space of the terms. More details about the free text retrieval in vaidurya is at [6]. Context sensitive search in Vaidurya To perform a context-sensitive search, Vaidurya assumes the existence of an internal hierarchical structure of the document (i.e., an ontology that defines an internal contextual model). The internal structure enables the user to query for keywords appearing only in specific contexts (i.e., within segments of text labeled only by the specified tag), thus potentially improving the search and retrieval accuracy. Thus a contextual query includes a set of keywords for each queried context element. In the retrieval process each document gets a rank according to the match for each queried context. This study focuses on the concept based search in Vaidurya, however, more details are provided at [6].

Figure 1 - The concept "Asthma" in MeSH, decomposed to tree-numbers, appearing in several locations in different concept trees Medical subject headings The MeSH thesaurus is organized hierarchically and includes 15 concept "trees", consisting of descriptors (concepts). Examples of such trees include Anatomy and Diseases. At the most general level of the hierarchical structure there are very broad concepts (e.g, 'Anatomy', 'Mental Disorders'). More specific headings are found at lower levels of the eleven levels hierarchy (e.g., 'Ankle' and 'Conduct Disorder'). The same concept may appear as multiple tree-numbers within MeSH, thus, in effect, having multiple ancestors. For example, the concept Asthma has four tree-numbers in MeSH, having three different parents (figure 1). Note that intermediate concepts such as Respiratory Hypersensitivity can also appear more than once as different tree numbers, possibly at different levels (figure 1).

Materials and methods Concept based search in Vaidurya In general, our study focused on a very broad class of search methods, which we refer to as double-operator methods. Our assumption is that a conceptual hierarchy is always composed of one or more concept trees, determined by the roots at level 1 of the hierarchy (level 0 being the root concept). For example, the 15 concepts trees in MeSH.

Vaidurya

To perform a concept-based search, Vaidurya includes optional specification of one or more concepts or subconcepts, using the logical operators conjunction (AND) and disjunction (OR), defining the constraints on the desired relations between the queried concepts, explicitly specified by the user. The first, called outer-op, defines the relation among different concept trees; the second, called inner-op, is defined within the same concept tree (Figure 2). Formally, a concept based query Qcb = {[t1,t2,..,tm], outerop> is denoted by a pair, in which the first element specifies a collection of the queried concepts trees, T = {t1, t2,.., tm}, in each one of the m concept trees t1 to tm. Each queried tree i is defined by a set of queried concepts

Vaidurya is a concept based and context sensitive search engine, developed originally, within the Digital Electronic GuidEline Library (DeGeL) [5], to search for textual and marked up clinical practice guidelines, including three types of implemented search methods: (1) full-text search, using standard key terms; (2) context-sensitive search, which exploits the semantic markup performed on guidelines in DeGeL, can be used also for search in a structured document; and (3) concept-based search, consisting on hierarchical concepts indexing structure. The documents can be classified manually or automatically using machine learning based method as we proposed in [11].

423

R. Moskovitch et al. / Experiments with Hierarchical Concept-Based Search

cik, in which i is the tree id and k is the queried concept id, and a local inneri operator AND or OR. The second element defines the outer logic operator. During the retrieval process, first, the documents, classified along each queried concept cij and its descendents, are retrieved. Then, based on the application of inner-op, a set of documents are retrieved for each queried hierarchy, in the case of AND the documents of each concept are intersected and in the case of OR they are unified. Eventually, the application of the outer-op logical operator on the documents retrieved for each tree, intersecting in the case of AND, and unifying in the case of OR, results in the final set of documents retrieved for the CBS having the same rank, which is later integrated with additional types of queries, such as full text or context sensitive search, using a weighted average formula. Four concept based retrieval algorithms were used in this study, defined by the outer and inner logical operators: (1) OR-OR, in which both logic operators set to disjunction. (2), OR-AND, in which the outer is set to disjunction and the inner to conjunction (3) AND-OR, in which the outer operator is set to conjunction and the outer operator is set to disjunction, and (4) AND-AND, in which both logic operators set to conjunction. In this study the inner operator in all the trees were set to the same value.

Figure 2 - The inner-op is located within the queried concepts tree and the outer-op defines the relation between the trees, in the final retrieval stage

Evaluation Test Collection For our detailed evaluation of the relative value of concept-based and context-sensitive search required a test collection in which documents are structured and classified along hierarchical concepts, we have therefore used the NGC CPGs collection. The NGC website is a repository of CPGs classified by MeSH concepts. The CPGs are classified along two tree-like hierarchical concepts, Disorders and Therapies. Each concept tree has roughly 2,000 unique concepts in a tree-like structure; overall, 5407 concepts were used at the time of the evaluation. In several regions, the concept trees were 10 levels deep, but the average depth was around 4-6 levels. There were 1136 CPGs; each CPG might have multiple classifications indexed by both of the concept trees, including indices that belong to the same tree but at different levels (nodes), not necessarily leaves. CPGs have on average 10 classifications per guideline. In order to use the NGC collection for an informationretrieval evaluation, we created a set of information needs,

queries, and corresponding judgments. Several physicians at the Palo Alto Veterans Administration Health Care System, E&C-Medical Intelligence Ltd Company and a physician from the Medical Crops of the Israeli Defense Forces defined 13 information needs (e.g., treatment of hypothyroidism in a particular population subset). Altogether, six physicians participated in the creation of the information needs, queries, and judgments. Each time, a subset of the physicians defined an information need; that group then agreed on the final information need. Eventually, these physicians scanned (manually) all of the guideline collection and identified CPGs that were relevant to the information needs (i.e., judgments). In order to evaluate the concept-based search and context-sensitive search methods, in addition to the full-text search, each information need was queried through a combination of a full-text query (FTQ), i.e., a list of keywords searched within the whole document, a context-sensitive query (CSQ), i.e., a query that searches for terms within three predefined context elements (knowledge roles that exist in the NGC ontology), and a concept-based query (CBQ), i.e., a list of concepts, all at the kth level of the concept tree. Thus, each query consisted of three components, each in a different format, each of which could be used, on its own or in combination with one or more other components, to query the guideline database. The typical FTQ consisted of two or three terms after stop-word removal. We selected three elements from the NGC document structure: “Target Population,” “Intervention and Practices considered,” and “Diseases and condition(s)” for the CSQ and full-text query. These elements are particularly meaningful when searching for a guideline to answer a clinical question that applies to one’s patient, and thus were suggested by the participating clinicians. For the CBQ two types of queries were formulated: (1) concepts from the 2nd level of the conceptual hierarchy, and (2) concepts from the 3rd level of the conceptual hierarchy. In this study we queried the CBQ using the four combinations of the outer and inner logic operators. Evaluation measures In order to evaluate the retrieval performance, we used the traditional precision and recall metrics. Precision is the proportion of relevant documents (defined for a specific query within the entire collection, also called judgments) within the set of the retrieved documents, and recall is the proportion of relevant-retrieved documents (judgments) retrieved from the set of relevant documents (total judgments), for a specific query. We also interpolated the averaged precision at eleven-points of recall levels 0.0, 0.1,..,1.0. Experimental plan In this study we wanted to examine the contribution of the CBQ to the FTQ and CSQ. We also wanted to examine the best level of querying, as well as estimating the best logic operators settings. Hypothesis I – adding concept-based search to full-text or context-sensitive search will improve the respective baseline performance. Hypothesis II – Querying concepts from the third level 424 will outperform querying at the second level.

R. Moskovitch et al. / Experiments with Hierarchical Concept-Based Search

Hypothesis III - There are significant differences among the results of the four types of searches.

Experiment II - CBQ in addition to single CSQ In experiment 2 we evaluated each of the eight combinations, in addition to a single context query. Figure 4 presents the four outperforming CBQs, in addition to the single CSQ baseline. Generally, at most of the recall levels all the four CBQs, including 3OA, 3OO, 3AO and 2OO, in decreasing order, outperformed the single CSQ baseline (1CSQ), while beyond the level of 0.3 recall the 2OO was bellow the baseline. In addition the additional four CBQs (not appearing in figure 4) were 2OA, 2AO, 3AA and 2AA. The four outperforming CBQs can be characterized by querying the 3rd level of the hierarchy, which outperformed the 2nd level. Three of them have OR set to the outer, and to the inner logic operators. Note that while all the four outperforming CBQs were the same, as in experiment 1, but in a slight different order, the best CBQ in the experiment two was 3OA.

To examine these hypotheses three main experiments were designed, in which we used CBQ in addition to a given baseline: (1) FTQ, (2) single CSQ, and (3) three CSQs. In each experiment we evaluated eight combinations of the CBQ resulting from three variables: (1) the queried level of the hierarchy second or third, (2) the outer logic operator having AND or OR, and (3) the inner logic operator having AND or OR, resulting in the four options: OR-OR, OR-AND, AND-OR and AND-AND. Table 1 presents all the eight combinations including an acronym which we will use in the report of the results. Table 1 - The eight concept based queries combinations Level

Outer

inner

Acronym

2

OR

OR

2OO

2

OR

AND

2OA

2

AND

OR

2AO

2

AND

AND

2AA

3

OR

OR

3OO

3

OR

AND

3OA

3

AND

OR

3AO

3

AND

AND

3AA

Figure 3 - CBQs applied in addition to the full text queries baseline, in which 3OO, 3AO, 3OA and 2OO outperform the baseline

Experiments and results We present the results of three experiments, in which the eight CBQs were applied in addition to a given baseline. As a result of the limited length of the paper we present only the four best CBQs in the figures, while report the order of the others performance in the text. We sorted the CBQs according to their average precision at 0, 0.5 and 1 recall level. Experiment 1 - CBQ in addition to FTQ In experiment 1 we evaluated each of the eight combinations, in addition to full-text queries. Figure 3 presents the four outperforming CBQs, in addition to the FTQ baseline. Generally at most of the recall levels all the four CBQs, including 3OO, which was significantly greater at the 0.05 significance level when compared at 0.5 recall level, 3AO, 3OA and 2OO in decreasing order, outperformed the FTQ baseline, while beyond the 0.6 recall level the FTQ outperformed. In addition the additional four CBQs (not appearing in figure 3) were 2OA, 2AO, 3AA and 2AA. The four outperforming CBQs can be characterized by querying the 3rd level of the hierarchy, which outperformed the 2nd level. Three of them have OR set to the outer, and to the inner logic operators. Note that the 3OO outperformed at most of the recall levels while the 3AO outperformed at the low recall levels.

Figure 4 - CBQs applied in addition to single context queries baseline, in which 3OA, 3OO, 3AO and 2OO outperform the baseline at most of the recall level Experiment III - CBQ in addition to three CSQs In experiment 3 we evaluated each of the eight combinations, in addition to three context queries. Figure 5 presents the four outperforming CBQs, in addition to three CSQs baseline. Generally, along all the recall levels all the four CBQs, including: 2OO, 3OA, 2OA and 3OO, in decreasing order, performed lower than the three CSQs baseline (3CSQ). In addition the additional four CBQs (not

425

R. Moskovitch et al. / Experiments with Hierarchical Concept-Based Search

appearing in figure 5) were 3AO, 3AA, 2AO and 2AA. Unlike the previous experiments, in which CBQs improved the baseline, here it decreased. We will refer to this in the discussion and conclusion section; however, again three of the four outperforming CBQs appear in the first and second experiments. In this experiment while there were two CBQs querying at the 2nd and 3rd hierarchy level, as well as the inner operator, the outer operator is OR at all the four top CBQs. Note that while 2OO was the fourth at both previous experiments, it is the first here (within the CBQs).

Previous studies examining CBS have shown that it does not necessarily improve, and might even worsen, the search-engine's performance [4]. This phenomenon might be due to the fact that current automated extraction modules are not yet sufficiently accurate, and users usually are not familiar with all the concepts when entering keywords. In the case of our CBS the user can manually specify queried concepts from a given predefined ontology of concepts. The limitations of this study are mainly caused by the size of the test collection, which is relatively small. However, in contrast to huge test collections, in which judgments are specified automatically, based on an ensemble of search engines, in our test collection the reviewers browsed manually each of the CPGs and indicated their relevance to the query. We are currently in the process of extending this method to enable a user enter simply a textual query which will be converted to a conceptual representation and will be queried in addition to the FTQ. Preliminary evaluation results on the Trec-Genomics are encouraging.

References [1] Humphreys BL and Lindberg DA. The UMLS project: making the conceptual connection between users and the information they need. Bulletin Medical Library Association, 81(2): p. 170-177, 1993. [2] Concard J, Yang C, and Claussen J. Effective collection metasearch in a hierarchical environment: Global vs. localized retrieval performance. Proc of SIGIR, 2002. [3] Wang W, Meng W, and Yu C. Concept hierarchy based text database categorization in a metasearch engine environment. Proc of WISE '00. 2000. [4] Hersh WR, Hickam DH, Haynes RB, and McKibbon KA. A performance and failure analysis of SAPHIRE with a MEDLINE test collection, JAMIA, Vol 1: p. 51-60, 1994. [5] Shahar Y, Young O, Shalom E, Galperin M, Mayaffit M, Moskovitch R and Hessing A. A framework for a distributed, hybrid, multiple-ontology clinical-guideline library and automated guideline-support tools. Journal of Biomedical Informatics, 37: p. 325-344, 2004. [6] Robert Moskovitch, Martins SB, Behiri E, Weiss A, and Shahar Y. A Comparative Evaluation of Full-text, ConceptBased, and Context-Sensitive Search. JAMIA, 14: p. 164174, 2007. [7] Hersh WR, Price S., and Donohoe L. Assessing thesaurusbased query expansion using the UMLS Metathesaurus. In Proceedings of AMIA, 2000. [8] Aronson AR and Rindflesch TC. Query expansion using the UMLS Metathesaurus. In Proceedings of AMIA Annual Fall Symposium, p. 485-489, 1997. [9] Srinivasan P. Retrieval feedback in MEDLINE. JAMIA, 3(2): p. 157-167, 1996. [10] Rada R, Bicknell E. Ranking documents with a thesaurus. Journal of the American Society for Information Science, 40: p. 304-310, 1989. [11] Moskovitch R, Cohen-Kashi S, Dror U, Levy I, Maimon A, and Shahar Y. Multiple hierarchical classification of freetext clinical guidelines. Artificial Intelligence in Medicine, 37: p. 177-190, 2006. [12] Salton G and McGill M. Introduction to Modern Information retrieval, New York: McGraw-Hill, 1983.

Figure 5 - CBQs applied in addition to three context queries baseline, in which the CBQs (2OO, 3OA, 2OA and 3OO) constantly decreased the baseline (3CSQ)

Discussion We presented Vaidurya, focusing on its concept based search methods, the hypotheses of the research, the corresponding experiments, in which CBQs in eight settings were applied to varying baselines. Testing hypothesis 1, in which we expected to have an improvement when applying CBQs in addition to varying baselines, shown a significant improvement when applied in addition to FTQ. An improvement was found in addition to a single CSQ, while decreased when used in addition to three CSQs. The reason for the difference between these three scenarios might be that the initial baseline precision in the last case (i.e., when using the three CSQs) was much higher than that when using FTQ or a single CSQ (see figures 3, 4 and 5). In addition, note that the improvement was greater when the baseline was lower (see figures 3 and 4). Referring to hypothesis 2, querying at the third level outperformed the second level, especially in experiments 1 and 2, while in the third it was even. Referring to hypothesis 3, three CBQs 3OA, 3OO and 2OO appeared within the four outperforming CBQs repeatedly within the three experiments. While 2OO was the last (within the four outperforming CBQs) in experiments 1 and 2, it was the first in experiment 3, which may be explained as well by the high level of the baseline performance. To summarize, an improvement by applying CBQ in addition to a textual search can be achieved, especially when querying at the third level, setting the outer logic operator to OR or AND, and setting OR to the inner logic operator. Note that using AND-AND achieved the lowest performance.

Address for correspondence [email protected]

426

Chapter 3. Sharing Data

427

This page intentionally left blank

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

StructConsult: Structured Real-Time Wet Read Consultation Infrastructure to Support Patient Care Craig Moriokaa, John David N. Dionisiob, Alex Buia, Suzie El-Sadena, and Hooshang Kangarlooa a

Medical Informatics, University of California, Los Angeles, United States Department of Electrical Engineering and Computer Science, Loyola Marymount University, United States

b

from emergency departments. Second opinions and overreads are also performed via teleradiology when the junior radiologist is unsure of the diagnosis and requires an expert opinion. A survey by Hunter et al. indicated that 60% of 287 academic radiology departments provide teleradiology services [3]. Kalyanpur et al. published an article on private radiology groups that have stationed board certified radiologist in India, in order to have 24 hour read outs of CT imaging studies [4]. This new emergent business model is known as NightHawk or Global Radiology Practices [5]. This allows the radiology group the ability to increase the number of procedures performed as well as their revenue. Primary care physicians usually treat patients for most maladies without consultation. For more difficult cases, the rate of referral by PCPs can be as high as 28.1% per 1000 patients [6]. The consultation process involves communication between physicians. Unfortunately, the communication process can cause serious problems for the patient. The Applied Strategies for Improving Patient Safety collaborative states that communication errors between clinicians, staff, and patients account for over 70% of all errors made in physician practices [7]. Breakdown in communication between clinicians accounts for 80% of all malpractice lawsuits [8]. The Physician Insurers Association of America (PIAA) report that communication errors between referring physicians and radiologists create the fourth most common compliant lodged against radiologists [9]. The PIAA reviewed 144 communication claims, and found that 10% of the written reports never reached the correct physician or patient. In another 10% of the claims, the radiologist’s issuance of a delayed written report affected the patient outcome in 75% of the cases. For 60% of the claims against radiologist, the most egregious mistake involved failure to deliver urgent or significant unexpected findings to the referring physician. In a recent study on generalist-subspecialist communication concerning children with chronic conditions, Stille et al. describe specific improvements in order to enhance communication between physicians [10]. The study advocated timely communication, understanding the reasons for the referral and the nature of the child's condition, or appropriate definition of what role the generalist and specialist should play in the treatment of the child. Physicians described numerous examples where communication had direct effects on patient outcomes. Efforts to

Abstract Our research addresses how to improve physician to physician communication of patient information, and how to prevent lapses of patient care as they are referred to other clinicians within the healthcare system. The wet read consultation is defined as a rapid response to a clinical question posed by a referring physician to a clinical specialist. This research involves the development of an imaging-based wet read consultation system called StructConsult (SC), which facilitates communication between non-imaging specialist (i.e., primary care physician (PCP), emergency room (ER) physician, or referring physician), and an imaging specialist-radiologist. To facilitate data mining and effective recall, SC utilizes a data model based on the Digital Image Communications in Medicine (DICOM) standard for grayscale presentation state and structured reporting. SC requires information from four sources: (a) patient-specific demographics, clinical hypothesis, and reason for exam, (b) sentinel image capture from a DICOM image study, (c) direct capture of radiologist’s image operations and annotations, and (d) radiologist's response to the chief compliant, and the reason for examination. SC allows users to add additional functionality to a Picture Archiving System to improve patient care. Keywords: summarization of patient data, DICOM presentation state, DICOM structured reporting, wet read consultation, teleradiology

Introduction Radiology is the one specialty that is inherently suited for the practice of telemedicine. In particular, the most common application of telemedicine in radiology is teleradiology. Teleradiology is the transmission of medical images to a remote location for the interpretation of radiographic images. Emergency departments, off-site outpatient imaging centers, and even rural hospitals are utilizing picture archiving and communication systems (PACS) to send images to radiology departments for immediate interpretations and/or second opinions [1-5]. The primary scenarios for the use of teleradiology involve off-hour and weekend interpretation of imaging studies

429

C. Morioka et al. /StructConsult: Structured Real-Time Wet Read Consultation Infrastructure to Support Patient Care

improve communication between pediatric generalists and specialists in the care of children with chronic conditions should emphasize the importance of timely information transfer, in order to avoid worsening the patient’s compromised health. The content of messages between physicians is also important, but lack of response when needed is more of a problem. Improving generalist/subspecialist communication has great potential to improve the quality of care. Forest et al. found that the top reason for referral was advice related to diagnosis and/or treatment [11,12]. Non-radiology medical clinicians — primary care, internal medicine, and emergency room physicians — typically do not have the proper training nor experience to diagnose difficult radiographic cases [13]. In research studies evaluating general clinicians versus radiologists, radiologists outperformed general clinicians in accuracy of image diagnosis [14,15]. A radiology consult provides two benefits: 1) the consultation helps the referring physician understand the patient’s present condition, and 2) the consultation provides the referring physicians with new medical knowledge. The goal of SC is to provide structured clinical consultations that address the specific clinical hypothesis posed by the referring physician. SC stores both the chief complaint, reason for the imaging exam, and the imaging evidence which substantiates the radiologist’s response to the reason for exam.

The three server-side components are shown near the center of the diagram: op-ros-ejb.jar is the central set of services provided by the software, as shown by the interfaces that it implements: Admin, UserAdmin, Info, Login, Order, Reconciliation, Station, Image Server, and Results. Other components rely on one or more of these interfaces to accomplish their respective tasks. The four end-user applications rely only on this component for all serverside interactions. pregen is the results generation daemon that prepares an imaging study for presentation on the Web application. When an imaging study is signed, the daemon converts the study’s selected key images from DICOM to a Web-compatible format such as JPEG or PNG, at the same time compositing any annotations that were made by the study’s reader. image server is the bridge between DICOM devices and SC. It is the gateway that receives incoming studies from these devices and notifies the other SC components of the studies’ availability. Received studies are stored on a file server that is mounted over the network on both the reading/annotation workstation and results generation daemon’s file systems.

Materials and methods SC addresses the communication barrier between referring physician and radiologist by enhancing the clinical workflow. The key components of the workflow that are improved include: the image order process, radiologist’s wet-read and response to reason for exam, and summarized structured report for the referring physician. The major software components of SC involve: image order, image capture, image order reconciliation, radiologist’s wet-read and response to reason for exam, and summarized structured report for the referring physician. Figure 1 illustrates the component or “wiring” structure of the SC software. The software consists of four end-user applications. Three of the applications are based on Java Swing, distributed via Java Web Start in the op-client.war archive, while one application is Web-based, deployed through the op-ros-web.war Web archive. •

• •

•

The referral order system is the Web application used for entering image orders and viewing their resulting images and annotations. The imaging study reconciler binds image orders to acquired imaging studies. The reading/annotation workstation displays the current worklist and corresponding imaging studies for reading, markup, and reporting. When an imaging study has been read and signed, it is “prepped” by the pre-genearation (pregen) component for presentation by the referral order Web application. The administration utility provides a user interface for configuring and setting up the SC installation, and user access and privilege.

Figure 1 - Component diagram of StructConsult software The overall design of the SC software follows recommended Enterprise Java Beans patterns, resulting in appropriate separation of classes, excellent portability, and flexibility of configuration and deployment. Every component of the suite has been tested on Windows XP, Mac OS X, and Linux. For optimal security, reliability, and stability, server-side components are typically hosted on a Unixbased operating system such as Linux or Mac OS X. Client machines tend to run end-user-centric operating systems such as Windows XP or Mac OS X. The components can also be deployed completely using established open source software (e.g., Apache, JBoss, PostgreSQL, Samba), thus

430

C. Morioka et al. /StructConsult: Structured Real-Time Wet Read Consultation Infrastructure to Support Patient Care

reducing the overall cost of an SC installation, especially when compared to existing commercial solutions.

DICOM presentation state The image viewer developed by our research group reads DICOM compliant radiographic image studies. When reviewing the images, the radiologist can capture the viewing conditions and image operations performed as a DICOM presentation state (PS). The DICOM PS captures the viewing state of the radiologist while the image study was dictated. The operations necessary to reproduce the same viewing conditions involve the exact image(s) reviewed, look-up table (LUT) values of the image displayed, scale size, rotation, linear measurements, and annotations. The objective of the PS is to capture the radiologist’s viewing conditions at the time the image study was reviewed. Once the user saves the DICOM PS, the DICOM PS object is sent to the image server for storage. The DICOM standard allows DICOM PS objects to be transmitted, stored, and retrieved from a DICOM compliant server.

StructConsult Organization Image order One of the major problems in medicine is improving physician communication between a referring physician and radiologist. The most prevalent criticism from referring physicians was that the radiology report did not answer the clinical question that they expected based on the patient’s chief complaint. Referring physician consultations with subspecialists can be more effective when the referring physician provides a specific question, clinical hypothesis, or specific reasons for exam concerning the patient condition [12]. Once the chief complaint and reason for exam are captured, the structured information is then used by the radiologist at the review station to determine which images in the image study will provide evidence linked to each particular reason for exam. The user interface for the referring physician is intended to capture two important pieces of information: 1) chief complaints, and 2) clinical hypothesis. The chief complaint is entered — abdominal pain. The reason for exam is also entered as a list of “Rule-out the following:” heart attack, ulcer, appendicitis, etc. Figure 2 shows the referring physician user interface. The referring physician enters the chief complaint (upper right panel) and finally enters the reasons for exam, or clinical hypothesis for the patient (lower right panel). Image capture After the image order is complete, the patient is sent to an imaging center to receive their exam. As part of SC, there is a DICOM image server that receives images from the patient scanning devices, or from the PACS server. DICOM image object definitions that can be received by our image server include the following: computed radiography, computed tomography, magnetic resonance, ultrasound, secondary capture, presentation state, and structured report objects.

Figure 2 - Referring physician order entry Web page Radiologist review workstation The radiologist review workstation allows the user to see patient demographic information, chief complaint, and reason for exam (see Figure 3). The reason for exam is shown in the upper panel in the right corner. The radiologist indicates for each reason for exam the following response: rule-in, rule-out, or unsure. When a reason for exam response is selected, the images that correspond to a particular reason for exam are captured as a presentation state. The response to a reason for exam is also captured as part of the structured report. The structured report can also capture free-text data keyed in by the radiologists as additional comments. Once a particular image study is chosen, the SC image review workstation allows the radiologist to examine the entire image study (see Figure 4). The toolbar along the top of the viewer allows basic image operations: horizontal and vertical flip, window and level, pan, scale, and rotation. The viewer also allows the user to draw line annotations, text, and different types of measurements (point, line, and area). Finally, the image viewer can present studies using different image layouts.

Image order/image study reconciliation The image order/image reconciliation process allows the administrator the ability to match up an image order with an image study. The SC infrastructure makes no assumption about the healthcare infrastructure. As healthcare within the United States can be fragmented, it cannot be assumed that the entire healthcare process is within the same healthcare entity. A patient may visit their PCP, or referring physician, who is unaffiliated with the image scanning facility. SC provides a manual method to reconcile a particular image order and image study. This reconciliation process can only begin once the image order is complete, and the patient’s image study has been received by SC’s image server. A worklist is provided to the SC administrator to identify outstanding imaging orders that need to be reconciled with the image study. The user selects the image order in one column, and then matches the appropriate image study in the other neighboring column. Patient demographic information is provided to help user ascertain the likelihood of a match between the order and the study.

431

C. Morioka et al. /StructConsult: Structured Real-Time Wet Read Consultation Infrastructure to Support Patient Care

physicians completed the survey. There were 5 questions asked of the users:

DICOM structured report The referring physician’s clinical hypothesis, DICOM PS of the image study, and the clinical findings that rule-out a clinical hypothesis are stored as a DICOM structured report (SR). The structured report is the underlying framework for capturing the clinical questions and responses for the patient. The primary components of a DICOM SR document consists of meta-information that describes the creator of the document, date of creation, time of creation, institution, and a unique identifier for this DICOM SR object. Other structured information captured by DICOM SR include: patient’s chief complaint, referring physician’s reason’s for exam, radiologist’s response to reason for exam, reference to specific DICOM image(s) referred to by DICOM PS, and reference to DICOM PS.

1. What was the availability of the Web-based image report? 2. Compare the Web-based system to the traditional phone order/fax report system. 3. What was the accuracy of the report content of the Web-based report compared to the faxed version? 4. When accessing the Web report from a browser, how long did it take? 5. What was the turnaround time of the Web report compared to the traditional phone order/fax report?

Summarized view of structured text and image Once the radiologist has structured their response to the chief complaint and reasons for exam by capturing the image(s), text comments, and structured responses, the information is stored in our image server as a DICOM SR object and within the PostgreSQL database. The results view of SC allows clinicians a summarized view of the patient. Patient demographic information, previous clinical documents, responses to the reason for exams, and the key images are stored as a Web-accessible page summarizing the patient’s condition. Figure 5 depicts our Web-accessible DICOM SR summarized view. The patient’s summary is a Web page that can be accessed anywhere given a user ID and password. The upper left panel in Figure 5 depicts the patient demographics with name, medical record number, gender, and patient’s birth date. The worklist to the right of the patient demographics is a list of available documents for the respective patient. All available clinical reports and labs can be selected from this panel. The text panels below the worklist show the most recent oncologist, radiology, and pathology reports. For this lung cancer patient, all responses to the reason for exams were positive and indicated by the red check mark shown on the right side of in the image box. The small thumbnail images along the top of the image panel are the DICOM PS key images captured by the radiologist. The user can select the smaller thumbnail image, which will appear in the larger image viewing panel. This summarized view of the radiologist’s responses to the referring physician’s clinical hypothesis completes the communication between clinicians.

Figure 3 - Radiologist review workstation depicts structured capture of reasons for exam (i.e., index lesion 3 is checked rule-in, upper right corner). Previous radiology reports are also available for review Each question was based on a 5-point rating scale (1 = superior performance, 2 = better performance, 3 = performance was equal, 4 = poor performance, and 5 = very poor performance). Questions 1 (avg=1.2, std=0.4, p=0.034, n=5), 2 (avg=1.5, std=1.0, p=0.046, n=4), and 4 (avg=1.2, std=0.5, p=0.034, n=5) were statistically significant using a Wilcoxon signed-rank test. Questions 3 (avg=1.8, std=1.0, p=0.052, n=4), and 5 (avg=1.6, std=0.9, p=0.052, n=6) were very close to significance.

Discussion The trend in the replies indicated that the users chose a score of 1, 2, or 3 indicating equal or better performance for the Web-based system for all questions. There were no user responses that indicated that the Web-based system was poor or very poor when compared to the conventional paper order/fax report. The goal of our SC system is to provide structured clinical consultations between referring physicians and radiologists. The SC infrastructure provides improved communication support for the referring physician, particularly for patients with complex medical conditions requiring close monitoring to insure proper quality of care. Utilizing our SC system insures that the communication between clinicians is accurate and justifi-

Results The SC system has been deployed at a primary care clinic/ imaging center. Harris Family Medical Center and University Center Imaging (UCI) in Melbourne, Florida have registered 48 referring physicians. The number of SC image orders from 01-01-2006 to 11-01-2006 was 11,093. This accounts for 32.4% of all images ordered. A majority of the image orders are still paper-based, but as the referring physicians become more comfortable using the Webbased system, a larger number of online image orders is expected. A survey was administered to the referring physicians using the SC system. Five of the referring

432

C. Morioka et al. /StructConsult: Structured Real-Time Wet Read Consultation Infrastructure to Support Patient Care

able. SC stores both the reason for the exam, and the imaging evidence that substantiates the radiologist’s response. Our system is not limited to only radiology consults; potentially one could provide accessible services to other subspecialists. Another benefit of the SC infrastructure is the ability to create teaching files, as the radiologist captures key images through DICOM PS. A long-term goal of this research is to generate summarized views of patient data accumulated over long period of care. The SC infrastructure will provide the structured image data necessary to complete this task.

This research is being supported in part by the National Institutes of Health Grant P01-CA51198-09A1.

References [1] Franken E, Harken K, and Berbaum K, Teleradiology Consultation for a rural hospital: patterns of use. Acad Radiol 1997;4:492-96. [2] Lee J, Renner J, Saunders B, Stamford P, Bickford T, Johnston R, Hsaio H, and Phillips M, Effect of real-time teleradiology on the practice of the emergency department physician in a rural setting. Acad Radiol 1998;5:8533-38. [3] Hunter T, Krupinski E, Hunt R, and Erly W, Emergency Department Coverage of Academic Departments of Radiology, Acad Radiol 2000;7:165-70. [4] Kalyanpur A, Weinberg J, Neklesa V, Brink J, and Forman H, Emergency radiology coverage: technical and clinical feasibility of an international teleradiology model, Emerg Radiol 2003;10:115-18. [5] Mun S, Tohme W, Platenberg R. and Choi I, Teleradiology and emerging business models. Journal of Telemedicine and Telecare 2005;11:271-75. [6] McPhee SJ, Lo B, Saida GY, and Meltzer R, How good is communication between primary care physicians and subspecialty consultants? Arch Intern Med 1984;144:126568. [7] Fernald DH, Pace WD, Harris DM, West DR, Main DS, and Westfal, JM, Event reporting to a primary care patient safety reporting system: a report from the ASIPS collaborative. Ana Fam Med 2004;2:327-32. [8] Levinson, W, Physician-patient communication. A key to malpractice prevention. JAMA 1994;272:1619-29. [9] Brenner RJ, Lucey LL, Smith JJ, and Saunders R, Radiology and medical malpractice claims: a report on the practice standards claims survey of the Physician Insurers Association of America and the American College of Radiology. AJR 1998;171:19-22. [10] Stille CJ, Korobov N, and Primack WA, Generalistsubspecialist communication about children with chronic conditions: an analysis of physician focus groups. Ambul. Pediatr. 2003;3:147-53. [11] Forrest CB, Glade GB, Starfield B, Baker AE, Kang M, and Reid RJ, Gatekeeping children and adolescents to specialty care. Pediatr. 1999;104:28-34. [12] Franken Jr. EA, Gergus GR, Koch TJ, Berbaum KS, and Smith WL, Added value of radiologist consultation to family practitioners in the outpatient setting. Radiology 1995;197:759-62. [13] Schriger DL, Kalafut M, Starkman S, Krueger M, and Saver JL, Cranial computed tomography interpretation in acute stroke. JAMA 1998;279:1293-97. [14] Kouri BE, Parsons RG, and Alpert HR, Physician selfreferral for diagnostic imaging: review of the empiric literature. AJR 2002;179:843-50. [15] Potchen EJ, Cooper TG, Sierra AE, Aben GR, Potchen MJ, Potter MG, and Siebert, J.E., Measuring performance in chest radiography. Radiology 2000;217:456-59.

Figure 4 - Image viewer component of radiologist review workstation shows DICOM PS capture of top 2 images. Each image has a 2D tumor measurement

Figure 5 - Web page with summarized state of patient showing DICOM PS (key images) and SR responses (red check boxes)

Address for correspondence: Craig A. Morioka, Ph.D. Assistant Professor UCLA Medical Informatics, Department of Radiological Sciences, 924 Westwood Blvd. Suite 420, Los Angeles, CA 90024

Acknowledgments We would like to thank Greg Tashima from Medaxis Corporation for his assistance in gathering the user survey and utilization data.

433

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Refining DICOM for Pathology – Progress from the IHE and DICOM Pathology working groups Christel Le Bozeca,b,c, Dominique Henina, Bettina Fabiania, Thomas Schrader d, Marcial Garcia-Rojo e , Bruce Beckwith f a

ADICAP, France; bINSERM, UMR_S 872, eq.20 Paris, F-75006 France; Univ Paris Descartes, Paris, F-75006 France; c APHP, Hôpital Georges Pompidou, Paris, F-75015 France; dDepartment of Pathology, Charite, Berlin, Germany, e Pathology Department at the Hospital General de Ciudad Real Spain; fDepartment of Pathology, Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, MA, 02215, USA

exchange of information called “Transactions”, from various functional components of a distributed healthcare environment, called “Actors”. IHE provides implementation guides for “Transactions”, using established standards as DICOM or HL7. IHE has developed in North America, Europe and Asia. The annual definition cycle of new profiles by users and suppliers, ending in the organization of international platforms of interoperability tests (called "connectathons"), confers its unique efficiency, transforming basic standards into "plug and play" solutions.

Abstract For making medical decisions, healthcare professionals require that all necessary information is both correct and easily available. We address the issue of integrating anatomical pathology department information into the electronic healthcare enterprise. The pathology workflow from order to report, including specimen processing and image acquisition was modelled. An integration profile pathology general workflow- was created in the framework of the Integrating the Healthcare Enterprise (IHE). This Integration Profile relies on 8 transactions based on HL7 or DICOM standards. An important issue was to define information entities (order, imaging study and report) and real-world objects (specimen, tissue sample, slide, etc). Joint efforts between IHE and DICOM WG26 has resulted in a proposed common model for “specimen” usable for both HL7 and DICOM transactions related to anatomic pathology.

In some cases, IHE recommends selection of specific options supported by these standards; however, IHE does not introduce technical choices that contradict conformance to these standards. If errors in or extensions to existing standards are identified, IHE’s policy is to report them to the appropriate standards bodies for resolution within their conformance and standards evolution strategy. In Europe, in 1996, the Association for the Development of Informatics in Cytology and Pathology (ADICAP) with the collaboration of different software vendors proposed a European de facto standard for image folders [4].

Keywords: pathology, standards, imaging, integration

Introduction

In the US, Laboratory Digital Imaging Project (LDIP) began in 2005 with a goal to establish an open access, voluntary use specification that will permit images generated in pathology and in clinical laboratories to be widely shared across different applications, databases, and operating systems for the purpose of enhancing image annotation, integration, archiving, publication, and analysis. This effort is intended to be complementary to DICOM, rather than as an alternative to it.

Information systems in anatomical pathology departments gather medical data (text, images, etc.) throughout various procedure steps from specimen processing to report editing. Since information systems are not typically integrated, information acquisition is time consuming with double data entry. Orders, images and reports are spread out over different systems which do not interoperate. Although standardization efforts conducted by HL7 [1] and DICOM [2] are progressing to provide integration solutions, HL7 or DICOM messages contain many optional data fields so that being DICOM or HL7 compliant does not imply direct integration. The goal of the Integrating the Healthcare Enterprise (IHE) initiative is precisely to specify how data standards should be implemented to meet specific healthcare needs and to make systems integration more efficient and less expensive [3]. Based on working groups including users and manufacturers, IHE defines “Integration Profiles”, that are real-word situations describing

Relying on this experience, the aim of ADICAP is now to promote the use of international standards (DICOM, HL7) in the development of information systems in anatomical pathology. In 2005, ADICAP and SEAP (Spanish association of Anatomic Pathology), with the collaboration of the Group promoting the Modernization of Hospital Information Systems in France (GMSIH), launched the IHE pathology initiative in Europe.

434

C. Le Bozec et al. / Refining DICOM for Pathology – Progress from the IHE and DICOM Pathology working groups

Although specific DICOM objects are defined for pathology, modification and/or extension are necessary for two main reasons. First, the current DICOM specimen module is not sufficiently detailed to capture the complexity of pathology practice, and second, some pathology-related image formats (whole-slide images, multispectral images, flow cytometry, etc) do not have applicable DICOM Information Object Definitions. A specific DICOM working group (DICOM WG26) has been recently created to address these issues. A specific working group (HL7 Pathology Special Interest Group) has been also recently created within HL7 to address the pathologists’ needs, in synergy with DICOM WG26, focusing on the orders and reports aspects of the pathology workflow.

since it is specimen-driven and when digital imaging is performed many types of imaging equipments (gross imaging, microscopic still imaging, whole slide imaging, multispectral imaging, etc.) may be involved for a single examination. Moreover, images of the same study may be related to different specimen (parts and/or slides) from one or even different patients (e.g., Tissue Micro Array). Finally, slides are always available to acquire more images, if needed. In radiology, the diagnostic process is patient-driven, an examination (study) usually involves a single image acquisition modality and all images of the study are related to one and only one patient.

The objective of this paper is to present a methodology to integrate anatomical pathology department into the healthcare enterprise. We first describe the IHE-Pathology efforts to model the anatomical pathology workflow in order to define new Integration Profiles. We also describe IHE-Pathology and DICOM 26 joint efforts to define a common model of “specimen” usable for both HL7 and DICOM transactions. DICOM Information Object Definitions for whole slide images is out of the scope of this paper.

Figure 1 - Anatomic pathology workflow The Integration Profile “Pathology General Workflow” was proposed for a first IHE cycle: (Specimen, Order and Report Management Workflow) and Pathology Image Workflow. This Integration Profile involves 8 Transactions exchanged by 8 Actors: 3 transactions dedicated to Order Management (Order Placer Management, Order Filler Management, Procedure Scheduled and Update), 4 transactions dedicate to Image Management and one procedure for Report Management. For each transaction, the workgroup proposes the use of the most suitable format (HL7 version 2, - HL7 version 3 (Clinical Document Architecture (CDA)) and DICOM).

Methods Modeling anatomical pathology workflow ADICAP, with the collaboration of the GMSIH, solicited participants to work on the Pathology Technical Framework: 12 pathologists and haematologists, 6 professional associations and 12 vendors. 7 working sessions were organized between September 2005 and January 2006. The working group first defined the pathologists’ needs and then created a corresponding Integration Profile for anatomic pathology. They identified the Actors and Transactions involved in this profile. Then, they reviewed the literature about order forms and reports in anatomic pathology in order to describe the main requirements for the structure and content of orders, imaging folders and reports.

General principles were defined with respect to information entities: Order, Image Study and Report. The order for the pathological examination is communicated between the Order Placer (of the Order Entry system) and the Order Filler (of the PIS). In the pathology department environment, the Order Filler also identifies the set of procedures and sub-procedures (procedure steps) that have to be performed in the process of fulfilling the order.

Defining HL7 and DICOM based transactions DICOM currently has Information Object Definitions dedicated to anatomical pathology, namely VL Photographic Image (XC) for gross imaging and VL Slide-Coordinates Microscopic Image (SM) for microscopic imaging. Following creation of the DICOM Pathology Working Group (WG-26), five IHE-Pathology-DICOM working sessions were organized between September 2005 and November 2006 to define modification/extension needed for the DICOM specimen module.

Order Quality assurance about order forms provides a list of mandatory items [5]: order identification (Order ID), order date & time, identification of the ordering physician and the ordering care department (including call back telephone number), patient identification (PID, name, visit number, etc.), identification of the care unit of the patient (if different from the ordering care department), priority of the order, (date & time when the results are expected to be available), etc. Each order may contain one or more Requested Procedure possibly reported by different pathologists. A Requested Procedure is a unit of work resulting in one report with associated codified, billable acts. For each Requested Procedure, the basic or special

Results General pathology workflow (pathology technical framework - volume 1) The diagnostic process in anatomical pathology (figure 1) differs from that in the clinical laboratory since it relies on image interpretation. It also differs from that in radiology

435

C. Le Bozec et al. / Refining DICOM for Pathology – Progress from the IHE and DICOM Pathology working groups

techniques involved in the processing of the corresponding specimen(s) may require different devices (automatons, image acquisition modality, etc). Each Requested Procedure may contain one or more Procedure Steps. A Procedure Step is the smallest unit of work in the workflow that is scheduled (work to do) and/or performed (work done) by a person or a machine (automaton, image acquisition modality, etc) on an object (specimen, tissue sample, tissue section, etc.) Figure 2 depicts an example of cases resulting from an order and of the breakdown in Requested Procedures and in Procedure Steps.

Actors and transactions

Report Since 1993, Association of Directors of Anatomic and Surgical Pathology has published recommendations for pathologic reporting [6]. A generic model of structured report can be derived from these templates. In complement, studies about quality assessment of reports provide lists of mandatory items and stress the positive role of checklists to enhance the reporting process [7,8]. According to “evidence-based pathology”, only features that are reproducible and relevant – with a demonstrated diagnostic or prognostic signification – should be reported in description and corresponding evidence available”[9,10]. A crucial issue is to identify a technical solution to handle templates of structured reports including findings and their evidences.

Figure 3 - General pathology workflow Six specific transactions exchanged between 7 actors are required to perform the General Pathology Workflow and are schematically represented in Figure 3. Placer Order Management (PAT1) contains all the messages required between the Order Placer and the Order Filler for the management of the life cycle of the order. Its main goal is to keep a consistent vision of the order, (content and status), between the two actors. Filler Order Management (PAT2) contains all the messages required between the Order Filler and the Order Placer for the notification of a new filler order, as well as the creation of the placer order that reflects it.

Image

Modality Worklist Provided (PAT3) is based on a query entered at the Acquisition Modality. In case of a general query, the list of Scheduled Imaging Procedures with selected demographic information and information about specimens is returned to the Acquisition Modality. In case of a query using the barcode identifying a given specimen being source of an imaging procedure (either part or slide), the specific information about this specimen is returned.

In pathology, the image folder (study) is defined at the level of the pathological examination or case. For each case, images acquisition may require different modalities (gross imaging, microscopic imaging, etc). A new series is created whenever an imaging procedure step is performed on a new specimen or slide or when a new type of image is created of the same specimen or slide.

The Procedure Scheduled and Update (PAT4) allows the Order Filler to send the Image Manager and Report Manager information about the scheduled procedure or procedure update. Report Management (PAT7) carries changes of the observation results and order status from Order Filler to the Enterprise Report Repository i.e. corrections, cancellations. Using the Image Availability Query (PAT9), the Order Filler or the Report Manager asks the Image Manager if a particular image or image series is available. The worklist provider informs other interested actors of the on-going status and completion of performed work. The transaction Modality Image Stored is based on two transactions already defined in the radiology domain (RAD8, RAD10) and allows an Acquisition Modality or an Evidence Creator sending acquired or generated images to the Image Archive. Acquisition Modality or Evidence Creator requests that the Image Manager confirm ownership for the specified DICOM objects (images, Key Image Notes, Evidence Documents or any combination thereof)

Figure 2 - Extract of the structure and content of an order: the breakdown in cases (corresponding to reports) and procedure steps and the organization of the corresponding images in studies and series

436

C. Le Bozec et al. / Refining DICOM for Pathology – Progress from the IHE and DICOM Pathology working groups

stored in the Image Archive, thus allowing the sender to delete those objects.

dice(s)). “Tissue dice(s)” within a block are tissue samples resulting from sampling processes within the pathology department obtained from one or more parts. They have been processed and embedded in a medium to allow the creating of tissue sections which are mounted on slides for imaging. Tissue dice(s) may be identified (Tissue dice ID, optional). The corresponding Cassette is mandatory identified (Cassette ID)

HL7 and DICOM based transaction (pathology technical framework – volume 2) Prior to the specification of the transaction a common model of “real world objects” (specimens, containers (blocks, cryomolds or cryotubes, slides, etc) was defined. Specimen identification mechanism

“Slide” refers to a container which holds a tissue section, a smear, a touch prep, etc. The slide is typically glass for visual light microscopy, but is usually a grid for electron microscopy. “Tissue sections” are created from Tissue Dice(s) embedded in blocks. “Touch preps” are prepared by placing a slide into contact with unprocessed tissue. “Smears” are created from taking a liquid containing cells (peripheral blood or other bodily fluids such as ascites) and spreading this liquid into a thin layer.

The specimen identification mechanism must allow keeping track of all these “real world objects” (figure 4). “Specimen” is the role played by any discrete physical object that is the subject of pathologic examination. This includes objects at all levels of processing, including fresh tissue, dissected organs, tissue embedded in paraffin, and sections made from embedded tissue. This extends the common definition of a specimen beyond the object itself received for examination (e.g., from surgery).

Tissue item(s) within a tissue section or a smear may be identified (Tissue item ID, optional). The corresponding Slide is required to be identified (Slide ID)

“Container” is a physical object which can be labeled (eg with a pathology department accession number, or a block label or a slide label, etc) and contains a tissue sample which may be analyzed or further processed.

“Tissue Microarray” (TMA) is a composite specimen which is typically created by taking a small core of tissue from many different tissue blocks and re-embedding them in a new block in an organized manner. Slides created from this TMA block thus have small fragments of many different tissues all of which may be processed at the same time, under the same conditions by a desired technique. These are typically utilized in research (figure 5).

“Box” refers to a container which contains Part(s). “Part” refers to a separately identified physical object (tissue) collected in care department upon which a pathologic or laboratory procedure is requested to be performed.

Figure 5 - TMA : Tissue items (“spots”) on slide come from tissue dice(s) (“cores”) sampled from different tissue blocks (from different parts and from different patients) Specimen description Specimen(s) and container(s) description requires a controlled vocabulary. A first step consisted in aligning DICOM Specimen Module and HL7 v2.5 Specimen (SPM) and Container (SAC) segments. Eleven DICOM tags of the Specimen Module correspond to items of the HL7 v2.5 SPM segment: Specimen ID; Specimen Type, and Type Modifier; Specimen Collection Method, Site and Date/Time; Specimen description; Specimen Handling

Figure 4 - Typical : Specimen can be identified by containers’ ID “Cassette” refers to a “container” which contains Tissue dice(s) that are embedded in Block (one cassette corresponds to one block that corresponds to one or more tissue

437

C. Le Bozec et al. / Refining DICOM for Pathology – Progress from the IHE and DICOM Pathology working groups

Code; Specimen Received Date/Time; Specimen Condition; Container Type and Condition and Parent Specimen ID. Four DICOM tags of the Specimen Module could be expressed using items of the HL7 v2.5 SAC segment: Container length, Container width, Container thickness and Container length, width, thickness units.

Architecture (CDA) seems to be the suitable format but solutions available to link textual items to DICOM images in an HL7 CDA document must be clarified. Our perspective is that vendors implement IHE-pathology principles. Thanks to an on going collaborative work involving IHE-Pathology, DICOM WG26 and HL7 pathology SIG, an implementation guide (Pathology Technical Framework) will be available for a first IHE cycle in 2008.

We haven’t found any existing item in HL7 v2.5 to express specimen state identifier and specimen state description. We have no more found a satisfactory solution to solve the specimen identification issue in case of TMA.

Acknowledgments This work has been supported by the ADICAP and the College of American Pathologists.

Discussion Quality assessment studies in anatomical pathology show that each of the different steps from specimen processing to report editing may be source of errors and that information systems integration supports error reduction [11-13]. This work is done in the framework of the IHE-pathology initiative to define the requirements of systems integration in anatomical pathology.

References. [1] HL7 organization : Health Level 7, http://www.hl7.org [2] DICOM 3 Standard, NEMA and Global Engineering Group: http://www.nema.org. [3] IHE - http://ihe.univ-rennes1.fr/whatisihe/. Accessed January, 2006 [4] Klossa J, Cordier JC, Flandrin G, Got C, Hémet J. An European de facto standard for image folders applied to telepathology and teaching. Inter. J. Med. Information. 1998, 48:207-216. [5] Burnett L, Chesher D, Mudaliar Y. Improving the quality of information on pathology request forms. Ann Clin Biochem, 2004. 41(Pt 1): p. 53-6. [6] ADASP. Consultations in surgical pathology. Association of Directors of Anatomic and Surgical Pathology. Am J Surg Pathol. 1993 Jul;17(7):743-5. [7] Campbell, F. and D.F. Griffiths, Quantitative audit of the content of histopathology reports. J Clin Pathol, 1994. 47(4): p. 360-1. [8] Imperato PJ, et al., Results of a cooperative educational program to improve prostate pathology reports among patients undergoing radical prostatectomy. J Community Health, 2002. 27(1): p. 1-13. [9] Fleming KA. Evidence-based pathology, J Pathol. Jun 1996; 179(2): 127-8. [10] Fleming KA. Evidence-based cellular pathology, Lancet, 2002; 359(9312): 1149-50. [11] Cowan DF. Quality assurance in anatomic pathology. An information system approach. Arch Pathol Lab Med, 1990. 114(2): p. 129-34. [12] Goldstein NS. Diagnostic errors in surgical pathology. Clin Lab Med, 1999. 19(4): p. 743-56. [13] Zarbo RJ, Meier FA, Raab SS. Error detection in anatomic pathology. Arch Pathol Lab Med, 2005. 129(10): p. 1237-45.

The results show that a first significant IHE cycle in anatomic pathology could involve the new integration profile “General Pathology Workflow”. The main contributions of this work were to analyze the specificity of the anatomical pathology workflow with respect to laboratory and radiology workflows and to define the structure and content of cases, orders, image folders and reports. There was an issue to make explicit the links between information entities (orders, image folder, reports, etc.) and real-world objects (specimens and containers, etc). The anatomic pathology specimen model that we have described was developed to be consistent with the Specimen Information Module included in HL7 version 3, which is based on the HL7 Reference Information Model (RIM). The HL7 model was designed for clinical laboratory specimens, but is sufficiently generic to be useful for anatomic pathology specimens. We consciously attempted to maintain consistency with the HL7 v3 Specimen Common Message Element Type (CMET), and in our definition of a specimen, we consciously adopted the terminology of the focal class specimen as a “Role” with an identifier, in accordance with the v3 model. Although the main output of the anatomical pathology workflow is a timely and clear report of a diagnostic opinion, images will be more and more associated as evidence to textual reports. DICOM seems to be a convenient format for image archiving and communication within the anatomical pathology department. For integration into the Electronic Healthcare Record, HL7 Clinical Document

Address for correspondence Dr. Christel Le Bozec. UMR_S 872 Eq20, Centre des cordeliers, 15 rue de l’école de médecine, 75006 Paris, France. [email protected]

438

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Generic, Web-based Clinical Information System Architecture Using HL7 CDA: Successful Implementation in Dermatological Routine Care Thilo Schulera,b, Martin Boekera, Rüdiger Klara, Marcel Müller b a

Department of Medical Informatics, University Medical Center Freiburg, Germany b Department of Dermatology, University Medical Center Freiburg, Germany

The aim of this paper is to share our experiences from designing and implementing a generic clinical information system architecture. This project was initiated to implement a special-purpose application in clinical dermatology. In order to make our decisions comprehensible and to prove our concept, the context and realization of this implementation will be illustrated.

Abstract The requirements of highly specialized clinical domains are often underrepresented in hospital information systems (HIS). Common consequences are that documentation remains to be paper-based or external systems with insufficient HIS integration are used. This paper presents a solution to overcome this deficiency in the form of a generic framework based on the HL7 Clinical Document Architecture. The central architectural idea is the definition of customized forms using a schema-controlled XML language. These flexible form definitions drive the user interface, the data storage, and standardized data exchange. A successful proof-of-concept application in a dermatologic outpatient wound care department has been implemented, and is well accepted by the clinicians. Our work with HL7 CDA revealed the need for further practical research in the health information standards realm.

Problem The outpatient clinic for patients with chronic wounds is a subunit of Freiburg university hospital’s dermatology department and needed an application to document their treatment. Up until this point paper files had been used to record visit findings and procedures. As the chronic wound clinic is a multi-provider facility, the main problems with the paper records were difficulties with legibility and conventions about how and what should be recorded. Further issues were the limited data reusability for research (no standardized recording practice and primarily no electronic format) and the non-existence of discharge letter writing assistance (e.g. partially pre-filled letter template).

Keywords: medical record systems, computerized, generic architecture, systems integration, hospital information systems; CDA, openEHR, HL7, semantic interoperability

Within the hospital an in-house developed information system is in use [3]. The system provides hospital wide services such as a central document and picture archive (e.g. discharge letters, radiology images), access to lab data, and management of administrative patient data via a range of client applications. The communication between distributed components is based on HL7 v2 [4]. Besides these general functions a number of custom extensions have been developed to reflect the information needs of single sub-/departments (e.g. a module to document the vein status).

Introduction The potential advantages of computerized patient records (CPR) versus conventional paper-based files have been known for many years [1]. CPRs were envisioned to improve both routine care and research in a multiple ways. The possible benefits range from obvious ones, like readability or availability, to visionary decision support scenarios. Studies were able to support many theoretical CPR advantages, while increased experience with CPRs and clinical information systems also reveals deficiencies. Ginneken [2] identifies low user-acceptance as a barrier that slows down the adoption of CPRs. This low useracceptance results from lack of integration and missing flexibility in content and use. Other impediments mentioned are an inability to adapt to change, the lack of data exchange standards, or an insufficient financial return on investment.

An external solution had also been considered for the chronic wound clinic. After determining scope (2 forms with around 200 data items) and basic requirements (analyzing existing paper recordings, conducting a series of three observation and interview sessions, and review of the existing products in the market) it became clear that no commercial product in the small market of chronic wounds documentation systems could fulfill the special needs. Lacking customisation flexibility, unsolved integration problems, incomplete features, and unacceptable conditions of use by one vendor were the main reasons for the decision to develop an in-house solution.

439

T. Schuler et al. / A Generic, Web-based Clinical Information System Architecture Using HL7 CDA

and further constrained by standardized, formal clinical content models called archetypes [10]. This so-called twomodel approach separates medical knowledge from technical knowledge to achieve semantic interoperability and future-proof health information systems. Part two of the CEN 13606 standard has adopted the openEHR Archetype Description Language (ADL).

Health information interoperability standards Establishment of semantic interoperability between distributed health information systems is probably one of the most important challenges of health informatics today [5]. Most expected CPR advantages [1] are based on this premise. It is widely accepted that only well designed health information standards can solve this problem.

Design decisions

There are currently two major standard initiatives that aim to achieve semantic interoperability of medical information: CEN 13606 [6, 7] & openEHR (http:// www.openehr.org/, accessed 29 Nov 2006) and HL7 v3 (http://www.hl7.org/, accessed 29 Nov 2006). While it is undisputed that standardized information structures have to be exchanged, the approaches to about how this can be done differ.

As a consequence of the depicted situation the following in-house solution design goals were rated with the highest priority: • • •

HL7 Clinical Document Architecture: HL7’s central design artefact is the Reference Information Model (RIM) from which message specifications are derived via a cascade of intermediary models. The idea is that this approach secures shared semantics. The HL7 Clinical Document Architecture release 2 (CDA r2, [8]) is an ANSI-approved exchange standard for medical documents. It has been developed according to the HL7 methodology and is fully based on the RIM. CDA r2 documents are XML instances with two main parts: a header setting the document context and the body containing the clinical report in a semantically enriched HTML-like markup. The CDA r2 specification purposely has a wide scope to be able to express any clinical document. Further constraint mechanisms are needed to enforce a particular structure. Currently, further constraints are defined by narrative ‘implementation guides’. In the future formal constraint expressions, called ‘HL7 Templates’, are envisioned for this task. In order to ease the adoption of the standard an incremental approach regarding the semantic enrichment is supported. The concept of levels reflects this design, while only a CDA r2 Level 3 document is envisioned to guarantee full semantic interoperability. CDA r2 Level 1 only expects a standard conforming document header while there are few restrictions for the body. In CDA r2 Level 2 the coarse body structure (sections) needs to be understood by the receiving system through definition of meaning in the form of terminology codes. Level 3 can be achieved by adding semantic markup (entries) for every narrative clinical statement. To adhere to the CDA’s human readability principle, level 3 markup can’t contain more information than the narrative.

• •

Customisation – to fit to clinical information requirements and workflow needs Integration – into the existing hospital IT environment Data sustainability and semantic interoperability – through standard conformance Generic methodology and components – to foster reusability in other clinical areas Pragmatism – to develop a solution that takes the limited resources into account and allows early utilization

Material and methods The design goals mentioned above determined the choice of development technologies and the applied methodologies. To allow the possibility of reusing the system for similar documentation problems within the department of dermatology (e.g. the clinic for dermatological autoimmune diseases) it was decided to build a generic architecture that can be adapted flexibly according to new needs. XML instances of a form description language implemented in RELAX NG were pictured to drive a GUI generator. Flexibility regarding data entry items implies flexibility of the underlying database. This requirement excluded the use of the central hospital database facilities and we used a variable XML format instead. In order not to create an isolated “data island” we decided to build a XSLT transformation mechanism that could export our data to the standardized CDA r2 format. Integration with the hospital information system (HIS) was to be achieved by regular sending of CDA documents as payload in HL7 v2 messages. Administrative patient data should be similarly imported from the administrative data module of the HIS. We conceived the architecture as a web application framework whose core features (form definition language, data storage) are based on XML technologies and tools. The presented initial version also uses technologies such as PHP5 (form generator) and MySQL (4 tables: patient data, user data, XML form definition and XML form data). The necessary security is achieved as the application can currently only be accessed from within the firewalled hospital intranet. Additionally LDAP authentication and https encryption were installed.

CEN 13606 and openEHR: The openEHR Foundation is a not-for-profit company behind an open community effort to produce specifications (requirements, technical and clinical models) and reference implementations. The aim is to achieve an “open, interoperable health computing platform, of which a major component is clinically effective and interoperable electronic health care records (EHRs)” [9]. While not a standards body itself, openEHR is dedicated to work with standards organizations. The revised European standard CEN 13606 is influenced by openEHR. Like the openEHR architecture, it promotes a stable reference model, whose classes can be aggregated

After the development of the generic architecture, analysis and deployment of the chronic wound application were the next steps. A number of form definition iterations were planned to achieve a customized solution. This approach follows the idea of rapid prototyping [11] and allows early

440

T. Schuler et al. / A Generic, Web-based Clinical Information System Architecture Using HL7 CDA

user involvement to secure a high-level of useracceptance.

rent values. This file is stored in the MySQL database together with metadata for retrieval. Transformation to the HL7 CDA level 2

Results

Based on former experiences with the first release of CDA [12, 13], we developed the framework according to the CDA r2 specifications. Provided correct code mappings exist in the form definition, an XSLT script that creates a CDA r2 Level 2 document can be derived automatically from the form definition. The CDA body content is created from the flexible form definition and the corresponding form data, while the metadata in the CDA header is based on information that is invariably required by the stable module.

Overall architecture Following the classic web paradigm the system architecture consists of a web server and database containing the application (PHP5) and a web browser on the client side to interact with the application. The application consists of: a form generator that creates HTML forms based on a formal XML form definition and modules for tasks such as user authentication, transformation to CDA r2, or integration with the HIS.

Integration with the HIS Integration with the existing HIS infrastructure is guaranteed by import of master data and export of CDA documents once they are approved by the supervising physician. Technically, import and export works by transferring XML documents via a HL7 v2 service interface provided by the HIS. Implementing the chronic wounds application Using the results from the scope and basic requirements analysis (conducted prior to the framework realization), an initial form definition was built by one author with good medical and IT knowledge. Consequently, 5 half-hour sessions with one medical user and one author were sufficient to reach the final definition. Figure 1 - Architecture of the system framework

In order to cover the use cases of the chronic wound clinic it was decided to have two types of forms: an ‘initial assessment’ form and a ‘follow-up’ form. The ‘initial assessment’ form contains the detailed patient history and information about special tests, that aren’t performed during every visit. It is primarily filled during the first visit, but it can also be updated later. A new ‘follow-up’ form is completed during each subsequent visit, documenting the current status and determining the further procedure.

HTML form generation The form generator component was written in PHP5 and dynamically generates HTML forms based on XML instances of a formal form definition language. These forms are used for editing and viewing. There is no restriction in the number of forms. Form definition language: A RELAX NG schema grammar restricts the form definition elements to 6 types that can be arranged using 3 layout patterns. Additionally the form can be divided in sections and subsections. Each sub/section XML tag has optional attributes to specify a terminology (e.g. LOINC or SNOMED) and the suitable code. The element types comprise the normal HTML input fields (text, checkbox, dropdown, upload) and widgets for date input (calendar) and for image display with planimetry. Each element has a mandatory label and an optional suffix. The available layout patterns are ‘beneath’ (field elements below each other), ‘float’ (field elements next to each other), ‘table’ (configurable tabular layout).

Discussion Proof of concept The first application of the presented framework has been successful. It is used daily by 3 nursing staff and 4 physicians. During the first 2 months of use about 500 visits of 150 patients have been documented. Although the system doesn’t

GUI features: The Graphical User Interface (GUI) is rendered according to the form definition file. By default, every section is displayed in its own tab (see Figure 2 for a screenshot from the chronic wound application). Data storage The structure of the form content data depends on the form definition. To store submitted form data, a simple XML file is generated by mapping element names to their cur-

441

T. Schuler et al. / A Generic, Web-based Clinical Information System Architecture Using HL7 CDA

Figure 2 - Chronic wound web-application GUI provide all possible features yet (see the section ‘Future’), early positive user reactions show that the system provides major improvements to the former solution. A study to analyse the system usability and the added value of the solution is planned.

CDA standard Standard support is crucial for inter-organizational information exchange and must be a goal of every modern clinical information system. We decided to use the HL7 CDA r2 standard. Its document orientation suits the formbased framework well, and more importantly, it was a pragmatic choice that could be implemented relatively easily on top of ubiquitous XML tools. Currently, the generated CDA documents conform only to Level 2. The generation doesn’t impose major difficulties provided suitable codes are found and set in the form definition. First experiments with Level 3 markup showed that it will not be easy to automatically create semantically correct CDA entry statements. Especially the necessary combination of several entries or the expression of post-coordinated SNOMED terms showed much arbitrariness.

The user involvement during requirements and implementation iterations was very valuable for both parties (medical and technical) and we can confirm similar experiences mentioned in the literature [2, 14]. Having the possibility to make quick adaptations to the form definition and displaying the result instantly helped the clinicians immensely to assess the current form and recommend improvements. The generator driven evolutionary approach described by Lenz and Kuhn [15] is similar in the rapid prototyping respect but manages system integration differently.

Total semantic interoperability means that a receiving system can derive the same meaning from a standardized information unit (e.g. a CDA document) as the sending system. This must be true for any CDA document. In our opinion for this long-term goal the CEN 13606/openEHR approach using coherent, standardized content models (archetypes) seems currently better suited. HL7 is aware of this “gap”, which HL7 templates are supposed to fill. Further research regarding CDA r2 Level 3 compared to CEN 13606/openEHR is needed.

Integration The presented architecture is potentially autonomous, but an integration with the central HIS components proved to be possible. The decentralized integration strategy is based on the standardized messaging exchange format HL7 v2. An direct extension of the HIS would have meant a prolonged development cycle and a high degree of inflexibility (limited existing data structures), which possibly could have lead to compromises in design.

442

T. Schuler et al. / A Generic, Web-based Clinical Information System Architecture Using HL7 CDA [4] Schrader U, Zaiss A, Büchele S, Binder M, Leupolz U, Klar R. Realization of a CPR Using HL7 Queries From Communication Server to Information Server. In: Kokol P et al., eds. Medical Informatics Europe '99. IOS Press, 1999; pp. 834-837. [5] Norgall T. Interoperability--a key infrastructure requirement for personalised health services. Stud Health Technol Inform 2005: 117: 125-133. [6] Health Informatics – Electronic health record communication - Part 1: Reference Model: CEN/TC251; 2005 March 2005. Report No.: prEN13606-1:2005:E. [7] Health Informatics – Electronic health record communication - Part 2: Archetypes: CEN/TC251; 2005 December 2005. Report No.: prEN13606-2:2005:E. [8] Dolin RH, Alschuler L, Boyer S, Beebe C, Behlen FM, Biron PV, Shabo AS. HL7 Clinical Document Architecture, Release 2. J Am Med Inform Assoc 2006: 13(1): 30-39. [9] Introducing openEHR [cited 29 Nov 2006]. Available from: http://svn.openehr.org/specification/TRUNK/pulish ing/openEHR/introducing_openEHR.pdf [10] Beale T. Archetypes: Constraint-based domain models for future-proof information systems. In: Baclawski K and Kilov H, eds. Eleventh OOPSLA Workshop on Behavioral Semantics: Serving the Customer (Seattle, Washington, USA, November 4, 2002). Boston: Northeastern University, 2002; pp. 16-32. [11] Fitton D, Cheverst K, Kray C, Dix A, Rouncefield M, Saslis-Lagoudakis G. Rapid Protoyping and User-Centered Design of Interactive Display-Based Systems. IEEE Pervasive Computing 2005: 4(4): 58-66. [12] Muller ML, Uckert F, Burkle T, and Prokosch HU. Crossinstitutional data exchange using the clinical document architecture (CDA). Int J Med Inform 2005: 74(2-4): 245256. [13] Muller ML, Butta R, and Prokosch HU. Electronic discharge letters using the Clinical Document Architecture (CDA). Stud Health Technol Inform 2003: 95: 824-828. [14] Wu RC, Abrams H, Baker M, Rossos PG. Implementation of a computerized physician order entry system of medications at the University Health Network--physicians perspectives on the critical issues. Healthc Q 2006: 9(1): 106-109. [15] Lenz R, Kuhn KA. Towards a continuous evolution and adaptation of information systems in healthcare. Int J Med Inform 2004: 73(1): 75-89.

Future Besides the necessary research mentioned in the preceding discussion sections, various practical and technical improvements to the framework could be explored and implemented to further address issues like the ones mentioned in the introduction (e.g. data reuse). Routine care would benefit greatly from a letter writing assistance tool or a history feature where certain entries can be viewed over time (e.g. numerical values as a chart). An “XML2relational” data dump tool would allow analysing the collected data with conventional statistics programs. At the moment, only one version of each form is valid. A variable versioning mechanism would overcome this restriction. Technically, a re-implementation of the architecture based on pure XML technologies such as XForms, the Apache Cocoon Framework, and a XML database would be interesting.

Conclusion This article describes a generic system architecture framework for health care applications. Through an autonomous concept special-purpose, form-based solutions that are customized to the needs of clinical users can be developed. The HL7 CDA r2 standard was used to ensure system integration. A proof-of-concept implementation for routine care shows its applicability. To what extend HL7 CDA or alternatives like openEHR can enable full semantic interoperability needs to be explored in further practical trials. Acknowledgments We would like to thank the medical and the nursing staff of the dermatological chronic wounds unit for their assistance and feedback during development.

References [1] Dick RS, Steen EB. The Computer-Based Patient Record: An Essential Technology for Health Care. Rev ed. Washington DC: National Academy Press, 1997. [2] van Ginneken AM. The computerized patient record: balancing effort and benefit. Int J Med Inform 2002: 65(2): 97-119. [3] Klar R, Zaiss A, Binder M, Schrgendorfer I. The minimum basic data set as the core of the Freiburg University Hospital information system. Medinfo 1995: 8 Pt 1: 586-589.

Address for Correspondence Thilo Schuler Department of Medical Informatics, University Medical Center Freiburg, Stefan-Meier-Str. 26, 79104 Freiburg, Germany [email protected]

443

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Analyzing the Key Variables in the Adoption Process of HL7 Alejandro E. Flores, Khin Than Win Health Informatics Research Centre, University of Wollongong, Australia

dard, and to outline alternative courses of action that permit the overcoming of those restrictions [4].

Abstract The intention of this paper is to provide an analysis of variables that could affect the adoption of HL7 message standard. Based on the review of 33 cases where HL7 was successfully implemented the authors present relevant evidence related to inherent limitation of HL7. The result from this study indicates that it is necessary to enhance the standard to overcome particular limitations and facilitate the implementation of inter-institutional software interfaces based on HL7.

The aim of this paper is to identify advantages and disadvantages of HL7 standard by the analysis of 33 specific cases. It will permit to identify benefits of the use, understand the adoption process and recognize limitation and barriers that should be considered during the implementation of HL7 standard. This will also offer an initial answer to why it is important to consider the development of new set of communication models based on HL7 in order to overcome the new requirement of connectivity and communication for specific application in the health care domain.

Keywords: Health Level Seven, adoption, connectivity and standardization

Methods

Introduction

As the main purpose of this research is to identify the key attributes that facilitate or limit the adoption of HL7. The methodology used in this study was based on an analytical generalization [5] based in the analysis of 33 articles related to the use of HL7 during the design and development of software interfaces for electronic health information systems.

Health Level seven (HL7) consortium was founded in 1986 to research and develop a set of standards for electronic data exchange in the heath care domain. The HL7 standard is a structured specification that can be used for interconnection and exchange of health records [1]. HL7 is the most widely used messaging standard for clinical and administrative data exchange among health care applications in the information technology industry [2]. HL7 has established a set of information and message models for the development and implementation of interfaces for communication and transmission of medical data among heterogeneous health information systems [2][3]. The aim of HL7 is to produce standards for a particular health care domain considering the holding of a strict and welldefined framework that ensure consensus, openness and balance of interest, and allow the development of specifications for the implementation of messages model and software interfaces[1].

Framework The Technology Acceptance Model (TAM) [6][7] was used as a framework for this analysis. The variables considered for this analysis were: (1) benefits, (2) adoption and (3) barriers. The variable cost was not considered because of the limited information provided by the authors of the studied papers. Scope The search was limited to articles published since 2000. The articles selected include implementation experiences in: (1) Electronic Health Records (EHR) Systems, (2) Cardiology Information Systems, (3) Electro Physiology Information Systems, (4) Radiology Information Systems, (5) Administrative, Support and Knowledge Information Systems, (6) Tele-medicine Services, (7) Regional Health Information Systems, and (8) Home Monitoring Health Information Systems.

At the actual level of development, the most intractable barrier for the use of HL7 has been the lack of standards for exchanging fine-grained, highly heterogeneous, structured clinical data among information systems that had been implemented under different platforms [2][3]. Moreover, the additional consideration of specific health information domain and inclusion of new information and communication technology adds levels of complexity to the initial hitch [4]. Therefore, it is necessary to explore the HL7 message standard and its application to different health care domains to establish limitations of HL7 stan-

Search strategy The search was realized in March 2006 and repeated in July and October 2006. The second and third searches were conducted to include recent cases where HL7 has been used as a framework for the development of commu-

444

A. E. Flores et al. / Analyzing the key variables in the adoption process of HL7

nication interfaces in the health care domain. The following key words were used during the search process: (1) HL7, (2) Health Level Seven, (3) health information standard, (4) interconnectivity, (5) software interfaces and (6) communication. The databases ScienceDirect, Proquest 5000, IEEE Xplore and SpringerLink were used as knowledge sources during the search process due to the immediate access for the authors.

tional information and communication standards. Moreover, the inclusion of international standards for exchange of information among health information systems, such as HL7, may have a direct and positive impact in the time processing and performance of medical data exchange. Langer [8] indicated that the implementation of HL7 standard enhances the performance and interoperability of health information systems and diminishes the time required for data exchange and access between hospital units and departments. Moreover, they consider that the incorporation of HL7 messages allows the friendly and easy access to multiple instances of patient's records and clinical information, which permit to enhance the quality of the health care delivery service. Ko et al. [9] agreed that the use of HL7 improves performance and efficiency of connectivity and interoperability among health information systems. However, they pointed out that large implementations, such as integrated institutional web-based application or inter-institutional health information systems, increase the risk of informatics attacks and could provoke loss of robustness, security and flexibility of the systems. Müler et al. [10] indicated that the implementation of communication standards is a cost factor in modern health care systems. This implies that a better and faster exchange of medical information could ameliorate medical care services. They also agree that HL7 framework, in special the Clinical Document Architecture (CDA), allows the development of efficient and well defined interfaces that enhance the exchange of medical documents in local health information structures and diminishes the time of transference.

More than 150 articles were identified during the search process. However, only 63 of them were considered for review based on the relevance over the adoption of HL7 in the health care domain. Three elements were considered as relevant to pick up the list of 63 articles: (1) the article presents information related to the development and implementation of software interfaces for data exchange among health care system, (2) the main standard used for the development was HL7, and (3) the information presented by the authors has been obtained during the implementation of projects since the year 2000. The Table 1 shows the criteria used to select the final list of 33 articles. The scope of these criteria was to narrow the articles selected to those that present the most recent information about HL7 implementation. Table 1 - Selection Criteria Criteria

Considerations

Version

HL7 v. 2.5 and HL7 v. 3.0

Implementation

Modeling, Implementation of Communication Interfaces, Software, and Web-based software and interface.

Message Format

XML

Publishing Date

Year 2000 and after.

Elements Implemented

HL7 messages, new vocabulary, and model representation based on the Reference Information Model (RIM) and/or Clinical Document Architecture (CDA).

In almost all the cases analyzed, the implementation of HL7 has permitted the enhancement of performance and diminution of time during the transaction and data exchange among electronic health information systems. However, the inclusion of security schemes should be considered to ensure the safe delivery of messages and information and overcome possible information threats and loss of robustness of the systems. Adaptability Since its beginning HL7 has been developed as a standard for software interface that should be able to connect different and highly heterogeneous software environments. For this reason adaptability has become a keystone attribute that must be accomplished by HL7.

Results The articles selected were classified into the four implementation categories described in table 1. The analysis was conducted considering the following variables: (1) Benefits/Limitations (performance and time saving, adaptability, and extensibility, modeling and implementation support; (2) Adoption of HL7 in health care; (3) Technical barriers for adoption of HL7

Most of the authors state that HL7 has presented high adaptability to the domains actually included in the standard scope [11][10]. In this sense, the HL7 Reference Information Model (RIM) has been successfully adapted to electronic health records systems, department information system and administrative and financial applications [11]. However, it has presented limited adaptability to nursing information systems, inter-institutional application

Benefit and limitation of HL7 Performance and time saving The performance and time saving during the exchange of data is a main decision variable for the adoption of interna-

445

A. E. Flores et al. / Analyzing the key variables in the adoption process of HL7

and specific health care systems such as general practice and radiology [4]. The Act1 class of the RIM has been unable to represent nursing activities and the HL7 vocabulary is limited for nursing information [4]. The actual data structure and vocabulary definition is unable to map complete data information for General Practice's information systems [12] and, even though, HL7 has demonstrated excellent performance during the exchange of radiology clinical information, it is still limited for large image exchange [13].

According to Fernandez and Sorgente [14], the ad hoc variation of the HL7 Unified Modeling Language (UML) is incompatible with existing standards. Moreover, the HL7 documentation is wide and complex making it difficult to understand. In addition, entities, roles and associations had been represented and structured for implementation not for abstract representation, e.g. roles are job descriptions without security specifications and associations that do not have names or semantic values. Indeed, they had been replaced by separate class representations. Moreover, to add extend class representations HL7 uses arbitrary names based on prefixes of the original classes and not stereotypes as usual in UML representations. In general, HL7 artifacts do not completely follow the UML patterns and software engineering rulers. This issue makes the standard unnecessarily complex for the elaboration of model representations and model extensions.

According to Müler et al. [10] CDA can be easily adapted to overcome local health system requirements such as electronic health records systems, decision support systems and knowledge applications [14]. However, some issues, related to adaptability requirements, efforts and cost to meet inter-institutional needs, should be addressed to provide a better support in the implementation of software interfaces for the exchange of clinical document among different actors of the health care domain [15][10]. They also suggested that CDA can be adapted to different inter-institutional scenarios by including additional data structures and vocabulary that allow overcome the two basic limitations: (1) CDA is limited to the scope of HL7 definitions [16] and any additional extension to the data or vocabulary definition is limited to local solutions [10]. Finally, Müler et al. pointed out that the inclusion of CDA permits diminish the cost associate to the exchange of clinical documents and could enhance the delivery of primary and secondary health care services.

The RIM provides explicit semantic and lexical representations of messages and fields. Additionally, it facilitates the data integration among health care applications, providing structural information models and a health base vocabulary [14]. Moreover, the RIM facilitates the mapping process over basic health care informational representations and model [20]. However, these representations are relatively limited if the standard is applied to some particular health care domains such as nursing systems [4], general practice’s information systems [12] or radiology information systems [21].

Bicer et al. [17] discussed about the necessity to develop message exchange frameworks that provide support semantic interoperability between different versions of HL7 message standard. This is one of the most common problems during the exchange of data between software interfaces developed under both version 2.x and version 3 of HL7.

The HL7 CDA gives a framework for design and interoperability of clinical documents [20]. Furthermore, the CDA provides support and representation for messages based on text, image, sound and multimedia contents, and allows the enhancement of vocabulary and information structure to reach particular requirement. However, it does not provide guides or recommendation for the development of structural or vocabulary extensions and most of the representations require adjustment and modification to local needs [22]. Additionally, any local extension to the model, vocabulary or documents structures must be considered as optional data or field during the interchange of data among health care providers [4]. This implies that, due to the message definition, some relevant information could not be interpreted by the destination bode.

In conclusion, HL7 has few adaptability issues for health information domains such as electronic health record, administrative and financial systems, and departmental information systems. However, due to limitations in the referential data representation and vocabulary, HL7 has restrictions that have to be considered for implementing health information systems in particular domains such as nursing system, general practice and inter-institutional application.

Lebak, Yao and Warren [23] suggest that HL7 should provide a better support for large scale system implementations that consider interconnection among different actors of the health care. It implies the consideration of a framework that includes support for the development and deployment of integrated, interconnected and secure software interfaces. [23], the integration of electronic health records, and provides a wide set of elements that support the modeling and implementation of robust interinstitutional software interfaces based on HL7 standard [15].

Modeling, extensibility and implementation support HL7 allows the incorporation of information structures and extension to the vocabulary and messages specifications. Moreover, the information model and vocabulary should be adapted during the process of software implementation to achieve local needs [4][11]. However, those extensions are limited in exchange of information among different health providers [10].

1

In Summary, the had hoc UML model representation used by HL7 artifacts do not follow the object oriented standards, this makes more complex the standard for develop-

The RIN Act class is used to represent intentional acts that are performed to benefit the patient and associated clinical activity.

446

A. E. Flores et al. / Analyzing the key variables in the adoption process of HL7

Thirdly, the CDA has been developed to provide a framework for document representation and message elaboration based on HL7 standards. However, the CDA framework is in a development stage. This implies that CDA does not provide a complete data representation for some specific health domains or local requirements [11]. Moreover, limitations of HL7 vocabulary and data structure make necessary the development of local solutions, which are not totally compatibles for inter-institutional information exchange [4][11]. In addition, actual vendor’s software does not provide complete support for integration of certain external data, and local implementations are restricted to internal needs [10]. These issues add levels of complexity to the development process and increase the cost if implement HL7-CDA messages interfaces over inter-institutional health information systems.

ers and increases the time and cost associated to the development of HL7 message interfaces. HL7 provides extensibility capabilities. However, extensions of HL7 standard are limited to local implementation. Moreover, inter-institutional implementations, such as Regional Health Information Systems, should consider this limitation to include homogenized message structures and message interpretations. Adoption of HL7 in health care The use and adoption of HL7 allows the implementation of integrated health care systems. In addition, HL7 provides a native and robust interoperability framework for software development and deployment. Moreover, HL7-CDA reduces the cost of moving existing documents to new standards [10] and enhances the work flow between health information systems. For these reasons most authors explicitly agree that HL7 is a recommended and required standard for information exchange among health care applications. However, the adoption of HL7 should consider several issues that should be addressed to the implementation plan. Some of them are adoption limitation over ad hoc UML modeling of HL7 [18], complexity of the implementation over large information systems, high cost, restrictions of vocabulary and the consideration of other communication standards that provide better support over specific domain, e.g. The Digital Imaging and Communications in Medicine (DICOM) for radiology exchange of information [13].

Finally, additional limitations are related the cost and time required for implementation and the complexity of the existent HL7 artifacts [13]. The implementation of HL7 messages based on CDA over XML requires an important amount of time and cost of development. In addition, the deployment of large health information system makes the development and implementation process highly difficult and requires additional resources [14]. In conclusion, HL7 has provided a helpful framework for developing and implementing health information messages interfaces. However, there still exist some issues to address in order to improve the standard.

Technical barriers for adoption of HL7

Discussion

HL7 provides a wide range of guidelines and specification for implementation of data structures and messages for software interfaces among health informatics applications [2]. Instead, HL7 has several technical limitations related to information model specifications, message definitions, document structures and vocabulary applied to specific health care domains.

This paper has presented an analysis based on HL7, both version 2.5 and version 3.0, implementations experiences over different health care domains since the year 2000. Those experiences ratified that HL7 provides a wide range of capabilities for the enhancement of message communication among health information systems. However, those experiences also had made manifest that HL7 has structural and technical limitation that could make difficult the adoption process. Those limitations are related to the message modeling and implementation, data structure representation and vocabulary presented in the RIM and CDA.

First, according to the definitions of HL7 standard, the message should contain a basic set of fields, which must hold the critical information required for exchange; additional information should be provided using the optional fields [1]. This fact does not represent a real inconvenience for local implementations [22][11]. However, this issue could increase the costs and efforts required during the development and deployment of HL7 messages for interinstitutional applications [10].

HL7 standard provides a basic framework for message modeling and implementing. However, in most of the cases analyzed the authors recommend to update the standard to overcome local needs. Furthermore, it is necessary to consider the development of a framework that provides guidelines for the development of inter-institutional message solutions. Additionally, it is also necessary to develop a message exchange framework that enhances the compatibility between version 2.x and version 3.0 HL7 messages.

Second, the RIM has presented issues during the data mapping and development of messages in some health care domains [4][12][20]. According to Danko et al. [4], the RIM class Act is unable to represent complete model structures for nursing information systems. Moreover, they suggested the additional attributes to the RIM-Act class and the enhancement of the HL7 vocabulary to include nursing information. Furthermore, these limitations also affect the development of software solutions in other domains such as general practice [12], and the exchange of referral and discharge letters [11].

Both the RIM and CDA have been implemented and used to enhance the data, information and document exchange among local and inter-institutional health information systems. At the local level both provide a framework for message design and extensibility implementation, and system deployment [1][10]. However, the data definition and vocabulary incorporated to the HL7 standard is limited to

447

A. E. Flores et al. / Analyzing the key variables in the adoption process of HL7

the existing in the RIM and CDA definition [20][10][19]. Moreover, the existing data and vocabulary definitions limit the mapping of data and generation of messages. These limitations can be overcome by incorporating extension to the models and vocabulary at the local level. Nevertheless, it is important to consider that according to the HL7 message standards [1][4], extensions must be considered optional attributes or optional values during the exchange of messages. These issues increase the complexity and cost of the development and implementation of health information software solution at inter-institutional level.

[10]

[11]

[12]

Conclusion

[13]

HL7 has demonstrated to be an important advance in the development of health information software for medical data exchange. However, the implementation of HL7 in specific cases, requires the development of new information models, message model and vocabulary that allows the implementation of those interfaces [4][12][13]. Moreover, the development of communication interfaces on specific scenarios would permit to enhance the actual information structure of HL7 standard. On the other hand, the definition and specification of HL7 message information models, for specific health domain software, allows the implementation of robust software interfaces. These interfaces would enhance the information exchange and interoperability among different local and inter-institutional health care software applications.

[14]

[15]

[16]

[17]

Acknowledgements The author acknowledges the Government of Chile and University of Talca (Chile) for the support of this research.

[18]

References

[19]

[1] Beeler G. HL7 Version 3 – an object-oriented methodology for collaborative standards development. Int. J. of Med. Info. 1998: 48 (1-3 Suppl): 151-161. [2] Henderson M. HL7 Messaging. Texas: Otech Inc, 2003. [3] Hinchley A. Understanding Version 3: A premier on the HL7 Version 3 Communication Standard. Munich: Alexander Mönch Pub., 2005. [4] Danko A, Kennedy R, Haskell R, Androwich I, Button P, Correia C, Grobe S, Harris M, Matney S, and Russler D. Modeling nursing interventions in the act class of HL7 RIM Version 3. J. of Bio. Info. 2003: Inf. 36 (4-5 Suppl): 294-303. [5] Yin RK. Case Study Research: Design and Methods. 3rd Ed. Thousand Oaks: Sage Publication, Inc, 2993. [6] Davis FD. User acceptance of information technologySystems characteristics, user perceptions and behavioral impact. I. J. Man-Machine Stud. 1993: 38 (3 Suppl): 475-487. [7] Davis FD. A critical Assessment of potential measurement biases in the technology acceptance model: three experiments. Int. J. H-C Studies 1996: 45 (1 Suppl): 19-45. [8] Langer S. Open RIMS: An open architecture radiology informatics management system. J. of Digital Imaging 2002: 15 (2 Suppl): 91 – 97 [9] Ko L, Lin j, Huang Ch, Chang J, Lai F, Hsu K, Yang T, Cheng P, Wen Ch, Chen J and Hsich S. HL7 middleware

[20]

[21]

[22]

[23]

framework for healthcare information system. Proceedings of the HEALTHCOM 2006 Müller M, Uckert F, Burle T and Prokosch H. Cross institutional data exchange using the clinical document architecture (CDA). Int. J. of Med. Info. 2005. 74 (2-4 Suppl) 245 – 256. Heitmann K, Schweiger R, and Dudeck J. Discharge and referral data exchange using global standards – the SCIPHOX project in Germany. Int. J. of Med. Info. 2003: 70 (2 Suppl): 195 – 203. Liaw S, Sulaiman N, Pearce C, Sims J, Hill K, Grain H, Tse J, and Mg C. Falls Prevention within the Australian General Practice Data Model: Methodology, Information Model, and Terminology Issues, JAMIA 2003: 10 (5 Suppl): 425-432. Um K, Kwak Y, Cho H, and Kim I. Development of an HL7 interface engine, based on tree structure and streaming algorithm, for large-size messages which include image data. J. of Comp. Methods and Programs in Biomedicine 2005: 80 (2 Suppl) 126-140. Katirai H and Ulrich S. Unlocking the value of clinical information: What you need to do now to enjoy the benefits in the future. Althoff KD et al. eds. WM 2005, LNAI 3782. Berlin Heidelberg: Springer-Verlag, 2005; pp. 330-338. Berler A, Pavlopoulos S, and Koutsouris. Design of an interoperability framework in a regional health system. Proceeding of the 26th Annual conference of the IEEE EMBS: 2004: 3093-3096. Dolin R., Alchuler L., Boyer S., Beebe C., Behlen F., Biron P. and Shavo A. HL7 Clinical Document Architecture, Release 2, JAMIA 2006: 7 (4): 30-39. Bicer V, Laleci G, Dogac A, and Kabak Y. Artemis Message Exchange Framework: Semantic interoperability of exchange message in health care domain. SIGMOD Records 2005:34(3 Suppl): 71-76. Fernández E and Sorgente T. An analysis of modeling flaws in HL7 and JAHIS. Proceedings of the 2005 ACM Symposium on Applied Computing. Orgun B. and Vu J. HL7 ontology and mobile agents for interoperability in heterogeneous medical information systems. Computers in Biology and Medicine 2006: 36 (7-8 Suppl): 817-836. Lyman J, Boyd J, Dalton J, and Egyhazy C. Applying the HL7 Reference Information Model to a Clinical Data Warehouse. IEEE 2003. Tellis W., Androiole P., Jovais C., and Avrin D. RIM Minus PACS Equals Film. Journal of Digital Imaging: 2002: 15 (1 Suppl): 20-26. Bilykh I, Jahnke J, McCallum G, and Morgan P. Using the Clinical Document Architecture as open data exchange format for interfacing EMRs with clinical decision support system. Proceeding of the 19th IEEE Symposium on Computer Based Medical Systems: 2006: 3093-3096. Lebak J. Yao J, and Warren S. HL7 – Compliant Healthcare Information System for home monitoring. Proceedings of the IEEE EMBS, 2004.

Address for correspondence Alejandro E. Flores Health Informatics Research Centre, University of Wollongong, Australia. Phone: +61 4221 3103

Email [email protected].

448

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

An XML Model of an Enhanced Data Dictionary to Facilitate the Exchange of Pre-Existing Clinical Research Data in International Studies Stephany N. Dudaa, Clint Cushmanb, Daniel R. Masysa a

Vanderbilt University, Nashville, USA University of California, San Diego, USA

b

collected the data independently as part of routine patient care. Successful harmonization of diverse, global data requires additional information that is known locally but not generally included in study records or patient care databases. Such metadata, or “data describing data,” may help answer the following questions that arise when already-existing data are considered for combined analysis:

Abstract Pre-existing clinical research data sets exchanged in international epidemiology research often lack the elements needed to assess their suitability for use in multi-region meta-analyses or other clinical studies. While the missing information is generally known to local investigators, it is not contained in the files exchanged between sites. Instead, such content must be solicited by the study coordinating center though a series of lengthy phone and electronic communications: an informal process whose reproducibility and accuracy decays over time. This report describes a set of supplemental information needed to assess whether clinical research data from diverse research sites are truly comparable, and what metadata (“data about the data”) should be preserved when a data set is archived for future use. We propose a structured Extensible Markup Language (XML) model that captures this information. The authors hope this model will be a first step towards preserving the metadata associated with clinical research data sets, thereby improving the quality of international data exchange, data archiving, and merged-data research using data collected in many different countries, languages and care settings.

•

Are the observations stored in a given variable in one data set equivalent to those stored in a variable in another data set, or are the variables similar but not identical for the purposes of a particular form of analysis? • What computer program has been used to store and format the data? • What is the “normal” range for this observation? • What are the valid values that this variable may take? • Who owns the data, and under what circumstances can it be used for analysis? • From what geographic region are the data derived? • In what language are the text-based data represented? • What quality control measures have been applied to the data set to ensure that it is accurate and complete? Historically, limited metadata related to these questions have been recorded in narrative form or made available by person-to-person communication with the owners of the research data. Over time, however, this information may be lost in the shuffle of old project records and in job transitioning among investigators and staff. The information loss resulting from reliance on memory, paper memos, and generally uneven documentation affects how accurately data sets can be combined for joint analysis, and how reliably data may be reused years after their creation [3].

Keywords: programming languages, software design, knowledge representation (computer), database management systems

Introduction The meta-analysis of clinical research and community health data from different regions of the world is the focus of an increasing number of international collaborations [1]. Merging such diverse data is particularly challenging in medicine, since geographic differences in patient and disease characteristics, medical resources, language, and cultural beliefs can have a profound effect on the comparability of clinical information [2]. Varying regional standards for measurement units, “normal” ranges, treatments and procedures, and medical definitions may prevent data from being interpreted correctly. These differences are magnified when each participating site has

If such “hidden” metadata can be identified, recorded, and transmitted alongside clinical research data sets in a wellstructured format, it may reduce information loss and enhance the usability of research results. Extensible Markup Language (XML) is an established standard for this type of structured information exchange: its hierarchical text format is flexible, simple to implement, and easily interpreted by both humans and computers [4]. Web-based

449

S. N. Duda et al. / An XML Model of an Enhanced Data Dictionary

applications and fields such as business, chemistry, and mathematics have employed XML extensively for information sharing, encoding semantic content, and data modeling [5-7]. In biomedicine, XML has been used to encode clinical guidelines, support electronic messaging between hospitals, and annotate nucleotide sequences [811]. The structure and syntax of such XML models is often described using XML Schema, the World Wide Web Consortium’s recommended means of expressing XML schemata[12].

developed systems of treatment centers in the developing world. While some sites may upload database files to their regional coordinating centers, others will struggle to deliver Microsoft Excel spreadsheets or plain text files. The internal structure of the data exchange format must accommodate all possibilities. Document Independence: The metadata solution should not rely on any single data management system to repackage data and metadata into a cohesive unit. Instead, the metadata document should accompany data files originating from preexisting local databases – such as those maintained at IeDEA’s participating sites – when they are submitted to collaborators or study coordinating centers.

We report here the development of methods for capturing and storing clinical research metadata in an XML format. Preliminary work will focus on data collected by the International Epidemiologic Databases to Evaluate AIDS (IeDEA), an initiative of the U.S. National Institutes of Health to “establish international regional centers for the collection and harmonization” of HIV data and the management of region-wide analyses [13]. Participating sites include hospitals and clinics located throughout the Americas, Asia, Africa, and Oceania. The IeDEA collaboration will focus on regional analyses in its first three years, after which sites may choose to submit their data sets worldwide data pooling and analysis.

Transparency: The syntax for recording research metadata should be easy to generate and interpret, for humans and computers alike. The information should be accessible to individuals without a specialized data viewer and it should be possible to automate part of the metadata documentation process. Design objectives for model content Data Descriptors: The model describing a clinical research data set should contain traditional data dictionary content, such as variable names, ranges, datatypes, and special values for missing or pending measurements. It is particularly important to capture this information for clinical databases in developing countries, where often no formalized data dictionary exists.

The majority of IeDEA member sites are based in developing countries, where the process of collaborative research differs significantly from that in developed nations [14]. Consequently, an appropriate data model must take into account the information exchange needs and capabilities of clinical research centers in the developing world. We assert that a simple, structured model designed to capture and preserve clinical research metadata can improve the quality of international data exchange and merged-data research, as well as preserve study metadata for future research projects.

Measurement: The documentation for numeric values such as laboratory results should include the measurement unit and a description of the measurement method used. Geography: The model should include some marker of location that indicates, geographically, the site(s) where the data have been collected and the location of the sponsoring or coordinating organization. This geographic code should be able to pinpoint the country, region, or city from which the data originated, but also allow fuzzy mappings to protect the privacy of individuals in small villages or accommodate the address-less locations of traveling or wilderness clinics.

Approach to model development Six IeDEA-participating sites submitted descriptions of their pre-existing patient care and research databases. We reviewed these descriptions and developed a list of elements required to represent the characteristics of the content. Approximately thirty participants, including statisticians, HIV researchers, epidemiologists, database managers, and international clinical trials specialists, reviewed the proposed element list and suggested additional fields they might use to determine the compatibility and usefulness of different data sets. The resulting metadata fields were categorized and mapped to a collection of XML examples and a corresponding XML Schema.

Contacts: The name, contact information, and website of the organization (clinic, hospital, research center) responsible for producing and maintaining the clinical research data should be recorded in the metadata, along with a list of persons responsible for preparing the data or metadata. Terms of Use: A section of text should clarify under what conditions the data set may be used.

The authors and collaborators identified the following design objectives for the metadata model, based on the input of the researchers and descriptions of locally-developed databases at IeDEA sites.

Quality: The data exchange model should include a description of the quality assurance measures applied to each variable as it exists in the current data set. Examples of quality assurance tests include range and valid values checking (either manually or using an algorithm), double data entry with comparison, and source document verification.

Design objectives for model Flexibility: An ideal model for collaborative international data exchange must be adaptable to a range of data storage formats, from elaborate databases in fully-electronic clinical trials data management systems to the locally-

Ontologies: The XML model should allow users to map locally defined variables to existing concepts in multiple controlled vocabularies and vocabulary versions. The

450

S. N. Duda et al. / An XML Model of an Enhanced Data Dictionary

resulting structure should be able to represent compound concepts, also known as data tuples or post-coordinated terms.

The tag, in contrast, encompasses the metadata related to the observations in the dataset. The four fields within the tag include , which contains the product name and version of the database software used, and , which contains the name, contact information, and geographic location of the hospital, clinic, or research institution that owns the data. The syntax of the hierarchy is shown in Figure 2 - XML description of the organization that owns the data set2. Geographic location markers, or geocodes, are recorded in latitude and longitude, as these measures are easy to compute using publicly accessible web tools such as EarthTools or Getty Thesaurus of Geographic Names Online [16, 17]. Data owners may also stipulate a Terms of Use clause, , that restricts the applications of their data.

Proposed model The model overview contains XML samples and brief descriptions of the more complex elements in the document hierarchy. The full XML Schema is available online at http://ccasanet.vanderbilt.edu/xml.php. The website is hosted by CCASAnet, the Caribbean, Central, and South America Network for HIV research, which is one of seven IeDEA-funded regional coordinating centers and is located at Vanderbilt University [15].) Model overview The XML structure has a single root element, <metadata>, that encloses all other metadata-related elements. The first levels of the hierarchy are shown in Figure 1 - Root structure of the XML model for exchanging pre-existing international research data1:

1 2 3 4 5 6 7 8 9 <email> 10 11 12 13 14 <description> 15 16 17 18 19 20

1<metadata> 2 3 ... 4 See Figure 2 - XML description of the organization that owns the data set2 5 6 7 ... 8 9 See Figure 3 - XML structure for storing information about the data set’s patient cohort3 10 11 12 13 14 15 See Figure 4 - XML structure for variables in a data set4 16 17 18 19 ... 20 21

Figure 2 - XML description of the organization that owns the data set A third element, , on line 6 of Figure 1, contains the names, titles, roles, and contact information for people associated with the creation, maintenance, or packaging of the data set. A high-level description of the data set, a description of the study cohort, and a set of geocodes demarking the data’s source regions are part of the fourth and final descriptive metadata element, , shown below:

Figure 1 - Root structure of the XML model for exchanging pre-existing international research data

1 2 <description> 3 <study_cohort> 4 5 ... 6 7

As seen above, the XML elements are partitioned into two sections of meta-content: metadata related to the data set as a unit (, line 2), and metadata related to the observations contained in the data set (, line 12). Metadata associated with the data set unit can answer questions such as “what software was used to maintain this database?,” “who owns this data?,” and “whom should I contact if I have questions related to this data set?” This information is captured between the tags.

Figure 3 - XML structure for storing information about the data set’s patient cohort The second section of the data exchange model describes the data files that accompany the XML metadata docu-

451

S. N. Duda et al. / An XML Model of an Enhanced Data Dictionary

ment. Each file is described within its own tag (Figure 1, line 13), and the set of such tags constitutes the structure of the model. These files may be database tables, spreadsheets, or even text files, but they contain variously formatted table information. All variables or equivalent table columns appearing in a data file are represented in the tag. The structure is detailed in Figure 4 - XML structure for variables in a data set4.

6 7 8 9 10 11

Figure 5 - XML structure for internal and external valid values lists The tag set, highlighted in Figure 5 XML structure for internal and external valid values lists5, specifies a list of names, codes, or numbers considered to be valid content for a given variable. An inline list of valid values is often one developed locally, by the database designers. In this case, all valid codes are specified between the tags, beginning on line 4 of Figure 5. A valid value set of type file indicates a value list included in a separate document transmitted along with the data set. A reference to external authorities suggests the valid values for this field are governed by controlled vocabularies such as SNOMED, ICD-9, or LOINC.

1 2 3 <description> 4 5 6 7 <measurement>... 8 9 10 11 <description> 12 13 14 15 16 17 ... 18 <missing>... 19 ... 20 21 See Figure 5 - XML structure for internal and external valid values lists5 22 23 24 25 26 See Figure 6 - XML structure for ontology mappings 27 28 29

Each also contains a structure that allows users to associate the variable semantics with one or multiple ontologies, specified by the vocabulary and version tags. The structure is shown in Figure 6 - XML structure for ontology mappings6. Multiple tags within a single content mapping indicate a concept tuple, or post-coordinated term. 1 2 3 4 5 6 7 8

Figure 6 - XML structure for ontology mappings The hierarchy of subsequent closing tags is specified in Figure 1 - Root structure of the XML model for exchanging pre-existing international research datagure 1. Many structures, such as , are optional, and any text field can take a language attribute. Repeatable elements such as or are fully specified in the XML Schema available online.

Figure 4 - XML structure for variables in a data set The XML fields in lines 2 through 6 of Figure 4 include a variable name, description, datatype, type length, and count. The <measurement> field, shown on line 7, records the measurement unit, if applicable, and a description of the measurement method used to ascertain a numeric or subjective value. The subsequent tag (line 8) encloses a list of quality assurance measures applied to the variable, along with the quality test date and result. Special value codes for missing and pending data and counts of these code occurrences are recorded in the field beginning on line 16, along with valid ranges for numeric variables and sets of valid values, discussed below.

Discussion The XML model described in this paper meets the specified design objectives and constitutes a complete structure for international clinical research data exchange. The metadata fields include traditional data dictionary values, mappings to controlled vocabularies, and markers of geographic location, as well as specifics about local data entry and storage methods, definitions, measurements, quality control procedures, data ownership, and use restrictions. In addition, the model employs a flexible data exchange language (XML) that is system-independent and interpretable by both humans and computers.

1 2 3 <description> 4 5

452

S. N. Duda et al. / An XML Model of an Enhanced Data Dictionary

By making this metadata uniformly available among IeDEA collaborators, researchers can analyze count and content metadata to determine in advance whether sufficient data are present to perform an analysis across global regions. Thus, the decision to make the data itself available for pooled analysis still remains with the owners of the data, while still allowing outside researchers to see what categories of information are available.

[3] Detmer DE. Building the national health information infrastructure for personal health, health care services, public health, and research. BMC medical informatics and decision making. 2003 Jan 6;3:1. [4] Wyke RA, Rehman S, Leupen B. XML Programming (Core Reference). Redmond, Washington: Microsoft Press 2002. [5] Medjahed B, Benatallah B, Bouguettaya A, Ngu AHH, Elmagarmid AK. Business-to-business interactions: issues and enabling technologies. Vldb Journal. 2003 May;12(1):59-85. [6] Miner R. The Importance of MathML to Mathematics Communication. Notices of the American Mathematical Society 2005:532-8. [7] Murray-Rust P, Rzepa HS. Chemical markup, XML, and the Worldwide Web. 1. Basic principles. Journal of Chemical Information and Computer Sciences. 1999 NovDec;39(6):928-42. [8] Achard F, Vaysseix G, Barillot E. XML, bioinformatics and data integration. Bioinformatics. 2001 Feb;17(2):115-25. [9] Araki K, Ohashi K, Yamazaki S, Hirose Y, Yamashita Y, Yamamoto R, Minagawa K, Sakamoto N, Yoshihara H. Medical markup language (MML) for XML-based hospital information interchange. Journal of medical systems. 2000 Jun;24(3):195-211. [10] Dolin RH, Alschuler L, Behlen F, Biron PV, Boyer S, Essin D, Harding L, Lincoln T, Mattison JE, Rishel W, Sokolowski R, Spinosa J, Williams JP. HL7 document patient record architecture: an XML document architecture based on a shared information model. Proceedings / AMIA Annual Symposium. 1999:52-6. [11] Shiffman RN, Karras BT, Agrawal A, Chen R, Marenco L, Nath S. GEM: a proposal for a more comprehensive guideline document model using XML. J Am Med Inform Assoc. 2000 Sep-Oct;7(5):488-98. [12] Walmsley P, Fallside DC. XML Schema Part 0: Primer Second Edition. W3C Recommendation. 2004 [last accessed 2006 Dec 1]; Available from: http://www.w3.org/TR/xmlschema-0/ [13] IeDEA: Project Overview. 2006 [last accessed 2006 November 29]; Available from: http://www.iedea-hiv.org/about/ [14] Duque RB, Ynalvez M, Sooryamoorthy R, Mbatia P, Dzorgbo DBS, Shrum W. Collaboration paradox: Scientific productivity, the Internet, and problems of research in developing areas. Social Studies of Science. 2005 Oct;35(5):755-85. [15] CCASAnet: Caribbean, Central and South America Network. 2006 [last accessed 2006 Nov 29]; Available from: http://ccasanet.vanderbilt.edu/index.php [16] EarthTools. [last accessed 2006 Dec 3]; Available from: http://www.earthtools.org/ [17] Getty Thesaurus of Geographic Names Online. [last accessed 2006 Dec 3]; Available from: http://www.getty.edu/research/conducting_research/ vocabularies/tgn/

Although current database modeling and healthcare messaging standards exist, these exchange formats are either too rigid or too complex to use to describe data collected outside the controlled environment of a clinical trial. Data dictionaries are often focused on database structure and fail to capture information outside of the database syntax, while medical messaging and data exchange formats such as HL7 are designed for the exchange of patient care documents among electronically-enabled hospitals and care providers, rather than the merging of clinical research data. We intend to validate the XML model presented here using real-world data sets contributed by IeDEA member sites. In areas where the model is shown to be not sufficiently flexible, we will update the schema to accommodate new categories of metadata. In addition, we intend to develop a tool to automatically populate most fields of the XML model by pre-processing the source database, and create a secure web interface for online maintenance of the metadata. This will reduce the burden of documentation placed on the data set owners. We will test the usefulness of the metadata and its XML model for the harmonization and meta-analysis of HIV electronic clinical research data from different countries, languages and cultural traditions. The work described here represents a two-fold approach to improving the documentation that accompanies clinical research data in international studies. We have described seven categories of metadata that researchers find useful in determining the semantics and compatibility of pre-existing research data sets, and have formulated an XML model to document this metadata in a flexible, structured format. Future work will assess whether this innovation will improve the quality of international data exchange and merged-data analyses, thereby creating a more sustainable model for research collaborations worldwide. Acknowledgments Supported in part by NIH Cooperative Agreement 1 UO1 AI069923-01, International Epidemiologic Databases to Evaluate AIDS (IeDEA), Latin American Region. The authors wish to thank Bryan Shepherd, Paul Biondich, William Tierney, Claudio Faulhaber, and Bill Lober for their reviews of and comments on the proposed metadata model.

Address for correspondence

References

Stephany Duda, 4th Floor Eskind Biomedical Library, 2209 Garland Ave, Nashville, TN 37212, USA; [email protected]

[1] Luukkonen T, Persson O, Sivertsen G. Understanding Patterns of International Scientific Collaboration. Science Technology & Human Values. 1992 Win;17(1):101-26. [2] Ioannidis JP, Lau J. Uncontrolled pearls, controlled evidence, meta-analysis and the individual patient. Journal of clinical epidemiology. 1998 Aug;51(8):709-11.

453

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Framework for Clinical Data Standardization Based on Archetypes Jose A. Maldonadoa, David Monera, Diego Tomása, Carlos Ánguloa, Montserrat Roblesa, Jesualdo T. Fernándezb a

Biomedical Informatics Group, ITACA Institute, Technical University of Valencia, Spain b Departamento de Informática y Sistemas, University of Murcia, Spain

the EHR architecture (EHRA) used to communicate data. Currently there are several international organizations [1] working on the definition of an EHRA. Health Level 7 [2] supports two message protocols: HL7 Version 2 and HL7 Version 3. The technical committee 251[3] of the European Committee for Standardization is working on a new full European Standard and future ISO norm for the communication of the EHR called EN13606 [4][5]. The OpenEHR foundation [6] maintains an EHR architecture designed to support the constructions of distributed, patient-centered, life-long, shared care health records.

Abstract Standardization of data is a prerequisite to achieve semantic interoperability in any domain. This is even more important in the healthcare sector where the need for exchanging health related data among professional and institutions is not an exception but the rule. Currently, there are several international organizations working on the definition of electronic health record architectures, some of them based on a dual-model approach. We present both an archetype modeling framework and LinkEHR-ED, an archetype editor and mapping tool for transforming existing electronic healthcare data which do not conform to a particular electronic healthcare record architecture into compliant electronic health records extracts. In particular, archetypes in LinkEHR-ED are formal representations of clinical concepts built on a particular reference model but enriched with mapping information to data sources which define how to extract and transform existing data in order to generate standardized XML documents.

Due to the complexity and the continuous evolution of the health domain a new approach for the development of EHR systems has been proposed. The methodology is known as the dual model methodology which is based on a clear separation between information and knowledge. The former is described through a Reference Model (RM) that contains the basic entities for representing any entry in an EHR. The latter is based on archetypes, which are formal definitions of clinical concepts in the form of structured and constrained combinations of the entities of a RM. Examples of Dual Model EHRA are CEN/TC251 EN13606 and openEHR.

Keywords: medical records, archetypes, standardization, data translation, information systems

Currently, in most organizations there is a vast amount of health related data that does not conform to an EHRA that need to be converted into standardized EHR extracts in order to be exchanged with other organizations. In this paper we address the problem of how to use archetypes to make public these legacy data in the form of standardized EHR. We argue that archetypes are suitable for this purpose. First, archetypes allow the formal description of the semantics of legacy and un-standardized health data. On the other hand, by mapping the structure of archetype definitions to the elements of the data sources, it is possible to generate normalized EHR extracts compliant with the underlying RM.

Introduction Health care is a sector where the need for sharing information is the norm rather than the exception. However, the health data of one patient is usually scattered among the different health facilities where she/he has been attended. This leads to distributed and heterogeneous data resources, all of them containing health data, making the exchange of data across systems and organizations very difficult. This situation has created a large gap between the potential and actual value of the information content of electronic health records (EHR). Closing this gap by making efficient use of the health data held by these systems, could improve significantly patient care, clinical efficiency and empower research activities.

Dual model approach The dual model approach distinguishes a reference model and archetypes. A reference model is an object oriented model that is used to represent the generic and stable properties of health record information. It comprises a small set of classes that define the generic building blocks to construct EHRs. It

Due to the special sensitivity of medical data and the wide range of ethical and legal constraints, data exchange must be done in a meaningful way, avoiding all possibility of misunderstanding or misinterpretation. The faithful communication in EHR crucially depends on the standardization of

454

J. A. Maldonado et al. / Framework for Clinical Data Standardization Based on Archetypes

specifies how health data should be aggregated to create more complex data structures and the context information that must accompany every piece of data in order to meet ethical and legal requirements.

Data model Archetypes impose a hierarchical structure to the EHR, therefore we have chosen a data model based on trees with labeled nodes to formalize their data instances. It is similar to the models presented in [9,10] but our data model supports both ordered and unordered nodes. Although archetypes do not impose an order on class attributes or attribute values, it is possible to define ordered multivalued attributes such as attributes whose value is a list.

An archetype is a formal definition of a distinct, domainlevel concept in the form of structured and constrained combinations of the classes of the RM. What is important is that for each concept in the domain we want to use, a definition can be developed in terms of constraints on structure, types, values, and behaviors of RM classes. Basically, archetypes are means for providing semantics to data instances that conform to some reference model by assuring that data obey a particular structure (combination of classes of the reference model) and satisfy a set of semantic constraints. This is achieved by linking data structures and content to knowledge resources as terminologies and ontologies. Their principal purpose is to provide a powerful, reusable and interoperable way of managing the creation, description, validation and query of EHRs.

The representation of data instances (in our context EHR extracts compliant with a RM) is straightforward. Each object is described by a data tree. The root node is labeled with the class name and has one child for each attribute. The children are labeled with the attribute names and each of them has one child labeled with the corresponding type (class) name. This mechanism is repeated iteratively. Atomic values are represented by a leaf node labeled with a value. Schema model For the representation of the definition section of archetypes we have developed a type system that allows the specification of decidable sets of data trees, i.e., in our context set of instances of the RM. We assume the existence of a finite set of primitive types C, i.e., the set of primitive types defined in the reference model, an infinite set of type variables T disjoint with C and an infinite set P of label predicates.

ADL (Archetype Definition Language) [7] is a formal language for expressing textually archetypes developed by OpenEHR that has also been adopted by CEN. ADL is a textual language for specifying constraints on data instances of an RM in a formal way. An archetype expressed in ADL is composed of four main parts: header, definition, ontology and revision history. The header section contains the archetype metadata. In the definition section is where the modeled clinical concept is represented in terms of a particular RM class. This description is built by constraining several properties of classes and attributes, such as existence, occurrences or cardinality or by constraining the domain of atomic attributes. It is important to notice that in this section only those entites that need to be constrained should appear. The ontology section is where the entities defined in the definition section are described and bound to terminologies. Finally the revision history section contains the audit of changes to the archetype.

Definition 1. A multiplicity list is a regular expression of the form:

t 1( l1 : u 1 ) . . .t n( l n : u n ) Where nt1, tiÎCÇT, li£ui and

(1)

( li :ui ) i

li i

t

li 1 i

t |t

ui i

| ...| t

Definition 2. A constrained multiplicity list (CML) is language definition expression of the form:

t

Archetype modeling The current ADL specification is not precise enough regarding archetype specialization; this hinders a precise understating of archetypes and their implementation. As a consequence our first work was to define a precise archetype modeling framework as a prerequisite for implementing tools providing enhanced support for archetypes. Since our main concern is the generation of standardized EHR extracts we have focused on the data definition facet of archetypes. We view the definition section of archetypes as a database schema which describes subset of instances of a class from a particular RM. In this section we present briefly the data model used to describe data instances (EHR extracts) and the schemas that formalize the definition section of archetypes, the composition and specialization of archetypes and the relationship between a business concept and its archetypes and between an archetype and its instances. An extended formal definition and additional examples can be found in [8], due to lack of space these are omitted here.

Where

( l1 :u 1 ) 1

( l1 :u1 ) 1

t

...t

...t n( l n :u n )

( ln :un ) n

| l :u |



(2)

is a multiplicity list, ldu, n

n

¦l

i

tu

i 1

¦u

i

and

i 1

tl .

Intuitively the language generated by a constrained multiplicity list is composed by all the word defined by the regular expression whose length (number of symbols) is between l and u inclusively. As an example consider the CML 1:2 1:3 |3:4| which defines the language {ABB,

A

B



ABBB, AAB, AABB}. We formalize the definition section of archetypes as a set of type definitions (a schema). Definition 3. A type definition has either the form t:=lt or t:=lt{rt}. Where t is a type name, lt is a label predicate and rt is a CML over the set the type names and primitive types.

455

J. A. Maldonado et al. / Framework for Clinical Data Standardization Based on Archetypes

A type definition has two parts. The first is a label predicate that describes the valid labels of nodes. The second is a CML that is used to describe the sequence of children that a node may have. An expression of the form t:= lt defines a sets of data trees t whose root node is labeled with a label that satisfies lt, its children are ordered and are describe by rt. On the other hand, an expression of the form t := lt{rt} defines a set of data trees t whose root node is labeled with a label that satisfies l, its children are unordered and at least one permutation is described by r.

|1..1| otherwise, and the regular expression is of the form t1(0:1)·… tn(0:1) where t1,..tn are the alternative types. Finally, domain constraints on primitive types are straightforwardly modeled as label predicates. As an example, let us consider the following ADL expression: PERSON[at001] matches { addresses cardinality matches {1..*} matches { ADDRESS[at002] occurrences matches {0..1} matches {…} ADDRESS[at003] occurrences matches {0..*} matches {…} }

Definition 4. A schema is a set of type definitions, one of which must be declared to be the root type, i.e. whether roots of data trees can be assigned this type.

Which can be modeled by the following set of type definitions:

We need to define the semantics of a schema, i.e. the set of data tree that it models. Intuitively, a data tree D conforms to a schema S if it is possible to assign to every node d of D a type Ti from S, and d satisfies the label predicate and the CML of Ti.

^

1:1 PERSON [at 001]: is _ PERSON ( X ) taddresses

^

^

person  X t

1:1

(1:1) addresses name

t



2:2

1:1

`

 0:1  0:* taddresses : is _ addresses ( X ) t ADDRESS [ at 002] .t ADDRESS [ at 003]

Both RM and definition sections of archetypes can be modeled as a schema. Each type in the schema represents an RM/archetype entity, i.e. a class or an attribute, and the CML represent its structure. The label predicate describes the name of the entity or the valid domain of atomic attributes. As an example consider the class PERSON from Figure 1. It can be formalized by the type definition: tPERSON := is_





1:*

`

t ADDRESS [ at 002] : is _ ADDRESS ( X ) ^...` t ADDRESS [ at 003] : is _ ADDRESS ( X ) ^...` Archetype specialization Archetypes can be defined by further constraining other archetype, i.e., by specialization, in order to obtain a more adequate or fine grained representation of a clinical concept. An archetype is specialized by providing narrower constraints on data. The overall idea is that all data instances that conform to the more specialized archetypes also conform to the more general but there can be data instances of the more general archetype which are not data instances of the specialized archetype. Multiple inheritance is not allowed. We formalize the inheritance relationship between archetypes by means of the subsumption relation [10], i.e., an archetype A is more general than archetype B if A subsumes B. The subsumption relation is also used to formalize the relationship between business concepts and archetypes, we say that an archetype A specialized a business concept B if B subsumes A. What makes the proposed subsumption relation interesting is that it not only captures the containment relationship between the set of data instances defined by two archetypes, but also captures some of the structural relationship between node objects from both archetypes. Intuitively, subsumption is based on defining a mapping function () between types, on inclusion between both label predicates and languages defined by the CML of these types. All these can be translated to archetype specialization. Subsumption mappings specify specialization relationships between node objects and attribute object from parent and child archetypes. This is compatible with the syntactical rules used in ADL to specify the specialization of node object. In ADL specialization of coded concepts, e.g. PERSON[at0001], is indicated by using the same root, followed by an extension, e.g. PERSON[at0001.1]. Note that this defines partly the subsumption mapping, e.g. (PERSON[at0001.1]) = PERSON[at0001]. On the other hand,

`

where is_person(X) is a unary predicate that is true only when X is equal to the string “person” and taddresses and tname are the types that, respectively, model the attributes addresses and name.

Figure 1 - Example of OO model Let see how different types of archetype constraints are modeled with our type system. Existence constraints of attributes can be easily expressed by the CML attached to the type that models the attribute. Cardinality and occurrence constrains are mutually related. Actually, it only makes sense to constrain the occurrence of node objects that are inside a block introduced by a multi-valued attribute. Cardinality constraints can be modeled by the length constraint of CML while occurrence constraints by regular expressions. Alternative constraints for monovalued attributes can be expressed by a CML whose length constraint is equal to |0..1| if the attribute is optional or to

456

J. A. Maldonado et al. / Framework for Clinical Data Standardization Based on Archetypes

inclusion of label predicates assures both that only class and attribute names from the reference model are used in archetypes, and the domain of atomic attributes in specialized archetype are a subset of the domain in the parent archetype. Finally, CML controls that super-types and subtypes have the same structure.

with the RM. The query extracts the relevant information and for each different combination of values of the identification attributes a new instance of the class is generated.

As stated before, RM and the definition section of archetypes can be modeled by the proposed type system. One interesting consequence is that “archetypable” classes of RMs can be considered as archetypes (from a data definition point of view). Therefore, the same logic can be applied both to the specialization of an exiting archetype or to the definition of a new one by constraining a RM class. This brings about the possibility of building flexible archetype editors capable of working with several RM or different version of the same reference model. This possibility has been explored in LinkEHR-Ed.

Archetype designers are responsible of defining the atomic attribute mappings and the system tries to generate [11] from them a set of candidate class mappings by taking into account the structure of the RM entity being used, the archetype constraints and the integrity constraints of data sources. This approach alleviates the work of defining how to populate archetypes since it is easier for the designer to indicate which data elements of the data sources are relevant to a certain archetype attribute, rather than to specify the possible complex query required to extract and transform all the relevant information. As a result, an XQuery expression is generated, which transforms the XML view of the data source into an XML document that satisfies the constraints imposed by the archetype and at the same time is compliant with the RM.

Mapping and data translation

Results

Since the health data to be made public resides in the underlying data sources, it is necessary to define some kind of mapping information that links entities described in the archetype to data elements in data repositories (e.g. tables and attributes in the case of relational data sources). In the health care domain very few generic EHR data transformation efforts exist. Commercial tools with limited capabilities exist but they are mainly focused on the generation of HL7 v2.x or EDI messages and none of them supports archetypes. Furthermore, definition sections of archetypes can not be represented by XML schemas. Thus, current tools for data translation for XML schemas can not be used for this purpose.

LinkEHR-Ed (http//pangea.upv.es/linkehr) is a visual tool implemented in Java under the Eclipse platform which allows the edition of archetypes that can be based on different RMs, mapping specification between archetype and data sources and the generation of data conversion scripts which generate XML documents compliant with the RM. Figure 2 describes the overall edition process. LinkEHR Ed is composed of four main components. Reference model manager. In LinkEHR-Ed new reference models expressed as a W3C XML Schema can be imported at any time. Note that the XML Schema is supposed to describe the XML documents compliant with the RM. The import process generates archetype representations of the RM archetypable classes that will be immediately available as basis for the creation of archetypes by means of specialization. Currently, many of the characteristic of W3C XML schemas are supported, such as all the data types, name spaces, imports and includes (reference models can be defined by several files) and several structures such as complex and simple types, elements, attributes, inheritance by extension and restriction, sequence, choice, all, attributes, patterns and groups and their respective facets. Two reference models have been imported and used successfully: EN13606 and OpenEHR. The XML schema of EN13606 schema has been developed by us from the UML model due to the lack of an official one. For OpenEHR the official schema (actually a set of XML schemas) has been used [6]. This feature has been very useful in keeping in pace with their evolution without modifying a single line of code.

In our scenario an archetype is considered to be a view that provides abstraction in interfacing between the data sources and the RM used to communicate the EHR extracts. Since EHR extracts have an inherent hierarchical structure, we have chosen XML as canonical data model, i.e. data sources are viewed a XML documents. There exists two kinds of mappings: atomic attribute mappings and class mappings. Atomic attribute mappings define how to obtain a value for an atomic attribute of an archetype by using a set of values from the data sources. For this purpose rules relating a set of source paths to an archetype path identifying an atomic attribute are used. This kind of mappings preserves node paths from the root, i.e. the node contexts. It is possible for an archetype attribute to have more than one mapping and it also possible to utilize functions involving more than one source path (for instance the addition of the value of two source attributes). A rule may also contain a condition specifying the subset of values of the data source that can be used to compute values. Obviously, there must be at least one of this kind of mapping for each mandatory atomic attribute.

Semantic validation module. In LinkEHR-Ed only the logic that guides archetypes specialization, which is based on the subsumption relationship described before, is hard coded. This module, given an archetype, tests whether the archetype is valid with respect the RM entity that it constrains (i.e., it is subsumed by the RM entity) or in the case that there exists a parent archetype it test that its constraints are narrowed that those of the parent archetype (i.e., it is subsumed by the parent archetype).

On the other hand, for each constrained class there exists a class mapping which contains both the query to be used to retrieve all the data necessary for generating data instances and the set of attributes that identify univocally the class instances. The combination of both components allows the conversion from source data to XML documents compliant

457

J. A. Maldonado et al. / Framework for Clinical Data Standardization Based on Archetypes

Mapping module. It is in charge of managing data sources schemas, attribute mappings and the generation of candidate class mappings. Given a candidate class mapping, it generates an XQuery expression that outputs standardized XML EHR extracts satisfying all the constraints stated in the archetype.

while providing a means for making public clinical information in the form of standardized EHR extracts, hiding technical details, location and heterogeneity of data repositories. Therefore, we use archetypes as a semantic layer over the underlying databases associating them with domain specific semantics. LinkEHR-Ed combines in an easy manner the formal representation of knowledge of a heath domain expert, represented by an archetype, with the mapping information to clinical data sources for semantic interoperability and standardization purposes.

Visual interface. LinkEHR-Ed provides two different interfaces, one for the health domain experts and one for the information technologies professionals. On the one side, the Health domain expert will be in charge of archetype definition and so they must have some knowledge about the RM they are working with. But the main idea for this perspective is to hide the underlying complexity of the system and the Dual Model architecture logic involved in the designing of an archetype as we can not presuppose any computer management skills for this expert. A set of available constraints and applicable restrictions is provided during design time. On the other side is the information technologies expert, who knows the structure of data sources of the organization and his role is to map the archetype definition tree nodes to them. A mapping definition interface fills nearly all this edition perspective. It is composed by a graphical representation of the archetype definition tree and a graphical representation of the diverse data sources available. Users can then add or modify mapping transformations between elements of both representations. In this case, LinKEHR-Ed can be seen as visual mapping and data translation tool.

Acknowledgments This work was supported in part by the Spanish Ministry of Education and Science under grant TSI2004-06475-C02.

References [1} Eichelberg M, Aden T, Riesmeier J, Dogac A and Leleci GB. A survey and analysis of electronic healthcare record standards. ACM computing Surveys 2005: 37(4): 277-315. [2] Health Level 7: www.hl7.org [3] European Committee for Standardization. Technical Committee on Health Informatics: www.centc251.org. [4] European Committee for Standardization. Health informatics-Electronic health record communication- Part 1: Reference model. Draft European Standard for CEN Enquiry prEN13606-1, 2006. [5] European Committee for Standardization. Health informatics-Electronic health record communication- Part 2: archetypes. Draft European Standard, prEN13606-2, 2006. [6] OpenEHR foundation: http://www.openehr.org. [7] Beale T, Heard S. Archetype Description Language 1.4 (ADL). The OpenEHR Foundation. [8] Maldonado JA. Historia Clínica Electrónica Federada Basada en la Norma Europea CEN/TC251 EN13606 PhD. Dissertation. Technical University of Valencia, 2005. [9] Beeri C, and Milo T. Schemas for integration and translation of structured and semi-structured data. Proceedings of the 7th International Conference on Database Theory (ICDT'99) , pp. 296-313, 1999 [10] Kuper GM, Simeon J. Subsumption for XML types. Proceedings of the 8th International Conference on Database Theory (ICDT'01), pp. 331-345, 2001. [11] Popa L, Velegraki Y, Miller RJ, Hernández MA, Fagin R. Translating web data. Proceedings of the 28th VLDB Conference, pp. 598-609, 2002.

Figure 2 - Archetype edition and mapping process

Address for correspondence Jose Alberto Maldonado, Ph.D. Biomedical Informatics Group. ITACA Institute. Technical University of Valencia Valencia 46022, Spain [email protected]

Conclusion In this paper, we have presented an archetype modelling framework and LinkEHR-Ed, a tool that allows the utilization of archetypes for upgrading already deployed systems in order to make them compatible with an EHR architecture standard. The overall objective is to maintain inproduction systems and applications without any changes

458

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Reaching Standards for Dissemination: A Case Study Helen Christensena and Kathleen Griffithsb ab

Centre for Mental Health Research, The Australian National University, Canberra, Australia

Policy makers and governments are now seriously interested in these applications, but they require guidelines for determining which applications are likely to yield positive public health outcomes, and which should be supported financially. One set of standards, which has been recently developed by the Society for Prevention Research (SPR), may be suitable, not only for prevention programs, but also for web applications designed as early intervention programs. Flay et al., 2005 [1] provide a set of 47 standards for efficacy, effectiveness and dissemination, which may prove to be useful in assessing web applications in terms of their success and their potential for implementation and wider dissemination. Currently, most web interventions are disseminated using marketing strategies, and their uptake is determined by market forces, consumer preferences and other factors. Little is known about the benefits or potential adverse effects of these interventions. Computer-based CBT programs have recently been endorsed by government bodies such as the UK-based National Institute for Health and Clinical Excellence, and it is likely that such bodies will similarly endorse Internet programs in the future. This endorsement will also be expected to drive demand.

Abstract At what stage are web applications ready for dissemination? A set of standards for efficacy, effectiveness and readiness for dissemination of (prevention) interventions has been developed by the Society for Prevention Research (SPR) (Flay et al., 2005)[1]. In this case study paper, we examine these standards criteria with reference to MoodGYM, an automated web application designed to reduce depression symptoms using cognitive behaviour therapy training. We examine evidence for its efficacy, its effectiveness in real world situations, and its capacity to meet recommended standards for dissemination. We conclude that MoodGYM has substantially met the standards of evidence required for dissemination. This successful application of the SPR standards to the evaluation of a web application, suggests that these prevention standards might be usefully applied to web/Internet context. Web applications might be assessed for dissemination by the International Society for Research into Internet Interventions (ISRII), or some other professional organization to assist policy makers in making decisions about the funding, adoption and promotion of applications. Keywords:

In this case report paper we describe the research taken to establish the efficacy and effectiveness of a web intervention – MoodGYM –an application designed to reduce symptoms of depression in community users. We ask (i) whether it meets the SPR criteria for efficacy, effectiveness or dissemination, and, if not, (ii) what additional research might be required. We also (iii) comment on the potential usefulness of SPR standards in the evaluation of web applications.

internet, depression, efficacy, effectiveness, implementation

Introduction Over the last 10 years, web applications have proliferated in the areas of mental health and substance abuse. This growth in the development and research of web applications is illustrated below using a review of the frequency of research papers identified using search terms for ‘web therapy’ trended for year from 1996 to 2003 (Figure 1)[2]. Web applications have been found to be both feasible and effective in reducing common mental health disorders such as anxiety and depression [3,4].

In the SPR standards, the demonstration of efficacy requires the demonstration of positive treatment outcomes under optimal conditions: The following are required: clear description of the project, and clearly stated outcomes [decrease in depression symptoms, over a (minimum) six month period, using psychometrically sound measures, with one form of data collection not susceptible to demand characteristics]. The strongest form of control design should be used where possible (RCT), and the sample requires definition, [how it was obtained], the statistical analysis should be unbiased [intention to treat, adjustment for pretest differences, differential attrition adjustment, correction for experiment wise error]. At least two different high quality studies/replicates must exist. Desirable standards include process variable measure-

Figure 1 - Web therapy papers as a function of year

459

H. Christensen et al. / Reaching Standards for Dissemination: A Case Study

ment, and measures of side effects [See Flay for more details [1]].

Materials and methods

An intention to treat analysis demonstrated that the effects of the website persisted at 12 months follow-up [8]. Figure 2 illustrates that both MoodGYM and the psychoeducation site were associated with better outcomes immediately, and at 6 and 12 months. A second study has now been undertaken of the use of the website in 29 schools around Australia, using a clustered randomized controlled trial design. Although not complete (two of the 29 schools currently in the intervention phase of the project) and the long term follow-up is yet to be collated, there is evidence of a small effect for the (Universal) intervention in students in Years 9 or 10 (mean age 14.3 yrs). An additional efficacy trial has been completed in Norway which found that MoodGYM (in combination with BluePages) reduced depression symptoms compared to a control condition [9]. A New Zealand trial is about to commence. The day-to-day implementation of these trials is under the control of researchers external to developers. The MoodGYM site measures a number of variables that might mediate the effect of the website on the reduction of depression symptoms. The major measure is the Warpy Thoughts Questionnaire. This questionnaire was developed by the Centre for Mental Health Research (CMHR), and has been validated against other scales of dysfunctional thinking. The users of the site record scores on this questionnaire three times over the course of the intervention. In the first instance, this permits the calculation of correlations between symptom change and thinking style [10]. A second desirable criterion for the efficacy of a trial is a measure of side effects. MoodGYM does not explicitly collect information on side effects. This represents a common omission in psychological therapy studies, but one that requires rectification.

Checklist and raters

Effectiveness standards

A checklist of standards was constructed based on the Flay et al. criteria. We reduced the number of SPR criteria from 47 to 19 in order to summarize outcomes within the space of this report. To achieve this some sub-criteria, definitions and some outcome criteria were not included. All published or completed research reports of the MoodGYM website were collected. Two raters reported whether the MoodGYM site reached standards for efficacy, effectiveness and potential for dissemination. Any difference between raters was resolved by references to criteria and published reports.

The 8 criteria for effectiveness are outlined in Table 1.

Effectiveness studies focus on the importance of the project in real-world situations, implementation procedures, fidelity and adaptation (p. 153). Standards include the provision of manuals and technical support, the delivery of the project, as in the real world, with a clear statement for whom the intervention works. The level and integrity of the intervention, in addition to the level of engagement is needed, including measures of adherence, and or involvement in both the control group and the treatment group. RCT designs are required, although often more difficult to implement. The sample investigated should be the sample for which the application is targeted. A desirable standard is replication with different responses. The practical importance or public health impact of studies requires assessment. Examples include cost effectiveness, effect size measures and percent relative change. Replication is also required. Programs worthy of dissemination must meet all requirements for efficacy and effectiveness, as well as additional standards for dissemination such as training of staff, and the provision of project support materials. The criteria include: the provision of provider materials, and evidence that the program can be implemented with fidelity, evidence for scalability, and exact estimates of cost. Monitoring and evaluation tools must be available to providers (p.167). A desirable standard is “a clear statement of the factors that are expected to assure the sustainability of the program or policy once implemented”.

The MoodGYM application provides a user manual and a clinician manual which can be purchased from the CMHR website at http://www.anu.edu.au/cmhr/shop.php. The user manual provides basic information on access to the site and trouble shooting for technical problems. The clinician manual provides week-by-week instructions, which allow clinicians to implement the self-help program with clinical populations, and normative data on website users. Table 1 - SPR Criteria Simplified Standards Criteria (Society for Prevention Research) Criteria for Efficacy Clear description of project (R) One high quality RCTs (ITT, sound measures) (R) Long term follow-up (at least 6 months) (R) Additional long term high quality RCT (R) Process variable measurement (D) Measures of side effects (D) Criteria for Effectiveness Provision of manuals and technical support (R)

Results Efficacy Standards All of the required standards for efficacy were met by the MoodGYM application, although one of two desirable standards was not met. The MoodGYM website provides a clear indication of its content and aims (http: moodgym.anu.edu.au). An efficacy trial published in 2004 compared the website to an attention placebo control condition (health coach) and to a psychoeducation site (BluePages) using participants from a community sample with elevated depression symptoms, recruited by survey using addresses from the Australian Electoral Roll [6,7].

460

Rating Yes Yes Yes Yes Yes No Rating Yes

H. Christensen et al. / Reaching Standards for Dissemination: A Case Study

Simplified Standards Criteria (Society for Prevention Research) Evidence of delivery in the real world to target (R) Level of exposure, engagement and adherence (R) Two RCTs in identified settings (R) Replication with different samples (D) Cost effectiveness (D) Dose analyses (D) Demonstration of public health impact (R) Criteria for Dissemination Provision of provider support and fidelity (R) Evidence for scalability (R) Clear cost information available (R) Monitoring and evaluation tools (R) Factors to assure sustainability (D)

MoodGYM has not yet met the standard of two randomized controlled effectiveness trials, although comparisons of spontaneous users and those in our first randomized controlled trial were positive [12] and the schools trial, mentioned earlier, is both an efficacy and effectiveness trial as it is implemented in real world conditions. An additional study is currently in progress, evaluating MoodGYM’s effectiveness in a general practice setting. Results from these evaluations may well provide the necessary evidence of effectiveness.

Yes Yes Current Current Yes Yes No Rating Yes

A study of the cost effectiveness of MoodGYM has also been conducted. There are few cost-effectiveness studies of Internet programs, although one computerized therapy program used in general practice settings has been assessed to date [See Kaltenthaler et al, 2006] [13]. Cost effectiveness analysis is particularly interesting for Internet programs given the scalability of Internet applications and their very low marginal costs [14]. The cost effectiveness study indicated that MoodGYM (M-CBT) was dominant over conventional treatments consisting of a general practitioner-delivered anti-depressant medication (GP-DRUG) and psychiatrist led cognitive behaviour therapy (P-CBT). The above assumes a similar level of efficacy for therapist administered CBT, for antidepressant medication and for Internet CBT. A separate randomized controlled trial of community users found that different components of the program were associated with improved outcomes [15], but, in particular, that extended CBT component of the program was associated with large effects.

Yes No Yes No

Note: R=Required, D=Desirable.

One of the advantages of the MoodGYM application is that it is both automated and self-directed. As such it requires less intensive training for those who implement it, especially in comparison to face-to-face interventions. Technical support is available via an email address posted on the site. One potential problem with the application when used outside a research trial that incorporates interviewer-instigated tracking is its failure to engage all registrants for the complete program of five modules. Studies of usage [11] and mental health change suggest that treatment benefits are obtained once two or more modules are accessed. As many spontaneous users engage with the site for more than this number of modules, we believe that adherence level is relatively acceptable for a proportion of users. Moreover, the interpretation of drop out from web applications, where there is no expectation held by most users that they are required to engage for hours of therapy online, is less straightforward than conventional drop out in non-Internet RCTs. Many users may well attain what they set out to achieve in two or three modules rather than the full six.

A final effectiveness criterion is the demonstration of the public health benefit of the application. This has not yet been conducted.

25 MoodGYM BluePages Health Coach

Figure 3 - Cost effectiveness comparisons. M-CBT=MoodGYM, P-CBT=Psychiatrist CBT and GP-DRUG=General Practitioner Anti-depressant medication[14]

CES-D

20

Dissemination standards

15

10 Pretest

Posttest

6 Month Follow up

Of the identified five standards in Table 1, MoodGYM meets three. These are fidelity, scalability, and monitoring tools. Fidelity (as in the provision of the exact user experience of the website) is guaranteed in automated web applications (although there is no guarantee that each user will access all pages in a similar way, or even receive the same pages if the application is personalized or tailored). The site is also scalable, with over 160,000 registrants, and up to 14,000 visitors a month. Moreover, organizations or researchers are able to download SPSS formatted data

12 Month Follow up

Figure 2 - Least squares means for CES-D estimated using time by condition model with unstructured covariance matrix. Note: CES_D: Higher scores indicate higher levels of depression symptoms [8]

461

H. Christensen et al. / Reaching Standards for Dissemination: A Case Study

from their specific trials or organisations, to allow monitoring and evaluation. MoodGYM is currently offered free of charge to users on the World Wide Web. The costs of the development and maintenance of the MoodGYM website are borne by our research group within a University. Clearly, this model of funding the site is not sustainable. The capacity to provide the service will cease when the research agenda has concluded or grants are completed. This as-yet-to-be developed model for sustainability represents the most serious threat to meeting the criteria for standards for dissemination.

will occur regardless of whether a website meets the SPR or other quality standards. This underscores the importance of researchers and policy makers introducing standards like these to rate the readiness of applications for dissemination. Major problems for sustainability A major problem we identified for the MoodGYM site was lack of sustainability. A number of methods to financially sustain the site might include a user-pays system, an employer-pays system for the workforces of large organizations, or a health company pays system delivered via health maintenance organizations. However, as developers of the site, we preferred to offer the site free of charge to individuals with mental health problems with the aim of maximising the potential public health impact of the application. To offer cost-free access, the site needs to gain support from government. The mechanisms within government to offer this support must be in place, and this itself may be a stumbling block. Because of the international nature of website applications, it is difficult to determine how users from different countries might contribute. For example, the MoodGYM site is regularly accessed by users of the UK National Health Service. There may be some capacity here for governments to agree to contribute to costs of approved sites. However, at this stage, in Australia, the sustainability for public health websites is yet to be convincingly demonstrated.

Discussion Usefulness of the efficacy, effectiveness and dissemination standards There may be merit in examining web applications with reference to the standards developed by the Society for Prevention Research. In this study, the areas in need of further research and development to reach standards for the MoodGYM site were relatively easily identified. For example, we identified the need to investigate and include side effect assessment in our online MoodGYM satisfaction questionnaire (available from the menu), and to consider methods to estimate the public health impact of our intervention more systematically. The review of the web application against the standards criteria suggests possible areas for the refinement of the criteria. For example, it is sometimes difficult to distinguish for automated applications such as MoodGYM whether an intervention study should be classified as an efficacy study or an effectiveness study. Some aspects that differentiate an efficacy from an effectiveness trial of a psychological intervention may be less relevant in an automated application. For example, in contrast to human-delivered therapy, automated delivery of web therapy ensures fidelity of treatment content whether or not the intervention is delivered in a research or specialist environment. Another potential area in which the SPR might be profitably refined relates to the definition of an acceptable ‘control’ group. Although there is a large ‘effectiveness’ trial of the MoodGYM site which was conducted entirely online [14], it does meet the specified criteria for an effectiveness trial, as the criteria state that the comparison group must be one of waitlist, attention placebo or TAU. In this trial, which is offered directly to users, the control condition was a onemodule version of the site. Perhaps effectiveness trials could be permitted to include a minimal intervention condition as a control.

Conclusion The standards of evidence for prevention research might be usefully applied to web/internet interventions. Web applications might be assessed for dissemination by the International Society for Research into Internet Interventions (ISRII), or some other professional organization to assist policy makers in making decisions about the funding, adoption and promotion of applications. Acknowledgments The authors are the developers of the MoodGYM site. The site is supported by e-hub at the Australian National University.

References [1] Flay BR, Biglan A, Boruch RF, Castro FG, Gottfedson D, Kellam S, Moscicki E, Schinke S, Valentin, JC, and Ji, P. Standards of evidence: Criteria for efficacy, Effectiveness and Dissemination. : Prevention Science 2005;6 (3): 151175. [2] Wantland DJ, Protillo CJ, Holzemer WL, Slaugher R, McGhee EM. The effectiveness of web-based vs non-webbased interventions: A meta-analysis of behaviour change outcomes. JMIR 2004; 6(4):e40. [3] Griffiths KM & Christensen H. Review of randomized controlled trials of Internet interventions for mental disorders and related conditions. Clinical Psychologist 2006; 10(1), 16-29. [4] Griffiths KM, Farrer L, Christensen H. Clickety-click: emental health train on track. Australasian Journal of Psychiatry, in press. [5] National Institute for Health and Clinical Excellence. Computerised cognitive behaviour therapy for depression and Anxiety. Review of Technology Appraisal 51.

Advantages of web applications This exercise of comparing MoodGYM against the standards criteria reveals the strengths of web applications compared to non-Internet interventions. For example, because of the capacity for tracking, web applications should be particularly strong in comparison to face-to-face interventions in the following areas: determining dose responses, process evaluation and the collection of routine implementation and evaluation data. Demand characteristics might be reduced by self-report data submission. Websites can be readily disseminated, and dissemination

462

H. Christensen et al. / Reaching Standards for Dissemination: A Case Study

[6]

[7]

[8] [9]

[10]

[11]

February2006. http://guidance.nice.org.uk/TA97/guidance/ word/English Christensen H, Griffiths KM, Jorm AF. Delivering depression interventions using the Internet: positive results from a large randomised controlled trial. British Medical Journal 2004, 328:265. Online 23-01-04. Griffiths KM, Christensen H, Jorm AF, Evans K,Groves C.. Effect of web-based depression literacy and cognitivebehavioural therapy interventions on stigmatising attitudes to depression: a randomised controlled trial. British Journal of Psychiatry 2004;185:342-349. Mackinnon AJ, Griffiths KM, Christensen H. Twelve month followup of the CBT site. Submitted for publication. Lintvedt OK Sorensen K Ostvik AR. Evaluating the effectiveness of an Internet based intervention preventing depression. A randomized controlled trial. 2006. Unpublished Thesis for Psychology Degree. Department of Psychology. University of Tromso, Tromso, Norway. Parslow R, Christensen H, Griffiths K, Groves C. The Warpy Thoughts Scale: a new 20 item instrument to measure dysfunctional attitudes: Factor analysis and associations with mental health messages. Cognitive Behaviour Therapy 2006;Vol 35(2) 106-116 Christensen H, Griffiths KM, Korten AE. Web-based cognitive behaviour therapy (CBT): analysis of site usage

[12]

[13]

[14]

[15]

and changes in depression and anxiety scores. Journal of Medical Internet Research 2002;4(1):e3. Christensen H, Griffiths K, Korten A, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. Journal of Medical Internet Research 2004;6 (4):e46.. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, Parry G, Rooney G and Sutcliffe, P. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technology Assessment, 2006; 10 (33). Butler, J, Griffiths, KM, Bartlett, L and Christensen, H. Direct costs of Internet based cognitive behaviour therapy for depression compared to face to face treatment and medication. Submitted for publication. Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K. Online randomised controlled trial of brief and full cognitive behavior therapy for depression. Psychological Medicine 2006 Vol : 36 (12) 1737-1746

Address for correspondence Centre for Mental Health Research, The Australian National University, Canberra, 0200, [email protected]

463

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Discrete Time-Space Geography for Epidemiology: From Mixing Groups to Pockets of Local Order in Pandemic Simulations Einar Holma, Toomas Timpkab,c a

Department of Social and Economic Geography, Umeå University, Sweden b Department of Computer Science, Linköping University, Sweden c Department of Social Medicine and Public Health, Linköping University, Sweden

approach to simulation of epidemics is based on stochastic models. These strategies have the advantages that they are computationally less complex and also can be validated using quantitative epidemiological data. Discrete-time, stochastic simulations have been used to study both the dissemination of infectious diseases and the outcome of interventions. To deal with the unrealistic assumption of homogenous contact patterns in large populations, individuals are usually allocated at each time step to discrete places, with specific localized mixing groups, where within social mixing is assumed to be more homogenous than between [5,6]. The computations in such a mixing group simulation can hereby be reduced to mixing people in the discrete spaces with homogenous mixing, while maintaining the dynamic state of infection of each sample person at each time step. Although numerous approaches have attempted to resolve the complexity-tractability/validation trade-off between social network-based and stochastic simulations, there is today no established solution to the problem.

Abstract The World Health Organization urges all nations to develop and maintain national influenza preparedness plans. Important components of such plans are forecasts of morbidity and mortality based on local social and geographic conditions. Most methodologies for simulations of epidemic outbreaks are implicitly based on the assumption that the frequency and duration of social contacts that lead to disease transmission is affected by geography, i.e. the spatial distribution of physical meeting places. In order to increase the effectiveness of the present methods for simulation of infectious disease outbreaks, the aim of this study is to examine two social geographic issues related to such models. We display how the social geographic characteristics of mixing networks, in particular when these significantly deviate from the random-mixing norm, can be represented in order to enhance the understanding and prediction of epidemic patterns in light of a possible future destructive influenza pandemic. We conclude that social geography, social networks and simulation models of directly transmitted infectious diseases are fundamentally linked.

A social geography of infectious disease The available methodologies for simulations of epidemic outbreaks are implicitly based on the assumption that the frequency and duration of social contacts that lead to disease transmission is affected by geography, i.e. the spatial distribution of physical meeting places of different types. However, current socio-geographic concepts and empirical findings have seldom been adequately applied to specify these assumptions. It is for example only the neighborhood around the living place that increases contact probability only for spatial reasons, i.e. the closest neighbors within some 100 meters to 1 km range [7,8]. Beyond that, the organization of workplaces and other meeting places are primarily not spatially but socially stratified. If the work place, friends, relatives, or shopping mall, etc. are around the corner or 20 km away does not significantly influence behavior; these places are still visited in favor of random places that are closer. The next distance that matters somewhat is the dwelling area (cp. village), say within some 3 km range. In Sweden, one third of the working population have their place of work within that range, often also a local shop, a primary school, and a day-care center. Obviously, the likelihood of meeting others at such a place chosen at random is larger than beyond. But more impor-

Keywords health informatics, methodology, social geography, epidemiological modeling, simulations

Introduction In response to the threat of a destructive influenza pandemic similar to that of 1917-1918, the World Health Organization urges all nations to develop and maintain national influenza preparedness plans [1]. Important components of such plans are forecasts of morbidity and mortality based on local social and geographic conditions. Such forecasts require both surveillance of disease activity and simulations of disease dissemination. Epidemiological simulations of contagious diseases based on social network theory, where hosts and contacts are modeled as ‘actors’ and ‘relations’, have attracted considerable attention [2,3,4]. While an arbitrary level of complexity may be included in network and agent-based simulations of spatial epidemics, computational intensity and analytical intractability mean that such models often lack transparency into the determinants of epidemiological dynamics. Another

464

E. Holm et al. / A Discrete Time-Space Geography for Epidemiology: From Mixing Groups to Pockets of Local Order

tantly, already within this spatial scale, you will almost never visit the other majority of workplaces, inhabitants, etc., except for your own specific ones. Once selected, they remain fixed with respect both to location and members for the duration of a pandemic. Next level that might matter is municipality range, approx. within 30 km (cp. the municipality). Most of everyday work (two thirds) and services are situated there. The following boundary is an extended commuting distance, approx. within 100 km (cp. county). Although, it is a relatively small fraction of the population that interacts daily on that distance. For instance, the median commuting distance all over Sweden is approx. 5 km. Beyond that, we have low frequency meetings within and outside the country more or less regardless of how many there are living in between or the distance. This low frequency travel is, however, important for the initial transmission of disease between regions.

between. In either case, most of those latter individuals you will never meet. This paper reports from a research program aimed at supporting pandemic response planning by simulations of contagious disease outbreaks in local public health contexts [10,11,12]. The methods section of the paper outlines the basic theoretical concepts necessary for application of social geography in the context of influenza pandemic simulations. The results section presents an application based on the Swedish ASTRID database. The final section summarizes the paper and draws conclusions.

Materials and methods Mixing groups as social pockets of local order Our approach to inform models for simulation of pandemics by social geography is to transfer Hägerstrand’s concept of ‘pockets of local order’ to homogenous localized population mixing groups. A network structure is used to represent the mixing groups as nodes with contact frequencies between individuals within them as well as the other properties of social pockets. Instead of a fixed arbitrary hierarchy of household groups and regions, we suggest a system of meeting place centered, floating, overlapping reference areas. We propose the following structure for modeling of the time individuals spend at social pockets.

A realistic social geographic representation is not the same as a mapping of the socio-spatial trajectories of all individuals in a population in full detail. Contrarily, most of such intrinsic peculiarities probably have minor impact on general transmission of infectious diseases and should therefore rather be modeled by help of more abstract entities with random variations. This is also the most prominent property of the mixing group model for stochastic simulations of epidemics. The most important thing is instead to represent the relevant discrete spaces of close contact, i.e. the only places where the transmission of infectious agents occur. Second is the spatial configuration of these meeting places, with Hägerstrands terminology, "the pockets of local order" [9]. They are constructed by people in order to perform activities for certain projects that must be done at a particular place and to shield these from destructive outside influences [8]. The spatial configuration is mainly a constraint on the initial set up and choice of meeting places. It can in most cases be regarded as a constant for the duration of the pandemic, not changing the set of participants at the places.

1. Close neighborhood (100-1000 m2around the family residence) 2. Dwelling area (range 3 km around the residence) 3. Municipality range (30 km around the residence) 4. Extended commuting range (100 km around the residence) 5. Rest of nation and world. The reference spatial entity is the family residence, where the infection enters and is distributed out with the family members. In principle, each family has its own unique set of ranges with a unique localization of the different types of meeting places for each individual family member. Their closest neighbor has almost the same range attributes but the set of meeting places within the ranges are very different for the individual members of each family. Information on the location of the meeting places is almost only necessary because of our interest in the spatial outcome of the disease diffusion. For global results, the individual flows between, and the characteristics of the meeting places would be enough.

Study aims In order to increase the effectiveness of the present methods for simulation of infectious disease outbreaks, the aim of this study is to examine two social geographic issues related to such models: 1. Contact endurance at meeting places. Most mixing groups used in simulation models represent the mixing of more or less the same set of individuals day after day during the evolution of the pandemic. The degree to witch this is the case is one of the most important distinguishing features between mixing groups and should be explicitly considered for modeling. 2. Social vs. ecological fallacy. In a mixing group simulation, populations are divided into submodules based on, e.g. municipalities or counties. Such regionalization that is heterogeneous with respect to distances, from a few hundred meters to hundreds of kilometers between random inhabitants. Depending on context this distance may or may not matters for whom you will meet, but not if it is 200 or 20000 people in

A network representation of mixing groups A central question for epidemic modeling is how to represent a differential description of who meets whom, how close and for how long time and how to represent the interaction (flows of people) between different geographical contexts. We know relatively well people’s behavior with regard to their social contracts (employments, compulsory schools, etc.) and can model visits at corresponding social pockets accordingly. What needs to be dynamically modeled is the time spent in pockets that are visited during leisure time. Following pockets of social order with rele-

465

E. Holm et al. / A Discrete Time-Space Geography for Epidemiology: From Mixing Groups to Pockets of Local Order

vance for transmission of infectious disease can be identified:

transmission of the infectious agent at a given contact. Estimates are from Swedish data and the literature [5,6,13]. This representation is mainly consistent with the mixing group model. A spatially explicit mixing group simulation can thereafter be divided into two stages. At the first stage, an abstract representation of contacts based on social contracts is generated from spatially explicit population data using fixed types of social pockets and itinerary types specific for age-group and socioeconomic categories. Each person is initiated by allocating the person first to an age-specific itinerary type, thereafter to the relevant types of social pockets, and finally to instances of social pockets, , called local pockets (localized mixing group) from the database. That allocation is based on observed or estimated cross sectional frequencies for each of the social pocket types over each of the ranges for certain groups of individuals.

Reference pocket with fixed location and mainly fixed set of mixing subjects 1. Family residence (by type, size) Pockets regularly visited by social contract with the same subjects 2. Place of work (by size, type of employees and social mixing patterns) 3. Day care center (specifically) 4. Primary school (specifically) 5. Other education Pockets visited at random occasions with the same subjects 6. Relatives’ homes (grandchildren, former wife etc.) 7. Friends’ homes (a large part of social interaction goes on here)

Table 2 Age

Pockets visited at random with random new subjects each time 8. Public services and entertainment

Own home 17.5/7

7-15

Own home 4/7

16-19

Own home 14/7

20-64

Own home 14/7

65-

Own home 17.5/7

Transition pockets (random or fixed occasions, random subjects) 9. Local daily travel with public means of transport (commuting, train, air transports - short duration but sometimes close contact with new people) Transitions (movements between communities) 10. Short and long haul recreation and leisure travel 11. Business travel (rapidly growing, partly replacing commuting and migration)

1km

0-6

3km

30km

100km

Day-care center 3.5/ 5 or Family day-care 3.5/5 Primary school 5/5 High school 5/5 Short-distance transports 0.5/8 Workplace 5/5 Short-distance transports 0.5/8

Table 1 Social pocket

n

p (c)

p (t)

Using empirical data, the patterns for visiting the social pockets can thus be represented as itinerary types for regularly visited social pockets and for arbitrarily visited pockets, respectively. While the pockets visited by social contract can be represented by one particular mixing group instance (school, workplace, etc.) during the instantiation, the arbitrarily visited social pockets are randomly instantiated (as one or several local pockets) within the corresponding geographical region determined from the individual’s place of residence. Table 2 shows itinerary types for pockets visited by social contract displayed by concentric geographical regions determined from the individual’s place of residence. Numbers in italics indicate 8hour time units spent in the pocket per week and number of visits per week (8-hour units/visits).

Contract

Family home

1-7

0.3-0.6

0.12

Yes/No*

Workplace

20

0.06

0.12

Yes

Day care Family day-care

3-6

0.35

0.12

Yes

Day-care center

10-20

0.15-

0.12

Yes

Primary school

15-25

0.03-0.04

0.12

Yes

High school

30-60

0.03

0.12

Yes

Schools

Public services Shopping mall

25-150

0.01-0.001

0.12

No

Arena

200-800

0.001-0.002 0.12

No

Short distance

40-80

0.03-0.04

0.12

No

Long distance

100-200

0.01-0.005

0.12

No

Public transportation

*

Table 3 shows itinerary types for arbitrarily visited mixing groups displayed by concentric geographical regions determined from the individual’s place of residence. The examples in the results sections are based on structures and data from the ASTRID database covering the Swedish population (9 million).

Relatives and friends visiting a family home do not own or rent the facility.

Table 1 displays some characteristics of such social pocket types. The contact probability p(c) reflects the probability for any person visiting the social pocket to meet one single person in the pocket, while p(t) reflects the probability of

466

E. Holm et al. / A Discrete Time-Space Geography for Epidemiology: From Mixing Groups to Pockets of Local Order

Table 3 3km

30km

0-6

Age Homes

1km

Homes Shopping malls

Homes Shopping malls

Homes

100km Homes

Nation

7-15

Homes

Homes Shopping malls

Homes Shopping malls Arenas

Homes Shopping malls Arenas

Homes

16-19

Homes

Homes Shopping malls

Homes Shopping malls Arenas

Homes Shopping malls Arenas Short-distance transports

Homes

20-64

Homes

Homes Shopping malls

Homes Shopping malls Arenas

Homes Shopping malls Arenas Short-distance transports

Homes Work places Long-distance transports

65-

Homes

Homes Shopping malls

Homes Shopping malls Arenas

Homes

Homes

local pockets, a school and a workplace are in place. The school might be within reach only for q whereas both p and q might work at the work place. The floating ranges are only used as a tool for the initial allocation of persons to local pockets. When that is done, the resulting set of "arrows", the network of connections, is the only thing necessary to feed into the simulation.

Results Mixing-group instantiation of social pockets In our instantiation of the social geographic model for epidemiological simulations, each social pocket type I has type specific properties, such as the average disease passage between pairs of visitors (infectious or susceptible) per unit of time. In order to enable adjustments to different infectious agents, this risk is factorized into contact probability per time unit and transmission risk per contact. The specific instances i of I are given the same properties as the type, and provided a random variation reflecting observed empirical distributions. In addition each i has a location and size etc. The outcome probability for a specific infectious/susceptible pair is also affected by those persons’ individual properties.

Each person also is given an individual computed state with regard to the disease. The only property that has to be dynamic is the person’s current infectious status. The set of people m(i) "mixing" at each specific social pocket i of type I, consists of persons being members of pocket i (p.r(i).member = true). That set is therefore also fixed for all persons and local pockets (meeting places) during simulation by the individual properties of the persons as derived from the database.

Parameterization of individuals

Simulation

Parameters for each individual is derived from the ASTRID database. The basic object in this modeling approach is each individual person p in the country. All other local pockets are relational properties of the person but also when convenient objects by themselves. The approach lends from time geography, micro simulation and agent based simulation [14,15]. Each person p has a set of individual properties (age, sex, education, origin, income, mother, family etc.). Person p’s family f has other properties common to its members (single, adults only, child family etc., size, location, housing type etc.) Person p also has individual relations r(I) to the social pockets types I visited by social contract (work, day care, school) and those visited less regularly but within a small set of persistent members (friends’ and relatives’ homes) and those visited irregularly with random members (public services, entertainment, long-distance travel, etc.). For the duration of a pandemic, p's properties, including location and social composition of family and meeting places, can be regarded as fixed, i.e. as an itinerary. For example, suppose that the partly overlapping ranges (family, 1 km2, 3 km neighborhood, 30 km commuting distance, and 100 km regional range) are outlined for two (p and q) out of the nine million localized individuals in the database. Say that two example

When running the simulation, a population is first broken down in administrative-political segments (by residence) and then divided into standardized units. The division into standardized units is, among other reasons, based on the necessity to validate the simulations against historical data. For each time step (8 hours) each person in the population is exposed to a context, time and person specific computed probability to transmit and receive the disease at each mixing group and thereafter updated accordingly to the outcome of the lottery. The consequence events are then conventionally modeled without time delay either immediately after the transmission or for aggregates at the end of the time step. After that the table for simulation is set and fixed, the main property changing value during simulation is the persons current infectious status. In a more elaborate model also other dynamic response actions can be introduced, like keeping children at home instead of at day care or school before they become infected or that hospital personnel obstruct by staying home from work in order to avoid personal risk. Such behavior responses can only be based on vague and anecdotic evidence but it is nevertheless crucial to enable such hypothetical experiments.

467

E. Holm et al. / A Discrete Time-Space Geography for Epidemiology: From Mixing Groups to Pockets of Local Order [5] Longini IM Jr, Halloran ME, Nizam A, and Yang Y. Containing pandemic influenza with antiviral agents. Am J Epidemiol. 2004 Apr 1;159(7):623-33. [6] Germann TC, Kadau K, Longini IM Jr, and Macken CA. Mitigation strategies for pandemic influenza in the United States. Proc Natl Acad Sci U S A. 2006 Apr 11;103(15):5935-40. Epub 2006 Apr 3. [7] Holm E, and Öberg S. Contagious social practice? Geografiska Annaler 2004;86B:297-314. [8] Ellegård K, and Vilhelmson B. Home as a pocket of local order: Everyday activities and the friction of distance. Geografiska Annaler 2004;86 B(4):281-296. [9] Hägerstrand T. Time-geography: focus on the corporeality of men, society and environment. In The Science and Praxis of Complexity. New York: United Nations University, 1985. [10] Timpka T, Morin M, Jenvald J, Eriksson H, and Gursky E. Towards a simulation environment for modeling of local influenza outbreaks. AMIA Annu Symp Proc. 2005; 729– 33. [11] Timpka T, Morin M, Jenvald J, Gursky E, and Eriksson H. Dealing with ecological fallacy in preparations for influenza pandemics: Use of a flexible environment for adaptation of simulations to household structures in local contexts. Medinfo 2007; [12] Eriksson H, Morin M, Jenvald J, Gursky E, Holm E, and Timpka T. Ontology-based modeling of pandemic simulation scenarios. MedInfo 2007;. [13] Halloran M., Longini IM, Cowart DM, and Nizam A. Community trials of vaccination and the epidemic prevention potential. Vaccine 2002;20:3254–3262. [14] Boman M, and Holm E. Multi-agent systems, time geography, and microsimulations. In Olsson MO, Sjöstedt G (Eds.). Systems approaches and their application: Examples from Sweden. Dordrecht: Kluwer Academic Publishers, 2004, 95-118. [15] Holm E, Holme K, Mäkilä K, Mattsson-Kaupi M, and Mörtvik G. The SVERIGE spatial microsimulation model – Content, validation, and example applications. Gerum Kulturgeografi 2002:4. Umeå: Umeå Universitet, 2002. [16] Keeling MJ, and Eames KT. Networks and epidemic models. J R Soc Interface. 2005 Sep 22;2(4):295-307.

Discussion Social geography, social networks and the epidemiology of directly transmitted infectious diseases are fundamentally linked. The foundations of epidemiology and early epidemiological models were based on population wide random-mixing, but in practice each individual has a finite set of contacts to whom they can pass infection; the ensemble of all such contacts forms a 'mixing network'. Knowledge of the structure of the network allows models to compute the epidemic dynamics at the population scale from the individual-level behaviour of infections [16]. We have in this study displayed how issues related to the social geographic characteristics of mixing networks, in particular how these deviate from the random-mixing norm, can be resolved in order to enhance the understanding and prediction of epidemic patterns in light of a possible future destructive influenza pandemic. Acknowledgments This work was supported by the Swedish Emergency Man-agement Agency (KBM) under contract 0700/2004. Henrik Eriksson, Britt Åkerlind and Kristina Kinnunen from Linköping University, Sweden, Johan Jenvald and Magnus Morin, VSL Research Labs, Sweden, Magnus Strömgren, Umeå University, Sweden, Elin Gursky, ANSER Analytic Services, Arlington, VA, USA, and James Nyce, Ball State University, USA have contributed valuable comments to the analyses presented in this report.

References [1] World Health Organization (WHO). Responding to the avian influenza threat. Geneva: World Health Organization, 2005. Available at http://who.int.org/csr/resources/ publications/influenza (accessed March 15th 2007). [2] Eubank S, Guclu H, Kumar VS, Marathe MV, Srinivasan A, Toroczkai Z, and Wang N. Modelling disease outbreaks in realistic urban social networks. Nature 2004 May 13;429(6988):180-4. [3] Browers L. MicroPox: a Large-scale and spatially explicit microsimulation model for smallpox transmission. http:// www.sgem.se/about_sgem/researchers/ associated_members.html [4] Burke DS, Epstein JM, Cummings DA, Parker JI, Cline KC, Singa RM, and Chakravarty S. Individual-based computational modeling of smallpox epidemic control strategies. Acad Emerg Med. 2006 Nov;13(11):1142-9.

Address for correspondence Professor Einar Holm PhD Department of Social and Economic Geography Umeå University SE-901 87 Umeå Sweden [email protected]

468

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A National Study of eHealth Standardization in Finland Goals and Recommendations Juha Mykkänena, Maritta Korhonenb, Jari Porrasmaaa, Tuula Tuomainenb, Antero Ensioc a

b

HIS R&D Unit, IT Services Centre, University of Kuopio, Finland Business and Administration, Savonia University of Applied Sciences, Kuopio, Finland c Ensitieto Oy, Varkaus, Finland

requirements of eHealth applications and information systems.

Abstract The role of standards is constantly increasing in health services, electronic health records, and eHealth applications. There are many areas of standardization which affect the healthcare work and health information systems. On a national level, the organization of the development and support for standardization should be a key priority. This paper summarizes a national study in Finland which reviewed the current status of eHealth standardization and made recommendations for the national standardization and the use of standards. The recommendations are related to the organization model and balanced participation, international and cross-domain collaboration and quality assurance of standards-related activities. In addition, education and support services and support for introductions and pilots are recommended to improve the know-how of standards in system acquisitions.

One of the central challenges for electronic health services is the interoperability of eHealth solutions [4]. Health information systems contain lots of diverse information which have to be preserved and used for a long time. Large amount of new information and knowledge is constantly emerging in healthcare [1]. In addition, information is often sensitive or has accountability requirements. It has been observed that in general, the interoperability of eHealth applications and health information systems is far from optimal. This is partially due to the fact that standards are missing, their existence is not known, or they have not been implemented to an adequate extent [2]. A central goal of the national health project in Finland is to produce interoperable national electronic healthcare records. This includes national solutions for the long-term storage of electronic patient records and a secure access to the patient records by health service providers, patients and other actors [5]. The national architecture requires that the electronic patient record systems correspond to the national specifications as well as the establishment of national healthcare IT services. These services include an electronic archive and related services for the registration of documents as well as solutions for messaging. The realization of the architecture requires steering by the

Keywords: standardization, interoperability, eHealth, health information systems

Introduction The role of standards in eHealth and national initiatives The provision and use of health services are facing increasing change pressures. The aging population, recruitment problems, biomedical advances, diversity of treatments and examinations, and increasing need of services by the elderly and by those having multiple diseases are seen as international trends [1, 2]. On the other hand, the national and international health and economic policies affect the market of the eHealth solutions. For example, the transition from hospital-centric to patient-centric and local health service provision has been seen as a panEuropean trend [3]. On the other hand, specialized services such as clinical laboratories are transferred to large specialized units, for example in Finland. Electronic health information is increasingly used to improve quality and to enable new care models and treatments [1]. Healthcare networks require connections between disparate healthcare units and professionals. These needs are reflected in the

standardization relevant to eHealth and HIS medicine and healthcare

healthcare IT and IS

IT, domain-neutral and cross-domain softwar e production / development security process description and definition interface technologies messaging and enveloping electronic documents eGovernmenance and architecture identification data communications

electronic health records security and confidentiality support for processes service and API interfaces archiving and long term storage message interfaces electronic clinical documents data types and formats architecture

information models and elements terminologies, classifications, codes guidelines, knowledge processes, pathways quality of care

Figure 1 - Areas of standardization addressed in the study

469

J. Mykkänen et al. / A National Study of eHealth Standardization in Finland - Goals and Recommendations

authorities and the establishment of an actor that is responsible for the implementation of the national services. Shared standards are a key factor in pursuing these goals. In addition to Finland, similar large-scale projects are underway in many countries. One central component of these initiatives is the promotion of the standardization of the healthcare IT.

ization possibilities and reusable models for subsystems or interfaces. In addition, healthcare financiers, medical research and the government benefit from standardization [3,2]. However, several problems related to standardization have been identified. Standards and standard families are incompatible and overlapping. Despite many standards available nationally and internationally, the introduction and assignment of different standards has not been completed. As the participation in the standards work is voluntary, not all relevant areas of standardization have been considered. In addition, the participation has been based on personal interest and it has not been properly coordinated, organized or resourced.

This paper presents the central results of a study which aimed to document the current status of the national healthcare IT standardization in Finland and to identify and suggest improvements in the organization and development of standardization and the use of standards. Dynamics of standardization To illustrate the context of this work, we briefly discuss some aspects of standardization as a basis for this study. A standard is a document approved by an accepted body which contains rules, guidelines or features for generic and repeated use in products, processes or services [6]. In general, standardization has many meanings and motivations [7]: the uniformity of production, the compatibility of technologies, the objectivity in measurement, the means for justice and a form of hegemony. Out of these goals, compatibility in the form of interoperability of software applications has been emphasized in recent years. Rapid change, global connectivity and the need for systems to interoperate are also increasingly evident in the future [8]. However, there is a wide range of interoperability standards available, both on healthcare-specific and domainneutral levels. The lack of implemented standards hinders the development and use of healthcare knowledge, causes risks to the patients and sub-optimal use of resources [2].

Materials and methods In November 2004, the challenges and problems of standardization were considered in a meeting by the representatives of the Finnish Ministry of Social affairs and Health, Ministry of Trade and Industry, the Association of Finnish Local and Regional Authorities, the Technical Research Centre of Finland (also representing HL7 Finland), the National Research and Development Centre for Welfare and Health, the Finnish Agency for Technology and Innovation TEKES, the University of Kuopio and the Savonia University of Applied Sciences. In conclusion, it was acknowledged that standardization is an important enabler of the health service provision, the development of the national economy, and the growth of the small and medium enterprises. The lack of sustained solutions for the development, utilization and participation in standardization was identified as a central challenge. The action was taken to produce a report which would review the previous studies and recommendations and combine them with a wider view of standardization. The result of the study would be recommendations for the improvements in the development of activities related to standardization on a national level.

Central tasks related to standards include the organization and steering of the standards work, the production and use of standards (experiments, introduction and use in products and procurements), education and support services, and the evaluation of standards and products conforming to them. The tasks are different in various phases of the lifecycle of standards, and they affect different participants. Standardization organizations are only responsible of some of the costs of these activities.

The work was scheduled from December 2004 to March 2005 for two projects funded by the Finnish Agency of Technology and Innovation TEKES. A literature survey was first performed, and national and international recommendations for eHealth standardization were reviewed, e.g. [3,4,9,10,2]. The study also produced a description of central international and national actors in eHealth standardization and a description of the different areas of standardization related to eHealth and health information systems. In addition, a previously defined framework for the evaluation and selection of standards was updated. The standardization areas were intentionally described from a wider view than only health IT to identify relationships and improvements beyond healthcare-specific considerations (see Figure 1). The standardization areas were divided in three main classes: the standards requiring specific medical or healthcare knowledge, the standards requiring a combination of IT and healthcare skills, and the IT or domain-neutral standards relevant in healthcare. These classes have some correspondence with the specificity levels and perspectives of the health informatics

Most standardization organizations emphasize the creation of standards as a response to the needs expressed by the companies and users. On the other hand, standards are created as a result of complex social negotiations [7]. These viewpoints to the creation of standards (functionalistic and constructivistic) emphasize the quality and accuracy, and the acceptance and dissemination of standards, respectively. In health information systems, standards support the vision of open systems which can be complemented with new products. The direct benefits of eHealth standards provided for healthcare organizations, professionals and patients include the reduction in costs and errors, and improvements in care quality, usability, work practices and the availability of information. In addition, the application developers must attain advantages from the use of standards, such as new markets, consistent and accurate requirements, improved integration with partners, special-

470

J. Mykkänen et al. / A National Study of eHealth Standardization in Finland - Goals and Recommendations

profiling framework by ISO TC 215 [10]. The notion of complementary standards to allow the modular construction of solutions [4] was one of the guiding principles in the identification and classification work.

tion and implementation were highlighted by software companies, as well as consistent solutions across different business domains. However, the opportunities of international markets were not among the most central goals. There were conflicting views regarding the special consideration of the existing systems in relation to standards, as well as the need for accurate identification of specific finegrained standards versus widely applicable frameworks.

After the identification of the main actors and areas, data collection was designed to survey the current status and the target state of the field. A recent web-based survey from the SerAPI project which included questions related to the responsibilities of integration and standardization was used as a basis. The survey was continued with an email survey to named experts in different organizations (hospital districts, companies, organizations in the board of the project). This survey had four main categories and 60 detailed points. The main categories were the goals and policies, areas of standardization, activities related to standardization and the objectives for improvement. Each respondent received questions from two or three categories to lower the response threshold.

According to the respondents, the most central areas of standardization were the consistency in the support and specification of healthcare processes, the structure of information, data types, semantic consistency and the storage of electronic patient documents. In addition, unified information models and desktop integration were among the most important areas in some responses. Terminologies, codes, knowledge, shared IT services and workflow support were advocated to some extent. There were different opinions about the necessity of the standardization of architectures, security solutions and many technical or domain-neutral aspects. The code sets and information models were seen as enablers for more advanced standards such as guidelines and processes1. According to the responses, eHealth solutions should be based on generic technologies and cross-domain standards, but the specific requirements of healthcare were identified in relation to security, safety and privacy, for example.

The questions of the e-mail survey were also discussed in several meetings with various experts. The number of responses in the web-based survey was 10 (of 18) and in the e-mail survey and meetings 13 (of 18), the total response rate being 66,7 %. In addition to the surveys, interviews were performed with several experts responsible of the different areas of official and industry standardization. Furthermore, the participants of the project board reviewed the work in two mid-project meetings. The results of the literature survey and the data collection were then used by the authors who wrote the descriptions of current and target state, and other parts of the report.

Many respondents saw a clear need for many different types of participants in the standardization. Healthcare organizations were emphasized in the standardization of healthcare-specific information, processes and guidelines. In the technology standardization (including healthcare IT), the role of companies and standardization organizations was highlighted. The role of authorities was also seen central, especially in relation to the healthcare-specific aspects and architectural guidance.

The main result of the study was the description of current status and issues and a set of recommendations related to the standardization of healthcare IT. For the recommendations, the material of the study was combined with the personal experience of the authors. The results of the study were published in a 92-page report in Finnish for further actions [11].

Goals, problems and challenges Based on the literature survey and the results from the data collection, the national problems and challenges were identified and the target state was specified. In eHealth standardization, no organization can master all areas. This stresses the importance of coordination, the utilization of experts in different areas and the relationship between international and national standardization. In addition, clear relationships between healthcare information and processes, healthcare IT and technical standardization, and the requirement to base the solutions on the actual needs of the market are central. Some research results suggest that global standards succeed only if they can be adjusted to the local processes and activities, and they are applied or modified according to the local requirements [7, 12]. This observation was supported by our study.

Even though the work covered many areas, the application of standards was delimited to consider mainly health information systems. This was due to the fact that the majority of challenges of the national project and of the hospital districts were related to the interoperability of these systems. Excluded were also the devices and device interfaces, for example the medical equipment. In addition, the scope of the work was limited to open standards and specifications, and it did not consider internal implementation technologies of applications or development models such as open source.

Results

The local or national definition of responsibilities, guidelines and resources is needed for the selection and production of standards and recommendations [9]. The selection and production of specifications, steering of these activities, support for their use in products and pro-

The results of the surveys and interviews The respondents of the surveys and interviews saw the quality assurance and applicability of standards as a key factor in relation to the goals and policies of standardization. In addition, the usability requirements and pragmatism of standards were emphasized. Fast introduc-

1

471

This statement has been emphasized also in other research [7].

J. Mykkänen et al. / A National Study of eHealth Standardization in Finland - Goals and Recommendations

curements, and education and conformance evaluation are central parts of these activities.

standardization steering groups are proposed for overall coordination and the healthcare-specific standards. 2. The continuity of domain-neutral and healthcarespecific IT standardization must be assured using permanent funding from various departments. 3. The primary preference for all domains, including healthcare, must be given to cross-domain and generic standards. Healthcare-specific standards should be developed and introduced cautiously and only on areas where they are essential. 4. The participation to the international standardization work and the observation of international key developments in standardization must be intensified and resourced. The goal is to identify mature standards which solve current local needs and to avoid contradictions with the international standardization. 5. The participation of healthcare application vendors and health service providers in the development, localization and introduction of standards must be supported by funding projects which aim at standards compliance and by developing models to support the balanced participation to the standardization. 6. The national standardization in healthcare IT must primarily support the goals of the national health project, especially the interoperable electronic health records. This requires a quick prioritization of the most immediate areas and goals, and the establishment of realistic funding and scheduling for them. 7. The status, normativeness and mutual relationships of healthcare IT standards, guidelines and recommendations must be defined unambiguously and accurately. 8. To ensure the interoperability of information systems and to support the introduction of standards, a support and education network must be created with adequate expertise to promote these recommendations and to support the steering groups. In addition to these high-level main recommendations, the report identified 51 detailed goals and 127 recommended actions to support these goals. The policy recommendations were related to the organization of the standardsrelated activities, steering and funding. The relationship recommendations focused on improving the international and cross-sectoral linkages in standardization. For the quality assurance of standards and guidelines, 20 recommendations were made. Central recommendations were also related to the improved know-how for the system acquisitions and to the establishment of education and support related to standards. In addition, the support for the introductions and pilots involving standards, and balance in the participation of standards activities received recommendations. In addition, detailed recommendations were made for many areas of standardization and national policies. The most urgent recommendations were related to the core information sets, clinical documents, architecture and security of the national electronic health records.

The coordination and ownership of standardization activities was identified as one of the key challenges in standardization. The various earlier recommendations for the organization models of standardization have not been realized. The collaboration and knowledge of different activities should be increased. Even though standardization requires participation from the industry and their customers, the government can maintain a working standards infrastructure. This support can be realized as clear recommendations about the selected standards, clarification of responsibilities and guidelines such as national architecture for electronic health records. The linkage between the standardization and system introductions, and the creation of conditions which balance the participation in standards activities can also be centrally supported [9]. The quality assurance and the evolution of requirements require special attention in standardization. There is a contradiction between the fast introduction of standards and their quality [9]. The subtle balance between accuracy and flexibility depends on the type of the specification. The conformance and certification require accurate specifications and dedicated services. Implementation examples, support for the projects which introduce standards, and guidelines for the procurement are useful means for the quality assurance. The relationship between official and industry standardization has not been clearly defined. Industry standardization has been more agile in responding to the needs of the users. The free availability of key specifications promotes their use. In addition, the commitment and participation in standards-related work has been increasing very slowly. In particular, the companies and healthcare organizations have not participated actively in the international standardization. In general, successful standards require demand from the market, and standards can not be easily enforced by the officials. New standards and specifications which are potentially relevant for healthcare, health IT or technology aspects, are continuously emerging. Constant learning by the users and developers is required. The careful selection of the specifications to be studied, and education about standards are necessary to keep the workload reasonable. The knowledge of key areas and activities of standardization should be included in the education of healthcare IT professionals.

Discussion Main recommendations Based on the surveys and previous parts of the study, the following eight main recommendations were made: 1. The standardization relationship between the healthcare IT and the domain-neutral IT must be intensified. Shared national goals, policies and procedures for IT-related standardization must be specified. National

Current trends of interoperability standardization Some central trends of standardization are related to the evolution towards advanced eHealth interoperability.

472

J. Mykkänen et al. / A National Study of eHealth Standardization in Finland - Goals and Recommendations

Information systems are increasingly evolving towards networked and service-oriented solutions, which increases the role of standardization. In technical standardization, the aim is to increase the flexibility of systems to support different processes and adaptability. On the other hand, accurate functional and semantic interoperability is pursued, and processes and workflows themselves are increasingly standardized. In particular, profiles which constrain the standards are increasingly used to promote plug and play interoperability or automatic adaptability. Such profiles are emerging on the technology level [13], on the functional level [14], on the semantic level [4] and in the field of application of interoperability standards [15, 3]. In addition, emerging support services such as conformance testing and certification are closely related to standardization.

Acknowledgments This work is part of the SerAPI, ZipIT-ojo and ZipII-ojo projects, funded by the Finnish Funding Agency for Technology and Innovation TEKES grants no. 40251/06, 644/06, 40353/05, 644/ 05, 40437/04 and 790/04 together with a consortium of companies and healthcare organizations.

References [1] Haux R. Health information systems - past, present, future. Int J Med Inf 2006: 75 (3-4): 268-281. [2] Klein G. Standardization of health informatics – results and challenges. Yearbook of Medical Informatics 2002, 103-114, IMIA, 2002. [3] Current and future standardization issues in the e-Health domain: Achieving interoperability. Report from the CEN/ ISSS e-Health Standardization Focus Group. CEN, 2005. [4] Beale T. Health Information Standards Manifesto. Revision 2.5. Deepthought informatics, 2004. http:// www.deepthought.com.au/health/HIS_manifesto/ Output/ his_manifesto.html. [5] Iivari A, and Ruotsalainen P. Principles for national data system architecture in the field of health care; from regional solutions to a national entity. Working Group Memorandum, Ministry of Social Affairs and Health, 2005. [6] Guide to the Project Management Body of Knowledge (PMBOK Guide). Project Management Institute, 2000. [7] Feng P. Studying Standardization: A Review of the Literature. In: Proceedings of the 3rd conference on Standardi-zation and Innovation in Information Technology (SIIT2003). IEEE, 2003, pp. 99-112. [8] Boehm B. Some future trends and implications for systems and software engineering processes. Systems Engineering 2006: 9 (1): 1-19. [9] Swann P. The Economics of standardization - Final Report for Standards and Technical Regulations Directorate. Department of Trade and Industry. Manchester Business School, University of Manchester, 2000. [10] Health Informatics - Health Informatics Profiling Framework. ISO/TR 17119:2004(E), Technical report, ISO TC 215, 2004. [11] Mykkänen J, Korhonen M, Porrasmaa J, Tuomainen T, Ensio A. Tietojärjestelmien standardointityön organisointi ja kehittäminen terveydenhuollossa: nykytila ja toimenpideehdotukset. Osaavien keskusten verkoston julkaisuja 3/ 2005. Helsinki: Stakes, 2005. [12] Berg M, and Timmermans S. Standardization in action: Achieving local universality through medical protocols. Social Studies of Science 1997: 27(2): 273-305. [13] Basic Profile Version 1.1. Web services interoperability organization, 2004. http://ws-i.org/Profiles/BasicProfile1.1-2004-08-24.html [14] Dickinson G, Fischetti L, and Heard S, eds. HL7 EHR System Functional Model - Draft Standard for Trial Use. Health Level Seven, Inc., July 2004. [15] Vegoda P. Introducing the IHE (Integrating the Healthcare Enterprise) concept. J Healthc Inf Manag 2002: 16 (1): 22-24.

Conclusions Achieving the benefits of standardization in the eHealth domain requires coordination, the selection and development of standards on many areas, the identification of central actors and close collaboration between them. The different areas require specific expertise and multidisciplinary collaboration. The success of standards is measured only through their utilization on the market. The support for the introductions of standards, the clear scope of standards, quality assurance, and the availability of standards and their support services create a vortex of successful standardization activities. The national participation in international standardization and the evaluation and selection of standards are necessities in this process. The rapid evolution of IT and complementary standards offer many opportunities. However, generic standards require healthcare-specific profiling and evaluation, and healthcare also has specific standardization areas. In a wider context, standardization and the use of standards promotes competition and open market, and progresses economy as a whole. The recommendations of this study were published and given to the project board after the report was finalized in 2005. Since then, the Social Insurance Institute of Finland has been selected as the national actor for the development of national healthcare IT services. HL7 CDA (Clinical Document Architecture) R2, DICOM and HL7 version 3 Medical Records have been recommended as some of the key standards of the national EHR in relation to clinical documents, medical imaging and messaging, respectively. In addition, some technical specifications such as X.509 for certificates, http(s) and SOAP for data communications and WS-Security for mediated secure messaging have been recommended. Furthermore, some of the key recommendations related to the steering model of standards-related work are being refined as of March 2007.

Address for correspondence Juha Mykkänen University of Kuopio, IT Services Centre, HIS R & D Unit P.O.B. 1627, Fin-70211 Kuopio, Finland [email protected]

473

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Building a Womb-to-Tomb Health Record in Hong Kong – an Application of Information Architecture Vicky Funga, N T Cheunga, Eric Hob, Clara Cheungb, Hudson Chanb, Kitty Tsangb, Joycelyne Cheungb, William Hob, Maggie Laua, Veronica Hunga, Austen Wonga, Anna Tonga, W N Wonga, Antonio Seka a

b

Health Informatics Section, Hospital Authority, Hong Kong Special Administration Region Information Technology Department, Hospital Authority, Hong Kong Special Administration Region

unique identifier for the data subject and care provider, the carers and the facilities; terminology standards for recording, storing and reusing information and supporting decision support systems; messaging standards for exchanging information amongst systems; information model such that various standards are able to work together; and security standards to enable the information to be communicated within a consistent security framework [5, 6, 7, 8].

Abstract The Hospital Authority developed the Information Architecture (IA) model in 2002 to support a fast, robust, flexible and accurate electronic patient record (ePR) to meet the hightempo health care environment in Hong Kong. With several successful applications in sharing data that were created for the same patients in various systems, the IA model was further developed to extend the longitudinal ePR to include one’s fetal data as entered in the mother’s record. This paper describes how various IA elements: Section, View, Form, Group, Entity, Content, Document supports the building of a true womb-to-tomb ePR for the HA patients. The future focus of Information Architecture in the HA will include building a Information Architecture Management System and linking the ePR with other patient records in the community.

Some common terms to describe the ‘longitudinal’ record are ‘cradle-to-grave’, ‘womb-to-tomb’, or even ‘sperm-toworm’ [1]. There is no question that one’s health could be related to conditions one had during his/her fetal life. This relationship is so important that the World Health Organisation requests countries to report these prenatal conditions that affect one’s health using International Classification of Diseases (ICD) [9]. Yet we find little discussion on how to electronically link the health record of one’s fetal life to that created after one’s birth. A powerful generic Information Architecture allows us to share data from the record of one’s immediate family and utilize this data in subsequent health care. This paper describes how the Hospital Authority (HA) realized the development of a true ‘womb-to-tomb’ record.

Keywords: Information Architecture, electronic health record, terminology, standards, patient master index

Introduction Healthcare is a person-centred process with multiple carers providing care to a person at different times at various locations. With the aim to improve a person’s health, a nation’s health and the world’s health, developing a longitudinal electronic health record (EHR) is one of the greatest challenges to the health informatics field.

Clinical information systems in the HA The Hong Kong Patient Master Index (HKPMI) At HA, all patients are uniquely identified with their Hong Kong Identity Number (HKID), a corporate based patient identifier. To date, the HKPMI has around 6 million unique HKIDs of living Hong Kong residents, which accounts for around 83% of the Hong Kong population. For each admission, outpatient visit or emergency attendance, a separate episode number is created. The episode number serves as a unique identifier to link all clinical information that created in that particular episode. The HKID links all clinical information in various episodes under the same patient.

Electronic health record refers to ‘a repository of information regarding the health of a subject of care in computer processable form’ [1]. It is essential to ensure the longitudinal EHR is functionally and semantically interoperable such that the systems are able to exchange and further process the received information which is understood with formal defined domain concepts [1,2]. In recent years, various countries started their own initiatives to build health information infrastructure to facilitate a person to access one’s health record regardless of where, when and by whom the record is created [3, 4, 5]. The keys to building a ‘longitudinal’ EHR include a universal

The Clinical Management System & ePR The Hospital Authority developed the Clinical Management System (CMS) from 1994. The CMS is an integrated

474

V. Fung et al. / Building a Womb-to-Tomb Health Record in Hong Kong – An Application of Information Architecture

workstation for doctors to directly enter clinical orders and patient documentation. To create a record in the CMS, there must be an episode in the Hong Kong Patient Master Index (HKPMI). From 2000, all essential clinical data from all HA hospitals and clinics were consolidated in the ePR repository, building up an enterprise-wide, longitudinal patient record [10]. Currently, the ePR houses 6.3Tb data (not including radiology images) with the records of 7.9 million patients, and garners 300,000 hits per day from clinical users. The Obstetrics Clinical Information System The Obstetrics Clinical Information System (ObsCIS) was first implemented in the HA hospital in 1994. The ObsCIS aims to support ongoing patient care and clinical audit. In 2005, out of 57,124 births in Hong Kong, 41,258 (72%) of them were born in HA hospitals, and the antenatal, delivery and postnatal information for all these mothers and newborns are recorded in the ObsCIS.

Figure 1 – The Information Architecture Model Every medical fact has a concept Central to the IA model is the “concept” attached to Entity, Document and Content. All Entity descriptions are added to the HA Clinical Vocabulary Table (HACVT) [12]. Each HACVT term is uniquely identified with a TermID. Where applicable, these terms will be mapped to reference terminologies, such as, SNOMED CT, or LOINC, ICD 9 CM, or ICD 10 to facilitate subsequent interpretation and data retrieval.

Information Architecture in the HA In 2002, HA developed a simple yet generic model, HA’s Information Architecture (IA), with the aim to build a fast, robust, flexible electronic patient record (ePR) from data being captured in various systems [11]. The IA framework was developed based on the following principles :

Contents may be codes, numbers, text or even images, and since coded Contents is referenced to the HACVT as well, mapping to the reference terminologies may be required.

• •

Simple – easily understood, and implemented Generic – able to apply to data captured in any technical environment • Flexible – support different presentations of the same data • Concept oriented – ensure the consistency of the meaning of the information More recently, the model has been refined with groupings of Entities into Forms and Groups to facilitate data capture and management. Under this model (figure 1), medical facts in ePR are described with : • • • • • • •

Every medical fact has a context Under the IA, all medical facts are stored in Entities. Entities are related through Groups and Forms. The IA defines relationships between these constructs, insuring that the context of data is preserved. Each Entity is identified with a unique EntityID. Each EntityID, apart from storing Entity description, also includes a set of attributes, such as data format or repetitive indicator. Every medical fact has a presentation

Sections – the macroscopic structure of ePR Views – dynamic presentations that fit according to various requirements Forms – source from which Entities are originated from Groups – logical grouping of Entities to facilitate data entry and subsequent data reporting and analysis Entities – the label that stores data Contents – the value that being stored, this could be a code, text data Documents – a special type of Content with human readable images

Sections and Views define how information is organized and presented in ePR. To support flexible data retrieval, the same Group / Entity / Content / Document can be put under different ePR Sections. For example, Psychiatric Nursing Discharge Summary document can be placed under Nursing Section and also the Psychiatric Summary Section. Although in 2002 other models were already in development, e.g. HL7’s Clinical Document Architecture [13], ASTM’s standard for the patient record [14], the Good Electronic Health Record [15], HA’s IA was able to scale rapidly to the demand of data retrieval on an enterprisewide scale in a busy clinical environment. The IA model takes a comprehensive approach in managing information from its generation to how it is used, presented and aggregated. It allows semantic interoperability between the different modules and subsystems of the CMS. IA helps the HA to build a flexible, comprehensive ePR which meets the high volume of data retrieval with

475

V. Fung et al. / Building a Womb-to-Tomb Health Record in Hong Kong – An Application of Information Architecture

subsecond response whilst still allowing advanced functionality.

The journey begins The mother-baby relationship table Since 2003, all newborns born in HA hospitals are admitted and a medical record is created. This allows clinicians to record the newborn’s data, e.g. laboratory results, directly in the newborn’s own CMS record instead of keeping them in the mother’s CMS record as before.

Figure 2 – Retrieving birth data for baby’s discharge summary

In 2005, it was decided to share the mother’s delivery data to the baby’s record. The first step is to ensure there is a correct linkage between the mother and the baby in the HKPMI.

A Section on Birth Record is created in the baby’s ePR to display the baby’s birth data (figure 3).

The mother’s delivery episode number and the newborn’s birth episode number are stored in the HA Mother-Baby Relationship Table (M-B table) which is incorporated in the HKPMI. This forms the basis for subsequent sharing of data from a mother’s ObsCIS record to the newborn’s record. Figure 3 – Birth record in baby’s ePR

Data to be shared The project is governed by the Perinatology Committee with representatives of senior clinicians from all Obstetrics and Paediatrics Department of the HA hospitals. The committee agreed that the following information is shared from the mother’s ObsCIS:

Subsequent application In 2006, a Neonatal Form was developed based on the IA and the same data sharing model. The ObsCIS data and mother’s antenatal problems are shared to the Neonatal Form via the ePR repository.

• birth datetime • maturity • mode of delivery • duration of membrane rupture • birth weight • Apgar score at 1, 5 & 10 minutes • related diagnoses and procedures For diagnoses and procedures, only those related to the fetus, the uterus, the membranes, liquor, and umbilical cord are passed to the baby’s record. Examples include preterm labour, cord presentation, oligohydromnios, fetal distress, and fetal growth retardation.

The way forward One of the greatest challenges in the use of the IA is to identify which Entities and which Contents should be referenced to terminologies like SNOMED CT. The CMS includes modules that are developed purely for patient care purpose but also some modules developed to facilitate auditing. Given the diversity, it is neither necessary nor worthwhile to reference all Entities and Contents to a Reference Terminology. Work started with defining Entities and Contents for ancillary services such as laboratory, and radiology – data will most likely contribute to the development of decision support systems – and has proceeded to other areas.

Sharing mother’s ObsCIS Data to baby’s ePR & CMS

Despite SNOMED CT [16] being the single largest clinical terminology covering quite a number of domains, an internal evaluation of SNOMED CT indicated that it is better developed in areas like findings, but would need improvement in terminologies on others like procedures and allied health. More work, such as using post-coordination, is required to use SNOMED CT to represent concepts for the whole spectrum of healthcare domains. The requirement of post-coordinated terms also creates challenges for future data retrieval and analysis [17].

Each of the selected ObsCIS information to be shared is assigned with a unique EntityID and referenced to SNOMED CT where applicable. With the same ObsCIS diagnosis / procedure, separate HACVT terms are assigned for the mother’s and the baby’s records to ensure correct ICD codes are generated in the corresponding records. The EntityID and TermID, plus the baby’s episode number is passed to the ePR repository where the information is separately stored under both the mother’s and the baby’s episode number. Birth data and related diagnoses and procedures will be retrieved from the ePR repository and stored in the baby’s CMS and ePR record.

Given the massive number of Entities and Contents in the HA CMS, the need of a tool to manage these Entities and Content is inevitable. To this end, the Information Architecture Management System (IAMS) is being explored. IAMS aims to provide robust functionalities such as semantic locality and lexical matching to facilitate the management of terminologies. Functions supported

Doctors can retrieve the baby’s birth data while preparing the discharge summary (figure 2).

476

V. Fung et al. / Building a Womb-to-Tomb Health Record in Hong Kong – An Application of Information Architecture

include mapping concepts to classification, managing the terminology lifecycle and managing reference information [18, 19]. In addition, IAMS will also manage relationship between individual IA elements.

[5] Canada Health Infoway. 2004. Electronic Health Record (EHR) Standards Needs Analysis. Retrieved 20 Oct 2006 from website : http://www.infoway-inforoute.ca/en/ WhatWeDo/Overview.aspx [6] Connecting for Health. The Data Standards Working Group final report. Markle Foundations; 2003 Jun 5. [7] Philip Aspden, Janet M. Corrigan, Julie Wolcott, Shari M. Erickson, Editors, Committee on Data Standards for Patient Safety. 2003. Patient safety: achieving a new standard for care. Institute of Medicine. [8] Fung V. White paper on standardization of health data. Retrieved 1 Oct 2006 from Hong Kong Special Administrative Region, Office of Government Chief Information Officer website: http://www.ogcio.gov.hk/eng/ infra/download/consortium_whitepaper_0609.pdf [9] World Health Organisation. 1993. International statistical classification of diseases and related health problems, 10th revision. [10] Cheung NT, Fung KW, Wong KC, Cheung A, Cheung J, Ho W, Cheung C, Shung E, Fung V, Fung H. Medical informatics – the state of the art in the Hospital Authority. International Journal of Medical Informatics. 2001 Jul;62 (2-3):113-9. [11] Cheung NT, Fung V, Kong JH. The Hong Kong Hospital Authority’s information architecture. Medinfo. 2004; 11 (Pt 2):1183-6. [12] Fung V, Cheung NT, Szeto K, Ngai L, Lau M, Kong JHB. Hospital Authority Clinical Vocabulary Table – the past, the present, and the future. 2004 International Federation of Health Records Organization Congress Proceedings. [13] Dolin RH, Alschuler L, Beebe C, Biron PV, Boyer SL, Essin D, Kimber E, Lincoln T, mattison JE. The HL7 Clinical Document Architecture, Journal of American Medical Informatics Association. 2001 Nov-Dec;8(6):552-69. [14] ASTM. E1384-02a, Standard Practice for Content and Structure of the Electronic Health Record (EHR), 2006. [15] Schloeffel P, Beale T, Heard S, Rowe D, Background and overview of the Good Electronic Health Record (GEHR), retrieved 15 Sept. 2003 from GEHR website: http:// www.gehr.org/Documents/BackgroundOverview_ [16] SNOMED Clinical Terms, retrieved 20 Sept 2006 from SNOMED website : http://www.snomed.org/snomedct/ documents/july_2006_release.pdf [17] Dolin RH, Spackman KA, Markwell D. Selective retrieval of pre- and post-coordinated SNOMED concepts. Proceedings of the American Medical Informatics Association 2002 Annual Symposium. [18] Chute CG, Elkin PL, Sheretz DD, Tuttle MS. Desiderata for a Clinical Terminology Server. Proceedings American Medical Informatics Association Symposium. 1999;:42-6. [19] Rector AL, Solomon WD, Nowlan WA, Tush TW. A terminology server fro medical language and medical information systems. Methods of Information in Medicine. 1995 mar;34(1-2):147-57.

Currently, the mother’s ObsCIS data are shared to the baby’s record at birth. For singleton, it is clear to whom the identified information belongs. More work would be required for linking correct fetal data to records for babies of multiple pregnancy. Another challenge to be addressed is to protect the mother’s privacy. The ObsCIS is adding a ‘confidential flag’ to indicate the mother particularly requested to keep the information confidential. This flag will also be passed to the baby’s record to alert paediatricians to be more careful when discussing the baby’s condition with a third party, e.g. baby’s father, where applicable.

Conclusion IA is a relatively simple but comprehensive concept when compared with other development in this area. Emphasizing on context, concept and presentation, the model can be flexibly applied to facilitate the building of a dynamic ePR while retaining the semantics of the data captured. The HA will complete building a platform to share data with the Department of Health (DH) by March 2007. Discussion has started on sharing the newborn baby’s discharge summary to the Maternal & Child Health Centre (MCHC) which is under the DH management. With nearly 90% of babies born in Hong Kong attending the MCHC, and 72% of babies born in HA hospitals, linking the HA and the MCHC baby’s record marks another significant step in building a womb-to-tomb record for Hong Kong residents. Acknowledgments We would like to thank members of the Obstetric & Gynaecology Coordinating Committee with their contribution to the development of the Obstetrics Clinical Information System. We are grateful to the Perinatology Committee for their support in the design and implementation of linking mother’s and the baby’s record.

References [1] ISO TC 215. 2005. ISO/TR 20514. Health informatics – electronic health record – definition, scope and context. [2] Norgall T. Interoperability – a key infrastructure requirement for personalized health services. Studies in health technology and informatics. 2005;117:125-33. [3] A strategy for building the National Health Information Infrastructure. National Committee on Vital and Health Services. Retrieved 4 May 2006 from website : http:// aspe.hhs.gov/sp/NHII/Documents/NHIIReport2001/ default.htm [4] National Programme for IT, National Health Services. Retrieved 24 Oct 2006 from website : http:// www.connectingforhealth.nhs.uk/

Address for correspondence Vicky Fung Rm 121N, Hospital Authority Building 147B Argyle Street Kowloon Hong Kong Email: [email protected]

477

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Another HISA – The New Standard: Health Informatics – Service Architecture Gunnar O. Kleina, Pier Angelo Sottileb, Frederik Endsleffc a

Dept. of Medcine, Karolinska Institutet, and Cambio Healthcare Systems AB, Stockholm, Sweden b GESI, Gestione Sistemi per l’Informatica, Rome, Italy c Dept. of Informatics, H:S, Copenhagen, Denmark

and cost-effective way whilst on the other hand it is vital to ensure the consistency and integration of the overall organisation, both at local and territorial level. This integration requirement is not only related to the need for improving clinical treatment of the subject of care but is also demanded by the necessity of to control and optimise the current level of expenditure for health.

Abstract In addition to the meaning as Health Informatics Society of Australia, HISA is the acronym used for the new European Standard: Health Informatics – Service Architecture. This EN 12967 standard has been developed by CEN – the federation of 29 national standards bodies in Europe. This standard defines the essential elements of a Service Oriented Architecture and a methodology for localization particularly useful for large healthcare organizations. It is based on the Open Distributed Processing (ODP) framework from ISO 10746 and contains the following parts: Part 1: Enterprise viewpoint Part 2: Information viewpoint Part 3: Computational viewpoint This standard is now also the starting point for the consideration for an International standard in ISO/TC 215. The basic principles with a set of health specific middleware services as a common platform for various applications for regional health information systems, or large integrated hospital information systems, are well established following a previous prestandard. Examples of large scale deployments in Sweden, Denmark and Italy are described.

It is today common with a large number of different databases and applications, isolated and incompatible, already operational in healthcare organisations to support specific needs of users. Even within the same centre, information systems are frequently fragmented across a number of applications, data and functionalities, isolated and scarcely consistent with each other. A main need for care delivery organisations is to integrate and to make available the existing information assets, with the interoperability of existing applications, thereby protecting investments made. During integration activities, continuity of service needs to be achieved whilst gradual migration of existing proprietary, monolithic systems towards the new concepts of openness and modularity occurs. The cost-effectiveness of the solutions, especially when projected on the scale of the whole healthcare organisation, represents a crucial aspect to be evaluated carefully.

Keywords:

The goal can be achieved through a unified, open architecture based on a middleware of information services independent from specific applications and capable of integrating common data and business logic. These services shall be made available to diverse, multi-vendor applications through many types of implementations. All aspects (i.e. clinical, organizational and managerial) of the healthcare structure must be supported by the architecture. This must be able therefore to comprise all relevant information and all business workflows, structuring them according to criteria and paradigms independent from specific subdomain aspects, temporary requirements or vendor specific technological solutions.

medical informatics, hospital information systems, standard, SOA, ODP, HISA, CEN, middleware, data-carrying integration platform

Introduction Healthcare structure consists of networks of units over a territory characterised by a high degree of heterogeneity and diversity, from organisational, logistic, clinical, technological and cultural perspectives. The structure of individual centres such as hospitals of different sizes and outpatient clinics including primary care is evolving from a vertical, aggregated organisation towards the integration of a set of specialised functional areas. These need to share common information and to operate according to integrated workflows within and between centres of a larger enterprise e.g. a county or a large city.

Standards and technological solutions already exist and will continue being defined for supporting specific requirements, both in terms of in situ user operations and with respect to movement of information. The architecture must be able to accommodate such requirements by allowing the specific models to be integrated with the complete information assets of the healthcare organisation and the

On the one hand it is necessary to support the specific requirements of each unit or user in the most appropriate

478

G. O. Klein et al. / Another HISA – The New Standard: Health Informatics – Service Architecture

communication messages to be “services” extracting or importing data from/to the common information as shown in Figure 1.

Open distributed processing The standard framework for Open Distributed Processing was first developed by the Object Management Group (OMG) and later approved by ISO/IEC as the International Standard ISO 10746 from 1996. This standard contains a rich set of specification elements and recommendations for the development of open distributed systems. It has successfully been applied in a large number of different industry sectors including, telecom and banking. At the time of issuing the first ODP standard, there was no available standard for information modeling. Since then UML (the Unified Modeling Language) has been developed and since it has been adopted as the method of choice for health information modelling by both CEN and ISO and many other organizations, UML was selected as the information modelling language for the new HISA standard.

Figure 1 - A Service Oriented Architecture

The ODP framework contains five viewpoints. In the HISA standard we have provided health specific advice and definitions for the three upper levels. The two lower viewpoints, Technology Viewpoint and Engineering Viewpoint are applicable for a concrete development and implementation project but there is no point in providing health specific restrictions in a standard.

The purpose of this standard is twofold: – to identify a methodology to describe healthcare information systems through a language, notation and paradigms suitable to facilitate the planning, design and comparison of systems; – to identify the fundamental architectural aspects enabling the openness, integration and interoperability of healthcare information systems. The architecture is therefore intended as a basis both for working with existing systems as well as for the planning and construction of new systems.

Results Part 1: Enterprise viewpoint The Enterprise Viewpoint specifies a set of fundamental common requirements at the enterprise level with respect to the organisational purposes, scopes and policies that must be supported by the information and functionalities of the middleware. It also provides guidance on how one individual enterprise (e.g. a regional healthcare authority, a large hospital or any other where this model is applicable) may specify and document additional specific business requirements. This should aim at gradually achieving a complete specification, adequate for all the characteristics of that enterprise.

Materials and methods The standards body and process This standard was developed within CEN/TC 251 (www.centc251.org) which is the technical committee for Health Informatics within the federation of 29 European national standard bodies. The work in health informatics has been mandated by the European Union and the European Standard is published as a national standard in all of the member countries following the approval based on a weighted vote. In this case only one country was opposing the approval.

The strategic paradigm The specification of the architecture shall start with a very concise, managerial-oriented document (the “Strategic Paradigm”) that identifies (at a high level of abstraction) the overall requirements and strategic objectives of the envisaged system. It describes, in natural language:

This work was developed within Working Group I: Information models by the Task Force HISA led by Gunnar Klein, who was at the time also chairman of CEN/TC 251 and with the two other authors as member of the core group with Frederik Endsleff having a particular responsibility for part 1 and Pier Angelo Sottile for parts 2 and 3.

– the rationale and the scope of the IT system with respect to the overall enterprise; – the fundamental organisational processes (as defined under terms and definitions) that can be identified in the enterprise and that are relevant for the envisaged system; – the fundamental constraints and objectives to be satisfied. The HISA standard gives important guidance for an Enterprise on how to model its detailed requirements. The importance of starting from a model of the care processes

Experts from the following countries also contributed actively to the development of this standard: Belgium, France, Germany, Hungary, Italy, Norway, Sweden and UK but also as non-European contributors Australia and the USA. During the extensive formal review process in several stages a total of 112 written submissions where adding to the final version.

479

G. O. Klein et al. / Another HISA – The New Standard: Health Informatics – Service Architecture

is emphasized and the following model is offered to clarify the different processes.

Figure 2 - Subdivision of the care process

Figure 3 – High level Information Model

The Enterprise Viewpoint deals with the requirement specification and contains use-cases, process descriptions and models leading to the identification of the overall Service Architecture and the basic clusters of objects. The following are identified:

HISA Information Objects in each package shall be classified as operational or descriptive: – “Operational”, usually representing the actual (clinical, Organizational, etc.) objects that are continuously generated during (and for) the daily activities. These include the personal and healthcare treatment information on patients, the individual resources used for carrying out the actual activities, etc.. – The operational information objects model the entities involved in the daily activities of the healthcare enterprise in the treatment of subjects of care and in the functioning of the enterprise itself. – “Descriptive”, usually Organization-related, specifying the criteria according to which the Organization works and is organized. It includes general classifications of clinical concepts, rules according to which the activities are performed, and more (e.g. the types of activities which are carried out in the radiology department, the diagnostic classification in use in the clinical setting, etc.). The descriptive information objects model the entities required for the overall knowledge base that is required by the healthcare enterprises to carry out daily activities related to the treatment of subjects of care and in the functioning of the enterprise itself.

• Subject of Care Workflow • Activities management workflow • Clinical Information Workflow • Management of authorizations • Management of resources • Management of dictionaries and coding • Interactions with other systems In the standard, there are detailed use case descriptions specifying common requirements on information management. It should be emphasized that this standard does not claim to have the description of all possible requirements, only the commonly shared processes of healthcare enterprises. Part 2: Information viewpoint The Enterprise Viewpoint of the HISA standard has identified certain processes. The Information Viewpoint is detailing the information structures using UML class diagrams. The information model is specified without any – explicit or implicit- assumption on the physical technologies, tools or solutions to be adopted for its physical implementation in the various target scenarios. The specification is nevertheless formal, complete and non-ambiguous enough to allow implementers to derive an efficient design of the system in the specific technological environment that will be selected for the physical implementation.

For each “operational” information object, therefore, the model foresees one “descriptive” information object, containing the main classification data, the properties, the rules and the default values that are necessary for the management of the live data instantiated in the “operational” object.

This specification does not aim at representing a fixed, complete, specification of all possible data that may be necessary for any requirement of any healthcare enterprise. The information model of the standard only specifies the set of characteristics identified as fundamental and common to all healthcare organisations.

The HISA information model is where relevant using existing standards. One example is the European EN 14822: Health Informatics - General Purpose Information Components which is based on the HL7 Reference Information Model. Part 3: Computational viewpoint This part of the standard specifies the fundamental characteristics of the computational model to be implemented by a specific architectural layer of the information system (i.e. the middleware) to provide a comprehensive and integrated interface to the common enterprise information and

480

G. O. Klein et al. / Another HISA – The New Standard: Health Informatics – Service Architecture

to support the fundamental business processes of the healthcare organization.

•

Order management for lab, imaging and consultancy services The same system is also used in the counties of Kronoberg, Jönköping, Östergötland, Västmanland, Kalmar and Värmland in Sweden and also in Odense in Denmark and in the Faroe Islands.

The computational model provides the basis for ensuring consistency between different engineering and technology specifications (including programming languages and communication mechanisms) since they must be consistent with the same computational object model. This consistency allows open inter-working and portability of components in the resulting implementation.

The “Policlinico A.- Gemelli” in Rome “Università Cattolica del Sacro Cuore” The UCSC information system consists of several applications based on a common architectural approach suitable to ensure the integrity and consistency of the information assets of the organisation, from the clinical, organisational and managerial point of view.

The basic computational objects, corresponding to the information objects, will be equipped with standard lowerlevel basic interfaces having the scope of adding, updating and deleting –in short maintaining-, listing, and getting one instance of the main classes described in the information viewpoint. These basic methods allow the access to and the manipulation of each element of the underlying model and secure the openness of the system.

It relies on a healthcare-specific middleware product from GESI (the DHE®, Distributed Healthcare Environment) of services allowing different applications to access the common information heritage and to perform common business processes through a set of services. On top of the DHE, several applications provide specific support to the user activities.

The higher-level computational objects implement more complex business transactions on the objects of the information model, simplifying and ensuring consistency of developments for common fundamental procedures of the organisation.

The following figure 4 shows at a high level of abstraction, the overall heterogeneous structure of the healthcare information system of the hospital and the centrality of the HISA-based DHE middleware.

Examples are: •

• • •

Patient/person area, including registering a person, Patient Administration (ADT), merging patient identifiers, period of care, etc. Activity management and life cycle, including requests, planning, booking, etc. Clinical and EHC record, including terminologies, classifications, problem-orientation, etc. Resource management, including standard usages

Examples of use of HISA based architectures In Europe there are a number of large healthcare organizations that have based their strategic planning on the HISA middleware principles and various technical solutions exist from different vendors and technical generations. Here are a few examples: UppsalaCounty Council, Sweden This is one of seven healthcare regions in Sweden that have decided to use the Cambio Spider middleware product. It is used for all its three hospitals and 35 primary care centres in one installation. This HISA implementation is based on a modern Java (J2EE) architecture with choice of Application Server and SQL database.

Figure 4 - The Gemelli system in Rome Copenhagen Hospital Corporation (H:S), Denmark

This system handles healthcare for all the 300 000 inhabitants plus as a regional highly specialised care for a million people. Today there are around 10 000 daily users of the system that includes, the following applications on top of the HISA platform:

H:S selected in 2002 the DHE product to implement its HISA strategy for 6 hospitals, in all comprising 4000 beds. The DHE serves as the joint data-carrying Integration Platform, forming the basis and common information heritage for a number of clinical and managerial applications.

• • • •

Among the applications are the medication module and the master patient index with over 10.000 users, fully rolled out and in integral daily running operation throughout the organisation.

Care Administration Resource Planning Electronic Health Record Medication management including electronic transfer of prescriptions

481

G. O. Klein et al. / Another HISA – The New Standard: Health Informatics – Service Architecture

H:S is from 1 January 2007 being merged with a.o. Frederiksborg County and Copenhagen County, into the Capital Region, responsible for regional healthcare of all of the region, including 10 hospitals and 1.6 million inhabitants. The master patient index and the medication module is for this purpose currently undergoing rollout in the Frederiksborg County.

common information and for performing common business logic

Several further DHE based applications are under current advanced development, such as Reporting, Accounting and the Patient portal, utilising the common information heritage.

Conclusion

This standard is not an alternative but a complement to other standards for health informatics such as specific messages developed by e.g. HL7 or the general EHR communication standards from CEN and openEHR [4].

After many years of research and standardization activities, there is now a formal standard for a Service Oriented Architeture (SOA) specific for the requirements of health care enterprises based on the general principles of Open Distributing Processing. It is now a European Standard but it has been submitted to ISO and is formally under consideration to become an International Standard.

Information is exchanged not only with the direct HISA interfaces (provided by the DHE) but also through messaging with other standards (utilising the underlying HISA interfaces) e.g.: •

•

The Danish national implementations of European standards in Edifact for e.g. laboratory communication and ePrescribing (MedCom). Information exchange with other messaging standards such as HL7, XML and Diagnostic Equipment such as for ECG and Imaging

Acknowledgements This work has been made possible through a large number of expert contributions from many countries in different phases from 1993-2006. In the first phase during the development of the European prestandard, the work was financially supported by the European Commission.

Discussion

References

HISA is a new standard defining a "Service Architecture” identifying the general principles to secure openness and vendor-independence:

[1] EN 12967-1: Health informatics – Service Architecture – Part 1: Enterprise Viewpoint. [2] EN 12967-2:.Health informatics – Service Architecture – Part 2: Information Viewpoint. [3] EN 12967-3: Health informatics – Service Architecture – Part 3: Computational Viewpoint. [4] Blobel BG. Advanced EHR architectures-promises or reality. Methods Inf Med 2006; 45(1): 95-101.

a) information must be separated from specific applications and accessible through services b) service logic must be independent from technological issues (i.e. multiple technologies and mechanisms must be allowed for accessing the same services) HISA is also identifying the fundamental elements of a comprehensive information model capable of supporting the whole healthcare organisation and finally the fundamental characteristics of a set of services for managing

Address for correspondence Gunnar O Klein, can be contacted at Karolinska Institutet, SE 171 77 Stockholm, Sweden E-mail: [email protected]

482

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Protecting Privacy while Sharing Medical Data Between Regional Healthcare Entities Tyrone Grandisona, Srivatsava Ranjit Gantab, Uri Braunc, James Kaufmana a

IBM Almaden Research, 650 Harry Road, San Jose, California 95120 b Pennslyvania State University, University Park, PA 16802 c Harvard University, Cambridge, MA 02138

ronment. A limited number of RHIOs exist today in the US and they vary in the ways they approach data sharing. The formation of a RHIO is based on the understanding that all the stakeholders agree to follow a specific set of guidelines.

Abstract Economies of scale, corporate partnerships and a need to increase the efficiency of Information Technology in the Healthcare sector are leading to the construction of Regional Health Information Organizations (RHIOs) across the United States. RHIOs are normally aligned by service provision given by particular healthcare payers (e.g. Blue Cross-Blue Shield, PacifiCare etc.) in particular geographies. Globalization has created a transient workforce that may require their healthcare provider access their patient data across several sovereign RHIOs. The barrier to enabling RHIO to RHIO collaboration lies in the need to respect the data disclosure policy of each RHIO, to adhere to the geography-specific healthcare legislation and also to not violate the express privacy wishes of the patient(s) involved. In this paper, we propose a datalevel control called Sticky Policy Enforcement which allows sharing to occur across RHIOs, while adhering to the concerns mentioned. Keywords: privacy, healthcare systems, collaboration

Introduction

Figure 1 - Regional Health Information Organization (RHIO)

Beyond the move to transform the American healthcare landscape by leveraging Information Technology to deliver better care, to reduce medical errors and to improve the quality of life, the existing healthcare topology dictates that computer networks be built that preserve the established business alliances that exist between payers and providers. This motivates the formation of connected information centers called Regional Health Information Organizations (RHIOs). The mandate to create a National Healthcare Information Network (NHIN) in the United States is based on the emerging existence of these RHIOs [1, 2]. A NHIN is a realization of a collaborative network of Healthcare Information Systems.

These guidelines detail policies regarding access, dissemination and processing of the patient data in the RHIO. Currently, the mechanism for cross-RHIO information sharing differ so greatly that collaboration across RHIOs to deliver care is still a daunting problem, but critical for the successful adoption of RHIOs. One of the key challenges is the protection of patient privacy when clinical documents are shared. This challenge is compounded by the facts that there can be no assumption of a central authority; that enforcement may involve multiple privacy policies based on source, destination and the documents involved in the transfer and the fact that data can be forwarded to an entity with additional rights, such as remote update rights.

Formally, a RHIO is an independent regional collective that facilitates the development, implementation and application of secure health information exchange among participating care providers. Each RHIO has independent policies regarding the privacy of health records stored within the RHIO. Figure 1 presents the typical RHIO envi-

This paper presents a first step towards addressing these concerns and marches healthcare systems towards achieving inter-RHIO collaboration while adhering to data disclosure constraints (e.g. privacy and security concerns).

483

T. Grandison et al. / Protecting Privacy while Sharing Medical Data Between Regional Healthcare Entities

The technology is called Sticky Policy Enforcement and provides a way to ensure that policy constraints are enforced wherever patient data travels.

enforcement. As only policy constraints that apply to the disclosed data are transferred, the communication impact is relatively small and the system does not require prior agreement among all medical organizations, states and patients.

In this paper, we will provide a discussion on the related work in the field, highlight the Sticky Policy architecture, demonstrate the technology by walking through the enforcement steps and provide concluding remarks.

Hippocratic database technology Our solution approach to the distributed privacy policy enforcement problem leverages the Hippocratic Database (HDB) Active Enforcement (AE) technology [6], which provides cell-level, policy-based disclosure management functionality, such that databases only return data that is compliant with company policies, applicable legislation, and customer preferences. The AE component ensures that enterprise applications accessing a database adhere to finegrained data disclosure policies. These policies, which may be security policies, privacy policies or data management policies, are distilled from the companys own policy, legal and regulatory requirements, customer preferences, and opt-in and opt-out choices. The component automatically rewrites user requests (i.e. queries) and returns only data that is consistent with these policies, allowing applications to enforce disclosure policies on arbitrary data elements at query execution time. A detailed description of HDB AE can be found in [7]. A quick overview of its operation is that, for a centralized or federated data system, HDB AE allows the definition of fine-grained disclosure policy, the creation of user preferences, the resolution of conflicts between preferences and policy and the enforcement of all the applicable constraints in an architecture that is application and database agnostic.

Related work The first mention of Sticky Policies appeared in computer science literature at the start of the 21st century [3]. It emerged from IBMs work on Enterprise Privacy Authorization Language and was recognized as a concept that is important for privacy preservation in distributed computer systems. The underlying notion behind Sticky Policy Enforcement is that the policy applicable to a piece of data travels with it and is enforceable at each point it is used. Though identified over a half a decade ago as a critical problem, application-independent solutions that were technically feasible and scaleable were not realized. Rivest and Lampsons work [4] embodies the earlier efforts in this space. Their focus was on the establishment of trust for a single disclosure object with a single policy. A data recipient is either granted access to the entire document, or must request authorization from the source. In healthcare environments, this is not sufficient. Sticky policy functionality should handle data disclosure to a party with welldefined constraints that allow data release to less privileged parties without requiring the originators involvement. This avoids the potential pitfall of having to contact a (potentially) large number of third parties before making a decision to disclose a specific piece of information.

HDB Active Enforcement technology was chosen as our platform because: (i) it offers a general platform for handling and codifying privacy policy and preference information; (ii) its enforcement mechanism is transparent to enterprise applications (integration currently assumes a database interface such as ODBC or JDBC); (iii) it is agnostic to underlying database technology; (iv) it allows policy changes without any modifications to the applications in use; and (v) in the typical case, it improves query processing speed.

The work by the Trusted Computing Group (TCG) consortium [5] represents another, more popular, approach to establishing the trust in single object, single policy environments. The general concern with the traditional approaches is that they require targeted application development and are not application and data agnostic, which is a mandatory requirement for situations with a complex web of pre-existing infrastructure, which are likely to be from differing vendors and running different, even proprietary, systems. Additionally, an ideal approach to Sticky Policy Enforcement should account for the fact that data changes occur frequently. It is not clear how approaches like that of the TCG would handle this without incurring a severe performance penalty.

Sticky privacy policy The format of our Sticky Policy package (Figure 2) consists of three parts: Data, Policy and Audit information. The Data segment contains the health documents requested. The Policy segment embodies the policy constraints applicable to the documents to be made available. The semantics of the policy entries are: 1. Requestor: The entity requesting access to part(s) of clinical document(s) from the source. The values for this entry could be taken from the roles mentioned in CDA standard. 2. Recipient: The entity that will be the final consumer of the data. The domain of possible values is similar to the set used for a requestor. 3. Purpose: The purpose for which the document(s) is being requested. 4. Retention: Time period until which access to the data is allowed. This could be computed based on various organizational policies.

System description The goal of our system is to enable distributed privacy policy enforcement. The difficulty in achieving this lies in the fact that there is no single entity with a priori access to all the policy constraints applicable to a document in a given state of the system. Our solution to this problem involves identifying the applicable privacy policy constraints for a document(s) to be shared and sticking them together, forming a single entity of transfer. In taking the approach of packaging policy with data, we maintain centralized decision making in a distributed

484

T. Grandison et al. / Protecting Privacy while Sharing Medical Data Between Regional Healthcare Entities

5. Copy-forward: The condition specifying whether the recipient is entitled to forward the requested document(s) to a third party after copying. The set of possible values are:

tampered with, and, 2) to ensure the non-repudiation of the sticky policy. Architecture & model In solving the distributed policy enforcement problem, we assume a trust model where authenticated users are honestbut-curious. Thus, our attack model focuses on the user who inspects the data they receive and attempts to gather data that they are not entitled to, and does not address malicious users who attempt to gain access to data they have not received even if it violates the policies.

Copy forward Yes w/notification

May copy and forward the data with a notification to the sender

Yes w/o notification

May copy and forward the data without any notification

No

May not forward at all

Ask

Must ask the sender on forward

Figure 3 illustrates the conceptual architecture of our system and showcases the creation and management of sticky policies.

6. Append/Modify: The Boolean condition specifying whether the recipient can append/modify the document. Append/Modify Yes w/notification

May append/modify with a notification

Yes w/o notification

May append/modify without any notification

No

May not append/modify at all

Ask

Must ask on append/ modify attempt

Figure 3 - Sticky policy enforcement architecture

The Audit section of the sticky policy consists of information including the source, requestor, a timestamp and digital signature to verify the authenticity of the sticky policy.

Our enforcement model consists of two approaches: Proactive enforcement involves prevention of unauthorized disclosure before it occurs by blocking operations or suppressing results that may lead to a privacy violation.

Figure 2 - Format of sticky privacy policy

Reactive enforcement involves detection of the violation through audits. This is based on an optimistic assumption that the environment is non-malicious.

Data × × ×

Clinical Document 1 .. Clinical Document n

These approaches differ in their points of execution. Proactive enforcement eliminates violations before they occur, which has limited use in scenarios when a priori knowledge of all the possible access situations is not possible. For example, in an emergency situation, violations should be allowed as long as they are auditable and necessary for the delivery of care. Reactive enforcement achieves this by tracking all the access information and assuming the existence of a trusted auditor system. The auditor must be able to access data from the source and recipient including any intermediaries between those parties. For the US healthcare environment, this could be the responsibility of the Department of Health and Human Services or its delegates. In the case of a violation, the system presumes the node to be guilty and demands records to certify innocence.

Policy × × × × × ×

Requestor Recipient Purpose Retention Copy forward Append Notification

× × × ×

From To Timestamp Verifiable Signature

Audit

The source and requestor information is used by the auditor to track the data while the timestamp is used to determine the causal ordering. The digital signature serves two purposes: 1) to maintain the integrity of the healthcare documents, i.e. guaranteeing that the document(s) are not

For distributed environments, proactive and reactive enforcement can be achieved through either a centralized or a federated approach leveraging a set of cooperating enforcers.

485

T. Grandison et al. / Protecting Privacy while Sharing Medical Data Between Regional Healthcare Entities

In our design, we perform proactive enforcement through HDB Active Enforcement technology. In the case of reactive enforcement, a centralized approach employs a single auditor which is trusted and authorized to investigate all aspects of a suspected policy violation. In a decentralized model, a set of cooperating auditors can be employed with each team responsible for a specific set of nodes, data or both.

Reactive enforcement is achieved by traversing the sticky policy audit logs to find a violation or to prove innocence. An audit begins with the suspicion of a privacy violation. We presume that any party with access to the data but without a sticky policy is guilty. In essence, the sticky policy is a certificate of innocence. This is similar to not having a license demonstrating legal ownership of a software product.

Beyond the issues of proactive and reactive enforcement, there is a question of where the enforcement occurs. We consider two possible locations: at the source and at the recipient. Enforcement at the source is simpler because it relies on the sources controls. Enforcement at the recipient places trust in all recipients. However, some enforcement can only occur at the recipient. For example, restricting the recipient from forwarding the result on to others is not something the source can enforce. Our approach is to attempt to perform all enforcement at the source and only rely on enforcement at the recipient when no alternative exists.

The auditor searches the database for the data item for which enforcement is presumed as violated. Once identified, the auditor checks the HDB metadata, and identifies the relevant policy and archive entries. If the ability to identify these entities does not exist (i.e. there is no policy or the archive entry is missing), then a violation has occurred. The auditor then tries to verify the signature on the sticky policy stored in the archive table. Again if the signature is not valid, a violation has occurred. Figure 4 - Sample sticky policy for healthcare RHIO sharing <patient>JamesBeach<suffix>PhD.... <section>History of Present IllnessJames Woods has suffered from calcific bursitis.... Vital SignsThe patient’s height, weight, and body mass index were measured to be 2.29489 meters, 400.05671738536824 pounds, and 34.45 kilograms per meter squared, respectively. ... ...

Scenario Initially, a request for clinical documents is placed through an application interface, which issues a query or a set of queries. On receiving the query, the proactive enforcement module rewrites the query to account for all applicable disclosure policies. In a traditional HDB-enabled system, the rewritten query would continue directly to the data systems native query execution engine. However, for our system, the rewritten query is redirected to the sticky policy module where the query is further modified to create a sticky policy result set. A digital signature is computed on the source using a User-Defined Function (UDF) and included in the sticky policy. The final sticky policy is then transferred to the recipient in its entirety. Figure 4 provides a sample healthcare sticky policy.

Requestor - Alice/ Admin Role/ ArizonaCare Recipient - Bob/ Staff Physician/ ArizonaCare Purpose - Emergency Case Retention - 30 days Copy-Forward - Yes With Notification Append/Modify - No

For this prototype, we chose to use XML as the data format for representing the sticky policy for multiple reasons. The interoperability among various EMR, HER and PHR systems used in the healthcare industry has been limited based on the proprietary interfaces and standards [8]. Although, XML processing as in the case of any text processing involves a lot of overhead, it offers the much needed features of platform independence and simplicity demanded by the healthcare industry. When the XML processing overhead becomes unbearable, specialized hardware and software can be used to mitigate this concern.

From - Trina/ Admin Role/ CalShield To - Alice/ Admin Role/ Arizona Care Timestamp - Nov 19 2006 Signature - ...

If everything has been successful so far, the auditor checks to make sure the sticky policy and data content agree. The auditor compares the policy in the policy table to its counterpart in the sticky policy and similarly compares the data in the sticky policy with that in the database. The auditor also verifies that the HDB metadata tables cover all the data included in the sticky policy. Even if everything checked so far is OK, the audit is not stopped.

For the purposes of this work, we assumed an agreement on vocabulary among interoperating parties. We also assume that proactive enforcement is achieved by leveraging HDB Active Enforcement. On the recipient, the AE component accepts the received sticky policy, assigns a unique id to the policy and stores an unaltered copy of policy for the purposes of auditing. The policy elements are then de-coupled and the corresponding data and policy constraints are extracted. The policy rules are then entered into the HDB metadata tables. The document(s) or their part(s) are then stored in a database and links are created from the entries in the HDB metadata

It is possible for an enforcement breach to have occurred before the current node even received the sticky policy. The auditor then traverses through the sticky policy to identify the node that forwarded the sticky policy. The auditor continues the audit up the chain to the originators

486

T. Grandison et al. / Protecting Privacy while Sharing Medical Data Between Regional Healthcare Entities

until he reaches the bounds of his jurisdiction or the first sender.

Future work Our future efforts will focus on 1) further deployment and enhancement of the current Sticky Policy functionality, and 2) innovating new approaches to enable Sticky Policy Enforcement in distributed systems with a central repository. On the first task, we plan to start by including technology that increases the transmission security strength. i.e. removing the assumption that the channel is inherently secure and encrypting the shared documents before transfer. Then we will remove the assumption of an honest-but-curious user and consider hostile environments; touching on the difficulties in considering Byzantine failures or collusion among several participants. For the second task, we will assume a system of shared policies and construct mechanisms to provide privacy guarantees when data is transferred.

Performance discussion We ran experiments to evaluate the overhead cost of sticky policy generation and sticky policy consumption. In the interest of terseness, we will just provide the results here. Our experimental platform used a synthetically generated dataset based on the Clinical Document Architecture [9]. All experiments were run using IBM DB2 UDB 8.2. The operating system was Microsoft Windows XP with Service Pack 2. The hardware consisted of a PC with Pentium-4 2.4GHz processor and a 60GB disk. The buffer pool was set to 1 MB. All other DB2 default settings were used. Overhead Cost for StickyPolicy Generation 2000 1800 1600

HDB query w ithout sticky policies

Conclusions

HDB query w ith stickypolicy generation

The construction of RHIOs and the sharing of information between them is an important pre-requisite for the successful creation and deployment of a National Healthcare Information Network (NHIN). Very little attention has been placed on technology to enable this RHIO to RHIO collaboration in a privacy preserving manner, till now. In this paper, we present Sticky Policy Enforcement technology, which provides mechanisms to perform proactive and reactive enforcement when sharing clinical documents between RHIOs.

1400 1200 1000 800 600 400 200 0 1000

2000

4000

8000

Nu m b e r o f Do cu m e n ts

Figure 5 - Overhead cost for sticky policy generation

References We examined document sharing for sets of 1000, 2000, 4000 and 8000 documents, which is well over the current limits for healthcare document sharing. It was observed that the overall cost introduced by sticky policy generation over the privacy preserving query processing in HDB is acceptable (Figure 5) considering that the generation is done using XML.

[1] The Goals of Strategic Network. http://www.hhs.gov/ healthit/goals.html. [2] The California Regional Health Information Organization. http://www.calrhio.org/. [3] Stufflebeam, William, Antn, Annie I., He, Qingfeng, and Neha Jain. Specifying Privacy Policies with P3P and EPAL: Lessons Learned. Proceedings of the 2004 ACM workshop on Privacy, Washington DC, USA (2004). [4] Rivest, Ronald L., and Lampson, Butler. SDSI A Simple Distributed Security Infrastructure. Working Document v1.1 [5] TCG Specification Architecture Overview. Revision 1.2 Retrieved 2006 from https://www.trustedcomputinggroup. org/specs/IWG/TCG_1_0_Architecture_Overview.pdf [6] K. Lefevre, R. Agrawal, V. Ercegovac, R. Ramakrishnan, Y. Xu, D. DeWitt. "Limiting Disclosure in Hippocratic Databases". Proc. of the 30th Int'l Conf. on Very Large Databases (VLDB 2004), Toronto, Canada, August 2004 [7] IBM Almaden Research. Hippocratic Database

Overhead Cost for StickyPolicy Consumption 30000

25000 XML Processing, Create and Insert

20000

Updating Metadata tables

15000

Updating archive

10000

Active Enforcement: User's Guide. Retrieved Nov. 28, 2006 from http://www.almaden.ibm.com/cs/projects/

5000

0 1000

2000

4000

iis/hdb/Publications/papers/ HDBEnforcementUserGuide.pdf [8] Srivatsava Ranjit Ganta, Eishay Smith, Sarah Knoop, Sondra Renly, James Kaufman. The Eclipse Open Health Framework, 5th International Conference on Healthcare Technology and Management (HCTM 2006), Chicago, August 2006. [9] Dolin RH, Alschuler L, Boyer S, Beebe C, Behlen FM, Biron PV, Shabo Shvo A, HL7 Clinical Document Architecture, Release 2, Journal of American Medical Informatics Association (JAMIA), 2006

8000

Nu m b e r o f Do cu m e n ts

Figure 6 - Overhead Cost for Sticky Policy Consumption For policy consumption, the time elapsed in updating the metadata tables and the archive is less than 30% of the overall policy consumption cost (Figure 6). As pointed earlier, XML hardware appliances may be used to reduce the overall consumption cost.

487

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Pulling Back the Covers: Technical Lessons of a Real-World Health Information Exchange Atif Zafar, M.D.a,b, Brian E. Dixon, M.P.A.a a

b

Regenstrief Institute, Inc., Indiana School of Medicine, Indiana University, Indiana

Abstract

The Indiana Network for Patient Care (INPC)

Several nations and local communities are striving to achieve widespread, secure exchange of clinical data between various health care providers and public health organizations. Most of the literature on health information exchange focuses on the financial, political, and privacy aspects of these initiatives. Perhaps just as important are the technical and organizational factors that have influenced development of data exchange methods and results. One mature network in the Midwestern United States has had success in establishing consistent, secure exchange of clinical data for more than ten years. Presented here are the technical lessons learned and design decisions made from this initiative with the hope that they can be used by others striving to connect disparate clinical information systems for the improvement of health care quality and safety.

Indianapolis has pioneered an extremely successful HIE initiative, the Indiana Network for Patient Care (INPC), launched in 1993 under the leadership of the Regenstrief Institute. Initially, the INPC provided data from one hospital to providers in emergency departments at three other hospitals. By 2005, a more mature network, with a membership comprising 95 percent of all hospital and emergency care in Indianapolis, expanded to include providers in other parts of the state. By the end of 2006, the INPC contained more than six million distinct patient registration records, 850 million discrete observations, 17 million text reports, 50 million radiology images, and 40 million orders. A number of factors have contributed to the INPCs success, including political and legal dynamics, which have been addressed in other publications [2][3][4]. Here we outline the reasons for the technical design decisions and functionality of the INPC, highlighting the technological and organizational factors that have contributed to the networks growth, ease of use, and sustainability.

Keywords: computerized medical records systems, computer communication networks, equipment reuse

Introduction Everything in health care ultimately revolves around the accessibility and effective use of clinical data. When physicians, nurses, and other health care professionals have the information they need when they need it, they serve patients better, in terms of both the quality and the safety of the care they provide. Making these data available electronically, then, appears to make good sense.

Technological factors Regenstrief has examined, deployed, refined, and evaluated a variety of operating systems, programming languages, software applications, and database management systems over its thirty-year history. The philosophical approach has been to select a technology, stick to it, and make it work. This means we rarely make radical changes (e.g., redesign a program written in one language using another language) unless there is a clear need (e.g., the new language is far superior to the old one). For example, early use of the Web (prior to 1995) for results aggregation brought many challenges. To enable asynchronous communication between clients and servers, we developed customized tools that possessed modern asynchronous JavaScript and XML (AJAX) functionality. Only recently have we begun to redesign our tools to utilize current AJAX frameworks.

Electronic clinical data abound. The problem is that they are often inaccessible to providers, because health care organizations tend to house their clinical data in distinct, isolated repositories. Many providers and policy makers now recognize that the sharing of data among hospitals, doctors, and other health care organizations in a given city, state, or region often referred to as health information exchange (HIE) can make health care safer, more efficient, and more effective [1].

Although sometimes slow to change, our organization is not afraid to experiment. We have worked with state-ofthe-art image-compression technologies, such as JPEG

488

B. E. Dixon et al. / Pulling Back the Covers: Technical Lessons of a Real-World Health Information Exchange

2000 and Wavelets, and motion video (MPEG) for display of radiology images and cardiac echo movies, respectively, in Web browsers. We have also experimented with voicerecognition and voice-annotation for clinical notes and display of laboratory results on mobile devices in wireless settings. Currently, we are experimenting with nomadic computing technologies so that clinicians can have an access-anywhere system for clinical data. The INPC is composed of many moving parts, some of which are legacies while others are more modern. It is not the formula of a certain operating system with a specific database management system that has produced success for the INPC. Rather, the technical success of the INPC may be attributed to its adherence to these philosophies and the principle that, when possible, one should build upon existing infrastructures rather than inventing or implementing new ones. Below we discuss this principle in the context of the INPCs security, speed, flexibility, and reusability. Figure 1 - Simplified data model for the Regenstrief Medical Record System (RMRS)

Security Secure exchange of information between the INPC and participants operates using point-to-point connections. In the past, the INPC has employed T1 lines (data pipes as weve referred to them in other publications). We are now phasing out T1 lines in favor of virtual private networks (VPNs).

Each silo represents mirrored data from one participating institution. Patient registry data, such as name, medical record number, and date of birth, and clinical data, like laboratory results, immunizations, and free-text notes recorded by the doctor during an encounter (e.g., clinical visit), are stored in the silos.

INPC security policy dictates the use of up-to-date encryption methods and good password hygiene and RSA keys. Currently the INPC relies on 128-bit SSL encryption to protect data on the rare occasion we use the public Internet to exchange information. Passwords must be changed at specified intervals, require a certain combination of letters and numbers, have a minimum length, and cannot be reused by the same individual.

Silos can be created technically using a variety of methods. Hierarchical databases using large flat files running on clustered VMS nodes have been employed in the past. More modern relational databases can also be used to develop silos. Each silo can function as a separate database, or indices can partition data stored in a table on separate physical disks.

The INPC also requires users to sign a confidentiality agreement and devices to be equipped with time-limit controls to prohibit unauthorized access. However, users do not log in directly to the INPC. User authentication is done via providers. First, users login to a provider portal or local area network (LAN), then they access INPC applications through a gateway created between the provider and the INPC network. The INPC relies on providers to implement access and time-limit controls on devices and ensure that users have signed a confidentiality agreement, which is required of them anyway to access the providers electronic resources.

The specific technologies employed to create silos are not as important as the concept. Creating federated vaults gives participants peace of mind that their data will be segregated and secure. Yet data in federated vaults can exist within a single network access storage (NAS) unit, reducing latency when retrieving data during a clinical encounter. To date, the centralized, federated model developed by Regenstrief has yielded better performance (speed) when compared to decentralized federated networks used by other HIE initiatives. A centralized, federated model also simplifies the process of data standardization ensuring identical blood urea nitrogen (BUN) results from various laboratories are interpreted the same way. The burden of reconciling various tests is shifted from individual provider organizations to Regenstrief. We can employ a single data model and dictionary and resolve errors as they arrive at the central hub for processing. Troubleshooting and mapping data elements to the standardized model requires overhead approximately 1-3 FTEs for the volume of messages we process. However, the costs and occasional painstaking mapping efforts are justified, because we value the quality of the data stored in the INPC. Our methods ensure that the

Federated data sharing model At its core, the INPC is a series of federated vaults, sometimes referred to as edge proxies or silos, storing data from the various participating institutions. Each institution has its own privileged silo where only data from that institution resides. The architecture of each silo closely resembles that of the Regenstrief Medical Record System (RMRS), a well known electronic medical record system [5]. A simplified data model of the RMRS is presented in Figure 1.

489

B. E. Dixon et al. / Pulling Back the Covers: Technical Lessons of a Real-World Health Information Exchange

data retrieved from the INPC is reliable and standardized, which builds trust amongst network users and makes retrieval and delivery methods efficient.

application, we call ours DOCS4DOCS, can be sent to the State Health Department for communicable disease reporting. Similarly, data received from private practices could be aggregated and presented to an ER physician for delivering emergency care. These compelling applications allow INPC stakeholders to get some value-added by joining the collaborative, with an understanding that their data will only be used according to the agreed upon terms within the data-sharing contract.

Standards Since its inception, the INPC has strived to provide optimal access to relevant clinical data at the point of care. To that end, the INPC has invested significant time and resources into the development and use of health information technology (health IT) standards (e.g., HL7, LOINC, CPT, etc.). These standards permit disparate systems to share data among one another, making them interoperable. They also permit the INPC to quickly add new types of data by reducing the time required to create customized interfaces for information delivery to providers.

Component-based architecture The INPC employs a number of component technologies to process much of the data that travel across the network. Technologies like interface engines, message processors, and a global patient and provider index perform specific tasks that are generic enough to be re-used from application to application. Thus each component can be optimized for its task and easily modified to include a newly interfaced system. This creates a network in which components are not only interoperable but also reusable.

Standards also enable data reusability, the ability to store a single concept and use it multiple times in a variety of applications. For example, a physician may order an HIV 1 AB (LOINC #7917-8) to indicate the presence of HIV in a patient. Once the result is reported to the INPC from the lab, three separate actions can be taken using the same HL7 message and LOINC code. First, the result would be stored in the institutional silo corresponding to the provider identified in the message. Second, the result could be delivered electronically to the physician using a clinical messaging application. Finally, the result could be reported to one or more public health agencies.

The idea of developing and reusing components is not unique to Regenstrief or the INPC. The object-oriented paradigm has influenced software development practices, with conventional modular techniques abandoned in favor of component-based approaches [6]. This is especially true in the open source software movement [7]. Regenstrief has embodied the philosophy of componentbased development into the INPC, which has enabled the network to remain flexible. Expansion over the last thirteen years has involved the addition of new participants (e.g., hospitals, laboratories), new applications that use the data for a variety of tasks, and new forms of data (e.g., we added pathology reports in 2003 and dictated notes in 2005). With each new addition, the network has required slight modification. Development time is shorter, because components can be reconfigured and redeployed faster than monolithic programs. New components can be developed more quickly, because insertion into the network does not require recompilation or reconfiguration of other components.

The example demonstrates that a single element, a standardized clinical message, can be used by three very different components of the INPC to store and exchange clinical information. Standards are employed so that the provider, physician, and health department interpret the result in the same way (e.g., all three receive a message indicating a positive value for LOINC #7917-8). This reuse of the same data is efficient, flexible, and costeffective. At the time of inception for the INPC, standards were immature and limited. We had to invent standardized methods for transmitting and mapping data between networked provider organizations. One such invention, the LOINC standard, was created because SNOMED and other existing terminologies lacked breadth for laboratory and some clinical concepts. We hope that other organizations can benefit from early experiments by us and other organizations. We believe that field tested standards, such as HL7 2.x and LOINC, can help others create interconnected systems in less than half the time it has taken us to develop the INPC.

Organizational factors Designing, constructing, and operation of a working technical infrastructure for interoperable exchange do not guarantee success. In addition to its technical infrastructure successes, the INPC has also benefited from a number of organizational factors that have shaped its development over the last 13 years.

Although we are experimenting with HL7 Version 3, current INPC members continue to transmit data using HL7 2.x. We encourage continued development and refinement of standards, and we will support them as they mature and become adopted by INPC participants.

Incremental evolution Incremental change has played a significant role in the INPCs long-term success. What began as an experiment to connect emergency rooms together slowly evolved into a large network that provides clinical information to emergency rooms, hospital staff in other departments, and ambulatory providers. This growth was guided by steady leadership that focused consistently on the INPC vision rather than on trends in the budding HIE industry.

Applications An important lesson learned from building an aggregated, standardized data repository is that data can then be reused for many applications. For example, the same dataset that is reported out to clinicians using a clinical messaging

490

B. E. Dixon et al. / Pulling Back the Covers: Technical Lessons of a Real-World Health Information Exchange

Some HIE projects can and will evolve more quickly than the INPC. However, leaders of such projects should temper expansion with a clear vision for their network and agreement from all their partners.

and communication initiative. Subsequent funding has come from the NLM, the Agency for Healthcare Research and Quality (AHRQ), the Health Resources and Services Administration (HRSA), the National Cancer Institute (NCI), the Indiana Genomics Initiative, and the Indiana Twenty-First-Century Fund [2]. Each grant supporting a portion of the INPCs development enabled Regenstrief to measure clinical, financial, and community outcomes. These data provided support to the INPCs business case, which allowed the network to secure additional funding for expansion of existing services and development of new ones.

Human resources Technology is not the single most critical factor for successful HIE. To succeed in the development of a broad clinical data exchange, the INPC employed many capable people to manage and support the technology of the network. A knowledgeable staff is necessary on both ends of the network, at each participating organization as well as at the data exchange entity. Technical difficulties and bugs are inevitable, so capable humans are needed to troubleshoot errors, resolve data issues, and continue to move the vision of the exchange forward. And as the INPC has grown, so too has its need for more staff members to effectively monitor all of its members relations and data connections.

In addition, the networks interoperable, flexible design supports a variety of clinical and research activities. Applications such as CareWebTM, DOCS4DOCS, the Shared Pathology Information Network (SPIN) anonymous query tool, and the Public Health Emergency Surveillance System (PHESSTM) for syndromic surveillance build on the INPCs core infrastructure. However, each creates a unique service for all or specific network members. An innovative, legally separate organization, the Indiana Health Information Exchange (IHIE), has also capitalized on the INPC infrastructure, creating a highly reliable, customeroriented organization to support care delivery organizations using Regenstrief technology. By vertically expanding the INPC in this way, senior leadership has successfully created new resource opportunities for the INPC to grow and improve.

For example, we recently had a lab send us an HL7 message using unexpected units (up/mL). The INPC exception processor detected the anomaly (unrecognized units), which resulted in 26,000 records being dumped into an exception queue for analysis. Turns out the lab system had an embedded typo (the units should have been ug/mL), and the problem was resolved after a phone call and a few emails. A more common problem we face is reporting of units in any other field, usually the notes field, except the appropriate HL7 units field (OBX-6). This is a problem common to all the labs from which we receive data, and it is a recurring problem for newly created tests.

Conclusion Reliable, up-to-date clinical data at the point of care remain the key to improving both the quality and safety of health care. Successful exchange of clinical data occurs only when all participating providers and organizations share not only data, but an understanding of what those data mean. Incremental change and growth are key to the success of data exchange networks. Over time, effective networks tend to expand the types of data they carry, as well as the applications for those data. Standards and reusable components help HIEs to maximize their efficiency through shorter development time and lower costs creating opportunities for integration with new systems and organizations. If other exchanges are as successful as the INPC, valuable improvements in care will be achieved in many communities.

Exceptions require human intervention as subtle differences between common clinical concepts are difficult for computers to resolve, despite several attempts in the past [8][9].Given the need for regular human intervention, we employ 2-3 FTEs to constantly monitor and troubleshoot the more than 150 message streams from the major hospital systems in Indianapolis, regional referral laboratories, specialty providers, several rural providers throughout the state of Indiana. We predict the need to add trained personnel in the future as the INPC continues to grow. Our human resources also drive innovation. For years we have benefited from the talents of National Library of Medicine (NLM) informatics fellows typically post-doctoral physicians. These individuals have been key players in designing, creating, testing, and evaluating innovative components of the INPC infrastructure, including add-on programs such as CHICATM and PHESSTM that extend the INPC beyond clinical messaging. Many of these fellows have stayed on as faculty at Regenstrief and the Indiana University School of Medicine, continuing to enhance the INPC and mentoring new fellows.

Acknowledgments This work was produced under Agency for Healthcare Research and Quality contract no. 290-04-0016. The authors thank staff at Regenstrief Institute, Inc. and the Indiana Health Information Exchange (IHIE) for their contributions to this work and tireless pursuit of health information exchange to improve the quality, safety, and efficiency of health care.

Sustainability

References

The ability of senior leadership to repeatedly make a clear, evidence-based business case for the INPC has contributed significantly to its sustainability. Initial funding for the INPC came from the NLMs high-performance computing

[1] Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, and Middleton B. The value of health care information exchange and interoperability. Health Affairs. 2005

491

B. E. Dixon et al. / Pulling Back the Covers: Technical Lessons of a Real-World Health Information Exchange

[2]

[3]

[4]

[5]

[cited 2006 May 1]. Available from: http://content. healthaffairs.org/cgi/content/abstract/hthaff.w5.10 [2] Biondich PG and Grannis SJ. The Indiana network for patient care: An integrated clinical information system informed by over thirty years of experience. J Public Health Mgmt. 2004 Nov; 10 Suppl 1:S81-6. McDonald CM, Overhage JM, Barnes M, Schadow G, Blevins L, Dexter PR, Mamlin B, and the INPC Management Committee. The Indiana network for patient care: A working local health information infrastructure. Health Affairs. 2005; 24(5): 1214-1220. Sears C, Prescott VM, and McDonald CJ. The Indiana network for patient care: A case study of a successful healthcare data sharing agreement. ABA Health eSource [serial on the Internet]. 2005 [cited 2006 Nov 29]; 2(1). Available from: http://www.abanet.org/health/esource/ vol2no1/Sears.pdf. McDonald CJ, Overhage JM, Tierney WM, Dexter PR, Martin DK, Suico JG, Zafar A, Schadow G, Blevins L, Glazener T, Meeks-Johnson J, Lemmon L, Warvel J, Porterfield B, Warvel J, Cassidy P, Lindbergh D, Belsito A, Tucker M, Williams B, Wodniak C. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inform. 1999 Jun; 54(3):225-53.

[6] Waguespack L and Schiano WT. Component-based IS architecture. Information Systems Management. 2004; 21(3): 53-60. [7] Madanmohan TR and Rahul D. Open source reuse in commercial firms. IEEE Software. 2004 Nov-Dec; 21(6): 62-69. [8] Schadow G, McDonald CJ, Suico JG, Fohring U, and Tolxdorff T. Units of measure in clinical information systems. J Am Med Inform Assoc. 1999 Mar-Apr; 6(2): 151-62. [9] Friedman C and Johnson SB. Natural language and text processing in biomedicine. In: Shortliffe EH and Cimino JJ, eds. Biomedical Informatics. 3rd ed. New York: Springer, 2006.

Address for correspondence Brian E. Dixon, M.P.A. Regenstrief Institute, Inc. 410 West 10th Street, Suite 2000 Indianapolis, IN 46202 U.S.A. (317) 423-5582 [email protected]

492

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Geographically Distributed Complementary Content-Based Image Retrieval Systems for Biomedical Image Informatics Sameer K. Antania, Thomas M. Desernoa,b, L. Rodney Longa, George R. Thomaa a

b

National Library of Medicine, National Institutes of Health, Bethesda, MD, USA Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany

features depicted in the image itself. It would be beneficial if the images could be retrieved by their visual content to help improve research, education, or medical practice. Content-Based Image Retrieval (CBIR) has received significant attention in the literature as a promising technique to ease the management of large image collections in a variety of domains [2-4]. Recently there has been an increasing interest in applying it to medical image repositories [5]. Rather than limiting queries to textual keywords, users can also query by example image or image feature (e.g., color, texture, or shape computed from a region of interest) to find similar images of the same modality, anatomical region, and disease along with the matching associated text records.

Abstract There is a significant increase in the use of medical images in clinical medicine, disease research, and education. While the literature lists several successful systems for content-based image retrieval and image management methods, they have been unable to make significant inroads in routine medical informatics. This can be attributed to the following: (i) the challenging nature of medical images, (ii) need for specialized methods specific to each image type and detail, (iii) lack of advances in image indexing methods, and (iv) lack of a uniform data and resource exchange framework between complementary systems. Most systems tend to focus on varying degrees of the first two items, making them very versatile in a small sampling of the variety of medical images but unable to share their strengths. This paper proposes to overcome these shortcomings by defining a data and resource exchange framework using open standards and software to develop geographically distributed toolkits. As proof-ofconcept, we describe the coupling of two complementary geographically separated systems: the IRMA system at Aachen University of Technology in Germany, and the SPIRS system at the U. S. National Library of Medicine in the United States of America.

While the literature lists several successful systems for content-based image retrieval and image management methods [2, 5], they have not made significant inroads in routine medical informatics. In addition, although many large imaging databases exist, such as the National Cancer Imaging Archive (NCIA) or the Lung Imaging Database Consortium (LIDC) created under the aegis of the Cancer Imaging Program1 at the U.S. National Cancer Institute (NCI), these efforts have concentrated on data collection and transmission but have left development of applications to the research community. Lack of CBIR adoption has been attributed partly to the difficulty of integrating current implementations with existing healthcare systems [6]. The following reasons may further explain this anomaly: (i) challenging nature of medical images, (ii) need for specialized methods specific to each image type and detail, (iii) lack of effective image indexing methods, and (iv) lack of a uniform data and resource exchange framework between complementary systems. Most systems tend to focus on varying degrees of first two items, making them very versatile in a small sampling of the variety of medical images, but unable to share their resources or strengths. This requires each project to redevelop what may exist as an advanced implementation, but inaccessible. The lack of suitable image indexing methods is a problem for large image collections. Image comparisons performed linearly are inefficient and too slow for practical use. This paper proposes to overcome these shortcomings by defining a data and resource exchange framework using open stan-

Keywords: medical informatics applications, image information storage and retrieval, Internet services

Introduction There has been an explosive growth in the acquisition of medical images for clinical diagnosis, and use in medical research and education [1]. Hospitals have been adopting technology such as Picture Archiving and Communication Systems (PACS) and Hospital Information Systems (HIS) to assist in the digital collection, organization, and storage of patient data. The goal of these systems is to make patient data more accessible; in reality, the amount of data that is entered and stored in these systems have created new challenges in effective information indexing and retrieval. Retrieval of image information from these systems is done using limited text keywords in special fields (e.g., unique patient identifier, fields in the image header). These keywords, however, do not capture the richness of

1

493

http://imaging.cancer.gov (Last accessed: March 27, 2007)

S. K. Antani et al. / Geographically Distributed Complementary Content-Based Image Retrieval Systems

dards and software to enable such specialized systems to act as geographically distributed toolkits. The approach enables communication between two or more geographically separated complementary systems with possibly different architectures and developed on different platforms, and specialized for different image modalities and characteristics. The resulting system provides the user with a rich functionality operating within a familiar interface on the Web-browser, making it portable and independent of location and underlying user operating systems. Figure 1 illustrates this concept.

based retrieval is unique and valuable for research into the retrieval of images in large repositories that are similar in type as well as pathology.

Background IRMA project The IRMA project2 aims to develop and implement highlevel methods for CBIR with prototypical application to medico-diagnostic tasks on radiological image archives. Stated goals include support for semantic and formalized queries to the medical image database with support for inter- and intra-individual variance and diseases. Example tasks are the staging of a patient's therapy or the retrieval of images with similar diagnostic findings in large electronic archives. Formal content-based queries also take into account the technical conditions of the examination and the image acquisition modalities. The system classifies radiological images in a general way without restriction to a certain diagnostic problem or question. Pattern recognition and structural analysis methods describe the image content in a feature-based, formal and generalized way. The mean image description enables a fast and reliable image comparison and retrieval. The project also includes an automatic classification and indexing process for insertion of new data into the system without manual interaction.

In this figure, each circle represents a CBIR system specializing in particular image types, pathologies, or CBIR techniques. Using a standard open protocol any system could act as a client using the services available elsewhere to provide the user with a rich medical image informatics resource.

CBIR: Intensity Data: Various CBIR: Shape, Color Data: Spine, Cervix Images CBIR: Texture Data: Lung Image CBIR: Texture Data: Brain Image

CBIR: Color, Texture Data: Histopathology

The IRMA project has several interfaces and can be characterized by its features: (i) Automated classification of radiographs based on global features with respect to imaging modality, direction, body region examined and biological system under investigation; (ii) Identification of image features that are relevant for medical diagnosis; these features are derived from a priori classified and registered images; and (iii) Image retrieval on similarity to an a priori selected feature set based on the visual similarity of certain image structures. Current image data consists of radiographs, with future plans to include medical images from other modalities. An IRMA retrieval interface supporting query refinement is shown in Figure 2.

Figure 1 - Concept: Geographically distributed framework of complementary CBIR systems As proof-of-concept, we describe the coupling of two leading, complementary geographically separated image informatics CBIR systems: The Image Retrieval in Medical Applications (IRMA) project at the Aachen University of Technology (RWTH) [7, 8] and the Spine Pathology & Image Retrieval System (SPIRS) project at the U.S. National Library of Medicine (NLM) [9-11]. IRMA and SPIRS retrieve images based on different approaches: Traditionally, IRMA has focused on image retrieval by computing overall (or global) image characteristics such as color, intensity, and texture. In particular, the image distortion model was developed and proven as a robust distance measure for differences on smaller sub-regions in the image [12]. Such an approach permits queries on a varied image collection and helps identify similar images, e.g., all chest X-rays in the A-P view. The IRMA system lacks the ability to find particular pathology that may be localized in specific regions within the image. In contrast, SPIRS can retrieve images that exhibit pathology that may be localized to a particular region under the assumption that the query to a large image collection containing images of only one type, e.g., vertebral pathology expressed in spine x-ray images in the sagittal plane. SPIRS lacks the ability to select pertinent images from a large varied image collection typical in a hospital PACS system, for example. We believe that combining the strengths of these two complementary technologies of whole image and local feature-

Figure 2 - IRMA retrieval interface with query refinement. 2

494

http://irma-project.org (Last accessed: March 27, 2007)

S. K. Antani et al. / Geographically Distributed Complementary Content-Based Image Retrieval Systems

detectable. Each of these key features may be identified by examining the boundary shape of the vertebra. SPIRS may be used to determine what features (e.g., protrusion on the anterior edge of the cervical vertebra) are consistently associated with a certain symptom (e.g., neck discomfort) or whether a certain feature is a precursor to more serious illnesses (e.g., arthritis). Pathology or medical condition may also be documented in the text of the patient record as a survey response or in the medical diagnoses. The SPIRS Web-interface is shown in Figure 3. In addition to the Web-interface, SPIRS also offers a service to its core shape similarity algorithms and data. Although SPIRS focuses on shape-based queries its framework is extensible to adopt features particular to other biomedical image and data collections, e.g., its core architecture is being extended to include color, texture, and spatial location in uterine cervix images from the National Cancer Institute at the NIH [11]

SPIRS project 3

SPIRS provides a Web-based interface for performing image retrieval from a database of digitized spine x-rays using the morphological shape of the vertebral body. Its framework enables interaction with and retrieval of relevant information from large databases of image and patient data using rich hybrid image and text query methods. A query editor enables users to pose queries by sketching a unique shape, or selecting or modifying an existing shape from the database. It aims to capture query semantics through support of advanced mechanisms like multiple partial shape matching. Additional text fields enable users to supplement visual queries with other relevant data (e.g., anthropometric data, quantitative imaging parameters, patient demographics). These hybrid text-image queries may be annotated with pertinent pathologies by selecting and weighting local features to indicate importance. Query results appear in a customizable window that displays the top matching results and related patient data. SPIRS provides a working proof-of-concept that is capable of accommodating large amounts of imaging data expected in the near future.

SPIRS-IRMA multilevel distributed CBIR Medical CBIR systems can be classified by the type of numerical features used to characterize the images: (i) global approaches extract a single feature vector from the entire image, (ii) local approaches assign the feature vector to distinct regions of interest, and (iii) structural approaches additionally cope with spatial and temporal relations between the image objects of interest. In this context the IRMA system is a global feature system with particular structural aspects of some image regions that are also computed. In contrast, SPIRS takes a more local approach to image similarity with some structural knowledge available from the vertebral image labels. In this sense, SPIRS-IRMA jointly may be considered a multilevel distributed CBIR system [11, 13]. In this joint system, IRMA serves as the front-end for the end user, while SPIRS provides specific shape similarity algorithms and supplies formatted responses to structured queries. In this proof-of-concept system, extensive data interchange was minimized through data mirroring. It is conceivable, however, that the data could also be securely shared in a purely service oriented setup. Screenshots from the combined SPIRS-IRMA system are shown in Figure 4.

Figure 3 - SPIRS interface with (a) crop and (b) detail view At NLM, the focus of CBIR research has been to develop systems capable of performing a range of queries on large medical multimedia databases comprising various biomedical images and patient health data. Such a database in current use contains digitized spine x-rays and associated person metadata that comes from a large nationwide survey, the Second National Health and Nutrition Examination Survey (NHANES II). NHANES is conducted regularly by the National Center for Health Statistics in the United States. The goals of NHANES include estimating prevalence of selected diseases, monitoring disease trends, and studying the relationship between nutrition and health. The NHANES II collection is considered very valuable to radiologists, bone morphometrists, researchers in osteo-arthritis, and medical educators. Domain experts reviewed a sample of the data and identified 23 key biomedical features that were exhibited in the x-rays. Of these, anterior osteophytes, subluxation/spondylolisthesis, and disc space narrowing were determined to be frequently occurring and reliably 3

Figure 4 - SPIRS-IRMA interface

http://archive.nlm.nih.gov/spirs (Last accessed: March 27, 2007)

495

S. K. Antani et al. / Geographically Distributed Complementary Content-Based Image Retrieval Systems

The system has plans for development in several phases. SPIRS-IRMA is utilizing only limited services provided by SPIRS with near term goals to use all available features. Future phases will also include permit users to upload their own images, as in the current IRMA system, and segmented or sketched boundary outlines of interest. Next steps include multi-resolution shape queries and support for query logging and relevance feedback.

interfaces and implement them such that the local interfaces also use the gateway for all system communication. SPIRS indexing and retrieval algorithms use a variety of shape similarity metrics embedded within feature indexing techniques for efficient retrieval essential for Web use [11]. All text data is stored in a MySQL database and is accessible via both the gateway and the CBIR algorithms. In this initial prototype both systems hold all images and shape contours to minimize data exchange. Future plans include allowance for secure image and feature data exchange. Communication protocol and data exchange format A distributed computational framework relies extensively on a robust communications protocol and data exchange format. In case of SPIRS-IRMA, the systems are looselycoupled over the Internet making it possible to use open communication standards. XML documents styled to a developed DTD are used for data exchange. The entire transaction is divided into three primary events, viz., querystatus, query, and queryresult. Each element in the XML file is designed for a particular event. For example, the element is used to determine if a desired service is available and to obtain a list of currently available services.

Figure 5 - The distributed architecture of SPIRS.

Data and resource exchange framework CBIR systems sharing their data and computational resources need to be developed with a distributed architecture as shown in Figure 5. Each CBIR system would require the following components: (i) a gateway that acts as a mediator between client and server-side components, (ii) the indexing and retrieval server, which performs the feature representation and similarity matching, (iii) the databases containing images and associated text data, and (iv) an open communication standard for sharing of data and commands between the systems. Key components are described using the protocol developed for the SPIRSIRMA system.

The element is used to make shape queries where each query contains: (i) the query vertebra contour (boundary data points), (ii) partial shape indices and weights (if any) (iii) requested matching method (iv) maximum number of responses requested, and (v) range of similarity result scores. SPIRS responds with the element populated with: (i) matching image - vertebra tuples, and (ii) similarity scores. The design of the SPIRS-IRMA collaboration, the DTD, and sample XML data, are discussed in [13].

Discussion

Gateway

The field of medical informatics has been unable to take advantage of image retrieval methods and systems in spite of their acknowledged importance in the face of growing use of image data in medical practice, research, and education. Image management and pathologically sensitive image content-based retrieval systems are thus increasingly necessary. The challenging nature of images and lack of comprehensive systems have hindered their acceptance. While it is difficult to develop a single comprehensive system, it may be possible to take advantage of the growing research interest and several successful systems with techniques developed for specific image collections. In addition to supporting rich segmentation, validation, indexing, query, retrieval, and visualization methods, inclusion of open interfaces using standard communication protocol, such as that described above, to the projects can enrich individual systems. The proposed approach has the following advantages:

The gateway is the entry point for all service requests. It can be implemented in any standard Web interface. In case of the SPIRS-IRMA collaboration, SPIRS has the service gateway implemented as a Java servlet. The gateway acts as a mediator between client requests and server-side components. It manages multiple simultaneous connections (users) as separate sessions and queues requests to the core CBIR engine. It translates query components that require information from the MySQL text database into SQL queries. The gateway is also responsible for formatting responses from server-side components and sending them to the client. Server-side components The core CBIR algorithms that operate on image feature data and text databases comprise the server-side components. While these algorithms rarely need any modification, their implementation is of particular importance. Typically, monolithic CBIR systems have indexing and similarity retrieval components closely wedded to the query and result visualization user interfaces. It is necessary to decouple these core algorithms from their user

•

496

Simplicity: The simple communication interface allows for rapid development of methods for individual systems by expanding the available resources reachable through open communication standards.

S. K. Antani et al. / Geographically Distributed Complementary Content-Based Image Retrieval Systems

•

Extensibility and Flexibility: By separating the user interface from the core image informatics algorithms, the proposed approach allows systems offering services to continue development of other techniques and add them as they mature. It is no longer necessary to rebuild entire applications. In unusual cases, the protocol allows removal of some services advertising their unavailability. • Security: Separating the core algorithms enables selective additional data security, user authentication, and encryption of the communication component (the gateway), where appropriate. The flexibility also allows sharing of just the methods. We have demonstrated some of these features through the SPIRS-IRMA collaboration. In the IRMA system, it is possible for a user to find images from their database similar to an uploaded image. The query is limited to this extent, however. It requires manual viewing of each resulting image to identify those with pathology similar to that in the query example. While the SPIRS system allows shape queries with multiple parts highlighted, which indicates their relative importance, and includes text fields for further refined responses, it is limited to the spine x-ray database. When completed, users familiar with the IRMA interface can extend their searches beyond finding similar x-rays to include localized searches. Goals include enabling image segmentation services for user-supplied images. The resulting system is a multi-level, pathologically sensitive, geographically distributed system.

Medicine (NLM), and Lister Hill National Center for Biomedical Communications (LHNCBC).

References

In this article, we have proposed a resource sharing strategy for increasing the impact of traditionally limited medical CBIR systems on medical informatics, and possible applications to clinical medicine, medical research, and education. We propose the use of open standards and a distributed framework that is simple to implement, extensible, flexible, and secure for the development of CBIR systems. We demonstrate its impact through a prototype SPIRS-IRMA combined retrieval system.

[1] Andriole KP. Addressing the Coming Radiology Crisis: The Society for Computer Applications in Radiology Transforming the Radiological Interpretation Process (TRIP) Inititative. White Paper (http://www.siimweb.org). November 2005. Last Accessed: March 26, 2007. [2] Smeulders A, Worring M, Santini S, Gupta A, Jain R. Content-based image retrieval at the end of the early years. IEEE Trans Pattern Rec. & Machine Intel. 2000; 22(12):1349-80. [3] Antani S, Kasturi R, Jain R. A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video. Pattern Rec. 2002;35(4):945-65. [4] Catarci T, Costabile MF, Levialdi S, Batini C. Visual query systems for databases: a survey. J. Visual Lang. & Comp. 1997;8(2):215-60. [5] Muller H, Michoux N, Bandon D, Geissbuhler A. A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. Int. J. Med. Info. 2004; 73(1):1-23. [6] Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS. ASSERT: A physician-in-the-loop contentbased retrieval system for HCRT image databases. Computer Vision & Image Understanding 1999;75(12): 111-32. [7] Lehmann TM, Güld MO, Thies C et al. Content-based image retrieval in medical applications. Methods of Information in Medicine 2004; 43(4): 354-61. [8] Thies C, Gueld MO, Fischer B, Lehmann TM. Contentbased queries on the CasImage database within the IRMA framework. Lec. Notes Computer Sci 2005;3491:781-92. [9] Antani S, Long LR, Thoma G. Content-based image retrieval for large biomedical image archives. Proc. 11th World Cong. on Med Info (MEDINFO) 2004, pp. 829-33. [10] Long LR, Antani SK, Thoma GR. Image informatics at a national research center. Computerized Med Imaging & Graphics. 2005;29:171-93. [11] Thoma GR, Long LR, Antani S. Biomedical imaging research and development: knowledge from images in the medical enterprise. Lister Hill National Center for Biomedical Communications, NLM, NIH; Sep. 2006. Report No.: LHNCBC-TR-2006-002. [12] Keysers D, Dahmen J, Ney H, Wein BB, Lehmann TM. A statistical framework for model-based image retrieval in medical applications. J Electronic Imaging 2003; 12(1): 59-68. [13] Antani S, Lehmann TM, Long LR, Güld MO, Neve L, Thoma G. Interfacing Global and Local CBIR Systems for Medical Image Retrieval. Accepted by Workshop on Image Research for Medicine (Bildverarbeitung für die Medizin). March 2007, Munich, Germany.

Acknowledgements

Address for correspondence

The authors would like to thank Leif Neve at NLM, William Hsu at University of California at Los Angeles, and Mark Oliver Güld at Aachen University of Technology for their assistance.

Sameer Antani, PhD National Library of Medicine, NIH 8600 Rockville Pike, MS 3824 Bethesda, MD 20894, USA Email: [email protected]

Conclusions and future work In addition to formalizing the implementation of all aspects of the SPIRS-IRMA system, we plan to develop and publish a formal specification to enable sharing of image informatics resources among various systems. As a test, we will expand this framework to the CBIR system under development for uterine cervix images that uses color, texture, and spatial location information in generating an image description.

This research was supported by the Intramural Research Program of the National Institutes of Health (NIH), National Library of

497

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Utilizing SELinux to Mandate Ultra-secure Access Control of Medical Records Peter R Croll, Matt Henricksen, Bill Caelli and Vicky Liu Information Security Institute, Queensland University of Technology, Brisbane, Australia

applications operate on commodity level computer operating systems that do not - and cannot - provide the required level of assurance needed for scientific research to be undertaken on sensitive data. They were never designed for this purpose.

Abstract Ongoing concerns have been raised over the effectiveness of information technology products and systems in maintaining privacy protection for sensitive data. The aim is to ensure that sensitive health information can be adequately protected yet still be accessible only to those that “needto-know”. To achieve this and ensure sustainability over the longer term, it is advocated that an alternative, stable and secure system architecture is required. This paper considers the adoption of a model targeted at health information that provides much higher degrees of protection. A purpose built demonstrator that was developed based on enterprise-level systems software products is detailed. The long term aim is to provide a viable solution by utilizing contemporary, commercially supported operating system and allied software. The advantages and limitations in its application with a medical database are discussed. The future needs in terms of research, software development and changes in organizational policy for healthcare providers, are outlined.

Information security mechanisms do exist to ensure sensitive information is protected and only accessible on a “need-to-know” and approval basis. It is imperative that both adversaries, such as external system “hackers” and technical/operations personnel with in-house knowledge be denied inappropriate access. The security mechanism known as Mandatory Access Control (MAC) is described below and can be used, as adapted, to enforce the necessary security and privacy processes required for handling sensitive health data. Such mechanisms have been studied and understood for over 30 years, mainly in defence related systems. However, they have not been evident in contemporary commodity level operating system or allied application level environments. In particular, the ICT industry’s move towards development of application systems based on so-called “Service Oriented Architectures (SOA)” and “Web Services” software environments presents new security challenges in the healthcare environment. Security standards, even for these high level service structures are often based on “interpreter” sub-systems such as XML, etc., and Internet/ World-Wide-Web “browser” packages; they are complex and are only in the early stages of development and deployment. Moreover, all of these structures critically depend upon the overall security and dependability of the underlying “middleware”, operating system and computer hardware systems. An application cannot be any more secure than the underlying systems upon which it depends. The same holds true for contemporary computer grid technologies, e.g., the Globus [1]. In other words, trying to adequately secure the shared “virtual machine” environments that grid technologies currently exploit is next to impossible (i.e., far too challenging for the foreseeable future).

Keywords: information security, health information systems, operating systems, access control

Introduction Advances in storage and communication technologies have made large repositories of data available even when they are maintained on separate systems and geographically distributed. The access to such data sets is often subject to varying degrees of legal, social and ethical constraints. Further, the data sets may not be available for open scientific research due to the sensitive and private nature of the information they contain. For example, individual personal health records offer significant value for medical and related research but such information cannot be readily accessed in a manner suitable for such research. The reasons include the need to abide by national privacy legislation, the reluctance to change to electronic record format due to security fears and the requirement to maintain end-user trust in the overall healthcare information system. Even when access rights have been granted - for example the data has been sanitized by having all personal identification removed - there are still legitimate concerns over the degree of privacy that contemporary IT applications will provide. The present reality is that IT

The primary aim of this project is to build a “Concept Technology Demonstrator”, based upon advanced cryptographic and information research and technologies (e.g., Cryptocards1 and Starlight InfoSec technology2) in order 1 2

498

“Cryptographic plug-in cards”, Eracom (Safenet) Australia. “Starlight InfoSec Technology”, Defence Science and Technology Organisation (DSTO), Australia.

P. R Croll et al. / Utilizing SELinux to Mandate Ultra-secure Access Control of Medical Records

to provide ultra-secure and sanitized access to the protected data sets.

configurations created by some of the recently arisen SELinux support groups.

The non-sustainability of current approaches

SELinux shares two fundamental properties with many MAC systems. Firstly, the super-user concept of DAC systems (i.e., “root” or “Administrator”) is banished, so that all users of the system are controlled by the same configuration policy. The policy is written by a security administrator. If an attacker can acquire the privileges of the security administrator, then the policy can be changed to suit his or her ends. But unlike the super-user, who controls any aspect of the system, the security administrator exists only to secure the machine, and not to make use of it.

There is a strong vested business interest by mainstream suppliers of computer operating systems and similar middleware products to perpetuate the belief that computer applications can be “made secure from within”, irrespective of other software or even hardware components. In other words, the correct use of their technology will ensure a sufficiently secure operating environment upon which application programs can run safely. Unfortunately, any such assumption is flawed since in reality they represent a “fortress built upon sand” [2].

Secondly, like some other MAC systems, SELinux is based upon the concept of “type enforcement”, in which all the objects in an operating system (be they files, network sockets or processes) are labelled and classified as “domains” or “types”. The system configuration determines how domains with label x are able to access types with label y. Typically the access rights will be described, in a broad sense, as “domain x can read type y”, “domain x can write to type y”, “domain x can execute type y”, combinations of these, or “domain x cannot interact with type y in any way”. This last option is the implicit default, so only positive relationships between domains and types need to be configured.

Current base operating systems in the commercial arena are based on what is designated as “Discretionary Access Control” or DAC. Essentially this design allows the “owner” of any object in a system to, at their discretion (from where the term DAC comes) pass on their access rights to any other person or entity in the overall information system’s environment. In particular, DAC really does not acknowledge the essential difference between a computer “user”, a person, and the individual “processes” in the system that act on his/her behalf. Moreover, the DAC structure assumes that the user is completely familiar with and trusts any program or related software system which he chooses to execute in the computer. In the current environment these assumptions, while possibly valid in the era of large mainframe computer systems with large “inhouse” software development and support organisations prior to the development of global “packaged” software industry, are simply no longer true. The DAC approach, moreover, assumes that users formulate their own “discretionary” policy, a situation that is no longer valid as overall information systems become subject to overriding legal, societal and enterprise policy requirements.

Nevertheless, the flexibility provided by SELinux tends to be its undoing, at least from the perspective of the casual user. Because there are many domains and types within even a single-user system, and because each possible positive interaction needs to be considered, configuration of access rights is a laborious and error-prone process. Add to this the fact that each system is guaranteed to be different to the one on which the prepackaged configuration was prepared, and a nightmare scenario in which SELinux denies essential accesses - such as allowing the system's Graphical User Interface to start - becomes a common one [5].

The applications will not be secure unless the underlying operating system and hardware have been specifically developed with security in mind. A system that is built to utilize a MAC mechanism will provide levels of security relating to all aspects of the computing system, i.e., “enforce an administrative set policy over all subjects and objects in a system, basing decisions on labels containing a variety of security-relevant information” [3].

There are well-known strategies that can help to reduce configuration complexities. One of the most popular of these is Role-Based Access Control (RBAC), which intersperses a “role r” into the relationship between domain and type, such that, for example, if “role r can read type y” and “if domain x is a member of role r”, then “x can read y”. Since there are a small number of roles relative to the number of domains and types, then the number of rules relating roles to types and domains to roles should be much fewer than the number of rules that relate domains directly to types. SELinux supports a primitive version of RBAC, yet a typical SELinux configuration file still runs to about 50,000 lines.

Practical implementations of MAC There are many implementations of MAC based systems, but one of the most popular is called SELinux (SecurityEnhanced Linux) [4]. It was designed and engineered by the National Security Agency, an intelligence-gathering organization belonging to the government of the United States. Released in 2000 as a patch to the Linux operating system, SELinux quickly gathered popularity within the Linux community due to its structural simplicity and the impeccable credentials of its designers. It now exists as an open-source module transparently integrated into the Linux OS kernel. Optimistically, the casual user may receive security support from SELinux without even noticing that the module is active, due to generic security

The RedHat company sells RedHat Linux Enterprise and sponsors the “Fedora Core” open-source software activity, both of which sport an extension of SELinux that includes “strict mode” and “targeted mode” structures. Strict mode is no different to the “vanilla” version of SELinux, but targeted mode protects only a subset of domains and types, usually those which have interaction with the external world via network sockets, etc. (that is, those objects which are most likely to be attacked by hackers). The

499

P. R Croll et al. / Utilizing SELinux to Mandate Ultra-secure Access Control of Medical Records

remaining objects within the system are labelled “unconfined” and can “run amok” with only the discretionary access controls regulating their behaviour. RedHat ships SELinux in the default mode of “targeted”, so that basic protection is afforded to the system without the mechanism becoming invasive, in turn preventing the user from being productive, and swamping RedHat support with basic administrative support requests. The flipside to this is that RedHat does not offer support to issues arising from the strict mode of SELinux. As will be seen later, this has a substantial impact on the use of SELinux to protect medical or other application data.

level relationships tend to be very static, for example, changing only when new software is installed, this is not especially disadvantageous for the normal use cases for SELinux but is not well suited for creating rapidly changing sandboxes. Our solution to this problem, which also avoids the problem of creating additional complex interactions between application and operating system level objects, is to create a proxy. The proxy runs at the application level and is secured in its own sandbox by SELinux, preventing unwanted interactions with other processes. The proxy regulates access by application-level process to protected data, using its own set of configuration files. In one sense, this solution can be viewed as nested SELinux, whereby the proxy represents a micro-instance of SELinux that deals only with application data. Operating system level processes see only a monolithic object (the proxy) representing application processes, meaning that the number of configuration rules between the two layers is linear rather than multiplicative.

Protecting medical application data The primary intention of SELinux is to protect objects embedded within the operating system, with security of application data being an afterthought. SELinux in effect partitions the operating system space into a set of “sandboxes”, protected areas between which communication is tightly regulated. The mechanism is generic, and consequently, the security administrator can create an additional series of sandboxes at the application level to protect medical and other kinds of data. For example, the administrator may configure a web-browsing sandbox that permits a web browser such as Internet Explorer or Mozilla Firefox to access the internet. In addition, the administrator may also configure a medical-related sandbox in which a medical application is permitted to access medical records. However, unless explicitly permitted, the web browser does not have access to the medical records. Neither does the medical application have the same level of exposure outside the network as the web browser. The security administrator can create arbitrary levels of complexity in the application layer by constructing sandboxes for different applications, yet the enforcement mechanism of SELinux treats them all equally and prevents unauthorized accesses. Whereas if a hacker attacked a DAC system through the network interface, and managed to acquire super-user permissions, in an SELinux scenario, the hacker would control only a single sandbox, and would need to launch additional exploits, each of which became increasingly infeasible with distance from the network interface.

The proxy deals with the added levels of interaction complexity at the application layer by using an enhanced version of RBAC, in which role permissions are inherited throughout a hierarchy. By collating roles into hierarchy, and associating the lowest member of each hierarchy with each type, this obviates the need to associate every role with every type. As an example, a vertical slice of a role hierarchy may consist of “Doctor is a subset of role Clinician” and “Surgeon is a subset of role Doctor”. Configuring the policy with “any user in the role of Clinician has access to type y” automatically covers the rules for “any user in the role of Doctor has access to type y” and “any user in the role of Surgeon has access to type y” by virtue of their membership of the family. Portions of the hierarchy can be overridden: configuring “any user in the role of Surgeon does not have access to type y” does not cause a contradiction but allows only Clinicians and Doctors access to type y. An option in this research was to build the extended RBAC functionality natively into SELinux for which the source code is freely available. However, the benefits to operating-system level objects, which are not ordered hierarchically, are unlikely to outweigh the disadvantage in branching the SELinux source code, consequently reducing the successful uptake of this solution.

An important caveat is that the “targeted” mode of Red Hat Enterprise Linux and Fedora Core does not permit application-level sandboxes, because all application process run in the unconfined domain. Any system supporting application level security is compelled to run in strict mode, which in turn means that it is likely not to be fully supported by its commercial vendor.

The mechanism by which the proxy works is very simple, and abstractly mirrors the SELinux mechanism. A client interacts with the proxy via a pair of Client and Server messages. For each client message received, the proxy sends exactly one server message.

Building an SELinux proxy Application data tends to be much more dynamic and flexible than operating-system level data. There may be many users of an application level database, whereas the number of owners of operating system processes tends to be very small. By default, SELinux is configured for four users, including system, staff, sys-admin and ordinary users. Adding new users involves recompiling and reloading the configuration policy, as does adding new rules for interactions between domains and types. As operating-system

The client authenticates itself to the proxy using a client message with type CREDENTIALS and with a payload containing the user, role and password that describe the client. Until the next such message is received, the proxy caches the credentials. This mimics the SELinux mechanism, which authenticates a user via a password before transitioning the user into the requested role. The proxy

500

P. R Croll et al. / Utilizing SELinux to Mandate Ultra-secure Access Control of Medical Records

generally responds to credential messages by sending a dummy OK response.

Our research did not consider key management issues between the client and the proxy, although the usual public key establishment protocols, such as Diffie-Hellman can be used.

The credentials are evaluated whenever the client requests access, either a read or a write, to a record in the proxy database. The proxy passes the credentials, along with the record identifier and the policy to the security filter. The security filter assesses the credentials, decides whether the record can be accessed in the way intended and passes this decision to the proxy. In the case of a read request, the proxy relays the appropriate record back to the client. If the client has requested a write, then the material passed in the payload of the REQUEST_WRITE_FILE message is appended or overwritten to the record.

To prove the effectiveness of the proxy, we developed a simple prototype of the proxy and a client, as shown in Figure 2. Auditing data for the client is shown in Figure 3. We used the proxy and client to demonstrate the security advantages of SELinux over DAC-based systems such as Windows XP. In DAC-based systems, it was relatively easy to use hacking tools such as rainbow tables [6] to break weak Windows system administrator passwords, and modify the proxy and client code to allow unauthorized and unaudited accesses. As the proxy was housed in its own sandbox under SELinux, traditional hacking tools did not provide an avenue for breaking into or changing the proxy. The issue remains that this security is present only in the unsupported “strict” mode of SELinux which is still too complex to deploy in commercial situations. Although the proxy significantly simplifies configuration of application data, it does not address problems at the operating-system level that need to be resolved. Further research in this area needs to focus on simplifying generic SELinux configuration, to allow realistic deployment of “strict” SELinux, which supports protection of application data. This is indeed happening, as witnessed by the development of modular policy logic in Fedora Core 5, which allows the configuration to be developed and loaded in blocks relating to the processes or daemons being protected. The efficacy of this strategy has yet to be solidly determined.

Figure 1 - Architecture of the SELinux proxy Whereas SELinux can protect data to the granularity of the file, the proxy has arbitrary granularity, as determined by tags exchanged between the proxy and its client. The client may wish to retrieve a single word from a database, or an entire collection of files. Our mechanism allows this with as little as a single configuration, although for more complex cases, the number of configuration rules will increase linearly in the number of database items. There are some cases when records must be accessible even in the absence of legitimate credentials. For example, if the authorized viewer of a patient’s case file is not present, but the patient requires emergency treatment, then the availability of the information is more important than its privacy. So the proxy is programmed to respond to a special role of “Emergency”, in which case it moves into auditing mode, until a new set of credentials with a differing role is provided. In auditing mode, all records can be retrieved and modified, but each action is recorded and flagged for review by the security administrator. Appropriate punishment for abusing this mode can be metered out at a social level. Our prototype does not handle differential records, whereby the deltas between subsequent versions of records are stored, although this would be advantageous for malicious or accidental modification of records in auditing mode.

Figure 2 – The Proxy Client

It is not essential for the proxy and the client to maintain an encrypted channel, since access control on the channel can be maintained by SELinux. For ease of configuration, all communication can be encrypted using commonly available algorithms such as the Advanced Encryption Standard.

Figure 3 – Auditing data for the proxy client

501

P. R Croll et al. / Utilizing SELinux to Mandate Ultra-secure Access Control of Medical Records

level are modifications of the basic concepts involved necessary. Likewise, the concept of “compartmentalization”, reflected in the SELinux type enforcement system, needs to be assessed in relation to its suitability for all levels of information services needed in a nationwide health information structure. At the same time, application software development needs to become aware of the new parameters afforded by the MAC facilities and determine to what level such applications may or may not make use of the security mechanisms and services offered, i.e., to determine the distinction between what may be labelled as “security aware” versus “security ignorant” applications. Moreover, the integration of existing software systems into this environment must be understood, requiring further research into appropriate techniques for system integration in higher security environments. In turn, this places new demands on education and training as ICT professionals need to develop the skills needed to understand, utilise and manage this new environment. This indicates that necessary or desirable changes in organizational policy and management structures for healthcare providers may be also needed. At present, full guidance to policy makers and operational management in relation to deployment of newer MAC based overall information systems do not appear to exist. This leads to the need for further research and experimental system development in the area to enable study of the economic, cultural, social and legal responses required.

Conclusions Sufficient evidence is emerging that the security requirements and obligations for the protection of sensitive health data cannot be sustained using contemporary data access control and protection mechanisms in current commercial, commodity computer systems. “Mandatory Access Control” or MAC, incorporated into basic operating systems and allied supporting software structures, provides an alternative, strict, security policy driven approach far superior to industry standard DAC mechanisms. MAC can strengthen protection from unauthorised access to sensitive health related information from both outside and inside an organization. This provides enhanced privacy protection from staff, including knowledgeable ICT professional staff members, gaining access to such sensitive data for which they are not authorised (i.e., view, modify, copy, transmit, delete, etc.). It further provides enhanced ‘boundary’ security from outside intrusion whereby adversaries, such as hackers and spyware operatives, are unable to gain full control of an information system. In the MAC case it can be demonstrated that damage can be limited to violation of an individual user’s account [7]. This research has found that a MAC based medical data system, although viable, still presents some key research and practical deployment challenges. In particular, the “strict” operational mode offered by SELinux may be seen as being too rigid for deploying Role-Base Access Control or RBAC structures with the required levels of flexibility needed in practical healthcare situations. Without this flexibility, system reconfiguration may be required each time a user is added or removed. This is infeasible in practice and is already the subject of a number of active research projects. It was shown with the demonstrator described in this paper that a compromise can be derived that provides an application level proxy to facilitate a secure, role-based access interface. A balance has to be struck between strict access control security and the degree of flexibility for dynamic modification of any system in the “real world”. Any approach taken should be determined from a privacy impact oriented risk assessment process. For example, such an assessment might readily determine a need for emergency over-ride capability to enable at least wide read-only access to medical/health data. Such a facility would, however, have to be subjected to new audit and control requirements as well as to limitations potentially related to time periods and the location of users. In this regard an implementation that can support dynamic reconfiguration in a manageable and understandable manner may be essential. Earlier MAC systems were simply not designed for this environment where the security policy may need to be dynamic, not just in detail but also in structure.

Acknowledgments This research was supported under the Australian Research Council's Special Initiative on e-Research funding scheme (project number SR0567386). The authors would like thank RedHat (Asia Pacific) Ltd. and the E-Health Research Centre (a joint CSIRO and Queensland Health initiative) who have been highly supportive.

References [1] Foster I, Kesselman C. The Globus Project: A Status Report. Proceedings of the Seventh Heterogeneous Computing Workshop, 1998, ISBN: 0-8186-8365-1. [2] Loscocco PA, Smalley SD, et al. The Inevitability of Failure: The Flawed Assumption of Security in Modern Computing Environments. Proc. 21st National Information Systems Security Conference, Crystal City, VA, 1998. [3] Loscocco PA, Smalley SD. Meeting Critical Security Objectives with Security-Enhanced Linux. Proceedings of the 2001, Linux Symposium, Ottawa, 2001. [4] National Security Agency. Security Enhanced Linux homepage. Available at http://www.nsa.gov/selinux, 2000. [5] Thompson, K. Does SELinux support X-Windows? http://www.crypt.gen.nz/selinux/faq.html#CP.7 Nov. 2006. [6] Oeschlin P. Making a Faster Cryptanalytic Time-Memory Trade-Off, Proceedings of Crypto 2003, pp 617-630, 2003. [7] Henricksen M, Caelli W, Croll PR. Securing Grid Data Using Mandatory Access Controls. to appear 5th Australasian Symposium on Grid Computing and e-Research (AusGrid 2007) Ballarat, Australia, Feb. 2007.

Future needs in terms of research also involve a better understanding of the complementarity of SELinux’s concept of “type enforcement” versus more traditional security structures based on hierarchical “Multi-level Secure” or MLS schemes. In the health information area it needs to be determined whether or not such hierarchical security schemes have a place or not and, if so, to what

Address for correspondence Prof. P.R Croll, Information Security Institute, QUT, 126 Margaret Street, Brisbane 4001, Australia. Email: [email protected]

502

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Proposal of a French Health Identification Number Interoperable at the European Level Catherine Quantina, François-André Allaertb, Paul Avillachc,d, Benoît Riandeye Marius Fieschic, Maniane Fassaa, Olivier Cohenf a. Service de Biostatistique et Informatique Médicale, CHU de Dijon, BP 77908, 21079 Dijon Cedex, INSERM EMI 0106, France b. Department of Epidemiology & Biostatistics, Mc Gill University, Montreal, Canada c. LERTIM Faculté de Médecine Université de la Méditerranée 27 Bd Jean Moulin 13385 Marseille cedex 05, France d. ISPED Université Victor Segalen Bordeaux II, 146 rue Léo Saignat, 33076 Bordeaux cedex, France e. Institut National d’Etudes Démographiques (INED), 133 Bd Davout, F-75980 Paris Cedex 20, France f. Laboratory TIMC-IMAG UMR 5525, CNRS University of Joseph Fourier Medical School of Grenoble, France

Abstract

Introduction

The French ministry of Health is setting up the Personal Medical Record (PMR). This innovative tool has long been expected by French Health Authorities, Associations of Patients, other Health’s associations, those defending Individual Liberties and the French National Data Protection Authority. The PMR will lead to improvements in many areas such as Diagnosis (Research and monitoring) Healthcare (Management of emergencies, urgent situations, Temporal health monitoring and evaluation), Therapy (Cohorts of patients for Clinical trials and epidemiological studies). The PMR will foster safe healthcare management, clinical research and epidemiological studies. Nevertheless, it raises many important questions regarding duplicates and the quality, precision and coherence of the linkage with other health data coming from different sources. The currently planned identifying process raises many questions with regard to its ability to deal with potential duplicates and to perform data linkage with other health data sources. Through this article, using the electronic health records, we develop and propose an identification process to improve the French PMR. Our proposed unique patient identifier will guarantee the security, confidentiality and privacy of the personal data, and will prove to be particularly useful for health planning, health policies and research as well as clinical and epidemiological studies. Finally, it will certainly be interoperable with other European health information systems. We propose here an alternative identification procedure that would allow France to broaden the scope of its PMR project by making it possible to contribute to public health research and policy while increasing interoperability with European health information systems and preserving the confidentiality of the data.

In the majority of industrialized countries, at the heart of many of the concerns relating to electronic processing of health information lies the problem of patient identification. In August 2004, the French government decided by law to initiate a national project concerning Electronic Health Records called the “Dossier Médical Personnel”, the Personal Medical Record (PMR) [1]. It intends to promote health care coordination, enhance the communication of health information and reduce iatrogenic accidents. The most important aim of this article is to show and present the principles of the French unique patient identifier created relative to the PMR. However, it will also explain and show its major disadvantage, which is its interoperability characteristic. For example, this current patient identifier seems to be incompatible with the identifier of the European health card [2]. This paper will thus demonstrate, in its first section that the French health identification number does not ensure interoperability at the European level. The second section proposes an alternative identification process that would allow France to improve the quality, security, precision and coherence of its PMR project. The developed and proposed health identifier will be extremely useful for health planners, those dealing with health policies, public health research and clinical and epidemiological studies, and this, at national, regional and international level.

Keywords:

•

Materials and methods The issues and criteria of a unique patient identifier The identification of the patient in a health-care structure and particularly in the framework of the electronic healthcare record of a care network is a major issue [3]:

electronic health records, unique patient identifier, security

503

Patient care continuity requires secure, precise, coherent and reliable patient identification through all of the health information systems.

C. Quantin et al. / Proposal of a French Health Identification Number Interoperable at the European Level

Unique Identifier (UID) 10) Unique Health Identifier (UHID) 11) National Health Identifier (NHI).

•

The reduction and management of identificationrelated error is one of the major constraints necessary to maintain and improve the quality of care; • A reliable unique format for patient identification is required in order to design and implement an interoperable health information system focused on the patient. In the context of the “Principle and Process of Patient Identification” project, the French Group for the Modernization of Hospital Information Systems (GMSIH) [4] inventoried the principles and the architecture of identification systems by specifying two aspects of the problem:

The five most important criteria and characteristics (Table 1) of the Health Care Client Identifier (HCCI) Designation are adapted from the AS5017-2006 and provided here as a guide to assigning a unique identifier [6]. Table 1 - criteria and characteristics of the Health Care Client Identifier (HCCI) Atomic (the HCCI Designation should be a single data item. It should not contain sub-elements that have meaning outside the context of the entire HCCI Designation. Nor should the HCCI Designation consist of multiple items that must be taken together to constitute an identifier).

1. “Identification” will specify which information will be used to uniquely identify a patient. Multiple identifiers can be proposed for the same patient in different applications. 2. “Merging” identification areas, requiring the implementation of specific methods (intra-health structures and among health structures). This group also inventoried the international patient identification experience of ten countries (Germany, Australia, Canada, Denmark, Finland, Luxembourg, the United States, New Zealand, The Netherlands and the United Kingdom) considered by the GMSIH as representative of best practices in the areas of socialized health care, electronic health care information systems or patient identification.

Content-free (the HCCI Designation should not depend on possibly changing or possibly unknown information pertaining to the health care client. Including content in the HCCI Designation will make it impossible to assign the ‘correct’ identifier if that information is not known. It also leads to invalid situations if the information changes: for example, what happens to an identifier based on sex if the health care client has a sex change procedure).

The American Society for Testing and Materials (ASTM), a standards development organization accredited by the American National Standards Institute, identified 30 criteria that were published in the Standard Guide for Properties of a Universal Healthcare Identifier [5]. The most recent standard on Health Care Client Identification we know is from Australian Standard® AS 5017-2006 published in June 2006 [6]. This standard includes data elements that jointly comprise a unique identifier for health care clients. It is explained that the combination of the client identifier and the health care establishment that assigned the identifier is one way to indicate unique identification. The logical structure for these data elements has shown that, there may be multiple identifiers collected for any one individual. Together, the data elements that makeup one complete Health Care Client Identifier are

Longevity (an HCCI system should be designed to function for the foreseeable future. It should not contain known limitations that will force the system to be restructured or revised radically).

a) Health Care Client Identifier Designation b) Health Care Client Identifier Geographic Area c) Health Care Client Identifier Issuer d) Health Care Client Identifier Type.

Unique (a valid HCCI Designation should identify one and only one health care client. A health care client should have only one primary HCCI Designation)

Permanent (once assigned, an HCCI Designation should remain with the health care client. It should never be reassigned to another client, even after the health care client’s death). Unambiguous (whether represented in automated or handwritten form, an HCCI Designation should minimize the risk of misinterpretation. Where using alphanumeric identifiers, be aware of possible confusion with the number ‘0’ with the letter ‘O’ and the number ‘1’ with the letter ‘I’)

Most health care clients have more than one identifier. The most commonly used identifier should be collected as the Primary Client Identifier (the first listed client identifier). This identifier is generally that assigned by the organization as the means of uniquely identifying the client. Often, there are other identifiers also related to the client, and these should be collected and recorded as Other Client Identifiers (listed second or subsequently in the identifier list). There can only be one Primary Client Identifier, but there may be many Other Client Identifiers collected for any one application. Examples of Health Care Client Identifiers include 1)Person Identifier 2) Medical Record Number (MRN) 3) Local Client Identifier 4) Health (care client) Identification Number 5)Unit Record (UR) Number 6) Enterprise Identifier 7) Area Identifier 8) State/Territory Health Identifier 9)

These criteria are designed to support four basic functions of a universal health care identifier: 1. Positive identification of patients when clinical care is rendered, 2. Automated linkage of various computer-based records on the same patient for the creation of lifelong electronic health care files, 3. Provision of a mechanism to support data security for the protection of privileged clinical information (does not attempt to address all safety concerns, however) 4. Use of technology for patient record handling to keep health care operating costs at a minimum.

504

C. Quantin et al. / Proposal of a French Health Identification Number Interoperable at the European Level

would be indirectly maintained at both sides. One and only one request form number would be associated with each social security number and one and only one INS would be associated with each request form number. The trusted authority would not require knowledge of the social security number to prevent collisions; they would need only to ensure that the INS had been assigned to no other request form number. Similarly, by checking that each form number is given to only one INS, they would prevent duplicates provided that no duplicate form number existed further up the chain.

The founding principles of the French Health Identification Number In September 2005, the French government defined the founding principles of the Health Identification Number (INS in France) of the personal medical file (PMR). This number must be: •

• •

• •

unique: to ensure that two people cannot have the same INS, thus avoiding erroneous attribution of a diagnosis. An identifier is therefore associated with a person whose identity can be validated. content-free: the number reveals no information about the holder (sex, age, place of birth…). public: the number can be legally used by all health care professionals authorized by the patient. It can be stored in the health care professional’s information system. The number is therefore not secret. permanent: will remain the same for the life of the patient and, possibly, beyond. irreversible: impossible to determine the identifier by calculating backwards from the social security number. However, if lost, the same number can be recreated.

The trusted authority would send the INS to the host along with other patient identifiers, such as first and last name and date of birth, but never the social security number. The host would thus possess only those identifiers that could not be used to reconstitute the social security number. Criticism of this method of generation The principal criticism [7] addresses the second main function, as defined in the ASTM Standard Guide for Properties of a Universal Healthcare Identifier [5]: Automated linkage of various computer-based records on the same patient for the creation of lifelong electronic health care files.

First proposal for the creation of the INS in the PMR

Thus, the fact that the proposed system leads to an institutional disconnect between the INS as a central health system identifier and the social security number. The main reason cited to justify this disconnect is the desire to prevent the creation of a population table of correspondence between the social security number and the INS. This argument is, however, open to criticism insofar as all health professionals and in particular large institutions whose patient databases cover a large proportion of the population of a region must have access to the correspondence between the social security number and the INS.

To meet the “uniqueness” condition, the French patient identification working group recommended that the identifier be associated with the social security number. The solution proposed by this working group involves the creation of a Health Identification Number (INS in France) for each patient managed at the patient’s request by an independent organization known as a “trusted authority”. The role of the “trusted authority”, which may not be a host, is to guarantee that the identifier is not and will not be duplicated. The trusted authority will also be in charge of the secure access, the respect of confidentiality and the integrity of all the identification workflows.

If the link between the Social Security Number (SSN) and the INS is broken, relating PMR data with other patient data for research purposes becomes impossible. For example, relating PMR with other sources is necessary not only to complete their data but also to ensure multi-source validation in conformity with the recommendations of the report of the academy of sciences regarding epidemiology. If research using PMR data is not possible in the short term because of the time necessary to create databases that are sufficiently structured, exhaustive and validated, it can be expected that the data will be of use in less than a decade, if an acceptable identifier is adopted. That’s why, like our British colleagues who complain that overzealous interpretation of UL laws stifles epidemiological research [8-11], we think that the current French identifier may cause particular difficulties regarding epidemiological and clinical research.

To preserve the anonymity of the identifier, the trusted authority creates the INS through the following process: •

The patient chooses an approved host and advises the PMR office using a request form and the host using a membership form. • By contacting the National Health Insurance organization the PMR office confirms the requester’s affiliation and that it is a first-time request • The trusted authority creates the INS using the request form number that it transmits along with patient attributes to the host. • The host then informs the trusted authority that the patient’s INS is operational The use of the request form number and not the social security number to generate the INS is designed to guarantee that the number is content-free and the INS creation process mathematically irreversible, i.e. no mathematical operation would permit backward calculation from the INS to lead to the social security number. The guarantee of INS anonymity would rely on the absence of a table of correspondence directly linking INS and the social security number. Detection of collisions and duplicates would occur by way of the request form number. Nonetheless, the relationship between the social security number and INS

Moreover, it must be possible to reconstitute the INS, which represents the true signature of a patient, whatever the nationality or the country of origin starting from features of identification that are always available. These methodological principles must guarantee the secure and perennial identification of all the patients. These principles must apply whatever the future use of the information. Indeed, the technical and methodological choices must be able to follow evolutions in the legislation, which may be modified according to the evolution of mentalities and cultures.

505

C. Quantin et al. / Proposal of a French Health Identification Number Interoperable at the European Level

diagnosis and healthcare, epidemiological research). The same solutions also derived through an irreversible encryption of the unique Social Security Number (SSN) have been proposed in Belgium [16] and New Zealand.

Our proposal It would be perfectly possible to preserve the confidentiality due to the patient and desired by patient associations by setting up anonymous procedures [12-14] such as those adopted by the Institut de Veille Sanitaire (Health Surveillance Institute) on the recommendation of the French National Commission for Data protection and the Liberties (CNIL), in the context of the follow-up procedures for the 30 diseases subject to mandatory reporting (including AIDS).

Regarding data access for public health research, the use of the Social Security Number (SSN), after hash-coding, would thereby allow the linkage of the main national databases. However this solution raises many questions. Firstly, in France, foreigners do not a have a complete social security number. For instance, the last characters, corresponding to the country of birth, are not systematically filled. Moreover, as the date of birth in some foreign countries is not known precisely, two different patients may have the same date and the same country of birth, resulting in collisions, and linkage errors. Secondly, as the Social Security Number (SSN), has a different structure according the country, it can not be a solution for a unique patient identifier, interoperable at the European level.

Unlike encryption methods that must be reversible to allow the legitimate recipient to decode the message, unidirectional hashing techniques are irreversible. Hashing produces a perfectly anonymous code (it is not possible to retrace the patient’s identity) that is always the same for a given individual so that patient data can be linked. There are many medical applications, which include the creation of national databases (such as those relating to the national follow-up of infected subjects - approximately 100,000 patients – an excellent example of what can be put to the service of epidemiological research, with complete patient approval) as well as regional and inter-regional databases in many areas (cancer, perinatality, genetic diseases). This system has also allowed (based on the hash-coding of the Social Security Number (SSN), the gender and the date of birth), standardized hospital discharge abstracts to be linked, classified into French Diagnosis Related Groups at the French national level and linked to the data of the national medical insurance information system. An anonymous procedure based on hash coding is also used for chaining patient files in Switzerland [15].

One solution would be to add personal patient characteristics such as family name, first name, date of birth (separately hashed) to the social security number, which would help to conform to the recommendations of the International Association of Medical Information Technology and where possible to ensure interoperability of this identifier with a European identifier [17]. In fact, in the national medical insurance information system like in the standardized discharge abstracts, gender and date of birth have already been associated with the social security number, in the same identification field, before hash-coding, resulting in a single code. We could thus propose to build the INS through the simultaneous hash-coding of gender and date of birth associated with the social security number.

In the case of the PMR the situation is not, in fact, more complex because several requirements must similarly be met in a similar way:

However, due to a simple error in the field for gender, linkage of data from the same patient can be definitively refused. It is important to note that this field is highly unreliable (numerous data entry errors have been observed in hospital settings). In addition, with regard to the quality of the linkage this variable is undiscriminating insofar as the probability of two people being the same gender is close to 50% and it does not discriminate for twins. It would therefore be preferable to replace this variable with one that is more sensitive, such as the first and last name of the beneficiary. As a consequence, our first proposal is to separately hash the social security number (removing the last digits), the date of birth and the last and first names, then merge this into a single signature of patient identity (Figure 2).

•

demands of the CNIL, patient and healthcare professional associations regarding confidentiality of personal information contained in the PMR, and respect of the law of August 13, 2004 concerning the use of data, • needs in public health or for individuals to have access to these data, particularly when the patient has given express consent. Ideally, hashing the social security number would help meeting these requirements (Figure 1).

Figure 1 - Hash coding Regarding confidentiality, insofar as the social security number could not be reconstituted using the INS, the link between them would be broken. Another advantage of using hash-coding is to meet the criterion of being focused (created and maintained solely for supporting health care cf. Table 1). As a consequence, using different keys for hash-coding will allow the creation of distinct identifiers according to different purposes (health administration,

Figure 2 - Extended identifier for European interoperability

506

C. Quantin et al. / Proposal of a French Health Identification Number Interoperable at the European Level

We could also propose, if possible to add biometric components (separately hashed). Biometric technologies are sometimes proposed in solving the problem of associating patients with their medical data, as they do not require the patient to bring any documents or remember information. Though this technology represents a real progress both in the identification and in the authentication of the patient, it raises many questions [18]. First, the accuracy and reliability of each biometric technology (for instance: finger print, iris scan, retinal scan and DNA) are not optimal in all circumstances. For example, fingers are frequently injured. Though retina scans and DNA analyses are the most accurate (with the exception of twins for DNA) they are also quite intrusive, which render even more difficult the possibility of combining these technologies, as proposed in some commercialized biometric systems. Moreover, the costs of the biometric solutions have to be considered. But the main problem lies in their acceptation by ethical organizations such as patients associations, national committees on ethics, human rights associations and national committees for data protection. In order to render this solution compatible with the current national procedures, it would be preferable to add the biometric component (separately hashed) to the hashed social security number, in order to ensure (at the European level) the interoperability of this identifier with national ones.

[3] Fieschi M, Les données du patient partagé: la culture du partage et de la qualité des informations pour améliorer la qualité des soins. Rapport au ministre de la santé. janvier 2003. [4] Group for the Modernization of Hospital Information Systems [cited 2006 December 4th]; Available from: http:// www.gmsih.fr [5] American Standards for Testing and Materials (ASTM). (Standard Guide for Properties of a Universal Healthcare Identifier (UHID), Designation: E1714-95, Volume 14.01, Issued June 2000 [6] Australian Standard® AS 5017-2006 on June 2006 www.standards.org.au [7] Quantin C, Guinot C, Tursz A, Salomez JL, Rogier C, Salamon R. Should epidemiological data be extracted from personal medical files? Revue d’Epidémiologie et de Santé Publique, 2006;54:177-84 [8] Walley T. Using personal health information in medical research. British Medical Journal 2006; 332; 130-1. [9] Ward H, Cousens S, Smith-Bathgate B, et al. Obstacles to conducting epidemiological research in the UK general population. British Medical Journal 2004; 329:277-9. [10] Verity C, Nicholl A. Consent, confidentially and the threat to public health surveillance. British Medical Journal 2002; 324:1210-3. [11] Iversen A, Liddell K, Fear N, Hotopf M, Wessely S. Consent, confidentiality and the Data Protection Act. British Medical Journal 2006; 332:165-9. [12] Quantin C, Bouzelat H, Allaert FA, Benhamiche AM, Faivre J, Dusserre L. Automatic record hash coding and linkage for epidemiological follow-up data confidentiality. Methods of Information in Medicine 1998; 37:271-7. [13] Quantin C, Bouzelat H, Allaert FA, Benhamiche AM, Faivre J, Dusserre L. How to ensure data security of an epidemiological follow-up: quality assessment of an anonymous record linkage procedure, International Journal of Medical Informatics 1998; 49:117-22. [14] ISO TS 25237 "Health informatics - Pseudonymisation practices for the protection of personal health information and health related services" [15] Borst F, Allaert FA, Quantin C. The swiss solution for anonymously chaining patient files. MEDINFO 2001;10 (Pt 1):1239-41. [16] Roger France FH, De Clercq E, Bangels S M. Purposes of Health Identification Cards in Belgium – EFMI, European Federation for Medical Informatics, IOS Press, 2005, Connecting Medical Informatics and Bio-Informatics [17] Quantin C, Allaert FA, Gouyon B, Cohen O. Proposal for the creation of a European healthcare identifier. Stud Health Technol Inform 2005; 116: 949-54. [18] Vaclav M, Zdenek R. Biometric authentication systems. A technical report. Retrieved April 15, 2006 from ecommonitor.com web site: http://www.ecom-monitor.com/ papers/biometricsTR2000.pdf

Conclusion Our proposal for a French Health Identification Number will make it possible to uniquely identify and link a patient to his specific medical data. By hashing the social security number it will be possible to link the information of the personal medical file to other national health information sources, with the aim to complete or validate Personal Medical Data (PMR) or conduct epidemiological research. Adding personal patient characteristics such as first and second names and date of birth and/or biometric identifiers (separately hashed, then merged) to the hashed social security number would also contribute to the establishment of European public health statistics by matching healthcare data of the patients' records with other administrative data (mortality, social information …). This data linkage would thus meet the requirements of anonymous data of the European directive on data protection. This proposal could also be used in the discussions about the creation of European health care identifier. Of course, even if anonymised, the use of biometric components would require, the authorization of the National commissions for data protection. Acknowledgements This research was supported by The French National Agency for Research (ANR).

Address for correspondence Professeur Catherine QUANTIN Service de Biostatistique et Informatique Médicale Centre Hospitalier Universitaire - BP 77908 21079 DIJON CEDEX - Tel: 33 3 80 29 36 29 Fax: 33 3 80 29 39 73 Email: [email protected]

References [1] Dossier Médical Personnel. [cited 2006 December 4th]; Available from: http://www.d-m-p.org [2] Pre-commercial procurement of innovation - A missing link in the European innovation cycle - March 2006. [cited 2006 December 4th]; Available from: http://europa.eu.int/ information_society/research/key_docs/documents/ procurement.pdf

507

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Clearinghouse: A Teleradiology Platform Emphasizing Security of Data and Communication Michael Spitzer, Lars Brinkmann, Frank Ueckert Department of Medical Informatics and Biomathematics, University Hospital, University of Muenster, Germany

uments, while at the same time providing a maximum of flexibility, data security and privacy.

Abstract The Clearinghouse application platform is a web based solution for secure digital exchange of radiological images and other clinical documents among authorized researchers and physicians. It implements a sophisticated security and role model to protect privacy and to minimize the risk of eavesdropping of patient data. The Clearinghouse serves as a centralized platform for distributed, distantly located medical research and health care. It is based on Open-Source software, thus ensuring continued support, maintenance, security and last but not least continuity of the platform. The use of the Clearinghouse minimizes turnaround times by superseding comparably slow and insecure conventional communication methods otherwise used for the exchange of radiological images and clinical documents, such as standard mail and courier services. Furthermore, it alleviates the integration of distantly located expert knowledge into diagnostic routines, culminating in an increased health care quality regardless of location of patients or physicians.

The integration of the Clearinghouse platform into the daily radiological routine enables effortless communication and collaboration with national and international experts. In reverse, patients in rural regions without ready access to specialists may benefit from expert opinions, improving the quality of their therapy. Due to the nature of e.g. rare diseases, and the fact that diagnosis, research and treatment are geographically separated in most of these cases, the Clearinghouse is especially suited to centrally organize and distribute relevant data. Exchange of data and knowledge in internationally relevant studies is intensified, while communication costs and turn-around times are lowered.

Materials and methods Traditional workflow The traditional workflow for exchanging clinical documents in general and radiological images in particular, involves mainly delivery by either patients themselves or by standard mail and courier services. Patients may be sent by their physicians to radiologists for taking radiological images, which in turn are sent back either via mail or are given to the patients to hand them over personally to their physician. If no other parties are involved in diagnosis and treatment based on such documents, this practice may be feasible, depending on the geographic distance of physician and radiologist.

Keywords: telemedicine, teleradiology, radiology information systems, PACS, remote consultation, computer security, information storage and retrieval.

Introduction In medicine and radiology applications the efficiency and contribution to health care quality of collaborations of distantly located participants is primarily dictated by the efficiency of the communication methods used. Travel times of several days for international mail are not unusual, and may be too lengthy to deliver advantages for a patient’s treatment. Furthermore, important original documents may be lost irrecoverably during delivery. In case of e.g. rare diseases the probability not to find expert knowledge on-site is comparably high, and local physicians without a strong background in such diseases are forced to (i) send these patients to specialists or (ii) to rely on conventional mail to communicate diagnostic findings and subsequent treatment with colleagues.

Once multiple, probably distantly located, parties are involved in diagnosis and thus require access to existing or newly acquired documents of a patient, these documents have to be sent out (preferably as copies) to all participants by mail, courier services or as fax. Taken that none of these services are trusted with original documents the risk of irrecoverable loss of data is minimized, however the risk of eavesdropping of private and possibly compromising data still remains. This risk may be diminished by using e.g. sealed envelopes which in turn increase cost and expenditure, yet this measure does not eliminate disclosure of private data by accidental loss.

The Clearinghouse application as an approach for a teleradiology platform aims to improve the speed and latency in communication of medical imaging and other clinical doc-

This traditional, decentralized workflow of exchanging patient data and documents bears several unratable risks of

508

M. Spitzer et al. / Clearinghouse: A Teleradiology Platform Emphasizing Security of Data and Communication

data loss or delay in schedule which may negatively influence a patient’s diagnosis and treatment.

privacy of patients ([2], [3]). All internal and external communication is encrypted to prevent eavesdropping and possible “man in the middle” attacks (cf. Figure 2). Both data classes (IDAT and MDAT) are transmitted separately to the client using encryption and merged in the client’s web browser only.

The Clearinghouse platform The Clearinghouse platform addresses these risks and shortcomings by unifying and optimizing access to a patient’s data, documents and clinical history. It poses a centralized web application, amalgamating the contributions of geographically separated institutions to a patient’s file and subsequent treatment. It features a sophisticated security concept and role model, both of which are described in the following paragraphs. On the client’s side, no additional software needs to be installed apart from a web browser and Java, all other components (cf. section Clearinghouse features) are provided online and instantaneously by the server.

Queries on treatment data of specific patients are internally carried out using temporary IDs (TempID, cf. Figure 1). The TempID is generated by the IDAT server and propagated to the client and the MDAT server. The client then automatically queries the MDAT server for the corresponding patient’s data using the TempID. The latter is invalidated upon completion of the transaction or after a specific idle period has passed. This procedure ensures that no transaction may be carried out twice by e.g. interception by unauthorized parties, and eliminates the possibility of malicious transactions not explicitly requested by authorized persons. The Clearinghouse platform is implemented using inexpensive Open-Source software and is strongly based on the LAMP concept (Linux, Apache, MySQL, PHP).

Figure 1 - Abstraction of the Clearinghouse security concept and communication flow Data Security Concept The security concept of the Clearinghouse platform (cf. Figure 1) is derived from a generic data security concept developed by the Telematic Platform for Medical Research Networks (TMF), Germany [1], and has been specifically tailored to meet the requirements of a web-based platform for digital image exchange. It implements a strict physical and logical separation of patient identification data (IDAT) such as name, gender and age, and medical treatment data (MDAT) such as digital images, clinical documents, medical evidences and diagnoses. In the case of DICOM objects, the header data is made pseudonymous by replacing data identifying patients in corresponding header fields with a so-called picture ID. It is generated by the MDAT component of the system and will not be propagated to other system components, except to clients (i.e. treating physicians) who are legally entitled to obtain knowledge of both data classes (IDAT and MDAT).

Figure 2 - Scheme of the Clearinghouse IT infrastructure Infrastructure The infrastructure of the central Clearinghouse teleradiology system (cf. Figure 2) resembles the theoretical data security concept as described in the previous section. The strict physical separation of the IDAT and MDAT data classes is realized by housing corresponding components in separate data centers, including system administration tasks carried out by different persons. Two web servers are used to deliver encrypted IDAT or MDAT content to the client’s web browser. Communication to and among system components (i.e. database servers and web servers) is carried out exclusively via secured lines, e.g. using a virtual private net (VPN) for the Clearinghouse system components or SSL encryption for communication with clients. Reference centers, e.g. reference radiologies, may be integrated via VPN or using SSL/TLS encryption of the DICOM network protocol ([4], [5]).

Each data class is stored on a separate database server featuring encrypted storage space. Each server is maintained by a separate system administrator and is located in the demilitarized zone of separate data centers, behind a firewall. Thus no persons except legitimate treating physicians (or in general, users with appropriate data access rights) are able to merge patient and treatment data. Furthermore, should one server be compromised, no association to the other data class is possible, thus protecting

509

M. Spitzer et al. / Clearinghouse: A Teleradiology Platform Emphasizing Security of Data and Communication

DICOM container, as well as protocols for the networked communication of DICOM objects (e.g. STORESCP). The Clearinghouse supports the whole DICOM workflow by integration of Open-Source software and libraries (Offis DICOM toolkit [6], XMedCon [7]).

Workflow The Clearinghouse platform transforms the traditional decentralized workflow into a centralized one. The web application forms a central venue for all participants and data corresponding to clinical documents. On the one hand, the platform is suitable for mutual exchange of documents between several physicians and experts, to confer clinical diagnoses with high efficiency and low latency. On the other hand, it is also suitable as a central platform for patient and data acquisition within the scope of a clinical trial or study. Here, participating physicians, institutions and clinics contribute patients and corresponding relevant documents to a specific study, with the study center having access (probably pseudonymous only, depending on study treaties) to all contributed data.

Apart from DICOM objects containing a single image the Clearinghouse is also able to handle multislice DICOM objects as generated by e.g. systems employed in cardiology diagnoses and treatment (e.g. heart catheter movies). Conversion of image objects into other formats (e.g. MPEG movies for multislice DICOM objects) is possible in almost any arbitrary way. Furthermore, up- and download of virtually any generic non-graphic file formats like PDF, Microsoft Office or Open Office documents etc. is possible.

Multiple studies may coexist within the same platform instance, without the need to have separate servers for each particular study. The security concept, including the role model, ensures that no data is exchanged between studies. Access to a study and patients or documents therein is granted by either the study manager and/or the person who contributes a specific patient. Read and modification rights may be granted to other users (e.g. of the corresponding study), while the right to delete objects, documents or patients is exclusively granted to the person who contributed the specific patient to the Clearinghouse platform. The study center always has read access to all contributed data and documents, as described above. The Clearinghouse allows dispatching DICOM objects to previously specified, trusted Reference Radiologies by implementation of the DICOM STORESCP protocol. This way, reference diagnoses, which are then included in the corresponding patient’s file, may be obtained when needed. Figure 4 - Thumbnail overview of image objects contained in a specific document (some data garbled for privacy reasons) The Clearinghouse enables straightforward upload of complete DICOM CD-ROMs by a sophisticated, operating system independent Java applet (cf. Figure 3). The applet is executed within the web browser of the user and automatically scans CD-ROMs for DICOM data. Once identified, these data are compressed, then transmitted to the platform via encrypted communication and stored within a previously defined document of the corresponding patient’s record.

Figure 3 - The Java upload applet of the Clearinghouse teleradiology platform for upload of complete DICOM CDs

For each patient a virtually unlimited number of documents may be created, and each document may comprise any number of objects. Primarily, a document is presented as a thumbnail overview of all image objects therein (cf. Figure 4), automatically sorted according to the header data (e.g. based on study ID and/or slice number). By clicking on an object a detail view is entered, with (in case of a DICOM object) a listing of all available header data (cf. Figure 5).

Clearinghouse features The Clearinghouse teleradiology platform is specifically suited to handle, besides generic bitmap graphics formats, digital medical images encapsulated in the DICOM format. The DICOM format has become the most important format for the generation, storage and communication of digital medical images. This standard includes a comprehensive description of data organization within the

510

M. Spitzer et al. / Clearinghouse: A Teleradiology Platform Emphasizing Security of Data and Communication

Figure 6 - The Radscaper Java DICOM viewer applet of the Clearinghouse teleradiology platform In summary, the Clearinghouse telemedicine approach has the potential to improve diagnosis and treatment while data security and privacy are improved at the same time.

Figure 5 - Detail view of a DICOM image object including header data (some data garbled for privacy reasons)

The cost-saving employment of the Clearinghouse platform is one of its major advantages. Due to consistent use of Open Source software the costs for installation, maintenance and support are comparably low. Furthermore, the platform constitutes an easily extensible and maintainable basis for future research projects.

To allow users the quick visualization of selected DICOM objects, the Radscaper [8] DICOM viewer (cf. Figure 6) was integrated into the Clearinghouse. As for the DICOM upload applet, the Radscaper viewer is written in Java and thus is operating system independent. This built-in DICOM viewer functionality relieves users to install standalone DICOM viewers on the client’s workstations, and allows for portable and more detailed analyses of DICOM objects. The Radscaper applet features several tools e.g. for adjusting window center and width, zoom, image rotation and measurement functionalities. A cinematic mode allows advancing through multiple slices with adjustable speed.

Comparison to other teleradiology solutions There are several solutions for teleradiological platforms, some following a similar architecture with a central web server communicated with by clients using a web browser [16], other implement a special Java-based solution for execution at the client’s workstation [17] or use the DICOM-e-mail facilities of the DICOM standard [18]. The latter approach allows for simple participation in such teleradiology networks by use of a standard Email application. However, collaborative features such as instant access for multiple users are not available except by sending mails to several recipients simultaneously. Furthermore, utilization as an electronic patient record is not feasible since other document types would have to be encapsulated in a DICOM container first. The Java-based solution described in [17] features a security concept comparable to the Clearinghouse in terms of encrypted communication, yet supposedly without the strict separation of IDAT and MDAT data classes, missing in [16] as well. Additionally, the solution in [17] features teleconferencing abilities and an interface to existing PACS applications, a topic that has not been tackled yet in practice by the Clearinghouse system. Main advantages of the Clearinghouse platform over the aforementioned solutions are:

Results Advantages By focusing on the exchange and annotation of digital image data the Clearinghouse platform improves the otherwise slow traditional transfer of medical images via standard mail. Both the risk of data loss and the response time are significantly decreased since • • • •

geographical distances are transformed into virtual “technical” distances, the centralized structure guarantees for immediate access to any document as well as for reliable backup, IDAT and MDAT are securely stored in physically separate databases (cf. section Data Security concept), the sophisticated role-based user model enables to define access to documents with fine granularity.

•

• •

511

no installation of specific standalone client software necessary, thus updates of the central system software are immediately available and usable by all clients, inexpensive setup and administration due to the use of Open-Source software and a centralized infrastructure, integration of reference centers by DICOMSCU/-SCP,

M. Spitzer et al. / Clearinghouse: A Teleradiology Platform Emphasizing Security of Data and Communication

• •

immediate availability of uploaded documents and digital images for all legitimate users, a security concept based on the generic concept of the Telematic Platform for Medical Research Networks (TMF), Germany. The concept is commonly accepted among local institutions and fully complies with at least German legal requirements.

[4] Digital Imaging and Communications in Medicine: http:// medical.nema.org [last access: 4th December 2006] [5] Hussein R, Engelmann U, Schroeter A, and Meinzer HP. DICOM structured reporting: Part 1. Overview and characteristics. Radiographics 2004: 24 (3): 891-896 [6] http://dicom.offis.de/dcmtk/ [last access: 4th December 2006] [7] Nolf E. XMedCon - An open-source medical image conversion toolkit. European Journal of Nuclear Medicine 2003: 30 (2):246. [8] http://www.divinev.com/radscaper/ [last access: 4th December 2006] [9] http://www.kinderkrebsinfo.de/e1664/e1676/e1758/e5468/ index_ger.html [last access: 4th December 2006] [10] http://www.kompetenznetz-leukaemie.de/ [last access: 4th December 2006] [11] http://www.kinderkrebsinfo.de/kpoh/ [last access: 4th December 2006] [12] Creutzig U, Jurgens H, Herold R, Gobel U, and Henze G. Concepts of the Society of Paediatric Oncology and Haematology (GPOH) and the German Competence Network in Paediatric Oncology and Haematology for the quality controlled development in paediatric oncology. Klin Padiatr 2004: 216 (6): 379-383 [Article in German] [13] http://www.netzwerk-ichthyose.de/ [last access: 4th December 2006] [14] Traupe H. Ichthyoses and related keratinization disorders. Management, clinical features and genetics. Hautarzt 2004: 55 (10): 931-941 [Article in German] [15] http://www.netzwerk-eb.de/ [last access: 4th December 2006] [16] Corella F, Lewison KP, Talukder M. A Teleradiology Architecture Featuring Security and High Performance. J Digit Imaging 2002: 15 Suppl 1: 214-215 [17] Munch H, Engelmann U, Schroter A, Meinzer HP. The integration of medical images with the electronic patient record and their web-based distribution. Acad Radiol 2004: 11(6): 661-668 [18] Weisser G, Engelmann U, Ruggiero S, Runa A, Schroter A, Baur S, Walz M. Teleradiology applications with DICOMe-mail. Eur Radiol 2006 [Epub ahead of print]

Current employment The Clearinghouse platform has already been proven to be beneficial for e.g. studies and participating institutions by facilitating secure, instantaneous and geographically independent access to data and documents via the Internet. It is already successfully tested and used routinely within the scope of the Ewing study [9], the Nephroblastome and AML study [10], the competence network of pediatric oncology and hematology ([11], [12]), the Network for Ichthyosis and Related Keratinization disorders ([13], [14]) as well as the network for Epidermolysis bullosa [15]. Future development The current development and research is based on already implemented functionalities of the Clearinghouse. It concentrates on the further development and full implementation of the data security concept as described in this document. Further projects involve the extension of DICOM integration and functionalities and embedding of a WYSIWYG editor for elementary online editing of documents, as well as internationalization and translation of the platform. Acknowledgments The project is partially funded by the German Federal Ministry of Education and Research within the scope of the Competence Network for Paediatric Haematology and Oncology GPOH/KPOH (code 01GI0413), the Network for Ichthyoses and Related Keratinization disorders NIRK (code 01GM0606), and the German beneficence for child cancer (Deutsche Kinderkrebsstiftung).

Address for correspondence

References

Michael Spitzer Department of Medical Informatics and Biomathematics University Hospital, University of Muenster Domagkstr. 11 48149 Muenster, Germany Email: [email protected]

[1] http://www.tmf-ev.de/ [last access: 4th December 2006] [2] Stanberry B. Legal and ethical aspects of telemedicine. J Telemed Telecare 2006: 12 (4): 166-175 [3] White P. Legal issues in teleradiology – distant thoughts! Br J Radiol 2002: 75 (891): 201-206

512

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

K-Box: Automatic Structuring and Exchange of Medical Documents Based on the Clinical Documentation Architecture (CDA) Minh H. Doan, Paul-Ludwig Lott, Marek Václavík, Prof. Dr. Frank Ueckert Department of Medical Informatics and Biomathematics, University Hospital, University of Muenster, Germany

or direct communication between different information systems in different organizations possible [2]. Some organizations joined together and implemented electronic communication between their systems. But the majority is still exporting their medical information by using word processing software to write a report. The importing party then has to copy the data from the written down report and paste it into their information system manually.

Abstract Sustainable health systems should understand information as a resource which also has to be managed efficiently. Today’s electronic documentation is only the mandatory first step to automatic information exchange between hospitals, physician offices, pharmacies and other participants. The K-Box is a modular system which allows different monolithic information systems to be connected and integrated by using the clinical document architecture (CDA) as the standard document exchange format. A prototype K-Box has been implemented and a trial run is scheduled. The workflow selected for the trial produces discharge summaries in CDA format.

Regarding building sustainable health systems, we are sure that the ability to automatic information exchange between the system elements would enhance the property “sustainability” of the system itself. The K-Box described in this paper is such a software system which allows different medical information systems to be connected and exchange structured information.

Keywords: information systems, systems integration, public health informatics

The idea behind the K-Box is to have handy servers (“boxes”) at hospitals, physician offices, pharmacies and other participants, which are connected to local information systems to provide them a unique communication and data exchange capability. At this phase of the project we focus on clinical document exchange between physicians such as exchange of discharge summaries. The “K” in the name of the project is representing “Kommunikation” as the German word of communication. But actually, beside the communication subsystem the K-Box also comprise another subsystem which we call the structuring subsystem. The structuring subsystem is able to convert every document it receives in a CDA (Clinical Document Architecture) document by using machine-learning supported methods. Based on the generated CDA particular information entities then can be extracted and imported directly into foreign systems.

Introduction Defining historical medical attendance records as an instance of information in the context of health system, today’s hospitals and other healthcare organizations should understand information as a resource beside other resources. Furthermore, we assume that within this context information will become one of the most expensive resources in the future. Missing information would mean additional or double tests in some cases and cause extra costs as the consequence. Therefore, the development of sustainable health systems also has to pay attention to save the resource information by endeavoring after its reuse. Today, in German hospitals medical data are processed and handled by hospital information systems (HIS’) which also include electronic patient records (EPRs) [1]. Also, most of the medical offices in Germany usual use computeraided patient management systems to save medical data related to constituent patients. Regarding the German health system we can notice, therefore, that the evolutional step from paper supported documentation to electronic documentation is mostly done.

The K-Box architecture is modular. Each K-Box installation can be extended with the support of new document formats or communication interfaces by adding pluggable components (“Plugins”) to the running system.

Materials and methods Component-Based Software Engineering

Even though the documentation is done electronically, there is a big gap in electronic communication between German healthcare organizations because the electronic documentation was not forced to follow a nationwide or international standard. As a result, there is no integration

The K-Box development follows the paradigm of Component-Based Software Engineering (CBSE). That means that we focus on software components in all phases of development. Furthermore, all components have their own

513

M. H. Doan et al. / K-Box: Automatic Structuring and Exchange of Medical Documents

development cycle, and are released independently from each other.

text reasoning. A semi-structured text contains additional information (e. g. presentation mark-up), enabling nonNLP techniques to reach comparable or even better results. Non-NLP methods are typically based on sets of extraction rules (wrappers), often modelled as finite-state automata [6]. Both approaches can utilize machine learning, i.e. to adapt the extraction process according to the received feedback.

The term of software component is not new to the software engineering domain. It was first used in 1968 by Doug McIlroy at the NATO software engineering conference in Garmisch where the discipline Software Engineering itself was born. In his article McIlroy’s described his vision based on the model of mass production in the industry about mass development of software by combining reusable components instead of programming blocks of similar functionality from zero every time [3]. The term reusability is often used in the context of Object-oriented Programming (OOP). From this background CBSE is often confused with OOP. In fact, reusability is the main motivation of both paradigms. However, we distinguish between white-box reusability and black-box reusability: White-box reusability means that we reuse written source code and add modifications to it to solve a similar but not the same problem. On the other side black-box reusability means that we reuse independent software components in binary format to solve, intra-domain, independent, and repeating sub-problems [4]. It is the art of CBSE to design components that are not too specific to a context to make them widely reusable but also not too abstract. In other words, OOP makes use of white-box reusability and CBSE makes use of black-box reusability.

We apply adaptive text extraction to discharge summaries, trying to turn them from an unstructured form (binary Word-File) to a structured XML-document conforming to Clinical Document Architecture (CDA). CDA is a document standard based on the HL7 v3 Reference Information Model. In Germany, the SCIPHOX-specification (CDA Release 1) developed into the “Electronic Medical Report” (eArztbrief), derived from “CDA Release 2”.

Results Architecture of the K-Box The K-Box architecture conforms to the layered model for message-based integration systems, as described by Keller [7]. Four functional layers can be distinguished: An adaptor layer which supports various communication protocols, an internal transport layer which is responsible for intra-component-communication, a format transformation layer which converts a document to a new format, and a workflow layer which routes documents though the K-Box.

OOP and CBSE are no competitive paradigms. On the opposite, it is wise to combine both techniques in an advantageous way. In our project, we use OOP to develop the components themselves and follow CBSE to design the component hosting core-system. The OOP language used at the K-Box project is Java. Information extraction To process incoming messages, a typical commercial EAI system relies on explicit structure definitions. It expects a description written in a vendor specific meta-language. This approach reaches its limits when the explicit message structure is unknown (e.g. undocumented legacy systems) or too complex to be expressed this way. Integration engineers usually apply workarounds to solve this situation: they “reconstruct” the message structure by generalizing available examples or they content themselves with a functional definition: “How do I extract a certain data element?” Learning the structure “by example” can also be performed automatically or semi-automatically, by means of information extraction (IE) and machine learning. IE with text data (text extraction) has been widely applied in the medicine domain for biomedical data mining. (e.g. [5])

Figure 1 - Modules of the K-Box architecture The entry point for a document into the K-Box is an inbound adaptor. Inbound adapters are components of the communication subsystem. They are responsible for the receipt of documents through different communication protocols, e. g. FTP, SCP, or SMB/NetBIOS. After a document is received the inbound adapter creates a new transaction within the K-Box. Each transaction is related to an internal document envelope (IDE) which contains the document itself as an attachment and further meta information in the header part of the IDE. The body part of the IDE is reserved for the CDA. Subsequently, the document is routed through the K-Box following the information

The task of text extraction comprises populating data slots in a given target format with information pieces from the input text. Text classification means analysing a text part and assigning it with a discriminating attribute (tagging), like “relevant” versus “irrelevant” or “regular mail” versus “spam”. The choice of the appropriate text extraction method depends on the structuredness of the text. Extracting information from an unstructured text (words, sentences, paragraphs) requires linguistic analysis (natural language processing, NLP) to maintain a certain level of

514

M. H. Doan et al. / K-Box: Automatic Structuring and Exchange of Medical Documents

from a predefined workflow. After the document is finally converted the result will be sent by an outbound adaptor.

nents extracts CDA body information using text classification. CDA section captions (Level 2) such as “Diagnosis” or “Therapy” are being assigned to a particular text-block. For higher domain independence the method considers only unstructured (free)

From the software design point of view, K-Box has been conceived as a component-based software system, which adds new functionality dynamically. The development of the component infrastructure takes place in the project module we call the core. Beside the core we declared four further project modules, namely: components, web, workflow and client (figure 1). kbox.core is a J2EE application containing the essential SessionBeans, a WorkflowManager and a TransmissionManager for tracking and logging. It also includes the generic AbstractIC and AbstractEC classes, designed as abstract MessageDrivenBeans (MDB). Each MDB is bound to a JMS-Queue (Java Message Service). The queues enable asynchronous message exchange between the components. The internal message type used in the KBox is a SOAP message with attachments, a protocol related to the Web Services technology. Within the core we develop kbox.components for external connectivity (resource adaptors - J2EE Connector Architecture) and message transformation.

Figure 3 - Screenshot of kbox.web

The kbox.web GUI allows K-Box administrators to manage and configure the K-Box core. It also provides a uniform view of the components and their properties.

formatting information. For this method to be effective, the system has to be trained with typical data. During the training process the user marks up a few documents manually and tells the system to which categories the document contents should be mapped.

kbox.workflow is a stand-alone Java application for graphical design of workflow objects. The constructed workflow object can be saved locally or uploaded to the connected K-Box core by employing the remote WorkflowManager.

Document transformation consists of three steps: plain text extraction, text-block classification and classified block processing. In the first step control characters, tags and proprietary mark-up are removed from the document by a learning algorithm comparable to those used in today's email-spam-filters. The normalized document is then passed to the second step where it is divided into distinct text blocks. Each block is then automatically assigned with a certain content-category (e.g. „Diagnosis“), thus adding semantic information to the document. After classifying all blocks, the document can be transformed to the destination format, using the semantic information from the previous step. Further, some more tightly focussed postprocessing can be applied according to the content category. For example, a text-block classified as “Diagnosis” is likely to contain ICD-codes. These can be extracted automatically and converted to a highly structured, CDA Level 3 compliant document. The classification is based on NGRAM-Analysis, an adaptive classification algorithm which already proved to be effective in other areas of information extraction. [8]

Figure 2 - Screenshot of the graphical workflow designer kbox.client serves as an alternative K-Box adaptor, meant for external systems without a J2EE resource adaptor. kbox.client is a generic, extendable JMS client, capable of sending byte stream messages to a specific EC in a configured K-Box.

Discussion

Information extraction from unstructured and semistructured documents

The K-Box is an integration tool with the focus on automatic structuring of unstructured medical documents. While commercial EAI-products are naturally developing into powerful and universal integration suites, a simple interface engine presents in many use cases a more transparent and manageable solution. Compared to other

Discharge summaries in an unstructured form are unsuitable for further automatic processing. Provided there are enough document examples available, machine learning can be applied. One of K-Box’s transformation compo-

515

M. H. Doan et al. / K-Box: Automatic Structuring and Exchange of Medical Documents

branches, healthcare seems to adapt new IT trends rather reluctantly [9]. Replacing “interface engines” with more complex, business-process oriented EAI-tools is still at the beginning [10]. Straightforward and lightweight solutions might be particularly appreciated in smaller enterprises, with minimalist equipment and limited budget, such as medical practices. This is one of our target groups.

[2] Kaiser RH. Elektronische Patientenakten aus aerztlicher Sicht. In: Jäckel, eds. Telemedizinführer Deutschland. Bad Nauheim: 2006; pp. 184-91. [3] McIlroy D. “Mass Produced“ Software Components. In: Bauer FL, Bolliet L, Helms HJ. Software Engineering. Report on a conference sponsored by the NATO Science Committee; 1968 October 7-11; Garmisch, Germany: pp. 138-57. [4] Balzert H. Lehrbuch der Software-Technik – SoftwareEntwicklung. 2nd ed. Spektrum - Akademischer Verlag. 2000. [5] Gaizauskas R, Demetriou G, Artymiuk PJ, Willett P. Protein Structures and Information Extraction from Biological Texts: The PASTA System. Bioinformatics. 2003; 19 (1): pp. 135–43. [6] Kushmerick N. Finite-state approaches to Web information extraction. In: Pazienza MT, eds. Information Extraction in the Web Era: Natural Language Communication for Knowledge Acquisition and Intelligent Information Agents; Springer-Verlag; 2005: pp. 77-91. ISBN 3-540-40579-8. [7] Keller W. Enterprise Application Integration - Erfahrungen aus der Praxis; dpunkt verlag; 2002. ISBN 3-89864-186-4. [8] Cavnar WB, Trenkle JM. NGram Based Text Categorization. Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval; 11-13: April 1994; pp. 161-9. [9] Laroia A. Healthcare’s Digital Heartbeat. eAI journal. August 2002; pp. 44-7; [10] Goth G. Integration Engines - A future of opportunities. Healthcare Informatics. 2005/08; pp. 14-8. Available from: http://www.healthcare-informatics.com/issues/2005/08_05/ trends.htm [cited Jan 30, 2006]. [11] Henzinger MR, Motwani R, Silverstein C. Challenges in Web Search Engines; Proc. of the 18th International Joint Conference on Artificial Intelligence. 2003; pp. 1573-9. [12] Kuhlins S, Korthaus A. A Multithreaded Java Framework for Information Extraction in the Context of Enterprise Application Integration. Proceedings of the 1st international symposium on Information and communication technologies; 2003 September 24-26; Dublin, Irland. Dublin: Computer Science Press Trinity College; 2003. pp. 535-40.

The component-based design should increase acceptance of K-Box among potential developers. The final component specification must find a reasonable trade-off between flexibility and complexity. A component repository should serve as a platform for component exchange inside the user community. The basic prerequisite is a complete development framework which is the objective of our current development. Information extraction from websites [11], often wrapperoriented and combined with machine learning [6], has been widely explored and exploited. However, there have been significantly fewer attempts to utilize these methods in the Enterprise Application Integration [12] or specifically in messaging systems. The goal of our approach is to minimize the need for explicit message structure descriptions or formal transformation-rules. At present we are exploring methods for precise and domain-independent extraction of CDA header data. Incorporating domain specific information might increase the quality of the extraction, but has to be added in a flexible, replaceable way. The K-Box supports protocols and data formats that are usually used in healthcare. Such an emerging standard for inter-sectoral communication is the CDA. This format has been chosen for the nationwide health telematics platform in Germany. By means of the freetext-to-CDA transformation the K-Box provides the link between an isolated standard-unaware information system and the centralized infrastructure. To receive a field test experience with the architecture, a prototype of the K-Box has been deployed in one of the departments at the University Hospital of Muenster. The implemented workflow transmits CDA discharge summaries to external recipients via encrypted emails. A trial run is the next step.

Address for correspondence Prof. Dr. med. Frank Ueckert Phone: +49 251 83-52773 Email: [email protected] Minh H. Doan Phone: +49 251 83-58215 Email: [email protected] Institut fuer Medizinische Informatik und Biomathematik Domagkstraße 11 48161 Muenster Germany

References [1] Uslu A, Stausberg J. Nutzen und Kosten der Elektronischen Patientenakte. In: Jäckel, eds. Telemedizinführer Deutschland. Bad Nauheim: 2006; pp. 151-5.

516

Chapter 4. Medical Knowledge, Ontologies and Terminologies

517

This page intentionally left blank

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Using Distributional Analysis to Semantically Classify UMLS Concepts Jung-Wei Fana, Hua Xua, Carol Friedmana a

Department of Biomedical Informatics, Columbia University, USA

taining its descendents, such as necrotic melanoma or choroidal melanoma, diffuse.

Abstract The UMLS is a widely used and comprehensive knowledge source in the biomedical domain. It specifies biomedical concepts and their semantic categories, and therefore is valuable for Natural Language Processing (NLP) and other knowledge-based systems. However, the UMLS semantic classification is not always accurate, which adversely affects performance of these systems. Therefore, it is desirable to automatically validate, or, when necessary, to semantically reclassify UMLS concepts. We applied a distributional similarity method based on syntactic dependencies and -skew divergence to classify concepts in the T033 Finding class in order to determine which ones were biologic functions or disorders. A gold standard of 100 randomly sampled concepts was created that was based on a majority annotation of three experts. Precision of 0.54 and recall of 0.654 was achieved by the top prediction; precision of 0.64 and recall of 0.769 was achieved by the top 2 predictions. Error analysis revealed problems in the current method, and provided insight into future improvements.

There are many such structured knowledge bases in the biomedical domain, e.g. the SNOMED-CT [1], MeSH [2], and Gene Ontology [3]. The most comprehensive integration of these resources is the Unified Medical Language System (UMLS) [4], which includes 139 source vocabularies in the current release (2007AA). Each concept in the UMLS has a unique identifier, called CUI, which is assigned one or more semantic types in the Semantic Network (SN) [5]. The SN also contains relations defined between the semantic types, and at the concept level, there are also various relations inherited from the source vocabularies. The semantic classification of a CUI, when appropriate, is very valuable for NLP applications. Textual patterns can be recognized on the basis of UMLS relations and the semantic types of UMLS concepts identified in text, or developers can define their own applicationdependent relations based on UMLS semantic types. For example, the template [(Food) interacts with (Pharmacologic Substance)] can be used to screen potential fooddrug interactions in text. The SN has been used to determine relations between the concepts extracted through specified semantic patterns [6] and used as semantic filters in literature-based knowledge discovery [7]. In this paper we focus on semantic classification within the UMLS because it is a comprehensive system in the biomedical domain with continuous maintenance and a large user population.

Keywords: semantic classification, UMLS, distributional similarity, natural language processing

Introduction Ontological knowledge bases are important for developing Natural Language Processing (NLP) and other knowledgebased applications because they provide well-defined specifications of the concepts, semantic classification, conceptual normalization, and indicate relevant semantic relations between the concepts. The specifications help define the meaning and semantic category of each concept unambiguously whereas normalization helps unify synonymous surface terms to enhance retrieval. For example, blood platelets and thrombocytes should be normalized into a single concept. The relations defined in the ontology can be used in semantic parsing or pattern recognition as well in logical inference. For example, a semantic parsing rule that uses a broad semantic class, would be satisfied for concepts that are children of the broad class. The parent/ child relations can also be useful in question answering systems by providing knowledge of hierarchical relation and use of inheritance. More specifically, information about malignant melanoma can be implied from texts con-

Although the SN semantic types are a valuable resource, there are issues about their granularity and correctness. For example, the current release assigns C0021494 Injections, Intravenous to T169 Functional Concept, but it seems that it would be more appropriate if it were assigned T061 Thera peutic or Preventive Procedure or a broader class procedure for general NLP purposes. Research has been performed to coarsen the granularity of the SN, which involves combining SN types [8] [9] [10], and to audit SN errors [11] [12]. However, the above coarsening approaches only aggregate existing types into broader classes without considering the validity of SN assignments themselves, and the auditing approaches work by finding contradictions derived from the existing semantic type assignments. The drawback of relying on existing assignments is that inappropriate assignments are retained along with the high-level reorganization. For example, through inheritance Injections, Intravenous would still not be con-

519

J. Fan et al. / Using Distributional Analysis to Semantically Classify UMLS Concepts

sidered a procedure-related class by the above methods. On the other hand, we cannot assume that all concepts in the Functional Concept class belong to procedure, since the SN type also includes concepts such as Solar system, Poisonous effects, and Hairy.

generalized the approach to one that measures similarity of concepts instead of individual terms. This is accomplished by forming each concepts distribution with the syntactic dependencies of all its synonymous terms in a training corpus. When reclassifying concepts of well-defined SN types into seven broader classes, we achieved a precision of 0.802 for the top prediction and 0.884 when considering the top two predictions. An advantage of the corpus-based approach is that it is automated and therefore highthroughput. In addition, it is based on real language usage and thus tends to be more reproducible than experts using their own judgments, which may vary from person to person and thus consistency would be more difficult to achieve. Consistency using multiple experts could be obtained but would necessitate a costly process involving consensus.

We have observed that some SN types are more problematic than others because they are high level, vague, or heterogeneous. We consider that a SN class is vague when it includes many CUIs that are very semantically heterogeneous or inappropriate for that class. For example, T033 Finding contains many concepts of the disorder sense (e.g. Nasal cartilage loss and Hemianopsia), but also contains many different types of concepts that are not clinically relevant, such as Spiritual state alteration, or that are not well-defined, such as Wanted, which requires additional context to determine the correct meaning. For example, the online Free Dictionary has over nine definitions for wanted (http://www.thefreedic tionary.com/wanted) with examples including context for clarity. This causes some applications, such as [6], to avoid using the concepts in these classes, but then relevant concepts are also dropped. For example, Progressive renal failure and Acute inflammation are in T033, and would be missed by applications using only well-defined classes, such as T047 Disease or Syndrome. Therefore, it is important to develop an automated method for reclassifying UMLS concepts, particularly those that are assigned vague SN types, into more appropriate semantic classes.

In this paper we focus on evaluating the classification of CUIs for the SN type T033 Finding because it includes many important concepts associated with disorders and functional information, e.g. some disorder-related adjectives like Hypertensive are only covered by T033. Our approach differs from the related work in that we used a typical distributional similarity measure, -skew divergence, based on more abundant types of syntactic dependencies obtained from a huge shallow-parsed corpus to reclassify/audit UMLS concepts. It also differs from other UMLS concept classification methods because our approach automatically proposes the most appropriate semantic class for a CUI, regardless of the SN type(s) that were originally assigned.

In the biomedical domain, Sibanda et al. showed that syntactic features are especially useful for determining the semantic categories of terms and clauses in discharge summaries, using Support Vector Machine classifiers [13]. Pedersen et al. used window-based contextual distributions from the Mayo Clinic Corpus of Clinic Notes to measure the semantic similarity between SNOMED-CT concepts, and reported high correlation with human experts [14]. Weeds et al. used distributional similarity based on syntactic dependencies and a nearest neighbor voting process to classify terms into semantic types of the GENIA ontology and achieved good accuracy [15]. In a previous paper [16] we showed that it was feasible to apply a distributional similarity approach to classify UMLS concepts. The distributional approach is based on Harris sublanguage theory: terms are syntactically dependent on other terms with unequal likelihoods especially in specialized domains [17], so that terms can be characterized and classified through the distribution of their syntactic dependencies. For example, the adjective psychogenic is more likely to be an adjective of a noun that is a disorder and unlikely to modify a noun that is a microorganism. We used Lees -skew divergence [18] as the measure of distributional similarity:

Materials and methods The distributional classification method can be outlined as follows, and is elaborated on below: 1) determining clinically relevant semantic classes for building distributional profiles, 2) obtaining a training corpus, 3) extracting syntactic dependencies from the corpus for CUIs in the relevant classes, 4) using the syntactic dependencies to build distributional profiles for the classes, 5) using syntactic dependencies to build distributional profiles for the test CUIs, and 6) using -skew divergence to compute the distributional similarities between the CUI and class profiles for classification. Based on previous work [16] we grouped subsets of the SN types (2006AC) into seven clinically relevant broad classes: biologic function, anatomy (above the molecular level), disorder, gene or protein, microorganism, procedure, and substance. For example, Laboratory Procedure, Diagnostic Procedure, and Therapeutic or Preventive Procedure were grouped to form the procedure class. Vague types, such as Finding, Functional Concepts, and Health Care Activity were excluded from forming the seven classes. We obtained a corpus of 199K MetaMap-processed abstracts fr1om the 2005 MEDLINE/PubMed Baseline Repository (MBR) database [20]. Perl scripts

S  P  C t , P  C t 2 = D  P  C t 2 || P  C t 1 +  1- P  C t 2

(1)

where 0 1, t1, t2 are two terms, C is the union of the syntactic dependencies of t1, t2 , P(C|t1), P(C|t2) are the corresponding distributions, and D is KL divergence. We

1

520

MetaMap is a program that performs part of speech tagging, shallow parsing, and statistical methods to map terms in free text to the UMLS concepts. See [19].

J. Fan et al. / Using Distributional Analysis to Semantically Classify UMLS Concepts

were created to extract the part-of-speech (POS) tags, identify the phrase types, and trace the mapped CUIs to the original extractions from the machine-readable format of the requested corpus. For example, the machine-readable output of a sentence containing the concept Hemianopsia would be processed as shown in Figure 1 (simplified for clarity). Then we used a set of context-searching rules to extract syntactic dependencies for each concept. For example, in Figure 1 the syntactic dependency {associated noun: notch} can be obtained for Hemianopsia from the sentence The macular notch in hemianopsia was studied because the attached prepositional phrase in hemianopsia modifies notch. We aggregated the syntactic dependencies of all the CUIs in each class to build the distributional profile for that class. For a CUI to be classified, the syntactic dependencies of that CUI were used to build its distributional profile. Classification was performed by computing the distributional similarities between the test CUI and each of the seven classes, and the class with highest similarity score was selected.

randomly sampled 50 CUIs from the set that were classified as biologic function or disorder. Recall and precision by the top prediction and the top 2 predictions were both calculated, where the latter means the correct class was within the highest 2 classes in terms of the similarities ranking. The gold standard was generated by taking the majority judgments of three medical experts. They were given both the recall and precision test sets. For each set, the testing CUI and corresponding strings in the UMLS were displayed, and the experts were asked to annotate each concept as biologic function, disorder, or neither of the two classes. Kappa statistics [21] for the inter-annotator agreement were calculated. We also performed an error analysis on the recall and precision set respectively.

Results Exemplary concepts of varying frequencies in the corpus are displayed in Table 1; we found that frequency does not reflect the usefulness of concepts, because concepts that seemed useful or not useful were distributed similarly over different frequencies. There were 365 CUIs (for evaluating recall) with 10 syntactic dependencies extracted from the corpus, and 238 of them (for evaluating precision) were classified as biologic function or disorder. There were 26 of the 50 sampled CUIs in the recall set annotated as biologic function or disorder, and they served as the denominator in evaluating recall. The recall and precision of our classifier is shown in Table 2. The Kappa statistics of all the three annotators on the recall set was 0.421, and was 0.462 on the precision set.

S: The macular notch in hemianopsia was studies. P: The macular notch NP W: The det W: macular adj W: *notch noun P: in hemianopsia PP W: in prep W: *hemianopsia noun E: hemianopsia noun P: was AUX W: was aux P: studied. VP W: studied verb

Table 1 - Concepts with different frequencies in the corpus Frequency

Figure 1 - A sentence with POS tags, phrase types, and CUI mappings reconstructed from MetaMap output. Notations: S- sentence, P- phrase and phrase type, Wword and POS, asterisk marks the head noun, E- mapped term, concept term, and CUI We semantically classified CUIs in T033 to determine the concepts in that type that were biologic function or disorder, but not other types because those two classes constituted the two most clinically important concepts in that class. There are a total of 55,445 concepts in T033 (2006AC), and 848 of them occurred in the 199K training corpus. We tested our method under feature-sufficient condition (10 syntactic dependencies) to ensure that we would have the appropriate number of features, since it would be possible to obtain more syntactic dependencies by expanding the training corpus. As we were also interested in studying if concepts that rarely occur in a large body of literature were of less value for clinical applications, we qualitatively evaluated some T033 concepts of varying frequencies in the training corpus.

Examples

0 or 1

Abdominal swelling, Gingival bleeding, Partial deafness, Abnormal or prolonged prothrombin time, Birth Place, Vegetarian

5 and < 10

Panting, Hypoesthesia, Hyperlactatemia, Diaphragmatic paralysis, Vegan, Vagabond

10

Albuminuria, Sudden death, Fever, Cardiomegaly, Hematuria, HIV positive, Nervousness, Problem, Unemployment, Divorced

Table 2 - Recall and Precision of the classifier

To evaluate recall of our method in classifying T033 concepts into biologic function or disorder, we randomly sampled 50 CUIs from the set of CUIs that had 10 syntactic dependencies. For evaluating the precision, we

Recall

Precision

By top prediction

0.654

0.54

By top 2 predictions

0.769

0.64

We performed an error analysis based on the results by the top predictions. The errors in the precision set could be

521

J. Fan et al. / Using Distributional Analysis to Semantically Classify UMLS Concepts

grouped into two types: 1) irrelevant concepts were classified as biologic function or disorder, 2) biologic function and disorder concepts were confused with each other. The first group had errors for concepts such as Wanted, Optimistic, and Spells. Some examples of errors in the second group are: Low birth weights as biologic function, Normal vision as disorder, and Hyperalgia as biologic function. Errors in the recall set could also be grouped into two types: 1) errors due to confusing biologic function with disorder, 2) concepts belonging to the two classes but classified otherwise. In the first category, Stress was classified as biologic function, as were Low birth weights and Unresponsiveness. In the second group Does move and Assisted were both classified as procedure, while the gold standard had them as biologic function and disorder respectively.

of differences between our method and the gold standard are Apprehension and Patient noncompliance, which were classified as disorder, whereas Optimistic and Well-being were classified as biological function; however, in the gold standard these are neither of the two classes. The above examples show that our method is very sensitive, but not specific enough to rule out imprecise classifications. The third error category occurred when there was a combination of a modifier (usually a qualitative adjective, such as low or normal) and a clinical attribute, such as weight or blood pressure. These errors could result from the limited coverage of our classes. For example, there was not a class for findings specified as normal. It is also possible that the distributional approach per se is not adequate to differentiate that type of nuance, and lexical features from the concept strings may need to be introduced. Concepts in the fourth error category were misclassified based on contextual distributions but could have been classified more appropriately using morphologic information. For example, the suffixes -algia and -opsia frequently occur with disorders. Thus, from the results, it appears that, other features in addition to contexts, should be used for semantic classification because syntactic dependencies are not adequate in certain situations.

Discussion From Table 1 we determined that some frequent concepts may be too general to be informative (e.g. Problem), suggesting the need to filter out some very general and uninformative concepts even before performing classification. In contrast, lengthy terms like Abnormal or prolonged prothrombin time are less likely to occur, which is consistent with the estimation by [22] that only about 10% of the UMLS strings could be found in MEDLINE. The Kappa statistics over the three annotators indicated only moderate degree of agreement, which manifests the difficulty of the task. However, noticing that one annotator often checked neither of the two classes when the other two agreed on biologic function or disorder, we considered the gold standard generated by voting to be adequate overall but future evaluations should be performed using more experts. The recall and precision by the top 2 predictions show that the similarity function did rank the correct class higher out of the seven, but the performance was not satisfactory. We hypothesize that the T033 set, with more heterogeneous, high-level, and ambiguous concepts, poses a harder task for automatic classification than the other well-defined semantic types we tested (e.g. Disease or Syndrome and Organism Function).

There were also borderline cases where one of the annotators agreed with our classification. For example, we classified Resting state as biologic function and Atypia as disorder, and each agreed with one annotator. There were some ambiguous cases, which stemmed from the strings associated with the concept. For example, the concept Sweating, included the two strings Sweats, function and Sweating symptom NOS from the source vocabularies. The UMLS did not sharply differentiate the two different senses. However, our method obtained the latter sense, while the annotators agreed on the former. Table 3 - Categories of the classification errors Category

We summarize and provide examples of the main classification errors in Table 3. The first category contained concepts that are very general English terms, and that do not have a well-defined meaning without additional context. For example, Wanted was classified by our method as a biologic function; we believe this term alone is much too vague to be assigned as a concept. Another example was Spells, which is also very ambiguous, and thus, all three annotators marked it as being neither of the two classes. When examining the corpus, we found it was classified by our method as disorder because frequently the term Spells occurred in contexts such as apneic spells, cyanotic spells, or spells of respiratory distress. It is likely that the annotators would agree on the disorder sense, if given the corresponding contexts. Literally, Spells could also mean the biologic function of being able to spell. The second category characterized behavioral/affective observations and also included many general English terms. Some examples

Examples

1 Common English term

Does move, Spells, Wanted

2 Behavioral/ affective observation

Apprehension, Patient noncompliance, Optimistic, Well-being

3 Modifier + clinical attribute

Low birth weights, Blood pressure normal, Normal vision

4 Disorder by morphology

Hyperalgia, Hyperosmolarity, Hemianopsia

In summary, we believe that most of the errors resulted from three main limitations of our current approach: 1) the specificity issue associated with the first two categories of error indicates the lack of a global cutoff value and/or a

522

J. Fan et al. / Using Distributional Analysis to Semantically Classify UMLS Concepts

negative class distributional profile, thus resulting in many false positives, 2) the third error category suggests that the coverage of our existing classes is not adequate to appropriately accommodate some concepts such as neutral clinical findings, 3) errors in the third and fourth error categories show that using only the contextual syntactic dependencies could miss morphological and lexical cues that are essential to precise semantic classification, especially for concepts that are equivocal for multiple classes when given only the contexts. For example, in previous work we observed that biologic function and disorder were easily confused by our method, contributing to one third of the misclassifications because many pathologic functions were classified as normal biologic function. We believe this issue was exacerbated when the method was applied to T033 because it contained many poorly defined concepts.

[4] Lindberg DA, Humphreys BL, McCray AT. The Unified Medical Language System. Methods Inf Med. 1993;32(4):281-91. [5] McCray AT. An upper level ontology for the biomedical domain. Comp Funct Genom 2003;4:80-84. [6] Rindflesch TC, Fiszman M. The interaction of domain knowledge and linguistic structure in natural language processing: interpreting hypernymic propositions in biomedical text. J Biomed Inform. 2003;36(6):462-77. [7] Weeber M, Klein H, Aronson AR, Mork JG, de Jong-van den Berg LT, Vos R. Text-based discovery in biomedicine: the architecture of the DAD-system. Proc AMIA Symp. 2000;903-7. [8] McCray AT, Burgun A, Bodenreider O. Aggregating UMLS semantic types for reducing conceptural complexity. Medinfo. 2001;10(Pt 1):216-20. [9] Chen Z, Perl Y, Halper M, Geller J, Gu H. Partitioning the UMLS semantic network. IEEE Trans Inf Technol Biomed. 2002;6(2):102-8. [10] Zhang L, Perl Y, Halper M, Geller J, Hripcsak G. A lexical metaschema for the UMLS semantic network. Artif Intell Med. 2005;33(1):41-59. [11] Gu HH, Perl Y, Elhanan G, Min H, Zhang L, Perg Y. Auditing concept categorizations in the UMLS. Artif Intell Med. 2004;31:29-44. [12] Cimino JJ. Auditing the Unified Medical Language System with semantic methods. J Am Med Inform Assoc. 1998;5(1):41-51. [13] Sibanda T, He T, Szolovits P, Uzuner O. Syntacticallyinformed semantic category recognizer for discharge summaries. Proc AMIA Symp. 2006;714-8. [14] Pedersen T, Pakhomov SV, Patwardhan S, Chute CG. Measures of semantic similarity and relatedness in the biomedical domain. J Biomed Inform (e-pub ahead of print). Jun 10, 2006. DOI 10.1016/j.jbi.2006.06.004. [15] Weeds J, Dowdall J, Schneider G, Keller B, Weir D. Using distributional similarity to organize biomedical terminology. Terminology. 2005;11(1):107-41. [16] Fan JW, Friedman C. Semantic classification of biomedical concepts using distributional similarity. J Am Med Inform Assoc. (in press) [17] Harris ZS. A theory of language and information: a mathematical approach. New York: Oxford University Press, 1991. [18] Lee L. Measures of distributional similarity. Proc Annu Meet Assoc Comput Linguist. 1999;25-32. [19] Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp. 2001:17-21. [20] The MEDLINE/PubMed Baseline Repository (MBR): http:/ /mbr.nlm.nih.gov/ [21] Green AM. Kappa statistics for multiple raters using categorical classifications. Proc 22nd Annu Conf SAS Users Group. 1997;1110-5. [22] McCray AT, Bondenreider O, Malley JD, Browne AC. Evaluating UMLS strings for natural language processing. Proc AMIA Symp. 2001;448-52.

Although the performance of our automated method was lower than expected, we believe automatic classification is possible and highly desirable, and will explore adding additional features to our method as well as developing complementary methods and performing more extensive evaluation. More specifically, future work will focus on 1) filtering out concepts that are too general to be useful before and/or during the classification, 2) reconsidering the formation of the classes and possibly incorporating more classes, and 3) reducing the classification errors that occur between biologic function and disorder. After the implementation refinements, we would also like to reclassify concepts under other types such as T169 Functional Concept. For example, we have observed that some T169 concepts such as intramuscular injection were appropriately classified as procedure by our method.

Conclusion We used a distributional similarity approach to classify the concepts of the UMLS T033 Finding class into biologic function or disorder. The precision and recall were 0.54 and 0.654 by the top prediction, and 0.64 and 0.769 by the top 2 predictions. The error analysis indicated this task was intrinsically difficult when using only a distributional similarity measure and also helped identify the main factors hampering performance. We conclude that there is room to improve the method and it is definitely worth exploring further. Acknowledgements We thank Drs. Amy Chused, Daniel Stein, and Peter Hung for annotating the gold standard. This work was supported by Grants R01 LM7659 and R01 LM8635 from the National Library of Medicine.

References [1] College of American Pathologists. SNOMED Clinical Terms: http://www.snomed.org/snomedct/index.html [2] National Library of Medicine. Medical Subject Headings: http://www.nlm.nih.gov/mesh/ [3] The Gene Ontology Consortium. Gene Ontology: tool for the unification of biology. Nature Genet. 2000;25:25-9.

Address for correspondence Jung-Wei Fan, M.S., email: [email protected]

523

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Reappraisal of Sentence and Token Splitting for Life Sciences Documents Katrin Tomanek, Joachim Wermter, Udo Hahn Jena University Language and Information Engineering (JULIE) Lab, Friedrich-Schiller-Universität Jena, Germany

of these preprocessing steps will unavoidably be propagated upwards in the pipeline. As a consequence, text mining modules, such as (named) entity recognition and relation detection, fed by erroneous segmentation results will inevitably suffer in terms of performance.

Abstract Natural language processing of real-world documents requires several low-level tasks such as splitting a piece of text into its constituent sentences, and splitting each sentence into its constituent tokens to be performed by some preprocessor (prior to linguistic analysis). While this task is often considered as unsophisticated clerical work, in the life sciences domain it poses enormous problems due to complex naming conventions. In this paper, we first introduce an annotation framework for sentence and token splitting underlying a newly constructed sentence- and token-tagged biomedical text corpus. This corpus serves as a training environment and test bed for machinelearning based sentence and token splitters using Conditional Random Fields (CRFs). Our evaluation experiments reveal that CRFs with a rich feature set substantially increase sentence and token detection performance.

Rather than adding further complexity to manually maintained rule sets, supervised machine learning (ML) is becoming more and more the method of choice for many demanding natural language processing (NLP) tasks. From a given set of training examples a statistical model is learned automatically which is then used to assign labels to unseen data. Those approaches are to a large extent datadriven, i.e., by exchanging the training material they can be ported to other domains and languages, possibly without further changes. This constitutes a clear advantage over rule-based approaches which require a lot of manual tweaking and tuning, especially when more complex rules have to be supplied [1]. Furthermore, supervised ML approaches have been shown to outperform rule-based ones on several NLP tasks, including sentence boundary detection and tokenization, both in performance and breadth of coverage [2]. Therefore, we redesigned the entire pre-processing cycle and developed a new approach to sentence and token splitting based on Conditional Random Fields (CRFs), a sequential machine learning technique.

Keywords: biomedical text mining, natural language processing, linguistics, text processing.

Introduction Natural language processing systems expect their input to be properly delimited into consecutive sentences and sentences to be properly segmented into their basic tokens. Real-world documents lack such fine structure and, typically, some simple heuristic pre-processor performs these analysis steps. In the newspaper domain, manually defined patterns for sentence splitting rely on the fact that, e.g., a period is a sentence boundary, if it is followed by an uppercase letter and not part of a known abbreviation (“e.g.”, “Dr.”, etc.). Tokenization is often performed by splitting at a closed set of special characters (especially punctuation symbols, quotation marks, parentheses, brackets, etc.). While such rules may be adequate for the newspaper domain, they are underspecified for the biomedical domain, in which complex naming conventions lead, in particular, to quite unsatisfactory tokenization results when few and simple hand-written rules are applied. Here, both sentence and token boundary symbols are much more ambiguous because they often appear within entity names (organism, protein, cell names, etc.) and their abbreviations, as well as within (chemical) formulae, bibliographic references. If these challenges are not met, in an analysis pipeline for text mining each error occurring in the course

For sentence and token boundary detection, two annotated biomedical text corpora are available which may hold adequate training data for a supervised ML approach, viz. the GENIA corpus [3] and the PENNBIOIE corpus [4]. Whereas determining the sentence boundaries in a text corpus may be seen as a comparatively easy task, deciding on token boundaries is not as straightforward as it might appear at first sight. In particular, in biomedical language such as found in PubMed abstracts (or in full articles), crucial semantic units, such as entity names or even references to biological processes, can be contained within larger string units and hence are not simply delimited by white spaces. Thus, a closer look at both corpora reveals that only PENNBIOIE addresses the problem of semantically motivated word token boundary annotation at all, whereas GENIA annotates tokens around the same closed set of special characters used for English newspaper language [5]. In this paper, we report first on the compilation and annotation of the JULIE corpus which provides sentence and word token boundary information in a semantically moti-

524

K. Tomanek et al. / A Reappraisal of Sentence and Token Splitting for Life Sciences Documents

vated and linguistically feasible way. The JULIE corpus is composed of documents from a large variety of biomedical subdomains and entity types with critical word token issues. We then report on an ML-based tool suite for sentence and word token boundary detection, the JULIE Tools. We evaluate both the corpus and the tools against GENIA and PENNBIOIE and against another ML-based tool suite, the OpenNLP Tools.1

Token boundary annotation For the annotation of word tokens in biomedical text (or any other domain), it is essential to determine which word tokens denote semantic units (i.e., entities) of interest and thus should be recognized as such (e.g., by named entity recognizers). In biomedical text, there are various symbols which may (or may not) denote word token boundary symbols, such as “-”, “+”, “/”, “ ’ ”, “=”, “%” “(”, “)”, etc., and thus are very ambiguous with respect to their status as token boundary symbols (TBS). The most important cases are:

Background To train ML-based tools for sentence and/or token boundary detection, high-quality annotated text data resources are needed. In particular, several ambiguous non-alphanumeric character symbols may denote sentence or word token boundaries (or not) and, furthermore, these symbols may (or may not) be part of names for biomedical entities, such as protein, cell, or organism names etc. In the following, we outline the annotation guidelines considered for the compilation of the JULIE corpus which we annotated with sentence and token boundary information.

•

Parentheses must usually be split from regular words, but, e.g., in chemical terminology they are part of the name, such as in “Ca(2+)” or “(S,S)-Tartate”. The same holds for enumeration list items such as “1)”, “(2)”, “a)”, “b)”, etc.; • Plus (“+”) symbols and hyphens may denote relevant semantic information, such as indicating the presentation (+) of an antigen on a cell or the absence (-) thereof: “CD34(+) T-cells”, “CD83+ dendritic cells”, “CD11c(++) B-lymphocytes”, “CD8alpha(-) DCs”. Here, it can be seen that biomedical language expresses complex biological processes by means of a single character symbol. In such cases, of course, these symbols should be tokens on their own; • Hyphens often concatenate entity names (such as protein names) with other words (“IL-2-specific”,3 “CD28-dependent”, etc.) or even with other entity names, such as cell names, as in “CD43-DC”. Lacking recognition of these entities as word tokens would prevent entity recognizers from detecting them at all; • Similar observations can be made with respect to slashes (“/”) which often separate two (or more) entity references (“IL-2/CD34”, “HA-1/2”, etc.); • Hyphens (and slashes) may also denote the knock-out status of a certain gene with respect to an organism, such as in “flt3L-/- mice”. One could argue that such problems might be overcome by simply splitting (i.e., marking a sentence or token boundary symbol) at every potential split symbol (i.e., at every parenthesis, hyphen, period, colon, etc.). This strategy would split a protein name such as “IL-2” into [IL], [–], and [2]. However, modules further up in a typical text mining pipeline (part-of-speech taggers, phrase chunkers, syntactic parsers, etc.) would not be able to perform adequately on such broken data because their linguistic representations (either rule-based or derived from training data) could not deal with such fragments.

Sentence boundary annotation For sentence boundary annotation, it is first necessary to determine potential sentence boundary symbols (SBS). For biomedical language texts, such as those from the PubMed literature database,2 we defined the “classical” sentence boundary symbols (“.”, “!”, “?”, “:”) and also two PubMed-specific ones (“)”, “]” ). In particular, for periods (“.”) and colons (“:”), we encountered many cases where they did not denote an SBS: • • •

•

•

•

1 2

General abbreviations (“e.g.”, “i.e.”, “et al.”, “ref.”, “viz.”, “Dr.”, “vs.”, etc.); Numbers (0.05, 1.2, .4); Entity names, e.g., organism names (“E. coli”, “F. oxysporum”, “f. sp. Lycopersicim”, “P. decumbens”). Classifying these symbols as SBS would break up organism names which are essential for disambiguating protein names and mapping them into their database entry (e.g., UniProt). Obviously, simply stating a rule that marks an SBS after every period if the following word starts with a capital letter would break up the organism entity “f. sp. Lycopersicim”. Moreover, it would also fail to recognize an SBS in cases where the beginning of the following sentence starts with a lower-case letter, as is the case with many protein names (e.g. “p53”, “tac”, etc.); Author and journal names in literature citations which are contained in many Pubmed abstracts (e.g., “Am. J. Physiol.”, “J. Biol. Chem.”, “L. Hoffmann”, “Schindler L.”); Other alphanumeric strings, such as EC numbers, chromosome locations, database identifiers (“EC1.7.3.3”, “LEN.PK113-78”); Colons followed by enumerations (“Several cytokines interact with each other: IL-2, IL-5, and IL-18.”).

Materials and methods JULIE Corpus Currently, only one biomedical text corpus, viz. PENNBIOIE [4], addresses both sentence boundary and tokenization annotation issues at all, although insuffi-

http://opennlp.sourceforge.net http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?DB=pubmed

3

525

Here, only the second hyphen should be split because the first one is actually part of the protein name “IL-2”.

K. Tomanek et al. / A Reappraisal of Sentence and Token Splitting for Life Sciences Documents

ciently. Furthermore, it is limited to two highly specialized biomedical subdomains, viz. the CYP450 enzyme and oncology, focusing on few organism types (human and mouse). No manual token annotation was done on the second well-known annotated biomedical corpus, GENIA [3], whose domain scope is also rather limited (transcription factors in human blood cells). Hence, by using different sets of MeSH terms,4 we assembled a PubMed text corpus both for sentence and for word token annotation which covered a more varied set of biological subdomains, including gene expression and regulation, stem cell biology/transplanta-tion and immunology. We refer to this as the Subdomain Corpus. Additionally, by using the MeSH thesaurus, we also assembled subcorpora with respect to biomedical entity types which are known to pose severe problems for sentence and token boundary detection, viz. organisms, chemicals, cell types and cell components. We refer to this as the Entity Corpus.

consecutive processing steps tokens are considered as atomic units; and if these are too coarse information is lost. JULIE Sentence Boundary Detector (JSBD) The input text is broken down into a sequence of (observation) units by splitting at all white space positions. For each such unit our sentence splitter has to decide whether it is at the end of the sentence or not (binary classification). Thus, the following piece of text from a PubMed abstract would be split into the following units (with the actual SBS after “T-cells”): … [on] [IL-2-activated] [CD34(+)] [cytotoxic] [T-cells.] [p3hr-1,] [the] [Burkitt’s] [lymphoma] [cell] [line,] [was] Each unit is represented by the following features: • •

the unit itself (lexical feature) and its size in characters; sentence boundary symbols (SBS):7 whether unit ends with SBS, whether unit contains SBS; • brief word class: capital letters are replaced by “A”, lowercase letters by “a” etc. and then identical consecutive letters are collapsed (e.g. IL2 ψ AA0 ψ A0); • orthographical features: based on regular expressions (e.g. HasDash, AllCaps, InitalCap, hasParenthesis, ...); • abbreviations: whether the unit is contained in the list of known abbreviations, whether the unit conforms to abbreviation classes (“[A-Z].”, “([A-Za-z].)+”, “[a-z]+.”); • local context: features of neighboring units in window [-1,1] copied. JSBD also employs a rule-based post-processing routine to avoid that a sentence is split within opened parentheses or brackets. This is a useful extension, as scientific papers and abstracts often contain complex bibliographic references within parenthetical or bracket-like expressions. Such a reference should be considered as one sentence. However, within these parenthesized sentences there are often many SBSs, especially periods, which could also be considered as sentence boundaries. Preliminary experiments showed that this processing routine, though it does not improve the overall performance, shifts the FP/FN ratio favorably, i.e., false positives are avoided (clearly at the cost of some more false negatives).

The Subdomain and Entity Corpora for sentence boundary information were automatically preprocessed with the OpenNLP Sentence Splitter trained on the PENNBIOIE corpus [6]. Then the annotations were manually inspected and, if necessary, corrected by a computational linguist (the second author of this paper) and a biologist. In addition, we have added the sentence annotations from GENIA and PENNBIOIE. Altogether, the JULIE Sentence Corpus contains 62,400 sentences. A similar procedure was performed for token annotation. Due to the more complex task, only the Subdomain Corpus was annotated with word token boundary information. Furthermore, we corrected the token annotation in the PENNBIOIE corpus when it did not conform to our guidelines. The JULIE Token Corpus contains both the annotated Subdomain Corpus and the corrected PENNBIOIE data, summing up to around 35,900 sentences. Both corpora (sentence and token) are here referred to as the JULIE Corpus. ML-based JULIE tools Our tools for sentence splitting and tokenization are based on supervised machine learning, i.e., from a given set of training examples a statistical model is learned. Such a model can be used to predict labels for unseen data. Here, we employ Conditional Random Fields (CRF), a sequential learning approach which assigns a label sequence to an observation sequence [7]. This approach fits well the inherent sequential structure of natural language text. As an implementation of CRFs, we employed the machine learning toolkit MALLET.5 In the following, the sentence and the token splitter (referred to as JULIE Tools6) are explained in detail. As a general rule, we tried to optimize our tools not only in terms of accuracy, but also with respect to the kind of errors made, viz. false positive (FP) and false negative (FN) errors. As for the sentence splitter, we prefer FNs over FPs. As many consecutive NLP components work on the sentence-level the costs of erroneously splitting a sentence is higher than not splitting. For the tokenizer we favor FPs over FNs because in many 4 5 6

JULIE Token Boundary Detector (JTBD) For the tokenizer we have defined a set of critical token boundary symbols8 (TBS) where it should check for possible token boundaries. It should be noted that periods, exclamation and question marks are not considered as TBS because these SBS symbols have been disambiguated by the sentence splitter before. This is in line with the paradigm of running NLP tools in a (sequential) pipeline, where each tool is designated to one NLP task and there is a (natural) order to the tools due to input/output dependen-

http://www.nlm.nih.gov/mesh/meshhome.html http://mallet.cs.umass.edu/ The tools can be downloaded from http://www.julielab.de

7 8

526

Sentence boundary symbols are, e.g., period, question and exclamation mark, etc. Token boundary symbols: { } , + – ( ) [ ] ; = / < > % &

K. Tomanek et al. / A Reappraisal of Sentence and Token Splitting for Life Sciences Documents

cies. Thus, tokenization is typically performed after sentence splitting.

the biomedical domain has already been shown [6]. Among other NLP tools, OpenNLP also provides a sentence splitter and a tokenizer, both based on conditional maximum entropy models [8], also known as logistic regression. Compared to the rich feature sets of the JULIE Tools, the OpenNLP counterparts have substantially fewer features (mainly lexical ones) coupled with a non-sequential learning algorithm.

At all white space positions and at each TBS we split the sentence into single units, the TBS itself is treated as a separate unit. Our example piece of text from above would thus contain the following units (for each sentence fragment): [on] [IL] [–] [2] [–] [activated] [CD34] [(] [+] [)] [cytotoxic] [T] [–] [cells] SBS [p3hr] [–] [,] [the] [Burkitt] [’] [s] [lymphoma] [cell] [line] [,] [was] For each such unit, JTBD decides whether or not this unit is the end of a token. A token thus consists of a sequence of n units where the last unit was labeled as token end but the other n-1 units not.

Sentence splitter evaluation To address the first question, we trained the JULIE sentence boundary detector on both the GENIA (18,529 sentences with approximately 486,000 word tokens) and the original PENNBIOIE (23,277 sentences with approximately 590,000 word tokens) corpora. The models learned from this training material were then evaluated against the JULIE Subdomain and Entity Corpora. We did not evaluate against the complete JULIE Sentence Corpus because it comprises both the GENIA and the PENNBIOIE corpus. The results are depicted in Table 1. There are only very small differences in the accuracy of the two models (accuracy about A=99.6). In addition, we also trained and crossvalidated JSBD on the complete JULIE Sentence Corpus which yields an accuracy of A=99.8. Here, we encounter a small improvement over the last three experiments. Table 1 also shows the performance of OpenNLP's sentence splitter on the complete JULIE Sentence Corpus (10-fold cross-validation). In this setting, JSBD performed notably9 better than OpenNLP's sentence splitter (A=99.8 vs. A=98.7).

Furthermore, we assign each unit its so-called super-unit: therefore we also split the sentence into larger strings at each white space as we did for sentence splitting. Such a string is considered a super-unit for a unit if it covers this unit in the sentence. In our example sentence fragments, we would have the following super-units: [on] [IL-2-activated] [CD34(+)] [cytotoxic] [T-cells] SBS [p3hr-1] [,] [the] [Burkitt’s] [lymphoma] [cell] [line,] [was] The super-units are needed as context information when building the features. Each unit is then represented by the following features: • • • •

•

•

•

the unit itself, the super-unit itself (lexical feature); whether the unit had white space to the right in original sentence; whether the unit is a TBS; features equivalent to the sentence splitter: size, brief word class, abbreviation class, local context, rich orthographical features of the unit; bracket information: whether the super-unit contains brackets (hasOpeningBracketOnly, isInBrackets, hasClosingBracketOnly,...); whether the super-unit is an enumeration (“(1)”), whether the super-unit has genitive (“enzyme's”), whether the super-unit has plural in brackets (“enzyme(s)”); other orthographical features of the super-unit with focus on hyphens, arrows, +/– symbols which are often contained in biomedical texts, and structure of chemical names

Table 1 - Sentence splitting performance: evaluated on different tools and different corpora tool

training material

JSBD

GENIA

JSBD JSBD OpenNLP

evaluation material

JULIE Subdomain + Entity Corpus PENNBIOIE JULIE Subdomain + Entity Corpus 10-fold cross validation on JULIE Sentence Corpus 10-fold cross validation on JULIE Sentence Corpus

accuracy 99.58 (FP=30%) 99.62 (FP=27%) 99.8 (FP=30%) 98.7 (FP=48%)

This can be explained by the rich feature set in our sentence splitter and the fact that we consider sentence splitting as a sequential learning problem. Also, our sentence splitter produces fewer false positives (FP) than OpenNLP's sentence splitter (FP=30% vs. 48%), which is more favorable for this task.

Experimental results and discussion

Tokenizer evaluation

Sentence splitting and tokenization performance is typically evaluated in terms of the accuracy (A), i.e., the number of correct decisions divided by the total number of decisions being made. Here, the total number of decisions equals the number of units. In our evaluation, we focus on two question, viz. (1) how well are the different corpora suited for training and (2) whether the JULIE Tools perform better than another well-known ML-based NLP tool suite, the OpenNLP Tools, whose general applicability to

We trained JTBD on GENIA and PENNBIOIE, and evaluated the models on the JULIE Token Corpus to determine how well these corpora are suited for training (see Table 2). As GENIA is only tokenized using newspaper language patterns, it is not so well suited for training; only an accu9

527

On such a level of accuracy, a 1-percentage-point difference is notable because this affects mostly critical cases (organism names etc.).

K. Tomanek et al. / A Reappraisal of Sentence and Token Splitting for Life Sciences Documents

racy of about A=71.5% is reached. PENNBIOIE is more apt because its word token annotation is more semantically motivated. A 10-fold cross-validation of JTBD on the complete JULIE Token Corpus showed that the performance is thus improved by approximately 1 percentage point. Table 2 also indicates that in comparison with the 10-fold cross-validation of OpenNLP's tokenizer on the complete JULIE Token Corpus (A=95.0), both JTBD's machine learning algorithm (sequential learning) and its rich linguistic feature representation (super-units/units) are superior (A=96.7). Looking at the tokenization decisions, the OpenNLP tokenizer runs into particular problems with hyphens which either get split not at all or in a rather inconsistent way. Thus, an expression like “IL-2-activated” is sometimes tokenized as [IL-2][–activated], [IL][–2][–activated], or not at all. Similar errors occur with expressions such as “CD34(+)”.

cialized to a subdomain; a correction because we removed some inconsistencies, annotation errors, and extended the semantic motivation of word token annotation. Our evaluation experiments run on these corpora indicate that forboth the sentence splitting and the tokenization task, a substantial improvement in performance could be achieved. Compared to a maximum entropy approach with poor feature sets (OpenNLP), these results suggest a ML approach based on CRFs for such NLP preprocessing tasks. In the future, we will optimize our tools in terms of feature representation as well as extend the JULIE Token Corpus to include more critical entity types. Acknowledgements This research was funded by the Germany Ministry of Education and Research (BMBF) within the StemNet project (funding code: 01DS001A to 1C), and by the EU’s 6th Framework Programm (the BOOTStrep project under grant FP6-28099).

From the above experiments we conclude, that for sentence splitting the respective corpus used for training is not that critical because sentence annotations are not very controversial within different biomedical subdomains. Tokenization, however, is a much more complex task with respect to the relevant biomedical semantic units to be annotated. Still, performance of tokenization can be improved significantly by employing a tool with a linguistically adequate representation and a rich feature set. In both cases, the extension of an annotation corpus by a set of critical (and rare) subdomain and entity cases, as is done in the complete JULIE Corpus, boosts performance.

References [1] Ngai G, Yarowsky D. Rule Writing or Annotation: Costefficient Resource Usage for Base Noun Phrase Chunking. In: Proceedings of ACL-2000. [2] Palmer DD, Hearst MA. Adaptive Multilingual Sentence Boundary Disambiguation. In: Computational Linguistics. Computational Linguistics 1997: 23(2): 241-67. [3] Ohta T, Tateisi Y, Kim JD. The GENIA corpus: An annotated research abstract corpus in the molecular biology domain. In: Proceedings of the HLT 2002. [4] Bies A, Kulick S, Mandel M. Parallel Entity and Treebank Annotation. In: Proceedings of ACL workshop on “Frontiers in Corpus Annotation II: Pie in the Sky”, 2005. [5] Marcus MP, Santorini B, Marcinkiewicz MA Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 1993: 19(2): 313-30. [6] Buyko E, Wermter J, Poprat M, Hahn U. Automatically Adapting an NLP Core Engine to the Biology Domain. Proceedings of the BioLink & Bio-Ontologies SIG Meeting 2006, Brazil. [7] Lafferty J, McCallum A, Pereira F. Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning, 2001. [8] Ratnaparkhi A. Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania, 1998

Table 2 - Tokenizer performance: evaluated on different tools and different corpora tool

training material

evaluation material

accuracy

JTBD

GENIA

JTBD

PENNBIOIE

JULIE Token Corpus JULIE Token

71.5 (FP=3%) 95.9 (FP=25%) 96.7 (FP=45%) 95.0 (FP=47%)

JTBD Open NLP

Corpus 10-fold crossvalidation on JULIE Token Corpus 10-fold crossvalidation on JULIE Token Corpus

Conclusion We introduced a sentence splitter and a tokenizer based on sequential machine learning methods and rich feature sets as well as novel corpora for the biomedical domain to train these tools. These corpora are an extension and a correction of the PENNBIOIE corpus: an extension because we added training material so that this corpus covers a more complete section of the biomedical domain and is not spe-

528

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Corpus-based Error Detection in a Multilingual Medical Thesaurus Roosewelt L. Andradea,b, Edson Pachecoa,b, Pindaro S. Canciana,b, Percy Nohamaa,b, Stefan Schulzb,c a Paraná University of Technology (UTFPR), Curitiba, Brazil Pontifical Catholic University of Paraná (PUCPR), Curitiba, Brazil c Department of Medical Informatics, University Hospital, Freiburg, Germany b

novel terms, names, and acronyms being constantly created and English terminology increasingly permeating nonEnglish medical documents.

Abstract Cross-language document retrieval systems require support by some kind of multilingual thesaurus for semantically indexing documents in different languages. The peculiarities of the medical sublanguage, together with the subjectivism of lexicographers’ choices, complicates the thesaurus construction process. It furthermore requires a high degree of communication and interaction between the lexicographers involved. In order to detect errors, a systematic procedure is therefore necessary. We here describe a method which supports the maintenance of the multilingual medical subword repository of the MorphoSaurus system which assigns language-independent semantic identifiers to medical texts. Based on the assumption that the distribution of these semantic identifiers should be similar whenever comparing closely related texts in different languages, our approach identifies those semantic identifiers that vary most in distribution comparing language pairs. The revision of these identifiers and the lexical items related to them revealed multiple errors which were subsequently classified and fixed by the lexicographers. The overall quality improvement of the thesaurus was finally measured using the OHSUMED IR benchmark, resulting in a significant improvement of the retrieval quality for one of the languages tested.

In light of this scenario, document retrieval systems necessitate domain-specific thesauri [1]. We understand by multilingual thesaurus some organized repository of linguistic symbols that are mapped to language independent, concept-like descriptors tailored to the needs of a certain domain. Furthermore, a thesaurus usually provides additional semantic relations between these descriptors [2]. According to [3], the main rationale for the use of a thesaurus is the supply of a controlled reference vocabulary to represent the contents of documents and thus abstracting away from linguistic variation, for the sake of ameliorating and simplifying document retrieval. In contrast to automatically generated word indices, such as the ones maintained by current Web search engines, thesauri are generally constructed manually, in a labor intensive process that requires the grouping of synonyms and translations into semantic classes, the addition of semantic relations between classes and the resolution of lexical ambiguities. It is therefore self-evident that such a scenario, which normally involves a group of domain experts, requires some measures for error detection and quality assessment.

Keywords:

The objective of this work is to demonstrate how a quality control mechanism can be implemented for the maintenance of the MorphoSaurus dictionary, a multilingual thesaurus tailored to support document retrieval in clinical medicine [4]. To this end, we exploit test samples of multilingual corpora exhibiting a high degree of content similarity, in order to discover weaknesses in the thesaurus content by comparing the distribution patterns of MorphoSaurus descriptors in these samples.

controlled vocabulary, information storage and retrieval, quality control.

Introduction The medical language presents several challenges to information engineering due to its specialized terminology and the large amount of texts found in literature databases, on the Web, and in medical record systems. Furthermore there exists a considerable mismatch between the language of medical science, the jargon used by health professionals, and the language used by laypersons such as patients and their relatives. Multilingualism is another important issue here, because even though the global tendency is to use English as the primary language of research, the local idioms are still used for patient-related everyday documentation and communication. Furthermore, medical language is extremely dynamic with

Materials and methods Subwords and semantic indexing paradigm The construction of a thesaurus starts with the identification of the most informative terms (i.e., words and phrases) for the domain under scrutiny [5]. The main difference between MorphoSaurus and virtually any other thesaurus is that the inclusion of lexical entries into the repository is

529

R. Andrade et al. / Corpus-based Error Detection in a Multilingual Medical Thesaurus

cle={myo-, muscle, mio, muscul, …} and #pain={pain,-algy, -algi, -algia, dor, schmerz,… }. The reason for this is the need to deal with composed meanings even in cases where a compound word cannot be properly dissected. There are principally three approaches to treat ambiguous MIDs:

guided, as much as possible, by criteria of semantic atomicity. The main corollary of this decision has been the definition of a new kind of lexical item, the so-called subword entry. Subwords are defined as self-contained, semantically minimal units [6], since we assume that neither fully inflected nor automatically stemmed words constitute the appropriate granularity level for lexicalized content description. In the medical sublanguage we observe particularly complex words such as “pseudo|hypo|para|thyroid|ism”, “append|ectomy”, or “tooth|ache”. Such words are constituted, in our model, by two or more subwords (in here, e.g., “pseudo-”, “ectomy”, “tooth”). In the MorphoSaurus system, subwords are listed as lexicon entries together with their attributes such as language and subword type (i.e., stem, prefix, suffix, invariant). Each entry is assigned to exactly one identifier representing a group of synonyms and translations. These concept-like semantic descriptors are named MorphoSaurus identifiers (MIDs).

i. Substitute each ambiguous MID by the sequence of the MIDs representing its non-ambiguous senses. ii. Substitute the ambiguous MIDs by exactly one MID which represents its most likely sense, computed from the MID frequency distribution. iii. Substitute the ambiguous MIDs by exactly one MID representing its most likely sense, according to its context, using a corpus-derived co-occurrence matrix. Whereas (iii.) has already experimentally tested [7], it has not yet been integrated into the MorphoSaurus system. For information retrieval purposes we use solution (i.), aware of the decrease in precision it may bring about.

The MorphoSaurus engine indexes texts by mapping every word to a subword sequence, which is then checked for morphological plausibility, using a finite-state automaton. Thus, invalid segmentations such as ones without stems or ones beginning with a suffix are rejected. Finally, each meaning-bearing subword is replaced by its corresponding semantic identifier (MID). This in turn constitutes the interlingual output representation of the system. For example, #physioterap is the MID of the meaning of the subwords krankengymnastGE, physiotherapEN, fisioterapPT or fysioterapSW, extracted from text words such as “krankengymnastisch”, “physiotherapist”, or “fisioterapia”.

The MorphoSaurus subword lexicon currently contains 90,550 entries, with 22,561 for English, 23,976 for German, 14,984 for Portuguese, 10,936 for Spanish, 7,812 for French, and 10,281 for Swedish. All of these entries are related in the thesaurus by 21,432 equivalence classes Fig. 2 depicts the process of semantic indexing, performed by the MorphoSaurus system.

Figure 2 - Morpho-semantic normalization scheme Pragmatics of thesaurus maintenance An important advantage of constraining a dictionary to subword entries is the achievement of a high coverage with much less lexicon entries than in conventional lexicons. Nevertheless, the principal problems in the management of such a resource, as mentioned above, also exist with MorphoSaurus and require a principled solution. Although the lexicographers’ work is based on written guidelines, many situations still require common-sense decisions which tend to produce more or less arbitrary results. Here, a particular difficulty of a subword thesaurus turns out to be that not only decisions about the semantic

Figure 1 - Paradigmatic and syntagmatic relations: “has_sense” and “has_word_part” respectively MorphoSaurus provides two different kinds of semantic relations between MIDs (cf. Figure 1): •

•

has_sense: This paradigmatic relation links ambiguous MIDs to their respective senses, e.g., the MID #head is linked to the MIDs #caput and #boss by has_sense. has_word_part: This syntagmatic relation links an MID having a composed meaning to its parts, e.g., the MID #myalg={“myalg-”, “mialg-”,…} to both #mus-

530

R. Andrade et al. / Corpus-based Error Detection in a Multilingual Medical Thesaurus

with f1 being the MID frequencies in a normalized corpus of one language and f2 the frequencies in another language; Sd scores the degree of imbalance between the MID occurrences between either language, and Sa relates the frequency of the MID under scrutiny with the frequency of the MID with the highest frequency in both corpora. Thus the S values range between zero and one. The overall score is therefore predominantly influenced by the degree of imbalance, but also gives an additional boost to highly frequent MIDs.

relations between entries but also the proper delimitation of subwords have to be decided by the lexicographers. Corpus-based error detection approach We developed the following methodology to support the detection of weaknesses in the thesaurus. According to the current workflow, the lexicographers are already using a moderated mailing list in order to facilitate the communication of errors and to support consensus decisions of difficult modeling issues. However, we observed that this process is guided rather by serendipity than by systematic contemplations, although good responses can be found to any sort of posting. Our objective has been to improve this process by an automated detection of errors in the content of the thesaurus.

Redefinition of workflow Guided by the sequence of problematic MIDs in the generated frequency lists, the lexicographers then started revising the thesaurus and the modifications were put down in a computer-based journal containing the following information in a semi-structured form: MID, problem description, problem class, solution, and rationale for modification.

Our methodology is based on the hypothesis that in closely related corpora [8, 9] (i.e., texts that deal with the same subject-matter in different languages), the statistical distribution of semantic identifiers exhibits a high degree of similarity. In consequence, any exception to this expected conformity is likely to indicate some kind of fault in the indexing process, or a weakness in the semantic relations between entries in the lexicon.

Progress assessment There are two different ways to assess the progress achieved in the thesaurus cleansing: •

Formative Evaluation: MID frequency lists are periodically generated, expecting a decrease of indices; • Summative Evaluation: performance of a multilingual document retrieval system is measured, using the MorphoSaurus approach for indexing both documents and queries. We subscribed to the second approach which is expected to yield stronger conclusions for the usefulness of the proposed error detection methodology.

General proposal Our proposal aims at triggering the lexicographic activities with a ranked list of potential problems generated by the comparison of the “semantic extract” of comparable corpora, i.e., the MIDs extracted from them. In addition, we monitor this work via the repeated execution of a summative quality metric which uses an information retrieval benchmark that has already been applied in previous studies [10]. This benchmark measures the general appropriateness of a thesaurus to support medical text retrieval.

According to [11], we use precision and recall as performance parameters in an IR system. Precision is defined as the proportion of relevant documents among all retrieved documents and recall as the rate of all relevant documents which could be retrieved. In IR systems which return all documents and produce a ranked output it is possible to measure precision at different recall points, thus yielding a precision / recall diagram. By interpolation it is then possible to compute precision values at defined recall points. As an overall assessment parameter, we computed the eleven point average value (AvgP11), defined as the arithmetic mean of the precision values at eleven recall points 0.0, 0.1, …, 0.9, 1.0.

Multilingual related medical corpora In order to create frequency distributions between the MIDs, generated from related multilingual corpora from different languages, we used the Merck Sharp & Dohme (MSD) manual of clinical medicine, a reference handbook of clinical medicine, available for the languages English (EN), Spanish (SP), Portuguese (PT), and German (GE), and freely available from the website http:// www.merck.com. These corpora were submitted to the MorphoSaurus indexer and a frequency table was generated for each language. Scoring of descriptors

For the precision / recall benchmark we used the OHSUMED collection, a subset of Medline abstracts, manually classified with regard to their relevance to a given set of authentic user queries [12]. In order to use this resource for benchmarking in a cross-language retrieval system, all queries had previously been translated to Portuguese, Spanish, German, and Swedish.

For each MID in each language pair, an S value is calculated based on MID frequency distributions, using both relevance (Sa) and imbalance (Sd) measures: 2S d – s a S = ------------------3

(1)

f1 – f2 S d = ------------f1 + f2

(2)

f1 + f2 S a = -------------------------------- f x1 + f 2 max

(3)

531

R. Andrade et al. / Corpus-based Error Detection in a Multilingual Medical Thesaurus

Table 1 - A sample of MID frequencies and related parameters. Top of the list for the language pair English / Portuguese

Table 2 depicts the most frequent problems:

During the process of problem analysis and correction it became clear that most of the highly scored MIDs spotted real problems which could be solved. Table 1 shows three rather extreme examples of imbalance between Portuguese and English, due to missing MIDs in one of the two languages. For example, the preposition “from” belongs to a MID which is marked for indexing, but its Portuguese analogue “de” is marked as a stop word and is therefore ignored for indexing.

The ambiguity is mainly due to ambiguous lexemes (and, the according MIDs) in one language but not in another. In some cases the ambiguous MID was found not to be mapped to the unambiguous ones and was therefore used for indexing, since the normal procedure consisting of the substitution of an ambiguous MID by the MIDs that represent its non-ambiguous senses, did not take place. This is a problem that can easily be corrected by including the missing has_sense links. • Missing or dispensable MIDs were common in borderline cases where a lexical entry had a very unspecific sense resulting in the fact that an entry was given a semantic identifier in one language but not in another one. An example is the preposition “from” (see discussion above and Table 1). The solution is to create a consensus about what should be considered as stop (sub)words, i.e., lexicon entries excluded from indexing. • The same sense was found in different MIDs (which generally did not contain lexemes of all languages). This problem could be solved by merging different MIDs. • Different senses were found in the same MID, and at least one of them was also present in another MID (generally with a focus on a different language). The solution consisted in splitting the non-uniform MID and redistributing its entries. Surprisingly, other problems occurred only with quite low frequency. For instance, problems of string delimitation rarely had an impact on the MID distribution. Instead they seem to raise the overall noise level. So we conclude that other approaches may be better suited for detecting this kind of problem.

Table 2 - Problems identified during the MID corrections

Summative evaluation measures

MID

Meaning

EqClass

f1

F2

•

S

#people

People

500783

6352

0

0.7155

#from

From

060077

4676

0

0.7026

#icas

icas

023555

0

3022

0.6899

During the correction period (three months), ten thesaurus backups were produced. Each of these backups was used for a complete IR experiment with the OHSUMED corpus. Each of these experiments in turn produced an AvgP11 benchmark value for each of the four languages. To get a better visibility of the performance, we plot the average of Avg11P of all experiments in Fig. 2.

Results Experiences with the correction process

Reason for MID high score

Frequency Portuguese / English

Frequency German / English

Frequency Spanish / English

Ambiguities

0.23

0.38

0.14

Missing or dispensable MID

0.49

0.18

0.53

Same Sense in Different MIDs

0.06

0.12

0.19

One MID with Different Senses

0.04

0.05

0.06

No problem

0.11

0.10

0.04

Unclassified

0.07

0.17

0.04

Fig. 2 demonstrates the evolution of the changes undertaken, using the IR benchmark described above. The AvgP11 values were calculated at 10 points within an evaluation period of nine weeks. During this time, about hundred hours were invested by experienced lexicographers who had already been working in the MorphoSaurus project before.

Thes 2005/08/02 – 2005/10/08 Figure 2 - Average of eleven point average value (AvgP11) evolution

532

R. Andrade et al. / Corpus-based Error Detection in a Multilingual Medical Thesaurus

For none of the languages under scrutiny there was a monotonous increase in performance. Comparing the first with the last AvgP11 value, there is a relatively insignificant growth of the values for Portuguese and German, namely by 1.8% and 2.6% respectively. This improvement seemed to be principally due to the addition of relations between MIDs and the rearrangement of MIDs. We even found an IR performance decrease of 1.9% in the case of English. We could argue that this value, just as the increase for German and Portuguese, lies within the range of normal variation, especially considering that the benchmark does not measure the whole information space but exactly the IR performance of a sample of 106 queries. Certainly, the more consolidated a resource, the less likely simple modifications will make it better. In contrast, the increment in performance of the Spanish benchmark amounted to a factor of 53% which cannot be attributed to chance. We interpret this finding as a good support for the hypothesis that the right problem selection – as done by our error detection approach – increases the performance in very short time.

observed for those three languages which already exhibited a good quality after years of maintenance (i.e., German, English, Portuguese), there was a considerable increase for a fourth language, Spanish. This language, which had been added only much later, had therefore not received such a high level of attention due to the lack of Spanish language skills among the lexicographers. Hence we can recommend the methodology especially for streamlining the lexicographers’ efforts, particularly in the case of yet unconsolidated portions of a thesaurus. In addition to what we applied during this experiment we suggest the following elements to be included into a workflow for thesaurus quality assurance: •

• •

The different degrees of maturity between the languagespecific subsets of the thesaurus also became obvious when comparing the values from Table 2. The main difference is the relatively low rate of missing or unnecessary MIDs for German / English, a fact which may be derived from the maturity of the German part of the thesaurus on the one hand, but also from a more concordant treatment of stop words in this language pair.

• •

Another interesting fact was that 10% of the MID disparities could not be attached to any thesaurus error. We interpret this as a consequence of lexical ambiguities occurring in one language but not in the other. So it frequently happens that one ambiguous MID (representing ambiguous terms in the same language) expands to one MID that is very common in the domain, and another one that is uncommon. As we disambiguate trough employing the expected frequency (cf. ii. in the second section) one or more readings will be ignored. For instance, if the English noun “head” corresponds a bit more likely to the sense of “caput” than to the one of “boss”, the latter sense will simply be ignored, which in turn may cause a disparity of the MID corresponding to the sense of “boss”, compared to other languages.

For each language pair MID frequency, lists ordered by imbalance should be generated periodically, with a period covering one day to one week, according to the intensity of lexicon maintenance effort; An overall indicator for MID imbalance for each language pair should be generated and recorded over time; For each MID edited, the imbalance score S should be recorded over time; Alerts should be generated for every MID which exhibits an increase in imbalance above a given tolerance interval; IR benchmarks should be generated on a weekly basis and AvgP11 values monitored over time. Alerts could be generated in case of strong decrease in value, or cumulated decrease, considering both sequences of measurement for one language, or single-point measurement for all languages.

Acknowledgements This work was supported by the CNPq, Brazil, (project number 550830/05-7), and by German-Brazilian cooperation project DLR/PUCPR (BRA 03/013). We are further indebted to Kornél Markó, Philipp Daumke, and Jan Paetzold for the development and maintenance of the MorphoSaurus engine.

References [1] Schulz S, Hahn U. Morpheme-based, cross-lingual indexing for medical document retrieval. in: International Journal of Medical Informatics. 2000; 58-59: 87-99. [2] Huge G. Natural Language Information Retrieval. Combining corpus linguistics and human memory models for automatic term association (1999). 75-98. [3] Foskett, DJ. Thesaurus. Reading in Information Retrieval. Morgan Kaufmann, 1997; pp. 111-34. [4] Markó K, Schulz S, Hahn U: MorphoSaurus - Design and evaluation of an interlingua-based, cross-language document retrieval engine for the medical domain. Methods of Information in Medicine, 4/2005(44): 537-45. [5] Ribeira-Neto B, Baeza-Yates R. Information Retrieval Data Structures & Algorithms, 1st. ed. New Jersey, EUA: Prentice Hall, 1992. [6] Daumke P, Schulz S, Markó S. Subword approach for acquiring and cross-linking multilingual specialized lexicons. LREC 2006 Workshop: Acquiring and Representing Multilingual, Specialized Lexicons: the Case of Biomedicine. 2006; Genova, Italy. [7] Markó K, Schulz S, Hahn U (2005); Unsupervised multilingual word sense disambiguation via an interlingua.

Conclusion In this paper we presented an approach for data-driven error detection in the maintenance process of a multilingual thesaurus in the medical domain. The usefulness of the method could be demonstrated by the fact that most problems detected corresponded to real errors which could be fixed by the lexicographers. This showed the heuristic value of this approach in the process of continuous quality assurance and formative evaluation of the resource. For the summative evaluation of our error-detection approach we used an IR benchmark during a nine-week period of thesaurus maintenance work. Whereas no significant increase in the benchmark parameter could be

533

R. Andrade et al. / Corpus-based Error Detection in a Multilingual Medical Thesaurus Proceedings of the 20th National Conference on Artificial Intelligence (AAAI '05): 1075-1080. [8] Fung P. A statistical view of bilingual lexicon extraction: From parallel corpora to non-parallel corpora. In: Véronis J (Ed.). Parallel Text Processing; 2000 [9] Rapp R. Identifying word translations in nonparallel texts. Proceedings of the Meeting of the Association for Computational Linguistics; 1995; p. 320-322. [10] Hahn U, Markó K, Poprat M, Schulz S, Wermter J, Nohama P. Interlingual indexing across different languages. In: Proceedings of RIAO'04, 7th International Conference "Recherche d'Information Assisté par Ordinateur"; (RIAO'04): Avignon, France. 2004 p. 100-115.

[11] Ribeira-Neto B, Baeza-Yates R. Modern Information Retrieval. 1st. ed. Boston: Addison-Wesley Longman Publishing Co.; 1999 [12] Hersh WR. Information Retrieval – A Health Care Perspective. New York: Springer; 1996.

Address for correspondence Roosewelt Leite de Andrade, LER, Pontifical Catholic University of Paraná. Rua Imaculada Conceição, 1155 – CEP 80215-901, Curitiba, Brazil Email: [email protected]

534

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Defining Medical Words: Transposing Morphosemantic Analysis from French to English Louise Delégera, Fiammetta Namerb, Pierre Zweigenbaumc a

INSERM, UMR _S 872, Éq. 20, Les Cordeliers, Paris, F-75006 France; Université Pierre et Marie Curie-Paris 6, UMR_ S 872, Paris, F-75006 France; Université Paris Descartes, UMR _S 872, Paris, F-75006 France b ATILF and Université Nancy 2, CLSH, Nancy, F-54015 France c LIMSI, CNRS UPR3251, Orsay, F-91403 France; INALCO, CRIM, Paris, F-75007 France

instance the aim of [1], where Morfessor, a tool for unsupervised learning of morphological segmentation [2], was used to contribute mappings between WHO-ART and SNOMED terms. Early work on medical morphosemantic analysis focused on specific components such as –itis [3] or –osis [4], then on larger sets of neoclassical compounds [5]. Lovis [6] introduced the notion of morphosemantemes, i.e., units that cannot be further decomposed without losing their original meanings. The Morphosaurus system [7,8] segments complex words using a similar notion called subword. The UMLS Specialist Lexicon [9], with its “Lexical tools,” handles derived words, i.e., complex words built through the addition of prefixes or suffixes. It provides tables of neoclassical roots, but no analyzer to automatically decompose compound words. DériF [10] morphosemantically analyses complex words. In contrast to [8] or [6], it computes a hierarchical decomposition of complex words. Moreover, it produces a semantic definition of these words, which it can link to other words through a set of semantic relations including synonymy and hyponymy. In contrast to the Specialist tools or to [11], DériF handles both derived and compound words. Designed initially for French complex words, then extended to the medical domain, its potential for cross-linguistic application was showed in [12]. Its transposition to English would fill a gap in the set of tools currently available to process complex English medical words.

Abstract Medical language, as many technical languages, is rich with morphologically complex words, many of which take their roots in Greek and Latin—in which case they are called neoclassical compounds. Morphosemantic analysis can help generate definitions of such words. This paper reports work on the adaptation of a morphosemantic analyzer dedicated to French (DériF) to analyze English medical neoclassical compounds. It presents the principles of this transposition and its current performance. The analyzer was tested on a set of 1,299 compounds extracted from the WHO-ART terminology. 859 could be decomposed and defined, 675 of which successfully. An advantage of this process is that complex linguistic analyses designed for French could be successfully transferred to the analysis of English medical neoclassical compounds. Moreover, the resulting system can produce more complete analyses of English medical compounds than existing ones, including a hierarchical decomposition and semantic gloss of each word. Keywords: natural language processing, morphosemantic analysis, word definition, neoclassical compounds

Introduction Medical language, as many technical languages, is rich with morphologically complex words, many of which take their roots in Greek and Latin. These so-called neoclassical compounds are present in many areas of the medical vocabulary, including anatomy (gastrointestinal), diseases (encephalitis, cardiomyopathy), and procedures (gastrectomy). Segmenting morphologically complex words into their components is the task of morphological analysis. When this analysis is complemented by semantic interpretation, the process is called morphosemantic analysis. Complex words are often “compositional,” in the sense that the meaning of a complex word is often a combination of that of its parts.

This paper reports work on the adaptation of DériF to English medical complex words. It focuses on neoclassical compounds and is based on the hypothesis that this type of words is similarly formed in related European languages (here French and English). Our goal is to have DériF analyse English words and present its results in English. We first describe the morphosemantic analyzer and our test set of words. We explain the modifications performed on this tool and the evaluation conducted. We then expose the results, discuss the method and conclude with some perspectives.

Morphosemantic analysis can therefore help processes interested in semantics, such as the detection of similar terms or the generation of definitions. This was for Selected for best paper award.

535

L. Deléger et al. / Defining Medical Words: Transposing Morphosemantic Analysis from French to English

For instance, the French word acrodynie (English acrodynia) is analyzed in the following way (N stands for noun, N* is assigned to a noun CF):

Material and methods The principle on which this work is based is morphosemantic analysis, that is morphological analysis associated to a semantic interpretation of words. In other words, we want to obtain a decomposition and a description of the meaning of a complex word based on the meanings of its parts. A complex word may be formed through any combination of the following word formation rules:

acrodynie/N ==> (1) [ [ acr N* ] [ odyn N* ] ie N] (2) douleur (de—lié(e) à) articulation (pain of—linked to joint) (3) maladie (disease) (4) eql:acr/algie, eql:apex/algie, see:acr/ite, see:apex/ite

•

derivation, which adds an affix (prefix or suffix) to a base word, e.g., pain/painful; • compounding, which joins two (or more) components together, those components being either neo-classical roots called Combining Forms (CFs) or modernlanguage words, e.g., thermoregulation, arthritis. For this work we chose to analyze neo-classical compounds (formed from CFs). However, a compound may also undergo derivation, so that mixed-formation words must also be addressed. Therefore we included not only “pure” compounds, but also those neo-classical compounds that were prefixed or suffixed (e.g., haemorrhagic).

It has been pointed out [12] that the method could be extended to other languages. Indeed the rules for generating lexically related words (item 4.) do not rely on any language-specific features so that this part of the system is fully language-independent. The morphosemantic parser however needs to be adapted to the language. To test the transposition of DériF to English, we prepared a list of test words. These words were taken from the WHOART terminology, which describes Adverse Drug Reactions (ADRs), since one of the intended applications of this work is to contribute to the pharmacovigilance domain by grouping terms describing similar ADRs. We selected the English terms of this terminology; since DériF works on single words and not on multi-word units, we split them into single words; and since we adapted DériF to analyze neo-classical compounds, we only retained those types of words. The selection was done both automatically by removing all words of 4 characters or less (these words are practically never morphologically complex), and manually by reviewing the list to look for neoclassical compounds (the work was done by a language engineer, LD). This gave us a list of 1,299 words to be decomposed out of a total of 3,476 words. These words were lemmatized and tagged with their parts-of-speech using the Treetagger1 part-of-speech tagger. We used a lexicon of tagged words from the UMLS Specialist lexicon2 to help TreeTagger deal with unknown words.

Our main hypothesis for the transposition from French to English is that a same linguistic analysis can be applied to neo-classical compounds of several languages. We assume that they are formed in a similar way and that the components involved are the same, the major differences being orthographic (such as -algia/-algie). Material We started from the French version of the DériF (“Derivation in French”) morphosemantic analyzer. DériF was designed both for general language and more specialized vocabularies such as medical language. It performs an analysis purely based on linguistic methods and implements a number of decomposition rules and semantic interpretation templates. It also uses resources which include a lexicon of word lemmas tagged with their partsof-speech and a table of CFs. The system goes further than simple decomposition and interpretation steps by predicting lexically related words. Another of its distinctive features is that it yields a structured decomposition of words and not simply a linear segmentation, so that we know which part is the head of the word.

Methods Adapting the Morphosemantic Analyzer The method of morphological analysis that we want to transpose to English is schematized in figure 1.

As input the system expects a list of words tagged with their parts-of-speech and lemmatized in their base form 㩛 no plural). It outputs the following elements: 1. a structured decomposition of the word into its meaningful parts; 2. a definition (“gloss”) of the word in natural language; 3. a semantic category, inspired by the main MeSH tree descriptors (anatomy, organism, disease, etc.); 4. a set of potentially lexically related words. The relation identified can be an equivalence relation (eql), a hyponymy relation (isa) or a see-also relation (see).

1 2

536

http://www.ims.uni-stuttgart.de/projekte/corplex/ TreeTagger/DecisionTreeTagger.html (last access 26/03/07) http://www.nlm.nih.gov/pubs/factsheets/umlslex.html (last access 26/03/07)

L. Deléger et al. / Defining Medical Words: Transposing Morphosemantic Analysis from French to English

automatically matched them to our English CFs and reviewed the results. Those roots that could not be matched were dealt with manually. The set of related CFs was also replaced by their English equivalents (obtained from the orthographic modifications mentioned above). The parts-of-speech were kept as is. An extract of the resulting table can be seen in table 1. The lexical relations between the CFs are labeled as follows: C)

TT TC CC

-0.210 -6.959

-0.042 -0.682

0.966 0.496

10.946 2.663 17.070 2.218

0.008 0.027

CETP (C-629A)

AA CA CC

2.940 0.376 12.862 1.183

0.707 0.238

-7.498 0.716

-1.207 0.089

0.228 0.929

CETP (TAQ1B)

B1B1 B1B2 B2B2

-2.171 23.954 -1.176 10.742

0.031 0.241

7.722 7.093

0.909 0.917

0.364 0.360

CETP (I405V)

II IV VV

-6.239 -1.022 -2.337 19.038

0.308 0.020

1.666 2.686

0.310 0.370

0.757 0.712

The effect of biometric, environmental and genetic factors on LDL cholesterol by multiple regression analysis

AGT (M235T)

MM MT TT

32.421 2.478 6.612 1.256

0.014 0.211

-2.938 -0.915

-0.207 -0.259

0.795 0.836

Model F statistic

1.781

1.798

The effects of the independent variables to LDL cholesterol were tested (Table 1). In males, the overall model was significant (F=1.781, p = 0.032) and 12.5 percent of the variance of LDL cholesterol level is explained by independent variables (R2=0.125). Biometric and environmental factors had significant influence on LDL cholesterol in one variable, drinking status. Drinking status was significantly related to LDL cholesterol (beta= -13.635, t-value= -2.169 , p54.5 years old). This indicated that age was the most important determining factor for LDL cholesterol.

In females, the overall model was significant (F=1.798, p = 0.028) and 10.3 percent of the variance in LDL cholesterol is explained by independent variables (R2=0.103). Biometric and environmental factors had no significant influence on LDL cholesterol, but two genetic polymorphisms had significant influence on LDL cholesterol, APOE and APOA5(-1131T>C).

Differences were observed between the two subtrees. Older subjects (>54.5 years old) are subsequently split by drinking status. Younger subjects (54.5 years old) are subsequently split by age again, with tracts having an age between 31.5 and 54.5 years old in one node and an age less than 31.5 years old in the other. Subjects between 31.5 and 54.5 years of age had higher LDL cholesterol (138.04 mg/dl) than (112.04 mg/dl) subjects less than 31.5 years of age.

Decision tree and rules for the prediction of LDL cholesterol

I: Improvement (reduction of the within-node variance) M.L: Mean of LDL cholesterol

We used the CART algorithm to build a decision tree for predicting the LDL cholesterol, and identified the risk factors that play important roles in explaining LDL cholesterol for the two groups, male (Figure 2) and female (Figure 3). The decision tree shows the risk factors that

Decision trees are charts that illustrate decision rules. Decision rules provide specific information about risk factors based on the rule of induction. As shown in Figure 2, the decision tree has 20 leaf nodes and 11 terminal nodes. Figure 3 demonstrates a decision tree having 18 leaf nodes and 10 terminal nodes. Each terminal node depicted in the

572

S. Ho et al. / Risk Stratification for LDL Cholesterol using Induction Technique LDL-C (M.L = 130.51)

LDL-C (M.L = 131.10)

Age > 54.5

¡Â 54.5

( I = 48.36 )

(M.L = 119.70)

-1131T>C

(M.L = 134.99)

( I = 41.34 )

TT

Drinking

Age

( I = 44.26 )

( I = 49.48 )

Yes

No

(M.L=113.67)

(M.L = 144.77)

¡Â 31.5

> 31.5

(M.L = 112.04)

M235T

( I = 24.43 )

TT

MT or TT

(M.L = 123.12)

(M.L = 136.21)

BMI

(M.L = 89.26)

BMI ( I = 47.65 )

( I = 38.63 )

> 23.40

¡Â52.5

(M.L = 110.27)

(M.L = 163.40)

BMI

(M.L = 151.37)

Age

APOE

BMI

( I = 16.75 )

( I = 25.61 )

( I = 26.34 )

¡Â51.5

> 51.5

E2/E3 or E3/E3

E3/E4

(M.L = 123.05)

(M.L = 138.10)

(M.L = 126.82)

(M.L = 149.59)

> 25.01 (M.L = 144.80)

¡Â25.01 (M.L = 175.68)

Node 14

( I = 23.44 )

¡Â27.03 (M.L = 131.31)

> 24.29

(M.L = 130.73)

(M.L = 129.53)

Node 10

> 52.5

(M.L = 133.82)

¡Â24.29

> 22.62

(M.L = 116.38)

MM (M.L = 173.44)

Age

( I = 11.11 )

¡Â23.40

BMI ¡Â22.62

( I = 40.29 )

TC or CC (M.L = 102.70)

(M.L = 137.37)

( I = 19.90 )

(M.L = 138.04)

-1131T>C

TC or CC

(M.L = 124.51)

> 27.03 (M.L = 150.93)

I405V

Exercise

( I = 10.19)

( I = 15.79 )

Node 10

TAQ1B

Node 12

( I = 21.22 )

TAQ1B

I405V

( I = 20.78 )

( I = 16.21 )

VV

II or IV

(M.L = 118.19)

(M.L = 133.91)

B1B2 (M.L = 131.37)

B1B1 or B2B2

Node 20

II or VV

IV

(M.L = 114.87)

(M.L = 130.29)

Yes

No

B2B2

(M.L = 123.96) (M.L = 146.70)

(M.L = 120.46)

B1B1 or B1B2 (M.L = 151.52)

(M.L = 166.13)

Node 18

Figure 2- Male CART algorithm decision tree

Figure 3- Female CART algorithm decision tree

decision tree can be expressed in terms of an ‘if-then’ rule, as follows:

For the male group, the first node in the table, node 10, indicates the mean LDL cholesterol is 173.44 mg/dl for 7 subjects. For node 10 in males, the index score is 132.89%, indicating that the mean LDL cholesterol for this node is about 1.32 times higher than the overall sample.

For male, /* Node 10 */ If [( 31.5 < Age d 54.5 ) and ( AGT-M235T genotype = MM )], then the mean of LDL cholesterol = 173.44 mg/dl

Cumulative statistics demonstrate that cases with elevated LDL cholesterol can be derived by procuring the best segments of the sample. In males, if only the best node, (node 10), is reviewed, cases with LDL cholesterol means 1.32 times higher than the overall sample are reached by targeting only 3.06 % of the sample. If the next best node is included, (node 20), then cases with a 1.29 times higher LDL cholesterol mean than the overall sample are returned, from only 6.99 % of the sample.

Risk assessment and target segmentation for the management of LDL cholesterol The gains chart produced by the decision tree can be used for as risk stratification for LDL cholesterol management. There are two parts to the gains chart, node-by-node statistics and cumulative statistics (Table 2). In regards to the target variable LDL cholesterol, which is continuous, gain charts provide node statistics relative to the mean of the target variable.

Table 2 - Male risk stratification for LDL cholesterol by gains chart Node

Node-by-node

Node( n) 10 20 14 4 18 19 8 17 5 12 11

7 9 11 13 91 7 29 18 19 16 9

Node(%) 3.06 3.93 4.80 5.68 39.74 3.06 12.66 7.86 8.30 6.99 3.93

Gain 173.44 166.13 163.40 144.77 133.91 131.37 123.12 118.19 112.04 110.27 89.26

Cumulative

Index (%) 132.89 127.29 125.20 110.92 102.60 100.66 94.33 90.56 85.85 84.49 68.39

Node (n) 7 16 27 40 131 138 167 185 204 220 229

Node(%) 3.06 6.99 11.79 17.47 57.21 60.26 72.93 80.79 89.08 96.07 100.00

Gain 173.44 169.33 166.91 159.72 141.79 141.26 138.11 136.17 133.92 132.20 130.51

Index (%) 132.89 129.74 127.89 122.37 108.64 108.23 105.82 104.33 102.61 101.29 100.00

* Node (n) : Number of cases for the node Node (%) : Percentage of the total sample cases falling into the node Gain : Mean of LDL cholesterol for the node Index (%) : Ratio of the gain score for the node to the gain score for the total sample

573

S. Ho et al. / Risk Stratification for LDL Cholesterol using Induction Technique

This study examined characteristics of the CART algorithm to demonstrate how they can be used to predict LDL cholesterol risk and provide management information from a patient database. The CART algorithm provided cumulative statistics demonstrating how efficient high LDL cholesterol cases were obtained by taking the best segments of the sample. The gains chart also provided valuable information about which segments to target and which to avoid. In addition, we presented rules that provided an occurrence relationship among the risk factors. Such information, which could not be obtained from multiple regression, can be used in examining the effects of individual risk factors on a specific segment of the target population.

Comparative assessment of multiple regression and CART A comparison of performance for the two models is shown in Table 3. The CART algorithm performed better prediction (24.53%, 20.66%) than multiple regression(12.5%, 10.3%) for male and female groups, respectively. In the CART algorithm, error estimate is calculated as the withinnode variance about the mean of the node. The total variance equals the within-node variance plus the betweennode variance. The within-node variance of this model for the male group is 975.144, while the total variance is 1292.14. The proportion of variance due to error is 975.144 / 1292.14 = 0.7547. Thus, the proportion of variance explained by the model is 100% - 75.47% = 24.53%. The within-node variance of this model for the female group is 863.255, while the total variance is 1088.03. The proportion of variance due to error is 863.255 / 1088.03 = 0.7934. Thus, the proportion of variance explained by the model is 100% 79.34 % = 20.66%

There were several limitations in this study. One limitation is its reliance on data from a single hospital, and thus was biased in terms of the affluence of the subject group. In order to generalize data from a larger sample, partitioning data into training and testing set or cross validation is a pre-requisite or requirement. Sufficient validation for the generalization of our findings was not performed.

Table 3 - Performance Comparison of Multiple Regression and the CART Algorithm Model

Fitness

Male

Future analyses will include an improvement to the CART algorithm. Furthermore, cost-effectiveness information will be incorporated into a data mining algorithm for each of the risk factors in order to estimate budgets for providing LDL cholesterol management services to the specific target population. This model provides a comprehensive analytic framework to construct the optimal design of clinical guidelines and healthcare policy for the prevention and management of LDL cholesterol.

Female

Multiple Regression

F statistics P-value R2

1.781 0.032 0.125

1.798 0.028 0.103

CART

Total variance Within-node variance Proportion of variance explained by model

1292.140 975.144 24.53 %

1088.030 863.255 20.66 %

References [1] Arai Y, Hirose N, Yamamura K, Nakazawa S, Shimizu K, Deficiency of choresteryl ester transfer protein and gene polymorphisms of lipoprotein lipase and hepatic lipase are not associated with longevity. J Mol Med. 2003;81:102-9. [2] Evans D, Buchwald A, Beil FU. The single nucleotide polymorphism .1131T>C in the apolipoprotein A5 (APOA5) gene is associated with elevated triglycerides in patients with hyperlipidemia. J Mol Med. 2003;81:645-54. [3] Horÿ€´nek A, Vra´bl€´k M, Cÿesÿka R, Ada´mkova´ V, Poledne R, Hubacek JA. T-11310->C polymorphism within the apolipoprotein AV gene in hypertriglyceridemic individuals. Atherosclerosis. 2003;167:369-70. [4] Liu S, Ma J, Ridker PM, Breslow JL, Stampfer MJ. A prospective study of the association between APOE genotype and the risk of myocardial infarction among apparently healthy men. Atherosclerosis. 2003;166:323-9. [5] Chen Q, Reis SE, Kammerer CM, McNamara DM, Holubkov R, Sharaf BL, Sopko G, et al. APOE polymorphism and angiographic coronary artery disease severity in the Women’s Ischemia Syndrome Evaluation (WISE) study. Atherosclerosis. 2003;169:159-67. [6] Austin MA, Talmud PJ, Farin FM, Nickerson DA, Edwards KL et al. Association of apolipoprotein A5 variants with LDL particle size and triglyceride in Japanese Americans. Biochimica et Biophysica Acta. 2004;1688:1-9. [7] Bose I, Mahapatra RK. Business data mining: a machine learning perspective. Information & Management. 2001;39:211-25. [8] Friedewald WT, Levy RI, Fredrickson DS. Estimation of the concentration of low-density lipoprotein cholesterol in

Discussion and conclusion In this hospital-based cardiovascular genome study of Korean men and women, we found that CART is a better method to discover the knowledge to predict LDL cholesterol by biometric, environmental, and genetic factors compared with the regression method. Based on the CART method, we found that genetic polymorphisms APOA5, M235T, CETP(I405V), and CETP(TAQ1B) for male and APOA5, APOE, CETP(I405V), and CETP(TAQ1B) for female, were associated with LDL cholesterol. The genetic polymorphisms combined with other risk factors generated the rules for the prediction of LDL cholesterol. Although genetic factors have been extensively investigated as risk factors for CVD, the results are still debatable [1-6,11]. In this present study, overall results displayed association of risk factors with LDL cholesterol which were very similar in both the CART and regression methods. However, the CART method provided more detailed results according to each segmentation and subgroup.

574

S. Ho et al. / Risk Stratification for LDL Cholesterol using Induction Technique plasma, without use of the preparative ultracentrifuge. Clin Chem. 1972;18:499-502. [9] Michael JA, Gordon SL. Data mining technique: for marketing, sales and customer support. John Wiley & Sons ; 1997. [10] Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and regression trees. Wadsworth, Belmont ;1984. [11] Hubacek JA, Sÿkodova´ Z, Ada´ mkova´ V, La´ nska´ V, Poledne R. The influence of APOAV polymorphisms (T1131>C and S19>W) on plasma triglyceridelevels and risk of myocardial infarction. Clin Genet. 2004;65:126-30.

Address for correspondence Sun Ha Jee, Ph.D., MHS Department of Epidemiology and Health Promotion, Graduate School of Public Health, Yonsei University, Seoul, Korea. Tel: 82 2 2228 1523 Fax: 82 2 392 7734 E-mail: [email protected]

575

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Results from Data Mining in a Radiology Department: The Relevance of Data Quality Martin Langacd, Nanda Kirpekarb, Thomas Bürklec, Susanne Laumannd, Hans-Ulrich Prokoschc a

Workflow Research Group, Imaging Science Institute (ISI), Tübingen, Germany Department of Radiology, New York University Medical Center (NYUMC), New York, U.S.A. c Chair of Medical Informatics, University Erlangen-Nuremberg, Erlangen, Germany d Department Image and Knowledge Management, Siemens AG Medical Solutions, Erlangen, Germany b

demonstrate that reliability of workflow performance measurement heavily depends upon data quality.

Abstract This work is part of an ongoing effort to examine and improve clinical workflows in radiology. Classical workflow analysis is time consuming and expensive. Here we present a purely data-driven approach using data mining techniques to detect causes for poor data quality and areas with poor workflow performance. Data has been taken from a operational RIS system. We defined a set of four key indicators for both data quality and workflow performance. Using several mining techniques such as cluster analysis and correlation tests we were able to detect interesting effects regarding data quality and an abnormality in the workflow for some organizational units of the examined radiology departments. We conclude that data-driven data mining approaches may act as a valuable tool to support workflow analysis and can narrow down the problem space for a manual on-site workflow analysis. This can save time and effort and leads to less strain for clinicians and workflow analysts during interviews.

We define data quality according to Juran in [3], cited in [2],[4]: "Data is of high quality if it is fit for the intended uses [...] in operations, decision making and planning". Aspects of data quality comprise e.g. completeness, correctness, comprehensiveness, accessibility, timeliness, objectivity, relevancy, interpretability and understandability [4][5]. Data mining is the process of extracting knowledge from large amounts of data. Typical data mining functionalities comprise clustering, the discovery of concept/class descriptions, associations or correlations, classification, prediction, trend analysis, outliers analysis, deviation analysis, and similarity analysis [6].

Methods Our working environment is the radiology department of New York University Medical Center (NYUMC). We focused on outpatient examinations of 2 NYUMC radiology sites during the time period 02/2006 to 08/2006. Site A performs ultrasound (US), computer tomography (CT), and conventional X-ray (XR) studies in three departments. Site B is responsible for magnetic resonance imaging (MRI). Site A has three, site B one big central unit and both have several smaller so called satellite units either in different buildings/floors or even spread across the city. All units run one radiology information system (RIS). Our source database is derived from this RIS and covered more than 60.000 examinations which are characterized by 30 attributes each. We built a data mining infrastructure in alignment with the knowledge discovery in databases process described in [6] which comprised the sequence steps data pre-processing, key indicator calculation, data mining, and pattern evaluation.

Keywords: data mining, data quality, workflow, radiology

Introduction Improving workflows in clinical environments and especially the radiology workflow is gaining importance due to limited financial resources, increasing commercial pressure, changing hospital reimbursement e.g. in DRG based systems and an increasing requirement for quality management in medicine. Measuring the current workflow performance and its quality are key to any improvement efforts. “If you cannot measure it, you cannot control it. If you cannot control it, you cannot manage it. If you cannot manage it, you cannot improve it" [1]. However, as we will show, a reliable data basis is required to determine reliable performance indicators. Poor data can delay decision making, cause bad decisions, or prevent appropriate decision making [2]. Redman [2] estimates that poor data quality costs the typical company at least ten percent of revenue. Here we present a data-driven approach using data mining techniques to detect causes for poor data quality. We will

Data pre-processing The following action sequence was performed: •

576

The data was loaded from an existing RIS-fed Oracle database containing the raw data into a staging database (Microsoft Access 2003).

M. Lang et al. / Results from Data Mining in a Radiology Department: The Relevance of Data Quality

•

•

•

•

A pre-processing step then excluded extreme outliers and inconsistencies like dangling references, resulting in a pre-processed data base. E.g. we excluded examinations with a specific tag which did not refer to the examined body region The next step was data transformation (data normalization and aggregation) and data reformatting (e.g. changing date formats, etc.) We used a combination of SQL queries and VBA scripts to define the sequence of transformation and reformatting tasks. As an example we merged date and time values from two independent string type fields into one date type field. Finally data reduction (reduction of data volume without affecting the analytical results, like e.g. data discretization) was performed. We removed attributes such as patient identifiers or accession numbers. Furthermore we reduced the deviation from the preset duration of the examination from minutes to a value between [0;1], specifying the fraction of the preset in percent. Then we loaded the cleaned and transformed data into the project dedicated data warehouse (Oracle 10g) which itself is the actual data base for our mining efforts.

Our database, derived from the operational RIS, comprised optional data from manual entry such as patient arrival time, the begin of an examination, and the patient departure time. The end time of an examination is also manually entered, but unlike the previously mentioned timestamps it is required by the system. Report creation and report finalization timestamps are auto-generated by the RIS. Normally patient arrival (as well as patient departure) should be tracked by the front-desk personnel of the respective unit. The begin examination timestamps are maintained by radiology technologists. In detail KI.3 data completeness was calculated as the fraction of radiology examinations which do not violate one or more of the following constraints: C3.1: Patient arrival time is available C3.2: Begin of examination time is available C3.3: Patient departure time is available Data correctness was even more difficult to define. As a coarse correctness indicator we searched for studies, which contain obviously nonsense values for the timestamps. KI.4: data correctness was calculated as the fraction of items, which do violate none of the following check constraints: C4.1: time(Pat. Arrival) < *time(Exam Begin) C4.2: time(Exam Complete) < *time(Pat. Departure) C4.3: time(Pat. Arrival) z *time(Pat. Departure) C4.4: time(Exam Begin) z *time(Exam Complete) C4.5: 1min < duration(Exam) d 5ScheduledDuration This reads as follows: t1 < *t2 means t1 is earlier than t2r1min. t1 > *t2 means t1 is later than t2r1min. Equality of two timestamps t1=*t2 is defined if t1=t2r1min. Two timestamps t1, t2 are considered to be unequal (t1 z *t2) if t1 < t2 r1min. The following rule (C4.6) was retrospectively abandoned due to too many violations.

Calculation of key indicators (KIs) Our primary goal was to concentrate on both data quality and workflow performance using a mixed set of key indicators. For workflow performance we defined: KI.1: report-turnaround time How long does it take from completion of an examination to finalization of the corresponding medical report • KI.2: adherence to the pre-assigned duration How many minutes does the examination deviate from its scheduled duration. Alternatively the relative deviation in percent of the scheduled duration was used. It is noteworthy, that the reliability of the key performance indicators KI.1 and K.2 of a radiology department depend on the availability and correctness of a relatively small set of timestamps. These underlying timestamps are e.g. the exam begin and completion timestamps or the report finalization timestamps. In contrast to the correctness of these timestamps (data correctness indicator KI.4) their availability (data completeness indicator KI.3) has no direct influence on the reliability of the performance indicators KI.1 and KI.2. However, a high fraction of unavailable timestamps may lead to a biased view of the performance. To measure data quality we used the following key indicators:

•

•

KI.3: data completeness Which fraction of items has all relevant information available (no attribute values are missing or undefined)

•

KI.4: data correctness Which fraction of items has "correct data" for all relevant information (no contradictory facts in the data)

C4.6: time(Exam Completion) z *time(Pat. Departure)

Data mining Data mining was performed on the computed key indicators KI.1 to KI.4 and the following information: patient sex, patient age, examining site, examining unit, medical category, report transcription time, report signing time, referring physician, modality type, imaging device, technologist, and responsible radiologist. We used the WEKA Software (v 3.5.3) [7] and focused on cluster analysis, outlier analysis, and classification of findings. For cluster analysis, multiple algorithms were used, such as the Simple-EM algorithm, an expectation maximization algorithm described in [6] or the Farthest-First algorithm described in [8] and implemented in [7]. For outlier analysis we considered instances as outliers if they exceed the following range: xi < Q1  1.5 IQR or xi > Q3 + 1.5 IQR. IQR is the Inter-Quartile Range, Q1 is the first quartile, Q3 the third. In addition we used the F2-test to determine if data follows a specific distribution and for testing two or more groups regarding statistical independence.

577

M. Lang et al. / Results from Data Mining in a Radiology Department: The Relevance of Data Quality

Cluster classification was done primarily by generating C4.5 decision trees as described in [9],[7]. In addition, we relied on the 1R classifier which uses the minimum-error attribute for prediction and discretizing numeric attributes see [10], and a nearest-neighbour-like algorithm using non-nested generalized exemplars as described in [11],[7].

if <Site is a satellite> then else Using this rule 97.49% of the instances would be classified correctly. The same approach applied to Site B resulted in similar findings and exactly the same rule, which would classify 92.68% of Site B instances correctly based on their satellite or non-satellite status.

Pattern Evaluation Table 1: Fraction of a) correct and complete, b) incomplete (ƒKI.3), and c) incorrect (ƒKI.4) data items

The findings from the previous step must be evaluated and valuable results should be used for workflow improvement. We concentrated on the following parameters to distinguish interesting and less interesting findings [6]: Simplicity, Certainty, Utility, and Novelty. Simplicity gives information with regard to how easily a finding can be understood. It can be quantified e.g. by measuring an (association) rule length, or the size of the mined (decision) tree. Certainty provides information about the classification reliability, the accuracy, the rule strength or the discriminating weight. The utility measurement quantifies the potential usefulness of a finding and can be quantified by calculating the rule support or noise threshold. And finally, a finding is of interest if it was not previously known or surprising (Novelty).

a) KI.3 and KI.4

b) ƒKI.3

c) ƒKI.4

Dept.

nonSatellite nonSatellite nonSatellite Satellite Satellite Satellite

A1

74,36% 29,48% 20,46% 5,20%

6,19%

68,03%

A2

79,26% n/a

9,04%

n/a

A3

82,56% 45,21% 5,44%

B1

85,37% 79,55% 11,39% 15,53% 4,06%

11,91% n/a

16,05% 12,67% 41,13% 5,71%

As an intermediate result at this stage we can say that there is a difference between site A and site B with regard to the data quality and/or workflow performance in the examined time period. Furthermore within the sites there is a difference between satellite units and the central unit in both cases. Now we tried to work out which of our key indicators, workflow performance (KI.1, KI.2) or data quality (KI.3, KI.4) was responsible, starting with the data quality indicators and comparing satellite to non-satellite units for sites A and B respectively. An overview of the findings is given in Table 1.

In this project, a rough pre-selection of interesting findings was done, which then was presented to and discussed with the involved and/or affected clinicians. This sifting through the intermediate findings leads to further open questions, additional issues to be examined, and to new and deeper insights into detected problems. Finally, workflow improvements were derived from the findings in consultation with the affected participants and the corresponding supervisors.

The fraction of data items entered that do not violate one of the data completeness rules C3.1 to C3.3 or the data correctness rules C4.1 to C4.5 are shown in Table 1 column a). Table 1 column b) and c) give a more detailed view. Table 1 column b) depicts the fraction of data entries having at least one incomplete timestamp (ƒKI.3). Table 1 column c) (ƒKI.4) gives detail about the entries violating one of the data correctness rules of indicator KI.4. Department A2 does not have satellite units, but was listed for comprehensiveness reasons.

Results In our case the objective of data mining was to classify the data to detect outliers and to find patterns in the data, which describe the set of detected outliers. Therefore we started the mining process with the application of clustering algorithms on the whole data base, including performance and data quality indicators KI.1 to KI.4 to detect groups of data items with a high degree of similarity. In this first step we detected three different clusters. To gain more information about the characteristics of those clusters, we used the R1 classification given by [7] which came up with the following rules:

We found for example that in department A1 the amount of data entries with missing values (ƒKI.3) is much lower in satellite units (5,20%) compared to the central unit (20,46%). But as the indicator KI.4 shows, the majority of data entered at A1 satellite units is bogus (ƒKI.4 = 68,03%), whereas the bogus ratio (ƒKI.4) at the central unit is much lower (6,19%). Departments A3 and B1 show also a lower amount of correct entries (Table 1 column a) at satellite units compared to the central unit. But in contrast to the findings at department A1, the satellite units of A3 and B1 perform worse regarding both KI.3 and KI.4. Obviously, there are differences between satellite and nonsatellite units with regard to data quality.

if <Site=A> then and if <Site=B> then Using just these rules would classify 89.29% data items correctly to the respective site. The third cluster did not prompt such a clear rule. As a consequence there must be a difference of data entered at site A (CT, US, XR) vs. site B (MRI) in terms of data quality or workflow performance. We now looked at the two sites separately and continued the same clustering using performance and data quality indicators KI.1 to KI.4.

Based on these findings we focused on the detection of coherences between the detected data quality issues at satellite units and front desk staff, technologists, radiologists, devices, types of examinations, patient characteristics, etc.

For site A we could assign two clusters. Classifying these clusters derived the following rule:

578

M. Lang et al. / Results from Data Mining in a Radiology Department: The Relevance of Data Quality

using the a F2-test. We detected a highly significant (D < 0,1%) deviation from uniform distribution for both sites A and B and rejected the null hypothesis at a level of confidence of 99.9%. Comparable results were found when comparing correctness of the patient arrival timestamp (C4.1 and C4.3) and technologists.

Therefore we performed different classification analysis based on smaller and more condensed views on the original data. We computed a large set of different transient aggregations comparing satellite units with central units for site A and B respectively. These aggregated views comprised for example the averaging (avg) of: •

avg(ƒKI.3) and avg(ƒKI.4) per weekday, site, and dept.

Discussion

– avg(ƒC3.1) per weekday, site, and dept. – ... – avg(ƒC4.5) per weekday, site, and dept. • avg(ƒKI.3) and avg(ƒKI.4) per tech., site, and dept. – avg(ƒC3.1) per tech., site, and dept. – ... – avg(ƒC4.5) per tech., site, and dept. • avg(ƒKI.3) and avg(ƒKI.4) per physician, site, and dept. • ... The classification based on these condensed data revealed - among other findings - the following rule for site B, using the 1R classifier after [10].

Within this study we detected interesting relationships between data quality of timestamps within a hospital RIS system and staff involved in the examination procedure. This information is extremely valuable when considering workflow performance measured on the basis of those timestamps and gives some insight in the likely working procedures. For example, in the satellite radiology units it might often be the case that patient arrival and departure is not encoded by front desk staff of the respective unit. We assume that in such cases the technologist enters those timestamps himself to continue the procedure but does so incorrectly. So there seems to be a defect in the satellite workflow because technologists are forced to track patient arrival. This needs to be confirmed by local workflow analysis projects and will influence our ongoing efforts to measure and optimize workflow performance.

if avg(ƒC3.1 per tech, site, dept)

1qW

`

UW qW 1 , qW @bqW OW

n i 1

Similarly, for training the fuzzy HMM model, the fuzzy version of the EM algorithm can be derived, again by using the fuzzy coefficient that multiplies the state transition coefficients and summing up using the Choquet integral.

n

(2) with defined xi : 0 hx0 d hx1 d ... d hxn , and the discrete step P i j : ­P ^xi , xi 1 , x j ` ® ¯0

1

, (6) where U t i, j [ P t , j  P t , j ] / D t i . G t i is the degree of certainty for a single state sequence finishing at time t in a state Si.

¦ >hxi  hxi 1 @Pin , i 1

Pij

t

1

n i

For the case of profile HMMs, the integration is done over the states that are a discrete set. In such cases, the discrete Choquet integral [21] can be used: I

^

max S q bq – D qW

q1 ,q2 ,...,qt 1

if i d j

Using fuzzy profile HMMs in phylogenetic analysis

otherwise

After obtaining a multiple sequence alignment using the fuzzy Viterbi algorithm, the methodology of performing and evaluating phylogenetic analysis is depicted in Figure 2.

(3) The fuzzy profile HMM O A , B , S with N states S={S1,S2,…,SN} that can be observed through a space of observations : with observations O=O1O2…OT corresponding to unknown state sequences Q=q1,q2,…, qT can be fully defined by the matrices A , B and S , where A is the fuzzy state transition matrix, B is the fuzzy observation matrix and S is initial state fuzzy density. Two fuzzy variables xX={x1,x2,…, xN} and yY={y1,y2,…, yN} are used to represent the state at time t and t+1 [17]. In these terms, S S  A is the grade of certainty that the initial state is in A. Respectively, for X 0  X and Y0  Y , D Y  X 0 | Y0 is the grade of certainty that the state at time t+1 is in Y0, given that the previous state was X0. Concerning the observation space : 0  :, bi : 0 is the grade of certainty that the current observation is in :0, given a current state Si. After defining the model we are able to address the three HMM problems. Specifically, the problem of HMM evaluation can be solved using the fuzzy forward-backward algorithm. D i t is the grade of certainty of O=O1O2…OT and xi at time t. The initialization step is D1 i S i š bi O1 , while the induction step becomes: D t 1 i

Figure 2 - Phylogenetic analysis using fuzzy HMM alignments

N

¦ D >P t , j  P t , j @ š b O n i

ij

n i 1

i

First, alignments are used to compute phylogenetic trees with the maximum parsimony and maximum likelihood methods.

t 1

(4) where the sum is the discrete Choquet integral, the š operator stands for the fuzzy intersection operator, and P i j is defined in Equation 3. From the above equation, it is possible to observe that the assumption of independence of the observation until time t is not necessary anymore neither is necessary the knowledge of the next state. The answer to the evaluation problem for the forward and backward variables respectively is: i 1

N

The maximum parsimony method constructs trees on the basis of the minimum number of amino acid changes required to account for the data. This often results in the generation of hundreds of equally most parsimonious trees, making it difficult to justify the choice of a particular tree.

(5) In the fuzzy case, the grade of certainty for a sequence is used to score the model. The Choquet integral is computed over the states at each time t, where the integration step ( P t, j  P t, j ) becomes a value j at time t+1.

Similarly, for the maximum likelihood method, testing all possible trees is impossible, and it is also not computationally feasible to estimate the model for each tree. Therefore, the accepted strategy is to infer a “reasonable” tree topology with faster – although less reliable – reconstruction methods and use that tree to estimate the parameters.

Respectively, the fuzzy Viterbi algorithm, which is used for the alignment of new sequences to the model, uses the Choquet integral and multiplication for the fuzzy intersec-

By implementing a bootstrap analysis [22] of the produced trees though, it is possible to acquire a measure for the reliability of the alignments. Bootstrapping can approximate

PO | O

¦ D i , PO | O T

i 1

n i

N

¦ E 1 i bi O1 i 1

n i 1

1247

C. Collyda et al. / Enhancing the Quality of Phylogenetic Analysis using Fuzzy Hidden Markov Model Alignments

the underline distribution by resampling from the original dataset and constructing a majority-rule consensus tree. Bootstrapping values can then be used as a confidence measure for the quality assessment of the alignments. In the conducted experiments the abovementioned phylogenetic analysis schema was used for both classical profile HMM and fuzzy profile HMM alignments, and the confidence levels of the bootstrapping procedure were measured.

Results In order to evaluate the performance of the fuzzy profile HMM alignments we have performed several comparative tests against the classical approach. The dataset was randomly divided in a 90%-10% manner into training and test set for the HMM models. Specifically, 97 sequences were used for training the model, while 11 sequences were used to create the multiple sequence alignment and consequently construct their phylogenetic tree. The test sequences came equally from the E6 protein HPV family and the homologous proteins obtained from PSI-BLAST. The classical profile HMM was then trained in 20 cycles, while the fuzzy profile HMM required 12 cycles for training. Alignments were obtained using the fuzzy Viterbi algorithm. Part of the resulting alignments yielding from the fuzzy profile HMM is depicted in Figure 3.

Figure 4 - Best trees for fuzzy and simple HMM by maximum likelihood method as calculated by the Phylip Package

Figure 5 - Best trees for fuzzy and simple HMM by Maximum Parsimony method as calculated by the Phylip Package Table 1 - Phylogenetic evaluation scores for different models and algorithms Methods ML Simple

HMM -13298.89

FuzzyHMM -13317.902

Figure 3 - Part of alignment using fuzzy Profile HMM

MP Simple ML with Bootstrap

8507 -13671.514

8067 -13633.146

In consequence, phylogenetic trees were inferred using the maximum likelihood (ML) and the maximum parsimony (MP) methods. Bootstrapping was also applied for each case. Table 1 illustrates log likelihood scores for the two cases of ML applications, while also contains the MP scores for each case. Optimal trees for the bootstrapped cases can be observed in Figures 4 and 5.

MP with Bootstrap

8609

8347

Discussion By inspecting the inferred trees it is possible to observe that both models can discriminate between the E6 proteins and the rest of the homologous sequences. Concerning the evaluation scores, in the maximum-likelihood case, the tree topology, as well as the evaluation scores is similar for the two alignments in both simple and consensus trees. The maximum parsimony method for the fuzzy case yields models with lower values for both simple and bootstrapping methods, compared to the simple HMM alignment. This means that the fuzzy HMM can produce more parsimonious phylogenetic trees, a property that seems to come naturally from the non-independence assumption of the fuzzy profile HMM. Another issue worth pointing is the ability of the fuzzy HMM trainer to converge in less

The fuzzy profile HMM was implemented in Java, partially using the BioJava API, while phylogenetic analysis was performed with the PHYLIP package [23].

1248

C. Collyda et al. / Enhancing the Quality of Phylogenetic Analysis using Fuzzy Hidden Markov Model Alignments

cycles, thus building the model faster. In this sense, the computational cost for performing the training stage in the fuzzy case is reduced without sacrificing the quality of the alignments. Instead, the relaxation of the statistical independence assumption provides enhanced biological meaning to the alignments.

[10] Mulder NJ. InterPro. In: Encyclopedia of Genetics, Genomics, Proteomics and Bioinformatics, Jorde LB, Little PFR, Dunn MJ, Subramanian S, editors, John Wiley & Sons, 2435-2439; 2005. [11] Diplaris S, Tsoumakas G, Mitkas PA, Vlahavas I. Protein classification with multiple algorithms. In Proceedings of the Tenth Panhellenic Conference on Informatics 2005, Volos, Greece. Bozanis P, Houstis EN (Eds), Springer, Lecture Notes in Computer Science; 3746:448-456. [12] Collyda C, Maglaveras N, Pappas C. A Multiple Sequence Alignment Tool Using Hidden Markov Models For Phylogenetic Analysis. In Proceedings of the 4th European Symposium on Biomedical Engineering 2004; Patra, Greece. [13] Tran D, Wagner M. Generalised Fuzzy Hidden Markov Models for Speech Recognition. In Proceedings of the International Conference on Fuzzy Systems 2002; Calcutta, India. [14] Collyda C, Diplaris S, Mitkas PA, Maglaveras N, Pappas C. Fuzzy Hidden Markov Models: A New Approach In Multiple Sequence Alignment. 20th International Congress of the European Federation for Medical Informatics (MIE 2006), Maastricht, Netherlands. A. Hasman et al. (Eds.), IOS Press, Studies and Health Technology in Informatics; 124:99-104. [15] Bidargaddi NP, Chetty M, Kamruzzaman J. Fuzzy Profile Hidden Markov Models for Protein Sequence Analysis. Proceedings of the IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology 2005; San Diego, CA, USA. [16] Narukawa Y, Murofushi T, Sugeno T. Conditions for choquet integral representations. Proceedings of the Eighth International Fuzzy Systems Association World Congress 1999; 911–925. [17] Krogh A, Brown M, Mian IS, Sjolander K, Haussler D. Hidden Markov models in computational biology: Applications to protein modeling. Journal of Molecular Biology 1994; 235:1501-1531. [18] Rabiner A. A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE 1989; 77(2):257-285. [19] Wang Z, Klir G. Fuzzy measure theory. Plenum Press; 1992. [20] Grabisch M, Murofushi T, Sugeno M, editors. Fuzzy measures and integrals : theory and applications. New York: Springer; 2000. [21] Mohamed M, Gader P. Generalized hidden markov models. i. theoretical frameworks. IEEE Transactions on Fuzzy Systems 2000; 8(1):67–81. [22] Felsenstein J. Numerical methods for inferring evolutionary trees. Systematic Zoology 1982; 27:783-791. [23] Felsenstein J. PHYLIP: Phylogenetic Inference Package, Version 3.65. Seattle: Department of Genetics, University of Washington, 1993.

Conclusions In this paper we presented a new methodology of phylogenetic tree inference that makes use of the fuzzy profile HMM for multiple sequence aligning. The fuzzy approach relaxes the independence restriction implied in classical profile HMMs, thus providing more biologically meaningful alignments. In terms of phylogenetic analysis this implies the constructions of more parsimonious trees in comparison with the classic HMM approach. Finally, we have shown this property by experimenting with HPV virus protein data. Future work involves the application of fuzzy HMM alignments in the creation of a whole new series of profiles that can then be used in protein classification. Acknowledgments This work was supported by the EPEAEK II project: “IRAKLEITOS: Scholarships targeted in basic research. Phylogenetic analysis of biological data”.

References [1] Fitch WM. Toward defining the course of evolution: Minimum change for a specific tree topology. Systematic Zoology 1971; 20:406-416. [2] Felsenstein J. Evolutionary trees from DNA sequences: A maximum-likelihood approach. Journal of Molecular Evolution 1981; 17:368-376. [3] Page RDM, Holmes EC. Molecular Evolution: A Phylogenetic Approach. Oxford: Blackwell Science, 1998. [4] Salemi M, Vandamme AM, editors. The Phylogenetic Handbook. Cambridge University Press; 2003. [5] Thompson D, Higgins DG, Gibson TJ. Clustal W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. Nucleic Acids Res 1994; 22:4673-80. [6] Altschul SF, Madden TL, Schäffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Oxford d University Press Nucleic Acids Research 1997; 25(17):3389–3402. [7] Eddy SR. Profile Hidden Markov Models. Bioinformatics 1998; 14:755-763. [8] Pearson WR. Empirical statistical estimates for sequence similarity searches. In J. Mol. Biol. 1998; 276:71-84. [9] Bateman A, Coin L, Durbin R, Finn RD, Hollich V, Griffiths-Jones S, Khanna A, Marshall M, Moxon S, Sonnhammer ELL, Studholme DJ, Yeats C, Eddy SR. The Pfam protein families database. Oxford University Press, Nucleic Acids research 2002; 30(1):276-280.

Address for correspondence Lab of Medical Informatics, Faculty of Medicine, Aristotle University, GR54124, Thessaloniki, Greece

1249

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Determining Transcription Factor Activity from Microarray Data using Bayesian Markov Chain Monte Carlo Sampling Andrew V. Kossenkova, Aidan J. Petersonb, Michael F. Ochsc a

The Wistar Institute, Philadelphia, PA, USA Howard Hughes Medical Institute at the University of Minnesota, Minneapolis, MN, USA c The Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, MD, USA b

Abstract

that may function similarly to known genes [1] or biomarkers of disease [2], efforts have been made to use microarrays to link to transcription factor promoter sites [3].

Many biological processes rely on remodeling of the transcriptional response of cells through activation of transcription factors. Although determination of the activity level of transcription factors from microarray data can provide insight into developmental and disease processes, it requires careful analysis because of the multiple regulation of genes. We present a novel approach that handles both the assignment of genes to multiple patterns, as required by multiple regulation, and the linking of genes in prior probability distributions according to their known transcriptional regulators. We demonstrate the power of this approach in simulations and by application to yeast cell cycle and deletion mutant data. The results of simulations in the presence of increasing noise showed improved recovery of patterns in terms of F2 fit. Analysis of the yeast data led to improved inference of biologically meaningful groups in comparison to other techniques, as demonstrated with ROC analysis. The new algorithm provides an approach for estimating the levels of transcription factor activity from microarray data, and therefore provides insights into biological response.

These methods overlook a key aspect of transcriptional regulation, as they rely on clustering of genes into groups with each gene belonging to a single group. However, most, if not all, genes are likely to be multiply regulated, as evolution has been very effective in borrowing function by using existing genes in new roles. Even among genes regulated in the yeast cell cycle, only roughly 10% are associated with a single cell cycle phase [4]. This results in the identification of a large number of clusters with genes improperly grouped away from those involved in the same biological processes (e.g., in analyses of the yeast cell cycle data, a typical cluster analysis shows more than 20 clusters). This results in a significant loss of power for identification of TF activity. Although our knowledge of transcriptional regulation is growing rapidly, in a recent study using Agilent human arrays, only ~1000 genes out of 20,000 were annotated with high reliability as to their TFs using TRANSFAC professional [5]. In order to recover the signal of TF activity, it is therefore highly desirable to maximize the signal by correctly grouping genes into multiple overlapping groups.

Keywords: Microarray analysis, Bayesian analysis, transcription factors

The problem of multiple regulation was identified reasonably early in the development of microarray technology. The application of singular value decomposition (SVD) to microarray data [6] addressed multiple regulation, however the orthogonality constraints led to less than ideal results, since biological processes are not independent. We took a different approach, applying our Bayesian Decomposition algorithm to microarray analysis [7]. This algorithm, described below, uses a series of constraints and a structure minimization argument to identify overlapping sets of genes.

Introduction The regulation of gene expression is a primary form of response in all cellular systems. This response is typically mediated by activation of transcription factors or complexes (both referred to here as TFs) that can induce or repress transcription of sets of genes by binding to upstream elements known as promoters. Identification of the activity level of individual TFs provides insight into biological processes activated or deactivated in specific samples. For instance, identification of upregulation of the ELK–1 TF can indicate activation of the oncogenic RAS-RAF pathway in a tumor cell, which is difficult to measure directly.

A significant advantage to our approach is the ability to encode biological knowledge through prior probability distributions. In this work, we demonstrate how knowledge of coregulation through TFs can be encoded into the algorithm, leading to improved statistical power for the determination of the activity levels of biological processes.

Microarrays provide insight into the global transcriptional response of cells, which should be useful for identification of TF activity. Although early studies using microarrays focused on “guilt-by-association” identification of genes

1250

M. F. Ochs et al. / Determining Transcription Factor Activity from Microarray Data using Bayesian Markov Chain Monte Carlo Sampling

Methods In order to recover signatures of transcription factor activity, the analysis of microarray data will need to isolate patterns related to a biological process governed by a transcription factor, identify the genes associated with this pattern in the background of multiple regulation, and link these genes to transcriptional regulators. We will describe the Bayesian Decomposition (BD) algorithm in general, then show specifically the modifications that allow direct inference of transcriptional regulation to improve statistical power. Bayesian decomposition

Figure 1 - The decomposition performed by BD

The fundamental factorization needed to identify overlapping groups of coexpressed genes is the recovery of a distribution matrix (A) and a pattern matrix (P) that multiply together to form a mock (“fitted”) data matrix (M), which reproduces the data matrix (D) within the noise (). This relationship can be written as D

MH

AP  H .

The prior encoded by the algorithm has three important features: positivity, correlated structure, and minimization. Positivity is incorporated by the inclusion of a one-dimensional atomic domain for each matrix in the model (A and P). In this domain, point masses (i.e., atoms) are created according to a prior distribution that is exponential in amplitude and uniform in location. The creation or destruction of atoms does not modify the prior distribution, which allows an approach to minimization of structure, as discussed below.

(1)

For microarray data, D is generated from replicated experiments and therefore represents the best estimate for the expression of each gene in each condition. The computed matrices then provide the assignment of genes to patterns, A, and the assignment of conditions to patterns, P, as shown in Figure 1 for a hypothetical analysis of the cell cycle. In this example, the data is approximated by the multiplication of A and P, so that a gene (N) with complex behavior (transcribed strongly in G1 and weakly in G2), can have that behavior explained as a mixture of simpler behaviors (G1 and G2).

Correlations are introduced between points in the model (A and P matrices) by the mapping of atoms from the atomic domain to the matrices [8]. The mappings use kernel functions to spread the amplitude of each atom to one or more matrix elements, allowing linking of samples through correlations in P and linking of genes through correlations in A. We have used correlated structure in P to perform supervised learning [9], and here use correlations in A to provide prior information on TF regulation.

The factorization of D into A and P is generic, and as noted above, approaches using orthogonality criteria have been used. However, biological patterns will not be orthogonal, as this would imply independence. In fact, SVD applied to cell cycle data does not even readily recover phase signatures [6], while BD recovers signatures for the cell cycle phases as well as a signature for the entrained metabolic oscillator [7].

We use “birth and death” MCMC techniques for creating and destroying atoms. Since the prior distribution is unaffected by these actions, atoms can be eliminated readily, as long as the fit to the data is not adversely affected. This, coupled to internal mechanisms of amplitude exchange, leads to a minimization of structure (i.e., number of atoms). In the simplest application, this matches sparse matrix approaches.

Probability distributions

Markov chain Monte Carlo sampling

BD implements a Markov chain Monte Carlo (MCMC) approach in order to solve Equation 1. The Markov chain uses a Gibbs sampler requiring relative probability measures between points in the distribution of possible solutions. These are provided according to Bayes’ Equation, p(A, P|D)P p(D|A, P) p(A, P).

The Markov chain begins with empty atomic domains, and thus empty A and P matrices. The algorithm attempts to birth atoms (created ex vacuo), move or exchange amplitude between atoms, and remove atoms in separate MCMC steps. Atoms are created according to the prior distribution and mapped through the kernel functions to the A and P matrices. The log likelihood is

(2)

The posterior probability, p(A,P|D), describes the probability of a model (A and P) given the data, and it is the distribution sampled by MCMC. The prior, p(A,P), provides the probability of the model independent of the data. A simple example is that a model with negative copies of mRNA can be ruled out a priori and has zero prior probability. The likelihood, p(D|A,P), gives the probability that the data comes from the model, and this probability is related to the F2 distribution.

log L =

­ 1

½

-D – A P , ¦ i¦ j®¯ --------2V ij ij ¦ ip pj ¾¿

(3)

so that changes in the likelihood can be easily calculated for any change in the matrices. The algorithm calculates this change in such a way as to allow resampling of amplitude to increase the speed of exploration of the posterior distribution.

1251

M. F. Ochs et al. / Determining Transcription Factor Activity from Microarray Data using Bayesian Markov Chain Monte Carlo Sampling

tern related to that TF. The dot product of the corresponding pattern, Pt*, with each row of Dt provides the weights, wt, for spreading an atom linked to this TF into the A matrix.

After an equilibration period determined by the user, sampling of the distribution is done by recording atoms and mapping them to the A and P matrices. Statistical measures, such as the mean and standard deviation for each matrix element, can be calculated. The rows of the inferred P matrix (patterns) link conditions, while the columns of the inferred A matrix assign genes to each pattern. Convergence is checked by insuring that the F2 fit to the data is stable, and that multiple chains starting at random points reach the same solution.

Figure 3 - Atomic domains and prior information Sampling with prior information We encoded TF information within BD by dividing the atomic domain related to the A matrix into two subdomains (see Figure 3). Atoms in the left half of the atomic domain spread to multiple elements in A through kernel functions, Kw, using the weight vectors, wt. The location of the atom determines the column of A and the weight vector wt that are used. An atom in the right half maps to a single element, allowing prior information not supported by the data to be ignored.

Figure 2 – Calculating weights for TF analysis Estimating transcription factor activity

As with BD, the new algorithm, BD-TF, starts with empty A and P matrices. As atoms are created, moved, or destroyed, changes are mapped to the matrices according to the scheme shown in Figure 3. If there is correlated structure in the data due to TF activity, a single atom will successfully recover this, and statistical power will be gained by use of Kw. Since the prior is defined on the atomic domain, the form is unchanged from the original BD algorithm. The calculated change in the log likelihood is affected by the correlation functions, Kw, but the likelihood function in Equation 3 includes summation across all matrix elements, so no change of form is needed. The equilibration and sampling proceed as in the original BD.

Previously we estimated TF activity without using prior information successfully for S. cerevisiae. We applied the original BD and subsequently linked genes associated with patterns to TFs [10]. However, for mammalian systems, a more statistically powerful approach will be required. The approach we propose here is to encode the knowledge of transcriptional regulation a priori during Markov chain sampling, thus borrowing power across genes. Normalizing expression for each transcription factor In order to map an atom to multiple matrix elements, it is necessary to determine the amount of the amplitude that should be devoted to each gene. This mapping cannot be made uniform across the matrix elements, as the overall copy number of mRNA produced will vary by gene and by biological process.

Analysis We have analyzed three separate sets of data and compared our BD-TF results to BD and to some standard techniques. We first used simulations of the yeast cell cycle, which allowed us to increase noise levels to determine the behavior of the algorithm across many levels of noise. Second, we analyzed the widely studied yeast cell cycle data set [1], permitting comparison with other methods. Third, we analyzed the Rosetta compendium of yeast deletion mutants, which included error estimates for all data points [11]. An open problem in the field is the correct dimensionality for analysis (i.e., number of patterns). We have estimated this previously by multiple methods for the data used here, while, for the simulated data, we use the simulated dimensionality.

We address this issue by implementing a preprocessing step to determine weight vectors for each gene linked through a TF. The overall approach is shown in Figure 2. The data, D, are divided into T overlapping subsets, with each subset containing all genes regulated a priori by a given TF. Each subset, Dt, is analyzed using the original BD without a priori correlations and positing P+1 patterns, where P is the total number of groups in T that contain any gene in Dt. This provides for a pattern for each TF and a pattern for routine metabolic function, which BD typically isolates separately. The rows of the At matrices are normalized to unit amplitude, and the column from each with the lowest variance is taken to represent the pat-

1252

M. F. Ochs et al. / Determining Transcription Factor Activity from Microarray Data using Bayesian Markov Chain Monte Carlo Sampling

For the simulated yeast cell cycle data, we created expression levels for 288 genes at 48 time points over two cycles. The P matrix comprised 5 overlapping patterns - four that reflect cell cycle phases over two periods and one representing a metabolic oscillator with an amplitude 5% as large as the cell cycle pattern amplitudes and with twice the frequency [7]. The A matrix assigned genes to patterns in the expression profile, with most genes assigned to 2 – 4 patterns, reflecting the fact that in yeast only ~10% of cell cycle genes belong to a single phase [4]. Noise was added to the data matrix, including different levels of additive and multiplicative noise, using the widely accepted noise model [12] D

N 0, V a eN 0,V m AS PS ,

experimental evidence. The set included the regulators Zap1, Ndt80, Mcm1, Gcn4, Dal80, Rtg1, Pdr1, Met4, Ume6, Ste12, Mot3, Gln3, Cbf1, Mig1, Rlm1, Msn4, and Msn2 with 5 – 19 target genes each. ROC analysis looked for recovery of the correlated groups. While this is somewhat circular, it indicated whether the prior information was being used appropriately by the algorithm. Unfortunately, the genes with known coregulation from cell cycle studies [13] were not in this data set, as they do not vary across the deletion mutants.

Results We present our results in three sections: 1) simulations of cell cycle data, 2) analysis of the Cho data with ROC analysis, and 3) analysis of the Rosetta data with ROC analysis.

(4)

where AS and PS are simulated amplitude and pattern matrices, and Va and Vm are additive and multiplicative levels of noise respectively. Simulated data matrices with 154 different noise levels were created, varyingVa from 0 to 6.5 and Vm from 0 to 3. The data matrix without noise had maximum amplitude 3.15 and mean amplitude 0.65. We simulated 4 replicate arrays and calculated mean and standard deviations for each simulated element in each D.

Figure 4 – F2 fits for simulation data

The Cho data was analyzed using BD, as described previously [7], and using BD-TF. Groups of coregulated yeast genes were assembled based on literature reports of regulatory relationships between TFs and target genes, and only coregulation groups with at least five members were retained. In addition, we sought to enrich for true TF-target relationships by including only genes supported by evidence both of TF binding and of alteration in transcription when the TF was absent or overexpressed. For the cell cycle analysis, the regulators Mot3, Ndt80, Ste12, Swi5, Cbf1, Fkh1, Fkh2, Swi6, Mcm1, Swi4, and Rlm1 provided prior information with 5 – 16 target genes each.

Simulations In Figure 4, we show the log2 ratios of the F2 fits to the AS matrix between BD-TF and BD. The heatmaps show the differences in the fits across different levels of noise and inclusion of prior information. In the figures, multiplicative noise increases to the right, from 0 to 48% of the peak signal; additive noise to the top, from 0 to 79% of peak signal; and the amount of prior information included (number of genes included from coregulation lists) increases as indicated by the percentages shown above each heatmap. Only the 36 simulations with the lowest noise levels are shown, as these cover levels of noise exceeding those in typical array experiments. Gray squares represent a neutral result (no improvement), while lighter squares represent improved fits with BD-TF and darker squares poorer fits. The advantage of using coregulation information increases as levels of prior information increase, as would be expected if the coregulation information is improving statistical inference by gaining power across genes.

In our previous work [7], we established that the best factorization used 6 dimensions (5 cell cycle phases, due to early and late G1 signatures as seen previously [4], and a metabolic oscillator signature). An ROC analysis was performed for BD by increasing the stringency of assignment of a gene to a pattern. Essentially, each gene had a mean value of its strength within a pattern and an uncertainty on that assignment from A based on MCMC sampling. By increasing the number of standard deviations away from zero required to assign a gene to a group, multiple estimates of the assignment of the genes to the patterns were made, allowing the ROC curve to be constructed. The gold standard for the analysis was based on the known molecular biology of gene coregulation independent of microarray studies, and this comprised 9 groups with 43 genes total [13]. The results were compared between hierarchical clustering, the original BD, and BD-TF.

Yeast cell cycle In Figure 5 on the left, we present results of the application of BD-TF to the cell cycle data using ROC analysis based on known coregulation [13]. We compared the results using original BD (circles), BD-TF (squares), and shrinkage-based hierarchical clustering (triangles; performed previously [13]). BD-TF obtained an area under the curve of 0.82, compared with 0.83 for BD and 0.56 for hierarchical clustering. The lack of improvement from use of coregulation information reflected the lack of such data here, as we had TF data on only 67 of 788 genes, which was not adequate to improve inference over BD. However, we include the results to show the value in BD and BD-TF

The Rosetta compendium data was previously analyzed using BD at multiple dimensions, and consistency analysis determined that 15 dimensions were optimal [10]. The data was reanalyzed at 15 dimensions using BD-TF as for the yeast cell cycle data, except we included targets whose regulation by a TF was supported by a single type of

1253

M. F. Ochs et al. / Determining Transcription Factor Activity from Microarray Data using Bayesian Markov Chain Monte Carlo Sampling

that arises solely from the proper assignment of genes to multiple groups.

References [1] Cho RJ, Campbell MJ, Winzeler EA, Steinmetz L, Conway A, Wodicka L, et al. A genome-wide transcriptional analysis of the mitotic cell cycle. Mol Cell. 1998;2(1):65-73. [2] Alizadeh AA, Eisen MB, Davis RE, Ma C, Lossos IS, Rosenwald A, et al. Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature. 2000;403(6769):503-11. [3] Vadigepalli R, Chakravarthula P, Zak DE, Schwaber JS, Gonye GE. PAINT: a promoter analysis and interaction network generation tool for gene regulatory network identification. Omics. 2003 Fall;7(3):235-52. [4] Lukashin AV, Fuchs R. Analysis of temporal gene expression profiles: clustering by simulated annealing and determining the optimal number of clusters. Bioinformatics. 2001;17(5):405-14. [5] Wingender E. TRANSFAC, TRANSPATH and CYTOMER as starting points for an ontology of regulatory networks. In silico biology. 2004;4(1):55-61. [6] Alter O, Brown PO, Botstein D. Singular value decomposition for genome-wide expression data processing and modeling. Proc Natl Acad Sci U S A. 2000;97(18):10101-6. [7] Moloshok TD, Klevecz RR, Grant JD, Manion FJ, Speier WFt, Ochs MF. Application of Bayesian Decomposition for analysing microarray data. Bioinformatics. 2002 Apr;18(4):566-75. [8] Sibisi S, Skilling J. Prior distributions on measure space. Journal of the Royal Statistical Society, B. 1997;59(1): 217 - 35. [9] Ochs MF, Moloshok TD, Bidaut G, Toby G. Bayesian Decomposition: Analyzing microarray data within a biological context. Annals of the New York Academy of Sciences. 2004;1020:212 - 26. [10] Bidaut G, Suhre K, Claverie JM, Ochs MF. Determination of strongly overlapping signaling activity from microarray data. BMC Bioinformatics. 2006 Feb 28;7(1):99. [11] Hughes TR, Marton MJ, Jones AR, Roberts CJ, Stoughton R, Armour CD, et al. Functional discovery via a compendium of expression profiles. Cell. 2000;102(1): 109-26. [12] Rocke DM, Durbin B. A model for measurement error for gene expression arrays. J Comput Biol. 2001;8:557 - 69. [13] Cherepinsky V, Feng J, Rejali M, Mishra B. Shrinkagebased similarity metric for cluster analysis of microarray data. Proc Natl Acad Sci U S A. 2003 Aug 19;100(17): 9668-73.

Figure 5 – ROC Curves for Cell Cycle and Rosetta Data Rosetta compendium The analysis of data from the Rosetta compendium data set using BD-TF (squares) was compared to K-means clustering (triangles) and the original BD analysis (circles) of the same data (Figure 5 right). Here, both the coregulation information and the “gold” standard gene lists were the same, so that the results demonstrated that the algorithm correctly used information about TF regulation and that such coregulation was reflected in the data. All techniques performed equally well at high specificity, however as sensitivity increased, BD-TF was superior due to a reduction of false positives with the inclusion of prior information on TF regulation.

Conclusion This work demonstrates 1) the value of inclusion of prior knowledge on transcriptional regulation in the analysis of microarray data, and 2) the present limits of that knowledge. While the simulations showed a clear advantage in using this knowledge, especially at typical noise levels, the analysis of cell cycle data indicated that more prior information would be helpful. Nevertheless, the superiority of the BD-TF approach over clustering for microarray analysis is clear. Our knowledge of transcriptional regulation is rapidly increasing, and we expect improved statistical power with BD-TF over the next few years. This power will be critical to improved inference of biological process activity, especially with heterogeneous and limited samples typical in clinical settings. These samples introduce noise to an analysis focused on understanding biological response, such as in therapeutic interventions, and techniques to gain statistical power through use of existing biological knowledge will be critical to make progress.

Address for correspondence Michael F. Ochs, Oncology Biostatistics and Bioinformatics, The Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins, 550 N. Broadway, Suite 1103, Baltimore, MD 21205, USA Email: [email protected]

Acknowledgements This work was supported by NIH: LM008309 to mfo and CA06973 to mfo through the Sidney Kimmel Cancer Center.

1254

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Ensemble Stump Classifiers and Gene Expression Signatures in Lung Cancer Lewis Freya1, Mary Edgertonb, Douglas Fisherc and Shawn Levyd a Biomedical Informatics Department, University of Utah, USA Department of Anatomic Pathology, M.D. Anderson Cancer Center, USA c Department of Electrical Engineering and Computer Science, Vanderbilt University, USA d Department Biomedical Informatics, Vanderbilt University, USA b

By combining data from different platforms, problems of data sparsity and overfitting can be mitigated. The microarray data sets available in repositories are growing at a rapid rate. The creation of data sharing initiatives such as oncomine.org and the Cancer Biomedical Informatics Grid (caBIG™) enable the combination of multiple data sets to find better classifiers.

Abstract Microarray data sets for cancer tumor tissue generally have very few samples, each sample having thousands of probes (i.e., continuous variables). The sparsity of samples makes it difficult for machine learning techniques to discover probes relevant to the classification of tumor tissue. By combining data from different platforms (i.e., data sources), data sparsity is reduced, but this typically requires normalizing data from the different platforms, which can be non-trivial. This paper proposes a variant on the idea of ensemble learners to circumvent the need for normalization. To facilitate comprehension we build ensembles of very simple classifiers known as decision stumps – decision trees of one test each. The Ensemble Stump Classifier (ESC) identifies an mRNA signature having three probes and high accuracy for distinguishing between adenocarcinoma and squamous cell carcinoma of the lung across four data sets. In terms of accuracy, ESC outperforms a decision tree classifier on all four data sets, outperforms ensemble decision trees on three data sets, and simple stump classifiers on two data sets.

However, combining data across platforms is challenging. First of all, there are multiple microarray platforms and these can differ in the types of probes arrayed (i.e. variables measured) for the specimens. A method for mapping the probes across platforms is first required to relate the results. The different means of getting expression levels from the platforms need to be consistently compared, which often requires normalizing the expression levels between platforms. A methodology that supports the generation of classifiers that find easily interpretable, robust mRNA signatures of cancer that generalize across experiments and platforms is needed. By robust mRNA signatures what is meant is a set of probes that are consistently associates with a cancer type. This paper describes a novel method for combining data sets to discover classifiers that use robust mRNA signatures which generalize across experiments and platforms. This is done by using a classifier to focus on a limited number of predictive probes in the data that persist across data sets. The result of our approach is a distinct classifier for each data set under investigation, but importantly the construction of each such classifier is informed by all the available data.

Keywords: microarray, decision trees, ensembles, stumps

Introduction Methods for finding robust mRNA signatures of cancer that remain consistent across experiments and microarray platforms (i.e., oligonucleotide and cDNA) have remained elusive in the bioinformatics literature. From a machine learning perspective this is expected, since many microarray data sets have a scarcity of sample (e.g., a few hundred); moreover, each sample has thousands of probes (i.e., continuous variables) resulting in a very pronounced curse of dimensionality. With thousands of variables from which to choose, the constructed classifier can overfit the specific data and cannot generalize to other data sets. It poses a challenge when applying machine learning techniques to discover a set of relevant probes that constitute a robust mRNA signature for the cancer. There are a number of papers describing the pitfalls of overfitting expression data and the failure of some classification models to do better than chance [1,2,3].

1

Empirical results by Holte [4] showed that simple, single test classifier trees, referred to as Stumps, can be surprisingly close in accuracy to more complex decision tree classifiers5 in many of the domains tested. Results on data sets used in machine learning have shown that combining multiple classifiers boosts classification accuracy by creating variance among the constituent classifiers. Our approach, Ensemble Stump Classifier (ESC), is a kind of subspace sampling. It combines different probes that have slightly dissimilar classification of the sample using a majority vote to boost accuracy due to the variance in the sample they accurately classify. Using Stump classifiers as the “base” classifier of the ensemble supports the creation of simple ensemble classifiers.

L.F. was supported in part by a fellowship from the National Library of Medicine (5T15LM007450). M.F. and D.F. were supported in part by a National Institute of Health grant (1R01LM008000) to M.E.

1255

L. Frey et al. / Ensemble Stump Classifiers and Gene Expression Signatures in Lung Cancer

Materials and methods

Building an ESC The following are the steps to generate a full ensemble stump classifier (as in Table 1b):

Four classifier-methods -- decision trees (C5.0), ensemble decision trees, Stump rule and ESC -- are applied to the data sets. Ten-fold cross validation is used to derive the accuracy measure for the data sets. Each data set is divided into 10 mutually-exclusive folds, consisting of 10% of the data. One of the 10-folds is held out as a testing set and the remaining 90% is used as the training set. Then a different fold is selected as the testing set and the remaining 90% serves as the training set. The average testing accuracy is calculated across all 10-folds.

1. Determine which probes match each other and construct rows based on the matching (i.e., “synonymous”) probes across the data sets. This was briefly described in the basic representations section, and is described in more detail in the match probes section. 2. Partition each of the three data sets into folds so that there are training and testing sets. In our experimental design we use 10-fold cross validation. 3. Learn the stump rules, one for each training set and matching probe set, from training folds. This was illustrated in the basic representations section (e.g., each Pij in Table 1b), and follows the same process given for stump rules. 4. Reorder the stumps across the data sets based on the probes that are “collectively” best overall. 5. Determine the best number of probes to be in the ESC based on training accuracy. Step 4 is implemented by ordering the stump rules generated in Step 3 according to the weighted average training accuracy across the data sets. The average accuracy is weighted by the size of the data sets. Step 5 is implemented by incrementally adding stump classifiers in order of quality (accuracy) until one fails to improve the accuracy of the ESC.

C5.0, ensemble C5.0 and stumps C5.0 is a commercial implementation of Quinlan’s decision tree approach [5]. The ensemble decision tree classifier is computed using C5.0’s implementation of the boosting approach of Freund and Schapire [6]. Stumps are single variable-test decision trees. Each Stump is made up of one probe at the root and has the best threshold for the training set that separates the two classes. Stumps are obtained by running C5.0 rules on a single data set with pruning set to a 1% confidence level with only single variable rules allowed. Ensemble stump classifier (ESC) The ESC is a new classifier approach that bypasses the need to normalize across multiple data sets. The base classifier for the ensemble is the Stump classifier. The outcome of our method is a separate classifier for each data set, but notably each classifier is instructed by all the available data.

Classification with an ESC Table 1a illustrates the data structures for three data sets with probes having expression values and each instance having a true class assigned. The true classes are assigned to the instance through an expert classifying the instance. For cancer tissues the expert is often a pathologist. Table 1b illustrates the data structure for an ESC consisting of three probes over three data sets. The thresholds and predictive classes are shown for the probes for each data set.

Basic representations ESC’s form of ensemble learning is different than boosting. The idea that learning is occurring over multiple data sets is key. To illustrate, consider three artificially constructed objects, one from each of three data sets (see Table 1a). Probes P1j (i.e., a matching probe set) represent “synonymous” probes across the three data sets. The three probes - P11, P12, and P13 -- can be thought of as the same probe with different scales and labels. The probe P11 has an expression value for the probe in match Probe Set 1 and Data Set 1.

Table 1c is the substitution of the expression values for the probes from Table 1a into the data structures for ESC in Table 1b. The predicted classes are present for each probe. The cells that are grey in Table 1c are incorrect class assignments given the expression values and the thresholds. The bolded classes are correct assignments. For example in Table 1c, P11 is incorrectly assigned while P12 and P13 are correctly assigned. Two out of three correct gives an accuracy of 66.7%.

The ESC algorithm repeatedly finds the “probe” (though with different labels and scales) that is “collectively” best over all data sets. This step leads to one decision stump per data set, each of which uses the same decision variable/ probe. For example, if probe P1 were found to be “collectively best” over three data sets, then the three decision stumps for each data set would have the form expressed in Table 1b.

The ESC majority vote classifier is obtained by going down a column and choosing the majority class predicted by the rules. An odd number of probes are used in the ensemble for simplicity and to ensure there is always a winner. For example in Data Set 1 the majority class is 1 because P21 and P31 are both correctly assigned to Class 1. For Data Set 2 the Majority Class is 2 because P12 and P22 are both correctly assigned to Class 2. Going across the majority vote row gives three out of three correct for an accuracy of 100%. Table 2 is the accuracy of the probe sets across the data sets and the ESC accuracy. Because we have not normalized the classifiers in ESC, the column or data set against which the test instance is compared must be known; we will address this limitation in the discussion.

The threshold for match Probe Set 1 in Data Set 1 is equal to 0.7 in Table 1b. Given an expression value for probe P11 equal to or below 0.7 the Stump rule predicts Class 1, otherwise Class 2. For Probe Set 1 and Data Set 2 the threshold is different than Data Set 1 (compare threshold 0.7 for P11 and threshold 10 for P12). Even though the probes are the same, the thresholds can be different between data sets. This occurs because the data sets are not normalized to each other. The threshold is only consistent within a data set.

1256

L. Frey et al. / Ensemble Stump Classifiers and Gene Expression Signatures in Lung Cancer

Table 1 - Obtaining accuracy given an instance and an example ESC. (a) Test instance with expression values and class. Data 1

Data 2

Data 3

Test Instance with expression values for probes

(P11=0.8, P21= 0.85, P31=1.5)

(P12=12, P22=14, P32=15.5)

(P13=0.02, P23=0.35, P33= 0.55)

True Class

Class 1

Class 2

Class 1

(b) Example ESCs for the three probe sets and data sets: Match Probe

Data Set 1

Data Set 2

Data Set 3

P1j

P11 Class 1 P11 > 0.7 -> Class 2

P12 Class 1 P12 > 10 -> Class 2

P13 Class 1 P13> 0.1 -> Class 2

P2j

P21 Class 1 P21 > 0.9 -> Class 2

P22 Class 1 P22 > 12 -> Class 2

P23 Class 1 P23 > 0.3 -> Class 2

P3j

P31 Class 2 P31 > 1.2 -> Class 1

P32 Class 2 P32 > 15 -> Class 1

P33 Class 2 P33 > 0.5 -> Class 1

ESC

Majority class predicted by P11, P21 & P31 Stump rules

Majority class predicted by P12, P22 & P32 Stump rules

Majority class predicted by P13, P23 & P33 Stump rules

(c) Class predictions given expression values from 1(a) and classifiers from 1(b). Bold is correct and grey is incorrect as compared to true class in 1(a). Match Probe

Data Set 1

Data Set 2

Data Set 3

P1j

0.8 > 0.7 -> Class 2

12 > 10 -> Class 2

0.02 Class 1

P2j

0.85 Class 1

14 > 12 -> Class 2

0.35 > 0.3 -> Class 2

P3j

1.5 > 1.2 -> Class 1

15.5 > 15 -> Class 1

0.55 > 0.5 -> Class 1

ESC

Majority class predicted by P11, P21 & P31 Stump rules > Class 1

Majority class predicted by P12, P22 & P32 Stump rules > Class 2

Majority class predicted by P13, P23 & P33 Stump rules > Class 1

Table 2 - Accuracy across the three data sets of the match probe sets and the ESC. Match Probe

P1j

P2j

P3j

ESC

lung with 4417 cDNA probes. The Garber et al. [10] data set consists of 52 samples with 39 adenocarcinomas and 13 squamous cell carcinomas of the lung with 24,192 cDNA probes.

Accuracy

66.7%

66.7%

66.7%

100%

Matched probes Before obtaining an ESC we must identify matching probes across data sets. While the other classifier methods that we have described will be applied to each of the four data sets independently, an ESC will be learned from the “combined” data set that we are about to describe.

Gene expression data There are four data sets. Collectively, these data sets are made up of two affymetrix arrays and two cDNA microarrays data sets. There are two types of tumor tissue: adenocarcinoma and squamous cell carcinoma of the lung. The Su et al. [7] data set consists of 28 samples with 14 adenocarcinomas and 14 squamous cell carcinomas of the lung with 12,533 affymetrix probes (i.e., continuous variables). The Bhattacharjee et al. [8] data set consists of 160 samples with 139 adenocarcinomas and 21 squamous cell carcinomas of the lung with 12,600 affymetrix probes. The Yamagata et al. [9] data set consists of 20 samples with 9 adenocarcinomas and 11 squamous cell carcinomas of the

A probe corresponds to part of a gene. Thus, different probes can be associated with the same gene. The same gene may be referenced by different probes in different data sets obtained on different platforms. In order to use an ensemble method that generalizes across different data sets, the Affymetrix and cDNAs probes must be mapped to each other. For this paper all probes are matched via Affymetrix U95A probe names. For the ensemble method the probe sets are matched as follows. Bhattacharjee et al. and Su et al. are joined via their almost identical chips, which are

1257

L. Frey et al. / Ensemble Stump Classifiers and Gene Expression Signatures in Lung Cancer

A measure of the complexity of the classifiers is given in the mean number of variables across folds for a classifier. The ESC converges to a classifier of three variables (genes: BPAG1, KRT5 and ABCC5) for nine out of the 10folds. As explained earlier, this number of genes does not result from a user-defined threshold, but to add more stumps (i.e., genes) to the ensemble would reduce training set accuracy. In the one remaining fold ESC converges to BPAG1, KRT5 and SIAT7B which results in a total of four variables used by the classifier. The thresholds in the rules are stable within each data set and hence there is consensus among the stumps in the cross-validation step Ensemble C5.0, C5.0 and Stump use varying number of probes for each data set, but they do not find probes that generalize across all four data sets.

U95Av2 and U95A Affymetrix chips respectively. Thus they have total overlap of 12533 unique probes. Garber et al. and Yamagata et al. also have similar cDNA probes with a unique overlapping set of 2106 accession id probes. Consequently, we have two pairs of similar platforms. It is trivial to map probes across platforms within each pair. To map across the pairs (i.e., across all four platforms) an online resource called ProbeMatchDB [11] is used to map these 2106 accession ids onto the U95A Affymetrix ids in a many-to-many mapping. The many-to-many mapping occurs because some of the 2106 cDNA probes have multiple U95A Af-fymetrix ids associated with them. All possible combinations of probes from the four data sets with matching U95A Af-fymetrix ids are used to construct the probe sets. This match-ing process results in the creation of 4491 probe sets. The probe sets are made up of four probes with one from each data set. A probe set can be thought of as a set of references for the same gene. In the discussion that follows we use “probe set” and “gene” synonymously.

Discussion ESCs are not limited to genomic data per se, and we are interested in their characteristics from a machine learning standpoint. Ensembles boost accuracy by insuring variability in classification behavior among the base classifiers. In bagging [12] this variance stems from bootstrap sampling and the instability of the classifiers that are constructed with these differing sample. In random subspace selection the requisite variance comes from selection of differing variables [13] on which to form the classifier. To some extent such designs might be motivated by a desire to use off-the-shelf, greedy decision tree induction, which is a standard base classifier of ensemble approaches. Another way that variance could be achieved would be to modify classifier systems to directly return a set of sufficiently good and sufficiently different classifiers. In fact, this is our approach, though our base classifiers are stumps – we incrementally add “best” stumps (as assessed across multiple data sets) until performance drops.

Results The four classification methods described above (i.e., ESC, Stump Rule, C5.0 and Ensemble C5.0) are compared using 10-fold cross validation testing accuracy, standard error and the average number of variables (Avg Var) for each classifier (See Table 3). Each C5.0 tree, Ensemble C5.0 forest and Stump Rule has been built in a way that is informed by a single data set. Thus, we will speak of classifications made by these approaches as using probes. The ESC method uses the same 4,491 match probe sets or genes for each data set. As we have noted, the construction of ESCs are informed by multiple data sets, but to classify a datum with an ESC we must know the data set (e.g., what column in Table 1) that test instance is drawn.

The limitation of knowing what column in ESC to apply to a particular data set is moot within a lab or facility since they will always compare on the data they collect. The benefit comes from being informed by data collected at other labs and facilities. In practice it can be used to identify signature genes that when combined enable the construction of high accuracy classifiers, even in hold-out data sets.

The ESC does better than C5.0 on all four data sets, better than ensemble C5.0 on three out of four sets and better on two out of four for Stump Rule. Note that the ESC uses probes that are robust across the four data sets rather than using the best probes for each given data set. The Stump Rule, C5.0 and ensemble C5.0 are all using the best probes within a given data set, which gives them an advantage for computing accuracy, although not on generalization to other data sets.

Table 3 - Adenocarcinoma vs. Squamous Cell Carcinoma for 10-fold cross validation test accuracy for ESC, Stump classifiers, C5.0 decision tree, Ensemble C5.0 and SVMs with standard error (SE) and average number of variables across folds (Avg Var) in the classifier. Data Set

ESC

Stump Rule

C5.0

Ensemble C5.0

Avg Acc %

SE

Avg Var

Avg Acc %

SE

Avg Var

Avg Acc %

SE

Avg Var

Avg Acc %

SE

Avg Var

Bhattacharjee

95.6

1.3

3

96.9

1.0

1

93.7

1.6

3

94.4

1.7

1.3

Su

93.3

4.4

3

83.3

7.5

1

91.7

5.7

1

90.0

7.1

1

Garber

91.8

3.4

3

92.7

3.1

1

87.0

4.8

1.4

88.3

4.4

1.2

Yamagata

95.0

5.0

3

40.0

10.0

1

80.0

11.1

1

90.0

6.7

1

1258

L. Frey et al. / Ensemble Stump Classifiers and Gene Expression Signatures in Lung Cancer

Whenever ensembles succeed in boosting accuracy it can be argued that the base classifiers, by definition, must be overfitting or underfitting the data. The ensemble then results in a classifier that moves towards a “best” fit to the data. In the case of an ensemble of decision trees that boost accuracy, the move is probably from overfit to better fit. Overfitting may account for cases where C5.0 underperformed Stump Rule on

Conclusion The ESC method does well with only three variables. These results suggest the existence of compact sets of genes with single thresholds, which can be measured using multiple modalities that consistently and accurately predict diagnosis. The building of data repositories and data exchange standards such as oncomine.com and the caBIG™ can assist in the discovering other robust mRNA signatures of cancer using ESC. The ESC finds in a greedy fashion the best available matched probes that can be used in an mRNA signature that generalizes across data sets.

some data sets. In the case of ESCs the move is probably from underfit to better fit. ESCs are not only very simple, are formed from stumps that generalize well across data sets, and as a result may underfit any given data set.

References

One reason stumps that generalize well across data sets are desirable with microarray data is that artifacts can be introduced into the data by the lab that collects the data. Thus, the very best stump for a particular data set may exploit an environmental peculiarity of the lab that is collecting the data. When dealing in thousands of probes, any small laboratory bias may systematically influence the value of one or more probes [14]. Problems with data or facility bias are found in other contexts. For example, Evans and Fisher [15] found that a feature (i.e., printing press) that was highly predictive of a printing defect in a particular printing plant provided no insights to why similar problems occurred at other plants.

[1] Simon, R. Diagnostic and prognostic prediction using gene expression profiles in high-dimensional microarray data. British Journal of Cancer 2003: 89; pp. 1599-1604. [2] Simon, R., Radmacher, M.D., Dobbin, K. and McShane, L.M. Pittfalls in the Use of DNA Microarray Data for Diagnostic and Prognostic Classification. J. Natural Cancer Institute 2003: 95(1); pp. 14-18. [3] Michiels, S., Koscielny, S. and Hill, C. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet 2005: 365; pp. 488-92. [4] Holte, R. Very Simple Classification Rules Perform Well on Most Commonly Used Datasets. Machine Learning 1993: 11; pp. 63-91. [5] Quinlan, J.R. Induction of decision trees. Machine Learning 1987: 1; pp. 81-106. [6] Freund, Y. and Schapire, R.E. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference 1996; pp. 148 - 156. [7] Su, A. et al. Molecular Classification of Human Carcinomas by Use of Gene Expression Signatures.* Cancer Research 2001: 61; pp. 7388-7393. [8] Bhattacharjee, A, et al. Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses. PNAS: 2001: 98 (24); pp. 13790-13795. [9] Yamagata N. et al. A training-testing approach to the molecular classification of resected non-small cell lung cancer. Clin Cancer Res 2003: 9; pp. 4695-704. [10] Garber, M.E., et al. Diversity of gene expression in adenocarcinoma of the lung. PNAS 2001: 98 (24); pp. 13790-13795. [11] Wang,P. et al. (2002) ProbeMatchDB—a web database for finding equivalent probes across microarray platforms and species. Bioinformatics, 18, pp. 488–489. [12] Breiman, L. Bagging Predictors. Machine Learning 1996: 24(2); pp 123-140. [13] Breiman, L. Random Forests. Machine Learning 2001: 45(1); pp. 5-32. [14] Shi, Leming et al. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nature Biotechnology 2006: 24(9); pp. 1151 – 1161. [15] Evans, B. and Fisher, D. Overcoming Process Delays with Decision Tree Induction 1994: 9(1); pp. 60-66.

ESC builds classifiers that generalize across data sets (labs, facilities), and informs data collectors about probes that generalize beyond their data collection processes. When ESCs learned across data sets are contrasted with the best classifiers within a data set, our methodology can also point out lab biases that should be remedied. ESCs were motivated initially by the desire to combine evidence from multiple sources of gene expression data, thereby mitigating the curse of dimensionality. We are interested then in what ESCs can tell in a biological domain. Notably, the ensemble method identifies the genes KRT5, BPAG1 and ABCC5 as informative across the four data sets that we examined. Importantly, the ESC method gives convergent support for the relevance of these probe sets relative to the findings in the original studies. Using hierarchical clustering Bhattacharjee et al. [8] found KRT5 and BPAG1 to be highly expressed in squamous cell carcinoma. Using hierarchical clustering Yamagata et al. [9] confirmed in their data that KRT5 and BPAG1 are highly expressed in squamous cell carcinoma. Using hierarchical clustering Garber et al. [10] also identified KRT5 and BPAG1 as highly expressed in squamous cell carcinoma. Using a SVM ranking method Su e al. [12] did not identify high expression of KRT5 or BPAG1 as predictive of squamous cell carcinoma. However, they did identify ABCC5 as predictive of squamous cell carcinoma. The convergent findings of these alternative methods provide additional support of the utility of the ESC method.

Address for correspondence Lewis Frey, Ph.D. 26 South 2000 East; Salt Lake City, UT 84112; USA

1259

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

From “Glycosyltransferase” to “Congenital Muscular Dystrophy”: Integrating Knowledge from NCBI Entrez Gene and the Gene Ontology Satya S. Sahoo1, Kelly Zeng2, Olivier Bodenreider2, Amit Sheth1 1

Kno.e.sis Center, Department of Computer Science and Engineering, Wright State University, Dayton, OH, USA 2 U.S. National Library of Medicine, NIH, Bethesda, Maryland, USA

In order to validate the hypothesis of possible association between the molecular function glycosyltransferase and the disease congenital muscular dystrophy, a researcher could simply search EG for the term glycosyltransferase, and all records containing the string “glycosyltransferase” in GO annotations would be returned. This approach, however, is suboptimal for at least two reasons. First, the term glycosyltransferase might appear as a substring in other GO terms (e.g., in UDP-glycosyltransferase), possibly leading to false positives. Conversely, not all GO terms related to glycosyltransferase actually contain the string “glycosyltransferase” (e.g., acetylglucosaminyltransferase, a kind of glycosyltransferase), possibly leading to false negatives.

Abstract Entrez Gene (EG), Online Mendelian Inheritance in Man (OMIM) and the Gene Ontology (GO) are three complementary knowledge resources that can be used to correlate genomic data with disease information. However, bridging between genotype and phenotype through these resources currently requires manual effort or the development of customized software. In this paper, we argue that integrating EG and GO provides a robust and flexible solution to this problem. We demonstrate how the Resource Description Framework (RDF) developed for the Semantic Web can be used to represent and integrate these resources and enable seamless access to them as a unified resource. We illustrate the effectiveness of our approach by answering a real-world biomedical query linking a specific molecular function, glycosyltransferase, to the disorder congenital muscular dystrophy.

To avoid false positives and false negatives, a careful researcher would likely start exploring the Gene Ontology database to create a list of glycosyltransferase-related terms by selecting the term glycosyltransferase itself (GO:0016757) and all its descendants, including specialized types of glycosyltransferase, such as acetylglucosaminyltransferase. This researcher would then look for the genes annotated with any of the glycosyltransferase-related terms. Resources such as the web browser AmiGO [1] support such searches and can retrieve the genes associated with any descendant of a given GO term. Finally, each of the genes found associated with any of the glycosyltransferase-related terms must be searched individually in EG, looking for mentions of the disease congenital muscular dystrophy (as an OMIM phenotype) in the corresponding records.

Keywords: knowledge integration, Semantic Web, RDF, Entrez Gene, Gene Ontology

Introduction A common scenario in biomedical research involves the correlation of genomic data with disease information, in other words, associating genotype and phenotype information. In the particular scenario illustrated in this paper, a researcher is interested in glycosylation and its implications for one disorder: congenital muscular dystrophy. The biological process of glycosylation results in the posttranslational addition of glycosyl groups (saccharides) to proteins (and lipids). Various enzymes, namely glycosyltransferases, catalyze glycosylation reactions.

The procedure described above is evidently inefficient, time consuming and error prone as several web interfaces need to be utilized (AmiGO and Entrez), and as the results of the search in one resource need to be copied and pasted as search terms in the other. The main reason for such inefficiency is that high quality resources such as GO and EG have been designed primarily for consultation by humans, not for automated processing by agents or integration in applications. Moreover, these resources have been developed by different groups, independently of each other and are therefore not interoperable. No system currently supports complex queries such as: Find all the genes annotated with glycosyltransferase-related terms in GO and associated with the disease congenital muscular

From the functional annotation of gene products with terms from the Gene Ontology (GO), a researcher can identify the genes having the molecular function of catalyzing the transfer of specific glycosyl groups (e.g., hexosyltransferase, for hexosyl groups). Known associations between these genes and diseases can then be mined from resources such as NCBI’s Entrez Gene (EG), where phenotypic information is recorded as pointers to the Online Mendelian Inheritance in Man (OMIM) knowledge base [3]. (See the Materials section for a presentation of GO and EG.)

1260

S. S. Sahoo et al. / Glycosyltransferase to Congenital Muscular Dystrophy: Integrating Knowledge from NCBI Entrez Gene and Gene Ontology

dystrophy in OMIM. Typically, querying across the different knowledge sources is accomplished manually through meticulous work or requires the development of complex and customized software applications.

The RDF triples often share nodes, thus forming a graph. For example, the two triples shown in Figure 2 share the node acetylglucosaminyltransferase (GO:0008375). The resulting graph is shown in Figure 2. The graph structure created by RDF is key to information integration in the Semantic Web.

In this paper, we propose an integrative approach to querying across knowledge sources. More specifically, we have applied Resource Description Framework (RDF) [4] standard developed by the World Wide Web Consortium (W3C) to integrate knowledge from GO and EG, and used this integrated resource to answer complex queries. We use the scenario presented earlier to illustrate the advantages of this approach. This work is a pilot contribution to the Biomedical Knowledge Repository under development at the U.S National Library of Medicine (NLM) as part of the Advanced Library Services project [8]. This repository integrates knowledge not only from structured resources (database and knowledge bases), but also from the biomedical literature (e.g., MEDLINE), in order to support applications, including knowledge discovery.

is_a GO:0008375

GO: 0016758

has molecular function EG:9215

GO:0008375

Figure 1 - Example of RDF triples

is_a

Background

GO:0008375

GO: 0016758 ha s m olecula

Information integration is one of the most challenging areas of research in Computer Science [11]. The use of heterogeneous schemas for data storage, that are designed primarily to ensure optimization of storage space, makes it extremely difficult for users to query data sources in an integrated manner. (The interested reader is referred to [12] for a survey of approaches to information integration.) The Semantic Web provides a common framework that enables the integration, sharing and reuse of data from multiple sources. The use of a representation formalism based on a formal language enables software applications to ‘understand’ and reason over information. Recent research in Semantic Web technologies has delivered promising results to enable information integration across heterogeneous knowledge sources.

r functi on

EG:9215

Fiigure 2 - Example of RDF graph

Materials The Gene Ontology (GO) seeks to provide a consistent description of gene products [13]. GO consists of three controlled vocabularies for biological processes (9,234 terms), molecular functions (7,456 terms) and cellular components (1,804 terms). The GO monthly releases are made available on the GO website in various formats, including RDF. The version of GO used in this study is dated of September 2006.

The Resource Description Framework (RDF) is a W3Crecommended framework for representing data in a common format that captures the logical structure of the data. This is in contrast to pure storage aspects addressed by traditional relational database schema. The RDF representational model uses a single schema in contrast to multiple heterogeneous schemas or Data Type Definitions (DTD) used to represent data in XML by different sources. Hence in conjunction with a single Uniform Resource Identifier (URI), all data represented in RDF form a single knowledge repository that may be queried as one knowledge resource. An RDF repository consists of a set of assertions or triples. Each triple is constituted of three entities namely, the subject – the triple pertains to this entity, the object – the entity that states something about the object and the predicate – the relationship between the subject and the object. For example, as shown in Figure 1, assertions such as acetylglucosaminyltransferase (GO:0008375) is a kind of hexosyltransferase (GO:0016758) and the gene LARGE (EG:9215) has molecular function acetylglucosaminyltransferase (GO:0008375) can be represented as RDF triples.

The Entrez Gene (EG) database records gene-related information from sequenced genomes and of model organisms that are focus of active research [9], totaling about two million genes. EG contains gene information about genomic maps, sequences, homology, and protein expression among others [9]. In contrast to GO, EG is not available in RDF, but in XML (converted from ASN1 by the program gene2xml provided by NCBI), and can be downloaded from the NCBI website. The version of EG used in this study is dated of July 2006.

Methods Our integration method can be summarized as follows and is illustrated in Figure 3. First, we extract manageable subsets from the two resources to be integrated. We then have to convert the EG subset from XML to RDF. Finally, we load both RDF resources in a common store, apply inference rules, and issue queries against it.

1261

S. S. Sahoo et al. / Glycosyltransferase to Congenital Muscular Dystrophy: Integrating Knowledge from NCBI Entrez Gene and Gene Ontology

We selected the eXtensible Stylesheet Language Transformation (XSLT) [6] for converting the EG XML information into RDF, because this approach allows for a clean separation between the application (using Java API for XML Processing (JAXP)) and the conversion logic (using XSLT stylesheet). Once the stylesheet is created, it can serve as an auxiliary file for existing programs realizing the XML to RDF conversion. In other words, the major interest of this approach is that no specific code is required for the conversion, because the transformation logic resides entirely in the stylesheet. Loading the two resources into a single data store Some of the requirements for our RDF store include native support for the RDF graph data model, support for persistence and indexing of the RDF triples, support for extensive collections of triples, and availability of a query language for the RDF graph. After surveying available RDF storage solutions, we decided to use Oracle Spatial 10g [7] as the RDF storage system. The RDF file resulting from the XSLT conversion of the original XML file for EG and the downloaded RDF version of GO are both loaded into a single RDF store. More precisely, the RDF resources are first converted to the NTriple format using the Jena API [10] and loaded into the RDF database using a utility program provided by Oracle. Applying inference rules

Figure 3 - Overview of the integration method

Unlike the Web Ontology language OWL, RDF provides no direct support for inference. However, inference rules can be implemented in the RDF store to make explicit the semantics of some predicates. For example, the relationships is_a and part_of used in GO are partial order relations, thus being reflexive, antisymmetric and transitive. The inference rules we created for implementing the transitivity and combination of these two relationships are shown in Table 1. The inference rules are stored in a rule base created in Oracle 10g.

Creating subsets The entire Entrez Gene data file (in XML format) is very large (50 GB) and unnecessarily difficult to manipulate. In order to obtain a manageable subset from EG, we restricted the gene records to two species: Homo sapiens (human) and Mus musculus (mouse). The resulting EG subset contains a total of 99,861 complete gene records (excluding obsolete records). Converting XML format Entrez Gene data to RDF A key element of our integration approach is the conversion of Entrez Gene from XML to RDF. There are many issues involved in the conversion of XML data into RDF format, including modeling the original semantics of the data, filtering redundant XML element tags, linking data entities using meaningful named relationships and identifying entities consistently within and across resources. Unlike traditional XML to XML conversion, XML to RDF conversion should exploit the advantages of the RDF model in representing the logical structure of the information.

Table 1 - Inference rules for is_a and part_of in GO Relation

is_a

part_of

is_a

IF <x is_a y> &

IF <x is_a y> &

THEN <x is_a z>

THEN <x part_of z>

IF <x part_of y> &

IF <x part_of y> &

part_of

THEN <x part_of z> THEN <x part_of z>

We chose not to convert the element tags of the native EG XML representation mechanically into the predicates of the RDF triples. Instead, we manually converted the XML element tags into meaningful and standardized relationship names that convey explicitly the semantics of the connection between the subject and the object. For example, the element was mapped to the more meaningful relationship named has_source_organism_taxonomic_name.

Querying the RDF Graph with SPARQL SPARQL [5] is a query language for RDF graphs, equivalent to SQL, the Structured Query Language, for relational databases. Unlike SQL, SPARQL does not require users to be familiar with the data model (e.g., tables, foreign keys), but simply to indicate how entities of interest relate to each other. For example, the structure of the query: Find all the genes annotated with the GO molecular function glycosyl-

1262

S. S. Sahoo et al. / Glycosyltransferase to Congenital Muscular Dystrophy: Integrating Knowledge from NCBI Entrez Gene and Gene Ontology

transferase (GO:0016757) or any of its descendants and associated with any form of congenital muscular dystrophy is represented in Figure 4.

atively small in comparison to the 411 million triples resulting from the conversion of the entire EG to RDF [2]. Biological query result: extended example The SPARQL query presented above returned one result, corresponding to one path in the graph between the GO term glycosyltransferase (GO:0016757) and OMIM disease names containing (variants of) the string “congenital muscular dystrophy”.

is_a GO ID

GO ID has m olecula r

functio

e enotyp soc. ph has as

OMIM ID

n

Gene ID

This path involved the human gene LARGE likeglycosyltransferase (EG:9215), annotated with the GO term acetylglucosaminyltransferase (GO:0008375), a descendant of glycosyltransferase (GO:0016757). Also involved in this path is the OMIM disease identified by MIM:608840. The name (textual description) of this disease is Muscular dystrophy, congenital, type 1D and contains the required substrings “congenital”, “muscular” and “dystrophy”. The instantiated RDF graph with path between glycosyltransferase (GO:0016757) and Muscular dystrophy, congenital, type 1D is shown in Figure 6.

OMIM name has textual description

Figure 4 - RDF graph corresponding to the query above SELECT distinct t,g,d FROM TABLE(SDO_RDF_MATCH( '(?t is_a GO:0016757) (?g has molecular function ?t) (?g has_associated_phenotype ?b2) (?b2 has_textual_description ?d)', SDO_RDF_Models('entrez_gene'), SDO_RDF_Rulebases('entrez_gene_rb'), SDO_RDF_Aliases(SDO_RDF_Alias('','')), null) ) where ( REGEXP_LIKE(LOWER(d), '((.)*(congenital)(.)*)') AND REGEXP_LIKE(LOWER(d), '((.)*(muscular)(.)*)') AND REGEXP_LIKE(LOWER(d), '((.)*(dystrophy)(.)*)'));

is_a GO:0008375

GO: 0016757 has m olecula

r functi on

EG:9215

pe henoty soc. p has as

MIM:608840

Figure 5 - Example of SPARQL query (simplified)

Muscular dystrophy, congenital, type 1D

has textual description

The query can be understood as finding a path in the RDF graph using a predetermined set of semantic relationships and would be formulated as follows. Because of the inference rules implementing the transitivity and reflexivity of the is_a relationship, the condition on the GO annotation “glycosyltransferase (GO:0016757) or any of its descendants” is easily expressed by ‘?t is_a GO:0016757’. The link between genes and GO terms is expressed by ‘?g has_molecular_function ?t’. Similarly, the link between genes and OMIM diseases is expressed by ‘?g has_associated_phenotype ?b2’ (OMIM ID) and ‘?b2 has_textual_description ?d’ (disease name). Finally, direct constraints are put on the GO term on the one hand (‘?t is_a GO:0016757’, to select glycosyltransferase (GO:0016757)) and on disease names on the other (where a regular expression is used to select disease names containing the strings “congenital”, “muscular” and “dystrophy”). The actual (but simplified) SPARQL query is shown in Figure 5.

Figure 6 - Instantiated RDF graph This simple SPARQL query provides an easy way of testing the biological hypothesis under investigation, i.e., the existence of a possible link between glycosylation and congenital muscular dystrophy. On manual inspection of the Entrez Gene record, we also note that the given gene may be involved in the development and progression of meningioma through modification of ganglioside composition and other glycosylated molecules in tumor cells.

Discussion Significance In this study, we demonstrated the feasibility of integrating two biomedical knowledge resources through RDF. We also provided anecdotal evidence for the benefits of such integration by showing how glycosyltransferase can be linked to congenital muscular dystrophy. The integrated resource is greater than the sum of its parts as it supports complex queries that could typically not be handled otherwise without tedious manual intervention or customized software applications.

Results One integrated RDF repository for Entrez Gene and GO The subset of Entrez Gene restricted to Homo sapiens (human) and Mus musculus (mouse) as biological sources comprises 99,861 gene records. Once converted to RDF, it consists of 772,530 triples. The RDF version of GO contains 293,798 triples. Overall, there are over one million triples in the store created for this experiment, which is rel-

Integrated resources based on a graph model are particularly important in an exploratory context where researchers need to “connect the dots” in order to validate an hypothesis. This approach also facilitates intuitive

1263

S. S. Sahoo et al. / Glycosyltransferase to Congenital Muscular Dystrophy: Integrating Knowledge from NCBI Entrez Gene and Gene Ontology

hypothesis formulation and refinement. For example, after verifying that glycosyltransferase is linked to congenital muscular dystrophy, our researchers may narrow the focus of their wet lab experiments to only hexosyltransferase out of the potential seven glycosyltransferases. Analogously, they can focus their research on Muscular dystrophy, congenital, type 1D, out of several other diseases.

Acknowledgments

Arguably, the graph data model of RDF resources is more intuitive than the database schemas. In fact, the RDF data model enables us to model the inherent logical relations between entities that mirror the human cognitive model of the real world. Additionally, the RDF data model offers more flexibility than database schemas for accommodating changes to the underlying model.

References

This research was supported in part by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM) and by the Integrated Technology Resource for Biomedical Glycomics (5 P41 RR18502), funded by the National Institutes of Health National Center for Research Resources.

[1} AmiGO: Gene Ontology browser [cited 11/29/06; Available from: http://www.godatabase.org/] [2] BioRDF subgroup: Health Care and Life Sciences interest group [cited 11/29/06; Available from: http://esw.w3.org/ topic/ HCLSIG_BioRDF_Subgroup] [3] Online Mendelian Inheritance in Man, OMIM (TM). McKusick-Nathans Institute for Genetic Medicine, Johns Hopkins University (Baltimore, MD) and National Center for Biotechnology Information, National Library of Medicine [cited 11/29/06; Available from: http:// www.ncbi.nlm.nih.gov/omim/] [4] Resource Description Framework (RDF), [cited 11/29/06; Available from: http://www.w3.org/TR/2004/REC-rdf-primer-20040210/] [5] SPARQL Query Language for RDF. W3C Working Draft 2006 [cited 11/29/06; Available from: http://www.w3.org/TR/rdf-sparql-query.] [6] XML Schema Language Transformation (XSLT) [cited 11/29/06; Available from: http://www.w3.org/TR/xslt] [7] Alexander, N., Ravada S., “RDF Object Type and Reification in Oracle”—Technical White Paper [cited 11/29/06; Available from: http://download-east.oracle.com/ otndocs/tech/semantic_web/pdf/rdf_reification.pdf] [8] Bodenreider O, Rindflesch TC. Advanced library services: Developing a biomedical knowledge repository to support advanced information management applications. Technical report. Bethesda, Maryland: Lister Hill National Center for Biomedical Communications, National Library of Medicine; September 14, 2006. [9] Maglott D, Ostell J, Pruitt KD, Tatusova T. “Entrez Gene: gene-centered information at NCBI”, Nucleic Acids Res. 2005 January 1; 33(Database Issue): D54–D58. [10] McBride, B.. Jena: A Semantic Web Toolkit. IEEE Internet Computing 2002;6, 6 (Nov. 2002), 55-59. [11] Sheth AP. “Changing Focus on Interoperability in Information Systems: From System, Syntax, Structure to Semantics”, in Interoperating Geographic Information Systems. M. F. Goodchild, M. J. Egenhofer, R. Fegeas, and C. A. Kottman (eds.), Kluwer, Academic Publishers, 1999, pp. 530. [12] Shvaiko P, Euzenat J. 2005. A survey of schema-based matching approaches. Journal on Data Semantics 4: 146-71. [13] The Gene Ontology Consortium. Gene Ontology: tool for the unification of biology. Nature Genetics 2000;25: 25-29.

Generalization The integration approach demonstrated in this study can be generalized to more complex queries and to additional information sources. For example, many additional constraints can be easily added to the query presented earlier by exploiting other properties represented in GO or EG. Examples of such constraints include restricting the annotations to specific evidence codes (e.g., TAS) and narrowing the query to a specific model organism. Only two resources are currently integrated in our RDF store. However, this approach can be generalized to other resources including pathway databases, microarray resources, disease ontologies and virtually all the structured knowledge bases currently under the umbrella of the Entrez system, including UniGene and HomoloGene. Knowledge extracted from unstructured sources such as the biomedical literature can also be integrated. Creating such an extensive repository of biomedical knowledge is one of the goals of the Advanced Library Services project under development at NLM. Unresolved issues and challenges In addition to scalability issues, which can be addressed by mature software and the next generation of hardware, challenges include the identification and organization of entities and relationships. Heterogeneous resources can interoperate in a RDF graph only if the entities shared by these resources are identified consistently. The namespace provided by the UMLS is expected to play an important role for the permanent identification of biomedical entities. In contrast to entities for which organizational schemes currently exist (terminologies and ontologies), the named relationships used to connect data entities during the conversion of EG from XML to RDF are currently not formalized in an ontology of relationships. As a consequence, only limited reasoning can be supported by the RDF graph. As sizeable ontologies of relationships become available, they too will be used for normalizing knowledge in our repository. RDF schemas and OWL will also be investigated.

Address for correspondence Olivier Bodenreider, National Library of Medicine 8600 Rockville Pike, MS 3841, Bethesda, MD 20894, USA. Email: [email protected]. Phone: (301) 435-3246.

1264

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Building a Research Model for Human Genetic Variation Knowledge Management Yulong Gua, James Warrena a

Department of Computer Science, University of Auckland, Auckland, New Zealand

genetic variation and environment in the origins and characteristics of human populations and causes, diagnoses, treatments and prevention of the disease; HGV scientists come from various domains, e.g. genomics, proteomics and clinical sciences [9]. The clinical and laboratory genetic services test and interpret the HGV of patients and/ or families [10]. There were over 500 HGV testing laboratories in the United States at 1997 [11]. The HGV testing is becoming a routine procedure in clinics and research, with at least 751 active laboratories and 936 clinical chemistry/ hematology centers at 2004 in the European Union alone [12]. An international survey at 2005 shows that 45% of early breast cancer patients discuss genetic testing with their physician and/or are referred to see a genetic counselor and 16.7% are then tested [13]. However, despite the significant growth of HGV knowledge, KM success is seldom reported:

Abstract Organizational knowledge management (KM) research studies the nature of knowledge, the scope of KM, the factors and mechanisms that affect KM outcomes, as well as theoretical KM frameworks. This paper discusses the implications of past studies for the KM efforts in the human genetic variation (HGV) research domain and presents a HGV-KM research model. This model identifies the context of HGV KM studies, the predispositions and factors that may impact KM outcomes, and important KM processes. It also represents the relationships among these issues. Applying the model, further studies will point the way for improved capture and dissemination of HGV knowledge from routine HGV research activities to contribute to the global genetics knowledgebase. Keywords: Knowledge Management (KM); knowledge processing; human genetic variation (HGV) research; KM approach; KM framework; KM process; HGV-KM research model

i) for managing high-quality knowledge stocks. Enormous efforts are put into storing HGV data in Locus Specific Databases (LSDB) and general databases, such as HGVbase [14], UMD LSDB [15], OMIM Database [16] and the proposed Central Database plus WayStation Submission tool [17-20]. However, the validity of data in these databases is of some concern, possibly due to inadequate data curation [21, 22]. ii) for supporting knowledge flow and enhancing knowledge processes. For example, although HGV testing laboratories collect and produce a lot of data with HGV details, knowledge flow from these laboratories to HGV research community is not occurring often [10, 23]. iii) for building core competencies and creating intellectual capital. With no attempt to improve the present “pattern” [24] of knowledge processing in HGV research facilities, the research ability and intellectual capital are not being managed or changed by KM efforts. With above KM problems, we apply KM theories to identify all the relevant factors; then, we develop a HGV-KM research model to present the anatomy among the significant issues.

Introduction Knowledge Management (KM) theories Knowledge is a fluid mix of framed experience, values, contextual information and expert insight; it includes both explicit knowledge that is transmittable in formal systematic language and tacit knowledge that has a personal quality and is hard to formalize and communicate [1-3]. Different views of knowledge lead to different perceptions of knowledge management (KM) [4, 5]. For instance, (i) if knowledge is an object or information access, KM should focus on building and managing knowledge stocks; (ii) if knowledge is a process, the focus of KM is knowledge flow and knowledge processing – knowledge creation, acquisition, codification, retention, storage/ retrieval, integration, coordination, transfer, sharing, distribution, application, valuation and use; (iii) if knowledge is viewed as an organizational capability, then KM centers on building core competencies, understanding the strategic advantage of know-how, and creating intellectual capital [6-8]. Human Genetic Variation (HGV) context

Implications of past literature

Human genetic variation (HGV) research aims to characterize the nature, distribution and evolution of genetic variations in humans and to study the relationship between

We synthesize past literature from leading Organizational Management journals (e.g., Organization Science and Harvard Business Review), Information Systems (IS)

1265

Y. Gu et al. / Building a Research Model for Human Genetic Variation Knowledge Management

periodicals, (e.g., MISQ and ISR), HGV journals (e.g., Human Mutation and Nucleic Acids Research), KM or HGV proceedings and books, then present the significant implications in this section.

cess capability' – with the latter being influenced by contingent knowledge tasks [32]. Knowledge management systems (KMS) are a class of IS to manage organizational knowledge, and to enhance knowledge processes [6, 33]. Developed upon IS Success Model [34, 35], Figure 1 shows how the individual’s and organization’s performance at workplace are improved from using KMS [36-38].

KM approaches in practice Most recorded KM practice took the product-centric or process-centric approach, reflecting type (i) or (ii) KM focus [6]; however, capability-centric KM exercise (type three) is rarely reported. The product-centric KM manage knowledge as an objective organizational asset [4, 25]. It relies on the transformation of implicit or explicit knowledge from employees’ heads to written information in documents and the subsequent management of these documents [26]. Accordingly, by using searchable document repository and content management systems [7], HGV knowledge can be captured, stored, retrieved and distributed in well-organized research documentations. On the other hand, Process-centric KM views knowledge as residing with a person and/or a business process. It provides pointers to experts [25] and implements business process management [27], by adopting database of experts, decision aids and expert system, workflow management system, groupware, the systems supporting ‘Community of Practice’ and ‘hardwiring’ of social networks, etc. [6, 28]. HGV studies require profound knowledge on the subjects and methodologies. Therefore, research done around the globe is frequently referred; and international collaborations are often performed. Taking these natures of HGV research into account, the key to managing HGV knowledge is to share it among the researchers [26], applying process-centric KM. Meanwhile, such KM approach may trigger the benchmarking, reengineering and optimizing of the HGV research processes. By tracking and sharing HGV research activities, instance decision making in single variant interpretation may be transited into best practice in studying the gene. This transition may eventually enhance an HGV research methodology and become a valuable intellectual capital, since medical data analysis may discover new models, into which available knowledge could be incorporated [29]. Thus, approaches in managing HGV knowledge may take all product-, process-, and capability-centric endeavors.

Figure 1 – KMS Success Model [36]

HGV-KM research model To manage knowledge in HGV context, the practical questions are “which activities are the promising targets for KM support?” and “what are the nature and benefits of effective KM support?” An HGV KM study is to point the way for improved knowledge capture and dissemination from routine research activities to contribute to the global genetics knowledgebase. Drawing on past literature, we develop a research model (Figure 2) with nine significant KM issues: I. KM context, II. KM process, III. knowledge process capability, IV. contingent task characteristics, V. technology and system quality, VI. knowledge and information quality, VII. perceived benefits and user satisfaction, VIII. knowledge infrastructure capability, and IX. KM outcome. The construct relationships are direct impacts (as arrows from cause to result), moderating effects on a relationship’s strength (by broken arrows) and weak connections (e.g. the quality of KM technologies as the extent to which knowledge processes and KM processes are computerized and integrated [36], represented by curves in the figure). In detail:

KM frameworks in literature Following KM approach review, this subsection extracts the implications of KM models as they offer best KM practices.

I. Context of a KM initiative or project [8] According to [8], the predispositions of a HGV KM project include (i) the properties of units, e.g. a researcher’s academic status and an organization’s social status; (ii) the properties of relationships between units, e.g. the contact frequency between dyadic units and the connection pattern among multiple units; and (iii) properties of knowledge, including explicit HGV data and tacit know-how in HGV research, external and internal knowledge, and uniquely possessed and public knowledge.

The result of knowledge creation, retention and transfer is affected by the properties of organization units, of unit relationships and of knowledge itself; and this effect is moderated by three key causal mechanisms – the three important KM processes: ability, motivation and opportunity [8]. Based on organizational capability perspective theory [30] and contingency perspective theory [31], two more preconditions for effective KM are discovered – the 'knowledge infrastructure capability' and 'knowledge pro-

II. KM process [8] A KM process presents what a KM project offers and how well it functions to enhance an organization’s (i) KM abil-

1266

Y. Gu et al. / Building a Research Model for Human Genetic Variation Knowledge Management

ity to codify implicit knowledge and produce information that makes sense to people other than the author [28], (ii) knowledge sharing motivation – the rewards and incentives, (iii) opportunity, and (iv) from process-centric KM perspectives, the management of HGV research activities and processes.

High-quality knowledge and information are complete, accurate, current (of linkages), informative, rich in expression and in detail [34, 37]. For example, a valid HGV result has to offer reliable and sufficient evidence for variant interpretation based on accurate and unambiguous variant description.

III. Knowledge process capability [32]

VII. Perceived benefits and user satisfaction [36]

This organizational competency is the result of contextual factors and efforts (Constructs I and II) [8]; it then decides the quality of processed knowledge (Construct VI), according to product-centric KM theories and KMS Success Model [36]. It also has direct impact on overall KM effectiveness and this impact is moderated by knowledgerelated tasks [32].

Perceptions on the benefits of KM technologies include perceived usefulness and ease-of-use, both of which are significant predictors of technology acceptance – the actual levels of system usage and user satisfaction [42]. This perspective of users is a result of knowledge quality; and it delivers ultimate KM outcomes in the organization [36, 38].

IV. Contingent knowledge tasks’ characteristics [32]

VIII. Knowledge infrastructure capability [32]

The characteristics of knowledge tasks (such as task content and task domain) decide if the right knowledge is captured and used [38]. An example of task domain is about knowledge creation tasks that may belong to any of the four modes of socialization, externalization, combination and internalization [39]. The moderating role of this construct suggests that the KM efforts that precisely suit a task provide more effective results than those that don’t [31, 32].

This capability represents the organization’s social capital – the network of relationship; and it delivers KM results through knowledge sharing via the network [32]. It is operationalized by (i) technologies (Construct V), (ii) the organizational structure that provides the relationships (i.e. is an organization’s property within Construct I) and (iii) the culture that provides a shared context (as a relationship’s property in Construct I). IX. Ultimate KM outcomes [36]

V. Technology and system quality [36]

KM outcomes include the KM project-improved organizational effectiveness – such as the ability to innovate and coordinate [30] – and individual/organizational performance [38]. The individual KM performance, as measured by 'correctness of decision' and 'confidence in decision', will in turn have an impact on the organization’s performance, e.g. on product quality [34, 36]. As a KM project’s outcomes, the KM efforts may cause positive or negative consequences that will trigger more or less use of the knowledge and the KMS [38].

In addition to the quality of general IS, such as 'ease of learning', 'integration of systems', and quick 'system response time' [34], KMS quality has three more dimensions: (i) the technological resources – the ability to develop, operate, and maintain a KMS, (ii) KMS form – the extent to which organizational memory and KM processes are computerized and integrated, and (iii) KMS level – the ability to bring past information to bear upon current activities [36, 37]. Given the teamwork nature of HGV research, quality KM technology should also support an HGV research facility’s social capital (Construct VIII) by facilitating collaborations, distributed learning, knowledge mapping and opportunity generation [30, 40, 41].

Discussion From past KM approaches and frameworks that recorded the nature of knowledge, KM scope, KM factors and mechanisms, we identify nine categories of contextual,

VI. Knowledge and information quality [36]

Figure 2 – HGV-KM Research model

1267

Y. Gu et al. / Building a Research Model for Human Genetic Variation Knowledge Management

cultural, structural, managerial, cognitive and technological issues that might be critical to the genetics research domain. Then we develop an HGV-KM research model presenting the dynamics among these nine constructs. There are a number of KM models proposed by both academics and practitioners that are not included in this paper, e.g. [43], [27], [24]. The reason for not covering them is that they focus more on single KM approach (product/process/capability-centric), but the HGV setting may have to apply all three strategies. Another limitation of our model is that it still needs validation, for instance, by empirical testing. However, it has revealed a promising research area that is to seek resolutions for the KM issues in genetics.

References [1] Davenport TH, and Prusak L. Working Knowledge: How Organizations Manage What They Know, Cambridge, MA: Harvard Business School Press, 1997. [2] Polanyi M. The Tacit Dimension, London: Routledge & Kegan Paul, 1966. [3] Nonaka I. A dynamic theory of organizational knowledge creation. Organization Science 1994; 5 (1): 14-37. [4] Schultze U, and Leidner DE. Studying knowledge management in information systems research: discourses and theoretical assumptions. MIS Quarterly 2002; 26 (3): 213-242. [5] Carlsson SA, El Sawy OA, Eriksson I, and Raven A. Gaining competitive advantage through shared knowledge creation: in search of a new design theory for strategic information systems. In Proceedings of the Fourth European Conference on Information Systems, Lisbon, 1996; pp. 1067-1075. [6] Alavi M, and Leidner DE. Review: knowledge management and knowledge management systems: conceptual foundations and research issues. MIS Quarterly 2001; 25 (1): 107-136. [7] Sambamurthy V, Bharadwaj A, and Grover V. Shaping agility through digital options: reconceptualizing the role of information technology in firms. MIS Quarterly 2003; 27 (2): 237-263. [8] Argote, L, McEvily B, Reagans R. Managing knowledge in organizations: An integrative framework and review of emerging themes. Management Sci 2003; 49(4 Special Issue) 571–582. [9] The Human Genome Organisation (HUGO). Mission Statement. 2006; http://www.hugo-international.org/ mission.htm, accessed at 2006-11-20. [10] ITPerspectives Inc. NHS genetic service information systems output based specification. 2005; http:// www.ngrl.org.uk/Manchester/Pages/Downloads/ITPGEN3-OBSv05.pdf, accessed at 2005-11-03. [11] Holtzman NA, Watson MS, eds. Promoting safe and effective genetic testing in the United States. Bethesda (MD): National Institutes of Health; 1997; http:// www.genome.gov/10001733, accessed 2006-12-3. [12] Ibarreta D, Elles R, Cassiman JJ, Rodriguez-Cerezo E, Dequeker E. Towards quality assurance and harmonization of genetic testing services in the European Union. Nat Biotechnol 2004; 22:1230-1235. [13] Brown KL, Hutchison R, Zinberg RE, and McGovern MM. Referral and experience with genetic testing among women with early onset breast cancer. Genetic Testing Dec 2005; 9 (4): 301-305. [14] The Karolinska Institute (Sweden) and the European Bioinformatics Institute (UK). Human Genome Variation Database. 2004; http://hgvbase.cgb.ki.se/, accessed at 200511-18. [15] Laboratory of Human Genetics Montpellier (France). The Universal Mutation Database (UMD) Locus Specific Databases. 2005; http://www.umd.be/LSDB.html, accessed at 2005-11-18. [16] National Center for Biotechnology Information (NCBI). OMIM – Online Mendelian Inheritance in Man. 2005; http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=OMIM, accessed at 2005-11-18. [17] Horaitis O, and Cotton RGH. The challenge of collecting mutations across the genome: The Human Genome Variation Society approach, Hum Mutat 2004; 23: 447-452.

Our current research direction is to validate the model proposed herein through iterative action research. We plan to evaluate the impact of IT on various KM processes and capabilities, as well as on overall KM outcomes. This will also validate the model constructs and the model dynamics in terms of construct relationships; for instance, we are expecting to discover significant correlations between the features of a KM project (including implementation of a KMS) and resulting changes in KM performance. In addition to triggering more or less use of the knowledge and the KMS, feedbacks from KM effectiveness (Construct IX) might contribute to subsequent increases of user satisfaction, and possible establishments of KM-related organizational capabilities, and even changes in the contextual properties, such as the contact frequency and connection pattern in the social network. In conclusion, longitudinal studies on HGV KM practice may further refine our research model and add more insights on its anatomy.

Conclusion Organizational knowledge management (KM) aims at effectively building and managing knowledge stocks, supporting knowledge flow and knowledge processes, building core competencies and creating intellectual capital. In the human genetic variation (HGV) research domain, KM efforts might improve the capture and dissemination of knowledge from routine HGV research activities to contribute to the global genetics knowledgebase. By synthesizing past literature, we have developed a KM research model with nine significant constructs; and we hope this paper will cast a light on future knowledge management research in the genetics domain. Acknowledgments We would like to thank many people for their support to this PhD project in the University of Auckland (UoA), especially the inspirational co-supervisor – Dr. Alexei Drummond, and our dearest colleagues Ms Karen Day, Dr. Martin Orr and Dr. Rekha Gaikwad. We are also indebted to Dr. Graeme Suthers, Prof. Andrew Shelling, Ms Deborah Prendergast, Ms Clare Stones and all the research students in the UoA Medical Genetics Group for their precious information and participation.

1268

Y. Gu et al. / Building a Research Model for Human Genetic Variation Knowledge Management [18] Scriver CR, Nowacki PM, and Lehväslaiho H. Guidelines and recommendations for content, structure and deployment of mutation databases. Hum Mutat 1999; 13: 344-350. [19] Scriver CR, Nowacki PM, and Lehväslaiho H. Guidelines and recommendations for content, structure, and deployment of mutation databases II: Journey in progress. Hum Mutat 2000; 15: 13-15. [20] Genomic Disorders Research Centre. Welcome to the Waystation. 2006; http://www.centralmutations.org, accessed at 2006-11-20. [21] Gu Y, Stanek J, and Warren J. Knowledge management (KM) technologies in medical genetic testing (MGT) laboratories – a literature review. In Proceedings of Health Informatics Conference (HIC 2006), Sydney, Australia, August 20-22, 2006. [22] Fredman D, Munns G, Rios D, Sjöholm F, Siegfried M, Lenhard B, Lehväslaiho H, and Brookes AJ. HGVbase: a curated resource describing human DNA variation and phenotype relationships. Nucleic Acids Research 2004; 32(Database-Issue): 516-519. [23] Gu Y, Warren J, Stanek J, and Suthers G. A system architecture design for knowledge management (KM) in medical genetic testing (MGT) laboratories. In Proceedings of the 10th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2006), Nanjing, China, May 3-6, 2006. [24] Firestone JM, and McElroy MW. Doing knowledge management. The Learning Organization 2005; 12 (2): 189-212. [25] Dennis AR, and Vessey I. Three knowledge management strategies: knowledge hierarchies, knowledge markets and knowledge communities. MIS Quarterly Executive 2005; 4 (4). [26] Bossen C, and Dalsgaard P. Conceptualization and appropriation: the evolving use of collaborative knowledge management. In Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility CC '05, August 2005, pp. 99-108. [27] Massey AP, Montoya-Weiss MM, and O'Driscoll TM. Knowledge management in pursuit of performance: insights from Nortel Networks. MIS Quarterly 2002; 26 (3): 269-289. [28] Brown JS, and Duguid P. Organizing knowledge. California Management Review 1998; 40 (3): 90-111. [29] Zupan B, Holmes JH, and Bellazzi R. Knowledge-based data analysis and interpretation. Artificial Intelligence in Medicine 2006; 37 (3): 163-165. [30] Gold AH, Malhotra A, and Segars AH. Knowledge management: an organizational capabilities perspective. Journal of Management Information Systems 2001; 18 (1): 185-214. [31] Becerra-Fernandez I, and Sabherwal R. Organizational knowledge management: a contingency perspective. Journal of Management Information Systems 2001; 18 (1): 23-55.

[32] Lindsey, K., “Measuring Knowledge Management Effectiveness: A Task-Contingent Organizational Capabilities Perspective,” Eighth Americas Conference on Information Systems, 2002, pp. 2085-2090. [33] Ruggles RL. Knowledge tools: using technology to manage knowledge better. Cambridge, MA: Cap Gemini Ernst & Young, Center for Business Innovation, 1997; http:// www.cbi.cgey.com/pub/docs/KnowledgeTools.pdf, accessed at 2001-09-01. [34] DeLone WH, and McLean ER. Information systems success: the quest for the dependent variable. Information Systems Research 1992; 3 (1): 60-95. [35] DeLone WH, and McLean ER. Information Systems Success Revisited. In Proceedings of HICSS, 2002, pp. 238-248. [36] Jennex ME, and Olfman L. A knowledge management success model: an extension of DeLone and McLean’s IS success model. In Proceedings of Ninth Americas Conference on Information Systems, August 2003. [37] Jennex ME, and Olfman L. Organizational memory/ knowledge effects on productivity, a longitudinal study. In Proceedings of 35th Hawaii International Conference on System Sciences, HICSS35, IEEE Computer Society, January 2002. [38] Jennex ME, and Olfman L. Assessing knowledge management success/effectiveness models. In Proceedings of the 37th Hawaii International Conference on System Sciences, 2004. [39] Nonaka I. The knowledge-creating company. Harvard Business Review 1991; 96-104. [40] Grant R. Toward a knowledge based theory of the firm. Strategic Management Journal, Winter 1996; 17: 109-122. [41] Leonard D. Wellsprings of Knowledge: Building and Sustaining the Source of Innovation. Boston: Harvard Business School Press, 1995. [42] Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of infoamtion technology. MIS Quarterly 1989; 13 (3): 319-340. [43] Bots PWG, and de Bruiin H, Effective Knowledge Management in Professional Organizations: Going by the rules. In Proceedings of the 35th Hawaii International Conference on System Sciences, 2002; IEEE Computer Society Press

Address for correspondence Yulong Gu, Department of Computer Science (Tamaki), University of Auckland, Private Bag 92019, Auckland, New Zealand. [email protected]

1269

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

ECTracker – An Efficient Algorithm for Haplotype Analysis and Classification 1

Li Lin, 2Limsoon Wong, 3Tze-Yun Leong, 4Pohsan Lai

1;2;3 School of Computing, National University of Singapore Dept of Pediatrics, National University Hospital, National University of Singapore

4

extracted patterns are used for predictive inference (or classification) to help in carrier detection, which is useful for medical prognosis and decision making.

Abstract This work aims at discovering the genetic variations of hemophilia A patients through examining the combination of molecular haplotypes present in hemophilia A and normal local populations using data mining methods. Data mining methods that are capable of extracting understandable and expressive patterns and also capable of making predictions based on inferences made on the patterns were explored in this work. An algorithm known as ECTracker is proposed and its performance compared with some common data mining methods such as artificial neural network, support vector machine, naive Bayesian, and decision tree (C4.5). Experimental studies and analyses show that ECTracker has comparatively good predictive accuracies in classification when compared to methods that can only perform classification. At the same time, ECTracker is also capable of producing easily comprehensible and expressive patterns for analytical purposes by experts.

In this paper, we present the design and implementation of the ECTracker method. We also examine its performance as compared to common data mining methods in supporting the targeted tasks. Specifically, we compared the expressiveness of the haplotype patterns discovered using ECTracker with the haplotype patterns discovered using the Decision Tree method (C4.5). Furthermore, we also compared the classification predictive accuracy of ECTracker with existing classification methods including Artificial Neural Network, Naïve Bayesian Network, Support Vector Machine and Decision Tree (C4.5) [2][3].

The ECTracker method There are two main steps in ECTracker. First, it identifies the genetic variations (or haplotype patterns) of hemophilia A patients to help analyze FVIII gene polymorphism for linkage analysis. Second, the haplotype patterns found in the first step are used to perform classification to facilitate carrier screening by medical practitioners. Details of the hemophilia A dataset will be introduced in the next section.

Keywords: datamining, classification, hemophilia A, genetic variations, haplotypes

Step 1 – Finding interesting patterns

Introduction

The first step of the ECTracker algorithm uses a level-wise neighborhood search method to enumerate all possible marker patterns of length one, two, and three etc, and then computes the statistical odds ratio of each of the patterns. Only those patterns that are significant are selected. The significance of a potential/candidate pattern is determined by computing its p-value. P-value calculates the probability due to chance alone of getting a difference larger than or equal to that actually observed in the data [4] [5]. A small p-value means it is difficult to attribute the observed difference to chance alone, and this can be taken as evidence against the null hypothesis of non-significance.

In this paper, we propose a new algorithm, called ECTracker1, for pattern extraction and classification of a specific type of biological dataset known as haplotypes. A total of 47 patients affected by hemophilia A and 47 matched normal controls from Singapore were genotyped with a set of markers located on chromosome Xq28 which tags the hemophilia A disease gene. Hemophilia A is an Xlinked recessive bleeding disorder that results from deficiency and/or abnormality of coagulation factor VIII (FVIII) [1]. The FVIII gene spans 186 kb of DNA and resides on 0.1% of the X chromosome (band Xq28).

Odds ratio is a test statistic that has been widely used in the biomedical arena to measure the magnitude of association between two categorical variables based on some data collected [6] [7]. Given a pattern x, odds ratio computes the ratio of non-association between x and the label L, to the association between x and L based on a set of data. For example, given a pattern, say (1,3), and there are V number of such pattern found in a dataset D associated with the class label Abnormal and S number of such pattern found in D associated with class label Normal. We are interested

We are interested in methods that are capable of performing the two tasks efficiently – first to extract expressive patterns for descriptive analysis, and second to perform classification. Intuitively, expressive haplotype patterns (or genetic variations) need to be extracted to provide medical practitioners with insights about the genetic manifestations of patients affected by hemophilia A. The 1

Initial findings of this work were presented as a poster in Asia-Pacific Conference on Human Genetics, 2004.

1270

L. Lin et al. / ECTracker – An Efficient Algorithm for Haplotype Analysis and Classification

ensures that we choose the pattern with the shortest pattern length for each p-value.

in finding out whether the marker pattern (1,3) is strongly associated with the label abnormal. Table 1 shows the contingency table for our example where P is the number of samples in the dataset associated with the class label Abnormal and N is the number of samples in the dataset with class label Normal. The odds ratio is computed based on equation 1 defined below.

Step 3: Select the first n patterns from sorted R following the sorted sequence to form the set ƒ for classification. Step 4: Perform classification on the training data D using the n pattern classifier ƒ and compute the true positive rate of the prediction.

Table 1 – 2x2 contingency table Abnormal

Normal

not(1,3)

P-V

N-S

(1,3)

V

S

 P – V S Odds Ratio, T = --------------------- N – S V

Step 5: If the true positive rate is less than the user defined minimum true positive rate, then repeat Step 3 and Step 4 with a different n value. In classifying an unseen case in Step 4, the first pattern that satisfies the case will classify it. If no pattern applies to the case, a scoring method will be used for each of the classes, where the class with the highest score classifies the case. However, if the scoring method produces the same score for each of the available classes, then the unseen case will take on the default class. The user is able to set the default class to “unknown” to allow the classifier to make no prediction when no pattern applies to the case. This is useful when there are samples that are identical in attribute values but belonging to different classes. Figure 1 shows the pseudocode for scoring the classes.

(1)

Step 2 – Predictive inference / classification The following describes the algorithm for predictive inference using the patterns derived from the previous step. Before presenting the algorithm, let us define the order of precedence of the derived patterns. This is used in selecting patterns for our classifier. Definition: Given two patterns, ri and rj, ri >> rj (also called ri precedes rj or ri has a higher precedence than rj) if 1. The p-value of ri is less than the p-value of rj, the smaller the p-value of a pattern the greater the statistical significance of that pattern. 2. Both patterns have the same p-values and ri  rj, the pattern length of ri is shorter than the length of rj. The pattern with shorter pattern length that can correctly classify an unseen case is preferred. 3. Both patterns have the same p-values and ri Œ rj, but ri is generated earlier than rj. Let R be the set of patterns derived in step 1, and D be the training data used to derive R. The basic idea of the algorithm is to choose a set of high precedence patterns in R as our classifier. The classifier is of the following format: , where ri  R, ra >> rb if b > a. The default_class is the chosen class for an unseen case when no pattern in the classifier could classify the unseen case. The default_class can be selected by the user. However, if the user decides to let the classifier select the default_class, then the majority class in the data D will be chosen as the default_class.

¦ ZC

x

Score(Cx) = 0 for each pattern ri  ƒ do if ri.class == Cx compute ZC

6. 7. 8. 9.

Score(Cx) = Score(Cx) + ZCx end end end

x

Figure 1 - Pseudocode for computing score of each class For each pattern ri in ƒ that classifies a class Cx, computes the individual pattern score using equation 2 as follows: Individual pattern score of ri for class Cx, ZC

x

2

 nummatch = ------------------------------------------------------------------------------------------patternlength – casepatternlength

(2)

Where patternlength refers to the pattern length of ri and casepatternlength refers to the pattern length of the case to be classified, and nummatch refers to the number of attribute matches between ri and the case pattern. The total score for a class Cx is computed as shown in equation 3 as follows:

The algorithm for building the classifier consists of five steps: Total score for class Cx, :c x =

for each class Cx do

1. 2. 3. 4. 5.

The unseen case will take on the class with the maximum : value. The objective of counting partial matches is for better noise handling.

(3)

Step 1: Sort the set of generated patterns R according to the relation “>>”. This is to ensure that we will choose the highest precedence patterns for our classifier.

We now describe how the scoring scheme handles noise with an example. Given that a pattern ABC is significant, its subset AB may or may not be significant since the odds ratio value is neither upward nor downward closed. However, if at a different odds ratio value, it is found that both ABC and AB are the shortest significant patterns for a class C1, this would mean that the attributes A and B are

Step 2: For each pattern r in sorted R, if there exist another pattern r’ such that the p-values of both r and r’ are the same, and r’  r, then remove r from sorted R. This

1271

L. Lin et al. / ECTracker – An Efficient Algorithm for Haplotype Analysis and Classification

Table 3 – Allelic frequencies of Intron 13 (CA)n Repeats

important for determining the attributes for the class C1. Now, if we have some other patterns say ADE and DEF that are significant for another class C2. If we now have a case pattern to classify, say A, then A will have a higher score for class C1. This is the desired effect since the pattern ADE for class C2 may become significant due to noise, but it is less likely for AB and ABC to become significant due to noise.

Allele Frequencies 24

23

22

21

20

19

15

1

2

3

4

5

6

10

This Study

0.01

0.10

0.06

0.26

0.52

0.04

0.01

Reported by [35]

0.013

0.05

0.11

0.29

0.45

0.07

0

Intron 13 (CA)n Repeats

The hemophilia dataset A set of five common PCR-based polymorphisms located on chromosome Xq28 which tags the hemophilia A disease gene were collected and analyzed from 47 patients and 47 matched normal controls. The five polymorphisms collected are two microsatellite repeats in introns 13 and 22, and three Restriction Fragment Length Polymorphisms (RFLPs), namely BclI-intron 18, HindIII-intron 19, and XbaI-intron 22.The exact location of the markers are shown in Figure 2.

Table 4 – Allelic frequencies of Intron 22 (GT)n/(AG)n repeats Allele Frequencies Intron 22 (GT)n/ (AG)n Repeats

This Study Reported by [35]

Figure 2 – Factor VIII gene

The allelic frequencies observed in this study and those reported by Hemophilia A Mutation, Structure, Test and Resources Site [8] are tabulated in Tables 2, 3, and 4. Our results for BclI, HindIII, Intron-13(CA)n and Intron22(GT)n(AG)n are significantly similar to those reported in [35] with F2 < 3.841 for BclI and HindIII, and F2 < 12.59 for Intron-13(CA)n and Intron22(GT)n(AG)n. They are all within 95% confidence interval. However, the frequency for XbaI is significantly different from those reported by [35] with F2 > 3.841.

Allele Frequencies (Reported by [35])

(+)

(-)

(+)

1

2

1

2

BclI

0.22

0.78

0.29

0.71

HindIII

0.78

0.22

0.75

0.25

XbaI

0.56

0.44

0.41

0.59

28

27

26

25

1

2

3

4

5

6

7

0.01

0.01

0.04

0.03

0.09

0.63

0.19

0

0

0

0.013

0

0.667 0.307

For descriptive analysis, we mainly report on the expressive and interesting patterns extracted from the remaining 30% of the dataset. Whereas for classification or predictive analysis, we divide the experiment into two parts: The first part compares the accuracies of the five classifiers based on the full hemophilia dataset. The second part we concentrate our study on the 30% of the dataset where those samples whose disease and normal haplotypes cannot be distinguished were removed.

Table 2 – Allelic frequencies of RFLPs

(-)

29

It was further found that 70% of the samples had exactly the same allele values in all the markers in both patient and normal controls, which means that the 5 markers/attributes in the dataset is insufficient for separating 70% of the samples. After removing those samples whose disease and normal haplotypes cannot be distinguished, there are 28 samples remaining – 18 samples belonging to the disease phenotype and 10 samples belonging to the normal/control phenotype. Tables 5 and 6 show the frequencies of the disease and normal/control haplotypes respectively.

Allelic frequencies

Allele Frequencies (This Study)

30

It was observed that samples with BclI-intron 18 allele 1 were always associated with HindIII-intron 19 allele 2 with F2 p-value < 0.001. This is an expected observation as there are reported linkage disequilibrium between BclI and HindIII alleles from literature such as Ahrens et. al. [9] and EL-Maarri et. al. [10]. The HindIII marker was thus excluded since BclI and HindIII are in linkage disequilibrium, we could easily predict the value of the other attribute base on the value of one attribute, and hence 4 markers are sufficient in for the analysis.

In the next sub-section, we describe the allelic frequencies of Factor VIII gene observed in our local population and the allelic frequencies reported by the authoritative resource website [8] for hemophilia A disease. The reporting of the allelic frequencies of our local population is useful for other medical practitioners not located in Singapore to decide whether they could make use of our discovery of the genetic variations for prognosis and counseling of their patients.

RFLPs

31

1272

L. Lin et al. / ECTracker – An Efficient Algorithm for Haplotype Analysis and Classification

Table 5 – Haplotype frequencies of probands with disease phenotype Marker

Disease Haplotypes

Intron-13 (CA)n

3

4

4

4

4

4

5

5

BclI

1

2

2

2

2

1

2

2

XbaI

1

1

1

2

2

1

2

Intron-22 (GT)n/(AG)n

3

1

3

3

5

7

No. of Probands

1

1

2

1

1

6

Expressive patterns derived by ECTracker

Total 5

10

2

1

2

1

2

1

1

2

4

5

6

6

1

2

1

1

1

18

Table 6 – Haplotype frequencies of probands with normal/ control phenotype Marker

Normal/Control Haplotypes

This experiment shows that as compared to the decision tree approach of C4.5, ECTracker is capable of deriving useful patterns even when the dataset is very small.

Total

Intron-13 (CA)n 1

2

2

3

3

4

4

5

6

BclI

1

1

2

1

1

2

2

2

2

XbaI

1

1

2

1

1

1

1

1

2

Intron-22 (GT)n/ 7 (AG)n

5

6

5

7

4

5

7

6

No. of Probands 1

1

1

1

2

1

1

1

1

The longest most significant pattern associated with the disease phenotype derived by ECTracker is 4117 (Intron13(CA)nBclIXbaIIntron22(GT)n(AG)n). This is an interesting observation as the haplotype occurs in 33.3% of the disease phenotype and 0% of the normal/control phenotype with F2 > 3.841 and odds ratio T = 0, which means that such observation occurs significantly greater than by chance. From Table 5, the haplotype occurs in 6 probands with disease phenotype as compare to other haplotypes which occur in no more than 2 probands. The shortest most significant patterns derived by ECTracker are 4**7 or 41** with F2 > 3.841 and odds ratio T = 0. This means that two markers alone were sufficient to define the disease haplotype, however, the longest most significant pattern provides a useful insight for the medical practitioners or scientists who seek to better understand the genetic variations of the disease.

Classification of the hemophilia A dataset There are a total of 94 records in the hemophilia dataset, 47 records belonging to the class patient and 47 records belonging to the class normal. The classification methods that we examined include C4.5, Naïve Bayesian Classifier, Neural Network, Support Vector Machine and ECTracker. Except for ECTracker, all the other four classification algorithms are available from the package WEKA. WEKA is an open source data mining and machine learning software [2]. Table 7 shows the performance of various classifiers when applied to the full hemophilia dataset, and Table 8 shows the performance of the classifiers when applied to the pruned hemophilia dataset.

10

Results Interesting pattern extraction

Table 7 – Analysis of classifiers based on full hemophilia dataset

Expressive patterns derived by C4.5 C4.5 deduced that haplotype patterns (or genetic variations) of 4***, 5***, or 10*** (Intron13(CA)n BclIXbaIIntron22(GT)n(AG)n) are highly associated with the disease phenotype. This derivation is not very useful as we could see from Table 6 that there are 3 probands with normal/control phenotype having intron-13 (CA)n allele values 4 and 5. Moreover, allele value 10 in intron-13 (CA)n only occurs once in the proband with disease phenotype (from Table 5), hence it is not able to give a generalize conclusion base only on allele value 10 of intron-13 (CA)n.

Accuracy Precision Recall Precision Recall of Class for Class for Class for Class Patient Patient Normal Normal

The possible reason for such a deduction by C4.5 may be due to the problem that the dataset is very small, and as a result the selection for partitioning attribute becomes biased for those attributes with more attribute values. Hence attributes with more attribute values will be assigned higher information gain as compared to attributes with fewer attribute values.

1273

C4.5

71.43%

0.708

0.944

0.75

0.3

Naïve Bayesian Network

64.29%

0.7

0.778

0.5

0.4

Artificial Neural Network

78.57%

0.833

0.833

0.7

0.7

Artificial Neural Network

71.43%

0.75

0.833

0.625

0.5

Support Vector Machine

82.14%

0.842

0.889

0.778

0.7

L. Lin et al. / ECTracker – An Efficient Algorithm for Haplotype Analysis and Classification

Table 8 – Analysis of classifiers based on pruned hemophilia dataset

In descriptive analysis, ECTracker is capable of extracting comprehensible and useful patterns from the hemophilia A dataset. Comparing with the patterns derived by C4.5, the patterns derived by C4.5 are less useful, as described earlier.

Accuracy Precision Recall Precision Recall for Class for Class for Class for Class Patient Patient Normal Normal C4.5

62.77%

0.615

0.681

0.643

0.574

Naïve Bayesian Network

62.77%

0.615

0.681

0.643

0.574

Artificial Neural Network

64.89%

0.675

0.574

0.63

0.723

Support Vector Machine

63.83%

0.623

0.702

0.659

0.574

ECTracker

67.02%

0.674

0.660

0.667

0.681

In predictive analysis or classification, ECTracker is capable of producing good predictive accuracies in classification that are comparable to those methods that only perform classification such as Artificial Neural Network and Support Vector Machine. The experiments have indicated that ECTracker is potentially an effective method for both pattern extraction and classification for biomedicine in particular, and datamining in general. The approach proposed here provides analysis and classification based on mainly the disease status of an individual. Continuously distributed quantitative traits such as blood pressure and choltesterol level may also be of significance to the clinicians. ECTracker can be extended to perform analysis and classification based on continuously distributed quantitative traits by defining a new scoring method for the interesting patterns. Further investigation will need to be done to assess the feasibility of such extension.

The classifiers were evaluated using 5-fold cross validation for the full dataset and leave-one-out for the pruned dataset. Since for each fold, different data samples are selected it is necessary run both steps in Section 2 fresh to avoid bias. Therefore, the patterns used for classification will vary for each fold.

References [1] Antonarakaris S, Kazazian H, Tuddenham E. Molecular etiology of factor VIII deficiency in hemophilia A. Human Mutation 1995; 5:1-22. [2] Witten IH, Frank E. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann; 2000. [3] Michie D, Spiegelhalter D, Taylor C. Machine Learning, Neural and Statistical Classification. Ellis Horwood: New York; 1994. [4] Ott L. An Introduction to Statistical Methods and Data Analysis. PWS-Kent Publishing Company, Massachussetts; 1988. [5] Theodorou, S.J., Theodorou, D.J. and Kakitsubata, Y. Statistical Analysis in Clinical Studies: An Introduction to Fundamentals for Physicians. Internet Medical Journal, 2004. [6] Le Moual N, Siroux V, Pin I, Kaufmann F, Kennedy SM. Asthma Severity and Exposure to Occupational Asthmagens. American Journal of Respiratory Critical Care Medicine 2005. [7] Deheinzelin D, Lourenco MT, Costa CL, Younes RN. The Level of Nicotine Dependence is an Independent Risk Factor for Cancer: A Case Control Study. Clinics 2005; 60(3):221-226. [8] HAMSTeRS Haemophilia A Mutation, Structure, Test and Resource Site http://europium.csc.mrc.ac.uk/WebPages/ Main/main.htm. [9] Ahrens P, Kruse TA, Schwartz M, Rasmussen PB, Din N.A. New HindIII Restriction Fragment Length Polymorphism in the Hemophilia A Locus. Human Genetics 1987; 76(2):127128. [10] EL-Maarri O, Kavakli K, Caglayan H. Intron 22 Inversions in the Turkish Haemophilia A Patients: Prevalence and Haplotype Analysis. Haemophilia 1999; 5(3):169-173.

ECTracker outperformed other classifiers with higher predictive accuracies on both full and pruned hemophilia A datasets. On the full hemophilia dataset, the predictive accuracy of ECTracker is 67.02%, followed by Artificial Neural Network and Support Vector Machine with predictive accuracies of 64.89% and 63.83% respectively. Both Naïve Bayesian Network and C4.5 have the same predictive accuracy of 62.77%. On the pruned dataset, ECTracker was able to accurately predict the phenotype of a sample given its polymorphic markers 82.14% of the time, followed by Artificial Neural Network at 78.57% of the time. C4.5 and Support Vector Machine were able to make accurate predictions 71.43% of the time. There are 94 records in the unprune hemophilia A dataset, which is reasonably large, and there are 23 records in the pruned hemophilia A dataset which is rather small. Our experiments show that ECTracker is capable of providing good classification accuracy on both small and large datasets when compared to other classification methods.

Conclusion In this work, we explored methods that are capable of extracting understandable and useful patterns, and also capable of performing inference on the patterns to make prediction. We applied these methods to find the genetic variations of a real dataset consisting of patients affected by hemophilia A to facilitate haplotype analysis by medical practitioners. We examined the issues of descriptive and predictive analyses using our proposed method called ECTracker.

Address for correspondence Email: [email protected]

1274

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Dynamic Query System for Supporting Phenotype Mining in Genetic Studies Angelo Nuzzo, Daniele Segagni, Giuseppe Milani, Carla Rognoni, Riccardo Bellazzi Department of Computer Science and Systems, University of Pavía, Pavía, Italy

interest, and the design and implementation of a data warehouse system for analyzing phenotypic data. Moreover, selecting which are the phenotypes of interest determines the subsequent genotyping choices, especially when a genome-wide scan is not feasible or suitable.

Abstract This paper describes an information technology infrastructure aimed at supporting translational bioinformatics studies that require joint management of phenotypic and genotypic data. In particular, we integrated an electronic medical record with an open-source environment for data mining to create a flexible and easy to use query system aimed at supporting the discovery of the most frequent complex traits. We propose a logical formalization to define the phenotypes of interest; this is translated into a graphical interface that allows the user to combine different conditions relative to the electronic medical record data (e.g., the presence of a particular pathology). The phenotypes are then stored in a multidimensional database. Then, the data mining system engine reads the filtered data from the database and executes dynamic queries for analyzing phenotypic data, presenting the results in a multidimensional format through a simple web interface. The system has been applied in a study on genetically isolated individuals, the Val Borbera project.

Once clinical data are collected, it is crucial to perform a series of queries and data aggregation steps to characterize the population and extract the most prevalent phenotypes. However, clinicians, biologists and epidemiologists are usually unable to explore the collected information, because the use of general query languages requires substantial technical skill, as well as knowledge of the underlying database structures. On the other hand, the need of performing “dynamic” queries hampers the implementation of a “standard” user interface for pre-defined queries. To address this problem, we are defining a dynamic query system based on data warehouse and mining concepts. The clinical data are copied into a data mart oriented to data analysis. Thanks to the integration in the overall system of open-source environment for data mining it is possible to design a simple interface for performing aggregation, counting and simple statistics on the majority of the variables contained in the clinical database.

Keywords: phenotype mining, complex traits, intelligent query, clinical data warehouse.

The implementation of the system has a number of steps which follows a workflow targeted at identifying the most important phenotypes which characterize a particular population. Such workflow includes the following steps:

Introduction A specific characteristic of the post-genomic era will be the correlation of genotypic and phenotypic information [1][2]; the emerging discipline of Biomedical Informatics may provide knowledge and tools for dealing with such an ambitious goal [3]. In this context, the studies aimed at the so-called genetic dissection of complex traits represent a first crucial benchmark for Biomedical Informatics and for translational bioinformatics.

1. The development of a relational database collecting clinical data on the target population 2. The translation of the database structure into a multidimensional data-base (data mart) oriented for query and reporting 3. The formal definition of the phenotypes to be searched and studied and their mapping in the database 4. Finally, the design and implementation of the data mining tool to easily extract the phenotypes and analyze their relationships The aim of the paper is to describe the IT infrastructure of the system and the biomedical informatics challenges we are dealing with. Preliminary results on the use of the system to support a study on genetically isolated individuals, the Val Borbera project [4], will also be reported.

The definition of an Information Technology infrastructure to support this kind of studies, and in particular the studies aimed at the analysis of large sets of phenotypes to discover the most prevalent diseases and then to integrate genotypic information, is a challenge which can be considered a paradigmatic goal of Biomedical Informatics. As a matter of fact, research on phenotypes requires the definition of an architecture for data collection, the implementation of an electronic medical record, the development of a system for the definition of the phenotypes of

1275

A. Nuzzo et al. / A Dynamic Query System for Supporting Phenotype Mining in Genetic Studies

Figure 1 - Overview of system’s components an the interactions among the 3 layers schema which is connected to the other tables with multiple joins. Such other tables, called the dimensional tables, only require a single join to be referenced by the fact table [5].

Materials and methods Objective The final purpose of the system that we have developed is to provide a versatile and easy to use data inspection tool to identify which phenotypes may be more successfully investigated in the population under study, so that suitable genotyping choices may be made subsequently. The system accomplishes this purpose by providing tools for two main tasks:

Typically a clinical database can be modeled by a star schema in which each record in the fact table represents a combination of a clinical measure and its values on a specific date for a specific patient. So the dimensions are individuals, measurement time and measurement values: all of them can be further specified using a snowflake model1.

1. for formally define the phenotypes to be investigated by a graphical user interface 2. for exploring clinical data to extract and analyze individuals with the same phenotypes (as they have been previously defined) Both aspects can be performed in a database where the information is stored in a non-normalized data structure, which is often referred to as data mart. As a matter of fact, while a normalized structure is always required for correct management of the database (as it concerns data security, integrity, reliability), the database is greatly facilitated by the use of query-oriented data schema. That’s why the basic common layer of our implemented system is a nonnormalized data structure, the so-called “star schema”, to which the two developed applications for query definition and data assessment are interfaced. Details of the components and their interactions are described in the following paragraphs, while a global overview is given in Figure 1.

The adaptation of the star-schema and of the snowflake models to the clinical context requires however several efforts. In fact, when taking into account the phenotype information in the analysis, we cannot model it as a dimension of the fact table, because it would be a nonadditive dimension with respect to the others. Additivity is the ability to use an aggregate operator (summation, counting, average) along the dimensions of the same fact [6]: in our case, the phenotype dimension would be additive only along the patient dimension, but not along the others (measurements and time), as it is defined by a set of measurements. To overcome this problem, we have defined a new fact table to model the relationship between phenotypes and individuals. The star schema and the phenotype tables are the physical models of the new multidimensional database. They represent two multidimensional “cubes” that together form the logical model of the database. The cubes may be merged in a single “virtual” cube, so that it is possible to use an Online Analytical Processing (OLAP) engine to perform

From the relational DB to the “star-schema” The typical data structure that has become a standard for all data warehouse applications is a multi-dimensional model called the “star join schema”. Unlike the EntityRelation model, the dimensional model is asymmetric. There is one large dominant table in the center of the schema, called the fact table. It is the only table in the

1

1276

A snowflake model is a model in which a given dimension has relationships to other levels of the same dimension. It is used to re-normalize complex dimensions to eliminate redundancy.

A. Nuzzo et al. / A Dynamic Query System for Supporting Phenotype Mining in Genetic Studies

data analysis (described in detail in the following paragraphs).

phenotype definition tables. The wizard is automatically setup by reading an XML file in which the two data structures are encoded. The conditions (attributes and their values) may be defined by combo boxes, which provide lists of attributes according to the measures table of the star schema, suggesting the admissible ranges of values for each attribute. Once the rules are defined, the corresponding SQL string is created by merging conditions by AND operators, in order to: i) store the rules in the phenotype section tables, and ii) select the subgroup of individuals satisfying that conditions and storing the relation individuals-phenotype. In the same way, it is possible to choose another graphical panel to selected some defined phenotypes to be merged together by the OR operators to create a new phenotype, which is stored in the phenotype section.

The phenotype definition tool Clinicians and biologists usually define a phenotype by a set of variables and the values they may take. In order to select (and then to analyze) the individuals satisfying that set of rules, it is necessary to write a suitable SQL statement to run a query to retrieve them. However, as the users may have no expertise in the use of a query scripting language, we provide a tool that automatically generate the proper SQL script to select individuals with the defined phenotype. To perform this task, a formalization of the phenotype definition is needed. The basic assumption is to consider a phenotype as a set of conditions in the form of attribute/ value pairs. Then using logical operators (AND, OR) it is possible to combine different conditions to define more and more complex phenotypes. In particular, the AND operator allows the specialization of a defined phenotype, while the OR operator is used to merge different phenotypes into a single, more comprehensive one. This procedure corresponds to a logical tree construction, in which the nodes are the conditions, the AND operator is used to go from the top to the bottom and the OR operator is used to add an upper node from the bottom to the top (figure 2).

The data analysis tool Dealing with clinical data to analyze phenotypic information implies taking into account heterogeneous data and viewing them at the same time. This means that it should be possible to perform a multidimensional inspection of the dataset. Whereas a relational database stores all data in the form of rows and columns, a multidimensional dataset consists of axes and cells organized in multidimensional “cubes”, the dimensions of which are the directions of drill-down investigations. The technique of multidimensional analysis is implemented in software tools called online analytical processing (OLAP) engines., OLAP, in fact, means analyzing large quantities of data in real-time. Unlike Online Transaction Processing (OLTP), where typical operations read and modify individual and small numbers of records, OLAP deals with data in bulk, and operations are generally read-only. The term “online” implies that even though huge quantities of data are involved — typically many millions of records, occupying several gigabytes — the system must respond to queries fast enough to allow an interactive exploration of the data.

Phenotype 7 OR

Phenotype 5

OR Condition 3 AND

AND

Condition 1

Condition 2

Condition 4

Phenotype 1

Phenotype 2

Condition 7 AND

Condition 5 Phenotype 4

Phenotype 3

AND

Condition 6

OR

As described above, the logical model of the star schema and the phenotype tables consist of two virtual cubes that may be merged in a single one, which is the input for the OLAP engine. In our system we use an OLAP engine written in the Java programming language: Mondrian [7]. It executes queries written in the MDX language (that has actually become a standard for data warehouse applications) [8], reads data from a relational database, and presents the results in a multidimensional format through a Java API, so that the presentation layer may be chosen by the final user. JSP pages are provided by default, so that the user can simply use a web browser for data visualization. The MDX queries have to be defined by the user, so we have developed a specific module (the “MDX generator” box in figure 1) that automatically create the MDX scripts directly from the attributes of interest chosen by the check box lists of the main page.

Condition 8 Phenotype 6

Figure 2 - Formalization of the phenotype definition The limitation of this model is that it is not possible to define any kind of AND/OR combination of the conditions under investigation, as in the corresponding SQL string, it will not be possible to interpret the correct order of the subgroups (i.e., the correct position of brackets in the query string). So it is necessary to define the ANDconditions first, and then combining them using the OR operator. The end users can create these definitions using a graphical wizard developed in the JAVA programming language. It interacts with the two sections of the non-normalized database, i.e., the star schema of the clinical data and the

Sharing and generalizing issues In order to make the graphical wizard usable on different databases, it has been made configurable via an XML file containing the star schema description. So the only prereq-

1277

A. Nuzzo et al. / A Dynamic Query System for Supporting Phenotype Mining in Genetic Studies

uisite for using it is to provide a star schema that is compliant to the model described above. Then the phenotype tables are automatically generated and populated by the GUI. Moreover, the phenotype definitions are also stored in XML files, so that existing phenotypes loading is performed by reading the XML files instead of the tables, and the XML repository may be shared with other scientists interested in analogue analysis. On the other side, whichever OLAP engine will be chosen, the only manual task needed is to code the virtual cube in the specific format required as input by the engine. Using Mondrian, it means to create the XML file containing the definition of the cube.

Results Figure 4 - Main panel: existing phenotypes are shown and details for the selected one

The tools described in the previous section have been tested for the exploration of the clinical database of the Val Borbera genetically isolated population project [4]. This study is conducted in collaboration with the DIBIT of San Raffaele Scientific Institute of Milan, for which we have provided the architectural IT infrastructure for data collecting and storing. The clinical data have been collected in a relational database in a high normal form, actually containing about one hundred clinical measures relative to more than 4000 individuals. So the first step was to create the correspondent star-schema for the multidimensional analysis. Here we present an example regarding the analysis of dysfunctions related to the thyroid.

So if the user wants to specialize an existing phenotype, he can select it from the “Phenotype’s Rules” window (figure 3) and add conditions using the AND operators. Otherwise, if some defined phenotypes have to be merged together by OR operators to create a new phenotype definition, they can be selected using the “generalization panel”, as shown in figure 4. In this case the combo box are filled by reading the XML phenotype files stored in the suitable repository.

Phenotype definition Before the use of a multidimensional analysis approach, the biologists had to ask a technician to extract individuals with the traits of interest by writing the SQL statement to be executed on the relational database. The statements are often some hundreds of lines long, due to the large number of join to be performed. Using the developed infrastructure, in contrast, the user has only to define the conditions by the graphical window shown in figure 3, that will be merged by the AND operators. The combo box are automatically filled in by reading the XML file in which the star schema is encoded.

Figure 5 - The generalization panel In both cases, the selection of individuals with the specified phenotypes is performed, and the relations are stored in the proper table. So the phenotype becomes a dimension which can be used to explore individual’s data. Data analysis and validation When the OLAP engine starts up, it reads the XML file containing the cube definition. The first page shows a set of check boxes containing the fields of the underlying tables, so that the user may choose the variable to be investigated (the phenotypes are among them). Once the features have been chosen, the engine loads information related to the individuals having that phenotype. Then a visual inspection of the measurements values can be done expanding or collapsing cells of the resulting table, so that the analysis can be executed at different levels of detail (figure 6). Automatic graphical reports can also be generated.

Figure 3 - Creation/specialization panel The defined phenotypes are summarized in the main panel of the GUI. The phenotype list shows the XML files that have already been created; details of the rules applied to define it are given below by selecting one of them (figure 5).

1278

A. Nuzzo et al. / A Dynamic Query System for Supporting Phenotype Mining in Genetic Studies

All the modules of the system are configurable via XML files, so that they can be reused to analyze other clinical databases. The only requirement is to translate the data structure into the data mart described in this paper, and to codify it in an XML file. Moreover, the phenotype definitions are also stored in an XML repository, so they may be reused and shared with other users to compare results. The system as been tested on a real dataset, the clinical database of the Val Borbera project, showing that it is easy to use and time-saving. Future development of the system will improve the graphical user interface. The phenotype definitions will be shown by graphs, corresponding to the logical model used to create it, so that the user may expand it directly by adding or removing nodes.

Figure 6 - The resulting dynamic table provided by Mondrian: the user may expand or collapse the cells to view the data at different levels of detail A first validation of the system is now undergoing; it consists of verifying how it improves the phenotype searching process. Biologists formulate several verbal requests, we then compared the results obtained by themselves using the system and the ones obtained by a technician performing a SQL statement (which is typically quite a lot of instruction lines long). The first initial report table is always the same for both tasks and for any condition exploited (many different phenotypes corning thyroid diseases, hypertension and diabetes have been defined and searched): this means that the query editor correctly runs. The advantage provided by the OLAP engine is then the capability of allowing a dynamic interaction with the data to perform a more detailed exploration.

Acknowledgments We gratefully acknowledge Cristiana Larizza for software development supervision and DIBIT group (D. Toniolo, C. Sala, C. Camaschella) for providing data of the Val Borbera project.

References [1] Lander ES, Schork NJ. Genetic dissection of complex traits. Science. 1994 Sep 30; 265(5181):2037-48. [2] Botstein D, Risch N. Discovering genotypes underlying human phenotypes: past successes for mendelian disease, future approaches for complex disease. Nature Genetics, 2003 Mar; 33:228-237. [3] Martin-Sanchez F, Iakovidis I, Norager S, Maojo V, de Groen P, Van der Lei J, Jones T, Abraham-Fuchs K, Apweiler R, Babic A, Baud R, Breton V, Cinquin P, Doupi P, Dugas M, Eils R, Engelbrecht R, Ghazal P, Jehenson P, Kulikowski C, Lampe K, De Moor G, Orphanoudakis S, Rossing N, Sarachan B, Sousa A, Spekowius G, Thireos G, Zahlmann G, Zvarova J, Hermosilla I, Vicente FJ. Synergy between medical informatics and bioinformatics: facilitating genomic medicine for future health care. JBI, 2004 Feb; 37(1):30-42. [4] Sala C, Bione S, Crocco L, Gatti M, Poggiali E,. Bellazzi R, Buetti I, Rognoni C, Camaschella C and Toniolo D. The Val Borbera Project: epidemiological and genealogical analysis of an isolated population in Northern Italy. Submitted to European Society of Human Genetics, 2006. [5] Kinball R, Ross M. The Data Warehouse Toolkit, Second Edition. Wiley and Sons, Inc., 2002. [6] Wyderka KC. Data Warehouse Technique for Outcomes Management. Health Management Technology, Nov. 1999; 20 (10):16-7. [7] Hyde J, Mondrian OLAP project, Pentaho Analysis Service. http://mondrian.pentaho.org/. [8] Spofford G, Harinath S, Webb C, Huang DH, Civardi.F. MDX Solutions, Second Edition. Wiley Publishing, Inc., 2006

Conclusion Current genetic studies are characterized by the collection of huge quantity of both clinical and genotypic data. The final goal of such effort is the “genetic dissection” of complex phenotypes; the first challenge of data analysis is therefore to identify which phenotypes must be investigated. However, this task may be difficult to be performed using “standard” tools for database navigation, such as SQL query, as they require technical skills for the end user to extract interesting information. In order to solve this problem, in this paper we present a dynamic query system based on data warehouse and mining concepts, which allows phenotypes definition by a graphical user interface and data exploration using OLAP tools. Both applications are based on a common underlying data layer, the structure of which is a data mart oriented to data analysis. The phenotype definition is based on a logical formalization and it is properly stored to be processed by the OLAP engine. Once the phenotypes have been defined, the OLAP engine allows the user to perform a visual inspection of the data through a set of results dynamically created. We have chosen an open source OLAP engine, Mondrian, and we integrated it with new components (the MDX generator) in order to automate other specific technical operations.

Address for correspondence [email protected]

1279

This page intentionally left blank

Chapter 10. Biomedical Image and Signal Processing

1281

This page intentionally left blank

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Identifying QT prolongation from ECG impressions using Natural Language Processing and Negation Detection Joshua C. Denny, MDa,b, Josh F. Peterson, MDa,b Departments of aBiomedical Informatics and bMedicine, Vanderbilt University Medical Center, Nashville, TN, USA

mia. Since the QT interval, the measurement of the time between ventricular contraction (“QRS complex”) and its repolarization (“T wave”), varies with heart rate, QT prolongation is typically assessed via a QT corrected for rate (QTc) [6]. A value greater than 440-460 ms is typically considered prolonged. Many drugs are known to prolong the QT interval; it is the most common reason for a drug to be removed from the market [2, 7]. However, many other factors can influence measurement of the QT (and thus QTc), including arrhythmias, intraventricular conduction disturbances, ECG measurement technique, and morphological changes in the ECG.

Abstract Electrocardiogram (ECG) impressions provide significant information for decision support and clinical research. We investigated the presence of QT prolongation, an important risk factor for sudden cardiac death, compared to the automated calculation of corrected QT (QTc) by ECG machines. We integrated a negation tagging algorithm into the KnowledgeMap concept identifier (KMCI), then applied it to impressions from 44,080 ECGs to identify Unified Medical Language System concepts. We compared the instances of QT prolongation identified by KMCI to the calculated QTc. The algorithm for negation detection had a recall of 0.973 and precision of 0.982 over 10,490 concepts. A concept query for QT prolongation matched 2,364 ECGs with precision of 1.00. The positive predictive value of the common QTc cutoffs was 6-21%. ECGs not identified by KMCI as prolonged but with QTc>450ms revealed potential causes of miscalculated QTc intervals in 96% of the cases; no definite concept query false negatives were detected. We conclude that a natural language processing system can effectively identify QT prolongation and other cardiac diagnoses from ECG impressions for potential decision support and clinical research.

ECG findings consist of two types: morphologic descriptions (e.g., QT prolongation or widened QRS) and interpretations of those findings (e.g., myocardial infarction, atrial fibrillation, or ventricular hypertrophy). While many have developed automated feature extraction programs based on ECG waveforms, automated algorithms are imperfect, with accuracies of 42-96% [8, 9]. These algorithms are generally superior for morphological descriptions than their interpretations [8, 9]. However, many factors, such as an arrhythmia or ischemia, can alter the accuracy of morphological descriptions as well. For these reasons, cardiologists’ interpretations of ECGs remain the consensus gold standard [8, 10, 11].

Keywords: electrocardiogram, Unified Medical Language System, natural language processing, concept identification, decision support.

Natural language processing and concept-based indexing to standardized vocabularies such as the Unified Medical Language System (UMLS) have been applied to radiology reports [12, 13], clinical notes [14, 15], and medical education documents [16], among others. Previously, we reported the use of the KnowledgeMap concept indexer (KMCI) to identify UMLS concepts from cardiologistgenerated ECG impressions [17]. KMCI is a general purpose concept identifier, using rigorous, scored-based algorithms to identify concepts from free text [16]. It accurately identifies unknown abbreviations, acronyms, and underspecified concepts (e.g., document phrase “1st degree block” for the closest UMLS match “1st degree atrioventricular block”). KMCI scores ambiguous concept matches using the context of other concepts matching around it to favor candidates that are likely to co-occur. Previously, we optimized and evaluated its performance on ECG impressions, finding a recall of 0.90 and precision of 0.94 [17]. It was especially accurate for myocardial perfusion changes, ECG rhythms, and extracardiac manifestations (recall and precision in excess of 0.98). This system, however, did not have the ability to detect negated or possible findings.

Introduction Electrocardiograms (ECG) provide significant medical information that has been largely untapped in medication decision support interventions. ECGs are commonly used to help diagnose cardiac diseases such as myocardial infarction and ischemia, arrhythmias, and cardiomyopathies as well as some extracardiac diagnoses. Recent studies have highlighted the risk of sudden cardiac death due to medications known to increase QT intervals [1, 2]. Medications can also affect the risk of myocardial infarction [3, 4], induce second and third degree atrioventricular block [5], and cause other, potentially fatal, arrhythmias [1]. A decision support system that warns providers when prescribing medications to patients with existing risk factors, such as QT prolongation or atrioventricular block, may be valuable in guiding prescription choice. QT prolongation is a key risk factor for development of Torsades de Pointes, a potentially fatal cardiac dysrhyth-

1283

J. C. Denny et al. / Identifying QT prolongation from ECG impressions using Natural Language Processing and Negation Detection

Table 1 – Comparison of KnowledgeMap Concept Identifier (KMCI) identification of negation to gold standard physician review Gold standard Negated or possible findings

Positive or Probable findings

Total concepts

Identified as negated or possible Identified as positive

722 (6.9%) 20 (0.2%)

13 (0.1%) 9725 (92.8%)

735 (7.0%) 9745 (93.0%)

Total Concepts

742 (7.1%)

9738 (92.9%)

10480

KCMI

In this paper, we report on the integration of negation detection algorithms into KMCI and its initial application on a four-year collection of ECGs to identify QT prolongation. The ultimate goal is a codified database of ECG impressions for the development of a medication decision support system.

Negation tagging algorithm and evaluation Many authors have developed negation tagging algorithms [18-20]. We applied a modified version of the NegEx algorithm [18] using regular expressions. We marked phrases as “negated”, “possible”, or “asserted.” For the purposes of decision support, we included findings indicated as “probable” or “likely” as asserted findings. We used a total of 205 phrases indicating negation or possibility, including symbols such as “?” and “r/o” as indictors of “possible.” We used a window of 8 words before or after a negating phrase. In this dataset, we found that most periods marked abbreviations rather than sentences. Semicolons, unmatched closing parentheses, and other negating phrases terminated the current negation window.

Methods Creation of ECG database Vanderbilt University Medical Center has developed an anonymized database of all orders, laboratory results, and ECGs for all inpatients admitted for 2-30 days from 19992003 as part of an ongoing research study investigating drug effects. The ECGs were imported in an XML format from an ECG management system. Every ECG includes machine calculated intervals and heart rate as well as a cardiologist-generated free-text impression. Cardiologists create an impression for all ECGs by selecting among personalizable stock phrases (“normal sinus rhythm”) and editing as necessary (“normal sinus rhythm with rare PVCs”), or typing comments de novo (“LA abnormality, PVCs, and inferolat ST-T changes”). Finally, cardiologists code each ECG with a standard severity: normal, otherwise normal, borderline normal, or abnormal. We extracted all ECGs from our research repository and loaded them into a relational database. There were 44,808 ECGs in the database with more than 155,000 sentences.

One author (JP, an internist), unfamiliar with the NegEx algorithm or its implementation, scored 5000 sentences from randomly selected ECGs via a color-coded HTML interface that highlighted the negating phrase and words modified by them. Only medical concepts or medical modifiers were considered for scoring. Concepts were marked as a correctly identified negated concept (i.e., a true positives, TP), false positives (FP), true negatives (TN), and false negatives (FN). We calculated recall of negated concepts as TP/(TP+FN); precision as TP/(TP+FP); and negative predictive value, the probability that the concept is not negated (i.e., an asserted finding), as TN/(TN+FN). Following the evaluation, three new negation phrases were added and validated over several thousand sentences with an improvement in matching before application to the entire dataset.

From this dataset, we randomly selected a test set of about 5,000 sentences for development of our negation algorithm. Another 5,000 randomly selected sentences were reserved as a validation set. All subsequent data analysis was performed on the entire dataset of ECGs.

Development of concept-based ECG database We applied KMCI to identify Unified Medical Language System (UMLS) concepts from the free text ECG impres-

Table 2 – Concepts employed to identify QT prolongation within ECG impressions. Concept query (Asserted and Possible Concepts only) 1 2 3 4*

C0151878 Prolonged QT interval C1560305 Prolonged QTc interval C0023976 Long QT syndrome C0429028 QT interval C0489625 QTc interval C0860814 QTc

}

with any of

{

Total unique ECGs *

C0205166 Long C0439590 Prolonged C0392744 Lengthened

ECG matches

True Positives

2294 55 2 100

2193 55 2 100

2,364†

2,363†

For query 4, one concept from each list must occur within the same sentence to be considered a match. †Some ECGs matched more than one concept query (e.g., “QT prolongation. Compared with [date], the QT has lengthened.”)

1284

J. C. Denny et al. / Identifying QT prolongation from ECG impressions using Natural Language Processing and Negation Detection

Table 3 – Comparison of QT prolongation identified in ECG impression to automated QTc by ECG machine. Concept query

QTc > 400

QTc > 450

QTc > 500

QTc > 550

ECGs matching criteria

2,364

34,059

11,804

2,518

620

KMCI positives

2,364

2,357

2,304

539

117

Sensitivity

1.00

0.98

0.23

0.05

Specificity

0.19

0.77

0.95

0.99

Positive predictive value

0.06

11,804

2,518

620

Negative predictive value

1.00

0.20

0.21

0.19

diagnoses that may be of interest for decision support (see Table 4). Each of these topics requires an interpretation of the free text. Topics were selected by finding the UMLS concepts representing the topic of interest in the database of matched concepts. For myocardial infarction, this involved the tree of concepts related to “myocardial infarction” and “infarct.”

sions, using the optimizations identified in prior study [17]. We added synonyms and derivational transformations to KMCI’s lexicon and modified the sentenceidentification algorithm to ignore most spaces and periods when determining sentence breaks. We used the 2006AC version of the UMLS [21]; the only restriction on concept matching is favoring underspecified concepts with words such as “heart” and “electrocardiogram” (see [17] for a full list). Candidate UMLS concepts with these words are penalized less than candidates with other words when the words do not match a document word. We applied the negation algorithm following the concept identification step to mark each concept as positive, possible, or negated. The concept-identified ECGs are linked to the original ECG impressions and calculated intervals, forming the identified ECG dataset.

Results Table 1 shows the results of the negation analysis. The 5,000 sentences in the negation test set contained a total of 10,480 UMLS concepts. Overall recall was 0.973 and overall precision 0.982. The negative predictive value of finding negation (probability that a statement was positive given it was identified as positive) was 0.998. All false negatives were due to three phrases not present in the regular expression list: “replaced ”, “ replaced by”, and “ is/are gone.” Several of the false positives were instances in which negating phrases were amid multiple concept words (e.g., “ST no longer depressed,” in which the negated concept “ST depression” is separated by the negation phrase “no longer”). KMCI typically identified the correct UMLS concept for these phrases. Misspellings also caused some errors.

Analysis of QT prolongation Through perusal of the ECG dataset, the authors identified the UMLS concepts representing “QT prolongation,” including any text indicating a probable or possible QT or QTc prolongation. To verify that we had found all concepts representing QT prolongation, we also did text searches for all string matches with “QT” or “QTc.” Each was manually verified by analyzing all unique strings containing the concept. We extracted these concepts, ignoring negated concepts, along with the automated QT and QTc intervals identified by the ECG management system. We compared the predictive value of the computer calculated QT and QTc intervals (a continuous number) via the area under the receiver operator characteristic curves (AUC) using the cardiologist interpretation as the gold standard and also by the positive predictive value of the commonly-used cutoff values of 400ms, 450ms, 500ms, and 550ms.

KMCI identified 375,838 concepts from the 44,080 ECGs in the database mapping to 23,080 unique admissions. Cardiologists identified 70% percent of the ECGs as “abnormal,” 12% as “borderline,” and 18% as “normal” or “otherwise normal.” Of identified concepts, 339,554 (90.3%) were asserted, 29107 (7.7%) possible, and 7177 (1.9%) negated. Table 2 shows the concepts used for the QT prolongation query and frequency of each in the database. There were 254 unique strings with a range of 2-18 words (median 11 words, weighted median 5 words) matching the QT prolongation query; 15 of these (e.g., “QT interval long for rate”) accounted for nearly 90% of all matching impressions. The overall precision was 2363/2364 (1.00). Table 3 shows the results of different methods of predicting prolonged QT intervals; 2,364 ECGs (5.3% of all ECGs) were identified as representing QT prolongation by our concept query. The average QTc interval for those with prolonged QT intervals was 487 ms (range 363-716 ms); ECGs without mention of QT prolongation averaged 429 ms (p 450 for potential reasons why the QT calculations may be incorrect and to ascertain if any positive ECGs were missed by our concept query. Since the ECG images are not stored in our anonymized database, we were only able to evaluate the ECG intervals and raw text of the cardiologist impression. Student t-tests were used to compare parametric data. AUC and statistical analyses were performed with Stata, version 9.2 (StataCorp LP, College Station, TX). Description of identified ECG database To investigate the possibility for the database for decision support, we developed preliminary queries for a number of

1285

J. C. Denny et al. / Identifying QT prolongation from ECG impressions using Natural Language Processing and Negation Detection

Table 4 – Number of ECGs expressing potential targets for decision support Concept

Total (%)

Myocardial infarction (MI)

Positive

6,355 (14)

Acute MI

3,381

Possible 2,919

Negated 55

208 (0.5)

167

36

5

Myocardial ischemia

2,312 (5.2)

1,855

444

13

ST segment elevation

1,015 (2.3)

895

92

28

107 (0.2)

93

7

7 14

Wolff-Parkinson-White bypass tract Atrioventricular block,

1st

degree

2,461 (5.5)

1,876

571

Atrioventricular block, 2nd degree

61 (0.1)

58

1

2

Atrioventricular block, 3rd degree

24 (0.05)

23

1

0

Pericarditis Atrial fibrillation

105 (0.2)

43

62

0

1,748 (4.0)

1,719

16

13

397 (0.9)

373

20

4

Atrial flutter Total number of ECGs

44,080

clinical document types, which had recalls of 78-97% and precisions of 85-91%[18-20].

tion; the QTc interval AUC was 0.91. Of the 100 manually-reviewed ECGs with QTc intervals longer than 450 ms but that were negative by concept query, 32% had a bundle branch block; 24% had various ST segment or T wave abnormalities; 24% had an arrhythmia, aberrant complexes, or a pacemaker; and 23% had myocardial ischemia or infarct. No ECGs contained comments suggesting QT prolongation. Only 4 ECGs had no significant electrocardiographic abnormalities that could not alter calculation of the QT interval.

We have grouped terms such as “questionable”, “cannot rule out”, and “borderline” together. A more granular approach to negating terms may be more predictive of specific clinical outcomes. Also, negating phrases such as “no longer” indicate that a patient has a history of a finding as well as the absence of it now; the current algorithm only identifies the latter. Such information may help determine treatment efficacy. Finally, we used a rather simple “negation window” technique consisting of a certain number of words before or after a negation phrase. Some commas separate clauses while others represent a list of concepts. A more advanced algorithm could use a parsed sentence and the presence of prepositions or coordinating conjunctions to correctly size of the negation window.

Table 4 shows concept findings over the entire database.

Discussion We studied the application of a concept-based, natural language processing system to identify QT prolongation within cardiologist-generated ECG reports. Commonly used cutoffs for diagnosing QT prolongation from the QTc intervals calculated by ECG machines were poor, with positive predictive values of only 6-21% when compared with the NLP-based approach. Physician review of ECGs with long QTc by calculation but not by the NLP query found 94% of these ECGs had findings that would confound automatic QTc calculation. No ECGs not identified by KMCI were identified as prolonged by the cardiologist. KMCI is a more accurate means of identifying QT prolongation than automated interval analysis. Due to the high provider override rates in most medication decision support systems, due in part to poor specificity [22], a medication decision support system for QT prolongation requires use of the cardiologist-generated impression.

The handling of “possible” findings, included in our QT prolongation query, would vary depending on application. For the purposes of decision support, inclusion of potential findings may help prevent adverse events. For example, one would want to avoid starting a drug known to prolong the QT interval in a patient that had a “borderline long QT.” In addition, many uncertain ECG findings require further workup. A patient with potential ischemia requires further evaluation, and one would likely discontinue cyclooxegenase-2 inhibitors in this patient. For research investigations, however, one may desire to exclude uncertain findings, as many ECGs indicating “cannot rule out” may represent benign changes. The next step in this investigation involves building a decision support tool. Many resources list possible QT prolonging medications, such as the medication registry maintained at the University of Arizona [7]. A decision support system for QT prolongation could intercept orders for high-risk medications. By investigating the occurrences of medication orders in our retrospective database of inpatient admissions in individuals with QT prolongation, we may be able to identify potential medications interactions and new causes of QT prolongation. Some of these interactions may be indirect, such as the addition of a

A modified version of the NegEx negation algorithm performed well in detected negation within this dataset with a recall of 97.3% and a precision of 98.2%. Negation within ECGs was rare; overall only 1.9% of all concepts were marked as negated and 7.7% as possible. Thus, the probability of a concept the system identified as positive truly being positive was 0.998. The high recall and precision of negation detection in this dataset is likely due to a constrained vocabulary and the relative simplicity of the ECG impression sentences compared to prior studies in other

1286

J. C. Denny et al. / Identifying QT prolongation from ECG impressions using Natural Language Processing and Negation Detection

drug adverse events and development of new decision support tools to improve cardiovascular medication safety.

potent cytochrome P450 inhibitor that raises serum concentrations of a known offending agent. Ideally, a medication intervention could not only intercept medications that prolong the QT interval but also those that significantly interact with those already prescribed.

Acknowledgements We would like to thank Mr. Mark Arrieta for his assistance in assembling the database of ECGs. This work is supported by two National Library of Medicine grants, T15 LM007450 and R01 LM007995.

Since this is a full-text concept index over a large dataset of ECGs, we also have the ability to support many other types of decision support and research venues. While a string-matching algorithm could potentially replicate the performance of the QT prolongation concept query, it would lack flexibility and scalability. By fully concept indexing, we can quickly assess multiple queries, enabling a broader range of research and decision support tools. In addition, the parent-child relationships between concepts in the UMLS ameliorate querying across broad concepts, such as myocardial infarction.

References [1] Ray WA, Murray KT, Meredith S, Narasimhulu SS, Hall K, Stein CM. Oral erythromycin and the risk of sudden death from cardiac causes. N Engl J Med 2004 Sep 9; 351(11): 1089-96. [2] Roden DM. Drug-induced prolongation of the QT interval. N Engl J Med 2004 Mar 4;350(10):1013-22. [3] Bresalier RS, Sandler RS, Quan H, Bolognese JA, Oxenius B, Horgan K, Lines C, Riddell R, Morton D, Lanas A, Konstam MA, Baron JA. Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. N Engl J Med 2005 Mar 17;352(11):1092-102. [4] Solomon SD, McMurray JJ, Pfeffer MA, Wittes J, Fowler R, Finn P, Anderson WF, Zauber A, Hawk E, Bertagnolli M. Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention. N Engl J Med 2005 Mar 17;352(11):1071-80. [5] Zeltser D, Justo D, Halkin A, Rosso R, Ish-Shalom M, Hochenberg M, Viskin S. Drug-induced atrioventricular block: prognosis after discontinuation of the culprit drug. J Am Coll Cardiol 2004 Jul 7;44(1):105-8. [6] Bazett H. An analysis of the time relationship of electrocardiograms. Heart. 1920;7:353-70. [7] Drugs that prolong the QT interval and/or induce Torsades dr Pointes ventricular arrhythmia. [cited 2006 12/2]; Available from: http://www.arizonacert.org/medical-pros/ drug-lists/drug-lists.htm [8] Tsai TL, Fridsma DB, Gatti G. Computer decision support as a source of interpretation error: the case of electrocardiograms. J Am Med Inform Assoc. 2003 Sep-Oct; 10(5):478-83. [9] Willems JL, Abreu-Lima C, Arnaud P, van Bemmel JH, Brohet C, Degani R, Denis B, Gehring J, Graham I, van Herpen G, et al. The diagnostic performance of computer programs for the interpretation of electrocardiograms. N Engl J Med 1991 Dec 19;325(25):1767-73. [10] Olsson SE, Ohlsson M, Ohlin H, Dzaferagic S, Nilsson ML, Sandkull P, Edenbrandt L. Decision support for the initial triage of patients with acute coronary syndromes. Clin Physiol Funct Imaging 2006 May;26(3):151-6. [11] Paoletti M, Marchesi C. Discovering dangerous patterns in long-term ambulatory ECG recordings using a fast QRS detection algorithm and explorative data analysis. Comput Methods Programs Biomed 2006 Apr;82(1):20-30. [12] Fiszman M, Chapman WW, Aronsky D, Evans RS, Haug PJ. Automatic detection of acute bacterial pneumonia from chest X-ray reports. J Am Med Inform Assoc 2000 NovDec;7(6):593-604. [13] Huang Y, Lowe HJ, Hersh WR. A pilot study of contextual UMLS indexing to improve the precision of concept-based representation in XML-structured clinical radiology reports. J Am Med Inform Assoc 2003 Nov-Dec;10(6):580-7. [14] Friedman C, Shagina L, Lussier Y, Hripcsak G. Automated encoding of clinical documents based on natural language processing. J Am Med Inform Assoc 2004 SepOct;11(5):392-402. [15] Nadkarni P, Chen R, Brandt C. UMLS concept indexing for production databases: a feasibility study. J Am Med Inform Assoc 2001 Jan-Feb;8(1):80-91.

In this study, we used a general purpose concept-identification program with the entire UMLS. We optimized the algorithm to enhance synonymy and favor underspecified matches that match cardiology-related concepts. By processing these ECGs in bulk, KMCI is able to “learn” a corpus of frequently-occurring concepts and thus favors those concepts and their related concepts when encountering ambiguous matches. Other general purpose concept identification algorithms may not perform as well without similar optimizations. The interpretation of our findings is limited. The performance of the negation algorithm and concept identifier may not translate to other repositories of ECG impressions; however, no specific optimizations were made that correspond to our institution. Second, while we evaluated the QT prolongation algorithm against a random set of ECGs for false negatives, we did not have access to the original ECG waveforms to reevaluate for potential false negatives due to errors in the cardiologists’ reading; however, we expect that this would be unlikely to dramatically alter our results. Furthermore, the prevalence of QT prolongation in the KMCI-identified dataset (5.3%) carries more face validity than the QTc interval prevalence of 26.6%. Third, since cardiologists read all ECGs in our institution in a timely manner, decision support based on the textual interpretation of ECGs is feasible. It may not be practical at other institutions in which formal ECG impressions are not rapidly available. Fourth, while we have accurately identified concept matches and their negation status, this is not the same as asserting normality. Our algorithm tells the presence or absence of “atrial fibrillation,” for instance, but cannot tell that there were no arrhythmias. These questions may be addressed by classifying concepts by type (e.g., “rhythm” or “perfusion abnormalities”) and defining normal status. Finally, our exploratory list of concepts in Table 4 has not been formally assessed for false negatives and provides only a rough incidence of these findings in set of ECGs.

Conclusion The combination of a negation tagging algorithm with an effective concept identifier allows rapid assessment of clinical syndromes such as QT prolongation, myocardial ischemia, or atrioventricular conduction disturbances. We believe this technique may enable large-scale research on

1287

J. C. Denny et al. / Identifying QT prolongation from ECG impressions using Natural Language Processing and Negation Detection [20] Mutalik PG, Deshpande A, Nadkarni PM. Use of generalpurpose negation detection to augment concept indexing of medical documents: a quantitative study using the UMLS. J Am Med Inform Assoc 2001 Nov-Dec;8(6):598-609. [21] Unified Medical Language System. [cited 2006 12/2]; Available from: http://www.nlm.nih.gov/research/umls/ [22] van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc.2006 Mar-Apr;13(2):138-47.

[16] Denny JC, Smithers JD, Miller RA, Spickard A, 3rd. "Understanding" medical school curriculum content using KnowledgeMap. J Am Med Inform Assoc 2003 JulAug;10(4):351-62. [17] Denny JC, Spickard A, Miller RA, Schildcrout J, Darbar D, Rosenbloom ST, Peterson JF. Identifying UMLS concepts from ECG Impressions using KnowledgeMap. AMIA Annu Symp Proc 2005:196-200. [18] Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG. A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform 2001 Oct;34(5):301-10. [19] Elkin PL, Brown SH, Bauer BA, Husser CS, Carruth W, Bergstrom LR, Wahner-Roedler DL. A controlled trial of automated classification of negation from clinical notes. BMC Med Inform Decis Mak 2005;5(1):13.

Address for correspondence: Josh Denny, MD 4th floor, Eskind Biomedical Library 2209 Garland Ave Nashville, TN 37232, USA [email protected]

1288

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Comparison of Supervised Classification Methods for Auditory Brainstem Response Determination Paul McCullagh, Haiying Wang, Huiru Zheng, Gaye Lightbody and Gerry McAllister Department of Computing and Mathematics, University of Ulster, United Kingdom

Abstract

the waveform while suppressing the uncorrelated EEG, extraneous noise and artifact. It is necessary to average approximately 1000-2000 trials before the noise is sufficiently suppressed, with signal to noise ratio enhanced proportional to the square root of the number of trials.

The ABR is commonly used in the Audiology clinic to determine and quantify hearing loss. Its interpretation is subjective, dependent upon the expertise and experience of the clinical scientist. In this study we investigated the role of machine learning for pattern classification in this domain. We extracted features from the ABRs of 85 test subjects (550 waveforms) and compared four complimentary supervised classification methods: Naïve Bayes, Support Vector Machine, Multi-Layer Perceptron and KStar. The ABR dataset comprised both high level and near threshold recordings, labeled as ‘response’ or ‘no response’ by the human expert. Features were extracted from single averaged recordings to make the classification process straightforward. A best classification accuracy of 83.4% was obtained using Naïve Bayes and five relevant features extracted from time and wavelet domains. Naïve Bayes also achieved the highest specificity (86.3%). The highest sensitivity (93.1%) was obtained with Support Vector Machine-based classification models. In terms of the overall classification accuracy, four classifiers have shown the consistent, relatively high performance, indicating the relevance of selected features and the feasibility of using machine learning and statistical classification models in the analysis of ABR.

Figure 1 - ABR waveform at a 70, 50 and 10 dB Stimuli Figure 1 shows responses for a healthy adult ranging from 70dB (normal hearing level, nHL) stimulus to threshold (10dB nHL). The characteristic shape is referred to as the Jewett waveform, comprising up to seven peaks, labeled IVII. As the level of stimulus is reduced, the different peaks of the waveform become less defined, and their latency is increased. The shape of the waveform from the long slope at the top of peak V is the strongest part of the waveform to remain as the stimulus diminishes. When the stimulus level is set near the subject’s hearing threshold the identification of wave V and its following negative slope assist with classifying the presence of the response. The shape of the ABR response will differ between subjects depending on a number of factors such as, electrode placement, filter settings, stimulus intensity, ear used, age, sex and subject’s skull thickness. For this reason the range of factors need to be taken into consideration before a clinical expert can make an interpretation. In particular the expert checks the latencies of waves I, III and V and examines the morphology of the waveform. Often sub-averages based on successive recording sessions are compared for consistency and a cross-correlation may also be taken between repeat recordings to assist in classification.

Keywords: Auditory Brainstem Response, wavelet decomposition, feature extraction, classification, decision support

Introduction The Auditory Brainstem Response (ABR) is evoked when a stimulus click is applied to a subject’s ear to determine hearing acuity and integrity of the auditory pathways. If the stimulus is perceived, a response changes their electroencephalogram (EEG) within 10ms from stimulus onset (Figure 1). The amplitude of the ABR signal is approximately 1 μVolt -5μVolts and is hidden behind the background EEG and noise (approximately 50μVolts). The components of the ABR are swamped by the endogenous electrical activity of the brain and the determination of a response can be difficult particularly at low levels of acoustic stimulation, as hearing threshold is reached. The ABR waveform is extracted by coherent averaging which exploits the deterministic nature of the signal to enhance

1289

P. McCullagh et al. / A Comparison of Supervised Classification Methods for Auditory Brainstem Response Determination

The interpretation of the ABR is subjective, thus clinical experts may not always draw the same conclusion [1], particularly at auditory threshold. Artificial intelligence methods have been used to provide objective assistance in response interpretation [2]. Useful information may be extracted from the EEG recordings using features extracted from the time and frequency domains [3]. Davey [4] showed that the ratio of the power of the post over pre stimulus time domain waveforms could be used as an effective method to classify ‘strong’ responses with an accuracy of 98.6%. Remaining ABR waveforms, i.e. those which did not have a strong response, were passed to a second stage of analysis, whereby repeat recordings were used to derive features based on cross-correlation parameters using both the time and frequency domains. Lower accuracies ranging from 65% to 82% were achieved, dependent upon features used. The lower accuracies reflect the more difficult classification process.

The data were pre-processed by band-pass filtering (100Hz-3kHz), sampled at 20kHz and then de-noised using a wavelet filter. Between 1000 and 2000 ensembles were averaged to provide one ABR waveform. Each waveform consisted of 480 data samples, half before the stimulus and half after, which related to 12ms before the stimulus and 12ms after the stimulus. The ABR waveform appears within 10ms of stimulus onset, which after bandpass filtering and sampling relates to 200 data points. A number of features were extracted from both time and wavelet domains. The Daubechies wavelet has been used to de-noise biosignals [10]. We computed a range of scaling lengths performed to 6 levels (A6: 0–156Hz, D6: 156–312Hz, D5: 312–625Hz, D4:625-1250Hz, D3:1250-2500Hz, D2:2500-5000Hz, D1:5kHz-10kHz). Levels D6, D5 and D4 showed to be of significant interest as they related to the key frequencies (200, 500 and 900Hz [11]) contained within the ABR. This is depicted in figure 2 which shows the Fast Fourier Transform (FFT) amplitude values for post-stimulus ABR waveforms at 70dB and 30dB stimulation intensities. It highlights the frequencies covered by the D6, D5 and D4 wavelet coefficients.

Lightbody [5] used wavelet decomposition [6, 7] on the same dataset following a two-stage classification process. Power ratios of the post over pre stimulus wavelet coefficients were used to classify ‘strong’ responses. Those remaining were then classified using correlation features of repeated tests derived from the wavelet domain. Overall accuracy of 76.4% was obtained using a C5.0 decision tree classifier. Strong responses were classified without error. The criterion to determine a ‘strong’ response required a combination of time and wavelet post stimulus to pre stimulus power ratios (time domain power>2 and wavelet domain power>1.6, determined heuristically). By combining features using the Demster-Shafer method, as used in evidential reasoning [8], Davey achieved a classification accuracy of 95.6% for ‘strong’ responses and 85% for lower level responses [9]. In this study we compared four additional classification techniques which are in widespread use by machine learning researchers. The aim was to determine whether one of these techniques provides superior classification accuracy for the quasi-stationary ABR evoked response. Three additional statistical descriptors, namely precision, sensitivity and specificity were used to further explore the data set. The data set comprised both high and low level recordings. A secondary aim was to determine whether a single ABR recording, irrespective of level, provided sufficient information for decision support. Hence the classification process does not involve correlation parameters from repeated recordings.

Figure 2 - FFT amplitude values for post-stimulus ABR waveforms at 70dB and 30dB stimulation intensities Although in figure 2 the D4 coefficients seemed to show the least promising power peaks they have double the number of coefficients than the D5 range helping to reveal more feature consistency. Figure 3 illustrates the D4 component for pre and post stimulus activity for decomposition on 256 data points. The decomposition was carried out on the full pre stimulus section and only waves I to V of the post section. Davey [4] showed that by using data from 1.5 to 9.5 ms post stimulus then the classification accuracy was improved. By focusing in on the ABR where waves I to V where most likely located removed the samples early in the waveform that may be more likely to contain artifacts and influence the classification. This same process was used for the wavelet decomposition of the post stimulus section. For both the pre and post stimulus waveforms data extension was required to extend the dataset to 256 which is the nearest dyadic number [5] so to support wavelet decomposition. Different methods to extend the data had been looked at to determine the method least likely to incur boundary issues [12].

Methods The study was performed on a database of 85 test subjects, provided by the Audiology Department of the Royal Group of Hospitals in Northern Ireland. Each subject had a range of test stimulus levels applied providing a mix of good, weak and non-response waveforms, all of which were classified by a clinical expert. There were 550 waveforms in total, 396 recordings with a YES classification and 154 with a NO classification.

1290

P. McCullagh et al. / A Comparison of Supervised Classification Methods for Auditory Brainstem Response Determination

first considered since a 6-level decomposition of a 128 sample waveform results in a small number of coefficients (8-D4, 4-D3 and 2-D6 coefficients). Then the individual bands were considered. The last features were derived by averaging the calculated features for each of the 128 sample windows, so to capture any overall trends. The features were used as inputs to four classification models Naïve Bayes (NB), Support Vector Machine (SVM), Multi-Layer Perceptron (MLP) and KStar. NB is a probalilistic classifier based on the Bayes rule of conditional probability. It assumes independence between features. It uses the normal distribution to model numeric attributes by calculating the mean standard deviation for each class [14]. SVM is a kernel based classifier. The basic training for SVMs involves finding a function which optimizes a bound on the generalization capability, i.e., performance on unseen data. By using the kernel trick technique, SVM can apply linear classification techniques to non-linear classification problems [15]. A MLP is a non-linear classification approach that may be trained using the back propagation algorithm. A MLP consists of multiple layers of computational units (an input layer, one or more hidden layer and one output layer). For the MLP model, the results were obtained using a model consisting of one hidden layer with six nodes when evaluating the top ten features, four nodes when evaluating the top six features and two nodes when considering the top three features (the choice of feature subsets is discussed later in the paper). Each MLP was trained for 500 epochs and the learning rate was set to 0.3. The final classification is KStar, a relatively simple instance based method. The class of a test instance is based upon the class of those training instances similar to it, using an entropy-based distance function to compute the similarity between two cases. The underlying assumption is that similar instances will have similar classes. All four classification models were implemented within the framework provided by the open-source platform Weka package [16]. This provides a collection of machine learning algorithms for data mining tasks, which may be implemented directly or called from Java code.

Figure 3: Extracted D4 wavelet coefficients (70dB stimulus), for pre and post-stimulus regions An alternative partitioning of the data has been considered by performing wavelet decomposition on a smaller section of the data containing 128 data points, therefore removing the need for data extension [13]. The window of data relates to a 6.4ms section of the waveform, so the choice of window within the full post stimulus section is vitally important to ensure that the subject’s response has been captured. This is crucial because as the strength of a response diminishes the peaks fade and latencies alter until peak V remains. Also, the exact position of the response differs from subject to subject. A sliding window of 128 data points was used to derive features between 1.5ms and 9.5ms of the post stimulus waveform, which was shown to be the time frame with the strongest waveforms in an ABR response [4]. Using a moving window of 128 samples from stimulus onset, the window containing the peak time domain power ratio was determined. The ABR response is expected to be at its maximum within 1.5ms and 9.5ms post stimulus. It was considered that a peak power before the 1.5ms mark could signify that the power ratio might be a result of something other than the ABR. A weighting based on the position of the peak power was used to numerically represent this doubt in the power ratio as a feature for classification. This weighting was applied to features DataAbs and DataPwr resulting in features wDataAbs and wDataPwr respectively. Whereby, DataAbs is a feature derived from the ratio of the post and pre stimulus D4 coefficients [5] (Figure 3) and DataPwr is the ratio of the post and pre stimulus time domain waveform [4]. In both cases only waves I to V in the post stimulus waveform were considered and the wavelet decomposition was performed on 256 data points resulting in 16 D4 coefficients (Figure 3).

In the evaluation of each classifier we used ten-fold cross validation, i.e. the entire dataset is partitioned into 10 subsets of approximately equal size. Each subset in turn is used as the test set while the other subsets are combined to form a training set. The quality of each classifier was assessed by the extent to which the correct class labels have been assigned. Each classifier is assessed by three statistical measures: Precision (Pr), Sensitivity (Se), Specificity (Sp). All the features were ranked using correlation coefficient-based ranking.

Each 128 sample window starting from sample 35 to starting from sample 75 were independently analyzed. Power ratios were calculated labeled RNxx, where R represents the ‘response’ post stimulus region in the waveform and N represents the ‘no response’ pre stimulus region of the waveform, and ‘xx’ represents the starting sample for the window of data. Similar ratios for the D4, D5 and D6 wavelet coefficients were calculated for each of the 128 sample windows. Using all the D4-D6 coefficients was

Results The top five features are illustrated in Table 1. Table 2 details accuracy, precision, sensitivity and specificity of the classifiers in separating the two classes, based on the top 5 discriminative features. (Note as this is a two-class problem, Se and Sp are reversed for Response=’NO’ and Response=’YES’).

1291

P. McCullagh et al. / A Comparison of Supervised Classification Methods for Auditory Brainstem Response Determination

Table 1 - Top five predictive features Feature

Name

Discussion The ABR test is routinely performed to detect hearing loss (Response=’NO’) in the Audiology clinic. Where the ratio of post-stimulus power to pre-stimulus power is high (>5), the classification decision can be made to high accuracy (98.6%). This is manifest as a clear Jewett response. Misclassification may be due to artifact attributed to stimulus, myogenic (muscle) activity, or eye blink artifact which corrupts the ABR. Such activity may be detected by an expert due to latency; stimulus artifact occurs within the first msec post-stimulus, and myogenic artifact normally occurs after wave VII. Eye blinks may be harder to eliminate as they may be more randomly distributed in the pre and post stimulus activity and hence overlap with the ABR components. The strategy is normally to remove eye blinks at source, as much as possible, by rejecting a contaminated trial.

Description

1

wDataAbs

Wavelet power feature: D4 (weighted)

2

DataAbs

Wavelet power feature: D4

3

RN75

Power ratio: 128 samples from 75

4

RN70

Power ratio: 128 samples from 70

5

RN65

Power ratio: 128 samples from 65

Table 3 details accuracy, precision, sensitivity and specificity of the classifiers in separating the two classes, based on the top 2 discriminative features. The accuracy and precision in determining a NO response are slightly reduced, implying that most discrimination is harnessed from the relative power measure from the wavelet domain, i.e. the D4 coefficient. Time may have a small role to play in the overall morphology. Table 2 - Prediction results for four classifiers using 10fold cross validation with top 5 features (wDataAbs, DataAbs, RN75, RN70, and RN65) Method

Ac (%)

Response=’NO’

Response=’YES’

Pr (%)

Se (%)

Sp (%)

Pr (%)

Se (%)

Sp (%)

NaiveBayes 83.4

93.5

82.2

86.3

66.8

86.3

82.2

SVM

81.9

83.2

93.1

54.9

76.8

54.9

93.1

MLP

81.9

87.9

86.2

71.4

68.3

71.4

86.2

Kstar

82.2

86.9

88.1

68.0

70.4

68.0

88.1

Table 3 - Prediction results for four classifiers using 10-fold cross validation with top 2 features (wDataAbs and DataAbs) Method

Ac (%)

Response=’NO’

Response=’YES’

Pr (%)

Se (%)

Sp (%)

Pr (%)

Se (%)

Sp (%)

NaiveBayes 81.4

86.7

86.9

68.0

68.4

68.0

86.9

SVM

80.2

81.5

93.1

49.1

74.8

49.1

93.1

MLP

80.7

84.5

89.1

60.6

69.7

60.6

89.1

KStar

81.2

85.0

89.1

62.3

70.3

62.3

89.1

1292

ABR responses with a lower post-stimulus power to prestimulus power ratio (L>N” (I-BIL first is normal, then changed to the low region, and finally changed to the normal region in one case like the right bottom in Fig 2) etc.

The input includes a set of time-stamped data points (events). The output includes a set of interval-based, context-specific unified values or patterns (usually qualitative) at a higher level of abstraction. TA can be generally considered in two phases: basic TA for abstracting timestamped data from given episodes (which are significant intervals for the investigation purpose) and complex TA for investigating specific temporal relationships between episodes that can be generated from a basic TA or from other complex TAs.

Figure 1 shows typical possible patterns (8 and undetermined) for rapidly changing tests, and Figure 2 shows typical possible patterns (21 and undetermined) for slowly changing tests [4]. Abstraction of rapidly changing test results From our observation and analysis, especially GPT and GOT were defined as rapidly changing attributes, which can go up in a very short period and go back to a stable” state. Thus two most representative characteristics of these tests are a “stable” base state (BS), and the position and value of peaks, where the attributes suddenly go up. Based on this, we formulated the following algorithm to find the base state and peaks of a test. Rapidly changing tests applied also to TTT and ZTT and they showed 9 patterns.

Basic temporal relations We started by a separation of two groups of tests, one with values that change rapidly in the short term such as GOT, GPT, TTT and ZTT and the other with values that change slowly in the long term such as T-CHO, CHE, ALB. Basic temporal abstractions typically extract states (e.g., low, normal, high), and/or trends (e.g., increase, stable, decrease) from a uni-dimensional temporal sequence.

Algorithm 1 (for rapidly changing tests) Input: A sequence of patient’s values of a test with length N denoted as S00 = {s1, s2, …, sN} in a given episode.

The essential ideas of our temporal abstraction methods here is to deal with long and irregular time-stamp sequences, and doing abstraction in efficient. We introduce the notion of “changes of state” to characterize the slowly changing tests, and the notions of “base state” and “peaks” to characterize the rapidly changing tests .

Output: Base state and peaks, and an abstraction of the sequence derived from them. Parameters: NU, HU, VHU, XHU: upper thresholds of normal, high, very high, extreme high regions of a test, a (real).

Temporal abstraction primitives

Notation:

From observation and analysis, we defined the following temporal abstraction primitives:

• • •

1. State primitives: N (normal), L (low), VL (very low), XL (extreme low), H (high), VH (very high), XH (extreme high). 2. Trend primitives: S (stable), I (increasing), FI (fast increasing), D (decreasing), and FD (fast decreasing). 3. Peak primitives: P (peaks occurred). 4. Relations: “>” (change state to), “&” (and), “–“ (and then), “/” (“X/Y” means majority of points are in state X and minority of points are in state Y). The thresholds to distinguish the state primitives of tests are given by medical doctors, for example, those of the test GOT are 40, 100, 200, respectively. We define structures of abstraction patterns as follows: <pattern> ::= <state primitive> <pattern> ::= <state primitive> <pattern> ::= <state primitive> <pattern> ::= <state primitive> <state primitive> Examples of abstracted patterns in a given episode are like follows:

Mi: Set of local maximum points of S BS: base state of S PEi: set of peaks of S

Figure 1 - rapidly changing test patterns Abstraction of slowly changing test results The key idea is to use the “change of state” as the main feature to characterize sequences of the tests. It can be seen that the “change of state” characterize information of both state and trend of the sequences.

1305

K. Takabayashi et al. / Temporal Abstraction and Data Mining with Visualization of Laboratory Data

From the beginning of a sequence, the first data points can be at one of the three states “N”, “H”, or “L”. It will happen that:

interactively and visually. Cviz is an attempt to integrate visualization into the knowledge discovery process. D2MS facilitates the trials of various alternatives of algorithm combinations and their settings. The data mining methods in D2MS consists of programs CABRO for tree learning and LUPC for rule learning [6]. CABRO produces decision trees using R-measure and graphically represents them in particular with T2.5D tool (trees 2.5 dimension). As shown in Figure 4, visualization made us easily recognized the different pattern of HBV and HCV.

Either the sequence changes from one state to another state, smoothly or variably (at boundaries), or the sequence remains in its state without changing. We provided 22patterns for slowly changing data.

Figure 4 - Rules for HBV and HCV by D2MS Figure 2 - Slowly changing test patterns

We examined statistical significance of the consequence according to the method of [7], which prunes discovered rules statistically as follows.

Temporal relationships The temporal relations between the abstracted events of laboratory data were also treated here as phenomena and a part of the rules by comparing the period of the state. They are classified into seven relations by Allen[8].

Assume a rule R: A  C (or R: A ¬C) with confidence conf(R). If conf (R) = p(C) then R is eliminated. To test whether conf (R) = p(C), we use the following test statistic where nA is the number of cases satisfying C Mining abstracted hepatitis data with LUPC LUPC is a separate-and-conquer algorithm that controls the induction process by several parameters. The parameters allow obtaining different results and this ability allows the user to play a central role in the mining process [6]. LUPC is developed to learn prediction rules from supervised data. Each rule found by LUPC is a conjunction of attribute-value pairs that may present an interesting pattern. The main features of LUPC are (1) its ability of finding rules with associate domain knowledge (such as finding rules containing or not containing specified attribute-value pairs), as well as finding rules for minority classes; (2) it is integrated with D2MS’s rule visualizer and thus supports the user in selecting the appropriate rules which result from different possible settings of parameters. The performance of LUPC depends on several parameter specified by the user: for min accuracy of rules, for min coverage of rules, for maximal number of candidate

Figure 3 - Temporal relations Complex temporal abstraction

rules in the beam search, and for maximal number of attribute-value pairs to be consider. By varying these parameters we can find different sets of rules [6]. When using the setting with default parameters of = 80%, = 3, = 200, and = 100, we found 119 rules characterizing the hepatitis B and 152 rules characterizing hepatitis C.

Mining abstracted hepatitis data with system D2MS The authors developed an interactive visualization tool in decision tree construction called D2MS (data mining for Model Selection) for supporting an effective cooperation of the user and the computer in classification. D2MS shares many features with WinViz [5] and Cviz that both use parallel coordinates. WinViz allows the user to visually examine a tabular database and to formulate query

1306

K. Takabayashi et al. / Temporal Abstraction and Data Mining with Visualization of Laboratory Data

Figure 5 - LUPC Rules can be illustrated in a left figure Evaluation The produced rules were evaluated by three experts. Table 2 - Top 5 rules selected by LUPC

Results By using D2MS, we discovered 33,477 rules for type B and C difference. These rules are complicated and sometimes contradictory to each other. For example, there is a rule if T-Bilirubin(Bil) is N(normal) , ZTT is N, and GOT is N with P(peak), then HBV, while there is another rule if D-Bil is N, TTT is N, GOT is N with P, then HCV, which are almost the same but the results are completely different. After pruning by statistical aspects between HBV and HCV, there are only 27 rules (0.08%) left (Table1). However, these rules seemed not attractive for medical doctors even if they are statistically significant. For example, Tcholesterol is normal is HCV in 171/260 cases (66%), or GPT is high with a peak and ZTT is mostly high partly normal is HBV. They are too simple or vague and must be carefully assessed.

In the viewer of LUPC we can see the accuracy and coverage on the upper left, rule itself in the middle and the relation of the rules and attribute value pairs in the figure to the right which can be manipulated by users (Figure 6). By handling it users can see the relations of each item. In this figure doctors could easily recognize if LDH is low to normal is false, then all the rules are related to HCV, while if creatinine is normal to low is false, then it is related to the rules of HBV except rule #48(center), so that the doctors could understand from a more comprehensive point. This technique was highly evaluated by medical doctors and some rules such as the top 5 were considered as meaningful.

Different datasets were found by using LUPC with various parameters including temporal relations between laboratory tests. Table2 presents the top five rules by LUPC from the point of coverage and confidence. From this table, especially rule3 and 5 are similar and could be merged as a rule that if TP decreasing high to normal and both ZTT and TTT is high, then it is HCV. In the evaluation of medical doctors, though most of them seemed not crucial or not useful in clinical medicine even the discovered rules covering many cases with high accuracy. However, some of the rules could be reasonable from the different clinical course of two types of hepatitis, especially when the experts checked and integrated the rules in the illustration. Table 1 - 27 pruned rules produced by D2MS and satisfying chi-square test

1307

K. Takabayashi et al. / Temporal Abstraction and Data Mining with Visualization of Laboratory Data

not realistic to study prospectively. There is a great possibility of new paradigms appearing before the study is completed. Retrospective studies are expected for these long term studies and data mining techniques will play a major role in this filed by creating high potential hypotheses. Even though we did not discover crucial rules to show the difference of laboratory data change between HBV and HCV, we proved to show that this combination of TA and data mining with visualization is useful and effective. Furthermore, it could be applied to other fields of medicine and would be a basic model for the universal analysis of data mining for temporal data analysis in medicine.

Conclusion Figure 6 - LUPC makes it easier for the users to understand the relations comprehensively

The rules that show the difference of the laboratory changes in the long clinical course between the HBV and HCV could be deduced by D2MS. Pruning by statistical significance could decrease the number of rules but obtained rules were not interesting in individuals. Visualization made it easier for doctors to find the relations and led them to find reasonable results. This combination technique of temporal abstraction and data mining with visualization could be applied universally.

Discussion We have presented a temporal abstraction approach to data mining the temporal hepatitis data. Most doctors do not believe that they can distinguish HBV and HCV from the laboratory data change, so it might be true when we cannot obtain any new findings from this data mining. However, we in fact obtained many rules to identify HBV or HCV with statistical significance. Some of them look very simple. To confirm, we need to analyze them by stratified analysis, removing the modification of other factors such as treatment and then compare again. By LUPC, we can estimate the patient data comprehensively and expect new findings in many diseases because it is difficult for human beings to find data changes over a long period of time. some experts of liver diseases mentioned that the cases with HBV and HCV were apt to show different clinical changes, and our results would reflect these changes.

References [1] Motoda, H., Active Mining: New Directions of Data Mining, IOS Press, 2002. [2] Horn, W., Miksch, S., Egghart, G., Popow, C., Paky, F., “Effective Data Validation of High-Frequency Data: Time-Point-, Time-Interval-, and Trend-Based Methods”, Computer in Biology and Medicine, Special Issue:TimeOriented Systems in Medicine, 27(5), 389-409, 1997. [3] Bellazzi, R., Larizza, C., Magni, P., Monntani, S., and Stefanelli,M. Intelligent Analysis of Clinic Time Series: An Application in the Diabetes Mellitus Domain Artificial Intelligence in Medicine, 20, 37-57, 2000. [4] Ho, T.B., Nguyen, T.D., Kawasaki, S., Le, S.Q., Nguyen, D.D., Yokoi, H., Takabayashi, K. Mining Hepatitis Data with Temporal Abstraction. ACM Inter. Conf. on Knowledge Discovery and Data Mining KDD-03, 369-377,2003. [5] Lee, H.Y., Ong, H.L., and Quek, L.H. Exploiting Visualization in Knowledge Discovery. First Inter. Conf. on Knowledge Discovery and Data Mining, 1995, pp. 198–203. [6] Ho T.B., Nguyen T.D., Nguyen D.D., Kawasaki S., Visualization Support for User-Centered Model Selection in Knowledge Discovery and Data Mining. Int. Journal Artificial Intelligence Tools, 2001;10(4): 691-713. [7] Bruzzese, D. and Davino, C. Statistical Pruning of Discovered Association Rules. Computational Statistics, 16 (3), pp. 387 –398, 2001. [8] Allen E and Jai S. A decidable temporal logic to reason about many processes. Proceedings of the ninth annual ACM symposium on Principles of distributed computing table of contents. Quebec City , Canada 233-246,1990.

One of the major problems in rule based data mining is that there are too many rules deduced for us to evaluate. To select most important rules we introduced the chi-square test which was effective to decrease the number of rules as well as statistical reasoning. Another is by LUPC, not only selecting minority classes from large unbalanced datasets but visualization, it is not difficult to separate the important ones from many rules for medical doctors. Other studies of data mining in medicine are mostly in the field of genomics and epidemiology and the analysis of laboratory data is quite limited. We provided the data of anti-phospholipid antibody syndrome to PKDD 1999 as model data in order to establish a new technique as well as hepatitis data. Current medical study is deeply inclined to use a prospective way or Cohort as a scientific study design, which implies a carefully planed experiment. However, when we think of a long-term experiment lasting over 10 years, it is

1308

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Automated Interpretation of Optic Nerve Images: A Data Mining Framework for Glaucoma Diagnostic Support Syed SR. Abidia, Paul H. Artesb, Sanjan Yuna, ,Jin Yua a

NICHE Research Group, Faculty of Computer Science, Dalhousie University, Halifax, Canada b Department of Ophthalmology and Visual Sciences, Dalhousie University, Halifax, Canada

Researchers have analyzed optic nerve data and CSLT based images with varying results. Bowd et al [3], working with retinal tomography images applied forward and backward feature selection methods for training Multi Layer Perceptron (MLP), Support Vector Machine (SVM) and linear discriminant functions; Park et al [4] used correlation analysis and forward wrapper model to select features from optic disc data for training SVM classifiers; Swindale et al [5] used a wrapper model for feature selection to train SVM classifiers.

Abstract Confocal Scanning Laser Tomography (CSLT) techniques capture high-quality images of the optic disc (the retinal region where the optic nerve exits the eye) that are used in the diagnosis and monitoring of glaucoma. We present a hybrid framework, combining image processing and data mining methods, to support the interpretation of CSLT optic nerve images. Our framework features (a) Zernike moment methods to derive shape information from optic disc images; (b) classification of optic disc images, based on shape information, to distinguish between healthy and glaucomatous optic discs. We apply Multi Layer Perceptrons, Support Vector Machines and Bayesian Networks for feature sub-set selection and image classification; and (c) clustering of optic disc images, based on shape information, using Self-Organizing Maps to visualize sub-types of glaucomatous optic disc damage. Our framework offers an automated and objective analysis of optic nerve images that can potentially support both diagnosis and monitoring of glaucoma.

We have developed a data-driven Glaucoma Diagnostic Support (GDS) system that features the automatic interpretation of CSLT topography images of the optic nerve to support (a) the classification of the optic disc images to distinguish between healthy and diseased optic discs; (b) the identification of the sub-types of glaucomatous optic disc damage. This is to help further sub-classify the glaucoma patients in order to provide treatments in line with the specific morphological patterns of damage [6]; and (c) the visualization of the temporal progression of the disease for a patient over a period of time.

Keywords:

In this paper we present an automated approach to CSLT topography image analysis to support glaucoma diagnosis. Our multi-stage approach is a hybrid of image processing and data mining methods. In Stage 1, we apply image-processing techniques to CSLT images to derive imagedefining features. In Stage 2, we apply data classification methods to the image’s shape-defining features to develop classifiers that can discriminate between healthy and glaucomatous optic discs. An important task at this stage is feature selection whereby we select an optimal subset of image features that exhibit high image classification capabilities. In Stage 3, we apply data clustering techniques to the optimal subset of image-defining features in order to identify the different sub-types of glaucomatous images in the image data-set. The emergent image clusters are subsequently used to both visualize the progression of the disease and the identification of noisy optic nerve images. In Stage 4, we apply rule-induction techniques to the optimal subset of features to induce classification rules (not discussed here). These symbolic rules provide practitioners with a justification of the diagnostic recommendations by our image classifiers. For our experiments, we worked with 1257 tomography images taken at different time intervals from 136 subjects (51 healthy subjects and 85 glaucoma patients).

glaucoma, data mining, feature selection, clustering, Confocal Scanning Laser Tomography, Support Vector Machines

Introduction Glaucoma is an eye disease that is characterized by slow progressive damage to the optic disc and corresponding deterioration of the patient’s vision [1]. At present, there is a gap in the understanding of the cause, the types and the natural course of glaucoma. The use of sophisticated imaging technologies, such as Confocal Scanning Laser Tomography (CSLT), capture 3-dimensional images of the optic disc that are used for diagnostic purposes [2]. However, the interpretation of CSLT images is a manual and subjective process—a trained professional has to manually define the margins of the optic nerve based on his/her training and expertise and then classify the optic nerve as normal or glaucomatous. The current process allows for misjudgments/errors in the interpretation of the CSLT image, inability to distinguish between actual and noisy images and variance in the diagnostic recommendations over a set of practitioners. The challenge, therefore, is to automate the analysis of CSLT images of the optic disc, in an objective and quantifiable manner, to support practitioners in the diagnosis and therapeutic management of glaucoma.

1309

S. SR. Abidi et al. / Automated Interpretation of Optic Nerve Images: A Data Mining Framework for Glaucoma Diagnostic Support

image classification. We have developed a two-pass image classification method that incorporates feature sub-set selection as an integral element (see Figure 1). In the first pass, MLP and SVM based wrapper models are simultaneously used to generate a Moment Feature Subset (MFS) consisting of low order moment features. In the second pass, we apply a Markov blanket filter method [8] based on an inferred Bayesian network to select the highly relevant moments from the MFS—i.e. the Optimal Moment Feature Subset (OMFS)—that offers reasonably high image classification despite using a small number of moments.

Methods Figure 1 illustrates the functional design of our GDS system. We explain the methods developed for each processing stage.

Pass I: Using MLP and SVM In the absence of any guiding principle to determine the size of the MFS, we devised an accumulative feature subset selection strategy as follows: (a) Generate training-set by incrementally adding the next higher order moments to an existing training set. We exploited the intrinsic partitioning of the 254 moments in terms of their order ranging from 1 to 29. Therefore, feature subset1 included moments with order2, feature subset2 includes moments with order 2 and 3, and so on. In total 29 different training sets were generated, where each training set covered all the images based on the moment orders chosen to represent it; (b) Train both a MLP and a SVM classifier separately using the 29 training sets. In total, 29 different MLP and SVM classifiers were trained. For training the MLP and SVM, we partitioned the images so that 75% images were used for training and 25% images were used for testing. For training the SVM, based on the training data a 5-fold cross validation was performed to find the optimal hyper parameters: C and O; and (c) Determine the classification accuracy of both classifiers, using the test images that are represented by the same number of moments as used to train the classifier.

Figure 1 – Functional design of our GDS system Stage 1: CSLT image processing This stage involves the extraction of shape-defining features from CSLT images. These features are used to develop the image classification and clustering models. We use an image processing technique referred to as Moment Methods that describes the properties of connected regions in binary images as Moment features. We use Zernike moments [7] which use a set of complex polynomials to describe the image’s properties by their order (n) and repetition (m) with respect to a digital image—the low order moments capture gross shape information and high order moments incrementally resolve high frequency information (representing detail) of the digital image. Two attractive features of Zernike moments for our purpose is that (a) moments can be made invariant to shifts, rotations and magnification changes; and (b) the optic nerve is centered in the image, thus avoiding the requirement for an independent segmentation stage in which the object is explicitly identified.

The next step was to determine the size of the MFS and based on it to select the most efficient MLP and SVM classifier. Our objective was to select the largest possible number of moments without compromising the classification accuracy. To do so, we plotted the classification accuracy of both classifiers and then identified the highest accuracy point on the plot (i.e. with respect to n moment groups) just prior to a downward trend in the classification accuracy as a result of the inclusion of the next higher moment group. The most low order moment groups that achieved the highest classification accuracy were selected as the MFS. And, the MLP and SVM classifiers trained using the MFS were deemed as the most efficient.

For each CSLT image we generated 254 Zernike moments, grouped in an incremental order ranging from 1 to 29— each group comprises a set of ordered moments. Low order moments capture fundamental geometric properties and high order moments represent detailed information of the image [7]. For efficient classification of CSLT images, it was important to select an optimal number of lower order moments. This is a non-trivial task because: (a) there is no objective measure to determine the exact number of (low order) moments needed to achieve high classification accuracy; and (b) there is no discernable relationship between the moments that can be utilized. Given these challenges, next we pursue feature subset selection in conjunction with image classification.

A comparison of the classification accuracy trends for both the MLP and the SVM classifiers (see figure 2) shows that both classifiers exhibited a similar classification accuracy trend—i.e. they both start with a relatively high accuracy with the first moment group and then the accuracy drops with the addition of the next few moment groups. But later the accuracy starts to pick up again such that for the MLP it peaks when the feature subset constitutes the first 8 moment groups, whereas for the SVM the accuracy peaks for the first 11 moment groups. It is interesting to note that the classification accuracy with higher order moment groups is relatively low as compared to the peak achieved with just the lower order moments.

Stage 2: Classification of CSLT images In the previous stage, we derived a 254 moment based representation for each CSLT image. In this stage, we pursue the classification of CSLT images based on a sub-set of low order moments. This stage therefore involves two tasks—i.e. firstly feature (sub-set) selection and secondly

1310

S. SR. Abidi et al. / Automated Interpretation of Optic Nerve Images: A Data Mining Framework for Glaucoma Diagnostic Support

21, 37 46} were within the Markov blanket of the class label. These six moments were selected to form the OMFS. Step 5: In order to determine the classification capability of the selected OMFS we used them to train a BN. Next, 5-folds cross validation's classification accuracy was calculated (see table 2 for results) and it was noted that the OMFS offers quite high classification accuracy.

Based on classification accuracy trend for both classifiers (shown in figure 2), we determined the MFS to contain the first 11 moment groups—i.e. the first 47 moments. With 11 moment groups the SVM exhibited the highest accuracy and the MLP produced its second highest accuracy level. 0.9 0.85

Stage 3: Clustering of CSLT images

0.8

In this stage we pursued the clustering of the CSLT optic nerve images, represented using the 47 moments in the MFS, to differentiate between the different subtypes of healthy and glaucomatous optic nerves. It may be noted that an important theme in glaucoma research is to develop an understanding of the large variation in the appearance of the optic nerve, both within groups of healthy subjects and in patients with glaucoma. It is therefore important, from a clinical standpoint, to recognise and differentiate between such patterns. However, the problem with the sub-classification of patterns of optic nerve damage is that it is a subjective task, giving rise to considerable levels of disagreement between trained experts. In this context, our aim was to develop an objective and automated method to characterize optic nerve images.

MLP

0.75

SVM 0.7 0.65 0.6 1

3

5

7

9

11

13

15

17

19

21

23

25

27

29

Accu m u lative F eatu re Su b set Gro u p

Figure 2 - Classification accuracy for both MLP and SVM Pass II: Using Markov blanket and Bayesian network In the second pass, we attempt to further reduce the size of the MFS in order to generate the OMFS that comprises only the highly salient moments. We use a filter model based on a Bayesian Network (BN) and the Markov blanket of the class label [8]. The choice of Markov blanket is guided by the observation that the correlation between most of the moments and their class label is weak, and the same is true for correlation between different moments. Hence, correlation based feature selection methods are not suitable here. We decided to use the Markov blanket approach as it considers every feature’s probability dependence relationship during the learning procedure of the Bayesian network's structure.

Our two phase clustering strategy was to: (a) partition the images into distinct clusters using Self Organizing Maps (SOM); and (b) draw clear and distinct boundaries around the clusters using the Expectation Maximization (EM) algorithm [9]. Phase A: Data clustering using SOM We used a SOM to cluster the CSLT images based on the similarities between image shapes, where each cluster may represent a different subtype of healthy and glaucomatous optic nerves. The training of the SOM was conducted as follows: (i) we determined the topology of the SOM to be hexagonal lattice comprising 192 units that were arranged as 16 rows and 12 columns; (ii) The units were linearly initialized along the two greatest eigenvectors of the covariance matrix of the training data—i.e. images represented using the 47 moments in the MFS; (iii) The SOM was trained using a sequential training algorithm by first running a rough training phase comprising 100 epochs starting with a large neighbourhood radius of 12 that was linearly reduced to 3 with a learning rate of 0.5. This was followed by a second fine-tuning phase comprising 1000 epochs with a small initial neighbourhood radius 3 that was reduced to 1 with learning rate of 0.1. In both cases a Gaussian neighbourhood function was used and the learning rate function was set to be inversely proportional to the training epochs; (iv) Finally, we achieved a trained SOM that placed similar images into close proximity, thus leading to the image clusters. We applied principal component projection to the learnt SOM to determine its projection. This was followed by developing a U-matrix representation of the learnt SOM by spreading a colour map on the projection. Based on the visualization offered by the SOM, it was noted that the data was partitioned into discernable clusters.

In a BN where CA is the set of children of node A, and QA is the set of parents of node A, the subset of nodes containing QA, CA and the parents of CA is called Markov blanket of A [8]. The Markov blanket of a specific feature is a subset of nodes in the BN; it comprises the feature's parent nodes, child nodes and all parent nodes of the child nodes. If we consider the class label node as the root node to learn a BN from data, then all nodes within the Markov blanket of the class node have probabilistic dependence relationship with it. The steps to generate the Markov blanket were as follows: Step 1: We used the K2 algorithm to learn the BN. Initially, the 47 moments in the MFS were discretized using an entropy-based method, resulting in 29 moments to be discretized into a single value. These moments were removed from the MFS. Thus we were left with only 18 moments for training the BN. The following moments were retained: moments {1, 2, 5, 6, 7, 12, 16, 21, 23, 25, 27, 33, 36, 37, 43, 44, 45, 46}. Step 2: A BN was trained using the18 moments in their original order. Five-fold stratified cross validation was used to evaluate the classification accuracy (see table 2 for results). Step 3: The 18 moments were ordered based on the chi squared statistical test score x2 between the moments and their class labels. The moments with the highest x2 were: {1, 43, 16, 25, 21, 23, 6, 5, 36, 2, 27, 33, 37, 7, 46, 45, 44, 12}. A BN was learnt using the ordered moments (see table 2 for results). Step 4: From the BN learnt in step 3, we inferred the Markov blanket of the class label and found that only six (6) moments {1, 6, 16,

Phase B: Defining the cluster boundaries After determining broad clusters of CSLT images, in this phase we objectively determine the cluster boundaries. The processing was guided by our assumption that the dis-

1311

S. SR. Abidi et al. / Automated Interpretation of Optic Nerve Images: A Data Mining Framework for Glaucoma Diagnostic Support

fier trained with the OMFS to distinguish between healthy and glaucomatous optic nerves.

tribution of the clusters within the learnt SOM is Gaussian. Therefore, we used the EM algorithm [9] as it is suitable to find distinct components in the case of Gaussian mixtures. Functionally, the EM algorithm initiates with an estimate of the number of components and their parameters. Our strategy was to maximize the likelihood of the optic nerve images into distinct clusters given the parameters and a maximum likelihood measure that indicated how well the Gaussian mixtures fit the data into clusters. We used a Bayesian Information Criterion (BIC) [9], where the best estimate (e.g., number of clusters) was chosen based on the highest BIC value.

Table 2 - Classification accuracy for different classifiers Feature Subset Size

2

3

4

5

6

7

254 moments

MLP

72.88%

254 moments

SVM

77.50%

47 moments in MFS

MLP

74.00%

47 moments in MFS

SVM

86.96%

18 moments (original order)

BN

77.21%

18 moments (chi2 order)

BN

80.88%

6 moments in OMFS

BN

83.82%

Pass II

6 moments in OMFS

SVM

80.26%

6 moments in OMFS

MLP

72.84%

Evaluation 2: Examining the CSLT image clusters Evaluation of the clustering stage involved mapping a series of optic nerve images for individual patients (i.e. test cases with explanations provided by experts) onto the SOM and noting the Compactness Factor (CF) between the activated units. The CF measures how close the images are with respect to each other in terms of the average distance between the centroid of all active units. The CF is an objective measure for evaluating the clustering goodness based on our initial observation that for a patient the series of optic nerve images are quite similar over a period of time; over time the differences are quite minute and should not lead to large variations between consecutive images. This implies that when visualizing the optic nerve images for a subject, the active units should be in close proximity and therefore yield a low CF.

Table 1 – Number of clusters vs. BIC values

BIC

Accuracy

Pass I

To achieve the cluster boundaries, using the EM method with BIC, we initialized the EM using 10 random re-starts method, and then selected a parameter setting to maximize the log-likelihood of our clusters from the SOM. EM clustering was performed for different number of clusters. Table 1 shows that the maximum BIC is achieved when K = 4. Hence, we determined that given the learnt SOM there are 4 clusters—one cluster represents health images and the three clusters for sub-types of glaucomatous images— in it that best fit the data (see Table 1). To finalize the cluster boundaries for the 4 clusters, we calculated the assignment probabilities of each CSLT image to all the cluster labels, the cluster label with the highest probability value was assigned to the CSLT image. Figure 2a shows the SOM with the emergent clusters, the clusters are coded using grey scale for visualization purposes.

K

Classifier

8

29100 30409 31354 30516 29125 27456 25486

Evaluation and discussion In this section we present the evaluation results for the various methods developed for stages 2 and 3 of our GDS system.

Figure 2a show that the results for patient 4209643, and it maybe noted that the 7 optic nerve images, taken over a period of time, map on to a single SOM unit resulting in a compactness factor of to 0. The numeral within the active unit shows the number of images mapping on to that unit. This demonstrates the best possible clustering outcome as the learnt SOM recognizes the similarity between all the ‘healthy’ optic nerve images for this patient. Figures 2b shows the 11 optic nerve images of patient 112455 being mapping on to 4 neighboring SOM units within one cluster, with a compactness factor of 0.20808. This result again implies the close proximity of the images for this patient. These results are in line with the visual observations of these images by experts, who also concurred that the images for these patients are quite similar in shape.

Evaluation 1: Evaluating CSLT image classification Table 2 presents the CSCLT image classification accuracy for the different classifiers trained in phase 2, using test images not previously seen by the classifiers. It is interesting to note that both the MLP and the SVM classifiers offered higher accuracy with the MFS as compared to the original 254 moments. This vindicates our hybrid feature sub-set selection strategy, and also confirmed the theoretical assumption that low order moments contain more shape information that is relevant for classification as compared to the information content of high order moments. In the second pass, we determined that the MFS could be further reduced to just 6 moments—i.e. the OMFS—without compromising the classification accuracy. The highest accuracy for MFS was offered by the SVM—i.e. 86.96%. The highest accuracy for the OMFS was 83.82% offered by a BN. Therefore, the compromise in classification accuracy is just 3 %, yet the gain in computational efficacy is quite significant. Note that the BN (with Markov Blanket) offers the most optimal classification results when compared with both MLP and SVM trained on the OMFS. We therefore selected the BN classi-

We use the learnt SOM to visualize the disease progression for a patient over a period of time. Images taken over time for a patient were mapped onto the SOM. The pattern of the active units indicated the potential progression of the disease from one cluster to another, where each cluster may represent images of a specific glaucoma sub-type. In Figure 3a the images fall into two adjoining clusters, and the path across the clusters suggests the progression of the

1312

S. SR. Abidi et al. / Automated Interpretation of Optic Nerve Images: A Data Mining Framework for Glaucoma Diagnostic Support

disease from one sub-type to another. Figure 3b shows the progression over time.

Concluding remarks We presented a data mining framework to objectively analyze medical images, and applied it to investigate glaucoma. The novel features of our approach are that: (a) we process CSLT images to derive shape information by using image processing techniques. This is in contrast to the traditional approach of using morphological features to analyze CSLT images; (b) we have developed a feature selection strategy that identifies the most salient imagedefining features without compromising the classification accuracy; and (c) we are able to visualize the CSLT images in terms of clusters of similar images. These clusters provide an opportunity to visualize the dispersion of multiple observations for a subject, and we show how this information can help to (i) determine a potential progression of the disease due to changes in the optic disc over time; and (ii) identify noisy CSLT images. We believe that our framework takes a step towards the automated and objective analysis of optic nerve images to support glaucoma diagnostics.

Figure 2a – SOM showing Figure 2b – SOM showing all images mapped to a unit all images mapped in one cluster Evaluation 3: Visualizing disease progression over time

Acknowledgments This project is supported by a research grant from Nova Scotia Health Research Foundation, Canada.

References [1] Hoskins H, Kass M. Primary open-angle glaucoma, BeckerShaffer's Diagnosis and Therapy of the Glaucomas. The C.V. Mosby Company, St.Louis, 1989: 277-307. [2] Zinser G, Wijnaendts-van-Resand RW, Dreher AW. Confocal laser tomographic scanning of the eye. Proc. SPIE 1161, 1980: 337–344, [3] Bowd C, Cban K, Zangwill LM, Goldbaum MH, Lee T, Sejnowski TJ. Comparing neural networks and linear discriminant functions for glaucoma detection using confocal scanning laser opthalmoscopy of the optic disc. Investigative Opthalmology & Visual Science, Vol. 43 (11), 2002. [4] Park J, Reed J, Zhou Q. Active feature selection in optic disc nerve data using support vector machine. IEEE World Congress on Computational Intelligence, 2002. [5] Swindale NV, Stjepanovic G, Cbin A, Mikelberg F. Automated analysis of normal and glaucomatous optic nerve head topography images. Investigative Opthalmology & Visual Science, Vol. 41 (7), 2000. [6] Broadway DC, Nicolela MT, Drance SM. Optic disc morphology on presentation of chronic glaucoma. Eye 17 (6), 2003: 798-799. [7] The CH, Chin RT. On Image Analysis by the methods of moments. IEEE Trans. Pattern Analysis by Machine Intelligence 10(4), July 1998: 96-513. [8] Hruschka Jr. ER, Hruschka ER, Ebecken NEF. Feature selection by Bayesian network. Canadian AI Conference, LNAI 3060, Springer Verlag, 2004:370-279. [9] McLachlan G, Krishnan T. The EM algorithm and extensions. New York: John Wiley & Sons, 1997.

Figure 3a - SOM showing Figure 3b – The disease the dispersion of images progression path. Note the over two adjoining clusters high CF between the images

Figure 4a - SOM showing a Figure 4b – The spike shows noisy image that is distant that the 2nd image is noisy, as from the other images. it does not follow the pattern. Evaluation 4: Identifying noisy CSLT images We used the learnt SOM to identify noisy CSLT images that typically occur due to various factors related to the capture of the optic nerve image. With the knowledge that consecutive images do not manifest drastic changes, if an image is noted to be significantly dissimilar from its neighbors it can be regarded as a noisy image. At present there are no objective means to identify noisy CSLT images. Figure 4 (a-b) shows 14 images for a patient, where the 2nd image is identified as a single noisy image because it is in a different cluster, whereas the remaining images all map onto just two other units that are very close to each other.

Address for correspondence Syed Sibte Raza Abidi, Faculty of Computer Science, Dalhousie University, Halifax, B3H 1W5, Canada. Email: [email protected]

1313

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Intelligent Querying and Exploration of Multiple Time-Oriented Medical Records Denis Klimova, Yuval Shahara a

Medical Informatics Research Center, Ben Gurion University of the Negev, Israel

(WBC) counts) and events (e.g., bone-marrow transplantation (BMT)), which create the necessary interpretation context (e.g., the therapy protocol used). The output includes a set of interval-based, context-specific parameters at the same or a higher level of abstraction and their respective values (e.g., a period of nearly 3 months of grade 0 bonemarrow toxicity in the context of that therapy protocol).

Abstract Querying and analyzing multiple time-oriented patient data is a key task during medical research, clinical trials or the assessment of the quality of therapy. In this paper, we present several aspects of the VISITORS system, which includes knowledge-based tools for graphical querying and exploration of multiple longitudinal patient records. We focus on the syntax and semantics of the knowledgebased aggregation query language for multiple time-oriented patient records, and on the graphical queryconstruction interface. The query language assumes an underlying computational method for deriving meaningful abstractions from single and multiple patient records, such as we had previously developed. The aggregation query language enables population querying using an expressive set of constraints. By using our underlying temporal mediator architecture, the time needed to answer typical temporal-abstraction aggregation queries on databases of 1000 to 10000 patients was reasonable.

Furthermore, the output temporal abstractions can be efficiently visualized. The KNAVE-II system, which we developed previously [2], supports the visualization and exploration of raw data and derived temporal abstractions for an individual patient or small number of patients. Evaluation of the KNAVE-II system in the oncology domain [3] has demonstrated that, by using KNAVE-II and its underlying temporal abstraction computational architecture, physicians can more quickly and more accurately answer clinical queries about patients. However, to analyze clinical trials, or for quality assessment purposes, an aggregated view of a group of patients is more effective than exploration of each individual record separately. In addition, certain patterns can only be discovered through the analysis of multiple patients. Therefore, we have extended the KNAVE-II system into a system called VISualizatIon of Time-Oriented RecordS (VISITORS) [4] which supports the visualization of a group of time-oriented records at different levels of abstraction.

Keywords: intelligent visualization, temporal abstraction, multiple patients, medical informatics, Human-Computer Interfaces

Introduction: Knowledge-based exploration of multiple time-oriented records A key task facing clinicians and medical researchers is the analysis of time-stamped, longitudinal medical records, in particular records of multiple patients. This capability is necessary to support, for example, quality assessment tasks, analysis of clinical trials, or the discovery of new clinical knowledge. Although the task of assessing patient data has been mostly solved through the increasing use of Electronic Medical Record (EMR) systems, there still remains the task of intelligent processing of multiple timeoriented patient records, including the capability for interactive exploration of the results. Standard means, such as tables, temporal statistical tools, or more advanced temporal data mining techniques, are often insufficient or can help only in particular cases.

The following three important features distinguish the VISITORS framework from other exploring data tools : 1. Time-oriented data are graphically displayed and explored for both individual and multiple patients. 2. The temporal dimension is a first class citizen. It can be explored in various granularities, such as hour, day, and month. We also support a calendar timeline and a timeline relative to special events (e.g., the six months following a particular intervention). 3. The computational reasoning supports not only a view of raw time-oriented data and their statistics but also a meaningful summarization of the raw data, based on the temporal-abstraction domain ontology and the KBTA computational mechanisms. The exploration interface is also based on that ontology, which supports a semantic exploration of the data and enables navigation of semantically related raw and abstract concepts. For example, the user can explore graphically an instance of a pattern that was derived by the KBTA pattern-detection inference mechanism and view all the

To solve the computational aspect of this problem, we have been using the knowledge-based temporal abstraction (KBTA) method [1] for automated derivation of meaningful context-specific interpretations and conclusions, called temporal abstractions, from raw timeoriented patient data, using a domain-specific knowledgebase (KB). In general, the KBTA method is defined as follows: The input includes a set of time-stamped parameters (e.g., platelet, red blood-cell (RBC), and white blood-cell

1314

D. Klimov et al. / Intelligent Querying and Exploration of Multiple Time-Oriented Medical Records

GetPatients (KB, DB, <PatientsConstraints>) Ÿ <patients>,

abstract components and raw data from which the pattern was derived. In this paper, we explain in detail the syntax and semantics of the VISITORS query model, which directly affect the semantics of the computational and display modules. We also introduce several of the graphic modules which we have implemented to assist users in the interactive definition of temporal aggregated queries and exploration of multiple time-oriented records. Finally, we provide results of a preliminary functional evaluation and discuss the implications of the VISITORS framework.

where GetPatients is an external procedure that queries the selected DB, KB is an appropriate knowledge base that includes the parameters definition used in <PatientConstraints> - a set of complex conditions defining criteria for patient selection, and <patients> is a resultant list of identification data of patients who satisfy the set of <PatientConstraints>. The <PatientConstraints> aspect of the query includes the list of Boolean and temporal conditions of three types:

Methods

•

General architecture The VISITORS system is an intelligent interface to a distributed architecture specific to the tasks of querying, knowledge-based visualization, and interactive exploration of time-oriented data. We assume that the necessary elements of the temporal abstraction framework (shown in Figure 1 by striped lines) are available. Figure 1 describes the overall architecture: End users (clinicians) interact with Query Builder of VISITORS to submit time-oriented queries regarding patients. The Temporal Abstraction Mediator, for example our previously described systems, the goal-directed IDAN Mediator [5], integrates the relevant data and knowledge from the appropriate sources, indicated by the user, to answer queries regarding raw data or to derive using a Temporal Abstraction Service the abstract time-oriented concepts. The resultant data can be visualized and explored by the user.

•

Demographic or non-temporal constraints (i.e., only the last value is relevant) such as patient’s ID, age, sex, physician, etc. The user can define Boolean constraints among the attribute’s values (i.e., OR/AND logical operators). However, we do not recommend using complex Boolean expressions, since they make understanding why a patient was included in the output less intuitive. The logical operator NOT was omitted for a similar reason. Time and value constraints for both raw and derived concepts. To construct the query, the user can define constraints on the value of a concept, its duration, and its start/end points (See Figure 2). Both absolute (i.e., calendaric) and relative (i.e., measured from a reference event) timelines are supported. Defining pair-wise temporal relations between interrelated concepts is supported using Allen’s temporal logic relation as well as relations among specific time points (i.e., start/end). We use the conjunction logical operator between the time and value constraints. Note that the computational functions and procedures that enabled us to derive the abstract parameters are defined in the knowledge base as part of the domain ontology. The name of the KB is specified in the query.

Figure 1- The distributed VISITORS architecture Semantics of the temporal aggregation queries Figure 2-Time and value constraints for concept

We assume that the clinical time-oriented databases (DBs) have the following virtual patient record structure: the patient identification data (e.g., patient ID), name of entity (e.g., measured parameter, medications, interventions, etc.), time of laboratory test or medication and value. For example:

•

Statistical Constraints enable the user to aggregate and filter the patient’s data on the basis of a specific statistical function. Using such constraints, the user is able to investigate who are the patients in the database who have specific values (or a range of values) within a given statistical range of threshold values. For example: “Select all patients whose state of WBC was derived as “low” or “very low” during more than 25% of the period Jun 1 – Oct 31 2006”. Figures 3 & 4 show an example of constructing a Select Patients Query with both demographic and knowledgebased constraints, whose informal definition is “Select all male patients, either younger than 20 or older than 70, whose hemoglobin (HGB) state was abstracted at least as

<John Smith,WBC,11.01.06 13:45:00,6.7*103 cells/ml>, In our work we have designed and developed a formal query language, which is ontology-based, i.e., we use domain knowledge to formulate and display the queries. We distinguish three types of aggregated queries. Select patients query This query retrieves a list of patients from selected database who satisfy a set of constraints, defined by user:

1315

D. Klimov et al. / Intelligent Querying and Exploration of Multiple Time-Oriented Medical Records

“moderate low” or higher, during at least seven days, starting at least two weeks after the allogenic bone marrow transplantation(BMT),and whose WBC counts were increasing during the same period.” That is, either young or old patients whose bone marrows have been recovering following the BMT procedure. The bottom part of the interfaces shown in Figures 3 and 4 displays the query that is automatically and incrementally being created from the user’s graphical specification (displayed in the top part). The graphical interface used for query construction has a similar structure to the main exploration interface of the VISITORS system, which we discuss in the next section (e.g., the ontology browser is displayed on the left side; the panel display used for query definition is similar to the one used for data display, etc.). The highlighted rectangular area denotes the ranges of the time and value constraints on the patient data or abstractions relevant to the query.

Figure 4-Definition of a query's knowledge-based constraints

Select time intervals query Given a set of time-oriented patient data, this query returns a list of time intervals that satisfy the constraints defined by the user. In other words, the goal of this query is to find when a certain portion of the patients has a specific value or value in a predefined value range.

For example, a typical Select Time Interval Query is: “Find [relative] time intervals following the BMT, during which the WBC count was increasing, and the state of the Platelet count was “normal” or higher, for more than 30% of the patients.” Its formal XML based definition would be:

Formally, the Select Time Interval Query has the following structure:

<SelectTimeIntervalsQuery>

GetTemporalIntervals (KB,DB, , [refPoint]) ? relation*(<start_time, end_time>),

) is a data structure that includes a list of temporal intervals distinguished by start_time and end_time time points. The optional parameter refPoint requires that the output be calculated from the denoted reference point (e.g., a key clinical procedure).

min_theshold=”30” max_theshold=”100”/>

is a logical expression without nested parentheses of the following atomic constraints related by conjunction logical operator:



,

Get patients data query

max_value=”very high”





max_thresh,

Given a list of patients ID’s and, optionally, a list of time intervals, the query retrieves the patients’ raw data, or the derived temporal abstractions of one selected concept within the selected time intervals for the selected patients. The default patient list is all of the patients in the DB, and by default there are no time-interval constraints. The formal expression of the query is the following:

where Concept is a concept name, the min_thresh and max_thresh are minimal and maximum thresholds values in percents of patients, and min_value and max_value is a range of values of the selected Concept.

GetDataConcept(KB,DB,Concept,<patients>,[])?

relation*(<patientn, start_timen,m, end_timen,m,valuen,m>), 1 n N, 1 m M, where GetDataConcept is an external procedure that queries the selected DB, KB is an appropriate knowledge base that includes the Concept definition, <patients> is a list of patients identification data, and is an optional parameter that constrains the time of the returned

Figure 3-Definition of a query's demographic constraints

1316

D. Klimov et al. / Intelligent Querying and Exploration of Multiple Time-Oriented Medical Records

that concept for the selected group of patients. In this case, the user explores the data of a group of patients named My patients (58 patients), previously retrieved by the Select Patients query. The 1st panel from the top displays all of the WBC laboratory test values during March 95. The top (red) line represents the daily maximal value of the WBC count. The 2nd panel shows the daily mean value of HGB for each patient during 1995. The top (red) line and bottom (blue) line represent, respectively, the monthly maximal and minimal patient HGB values within the group. The 3rd panel displays the distribution of the WBC state-abstraction values for each month of 1995. For example, in Aug 95, 29 % of patients in the group have had a moderately_low value. The bottom panel displays statistical and temporal associations within the specific time period among the selected raw-data or abstracted concepts. In this case, WBC and RBC values are displayed over March 1995, and the Platelet and HGB values over 1995. Only 25 patients in the group have data in the selected time intervals. Values of all parameters for each patient are connected by lines.

data. The output relation*(<patientn, start_timen,m, end_timen,m, valuen,m>) is a data structure that includes the time-oriented data of N patients in <patients> list with M data records for each patient. In contrast to the previous two queries, the Get Patients Data Query does not have a separate interface. This query is constructed automatically during the users’ exploration of the patient’s data in the main interface of VISITORS. The main interface in VISITORS We described previously several visualization tasks in VISITORS [4]; here we intend to explain only the main design principles of the interface. The main interface in the VISITORS system is divided into three logical parts (See Figure 5): The top panel (A) is used for the patients selection tasks. The user can select previously retrieved groups from the table, choose patients from a DB, input the patient ID by himself, or construct a new Select Patients Query. The middle small panel (B) is used for time interval selection. The user can use the previously returned intervals, define a new interval, or construct a new Select Time Intervals Query. Note that both calendaric and relative timelines are supported. The main part of the interface (C) is used to explore the patients' time-oriented data. The left side includes a browser to the clinical domain ontology, retrieved on the fly from the relevant domain KB. Clicking on a concept node in the ontology tree displays the data of

The functional evaluation We are currently performing a functional and usability evaluation of the Query Builder module in an oncology domain. Evaluators were asked to construct eight Select Patients and two Select Time Intervals queries at different difficulty levels (Examples of queries are showed in Table 1).

Figure 5-The main VISITORS interface

1317

D. Klimov et al. / Intelligent Querying and Exploration of Multiple Time-Oriented Medical Records

Table 1 – Examples of aggregated queries Complexity

extension to standard SQL tools [6, 7]. However, these systems are limited in the temporal analysis by SQL capabilities. Moreover, they do not include the temporal reasoning mechanisms. In the area of visual querying, several tools have been proposed. The TimeFinder system [8] is a visual exploration and query system for exploring time-series data sets, based on a direct manipulation metaphor. Chittaro and Combi [9] provide visual representations framework of temporal intervals and relations. However, these techniques focus on query and exploration only of raw longitudinal data. Attempts to support intelligent query and retrieval are provided in the TimeLine system [10], and in knowledge-based spatial temporal query language (KSTL) [11] which support, however, only the medical imaging domain. Thus, we aimed to develop a domain-independent visual query and exploration system that enables clinicians to explore multiple longitudinal patient records, in an intelligent manner that supplies querying both raw data and their knowledgebased meaningful interpretations.

Examples of Queries

Select Patients Query Easy

Find male patients older 50 who have had during Sep 95 WBC laboratory test value less than 4000 cells/ml.

Moderate

Find all patients who had the HGB state value in the value range “low” to “normal,”, within the following time constraints: the episode of HGB starts between 3 to 4 days after allogenic BMT and its duration is at least 20 hours but no more than 60 hours.

Hard

Select male patients, whose bone-marrow toxicity grades have been decreasing for at least seven days during the period starting from 2 weeks after the allogenic BMT procedure, and whose liver toxicity grades were also decreasing during the same period.

Acknowledgments This research was supported by Deutsche Telekom Company and Israel Ministry of Defense award no.89357628-01

Select Time Intervals Query Easy

Hard

References

Find time intervals when the state of WBC was considered as “normal” or higher for less than 30% of the patients

[1] Shahar Y. A framework for knowledge-based temporal abstraction. Artificial Intelligence, 90(1-2), 1997. [2] Shahar Y, Goren-Bar D, Boaz D, Tahan G. Distributed, intelligent, interactive visualization and exploration of timeoriented clinical data and their abstractions. Artificial Intelligence in Medicine 38 (2), 2006, pp. 115-35. [3] Martins SB, Shahar Y. Evaluation of KNAVE-II: A tool for intelligent query and exploration of patient data. Proc. of Medinfo 2004. [4] Klimov D, Shahar Y.A framework for intelligent visualization of multiple time-oriented medical records. AMIA Annu Symp Proc. 2005, pp. 405-9. [5] Boaz D, Shahar Y. A distributed temporal-abstraction mediation architecture for medical databases. Artificial Intelligence in Medicine 34 (1), 2005, pp. 3-24. [6] Nigrin D, Kohane IS. Temporal expressiveness in querying a time-stamp–based clinical database. J Am Med Informatics Assoc, vol. 7, 2000, pp.152-63. [7] Combi C, Cucchi G. GCH-OSQL: a temporally-oriented object-oriented query language based on a three-valued logic. Proc of the 4th International Workshop on Temporal Representation and Reasoning, 1997 [8] Hochheiser H, Shneiderman B. Visual Specification of Queries for Finding Patterns in Time-Series Data. Proc. of Discovery Science 2001, 2001, pp. 441-46. [9] Chittaro L, Combi C. Visualizing queries on databases of temporal histories: new metaphors and their evaluation. Data and Knowledge Engineering, 44, 2003, pp. 239-264. [10] Aoyama D, et al. TimeLine and visualization of multipledata sets and the visualization querying challenge. Journal of Visual Languages & Computing 18 (1), 2007, pp.1-21. [11] Chu W, Alfonso F, and Ricky K. Knowledge-based image retrieval with spatial and temporal constructs. IEEE Transactions on Knowledge and Data Engineering 10(6).

Find time intervals following the BMT procedure during which more than 20% of the patients had “moderate” anemia, and more than 30% of the patients had “low” or “very low” WBC counts

In all tests we used a retrospective DB of more than 1000 oncology patients after bone-marrow transplantations (BMT), who were followed for two to four years. In addition, we assessed the performance of the overall architecture when answering these queries. Results show that performance times using IDAN architecture were reasonable: several seconds for answering aggregated queries with demographic and time and value constraints for raw and abstracted parameters. We intend also to evaluate the system in the diabetic domain.

Discussion In this paper we have presented the novel ontology-based multiple records query language, which enables users to construct a new query and retrieve a set of relevant patients or time intervals, using a broad set of constraints. Such aggregated queries, and the graphical Query Builder tool used to construct it, are part of the VISITORS system that enables clinicians to query, visualize and explore both raw time-oriented medical data and meaningful interpretations (including complex temporal patterns), derived from the these data, based on a domain knowledge base. The main advantage of our system is successful integration of different methods, such as information visualization and knowledge-based temporal reasoning.

Address for correspondence Mr. Denis Klimov, Tel: 972-8-6477160, email: [email protected] Department of Information Systems Engineering, Ben-Gurion University of the Negev, P.O.Box 653, Beer Sheva, 84105, Israel.

In previous studies the time-oriented aspect of querying was commonly addressed by adding a temporally-oriented

1318

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks for Image Retrieval Evaluation Henning Müller a, Célia Boyer b, Arnaud Gaudinat b, William Hersh c, Antoine Geissbuhler a a

Medical Informatics Service, University and Hospitals of Geneva, Geneva, Switzerland b Health On the Net (HON), Geneva, Switzerland c Oregon Health and Science University (OHSU), Portland, OR, USA

patients and medical professionals with a multilingual search interface1 [2]. Much research has been done on the searching of medical texts [3] but less on how images are used and searched for, although the amount of image data being produced is rising [4]. Many medical image databases are available within institutions, mainly for teaching, but some are also made available on the Internet. These include Casimage, HEAL (Health Education Assets Library), MedPix, and the Pathopic datasets. MIRC2 (Medical Image Resource Center) is an initiative of the Radiological Society of North America (RSNA) to unite teaching files under a single interface. These databases contain thousands of annotated images. Unfortunately, the images are only rarely indexed in search engines such as Google as they are usually only available through the search in database fields. Another problem is that the annotation is often incomplete and information on the image modality is not always given. A search for “lung CT” with Google image search in October 2005 brought 160 results, about half of them lung CTs. The abovementioned databases alone contain several thousand lung CTs.

Abstract Medical institutions produce ever-increasing amount of diverse information. The digital form makes these data available for the use on more than a single patient. Images are no exception to this. However, less is known about how medical professionals search for visual medical information and how they want to use it outside of the context of a single patient. This article analyzes ten months of usage log files of the Health on the Net (HON) medical media search engine. Key words were extracted from all queries and the most frequent terms and subjects were identified. The dataset required much pre-treatment. Problems included national character sets, spelling errors and the use of terms in several languages. The results show that media search, particularly for images, was frequently used. The most common queries were for general concepts (e.g., heart, lung). To define realistic information needs for the ImageCLEFmed challenge evaluation (Cross Language Evaluation Forum medical image retrieval), we used frequent queries that were still specific enough to at least cover two of the three axes on modality, anatomic region, and pathology. Several research groups evaluated their image retrieval algorithms based on these defined topics.

Outside of medicine, visual information retrieval has been an extremely active research domain for more than 15 years [5]. Studies on domain-specific user requirements have been performed, for example for journalists searching for images [6] or in the cultural heritage domain [7]. In the medical field, visual information retrieval has been proposed many times as extremely useful [8, 9]. Still, most research has a limited focus on retrieval for one particular group of images [9]. Although this might be a domain with high potential impact, teaching and research are more likely to profit first from possibilities to browse very large and diverse image collections by visual properties. In the context of ImageCLEFmed [10], a challenge evaluation for medical image retrieval, two surveys were performed among medical image users [11, 12] to find out more about typical information needs and search tasks. CLEF (CrossLanguage Evaluation Forum) is a challenge evaluation for retrieval of multilingual information. ImageCLEFmed in particular evaluates the quality of retrieval from multilingual medical image retrieval available on the Internet. The

Keywords: log files analysis, image retrieval evaluation.

Introduction An increasing amount of medical information is being produced digitally, making it available for further processing and use, i.e., for teaching and research. Much of the produced data and experiences from past cases can be used to create tools for diagnostic decision aid. A great deal of medical information is also available on the Internet, as there are increasing requests for medical information by patients and professionals [1]. MedLinePlus is one example of a repository created to inform non-professionals, patients searching for information. Another example is Health On the Net (HON), which develops quality criteria for medical web pages and has an accreditation service for pages adhering to several quality criteria. HON also runs web search engines for medical web content aimed at

1 2

Selected for best paper award.

1319

http://www.wrapin.org/ & http://www.hon.ch/HONselect/ http://mirc.rsna.org/

H. Müller et al. / Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks

surveys include five user groups: medical professionals for diagnosis, teaching, and research as well as medical students and librarians. The goal of the work descried in this paper was to create realistic search tasks for ImageCLEFmed3 based on information needs of web users. The analysis resulted in 30 search tasks used by participating research groups. Among the techniques used was analysis of log files, an active research domain [13], mainly to analyze web page design.

•

Unify coding issues, to remove accents, Umlauts, national symbols, and any sort of non-text: –“()+–. • Remove commands and options send by web robots or search engines. • Remove URLs or fragments of URLs. • Convert all characters to lower case. • Change plural of frequent terms such as “images”. • Remove frequent terms to define the target media: image(s) (5’796), media (512), video(s) (334). Over 100 rules for normalization and removal were defined and applied to clean the data. Even after the removal steps, it was apparent that an extremely large number of different queries remained. In total, there remained 5’365 different unique queries (of 16’677) for the Google queries and 17’643 different HON queries (of 37’293). This meant that almost half the queries were unique being asked only once, which made a systematic evaluation of the entire dataset hard. The number of words per query was small. Google queries contained an average of 2.01 words in our study and HON queries 1.50 words, after removing the words image, video and media. This resulted in 191 empty queries for Google and 150 for HON. The same number of queries contained only a single character.

Materials and methods Used data sets The data used for this study were log files containing query terms of the HONmedia4 search engine. The examined period of queries included ten months, from January 1, 2005 to October 31, 2005. This period was sufficient for a representative evaluation of search terms. Variations of search frequency or quality over the months were not part of our analysis. The original data set contained 53’970 queries. With each automatically extracted query term, the date and time of the query was stored. It was also stored whether the query was directly done via the HONmedia interface or referred to from Google towards HONmedia search. Many queries were in French, as the French-speaking medical community frequently uses the HON query engine. It was not possible to perform an automatic translation of the topics, as language detection is hard with only very few words. Other languages identified for the queries were English, German, Spanish, and Italian.

Removal of unclear queries After term normalization, it became clear that there are queries unimportant for further analysis. First, a group of queries concerned sexually explicit queries: In the Google queries, the following frequent terms were removed: xxx (334 times). For HON the following terms were removed: penis (114), vagina (108), breast (102), sex (65), clitoris (32), gynecology (24). Another group of queries implicitly contained similar ideas; for Google these were: accouchement (childbirth, 143), cesarienne (33). For HON: home childbirth (239), nurse (130), birth (69). Third, another group of queries were processed to remove those not containing a precise information need, some of them, such as the term “search,” were simply placed by web robots trying to access information stored in web-accessible databases. For Google this included the following terms: medical images (508 times), HON (116), health (62), medical illustrations (32), repository (30). For HON, these terms included: search (1493), medical images (79), doctor (70), anatomy (65).

Pre-treatment of the data and evaluation techniques The analysis of the data was done on a Linux computer using Perl to analyze the text files. The original data sets were transferred to pure text and the information on time and date of the query were discarded. Perl was used mainly to pre-treat the data. As data were extracted automatically and as robots perform queries on web interfaces there are many different formats for queries (sometimes broken), plus a variety of international character containing umlauts and accented characters sets that need to be combined.

Results The data contained two groups of queries, queries directly asked via HON and queries forwarded via Google. These groups were treated separately. A total of 37’293 queries were directly performed via HONmedia and 16’677 were forwarded via Google.

Most frequent queries and terms After normalization and removal of queries, we analyzed the most frequent remaining terms. Table 1 shows the most frequent remaining terms forwarded from Google. This list contains very specific medical search requests, from specialists rather than patients. Most of the terms are in French, actually all of the most frequent 20. The specialized nature of the terms and the fact that they are in French can be explained with the fact that only these technical queries link towards HONmedia.

Text normalization First, normalization was necessary for the text to remove differences in coding of the strings, parameter options transmitted and for broken queries containing graphical symbols. We did not treat the word order in the queries. The steps were mainly based on a manual analysis of the data:

3 4

http://ir.ohsu.edu/image/ http://www.hon.ch/cgi-bin/HONmedia/

1320

H. Müller et al. / Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks

Table 1 - Most frequent terms forwarded from Google Term

Frequency

Nerf sciatique Kyste pilonidal Leucemie aigue myeloblastique Glossite exfoliatrice marginee Fracture humerus Grenouillette sublinguale Hematome sous dural Polype nez Appendice xiphoide Leucomalacie periventriculaire Leucemie Purpura rhumatoide Scarlatine Hematome retroplacentaire Kyste thyreoglosse Leucemie myelomonocytaire chronique Leucoplasie Apophyse odontoide Hidradenite Scoliose

154 76 72 67 66 60 57 56 53 51 46 46 44 40 39 39 38 37 37 34

is frequently used as it corresponds to the information needs. Table 4 - Frequent terms regarding symptoms

Frequency 381 242 211 160 101 98 93 77 69 69 67 64 60 58 58 58 57 57 55 54

64 58 42 37 30

Table 5 - Frequent terms regarding treatments Term

Frequency

Injection Surgery Stethoscope Anesthesia Vaccination

67 46 36 24 22

Table 5 lists terms concerning treatments, taken in a wide sense, as stethoscope is not a treatment. Table 6 - Frequent terms regarding anatomic region

Table 2 - Most frequent terms from the HONmedia search Heart Asthma Brain Diabetes Liver Cancer Marfan Kidney Lung Knee Injection Bacteria Eye Foot Pain Ear Pancreas Aids Blood HIV

Frequency

Bacteria Pain Burns Stress Blood pressure

Table 4 shows symptoms searched for, where symptom is taken in a broad sense. Bacteria is not a symptom but might be interpreted from patients with flu-like symptoms looking for more information on a particular situation.

Table 2 shows the most frequent terms directly queried with HONmedia. These terms are more likely to be from patients than specialists. The first 20 contain only a single word. More terms are in English than in French, actually all top 20, whereas a large number of the less frequent terms are in French. Most terms are of two groups: Terms describing an anatomic region or a disease. Only other terms found in the most frequent 20 are concerning symptoms or a treatment in the largest sense, such as injection, bacteria and pain.

Term

Term

Term

Frequency

Heart Brain Liver Kidney Lung

381 211 101 77 69

In Table 6, frequent anatomic regions are listed that correspond well to the most frequent causes of death [14]. Also the search terms regarding pathology correspond well to diseases mentioned in [14]. Only Marfan is surprisingly frequent. The 500 most frequent terms were analyzed accounting for almost half the search terms in total. Besides the identified five axes, some other terms are frequently queried, which are hard to classify: Human body (41), smoking (38), CPR (computerized patient record, 33), cardiology (26). It is hard to know what images or videos the users were searching for.

Classified term occurrences important for us Table 7 - Frequent terms regarding pathology

This section analyzes only queries directly from HON as they correspond better to our needs concerning patient information search. We particularly note the most frequent terms for anatomic region, pathology, imaging modality, symptom and treatment, as these are axes to model search tasks along.

Frequency

Ultrasound Ecg/ekg MRI X-ray Endoscopy

47 34/32 33 21 18

Frequency

Asthma Diabetes Cancer Marfan Aids/HIV

242 160 98 93 57/54

Constraints to define search tasks based on the results

Table 3 - Frequent terms regarding modality Term

Term

From the most frequent concepts and the average number of query terms it becomes clear that users express fuzzy information needs and describe them with few terms. As the information in the HON queries corresponded better to our goal, we only used these. It is clear that information needs are often broad and it seems to aim at general illustrations (CPR, human body, AIDS …) than towards precise images of a particular modality and anatomic region. Illustrations also need to be taken into account as

Table 3 shows modalities searched for. Interestingly, a commonly used modality (CT) is not mentioned often, whereas ECG, often discarded in medical image databases,

1321

H. Müller et al. / Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks

frequent query words such as doctor, nurse, injection or bacteria show.

The 30 query topics generated in this way were sent to all 40 participating research groups together with an image database. After retrieval experiments by participating groups and pooling of results, a group of physicians performed relevance judgments to compare the retrieval results of the participating retrieval systems. More about the results can be read in [15].

Table 8 - A collection of longer queries Term

Frequency

Autonomic nervous system Heart conduction system Artrite reumatoide juvenil Lupus vasculitis central nervous system Fetal alcohol syndrome Sickle cell anemia Epilepsy frontal lobe Respiratory distress syndrome adult Spinal cord compression Shoulder impingement syndrome

16 10 9 8 7 7 6 6 6 6

Discussion and conclusions The normalization of query terms that we applied is not completely sufficient for a system that is used in several languages. A translation of the terms towards a single language or terminology would be best but with most queries being single words, this is difficult. At least 10 languages were identified. Spelling errors and abbreviations were other problems. Part of this was corrected with manual analysis but a large number of queries for the same terms could not be combined.

Other queries contained expected concepts but not as detailed as desired. If looking for images of the heart, all modalities, views and pathologies combined produce an extremely large number of images to be found. Such tasks are not suited to find out more about the quality of a retrieval system. For this reason, we evaluated the most frequent queries with at least three words. Table 8 lists these frequent search terms. The table shows that several terms still contain a single concept (autonomic nervous system). Most queries contain two distinct concepts, either pathology and anatomic information (epilepsy frontal lobe) or a disease and a patient group (respiratory distress syndrome adult). Still, few of these queries can be taken as query tasks for a benchmark directly.

It can be seen that many queries for visual medical content are being performed with HONmedia search. About 52’000 queries in ten months is a large number for a small specialized search engine. Some queries are not for medical content but erotic, which is a phenomenon known by all search engines, particularly searches for images. Many queries are for illustrations of broad concepts, where the users seem to be willing to browse through a large number of varying results without a clear idea in mind and rather to illustrate an article or a presentation. Most queries are for a particular anatomic region or a certain disease. Users of the search engine do not seem to be used to supplying precise information needs concerning images. They follow the behavior of textual Internet search using broad concepts. Most image databases on the web are not well annotated and much of the information is incomplete resulting possibly in poor results.

Example Query tasks of ImageCLEFmed Finally, it was decided to use 30 real but rarer queries of the log files that cover at least two of the axes modality, anatomic region, and pathology. Example topics with example query images can be seen in Figures 1 and 2.

Compared to text analysis and retrieval, medical visual information retrieval is still in its infancy. Currently, large data sets are being created and made available. Still, the applied search methods are mostly based on text, only. Techniques for visual retrieval do exist [9] and if we want to apply them in real clinical settings we need to build prototypes and make users familiar with the techniques, the possibilities and the limitations. In this sense, ImageCLEFmed is an important initiative for bringing image retrieval systems closer to routine use, through evaluating their quality. To do so, the common image databases need to be shared and realistic visual information needs have to be defined. For this, resources such as the HONmedia log files are important for us as only few medical visual search engines exist in routine use. It is also important to educate users to define their information needs more precisely using text as well as visual means and also relevance feedback.

Show me chest CT images with nodules. Zeige mir CT Bilder der Lunge mit Knötchen. Montre-moi des CTs du thorax avec nodules.

Figure 1 - A visual query of ImageCLEFmed 2006

An interesting future research topic is the analysis of query terms over short time frames. How does this behavior change with respect to events in the world (such as the bird flu)? Could the beginning of a flu outbreak be detected through keyword changes for related terms? Medical

Show me microscopic images showing parvovirus infection. Zeige mir Mikroskopien mit einer Parvovirusinfektion. Montre-moi des images microscopiques qui montrent une infection parvovirale

Figure 2 - A semantic query for ImageCLEFmed 2006

1322

H. Müller et al. / Analyzing Web Log Files of the Health On the Net HONmedia Search Engine to Define Typical Image Search Tasks

image search on the Internet and in institutional databases has a high potential but more research is needed and particularly prototypes that can be made available to the users for testing to find out more about concrete information needs.

[8] Tagare HD, Jaffe C, Duncan J, Medical Image Databases: A Content-Based Retrieval Approach, Journal of the American Medical Informatics Association (JAMIA), 4(3):184-198, 1997. [9] Müller H, Michoux N, Bandon D, Geissbuhler A, A review of content-based image retrieval systems in medicine – clinical benefits and future directions, International Journal of Medical Informatics, 73, pp 1-23, 2004. [10] Clough P, Müller H, Deselaers T, Grubinger M, Lehmann T, Jensen J, Hersh W, The CLEF 2005 Cross-Language Image Retrieval Track, Springer Lecture Notes in Computer Science LNCS 4022, pp 535-557, 2005. [11] Hersh W, Müller H, Gorman P, Jensen J, Task Analysis for Evaluating Image Retrieval Systems in the ImageCLEF Biomedical Image Retrieval Task, Slice of Life, multimedia in medical education, Portland, OR, USA, 2005. [12] Müller H, Despond-Gros C, Hersh W, Jensen J, Lovis C, Geissbuhler A, Medical professionals’ image search and use behavior, Medical Informatics Europe, 2006. [13] Paliouras G, Papatheodorou C, Karkaletsis V, Spyropoulos CD, Tzitziras P, "From Web Usage Statistics to Web Usage Analysis," IEEE Conference on Systems Man and Cybernetics, 1999. [14] Anderson RN, Smith BL, Deaths: Leading causes for 2001, National Vital Statistics Report 52(9):1-86, 2003. [15] Müller H, Deselaers T, Lehmann T, Clough P, Kim E, Hersh W, Overview of the ImageCLEFmed 2006 Medical Retrieval and Medical Annotation Tasks, Springer Lecture Notes in Computer Science (LNCS), 2007 – to appear.

Acknowledgements This work was partially supported by the Swiss National Science foundation with grant 205321-109304 and by Grant 0325160 of the US National Science Foundation..

References [1] Rice RE, Influences, usage, and outcome of Internet health information searching: Multivariate results from the Pew surveys, International Journal of Medical Informatics, 75:8-28, 2006. [2] Gaudinat A, Boyer C, WRAPIN (Worldwide online Reliable Advice to Patients and Individuals). In MEDNET 2003, The 8th Annual World Congress on the Internet and Medicine, Geneva, Switzerland. [3] Hersh WR, Hickam DH, How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review, Journal of the American Medical Association, 1998, 280: 1347-1352 [4] Gould P, The rise and rise of medical imaging, physicsweb, 16(8), August 2003. [5] Smeulders AWM, Worring M, Santini S, Gupta A, Jain R, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12) pp 1349-1380, 2000. [6] Markkula M, Sorminen E, Searching for photos Journalists' practices in pictorial IR. In JP Eakins, DJ Harper, J Jose (Eds.), The Challenge of Image Retrieval Newcastle upon Tyne, UK, 1998. [7] Choi Y, Rasmussen EM, Users’ relevance criteria in image retrieval in American history, Information Processing and Management 38: 695-726, 2002.

Address for correspondence Henning Müller (PhD) University & Hospitals of Geneva, Medical Informatics Service 24, rue Micheli-du-Crest, 1211 Geneva 14, Switzerland Tel +41 22 372-6175, Fax +41 22 372-8680 [email protected].

1323

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Improving Computer Aided Disease Detection Using Knowledge of Disease Appearance Tatjana Zrimeca,b James S. Wongb, a

b

Centre for Heath Informatics, University of New South Wales, Australia School of Computer Science & Engineering, University of New South Wales, Australia

automated detection algorithms, being developed to analyse CT scans are texture based. The classical approach is to use a set of image features to describe the image content and to use some classification scheme to distinguish between different patterns. For example, Uppaluri et al. [2] used twenty-two independent texture features to characterise a tissue pattern in the overlapping square regions of the lung. A Bayesian classifier was trained to discriminate between six different patterns. Uchiyama et al. [3] proposed a similar texture based technique. They trained an Artificial Neural Network with twelve features, calculated on regions with different sizes, to classify new regions. The system was trained to distinguishing between seven different patterns, which included normals and six patterns associated with diffuse lung diseases.

Abstract The accurate quantification of disease patterns in medical images allows radiologists to track the progress of a disease. Various computer vision techniques are able to automatically detect different patterns that appear on images. However, classical pattern detection approaches do not perform satisfactorily on medical images. The problem is that texture descriptors, alone, do not capture information that is pertinent to medical images, i.e. the disease appearance and distribution. We present a method that uses knowledge of anatomy and specialised knowledge about disease appearance to improve computeraided detection. The system has been tested on detecting honeycombing - a diffuse lung disease pattern in HRCT images of the lung. The results show that the proposed knowledge guided approach improves the accuracy of honeycombing detection. A paired t-test, shows the improvement in accuracy to be statistically significant (p < 0.0001).

Our system, developed to automatically detect lung disease patterns, adopts a similar approach. However, we use a much bigger set of image attributes to describe the content of the image. We experimented with different attributes subsets and different learning schemes to improve the system’s performance. The results reveal that classical pattern detection approaches do not perform satisfactorily on medical images. The problem is that texture descriptors, alone, do not capture information that is pertinent to medical images, i.e. the disease appearance and distribution. Therefore we incorporated domain knowledge of lung anatomy and lung structure to help and improve image analysis.

Keywords: lung HRCT, lung diagnosis, computer aided diagnosis, medical imaging, honeycombing detection

Introduction Medical imaging systems are constantly improving in image quality because of increased image resolution. This results in a growing number of images that have to be inspected for diagnosis. For example, high resolution CT (HRCT) imaging protocols of the lungs can generate from 40 up to 600 images per study. These high-resolution axial images provide anatomic detail similar to that available from gross pathology specimens of lung slices [1]. Now radiologists can clearly see the alterations in lung anatomy caused by a disease process. Unfortunately, image analysis is still performed manually, which is often a difficult and time-consuming task. Consequently, there is an increasing need for computerised image analysis to facilitate imagebased diagnosis.

In this paper we focus on detecting honeycombing, an important diffuse lung disease pattern, in HRCT images of the lung. As the goal of the system is to provide radiologists with a second opinion on a lung diagnosis, it is important to achieve high accuracy. In this paper we present a new method developed to improve computeraided detection. The method uses specialised knowledge of disease appearance in axial images. It also uses information about lung regions that are used in radiology reporting [9]. To determine if using knowledge can significantly improve the system’s performance, we incorporated the knowledge-guided approach into two classification methods, one based on decision tree learning and the other using Naïve Bayes.

We are developing a system for computer-aided detection of diffuse lung diseases, a large group of disorders that primarily affects the lung parenchyma. They are characterised by specific abnormal findings, mostly texture-like in appearance. Consequently, most of the

In the remainder of the paper, we present a computer-aided detection system using a classical pattern detection

1324

T. Zrimec et al. / Improving Computer Aided Disease Detection Using Knowledge of Disease Appearance

approach. We then present the improved system and compare their performance.

mark up system [4], we collected a set of images with marked and labelled regions of honeycombing and other lung diseases patterns. Figure 2 (top) shows example of an HRCT image with marked regions. Having examples of image regions with and without a disease, we were able to use supervised machine learning to generate rules for recognising different patterns in HRCT images.

Materials and methods Lung diseases – honeycombing pattern Honeycombing is one of the main indicators of diffuse lung diseases. It can be seen in many diseases leading to end-stage pulmonary fibrosis. Honeycombing is characterised by small, uniform (2-10mm) cystic air spaces with well-defined thick walls, (See Figure 1 left). Honeycomb cysts usually form clusters that have the characteristic appearance of “honeycombing” on HRCT images. The visual appearance of honeycombing in cross-section scans is a combination of dark and light patches. It is one of the more difficult disease patterns to detect because honeycombing can often be mistaken for other normal structures in the lung, for example, bronchi and pulmonary vessels (see Figure 1 right).

The method consists of there main steps: 1. Data preparation: a) Image pre-processing and segmentation b) Feature extraction - calculating attributes for regions of interest 2. Knowledge generation - training: a) Feature selection – finding informative attributes for a particular disease pattern b) Generating rules via machine learning 3. Knowledge verification -testing the quality of the learned rules: a) On part of the training data b) On new data Data pre-processing and segmentation As we are interested in detecting patterns in the lungs, we first pre-process the images and segment the lungs. We have developed a lung segmentation technique based on adaptive thresholding, morphological operators and active contour snakes. Adaptive trhesholding is applied to segment the darker regions in the image that represent the airfilled lung. Morphological operators are then used to include structures within the lung that have a high attenuation. Active contour snakes [5] are used to generate the lung contours (see Figure 2 bottom row).

Figure 1 - Left lung- outlined region with honeycombing. Right lung –outlined example of bronchovascular structures, which has similar appearance as honeycombing. The HRCT images The HRCT images used for developing the computer aided disease detection system were obtained from a radiology practice. They were acquired using a SIEMENS scanner and a standard imaging protocol with 750ms exposure time. The HRCT generates volume data with spatial resolution 512x512 and 1.0mm slice thickness. For diffuse lung disease detection, radiologists usually use data with 10mm or 15mm slice gap, The data are stored in DICOM format as 16-bit greyscale images.

Feature extraction Having segmented the lung, we proceed to extract features from the image that best represent the underlying texture. A set of attributes was calculated for each pixel and it’s surrounding area. We used a ROI with two window sizes, 7x7 and 15x15 pixels, to capture the characteristics of small and larger honeycombing cysts.

Describing the disease

We calculate first and second order texture attributes and grey-level difference for each ROI [6]. The first order texture attributes measure the grey-level distribution within the ROI. Those attributes include: the mean HU1, variance, skewness, kurtosis, energy and entropy. The second order features describe the spatial distribution of the grey-levels within these ROIs. To do this, a co-occurrence matrix is calculated that specifies the frequency of a particular greylevel occurring near another grey-level. Each pixel, with it’s surrounding area, is represented by 63 attributes per window, resulting in a feature vector with 126 attributes, (63 for ROI7x7 and 63 for ROI15X15).

There are specialised texts that describe how to interpret HRCT images, accompanied by illustrative examples [1]. Although this highly specialised knowledge is very useful for humans, computers cannot make direct use of it. For computers to recognise a disease in medical images, the content of the image is represented by a set of image attributes, calculated for regions of interest (ROI) in an image. The values of these attributes depend on the characteristics of the regions. We can use these characteristic values to distinguish between normal and pathological regions as well as between different pathologies. The method The easiest way for a radiologist to communicate expert knowledge about how diseases appear in HRCT images is to provide examples. Using a specially developed image

1

1325

Hounsfield unit (HU) a unit used in medical imaging (CT or MRI scanning) to describe the amount of x-ray attenuation of each "voxel" (volume element) in the three-dimensional image.

T. Zrimec et al. / Improving Computer Aided Disease Detection Using Knowledge of Disease Appearance

P(c | F )

P(F | c)P(c) P(F )

(1)

where c is the class and F is the vector of features. The class with the highest probability is assigned to the ROI. Although the Naive Bayes approach is optimal when the features are independent, in reality it still works well without this assumption. The decision tree learner generates a tree in which nodes represent tests on attributes, branches represent attribute values and leaf nodes represent classes, i.e. decisions. More informative attributes appear higher in the tree. To train the system to detect honeycombing pattern, we used 42 images from 8 different patients that showed patterns representative of honeycombing and nonhoneycombing tissue. After prepossessing and segmentation, feature extraction and selection were performed as described previously. A set of 18,407 labelled region of interest were used for training the machine learning algorithms from which 9,467 ROIs contained honeycombing and 8,940 ROIs did not. Two classifiers were built, one using Naïve Bayes and the second using decision tree induction. Knowledge verification Ten-fold cross validation was used to estimate the accuracy of the learned classifiers. In this validation scheme, 90% of the training data are randomly sampled for learning and 10% for testing. This is repeated 10 times and the results averaged.

Figure 2 - Top: an image with marked and labelled regions by a radiologist showing disease patterns; bottom left: original HRCT image, bottom right: segmented lungs.

The performance of the classification during training and testing was evaluated by calculating accuracy, sensitivity, and specificity. In our case accuracy measures the proportion of the lung that is classified correctly. Sensitivity determines the proportion of actual honeycombing that has been detected as honeycombing. Specificity measures the amount of non-honeycombing that has been classified as non-honeycombing.

Knowledge generation process - feature selection With a feature vector of 126 dimensions, the classifier generated would be computationally intractable. We reduce the dimensionality by selecting a subset of features that best discriminates honeycombed and non-honeycombed regions. In this study, we use Correlation-based Feature Selection (CFS) [7]. CFS selects subsets of attributes that are highly correlated with the class and that have low intercorrelation.

Although the accuracy, sensitivity and specificity were comparable with the results published in the literature, we were not satisfied with the system’s performance. It produced some spurious honeycombing classifications in regions where honeycombing cannot appear.

Generating rules via machine learning Even with the reduced set of attributes, it is difficult to manually determine the attribute values that are characteristic for a particular pattern. We used supervised machine learning to automatically generate rules for discriminating between different patterns. In supervised learning, a set of pre-classified training examples is used to generate classification rules. In our case, the training examples consist of a set of attribute values representing a region with or without honeycombing pattern. The training set was prepared from the images with labelled regions provided by radiologists (see Figure 2 top).

Improvements based on domain knowledge Instead of developing post-processing methods for handling misclassifications, we decided to make use of domain knowledge about the lung structure as well as expert knowledge about the appearance of diseases. For example, Web [1] pp 91, states that “Honeycombing results in cysts …which have a peripheral predominance”. This simple statement is not simple to implement. We first had to develop a model of the human lung. Next we had to develop algorithms that use anatomical knowledge to automatically generate lung regions, such as, peripheral, central, apical and basal, which are frequently used in disease reporting [9], [10]. These enabled us to determine the

We experimented with two machine learning algorithms to build the classifier: Naive Bayes and the decision tree learner, J48, both implemented in Weka data mining environment [8]. The Naive Bayes algorithm is based on a probability model. The probability of a class given a feature vector is determined using Bayes’ rule:

1326

T. Zrimec et al. / Improving Computer Aided Disease Detection Using Knowledge of Disease Appearance

accuracy shows that the technique is well suited for honeycombing detection. A paired t-test, shows the improvement in accuracy to be statistically significant (p < 0.0001). Table 1 – Results of the two classifies: Decision Tree Induction J48 - (DTI-J48) and Naïve Bayes used with classical and knowledge-guided approach. Classifier

Figure 3 - An HRCT image with lung regions: blue-central, red - peripheral.

Accuracy

lung periphery on each axial scan, which helped in desease classification.

DTI-J48

DTI-J48

Naïve Bayes

Naïve Bayes

Classical

Knowledgeguided

Classical

Knowledgeguided

88.20%

89.70%

85.50%

87.20%

Sensitivity

96.70%

96.60%

97.50%

97.40%

Specificity

86.80%

88.60%

83.50%

85.50%

Testing on new, previously unseen data - Images from 8 patients, 4 patients with homecoming present and 4 patients with different disease patterns were used to test the performance of the detection system. The evaluation was performed on six images with honeycombing present and six images with out honeycombing. In total, there were 3150 regions with honeycombing and 60318 regions without honeycombing. Sensitivity of the algorithm dropped to 85%.

Knowledge guided classification In many systems (e.g. [2, 3]), all regions within the lung are classified starting from the top of the image. However, for diseases that show honeycombing, the pattern spreads from the periphery of the lung. We developed a knowledge-guided strategy for classification. This strategy uses seeded region-growing [11] and works as follows: •

The algorithm initially only classifies peripheral regions. Peripheral regions are determined using the lung regions masks. ROIs in the periphery of the lung with honeycombing are set as the “seed points” for the algorithm. • The algorithm only classifies a region of interest if it is near other ROIs already classified as honeycombing. • The algorithm will stop when there are no more ROIs to consider. In summary, the knowledge guides the system to classify all ROIs that are either in the periphery of the lung or in close proximity to other ROIs classified as honeycombing.

Conclusion The accurate quantification of disease patterns in medical images allows radiologists to track the progress of a disease. We have developed a system that uses machine learning to automatically detect honeycombing patterns in HRCT images of the lungs. Applying a classical, texturebased, approach resulted in over detection of honeycombing. It was not possible with simple post-processing to remove the false positive regions. We improved the perfor-

Results In order to test the clinical viability of the system, we evaluated the performance of our system on part of the training data and on new, previously unseen data. Testing on the training data - We used ten-fold cross validation. The number of ROIs used for testing varies for each fold, as the size of the lung in each slice affects the number of ROIs that we extract. On average 9,337 ROIs were used for testing (876 ROIs for honeycombing and 8,461 ROIs for non-honeycombing). The number of ROIs containing non-honeycombing was significantly larger as most of the lung region does not show honeycombing. The average of the results are shown in Table 1. From the results presented in Table 1, it can be seen that the knowledge-guided approach improved the accuracy of the honeycombing detection. The improvement in accuracy is attributed to a decrease in false positive classifications (sometimes by over 2%). The increase in

Figure 4 - Left: classification without using knowledge; right: knowledge-guided classification; red regions are classified as honeycombing; the blue line is the mask defining the periphery.

1327

T. Zrimec et al. / Improving Computer Aided Disease Detection Using Knowledge of Disease Appearance

mance of the system by using knowledge-guided classification.

[3] Uchiyama Y, Katsuragawa S, Abe H, Shiraishi J, Li F, Li Q, Zhang CT, Suzuki K, Doi K. Quantitative computerized analysis of diffuse lung disease in high-resolution computed tomography. Medical Physics 2003:30(9): 2440–2454. [4] Rudrapatna M, Sowmya A, Zrimec T, Wilson P, Kossoff G, Wong J, Busayarat S, Misra A, Lucas P, LMIK – Learning Medical Image Knowledge: An Internet-based medical image knowledge acquisition framework, Electronic Imaging Science and Technology - IS&T/SPIE's 16th Annual Symposium, San Jose, CA. 2004. [5] Papasoulis J, LMIK - Anatomy and Lung Measurements using Active Contour Snakes, Undergraduate thesis, supervisor: T. Zrimec, Computer Science and Engineering, University of New South Wales, Sydney, Australia, 2003. [6] Haralick RM, “Statistical and Structural Approaches to Texture,” Proceedings of the IEEE 67, pp. 786– 804, 1979. [7] Hall, MA. Correlation-based feature selection for discrete and numeric class machine learning. In: ICML ’00: Proceedings of the Seventeenth International Conference on Machine Learning, San Francisco, CA, USA, Morgan Kaufmann Publishers Inc. 2000; pp. 359–366 [8] Witten, IH, Frank E. Data Mining: Practical machine learning tools and techniques. 2nd edition, Morgan Kaufmann, San Francisco, 2005. [9] Zrimec T, Busayarat S, Wilson P. A 3D model of the human lung with lung regions characterization. ICIP 2004 Proc. IEEE Int. Conf. on Image Processing, 2004, pp. 1149–1152. [10] Zrimec T, Busayarat S, 3D modelling and visualisation of the human lung, Proc. 2nd Int. Symp. on 3D Data Processing, Visualization and Transmission, Thessaloniki, Greece, September 2004. [11] Adams R, Bischof L. Seeded region growing. Pattern Analysis and Machine Intelligence, IEEE Transactions on 1994: 16(6): 641–647.

We tested the system with part of the training data and with new, previously unseen data. The results showed a high degree of accuracy (89.7%) and sensitivity (96.6%) on the training data. The accuracy sensitivity, however drop from 97 to 71% when testing on new data. Experiments in building classifiers with different machine learning algorithms, Naïve Bayes and J48 decision tree learner, showed that the knowledge-guided classification performs better in both cases. The results showed that using knowledge-guided classification using the texture based Naïve Bayes classification lead to significant improvement according to pared t-test. The domain knowledge not only improved the results of the classification, but it also improved the representation of the results. For example, a computer aided system without knowledge of lung structure reports that 5% of the fifth image contains honeycombing and that 20 % of the images 9, 10 and 11, also contain homecoming. Our system reports that 5% of honeycombing was detected in the apical area and 20% in the basal area, predominantly in the lower lobe of the left lung. Being able to use knowledge of lung anatomy in image analyses will significantly improve the detection and quantification of other lung diseases. Acknowledgments We thank Dr. Peter Wilson for providing medical knowledge and guidance and Medical Imaging Australia for providing data. We thank Claude Sammut for his comments. This research was supported by the Australian Research Council.

Address for correspondence

References

Tatjana Zrimec Centre for Health Informatics University of New South Wales Sydney, 2052 [email protected] Tel: +61 2 9385 9034 Fax: +61 2 9385 9006

[1] Webb WR, Müller NL, Naidich DP. High-Resolution CT of the Lung. 3rd ed. Lippincott Williams & Wilkins, Philadelphia, 2001. [2] Uppaluri R, Hoffman EA, Sonka M, Hartley PG. Computer Recognition of Regional Lung Disease Patterns. American Journal of Respiratory and Critical Care Medicine 1999:160(2): 648–654.

1328

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

MR Atlas for Articular Cartilage Morphology: Potential to Detect Shape Differences Hussain Z. Tameema,b, Usha S. Sinhab a

Department of Biomedical Engineering, University of California, Los Angeles, United States b Medical Imaging Infomatics Group, University of California, Los Angeles, United States

replacements surgeries among Medicare recipients. Well over 200,000 knee replacement procedures for OA are performed every year in the United States [2].

Abstract An atlas of the cartilage was created using free form transformation of MR images of the cartilage from 20 subjects. The deformation required to move each voxel to its corresponding location in the atlas is used to determine the differences in shape between cartilages of subjects in a population. Based on these active shape models, it is possible to localize regions of high morphological variance in population cohorts. The atlas, reported here, is based on 20 male subjects; ten symptomatic of arthritis and ten asymptomatic. The active shape models based on this atlas show regions of high morphological variance corresponding to cartilage thinning in the arthritic group. This method has the potential to differentiate between normal and arthritic population groups by detecting subtle morphological changes in articular cartilage.

As of today, there are no reports of any disease modifying therapies for knee OA and all treatments are predominantly designed to relieve pain [10]. Approaches to prevent knee OA development, progression, or related disability are also very limited, in large part due to incomplete knowledge of potentially modifiable factors responsible for these outcomes. In this paper, we report a method based on free-form transformation to generate an average shape atlas of the femoral cartilage and apply it to study shape differences in a population cohort. This method has potential applications in the detection of subtle shape differences in normal and diseased population groups.

Keywords:

Background

active shape models, articular cartilage, image registration, magnetic resonance imaging, morphometry

Previous research on cartilage morphological assessment includes its volume and thickness measurements and the impact of various factors on normal knee, such as sex, body weight and height, maturity and age, body mass index, leg length and foot size, knee bone size, bone mineral density, muscle mass, level of physical exercise and genetics [11,12,13,14]. But none of the research to date focuses on studying the shape changes in cartilage between normal and diseased state population. Also there are studies which indicate the use of mathematical frameworks such as principal component analysis (PCA) to describe general shape variations. Marcus [15] used PCA to study the variation in the skull measurements of rodent and bird species. The resulting principal modes were interpreted as size and gross shape components. Cootes et al. [16] applied the theory of PCA to build statistical shape models of organs based on manually chosen landmarks. This model provided the average positions of the points and the principal modes of variations were computed from the dataset. The ability of the method to locate structures in medical images was demonstrated in a set of experiments with echocardiograms, brain ventricle tracking and prostate segmentation. Le Briquer and Gee [17] applied PCA to analyze the displacement fields obtained from registering a reference image volume of the brain to a set of subjects, based on the elastic matching framework. The

Introduction Osteoarthritis (OA) is a complex, progressive disease of the joint characterized by degenerative and regenerative morphological and structural changes in the articular cartilage and subchondral bone [1]. OA is a slowly progressing disease characterized clinically by pain, enlargement and deformity of the joints, and limitation of the motion. OA is the most prevalent form of arthritis and leading cause of disability and work limitation among adults resulting in enormous cost to society [2, 3]. Approximately 21 million American adults have physician diagnosed OA, [4] a diagnosis usually based on the combination of joint symptoms and radiographic changes. The prevalence of OA in a population is difficult to determine because: 1) the degree of radiological changes in symptomatic individuals varies greatly and 2) many individuals with radiographic evidence of OA have no symptoms. By age 60 nearly half of the population has radiographic evidence of OA in one or more joints, and by age 80 these findings are universal [5, 6]. Among the various sites being affected in OA, knee is the major source of reported disability and loss of function. About 40% of the adult population age 55 and older has frequent knee pain or definite x-ray evidence of knee OA [7-9]. Advanced OA accounts for majority of knee

1329

H. Z. Tameem et al. / MR Atlas for Articular Cartilage Morphology: Potential to Detect Shape Differences

analysis provided the inference of morphological variability within a population and was the basis for the construction of a statistical model for brain shape, which could be used as prior information to guide the registration process. Duchesne et al. proposed shape models for segmentation of the medial temporal brain structures [18].

Materials and methods Image acquisition We obtained the images from a pilot study conducted for the National Institute of Health OA initiative (OAI) version 0.A.1. MR images were acquired using a waterexcitation double echo steady-state (DESS) imaging protocol with sagittal slices at 3.0T (Magnetom Trio£, Siemens). The imaging parameters for the sequence were: TR/TE: 16.3/4.7 ms, matrix: 384f384, FOV: 140 mm, slice thickness: 0.7 mm, x/y resolution: 0.365/0.365 mm. Figure 1 shows a sagittal slice of the magnetic resonance image obtained using DESS sequence.

Figure 2 - Schematics of steps involved in atlas creation Step I: We randomly chose one subject from the pool of 20 to serve as a reference to which the rest of the images were aligned utilizing the mutual information based affine transformation. Affine transform corrects for subject positioning and global size differences. Step II: An elastic registration based on demons algorithm [19] was employed to locally map all the images in the group of subjects to the reference image using the affine transformation parameters as an initial estimation. This provides 3D deformation fields that can map the spatial locations on an individual in the group to the coordinate system of the reference. The registration algorithm computes the transformation iteratively using equation 1.

Qn+1 is the correction vector field at iteration n+1, GV is the Gaussian filter with variance V2, … denotes the convolution, C is the scaling factor and T and S are the target and transformed images respectively. The algorithm estimates the displacement which maps a voxel at location (x,y,z) in T to the corresponding anatomical location in S. The algorithm is implemented hierarchically and to preserve the morphology, deformation vector fields were computed utilizing both the forward and backward transformation.

Figure 1 - MR image acquired with a DESS sequence with water-excitation at 3.0 T. The images are obtained from National Institutes of Health (NIH) OA initiative The OAI data consists of a stratified random sample of 200 participants based on gender, sub-cohort assignment (progression and incidence) and clinic (four recruitment centers). The progression sub-cohort contains participants with symptomatic knee OA at baseline where symptoms are pain, aching or stiffness in or around the knee on most days for at least one month during the past 12 months. The incidence sub-cohort contains participants with no symptomatic knee OA at baseline, but has characteristics that place them at the risk for developing symptomatic knee OA during the study. For this study we randomly chose 20 male participants, 10 each from progression and incidence cohort group.

Step III: A mean intensity image with the shape of the reference image is created by averaging the globally and locally transformed images of the group. Step IV: A mean deformation field that encodes the shape variation between the reference image and average shape of the elements in the subject group is created by averaging over 3D deformation vector fields of the individual subjects of the group. Step V: Inverse average deformation field is applied to the average intensity image to generate and average intensity and deformation image template for the group under study.

Atlas creation We manually segmented the cartilage for all 20 subjects and then interpolated the raw data to a pixel resolution of 0.365x0.365x0.365 mm3. We created the atlas in the following steps, which are illustrated in figure 2.

Step VI: Steps 1-5 are iterated until no significant change in the deformation field is observed relative to the previous computation. At the end of each iteration the original reference image is replaced by the average template

1330

H. Z. Tameem et al. / MR Atlas for Articular Cartilage Morphology: Potential to Detect Shape Differences

Figure 3 - Accuracy of registration. Left to Right: reference, test, result of affine transform and free form deformation image. Top and bottom row showing slices at different locations. Outline from reference image shown superposed on test & aligned images. constructed at Step V generating both average shape (morphometric) and intensity atlases that represent the centroid of the population data set.

d

Active shape models are used to represent the variance in cartilage shape within a given population. Active shape models based on principal component analysis of the deformation fields were created using the data from the last iteration of the atlas creation procedure [17]. With reference to the atlas creation, each iterative process results in an average atlas and deformation field dir. The deformation field, dir, is the amount required to move the voxel from its original position (after global affine transformation) to the corresponding location in the atlas. Here r refers to the voxel, i represent the subjects (1-20 used to create the atlas) and d represents the deformation vector. The following analysis based on principal component analysis for data reduction was performed on all n voxel of the cartilage and consists of the following steps. Step I: Calculation of the mean deformation for N subjects at each voxel as shown in equation 2 where dmean is the mean deformation at any voxel over all subjects.

Figure 4 - Left: Shows the sharp edges on the atlas. Right: 3D volume reconstruction of the cartilage from the atlas

(2)

Results

Step II: Computation of the deviation from the mean value as shown in equation 3 'di

di  dmean

The alignment of test image to the reference image using affine and freeform transformation is shown in figure 3. A region of interest is selected on the reference image and overlaid on the test, affine and the image obtained after free form transformation to show the accuracy of registration. It can be clearly seen that the affine transform corrects for positional and global scale changes and the local deformation corrects for physiological changes. The accuracy of the alignment can be visually evaluated by the sharp edges of the cartilage in the 2D images of the atlas as

(3)

Step III: Calculation of nfn covariance matrix, C, to find the basis for the space as shown in equation 4 C = 1/N) (6'didiT)

(5)

The shape variations by r2 SD from the mean shape along the first two principal modes were generated using eigenvalues and eigenvectors derived from this analysis. These images were synthesized by setting the weights of the first or the second modes at r Oi and r2 Oi (where i = 1, 2 for the first and second eigen mode respectively) and all other weights to zero. The synthesized images provide a visual representation of the possible variance in shape of the cartilage based on 20 image sets to create the atlas.

Active shape models

dmean = 1/N) (6di)

dmean  QWs

(4)

Step IV: Diagonalization of the covariance matrix to obtain the eigenvectors, Qk and the eigenvalues, Ok. Step V: Construction of the linear model as shown in equation 5, where Q = (Q1, Q2, …., Qk) is the matrix of the first eigenvectors, and Ws is a vector of weights, also called the shape coefficient. This results in a shape model.

1331

H. Z. Tameem et al. / MR Atlas for Articular Cartilage Morphology: Potential to Detect Shape Differences

Figure 5 - 3D active shape models. Average shape shown in the middle top row with standard deviation (SD) variations along the first and second mode shown in rows 2 and 3: Left to right - 2SD, -1 SD, +1 SD and +2 SD. Arrows indicate regions with changes from average bone shape shown in figure 4. At the end of each iteration the reference image is replaced by the average image which moves closer to the centroid of the images in the group. There is no significant difference in the atlas after the third iteration confirming its convergence. Figure 4 shows 3 dimensional rendering of the cartilage generated from the atlas volume.

PCA has been used to study gender and disease-related morphological differences in the corpus callosum, putamen, ventricles and hippocampus [20, 21]. It should be noted that within the scope of this paper we demonstrate the feasibility of generating the active shape models and that the application for classification will require far more image volumes to be included in the training set from the population cohorts of subjects under investigation.

Figure 5 shows the active shape models and the variations in shape for r2SD along the two leading eigenmodes. The outline of the cartilage is overlaid on the variations along the first and second modes that show the variations seen in the current set of 20 subjects. The first mode shows the larger changes since it captures the largest variation in the data as compared to the second mode.

Conclusion We have developed an atlas for the cartilage and active shape models which when combined can be used to detect the subtle shape changes in cartilages. We see significant changes using this technique within the group of 20 subjects we selected. These models have a potential to be used in the future to discriminate normal and diseased states with larger databases.

Discussion We successfully developed an atlas for the articular cartilage derived from the MR images at 3T. This is the first report of creating a cartilage atlas from images acquired at high resolution and isotropic resolution (0.365x0.365x0.365). The results show very accurate alignment which could be used for clinical purposes. We are currently working on creating a 3D Active contour without edges segmentation algorithm proposed by Chang T and Vesse L to extract the cartilage from the MR images.

Acknowledgments We would like to thank The OAI study sponsored by NIH for providing the data used in this study

References [1] Aubray JH, Jr, Sokoloff L 1989 Pathology of osteoarthritis. In: McCarty DJ (ed.) Arthritis and Allied Conditions, 11th Ed. Lea & Febiger, Philadelphia, PA, USA., pp. 1571-1580. [2] Dunlop DD, Manheim LM, Yelin EH, Song J, Chang RW. The cost of arthritis. Arthritis Rheum 2003; 49(1):101-13. [3] Guccione AA, Felson DT, Anderson JJ, Anthony JM, Zhang Y, Wilson PW, Kelly-Hayes M, Wolf PA, Kreger BE, Kannel WB. The effects of specific medical conditions on the functional limitations of elders in the Framingham study. Am J Public Health 1994;84:351-8. [4] Lawrence RC, Helmick CJ, Arnett FC, Deyo RA, Felson DT, Giannini EH. Estimates of the prevalence of arthritis

We hypothesize that it is possible to automatically determine the cartilage location by analyzing its overall shape variation. If this hypothesis holds, then we can reverse the process and use the information obtained from the shape analysis to automatically segment cartilage from rest of the structures. In future we intend to use the active shape models and show that unsupervised learning can be used to explore the anatomy and facilitate segmentation. This methodology could potentially be used to classify different population groups. Structural shape characterization using

1332

H. Z. Tameem et al. / MR Atlas for Articular Cartilage Morphology: Potential to Detect Shape Differences

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

and selected musculoskeletal disorders in the United States. Arthritis Rheum 1998; 41:778-99. Bagge E, Bjelle E, Valkenburg HA, Svanborg A. Prevalence of radiographic osteoarthritis in two elderly european populations. Rheumatol Int. 1992;12:33-8. Van Sasse JLCM, Van Romunde LKJ, Cats A, Vandenbroucke JP, Valkenburg HA. Epidimealogy of ostearthritis: Zoetermeer survey. Comparison of radiological osteoarthritis in a Dutch population with that in 10 other population. Ann Rheum Dis 1989;48:271-80. Urwin M, Symmons D, Allison T, Brammah T, Busby H, Roxby M, Simmons A, Williams G. Estimating the burden of musculoskeletal disorders in the community: the comparative prevalence of symptoms at different anatomical sites, and the relation to social deprivation. Annals of Rheumatic Diseases 1998; 57:649-55. Peat G, McCarney R, Croft P. Knee pain and Osteoarthritis in older adults: a review of community burden and current use of primary health care. Annals of Rheumatic Diseases 2001; 60:91-7. Felson DT, Zhang Y. An update on Epidemiology of knee and hip osteoarthritis with a view to prevention. Arthritis Rheum 1998;41(8):1343-55 Felson DT, Lawrence RC, Hochberg MC, McAlindon T, Dieppe PA, Minor MA, Blair SN, Berman BM, Fries JF, Weinberger M, Lorig KR, Jacobs JJ, Goldberg V. Osteoarthritis: New Insights. Part 2: Treatment Approaches. Ann Internal Med 2000; 133(9): 726-37 Cicuttini F, Forbes A, Morris K, Darling S, Bailey M, Stcukey S. Gender differences in knee joint cartilage volume as measured by magnetic resonance imaging. Osteoarthritis Cartilage 1999; 7:265-71. Ding C, Cicuttini F Scott F, Glisson M, Jones G. Sex differences in knee cartilage volume in adults: role of body and bone size, age and physical activity. Rheumatology (Oxford) 2003. Cicuttini FM, Teichtahl AJ, Wluka AE, Davis S, Strauss BJ, Ebeling PR. The relationship between body composition

[14]

[15]

[16]

[17] [18]

[19]

[20]

[21]

and knee cartilage volume in middle-aged subjects. Arthritis Rheum 2005;52:461-7. Jones G, Ding C, Scot F, Cicuttini F. Genetic Mechanism of knee osteoarthritis: a population based case-control study. Ann Rheum Dis 2004; 63:1255-9. Marcus L. Traditional Morphometrics In: Rohlf FJ, Bookstein FL, editors. Proceedings of the Michigan Morphometrics Workshop, The University of Michigan Museum of Zoology, Ann Arbor, 1990. p. 77-122. Cootes T, Hill A, Taylor CJ, Haslam J. Use of active shape models for locating structures in medical images. Image Vision Comput 1994; 12548:126-32. Le Briquer L, Gee JC. Design of a statistical model of brain shape. Lect Notes Comput Sci 1997; 1230:477-82. Duchesne S, Pruessner JC, Collins DL. Appearance-based segmentation of medial temporal lobe structures. NeuroImage; 2002,17:515-532 Thirion JP. Image Matching as a diffusion process: An analogy with Maxwell’s demons. Medical Image Analysis, 1998; Vol 2 (3), 243-260 Kim SH, Lee JM, Kim HP, Jang DP, Shin YW, Ha TH, Kim JJ, Kim IY, Kwon JS, Kim SI. Asymmetry analysis of deformable hippocampal model using the principal component in schizophrenia. Hum Brain Mapp. 2005 Aug; 25(4):361-9 Shenton ME, Gerig G, McCarley RW, Szekely G, Kikinis R. Amygdalahippocampal shape difference in schizophrenia: the application of 3D shape models to volumetric MR data. Pyschiatry res. 2002 Aug 20; 115(1-2):15-35

Address for correspondence Hussain Tameem UCLA Medical Imaging Informatics 924 Westwood Blvd, Suite 420 Los Angeles, CA 90024 USA Email: [email protected]

1333

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval Jayashree Kalpathy-Cramera, William Hersha a

Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon, USA

and painstaking task, and is prone to errors. Consequently, there is a desire to be able to automate the task of indexing these collections with a goal to improve the ability to search and retrieve relevant documents.

Abstract Medical image retrieval can play an important role for diagnostic and teaching purposes in medicine. Image modality is an important visual characteristic that can be used to improve retrieval performance. Many test and online collections do not contain information about the image modality. We have created an automatic image classifier for both grey-scale and colour medical images. We evaluated the performance of the two modality classifiers, one for grey-scale images and the other for colour images on the CISMeF and the ImageCLEFmed 2006 databases. Both classifiers were created using a neural network architecture for learning. Low level colour and texture based feature vectors were extracted to train the network. Both classifiers achieved an accuracy of > 95% on the test collections that they were tested on. We also evaluated the performance of these classifiers on a selection of queries from the ImageCLEFmed 2006. The precision of the results was improved by using the modality classifier to resort the results of a textual query.

Medical image retrieval systems have traditionally been text- based, relying on the annotation or captions associated with the images as the input to the retrieval system. The last few decades have offered advancements in the area of content-based image retrieval (CBIR) [1]. CBIR systems have had some success in fairly constrained medical domains, including pathology, head MRIs, lung CTs, and mammograms [2]. However, purely content-based image retrieval systems currently have limitations in more general medical image retrieval situations, especially when the query includes information about pathology [3, 4]. Mixed systems (using both textual and visual techniques) have demonstrated improved retrieval performance, especially with regards to precision at the top of the list [4]. Medical image databases used for image retrieval or for teaching purposes often contain images of many different modalities, taken under varied conditions with variable accuracy of annotation. This can be true for images found in various on-line resources, including those that access the on-line content of journals1.

Keywords: medical imaging, neural networks, image annotation, content-based image retrieval

Introduction

Image modality is an important, fundamental visual characteristic of an image that can be used to aid in the retrieval process. However, the annotations or captions associated with images often do not capture information about the modality. Images that may have had modality associated with them as part of the DICOM header can lose that information when the image is compressed to become a part of a teaching or on-line collection. There have also been reported errors in the accuracy of DICOM headings [5].

Medical images form a vital component of a patient’s health record. Effective medical image retrieval systems can play an important role in aiding in diagnosis and treatment; they can also be effective in the education domain for healthcare students, instructors and patients alike. As a result of advances in digital imaging technologies, there has been a large growth in the number of digital images stored in recent years. In addition to the Picture Archival and Communication Systems (PACS) that are becoming omnipresent in hospital and clinics, there are numerous online collections of medical images. On-line atlases of images can be found for many medical domains including dermatology, radiology and gastroenterology. The sheer volume of medical image data provides for numerous challenges and opportunities in the arena of medical image retrieval.

The medical image retrieval task within ImageCLEF has provided both a forum as well as test collections to benchmark image retrieval techniques. The ImageCLEF campaign has been a part of the Cross Language Evaluation Forum since 2003 [3]. CLEF itself is an offshoot from the Text REtrieval Conference (TREC). In 2004, ImageCLEFmed, a domain–specific task, was added to evaluate medical image retrieval algorithms and techniques.

Historically, the task of indexing and cataloging these collections has been performed manually. This is an arduous

1

1334

http://gm.arrs.org/ (accessed 3/26/2007)

J. Kalpathy-Cramer et al. / Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval

Approaches combining both visual and textual techniques for retrieval have shown some promise at medical image retrieval tasks [3]. In 2005, a medical image annotation task was added to ImageCLEF. The goal of this task was to correctly classify 1000 test images into 116 classes given a set of 10,000 training images. The classes differed primarily in anatomy and view of the image. It should be noted, however, that these images were primarily of a single modality (X-rays).The goal of the ImageCLEF medical image retrieval task of 2006 was to retrieve relevant images for thirty topics from a test collection of about 50,000 annotated images of different modalities. These tasks were divided by the organizers into those expected to be amenable to textual, visual, or mixed retrieval techniques.

electrocardiograms (ECGs), as well as a few miscellaneous images.

We participated in both the medical image retrieval and the automatic medical image annotation tasks at ImageCLEF 2006 [6, 7]. The techniques developed for those tasks have been extended for the more general task of medical image modality classification and annotation. Using medical image modality for image annotation and retrieval has recently been studied. Florea et al [8] have compared the efficacy of two different systems (MedIC and MedGIFT) in classifying the modality of a database with six standard modalities for radiology and nuclear medicine images. In this paper, we compare the results obtained on our system with those described in previous publications [8] for the six modalities of the CISMeF database. We will also extend this technique to classify colour images from the ImageCLEF medical retrieval task collection [6] into six categories. We will finally report on the improvement in precision that we observed for a selected number of tasks of the ImageCLEF medical retrieval task for 2006 by incorporating the modality classifier in series with a textbased retrieval system.

Figure 1 - Hierarchical classification scheme for images A neural network-based scheme using a variety of low level, primarily global image features was used to create a six-class classification system for the grayscale images. The multilayer perceptron architecture used a hidden layer of approximately 50-150 nodes. The classification system was created in MATLAB2, in part using several routines modified from the Netlab toolbox3. We experimented with a variety of feature vectors as inputs to the network. A combination of texture and intensity histogram features provided the best classification [10, 11]. All images were first resized while maintaining the aspect ratio such that the smaller dimension was 256 pixels. The image was divided into five overlapping blocks. Grey level correlation matrices were computed for each block using four angles and an offset of 1 pixel. Contrast, correlation, energy, homogeneity and entropy were calculated for each matrix. A quantized grey scale histogram was then appended resulting in a 132-dimension feature vector for each image for the texture. All inputs to the neural network (the image feature vectors) were normalized using the training set to have a mean of zero and variance of 1.

Methods We employed a supervised machine learning approach to problem of medical image modality classification using a hierarchical classification scheme as seen in figure 1. There were two primary databases that were used to create and test the classifiers. We worked with a small subset of the CISMeF database as the primary target for our greyscale (radiographic and nuclear medicine) image classifier [9]. This database had a set of 1332 images classified into one of six classes based on modality. These include angiography, computerized tomography scans (CT), Xray, Magnetic resonance (MRI), ultrasound, and scintigraphy. The images in this database had been acquired under differing conditions over a long period of time. Consequently, there was considerable intra-class variation in quality, size, contrast, illumination and background.

The 1332 images in the database were randomly split into a training set of 1000 images and a test set of 332 images. A small random subset of the training images was initially used to create the classifier (200 images). The classifier

The imageCLEFmed database contains 50,000 images of differing modalities, including radiography and nuclear medicine, as well as microscopic and histopathological images, photographs and gross pathology images, power point slides, electroencephalograhical images (EEGs) and

2 3

1335

www.mathworks.com (accessed 3/26/2007) http://www.ncrg.aston.ac.uk/netlab/index.php (accessed 3/26/2007)

J. Kalpathy-Cramer et al. / Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval

was then applied to the entire training set and images that were misclassified were then added to the images used to refine the classifier. The classifier was finally tested on the test images. A similar scheme was used to create the classifier for colour images. We believe this novel idea can improve the retrieval performance of purely textual systems or for images for which the associated modalities are not known. Although modality detectors for grey-scale medical images have been reported [9], we are unaware of a similar effort for classification of other categories of medical images like those produced by histo-pathology and endoscopy. The images used for this classification task were taken from the test collection of images used in the ImageCLEFmed retrieval task. 2250 colour images in this collection were broadly categorized into six categories as microscopic, gross pathology, EEG/ECG or other charts, powerpoint slides, endoscopic images and other. There was considerable intra-class variability in this dataset. These 2250 images were again split randomly into training (1750) and test images (500). A similar training methodology to that described above was used to incrementally improve the classifier, starting with a smaller subset of the training database.

Figure 2 - Sample textual and visual queries at ImageCLEFmed 2006 Once our classifiers had been trained to achieve >95% classification accuracy, they were tested on a random subset of the ImageCLEFmed topics. The schematic of our modified retrieval system is shown below. The query was initially fed to our Lucene4 based text retrieval system. The queries were manually edited by one of the authors. The resulting images were subsequently classified by the hierarchical classifier for modality. Images of the desired modality (as stated in the query or as discerned by the automatic classifier based on the sample images) were moved to the top of the list while maintaining the ranking of the textual system within a class.

A two-layer architecture with 25-150 hidden nodes was used for the neural network. The feature vector in this case consisted of colour histogram features, as well as texture features obtained using the grey level correlation matrices. The image was split into 9 uneven blocks. Colour histogram properties of image after conversion into the L*A*B* colour space were calculated, while texture features were calculated after converting the image to greyscale These neural network classifiers can be created to further classify images within a given modality. For instance, xray images could now be classified to account for anatomy. Anatomical classifiers were used in the automatic annotation task at ImageCLEFmed. The tasks had been stated in English, German and French, and had and provided example images. All but three of the tasks stated the desired modality of the image to be retrieved. Two examples of the tasks are shown in figure 2.

Figure 3 - Image retrieval system used for the ImageCLEFmed 2006 test collection We compared the results of our purely textual system with that including the addition of the modality classifier.

Show me images of a hand x-ray. Zeige mir Röntgenbilder einer Hand. Montre-moi des radiographies de la main.

Results A classification accuracy of 96.4% was achieved on the CISMeF database. The confusion matrix suggests that the primary misclassification occur between the MRI and CT scan classes. This is not surprising as these classes are visually quite similar. Florea et al [8] have reported similar results both in terms of accuracy and inter-class misclassification patterns. The classification of grey-scale medical images into commonly occurring modalities using low level image features and machine learning techniques appears to be a tractable task. We expect to achieve over 98% accuracy with further refinement of our machine

Show me blood smears that include polymorphonuclear neutrophils. Zeige mir Blutabstriche mit polymophonuklearer Neutrophils. Montre-moi des échantillons de sang incluant des neutrophiles polymorphonucléaires.

4

1336

http://lucene.apache.org/ (accessed 3/26/2007)

J. Kalpathy-Cramer et al. / Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval

learning approach by the use of more advanced cross-validation, bootstrapping, boosting or bagging techniques.

Task 1 Precision 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

Preliminary testing of the classifiers on 2250 colour images of the imageCLEFmed test collection resulted in a modality classification accuracy of 98.6%. Most of the misclassifications involved the “other” class with contained a set of miscellaneous images not belonging to the other five specific categories The colour modality classifier was tested on a small random subset of the ImageCLEFmed 2006 topics. The topics for imageCLEFmed 2006 fell into three categories (visual, mixed, semantic) consisting of 10 tasks each. Although visual techniques had, in general, performed extremely poorly at the semantic tasks, use of some visual information (primarily modality) was shown to increase the precision [4].

P5

Grey-scale

1831

Photograph/gross pathology

90

Microscope

71

Other

P30

P100

OSHU textual

P200

P500 P1000

Best system overall

Figure 5 plots the precision for varying number of documents retrieved for the purely textual system, the improvement with the use of the modality classifier and the overall best system (mixed visual and textual based) that participated in ImageCLEFmed 2006. This increased the precision of the query as seen in figure 5. The improvement in precision at the top of the ranked list (P5 – P200) is better with the use of the modality detector compared to a purely textual search. We should note that a perfect modality classifier will only improve the precision of the search and not the recall if it is applied in the serial manner described above. The mean average precision (MAP) would still be limited by the number of relevant images that are retrieved by the textual search (recall of the textual search). Even in searches that are expected to be semantic, we see an improvement in precision by using the modality classifier as seen in figure 6 and 7.

Number of images 2000

P20

Figure 5 - Improvement in precision resulting from modality classification.

Task 1 - Show me images of the oral cavity including teeth and gum tissue

Total returned by textual query

P15

OHSU textual + modality classifier

Analysis of our textual results indicated that in many queries, especially those of a visual or mixed nature, up to 75% of the top 1000 results were not of the correct modality. A compelling example is given in figure 4 and table 1. Only 90 of the top 2000 images returned by the textual query were of the desired modality.

Image type

P10

Task 2 - Show me microscopic images of tissue from the cerebellum (semantic query)

8

Figure 4- Sample query suitable for visual retrieval at ImageCLEFmed 2006

Image type

These images were then classified using our modality classifier. The ranked list of retrieved images was resorted taking into account the desired modality based on the query.

Number of images

Total returned by textual query

2000

Greyscale

1476

Photograph/gross pathology

408

Microscope

116

Figure 6 - Sample query suitable for visual retrieval at ImageCLEFmed 2006

1337

J. Kalpathy-Cramer et al. / Automatic Image Modality Based Classification and Annotation to Improve Medical Image Retrieval

The precision of this search was similarly improved by the use of the modality detector as seen in figure 7.

Rouen Hospital, France, for proving the CISMeF images used in this paper. We would like to thank Dr. T. Lehmann at RWTH Aachen University, Germany, for access to the IRMA image database. We acknowledge the support of NLM Training grant 5T15 LM07088-15 and NSF Grant ITR-0325160.

Task 25 Precision

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

OHSU textual + modality classifier

[1] Smeulders AWM, Worring M, Santini S, Gupta A, Jain R. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000;22(12), 1349-80. [2] Müller H, Michoux N, Bandon D, Geissbuhler A. A review of content-based image retrieval systems in medicine – clinical benefits and future directions. International Journal of Medical Informatics 2004; 73, 1-23. [3] Müller H, Deselaers T, Lehmann T, Clough P, Hersh W. Overview of the ImageCLEFmed 2006 medical retrieval annotation tasks, Evaluation of Multilingual and Multimodal Information Retrieval, Seventh Workshop of the Cross-Language Evaluation Forum, CLEF 2006, editors: Peters C, Clough P, Gey F, Karlgren J, Magnini B, Oard DW, de Rijke M, Stempfhuber M, LNCS 2006,Alicante, Spain, to appear. [4] Hersh W, Kalpathy-Cramer J, Jensen, J. Medical Image Retrieval and Automated Annotation: OHSU at ImageCLEF 2006. Working Notes for the CLEF 2006 Workshop, Alicante, Spain. http://www.clef-campaign.org/ 2006/ [5] Güld MO, Kohnen M, Keysers D, Schubert H, Wein BB, Bredno J, Lehmann TM, Quality of DICOM header information for image categorization, Proceedings SPIE, 4685, 280-287, 2002. [6] Hersh W, Müller H, Jensen J, Yang J, Gorman P, Ruch P. Advancing biomedical image retrieval: development and analysis of a test collection. J Amer Med Inform Assoc 2006; 13(5). [7] Lehmann TM, Güld MO, Thies C, Fischer B, Keysers D, Kohnen M, Schubert H, Wein BB. Content-based image retrieval in medical applications for picture archiving and communication systems, in Medical Imaging, SPIE Proceedings, 5033, 440-451, San Diego, California, 2003. [8] Florea F, Müller H, Rogozan A, Geissbühler A, Darmoni S, Medical image categorization with MedIC andMedGIFT. Medical Informatics Europe (MIE 2006). [9] Douyere M, Soualmia LF, Neveol A, Rogozan A, Dahamna B, Leroy JP, Thirion B, Darmoni SJ. Enhancing the MeSH thesaurus to retrieve French online health resources in a quality-controlled gateway. Health Info Libr J 2004; 21(4):253-61. [10] Haralick R. Statistical and structural approaches to texture. Proceedings of the IEEE 67, 786-804, 1979. [11] Howarth P, Ruger S. Evaluation of texture features for content-based imageretrieval. In: Proceedings of the International Conference on Image and Video Retrieval, Springer-Verlag, pp. 326-324, 2004.

P1 00 0

P2 00

OSHU textual

P5 00

P3 0

P1 00

P2 0

P1 5

P5

P1 0

References

Best system overall

Figure 7 - Improvement in precision resulting from modality classification Four of the six tasks tested showed improvement in precision by the use of the modality detector for colour images. There were two tasks amenable to textual methods for which there very little change in precision with the addition of the modality information. We plan on testing the performance of the modality detector on the complete set of tasks for ImageCLEFmed 2005 and 2006. We also intend to index the entire collection of 50,000 images used in the ImageCLEFmed test collection using the modality classifier. Information about the class membership of an image will be added to the metadata. This should improve the performance of the retrieval in two ways. Clustering of the data by modality and perhaps anatomy will speed up the search process as fewer documents will have to be compared to the query image/text. Secondly, we expect that the overall precision of the search will improve by considering the modality of the image that is desired by the user. However, we can expect a small degradation in the recall due to potentially misclassified images not being searched.

Conclusion We have developed a neural network based, hierarchical classifier for the modality classification of medical images. This system can classify colour images including histo-pathological and endoscopic images, and photographs as well as grey-scale (radiological and nuclear medicine). The classifier uses a histogram and texture properties as inputs to the two level neural network. This classifier results in a classification accuracy of greater than 95% for the grey-scale images of the CISMeF database as well as a selection of colour and grey-scale images from the ImageCLEFmed database. The use of this classifier increases the precision of retrieval of our primarily text based retrieval system by moving images of the desired modality to the top of the ranked list.

Address for correspondence Jayashree Kalpathy-Cramer 5th Floor, Biomedical Information Communication Center 3181 S.W. Sam Jackson Park Rd. Portland, Oregon 97239-3098

Acknowledgments

Email: [email protected]

The authors would like to thank Henning Müller for his valuable input in discussions as well as providing access to the CISMeF images. We would also like to thank Jean-Nicholas Dacher at the

1338

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Quantification of Myocardial Perfusion for CAD Diagnosis Hartmut Dickhausa, Markus Erbachera, Helmut Küchererb a

Department of Med. Informatics, University of Heidelberg, Heidelberg, Germany b Department of Cardiology, University Hospital, Heidelberg, Germany

For example: grade 1 is defined as: “dye slowly enters but fails to exit the microvasculature. There is no blush or opacification on the myocardium in the distribution of the culprit lesion that fails to clear from the microvasculature and dye staining is present on the next injection 30 sec later.”

Abstract We introduce a computer based algorithm for objective quantification of myocardial perfusion to support the diagnosis of cad patients. This new method is based on conventional cine angiographic films. In order to achieve maximal quality of the digital subtraction angiography images, the sequence is synchronized with the ECG. Optionally, the digital images can be motion compensated by a two step matching method. The spatio-temporal spread of blood, or the so-called blush, through the microvasculature to the myocardium - indicated by dye injection – represents a characteristic pattern for the myocardial perfusion. This dynamic temporal pattern is characterized by typical features as the maximal value of blush intensity, of increase and of decrease velocity which correspond with the different phases of flooding in and washout. On the basis of 100 different temporal blush profiles, an algorithm is established which classifies the acquired blush patterns into 4 different grades.

Despite the principal advantages of blush characterization, qualitative descriptions like these are difficult to apply in an objective and reproducible stratification. For that reason, we tried to establish a computer assisted procedure to quantify blush grades corresponding with those of Gibson, however, in an objective and formal description. Furthermore, we developed a computer aided tool to visualize the spatial and temporal spread of the adelomorphic blush, of the myocardium from cineangiographic films after dye injection into the arteries. Under the control of the cardiologist, a specific blush grade is assigned for a specific myocardial region related to the three main supplying coronary arteries.

Keywords:

Materials and methods

angiography, blush grade, CAD diagnosis, myocardial perfusion

In this study, 100 films from patients with various extent of CAD recorded in different projections, are quantitatively examined before and after PTCA. Because the heart is supplied by three different arteries: right coronary artery (RCA), left artery descending (LAD) and left circumflex (LCX), we look for blush occurrence in correspondent myocardial areas supplied from these vessels.

Introduction Currently, the coronary angiography is still the gold standard for coronary artery disease (CAD) diagnosis, although other methods have been proposed and discussed [1]. In almost all cases, the required invasive procedure of inserting a catheter for dye injection is combined with a percutaneous transluminal coronary angioplasty (PTCA). Therefore, the angiographic procedure allows an immediate estimation of the success of the therapeutic intervention.

In order to enhance the contrast between blush and the arteries and surrounding tissue, images after dye injection have to be subtracted from that prior to injection, respectively in standard position. The initial image without dye is called a mask and is mostly established by averaging two or three consecutive frames in order to smooth small artifacts and noise. The difference of images shows the highest contrast between filled vessels, microvasculature and the surrounding, if we take the logarithmically transformed angiograms. In order to consider the motion of the heart and its vessels as well as the inserted catheter during the heart cycle, one has to choose images for subtraction which correspond to identical heart geometry and position. This can be achieved by synchronization of the image acquisition to the R peak of the simultaneously recorded ECG. Fortunately, the time interval between two cycles,

For quantifying complete and sustained reperfusion of the infarcted myocardium and prognostic statements as for identifying patients at high risk, myocardial blood flow, expressed in so-called blush grades, is much more appropriate than any other angiography related measure. This is demonstrated by Stone et al. [2]. However, so far only qualitative descriptions for different extends of blush or myocardial reperfusion exist. Gibson et al. [3] presented 4 different grades which classify the perfusion in relation to its temporal dynamic and intensity.

1339

H. Dickhaus et al. / Quantification of Myocardial Perfusion for CAD Diagnosis

about 1 sec, is relatively small, so that the resulting time course of dye spreading is still sufficiently documented.

Movements of the operation table or of the camera adjusted by the cardiologist for optimal viewing conditions are primarily compensated by a rigid matching procedure. Secondly motion of the inner organs particular caused by breathing needs an elastic compensation. In most cases the table or camera movements are larger but can be corrected more easily than the breathing effects. This two step procedure minimizes the extent and time for the elastic registration [4]. One general problem characterizes both registration steps, even the continuous spread of dye during the digital acquisition procedure. This means that besides the mentioned motion artefacts we have to consider differences between succeeding frames resulting from the dye distribution. In order to minimize this problem we apply the matching procedures always between two consecutive image frames. That means that the registration is performed every time relative to the previous frame. Finally all single transformations are summed up. Figure 2 demonstrates this process schematically. To calculate each transformation step, the rigid one as well as the elastic one, primarily the vessels have to be segmented for each frame. The pixels identified as vessels are then skipped during the calculation, in order to minimize the problems caused by the spreading dye.

Figure 1 - Acquisition of ECG synchronized frames from the cine angiographic film. The black marked frames related to the R peak of every cardiac cycle are used for creating the sequence of difference images, displaying the spreading blush across the heart.

The spatial distribution of the blush is represented by the gray value averaged from myocardium pixels excluding the arteries. That means we have to identify all myocardium pixels belonging to the blush. This task is very hard even for an experienced cardiologist. We perform this segmentation by an interactive procedure. For all pixels, we trace the intensity profile, respectively the gray value, over time. The incoming wave of dye after injection follows primarily the main coronary arteries, flows through the smaller vessels into the microvasculature, and reaches the myocardium. The washout is collected in the venous vessels. This time dependent process can be represented in its spatial distribution by detecting the maximal values of the intensity profiles of all image pixels. All pixels identified by their maximal gray values at a certain time instant represent more or less a specific structure of the pathway to the venous system. As it is demonstrated in Figure 3, it is easy to choose a color map representing the arteries map c) or d) or the perfused myocardium, pixel map f). For all segmented pixels of map f) we calculate the average gray value, cycle for cycle and plot the corresponding time course of blush intensity.

In figure 1 the acquisition procedure is schematically demonstrated. With increasing time we get a sequence of difference images displaying the temporal progression of dye and its spatial spreading through the arteries and small vessels into the microvasculature and the myocardium. The quality of these images demonstrating the blush is highly dependent to any change of body position or motion of inner organs, like the diaphragm, during breathing. For this reason, the patient is asked to stop breathing for about 20 sec, and the camera as well as the operating table have to be fixed for that period. However, in clinical routine it is difficult to accomplish these conditions completely. Therefore, we apply additional compensation procedures in two consecutive steps.

The cardiologist has to position a specific ROI at a region of the myocardium besides the arteries where a blush would be expected under normal conditions. We also calculate for this specific region the average gray values for different instants of time as characteristic time courses for a restricted perfusion. The different time dependent profiles have to be characterized by a few parameters which can be used to distinguish between different grades of blush. Furthermore, we would like to define parameters which can be easily interpreted in context with the existing qualitative characterizations of Gibson, and finally they should be related to the pathophysiological background.

Figure 2 - Schema of the registration procedure between frame n and the mask: the registration is split into n single consecutive transformations and finally the sum of all transformations is performed.

1340

H. Dickhaus et al. / Quantification of Myocardial Perfusion for CAD Diagnosis

Figure 4 - The time course of averaged gray values from corresponding areas of the perfused myocardium. 4 specific parameters: Imax: Increase-slope, Dmax: Decrease-slope, Gmax: maximal gray-value, Tmax: time of Gmax are indicated

Results For all 100 films the corresponding time courses are established and automatically characterized by Gmax, Tmax, Dmax and Imax. It is of particular interest to demonstrate the difference of blush spreading before and after a PTCA intervention by different temporal patterns. Different degrees of stenosis are related with different extensions of blush and different patterns of dynamic profiles. The patient of Figure 5 had an occlusion of RCA. After PTCA the artery looks absolutely normal (Figure 5, lower trace). By positioning of a ROI in the supply area of this vessel we calculate the corresponding blush profile with its characterizing parameters. However, for the same region before PTCA we found a very flat profile (Figure 5 upper trace), which shows us that there is not any perfusion of the myocardium. Nevertheless, the reperfusion after PTCA is obvious. In this case we have consistent results with the re-opened artery and the blush profile. However, sometimes the situation demonstrated on the angiograms is not so clear. For these cases we get much more information by the quantified blush profiles.

Figure 3 - Color maps for characterizing of specific structures during blood perfusion. The temporal incidence of the maximal dye intensity is mapped on to the pixel matrix. For different instances of time, corresponding with map b) to f), we display different anatomic structures which are perfused at this time; a) shows all maps superimposed: background, artery (LAD), branching, myocardium, vein; b) - f) the separated structures. Map f) is of particular interest because it represents the pixels of the myocardium. We defined 4 different parameters for each intensity profile which are displayed in Figure 4: the maximal intensity Gmax; the time of Gmax; the maximal slope Imax of the rising profile, which corresponds with the maximal rising velocity of the perfusion; and the maximal slope Dmax of the falling profile, which corresponds with the maximal outwash velocity. For all calculated blush profiles, these four parameters are extracted as a typical characterizing feature set.

In order to relate the various blush profiles to blush grades a non-parametric classification procedure based on the four specific parameters is under development. First promising results are in good agreement with the subjective grading of clinical experts.

1341

H. Dickhaus et al. / Quantification of Myocardial Perfusion for CAD Diagnosis

remaining 10 films show such considerable motion artefacts of the diaphragm that an evaluation was not possible. In all of these cases the diaphragm moved over the whole hearth and modified its grayvalues significantly. Figure 6 shows the effect of motion compensation.

before motion compensation

before PTCA

after motion compensation

60

30

average gray value

average gray value

35

25 20 15 10 5

50 40 30 20 10 0

0

0

0

2

4 6 time in seconds

8

10

2

4 6 time in se conds

8

10

Figure 6 - The left image shows the extent of usually recorded motion artefacts whereas the right image demonstrates the benefit of motion compensation. The time courses before and after motion compensation are plotted in the same diagram below

Discussion Our presented approach on the basis of digital subtracted angiographic images is an interactive procedure which helps the cardiologist to assign blush grades in strong relation to what he is used to do. He gets an intuitive impression of the spreading blood after the dye injection by displaying the processed colormaps. It is easy to confirm a quantitative hypothesis by looking precisely to selected areas of interest of the myocardium, and its temporal gray value profile or to integrate the average dye activity of the whole area.

after PTCA

average gray value

35 30 25 20 15

The advantages of the developed system are given in a highly adaptable degree of automation. Dependent on the quality of angiograms, the more or less experienced user can interfere in different kinds. For example, simple adjustment of the contrast or corrections of motion artefacts lead to an improved quality of the digitally subtracted image sequence. The automated classification procedure is also easily adaptable to other types of blush patterns. Up to now it was our aim to implement the descriptions for blush grading of experienced cardiologists as it was proposed by Gibson. However, if we wish to characterize any other specifity of the perfusion, we can as well introduce more quantitative features related to the blush patterns, for example temporal measures. Furthermore, other approaches of motion compensation could be incorporated and evaluated for their practicability in respect of typical

10 5 0 0

2

4 6 time in seconds

8

10

Figure 5 - Angiograms before and after PTCA of a patient with a complete stenosis of 100% for RCA. The corresponding temporal blush profiles document obviously the improvement of myocardial reperfusion From the 100 digitized angiographic film sequences we could evaluate only 60 without considerable motion compensation. The introduced two step registration method allowed an additional evaluation of 30 more series. The

1342

H. Dickhaus et al. / Quantification of Myocardial Perfusion for CAD Diagnosis

angiographic artefacts. Of course, we have to look for more sequences to incorporate them in our learning sample. Therefore, we developed an intuitive user interface which helps the cardiologist to follow the different steps of the acquisition procedure and which offers him the various facilities to derive a reliable diagnostic statement. During the current evaluation of our system we are discussing the usefulness of a more automated selection of the ROI for blush profile quantification. However, up to now the experienced cardiologists are not convinced that this feature increases the clinical benefit significantly.

Survival in high risk Patients undergoing primary or rescue PTCA. J Am Coll Cardiol 2000: 35: 403A [3] Gibson CM, et al. Relationship of TIMI Myocardial Perfusion Grade to Mortality after Administration of Thrombolytic Drugs. Circulation, 2000: 101(2): pp. 125-130 [4] Modersitzki J. Numerical Methods for Image Registration. Oxford University Press, 2004

Address for correspondence [email protected] Prof. Dr. H. Dickhaus Institute for Medical Biometry and Informatics Department of Medical Informatics University of Heidelberg Im Neuenheimer Feld 400 D-69120 Heidelberg Germany Tel.: 0049 6221 567483 Fax: 0049 6221 564997

References [1] Bibra H. Assessment of myocardial perfusion - role of contrast echocardiography. Z. Kardiol 2001: 90: pp. 848-859 [2] Stone GW, Lansky AJ, Mehran R, et al. Beyond TIMI 3 Flow: The Importance of Restored Myocardial Perfusion for

1343

This page intentionally left blank

Chapter 11. Education and Training

1345

This page intentionally left blank

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Enabling the Safe and Effective Implementation of Health Informatics Systems – Validating and Rolling Out the ECDL/ICDL Health Supplement Michael J. Rigbya, Carol Hulmb, Don Detmerc, Luca Buccolierod a

Centre for Health Planning and Management, Keele University, United Kingdom b British Computer Society, Swindon, United Kingdom c American Medical Informatics Association, Bethesda, MD, United States of America d SDA Bocconi and CeRGAS Centre for Research on Health Care Management, Bocconi University, Milan, Italy

Health informatics systems are a vehicle for collecting, processing, and making available relevant health information at the point of need. Therefore such support systems should not dominate or hamper health practitioners’ activities except so far as they can make them more effective or efficient. However, all too often end users – health practitioners and their support staff – can find such systems threatening and restricting, and in turn this means that systems are used inefficiently and ineffectively, and either perform badly or fall into disuse.

Abstract Sound understanding by end-users (health professionals and support staff) of key e-health principles and challenges is essential for the safe, effective, and sustainable use of health informatics systems. This is separate from, and ideally a precursor to, training on a specific system. However, hitherto this aspect has been little understood or addressed. Over the last few years, the concept of a customised Health Supplement to the well-established European/International Computer Driving Licence has progressed from idea to reality, through initial support for consultation by the then UK NHS Information Authority, followed by establishment of an international Expert Group by the global ECDL/ICDL Foundation. As a result, the ECDL/ICDL Foundation has developed a formal syllabus. This paper reports successful piloting, and progress in the development of local training and testing materials and national implementation plans, in three countries on two continents.

Resistant mature staff Though often a surprise to many, this picture of end-user poor compliance is well documented over a period of time, and thus should be anticipated [1]. Health professionals have in round terms a professional lifespan of approximately forty years. In nearly every country those practicing for more than ten years will have undertaken their basic professional education and early practice in an environment based on paper-based records. In many countries of the world electronic systems will only have been brought in even more recently. Furthermore, the most senior and experienced health practitioners – usually the professional champions and leaders – will certainly have been educated ahead of the e-health revolution. It is into this workforce environment that governments, health policy makers, and health informatics system advocates are seeking to introduce radical and comprehensive electronic systems.

Keywords: health informatics education; end-user; ECDL; ICDL; health informatics sustainability

Introduction As defined by the theme for Medinfo 2007, there is rapidly growing recognition and dependence in all countries on the future major role of health informatics systems in enabling national health systems to be increasingly effective, efficient, and evidence based. Whilst each country starts from a different base line, all are moving forward at their local pace in increasing the use of informatics systems.

An atypical IT system user population A further complication is that not only are health information system end users more well versed in paper-based systems, in many other ways too they differ from the end users of almost all other organisation-wide modern electronics records systems, such as those in banking or the travel industry [2,3]. Not only are health users senior and mature staff rather than new entrants; they are totally dependent on information systems in order to carry out their principal daily business. However, this core work is not itself focussed on the information system - in general the use of the information system only forms between 10% and 20% of their duties - and to them this is a low-skill element of their work. Thus senior doctors, nurses, and other

Rightly, the priority is increasingly for sustainability, which must include the reliable and efficient use of such systems, in line with intention and with capability. However, the fundamental oversight is to focus solely on systems design and implementation, important though they are. What is so often overlooked is the fact that systems are only as good as their users’ understanding and capacity to use them appropriately.

Selected for best paper award.

1347

M. J. Rigby et al. / Safe and Effective Implementation of Health Informatics – Validating and Rolling Out the ECDL/ICDL Health Supplement

health professionals will not give the same level of importance to being trained in new-generation information systems as they will to new clinical or healthcare techniques. Yet, even though being only part-time users, they are senior and often autonomous employees, and they have professional accountability over and above their duty to their immediate employer. Thus they will only be effective, reliable and regular users if they both understand the way to harness a system, and trust in its integrity and that of all partner users (who are largely unknown to them individually). The deeper analysis undertaken of these differences of health informatics system end user profiles compared to others system user profiles has been published [2, 3].

which needs a new mind set in order to negotiate it effectively to find key and relevant information items. It would be unreasonable to expect a surgeon to use a new type of instrument or a radically new surgical procedure without adequate training, yet governments and policy makers worldwide are inappropriately labelling as “obstructive” health practitioners who are reluctant to change information management approaches with which they are familiar, for ones which they find unknown and intimidating. Further, ‘smart’ systems may make good evidence-based calculations, recommend particular treatment patterns or warn against particular prescribing intentions. These are safe provided the end user understands the rationale in both the clinical and computing logic inbuilt, but carry risks if the end user does not understand and know how to ascertain that logic.

Change-inducing systems Added to these challenges, health informatics systems in themselves do not support clinical practice in the simplistic ways that telephones support and replace meetings and written correspondence, or dictation systems replace the need for the physical presence of a secretary. Instead, health information systems require a very different pattern of working, ranging from new data recording processes, through to how to search a past record (which is navigated very differently if it is held in computer files compared with if it is a large collection of paper charts in a physical folder).

The risks of new constraints A key aspect of most health informatics systems is the fact that they require a standard approach to the description of histories, investigations, results, diagnoses, and interventions - in other words, the benefits of standard terminologies and taxonomies should reduce ambiguity and render observations and findings interpretable accurately by all. However, the converse of that is that an individual’s well-developed means of indicating valuable subjective information such as uncertainty, provisional views, or feelings as apposed to hard evidence are rendered impossible. This may either exclude uncertain information, or result in it being recorded with a spurious impression of certainty. Users may know the clinical approach and specific skills of colleagues who handrecord narrative information, but this authorship and personalisation may be lost with electronically captured and stored data.

Informatics system sustainability but practice challenge Therefore sustainability is highly dependent as much upon the pattern of use as it is on the pattern of design, yet is affected also by the user population’s characteristics. Thus it is clearly naive to expect the complete cohort of the most senior and experienced practitioners in their country universally to welcome and endorse new systems which require radically different physical and cognitive skills, and which require immediate changes to patterns of data recording and assimilation honed over a life time. Thus, however good the system introduced, it will not be sustainable if the principal stakeholders are not comfortable with issues.

Risks of misuse The very strength of health informatic systems – that they can search and present information from very large databases extremely quickly – is a potential risk as well as being a core purpose. Files can become increasingly comprehensive, and information can be obtained about many people. Careless use of such information can lead to inappropriate divulgence of confidential information, and without safeguards there are clearly risks of an ethical or malicious misuse. Thus all end users need to be educated to avoid these risks, and to adhere to robust corporate policies to control usage.

Counteracting the principal risks of health information systems Apart from recognising the challenge to traditional practice of new health informatics systems, it is also important to recognise the inherent risks these bring if introduced without due preparation. Moreover, professionals will be aware in general terms of these risks, and be likely to militate against introduction of such systems with good intent unless they feel that these risks have been addressed and controlled. There are three types of such risk, as shown below.

For all these reasons it is therefore important for end users to be adequately educated as to how to use systems soundly and effectively. This is a key part of the sustainability of systems. Evidence (or even suspicion) of misuse of systems, or of poor clinical decision making because of inability to use systems, will provide a rapid means of ensuring their demise. Such evidence or suspicions may come either through professional sources, press reports, or collective patient anxieties.

Risks generated by the need for new skills The use of a health informatics system requires a radical range of new skills. These commence with the basics of operating any computer system, through the skills required to record data electronically as apposed to by hand writing or filing a chart, to the skills needed to navigate a record which is stored in a highly structured and efficient way but

1348

M. J. Rigby et al. / Safe and Effective Implementation of Health Informatics – Validating and Rolling Out the ECDL/ICDL Health Supplement

many practical steps have been made towards achieving this reality.

The move to a health end-user qualification The need to recognise end users

In 2004 the NHS Information Authority, the then lead body in this field for the National Health Service in England, agreed to support two consultation workshops – one for key opinion leaders in health informatics from eight European countries, and one for a range of delegates from the National Health Service across the United Kingdom. As a result of the strong enthusiasm at both these meetings, the European Computer Driving Licence Foundation (the global regulatory and licensing body for the ECDL/ICDL) agreed to consider formalising the development process. The ECDL Board endorsed this, and in 2005 an Expert Group was set up comprising representatives of six European countries and of the United States of America. The resultant recommended syllabus was signed off by the Expert Group in early 2006.

The educational needs of end users are very different from those of technical health informatics staff. Over a decade ago a European Commission Concerted Action entitled EDUCTRA identified the informatics educational needs of health professionals as being different from those of IT staff [4]. This work suggested a range of necessary learning outcomes for each group – though for health professional users they focussed primarily on basic curricula for new entrants. Subsequently the International Medical Informatics Association (IMIA) addressed this topic, and produced recommendations of what should form the basis of health informatics education globally for each of the two staff communities. Both before and since these recommendations, the prime focus has been on informatics education, with development of many formal courses, and on the introduction of some informatics training into basic health professional education. Neither of these groups, however, form the general body of the practice community to whom new organisation-wide informatics systems are introduced or imposed.

The ECDL/ICDL health supplement content The final ECDL/ICDL Health Supplement consists of a competencies framework defining knowledge and skills the candidate needs to possess in order to operate a health information system safely. It excludes generic issues covered in health professional training or staff induction (such as basic principles of confidentiality). Regarding computer recording, it focuses on those aspects which are different, or have different emphasis or importance, in health applications.

The concept of an end user qualification To overcome these anxieties and risks it seems self-evident that an appropriate end-user educational programme, and related qualification, for health informatic systems users should be developed, but this was not being addressed. Meanwhile, virtually all countries in the world have a qualification requirement for drivers of motor vehicles or pilots of aeroplanes, as such equipment is seen as extremely beneficial yet extremely risky if misused through ignorance or lack of skills. It is seen as a societal responsibility to provide a qualifications framework and regulation, and a citizen responsibility to ensure qualification before becoming a user.

The core contents of the syllabus are copyright to the ECDL/ICDL Foundation, and comprise the following topics: •

•

This approach has already been taken with the more general use of computers, with the development of the European Computer Driving Licence (ECDL) to a standard international curriculum, as most of the issues of using computers safely and effectively are generic and universal. This has now developed into the global International Computer Driving License (ICDL), available in virtually all countries of the world [6].

•

It therefore seemed logical to develop a specific supplement or module for the ECDL/ICDL, given the risks and responsibilities inherent in using such systems. This concept was first promoted in 1999 in a European context [7]. Subsequently, the idea was developed at conceptual level in more detail, and support gradually developed [8]. Of significance for Medinfo 2007, details were sought by a principal Australian health informatics journal [9].

•

Concepts – Health Information Systems – HIS Types Due Care – Confidentiality – Access Control – Security User Skills – Navigation – Decision Support – Output Reports Policy and Procedure

For each topic a number of defined knowledge areas or competencies are specified. The content is designed to accommodate specific national language and terminologies, organisations, and legal and professional frameworks. The normal pattern of assessment will be electronic, through a testing framework available on line or by other electronic means. It is based on the assumption that the candidate will already be competent in basic computer user skills.

Practical steps to development Following these moves towards the development a health supplement to the ECDL/ICDL as the best means of meeting this need, and thereby ensuring sustainable and safe implementation of health informatics systems through education, assurance, and empowerment of end users,

International trials and validation Since the specification phase, rapid progress has been made in significantly different countries, with very differ-

1349

M. J. Rigby et al. / Safe and Effective Implementation of Health Informatics – Validating and Rolling Out the ECDL/ICDL Health Supplement

ent health systems, different languages, and also different terminology and nomenclature within the same language group.

Italy The Italian Association for Computing and Automated Calculation (AICA), the Italian national ECDL licensee [11], instructed CERGAS Bocconi to work out the Italian health syllabus, based on the core syllabus, and the test structure. The related “ECDL Health Manual” has also been developed and printed. Between January and March 2007, Italy has implemented two pilot editions of the course, delivered to about 60 medical doctors and nurses of the Local Healthcare Units of Milan and Dolo (Venice). Participants have been offered four courses of ECDL Start (24 teaching hours), plus a specific course of ECDL Health (8 teaching hours). At the end of the pilot courses, in May 2007, final examinations will be held and skill cards issued accordingly (including, for the first time ever in Italy, those relating to ECDL Health). The examination to obtain the ECDL Health skill card will include practical exercises simulating the use of patient records management software. The courses will be included in national and regional programmes aimed at the continuous education of the NHS medical personnel and will enable participating medical doctors to obtain some compulsory education credits. These courses are expected to become a key element of the education and training programmes nationwide.

United Kingdom In the United Kingdom, the British Computer Society as national licensee for the ECDL, and with a strong relationship with the National Health Service, organised piloting of the syllabus utilising an interim training manual and testing framework in six very different sites. These encompassed very different localities, and different healthcare environments ranging from primary care through secondary care to mental health, and different health professions from research staff to medical consultants, and also health informatics and health data experts. This pilot involved 84 persons, who were all very positive on the value of the knowledge and competencies covered in the syllabus. The only significant comments received were about the interim testing framework, which was only ever intended to be temporary in order to facilitate consideration of the syllabus. More detailed reporting of these results is in press [10]. Consequent upon these successful pilots, a full electronic training resource has been developed from the interim one, and a definitive electronic testing framework built. The ECDL Health Supplement was launched to English NHS staff by NHS Connecting for Health in April 2007, with the on-line tutorial and testing available to staff free of charge.

The Italian experience stands out for its special focus on the preliminary planning of the initiative, also being based on a scientific research project conducted in 2004 aimed at measuring the potential benefits of information education and training in the healthcare sector [12]. This research project has analysed and evaluated the “cost of IT ignorance” in the Italian healthcare sector through a sample survey, empirical measurement tests, and experience of a similar research project conducted on private businesses. Ignorance in the information field has proved to be a notable hidden cost for the Italian healthcare sector and the potential value of information education and training of the NHS personnel amounts to about 2 billion Euros per year. After being published and officially presented, the results enhanced the institutional awareness regarding development of targeted educational programmes.

United States of America The American Medical Informatics Association (AMIA) and the national ICDL licensee, ICDL-US, worked closely during 2006 to create a US-version curriculum for ‘anyone in a health-related entity who touches a keyboard containing person-specific health information’ as well as an examination to certify mastery of this content. This will be entitled the Digital Patient Record Certification or DPRC. The curriculum group adapted the syllabus developed by the ECDL Foundation group; the US version was then reviewed by the ECDL group. The test will be piloted in early 2007 with the expectation that the program will be functional late in 2007. ICDL-US and AMIA found the partnership to be mutually beneficial and there is a desire to work together on other products for the North American region.

Discussion Development of any new qualification takes a considerable period of time, commencing with identification of the need. This initiative has sought to achieve this in a way that matches the differing needs of countries globally, and to pilot and validate it in differing countries in two continents. Though the piloting, and the development of the educational and testing frameworks and systems, was grounded on the needs of the individual countries involved, the resultant tools and products are likely to yield wider benefits and use by other countries wishing to follow suit.

AMIA has two major educational initiatives underway and this initiative is part of its “Got EHR?” campaign. AMIA also seeks to educate the general public about electronic health records and especially integrated personal health records, particularly as an integral part of the electronic medical record. The campaign also strives to increase the use of EHRs in small practice environments. The second initiative is the “10x10” Program, which is an effort to educate 10,000 applied clinical informaticians by 2010. This program now involves two universities and will involve at least five by the end of 2007.

As well as being a key contribution towards sustainable health informatics systems, this initiative also marks a policy development for the ECDL/ICDL Foundation. It began the development of the Health module as a global

1350

M. J. Rigby et al. / Safe and Effective Implementation of Health Informatics – Validating and Rolling Out the ECDL/ICDL Health Supplement

concept with the help of a group of subject matter experts. However, it became clear that the application of Health informatics systems required a certification that was closely tailored to national requirements including patterns of practice, culture, language, and legislative frameworks. As a result the ECDL Foundation adopted a modem of product endorsement. The core syllabus for the Health supplement is specified by the Foundation, but individual national license holders propose the format in which the national certification is developed and assessed within a specific country, ensuring this has local relevance. Each national certification is then endorsed by the ECDL Foundation. The Health certification is thus tailored to each individual country's health system. It not only uses the languages of the country, but the terminologies and taxonomies for health care practice, as well as professional and legal codes. The ECDL/ICDL Health Supplement is the first successful implementation of this Endorsed Product concept.

References [1] Anderson JG, Aydin CE, Jay SJ (eds.). Evaluating Health Care Information Systems: Methods and Applications; Sage, Thousand Oaks CA, 1994. [2] Rigby M. Improving Governance and Reducing Risk in Electronic Patient Record Systems: Ensuring Appropriate Competencies for Support and End-User Staff. In M Fieschi, E Coiera, Y-C J Li (editors). Medinfo 2004 Proceedings of the 11th. World Congress on Medical Informatics, IOS Press, Amsterdam, ISBN 1 58603 444 8, 2004, 926-930. [3] Rigby M. Protecting the Patient by Promoting End-user Competence in Health Informatics Systems – Moves towards a Generic Health Computer User “Driving License”; International Journal of Medical Informatics, 73, 151-156, 2004. [4] Staff of EDUCTRA Concerted Action. Education and Training in Health Informatics: Guidelines for European Curricula; in Hasman A et al. Education and Training in Health Informatics in Europe; IOS Press, Amsterdam, 1995. [5] International Medical Informatics Association. Recommendations of the International Medical Informatics Association (IMIA) on Education on Health and Medical Informatics; Methods of Information in Medicine, 39, 267277, 2000. (also available on www.imia.org/wg1/rec.pdf) [6] www.ecdl.com/main/index.php [7] Rigby M, Draper R, Sheridan A. Confidentiality, Competence, and Confidence - Ensuring Ethics whilst Recognising Reality; in Proceedings, EuroRec ’99 Third European Conference on Electronic Health Care Records, 1999, Sevilla, España; Sadiel, Sevilla, 1999. [8] Rigby M. Protecting the Patient by Ensuring End-User Competence in Health Informatics Systems – Moves Towards a Generic Health Computer User “Driving Licence”; in Proceedings of IMIA Working Group on Education Conference, Portland, Oregon, 2003. [9] Rigby M. The Health IT roll out: supporting the end users; Health Information Management, 33, 3, 93-94; 2004. [10] Rigby M, Hulm C. Piloting in England of a Health Supplement for the European Computer Driving License (paper presented to HC 2007 and in press). [11] www.aicanet.it [12] Buccoliero L, Caccia C. Un approccio aziendale ai

Conclusion With the increasing importance of health informatics systems, and the need to ensure their effective and safe usage, there has been a steady and increasing recognition of the importance of end user competence as a contribution to effective sustainable implementation and development of such systems. Moreover, given that the issues and risks are basically generic, coupled with increasing mobility of health staff, the advantage of devising a generic solution has become self evident. From this position, the progress in the last two years through an expert committee identifying and confirming the core syllabus, and three different countries undertaking trials and detailed implementation plans, is significant and encouraging. Like motorcars, aeroplanes, or other items of advanced technology, health informatics systems are only as good as the competence (and confidence) of their users. Hitherto this has gone largely unnoticed, except for possible training in a particular system’s operational instructions. The ECDL/ICDL Health Supplement has broken new ground, by recognising the high importance of the education and empowerment of the end user, whatever their level or profession. It thus makes a vital contribution to the sustainability of health informatics systems.

sistemi informativi in sanità. [A corporate approach to information systems in the healthcare sector.] In Borgonovi E, Camussone PF, Occhini G, (Eds.) L'Ignoranza informatica: il costo nella sanità. [IT ignorance: how much it weighs on healthcare] Milan, McGraw-Hill., 2004: 31-69.

Moreover, this is a global solution, linking common generic requirements with local need through the ECDL Endorsed Product concept. Having been developed by an international expert group, endorsed by the ECDL/ICDL Foundation, and now validated simultaneously in three very different countries, this product is now available for use in any nation.

Address of correspondence Professor Michael Rigby, Professor of Health Information Strategy, Centre for Health Planning and Management, Darwin Building, Keele University, Newcastle-under-Lyme, Staffordshire, ST5 5BG, UK. Telephone +44 1782 583191, Fax +44 1782 711737, email: [email protected]

1351

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

A Multi-method Approach to Assessing Health Information Systems End Users’ Training Needs Yiyu Qiua, Ping Yub Dr. Peter Hylandc a,b

c

Health Informatics Research Centre, University of Wollongong, Australia School of Information and Technology and Computer Science, University of Wollongong, Australia

which in turn will lead to loss of organizational resources like skilled healthcare workers. It is important, therefore, that significant investment be made into end user training and support in order to ensure that the introduced HIS will be accepted and used by the healthcare workers.

Abstract Many existing training programs for Health Information Systems (HIS) do not adequately satisfy end users’ needs nor meet training objectives. This is because they do not envisage the problems that users may encounter when performing specific tasks. Therefore the first priority for the development of an effective training program is to precisely assess the end users’ training needs, a process called Training Needs Assessment (TNA). Applying traditional approaches for TNA, such as interviews or surveys alone, however, may be insufficient because they are limited in their capacity to reveal the cognitive processes of end users. Usability testing, with its ability to gather data about human computer interaction, overcomes the deficiencies of these traditional approaches. This paper proposes a multi-method approach, which combines usability testing together with traditional methods, such as interviews or questionnaire surveys to assess HIS end users’ training needs. This innovative method is able to precisely reveal the training needs for different levels of HIS users. A case study, which applied this method to assess the training needs for users of a nursing information system demonstrates its feasibility.

An effective end user training program should have the capacity to deliver a timely, effective, efficient and enjoyable learning experience to the end users [4, 7]. In other words, it plays the role of closing the gap between the complexity of an HIS and the users’ cognitive capacity to master it. The majority of existing training programs, however, are not as effective as they promise [8, 9]. The most common problems of these training programs are that they are ill-directed and inadequately focused [10]. For example, some training programs provide healthcare professionals with huge amounts of unnecessary information because they have been developed as a “one size fits all” solution [8, 9]. The fundamental flaw is that training needs assessment, namely, the process of assessing the training objectives [11], is not properly conducted for these programs so the training program designer can not accurately envisage the problems end users may encounter when performing specific tasks using the HIS. Thorough TNA is required to improve end users’ learning outcomes and to enable them to become familiar with a new HIS efficiently.

Keywords: health information systems, end users, training needs assessment, usability testing

In order to conduct a thorough and accurate TNA, the method of analysis should be scientifically designed, and this is the topic to be discussed in this paper. Firstly this paper will critique the traditional methods for TNA. Afterwards it will propose a novel method for TNA, followed by a case study demonstrating how to integrate the new approach with traditional methods.

Introduction A well designed Health Information System (HIS) can not only save staff time in entering and retrieving client data, but can also increase the accuracy and completeness of such data. The adoption of HISs, however, is not common in the current Australian healthcare sector [1, 2]. One of the reasons is that shifting from the traditional paper-based documentation to electronic documentation requires the users of an HIS to have basic computer skills, to be familiar with the HIS, and to change their practices of information management [3]. Managers are more and more aware that end user training is an essential strategy to accomplish this [4, 5] and that the failure to implement appropriate end user training strategy will leave staff feeling frustrated and threatened by the new system [6]. In the worst case, they may even reject the new system or resign,

Need for new approach in training needs assessment The primary purpose of a TNA is to identify what knowledge and skills end users should have in order to enable them to effectively interact with an HIS. Through identifying the usage problems, the gap between the necessary and the actual knowledge and skills that a user has for effectively interacting with this IS can be inferred [12]. In other words, experimental Human Computer Interaction (HCI) data such as usage problems, mistakes or inefficient behaviors, are effective indicators of what they do not

1352

Y. Qiu et al. / A Multi-method Approach to Assessing Health Information Systems End Users’ Training Needs

know [13], which, in turn, suggests what they need to know. Traditional methods of TNA, however, lack the ability to collect such detailed information about the cognitive process end users follow in their interaction with a new IS.

retrieval and educational research [22, 23]. From the perspective of informatics, cognitive science can provide a framework for the analysis and modeling of complex human performance in IS [23]. Theories and methods from the cognitive sciences can illuminate different facets of the design and implementation of IS [22, 23]. They can also play an important role in understanding and enhancing human performance on a wide range of tasks. These tasks may include developing training programs to reduce errors and increase efficiency for healthcare [23].

The common methods of assessing computer users’ training needs are self-reported questionnaire surveys and interviews with end-users [13-15]. Questionnaire surveys have a number of distinct advantages, including the ease of distributing questionnaires to a large number of users and the automated analysis of the results with statistical packages. The typical process of quantitative assessment consists of managers setting the required level of skills for a particular task, then a staff member is requested to rate his/her skill level against this standard [16]. A comparison between these two sets of data suggests this staff member’s skill gap in accomplishing the task, however, this method can only identify difficulties of which the designers or skilled external consultants are already aware [15]. It can not detect all the challenges or mistakes that a user may face or make in using a particular HIS.

Usability testing encompasses a range of methods for identifying how users actually interact with a complete software product. Empirical testing is a form of usability inspection that observes actual user interactions with an interface (Neilson, 1994). Given that many HISs fail due to usability problems, organizations are starting to show interest in usability testing. Some preliminary studies have been performed testing clinical information systems. For example, Kushniruk [24] introduces a laboratory-based usability testing method to evaluate the effectiveness of HISs. According to Kushniruk [24], usability testing refers to “the evaluation of an IS that involves testing of participants who are representative of the target user population, as they perform representative tasks using an IS”. During the testing, all interactions a user has with an IS are recorded (e.g. video recordings made of all computer screens or user activities and actions). In addition, this technique generally includes the collection of “think aloud” reports, involving the recording of users as they verbalize their thoughts while performing particular tasks [24].

Similar problems underlie other commonly used methods like interviews or focus group discussion, where end users are asked to reflect on their prior experience with an IS. Such qualitative assessments provide opportunities for users to express their perceived difficulties in using an IS but these conventional methods are not adequate to assess the learning needs of users, particularly for users with different levels of experience with the system. For example, novice users may not have sufficient knowledge about this new IS to enable them to realize the problems they may encounter in using it [17, 18]. Even experienced users may not be able to clearly recall their problems [19]. Verbalizing the process that a person follows to complete a task is also problematic, as it involves the expression of sequences of psychomotor movement in interaction with an HIS [20]. In addition, users’ perceptions of the same problem may be different because of differences in their educational or technical levels [21].

In brief, this approach focuses on classifying users’ cognitive ability and then identifying the problems they encounter during their interaction with the IS. With its ability to gather rich empirical HCI data, this method provides an excellent opportunity to complement the weaknesses of interviews or questionnaires for assessing training needs. Although this method was originally designed for testing the usability of an IS, it is also useful for identifying training needs. In the early ’90s, Simpson proposed a framework to describe how testing methods could be used in the planning phase of designing online documents [25]. A recent case study has used novice users’ interaction with a search engine to reveal the knowledge and skills that such users need [18]. Our preliminary study which employed the usability testing method to assess the training needs of nursing students also demonstrates the feasibility of this method [26].

Thus the use of traditional TNA methods alone may not reveal detailed information about the cognitive process end users follow in their interaction with a new IS [21, 22] so data gathered from these methods are not adequate to identify end users’ training needs. In other words, basing the selection of training strategy on the analysis of such incomplete data may lead to ineffective programs because there is a significant gap between what is perceived to be a problem and the actual problems that a user encounters. Traditional methods for TNA need to be complemented by more effective new approaches. Careful observation of how users encounter and react to problems in interacting with a particular HIS is necessary. The next section will explain our proposed methods and suggest strategies for dealing with the problems discussed above.

A multi-method approach for training needs assessment We have demonstrated the viability of cognitive usability testing methods in capturing process data on how an end user interacts with a particular IS. We propose that a multimethod approach combining usability testing with conventional methods like interview or questionnaires can precisely and thoroughly understand the process end users follow in processing information in a particular HIS. It can also suggest the knowledge and skills that these users need to learn in order to use the HIS.

Cognitive usability testing method There is a growing role for the cognitive and behavioral sciences in health informatics, particularly as it pertains to human factors, and other areas such as information

1353

Y. Qiu et al. / A Multi-method Approach to Assessing Health Information Systems End Users’ Training Needs

The process of our proposed approach is to:

•

•

observe how novice, intermediate and veteran users use an HIS to complete representative tasks. • interview or survey these users before or after conducting cognitive usability testing, in order to ascertain their level of the knowledge and skills that are relevant to the HIS that they are trained to use • extract patterns of strategies used by the novice, intermediate and veteran users to complete various tasks with this HIS. • identify the key knowledge gaps (learning needs) of different levels of learners based on the different interaction patterns that they display when using the HIS. In the following sections we describe three approaches which can be utilized for an innovative TNA.

Ease of use: used when participants comment on the level of “ease of use” of the system (from easy to hard) or any confusion or frustration experienced.

A case study The NIS described in this case was the Care Planning Assessment Tool (CPAT), owned by the Hammond Care Group. The CPAT was introduced to help nurses carry out systematic assessments for nursing home residents. The “Clients” menu enables users to perform the most crucial functions of the CPAT, i.e., doing data entry and assessments for clients (i.e. residents), see Fig 1 for a screen shot of the assessment screen. In the assessment screen, users can enter detailed assessment results for a resident. The program can then generate various assessment related reports.

The following three issues need to be considered when performing usability testing.

This particular study aimed to assess the training needs for different levels of the CPAT users, so as to develop the “right” training materials for the “right” groups of users. The training materials for the CPAT are:

Participants

•

Usability testing approach

Participants for the experiment are potential or actual end users. Based on their experiences with the HIS (e.g. measured by the time and frequency of their usage of the system), they can be categorized as novice, intermediate, and experienced users. The recommended sample size varies between usability experts. As Nielsen and Mack suggest [27], usability testing can be carried out using only five participants, and the results will demonstrate 85% of the usability problems. Kushniruk et al. [21] suggest that up to 80% of usage problems can be detected from 8 to12 participants evaluating a system.

•

User Manual: this is usually for novice or first time users and should be very detailed. Online Help: this type of training is traditionally for the relatively experienced users who require help while using the product; it usually contains information on how to conduct a task.

Outcome measurement Kushniruk et al. [21] suggest that the usability testing should involve setting up recording equipment that allows for continuous recording of computer screens during the process of human-computer interaction. To achieve this goal, Camtasia Studio, screen recording software, can be used to record each participant’s mouse movement and keyboard strokes. In addition, participants’ “think aloud” reports can be audio-taped. The data analyzed for usability analysis included both the video and the audio file. Fig 1 - The assessment screen of the CPAT

Data analysis technique

Procedure

Prior to analyzing the video data, a coding scheme should be defined for use in identifying specific occurrences of users’ problems and aspects of cognitive processes from transcripts of a participant’s “think aloud” comments. A coding taxonomy developed by Kushniruk can be used for analyzing human computer interaction data [22, 24]. •

•

Based on users’ knowledge and experience with the CPAT, they are classified into two categories: novice and experienced users. The process of the training needs assessment in this case study was to: •

Navigation: used when participants comment that they are navigating, or indicate that they are incapable of moving through the interface to find the relevant information or accomplish what they are supposed to do. Understanding: used when participants comment on understanding the meaning of labels, instructions or errors.

•

•

1354

conduct laboratory-based usability testing to observe how novice CPAT users complete a series of data entry tasks using the CPAT; interview a cohort of experienced CPAT users to understand the problems they often encounter in using the CPAT software analyze two sets of data to identify knowledge gaps;

Y. Qiu et al. / A Multi-method Approach to Assessing Health Information Systems End Users’ Training Needs

•

integrate the findings into the design of training materials.

Findings from the focus group discussion The normal method of learning the CPAT was labor-intensive, one-on-one coaching, followed by self-directed practice. If the user encountered any problems, they could either approach the trainer or try to solve the problem by themselves. A list of frequent usage problems was identified through focus group discussion. It ranged from system-related problems to computer-related problems.

Usability testing experiment Eight participants for the experiment were recruited from third-year nursing undergraduate students from the University of Wollongong. All the participants majored in geriatrics. They were potential users of the CPAT but had no previous usage experience with the software. The participants were asked to perform the following three representative tasks supported by the software:

Discussion The findings from the TNA identified both procedural and conceptual usage problems in the format of HCI (usability testing) and verbal expression (focus group), which provided valuable input into the design of both the user manual and online help for the CPAT software. For the user manual, usability testing vividly revealed three types of novice-user knowledge gaps, which would become the main focus of the user manual. In addition, frequently asked questions gathered from focus group discussion can be effective contents in the user manual as these problems are also common for novice users. For the online help, problems that were identified in usability testing and were summarized in focus group became the core contents of online documentations. In addition, feedback from experienced users suggested that the learners like functions like video demonstrations. Therefore, this new approach for TNA has proved its capacity to precisely and thoroughly identify the training objectives for both novice and experienced users.

• entering data for a resident; • doing an assessment for a resident; • generating a change monitoring report. The participants were encouraged to “think aloud” or verbalize their thoughts if they were uncertain about how to conduct the above documentation tasks using the software. Findings There were eight video and audio data collected from the experiment. Analyzing the triangulation of audio and video data identifies a series of problems that novice CPAT users encountered in this training session. An excerpt of a coded section of such triangulation is given below to show how users’ interaction with the software was coded. 20:40 – user finished scoring “communication problems” and intended to answer the next group of assessment questions. “How should I go to the next group of questions?”

Conclusion

Navigation - having problems navigating between assessment criteria. By coding all of the participants’ usage problems, three groups of problems that users encountered were identified: • • •

In this paper we have firstly presented the strengths and weakness of a number of traditional methods (interviews and questionnaires) that are used for TNA for novice users of an HIS. Next we described the capacity of cognitive usability testing to capture the cognitive process of HIS users in their interaction with the system. We argue that using the conventional methods alone has limitations and that they could be complemented through combining cognitive usability testing with the conventional methods. This proposed new approach has been explained in detail, particularly through demonstration of a case study, which involved assessing the training needs for users of a nursing information system through two approaches: cognitive usability testing and focus group discussion.

basic computer skills, e.g., users do not recognize the drop-down icon in the selection fields. knowledge about the software, e.g., users do not know where to score questions about assessment criteria domain knowledge about nursing documentation, e.g., users do not understand some assessment questions

Focus group discussion A semi-structured focus group discussion involving nine veteran users from the Hammond Care Group was conducted to explore their learning and work experience with the CPAT. The participants were the actual users of the CPAT. Their roles included facility manager, trainer, consultant and dementia care worker. Most of them have more than three year’s CPAT usage experience. Participants were prompted to provide their answers for the following questions: • • •

Acknowledgement We would like to acknowledge the contributions by: All the participants in the laboratory and focus group. Mr. Richard Fleming, owner of the CPAT application, for his support in providing introduction to the CPAT software for the participants of this experiment. Dr. Patrick Crooks and Ms. Joanne Joyce and staff in Nursing school in University of Wollongong for encouraging and recruiting their students to participate in the experiment. Dr. Madeleine Cincotta, for her generous help in editing and proof-reading the final manuscript.

How did you learn to use the CPAT? Could you recall any problems encountered when using the CPAT? What kind of help do you expect when you encounter problems?

1355

Y. Qiu et al. / A Multi-method Approach to Assessing Health Information Systems End Users’ Training Needs [19] Smith SM. Frustrated feelings of imminent recall. In: metcalfe J, Shimamura AP, eds. Metacognition: knowing about knowing 1994. [20] Norton B. Selecting learning experiences to achieve curriculum outcomes. In: Billings DM, Halstead JA, eds. Teaching in nursing : a guide for faculty. St. Louis, Miss: Elsevier Saunders 2005. [21] Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of Biomedical Informatics. 2004;37:56-76. [22] Kushniruk AW, Patel VL. Cognitive approaches to the evaluation of healthcare information systems. In: Anderson JG, Aydin CE, eds. Evaluating the organizational impact of healthcare information system: second edition: Springer 2005. [23] Patel VL, Kaufman DR. Cognitive Science and Biomedical Informatics. In: Cimino EHSJJ, ed. Biomedical Informatics: Computer Applications in Health Care and Biomedicine (Third Edition). New York: Springer-Verlag 2006. [24] Kushniruk AW. Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Computers in biology and medicine. 2002;32:141-9. [25] Simpson M. How usability testing can aid the development of online documentation. Proceedings of the 8th annual international conference on Systems documentation; 1990; Little Rock, Arkansas, United States: ACM; 1990. [26] Qiu Y, Yu P. Nursing Informaion Systems: applying usability testing to assess the training needs for nursing students. Methods of Information in Medicine. 2006;to be appear. [27] Nielsen J, Mack RL, eds. Usability inspection methods New York Wiley 1994. [28] Courage C, Baxter K. Understanding your users: a practical guide to user requirements methods, tools and techniques 2005. [29] Crowl TK. Fundamentals of educational research. Madison, Wisconsin: Brown & Benchmark 1996. [30] Wilkinson D, Birmingham P. Using research instruments : a guide for researchers. New York ; London: RoutledgeFalme 2003. [31] Morgan DL, ed. Successful focus groups: advancing the state of the art: Sage Publications 1993. [32] Kirakowski J. Questionnaires in usability engineering. 2003. [33] Kushniruk AW. Analysis of complex decision making processes in health care: cognitive approaches to health informatics. J Biomed Inform. 2001;34(5):365-76.

References [1] Gosbee J, Ritchie E. Human-Computer Interaction and Medical Software Development. Interactions. 1997;4(4): pg 13--8. [2] Hunt EC, Sproat SB, Kitzmiller RR. The nursing informatics implementation guide. New York: Springer 2004. [3] Yu P, Comensolia N. An exploration of the barriers to the adoption of information technology in Australia aged care industry. Health Informatics Conference 2004; 2004; Brisbane, Australia: HISA; 2004. [4] Gjestland C, Van Slyke C, Collins R, Cheney P. End User Training: A Decade of Research; 1997. [5] Huang AH. End-User Training Management Systems. Journal of Educational Technology Systems. 2002;31(1):1933. [6] Coiera E. Guide to health informatics: Arnold 2003. [7] RadhaKanta M, Vincent SL. Evaluating end-user training programs. Association for Computing Machinery Communications of the ACM. 2005;48(1):n/a. [8] Kidd MR, McPhee W. The 'lost generation': IT education for healthcare professionals. The Medical Journal of Australia. 1999;171(10):p510-1. [9] Ward JPT, Gordon J, Field MJ, Lehmann HP. Communication and information technology in medical education. The Lancet. 2001;357(9258):p792. [10] Washer P. Barriers to the use of web-based learning in nurse education. Nurse education today. 2001;21:455-60. [11] Ostroff C, Ford JK. Assessing training needs: critical levels of analysis. In: Goldstein IL, ed. Training and Development in Organizations. San Francisco: Jossey-Bass 1989. [12] Gupta S, Bostrom RP. End-user training methods: what we know, need to know. Proceedings of the 2006 ACM SIGMIS CPR conference on computer personnel research; 2006; Claremont, California, USA: ACM; 2006. [13] Howard S. User interface design and HCI: identifying the training needs of practitioners. SIGCHI bulletin; 1995; 1995. p. 17-22. [14] Nelson RR, Whitener EM, Philcox HH. The assessment of end-user training needs. Communications of the ACM. 1995;38(7):27-39. [15] O'Brien E, Hall T. Training needs analysis: the first step in authoring e-learning content. 2004 ACM symposium on applied computing; 2004; Nicosia, Cyprus: ACM Press; 2004. [16] Boydell T, Leary M. Identifying Training Needs. London: Institute of personnel and development 1998. [17] Sein MK, Bostrom RP, Oflman L. Individual differences and conceptual models in training novice end-users. Human computer interaction. 1989;4(3):197-229. [18] Lazonder AW. Exploring novice users' training needs in searching information on the WWW. Journal of computer assisted learning. 2000;16:326-35.

Address for correspondence Yiyu Qiu Email: [email protected] 69/39 Northfields Avenue, Wollongong, Australia 2500

1356

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Making Health Informatics Competencies Useful: An Applied Health Informatics Competency Self-Assessment System H.D. Covvey, S. Fenton, D. Mulholland, K. Young, B.A.E. Waterloo Institute for Health Informatics Research, University of Waterloo, Waterloo, Ontario, Canada

camp program have been launched as attempts to correct this situation. However, there is still minimal awareness of the nature of HI, the roles that HI professionals perform, the education required to play a role in this field, and the opportunities for employment for graduates.

Abstract Years ago we undertook to define Health Informatics (HI) competencies. This effort resulted in the creation of a document that articulated HI roles, the challenges faced by HI professionals, the high-level tasks that they needed to undertake to address these challenges and the competencies (skills, knowledge, and experience) they needed to complete these tasks. Unfortunately, in so doing we created what is arguably the most boring book in history, shoes contents are very difficult to extract, use, maintain and improve. We report here the completion of a pilot of a system that we believe corrects this situation. It is a webbased tool that incorporates all of the material, from roles to detailed competencies, enabling them to be accessed and used for a variety of purposes, the most notable of which is professional self-assessment.

Moreover, many health-related public and private sector organizations that would benefit from HI expertise are only just becoming aware of the competencies required of health informaticians, or that education and training is available in our universities and colleges. One consequence is that recruitments are not properly informed regarding required competencies, when candidates are found, poor choices can easily be made in hiring. The field of Applied Health Informatics (AHI), that most relevant to health organizations and the vendors that serve them, blends the informatics and health disciplines to find and deploy the best possible technical solutions to a wide range of healthcare information-related challenges. Thus, people in AHI need to have a solid technical background, a detailed knowledge of the healthcare industry and of the capabilities it has produced, and a wide range of personal, organizational and cultural, business, managerial and analytical skills.

Keywords: health informatics education, competency assessment

Introduction Canada Health Infoway and other Canadian organizations involved in the deployment of ehealth infrastructure have pointed out that they face major challenges in accessing human resources with Health Informatics backgrounds. There have been estimates that as many as 2,000 positions in Canada go unfilled or are filled by less-than-fully qualified individuals.

AHI self-assessment system We undertook several years ago to address the need for Health Informatics career awareness in part by beginning the development of a unique and innovative web-based Health Informatics competency self-assessment tool. This tool helps students, potential students, and the individuals who want to hire them: (1) to understand the types of roles that exist for individuals trained in Applied Health Informatics, (2) to understand the competencies required to fill these roles, (3) to self-assess their (or their candidate’s) knowledge and skills against the competencies required in these AHI roles, (4) to take objective tests to validate their perceptions of their competencies and (5) to find educational resources that enable them to address deficiencies that are identified.

Canada already has university and community collegebased programs in HI, but these produce less than a hundred HI professionals per year. Even with a number of new education programs emerging, there is little hope of increasing this number significantly over the next 3-5 years. In addition to this, the reality is that few students in our high schools or many of our post-secondary institutions have even heard of opportunities for careers in HI, or even what HI is! This lack of awareness was echoed by academic experts and industry leaders from Canada, United States and Britain at the eHealth Conference 2005 in Toronto, Ontario who called for aggressive action to resolve the health informatics human resources gap. Rapid education deployment programs like AMIA’s 10 x 10 program and the Waterloo Institute for Health Informatics Research Boot-

The primary purpose of this tool is to serve as a self-assessment system and to provide an index to learning resources. It is intended as a resource for students, teachers and employers related to understanding the skills, knowledge and experience expected of competent Applied Health

1357

H. D. Covvey et al. / Making Health Informatics Competencies Useful: Applied Health Informatics Competency Self-Assessment System

Informaticians. Furthermore, the competencies and competency categories embodied in the tool and supporting documentation1 can assist curriculum developers in defining the educational content for Applied Health Informatics education programs. The system is also intended to be able to serve as a “front-end” or entry point to an educational program that can be used to document progress related to the competencies the student needs and has acquired.

sity of Waterloo, led by D. Cowan [4]. WIDE is intended as a rapid pilot system definition environment, based on a decade of Software Engineering research, composed of software services and are customized using declarative techniques. The goal of WIDE is to reduce technological barriers to system design and development. WIDE is primarily based on open source software technology and consists of a number of services and supporting frameworks. Applications can include input forms or reports containing extensive multimedia materials such as imaginative use of maps or any 2-dimensional diagram, websites, databases, indexing and searching methods, agents, and push technologies. WIDE also contains a knowledge management system that supports documentation of technical information and best practices. The structures underlying the services are usually expressed in an XML-based declarative language that uses metadata and XSL. In the WIDE metadata context, “programming” has effectively been replaced with a declarative methodology thus making it possible to provide a wizard or formsbased approach to building Web-based systems. Internally WIDE uses a bootstrap approach; its extensions are implemented using its own metadata technology. WIDE can support a rapid development paradigm and new applications can be quickly built and demonstrated. The components of WIDE are described next.

It should be noted that our framework to define competencies and the tool we built to support self-assessment and access to learning resources are entirely general and can be applied to any discipline, not just to Health Informatics. This makes our framework and tool reusable and of broad potential interest.

Methodology In earlier work [1], we led a process that defined HI competencies and supporting curricula using a team of approximately 100 health and Health Informatics professionals (including HI teachers/researchers, curriculum developers, human resources professionals in healthcare organizations and HI companies, vendor staff, government representatives, and current and candidate students). This work involved the comprehensive documentation of HI roles, the challenges faced by professionals in each of these roles, the high-level tasks (which we called “microroles”) that professionals in these roles need to undertake to address the challenges, and the competencies required to accomplish the tasks/micro-roles [1,2]. This material was then used to define content that could be used as the basis for a number of programs world-wide [3]. One such program has been established at Conestoga College, in Kitchener, Waterloo, Ontario, which registered its first students in the fall of 2005. The competency definition project was funded by the Canadian Institutes for Health Research (CIHR), its products are frequently cited, and today it stands as one of the few definitions of HI competence that have been derived using a logical framework (a work breakdown structure) rather than being solely based on the preferences and opinions of teachers with their own programs.

A mapping services framework. Interactive maps are delivered from a map-server, which supports zoom-in or zoom-out functionality and positioning over areas of interest. When connected to a database or other directory the maps can be used to display and interact with the location of a geo-referenced object. Map searches can be defined by a circle, or general polygon. The mapping service framework does not use traditional GIS software. A diagram and chart services framework. This framework manages and delivers specified interactive diagram and chart types upon request for presentation of data on the Web or in other formats. The diagram and chart services are also based on the Scalable Vector Graphics (SVG) open W3C standard and so provide similar functionality to the mapping service framework.

Once having completed the documentation of competencies (there are on the order of 400 for AHI alone) and the other components of this project, we recognized that the form they were in needed to change. In particular, they were incredibly boring to read, making improving them an un-motivating task. In 2001, F. Lau at the University of Victoria made the suggestion that we consider some way of encapsulating or packaging the competencies and other content in a directly usable form. We considered this suggestion and realized that significant value could be derived by packaging the competencies in a software tool that made them accessible, available and integrated for review, understandable, and actually usable by individuals in the field.

An XML-based metadata framework. The structure of databases, websites, agents, and applications including reports and input forms with maps and diagrams are described using XML. They are transformed into operating applications through the use of XSL “programs.” Any application can describe and subsequently access databases or Web sites reachable anywhere over the Internet. A report services framework. This framework supports the management of interactive report and input form types including maps, charts and diagrams and delivers them on request for presentation on the Web or in other formats. The user indirectly specifies the form type and the data that that is to be presented or requested; the framework chooses the report or input form type and populates it with the requested data.

We called this tool “WebSAT” and built it using the Webbased Informatics Development Environment (WIDE) developed by the Computer Systems Group at the Univer-

1358

H. D. Covvey et al. / Making Health Informatics Competencies Useful: Applied Health Informatics Competency Self-Assessment System

A content management services framework. This framework supports the management of text and multimedia information in a database where it can be viewed, searched, maintained and then published for use on the Web or in other formats.

In order to make the initial version of the system of manageable scope, we limited ourselves to providing a tool for those interested in assessing their own competencies relative to those required by AHI professionals. Furthermore, we did not address many of the user interface issues. For example, we did not allow individuals to assess themselves first at a high-level (versus broad categories of competencies) and then at a more granular level within these categories.

An access control service framework. Access to any content such as a database, website or other text and multimedia content can be provided with multi-level access controls to determine who can read or change data.

This past year we extended the development of this tool through the able assistance of a student working in the Undergraduate Research Assistant (URA) program in the David Cheriton School of Computer Science (D. Chodos). This student added references to Web-based educational materials for sample set of competencies. This allows users to click on a link and access educational documents, on-line educational programs, and on-site courses that provide a means to correct competence deficiencies. This has turned out to be an excellent capability that supports lifelong learning in this field. This latter work was reported at the recent eHealth 2005 Conference in Toronto [5].

A Web and database searching service framework. This service framework contains an indexing agent and search engine that will index known websites and databases and support searching. The results from Web searches are categorized based on different search criteria such as the proximity of words in a phrase. The results of combined database and Web searches can be presented together. The results of the two searches can be compared to see if new results have appeared in the intervening time interval. A push/notification service framework. The general push/notification service framework allows developers to create systems that allow users to specify conditions under which they wish to be notified or have information pushed at them.

During the last 6 months we have made major improvements to the system: •

A new user interface has been developed that makes use of the system more intuitive. • Users can now assess themselves at a high-level (compared to competency categories) or at a detailed level, reducing the effort if the user does not need to go to a deeper level. • Improved graphics show the comparison of the user’s stated competencies to those required by the selected role. • There is a new query capability that allows recruiters to retrieve the competencies required for various roles, based on the importance of the competencies. • The system supports the self-assessment of experience. • A capability to take objective tests is now included so users can validate there perceptions of their competencies. Finally, the overall system now has a better look and feel to improve the user experience.

An agent service framework. The agent service framework supports the description of agents that will act autonomously to perform utility tasks within an application. Agents are often defined to manage redundancy. For example, agents could be defined to verify the content of “local” databases against authoritative sources or to allow a user to type information once while submitting the data to multiple databases or Web sites. The academy – a knowledge management framework. The academy framework is used to support widespread dissemination of “documentation” and knowledge describing how applications can be built from the WIDE Toolkit.

Description of WebSAT WebSAT (Web-based Self-Assessment Tool) is a Webbased AHI self-assessment system that enables individuals to review HI competencies, assess their own competencies and compare these to the competencies required for specific roles. The development of this system was funded using internal resources and volunteer labor and is now available as a demonstration website on request to the author. In return, we ask users to complete a brief assessment of the system for us.

Mode of operation WebSAT operates as follows: •

Our competency definition work identified three types of HI professionals: (1) AHI (Applied Health Informatics) professionals, who define the requirements for, procure, deploy, implement, manage, guide the use of, and evaluate HI systems and methods in health enterprises and their supporting industries; (2) RDHI (Research and Development HI) professionals, who teach, do research, and develop innovative HI tools for the health system and are typically found in academia and private industry research labs; and (3) clinicians who need HI competencies to be good clinicians (called Clinician HI or CHI).

•

•

1359

Users who access the system are asked to register under a user identity and password that they create. The user can review the definitions and detail of any of the components used to define competencies, including possible roles, challenges, skills, etc. This is the part of the system that supports the understanding of AHI itself and the review of details that the content developer may add, delete or improve. The user can then select one of the potential AHI roles, after reviewing descriptions of each role. The users stated competencies will be compared with this role (this can be changed at any time).

H. D. Covvey et al. / Making Health Informatics Competencies Useful: Applied Health Informatics Competency Self-Assessment System

•

The user can then access approximately 20 “competency categories” (groups of similar competencies), each containing multiple specific competencies, and can assess him/herself as to the level of knowledge or skill he/she has. Competencies are assessable as one of seven levels from “no knowledge” to “expert”). These levels include: UNA=Unacquainted; ACQ=Acquainted; PAM=Passing Familiarity; GAM=General Familiarity; FAM= Working Familiarity; CAP=Capable; EXP=Expert) • If the user wishes, she/he can drill down to detailed competencies within the competency categories. • The system allows corrections and the saving of input for later completion. • Once the user has responded to all competencies at whatever level of detail desired, the system compares the user’s competencies to the selected role and provides graphical feedback (a bar chart of responses versus requirements) as well as textual feedback. Different roles can be tried, to see if there is a better fit with these. • For areas requiring further work, the users have the opportunity to receive system guidance to educational resources. • Users can access objective tests for each competency. The competency categories included in the system are:

Results WebSAT has now been tested by both by graduate students in our HI program and by students in a new HI program at a nearby community college. Students have reported a high degree of satisfaction with the system. They found the system useful and informative, and it gave them a clear view of what they still need to learn and where they stand relative to the requirements of various roles. We have recognized, however, that other improvements are possible, and we continue to enhance the system further along the following lines: 1. We are undertaking further improvements to its user interface so that it can be more efficiently used by inexperienced users. 2. We are completing the addition of learning links to Web-based educational sources to be referenced by students with identified weaknesses. 3. We are adding additional objective tests and creating a new testing engine. The existing one does not support sufficiently complex multiple choice options. 4. We will extend the tool to incorporate Research and Development HI (academic-level) self-assessment to the same degree as the AHI assessment. 5. We are in the process of offering the system to other programs for use as a tool for students, and we are seeking recruiters to test its support for their activities.

1. Personal Competencies for AHI 2. General Computing Competencies 3. Health Computing for AHI Professionals 4. Key IT Usage for AHI Professionals 5. General Health System Competencies 6. General Business/Management 7. General IS Department Management 8. Team and Human Resources Management 9. Re-Engineering and Management of Change 10. Strategic and Operational Planning 11. Assessment of the Value, Effects, and Cost of IT 12. General Technology/Systems Life-Cycle Management Competencies 13. Procurement Competencies 14. Systems Implementation and Integration 15. Systems Maintenance and Support System Customization/Ad Hoc Development 16. Project Management Competencies 17. Education and Training Competencies 18. Vendor/Service Provider Competencies 19. User and Process Observation and Assessment Competencies 20. Security Management Competencies 21. Information and Data Collection, Analysis and Management Competencies A total of approximately 400 separate competencies allocated under these categories are addressed by the system.

Summary and Conclusions We have developed a Web-based competency self- assessment system for Applied Health Informatics. This system allows individuals to assess themselves as to the congruence of their competencies with those required for specific roles. The system has been tested on students and is in the process of being enhanced and disseminated. Individuals interested in accessing the system personally or using it as a component of their programs are invited to contact the first author. The system is offered without charge. Acknowledgements This work was supported by NSERC under the Executive Industrial Research Chair program and by Canarie Inc. The authors are grateful for the contributions of D. Chodos, research assistant, at the University of Waterloo. The authors are grateful to F. Lau at the University of Victoria who first suggested the development of this system.

References [1] Covvey HD, Zitner D, Bernstein RM. “Pointing the Way: Competencies and Curricula in Health Informatics”, H/IT Advisors, March 2001 (Book: at http://hi.uwaterloo.ca, May 2007). [2] Covvey HD, Zitner D, Bernstein R. “The Development of Model Curricula for Applied Health Informatics”, Medinfo 2001, London, England, September 2-5, 2001. [3] Covvey HD, Zitner, D Bernstein RM, MacNeill JE. “The Development of Model Curricula for Health Informatics”,

1360

H. D. Covvey et al. / Making Health Informatics Competencies Useful: Applied Health Informatics Competency Self-Assessment System Proceedings of the International Medical Informatics Association WG 4 Meeting, April 2003. [4] Cowan D, Fenton S, Mulholland D. Lowering Technological Barriers to Developing, Deploying and Maintaining Web-based Interactive Information Systems, to appear in the International Journal of Technology, Knowledge and Society 2005.

[5] Chodos D. Accessing Applied Health Informatics Educational Resources via a Web-based Self-Assessment Tool, eHealth 2005 (Poster), May 2005.

Address for correspondence WIHIR, DC3333, University of Waterloo, 200 University Ave. West, Waterloo, ON, Canada, N2L 3G1; [email protected]

1361

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

E-learning for Students in their First Year: A French Experimentation at the Medical School of Grenoble Jean-Marie Renarda, d, Daniel Pagonisb,c, d, Jean-Philippe Vuillezc,d, Jean-Paul Romanetc, d, Bernard Selec, d a

CERIM, CNRS-EA2694, Faculty of Medicine, Lille2 University, France b SIIM, University Hospital, Grenoble, France c TIMC, Faculty of Medicine, Grenoble University, France d UMVF, French Virtual Medical University, http://www.umvf.org

sponds today to 12% of the number of registered students (year 2006, 1300 registered students against 166 students allowed to continue the second year course). The amphitheaters have a normal capacity of 650 places whilst 800 students actually attend the lectures. This number cannot possibly be increased for safety reasons.

Abstract A local study carried out in the Medical School of Grenoble shows that teaching in the first year in medicine studies satisfies neither the students, nor the teachers. The Faculty of Medicine of Grenoble decided to set up a reform in order to offer a high quality education. This reform leads to a complete reorganization of the curriculum and to the intensive use of new information and communication technologies of information, in particular, the use of multimedia documents. The communication and information technologies team of the Faculty of Medicine of Grenoble carried out an innovating and daring reform to start at the academic year 2006-2007. The new course is built on three activities: self learning on multi-media resources, meetings with teachers for questions-answers sessions and tutorials animated by older students. This article reports the first results for this successful project. In the academic year 2006-2007, are concerned 1290 students, 40 teachers and 8 disciplines.

A contest which no longer fulfils its role The contest no longer fulfils its double role of regulating the number of students and selecting the best ones. Studies showed that the harder a contest is, the less effective the selection of the best students is. This strong selection has many direct and indirect consequences such as some influence on the content of the lessons. Difficult lessons only exist for their selective character and not for their relevance. Inadmissible behaviours

Introduction

The extremely selective aspect of the contest has generated an attitude of unfair competition behaviour, such as voluntary disturbance during the lectures by a few students, voluntary locking of the books to limit there availability, paper planes sent through the amphitheatres, going to the extreme: sale of already annotated lectures showing voluntarily added errors. These behaviours go against the university values of solidarity and equity amongst students.

Observed difficulties during the first-year curriculum

Less equity between the students

A degradation of working conditions

Many a student registers in private courses which offer method and regularity in work in order to make up for the effects of these bad working conditions. This is also a means for the getting of annals and self assessment in relation to others. In the French system, the inscription to these courses generates a discrimination against money since the first access price exceeds 1500 € .

Keywords: e-learning; first year medical curriculum; multi-media; Grenoble, France.

This degradation was explained as a consequence of years of teaching without innovations or improvements. From a teaching point of view, the acquisition and transfer of knowledge remain extremely antiquated. The majority of learners remain passive. The student remains isolated before a task that cannot be managed without help. This may generate harmful consequences on his learning capacity..

Effects over the forthcoming years One of the indirect but considerable effects of this situation is that the motivation to work is getting slack. This is understandable considering the very great efforts provided for during the first year. Teachers estimate that students need three years to come back to a normal working attitude.

A regular increase in the number of students The increasing number of students is a factor which amplifies the degradation of the working conditions. Since 1975 the number of admission in second year has dramatically increased. The “numerus clausus” of Grenoble corre-

1362

J. M. Renard et al. / E-learning for Students in their First Year: A French Experimentation at the Medical School of Grenoble

for the years between the second and the third year (20032004) of the medical curriculum. Another particularity is the strong motivation of all the teams involved in teaching for an intensive use of new communication and information technologies.

Solutions already tested by other universities were not very conclusive Faculties of Medicine, facing the same difficulties, and drawing the same conclusions adopted various solutions: •

•

One of them is the repetition of the courses, which obliges the teachers to reproduce at various gaps the same course in many amphitheatres. That demands a great involvement from the teachers and availability of many extra amphitheatres (with a capacity exceeding 700 places). The risk is this might encourage the action of disturbing students, increase the loss of interaction between teacher and students and generate problems with regards to safety. For all of these reasons, this solution cannot be applied in all Faculties. Another very widespread solution adopted by various Faculties is the broadcasting of the course by Videotransmission in a secondary amphitheatre. The principle consists in diffusing on line and on large screen in a secondary amphitheatre the course done by the teacher lecturing in the first amphitheatre. This reduces the number of mobilized teachers. However, to be added further to the already listed problems, the high costs for expensive, fragile and not always reliable equipment which cause considerable discontent among students. Answering the student’s questions is difficult and this increases the risk of disturbance.

New methods of training The reform does not change either the curriculum program or the organization of the contest. The year (2006-2007) is divided into 2 six-month study periods. Each semester ends with a part of a contest (January and May 2007). One six-month period is divided into 12 training sequences, each sequence divided into 4 weeks. Each week is devoted to a different training activity. •

•

The need for an important reform Whatever the solutions considered by the universities, the quality of the teaching is unfortunately not improved, on the contrary it is damaged. To face the whole of these drawbacks, the Faculty of Grenoble team has proposed a reform which leads to an original solution.

•

Materials and methods Elaboration of an original reform Emergence of the project of a reform for the first year medical curriculum, all the actors at the medical school carried out a complete thinking on the modernization of teaching. Administrative people (Dean...), teachers, and students (elected representatives) endeavoured to find a solution which would improve the quality of the teaching. In there analyse, they took into consideration different dimension like behaviours [1,2], cognitive sciences [3,4] and possibilities offered by the communication and information technologies [5,6]. A reform combining a new organization of teaching associated with an intensive use of communication and information technologies appear to be the best alternative.

•

The first week is devoted to the study of the courses on DVD-Rom: one or two different disciplines are studied over the week in the form of multi-media courses animated and wired for sound by the teacher’s commentary. The second week is used for the formulation of all questions destined to teachers. These questions exclusively relate to the multi-media courses studied during the previous week. They are the base of the Question/ Answer Meetings. The questions are formulated on the Faculty of Medicine of Grenoble’s web site which is destined to the PCEM1 students (www.medaticegrenoble.fr). The on-line form is open each week, from Saturday to Tuesday exclusively. The access is secured by a login-password provided for at the time of final registration. The third week is devoted to the Question-Answer Meetings. Each discipline studied during the sequence is followed by a 2 hour Question/Answer Meeting. These meetings are held in small amphitheatres. Students are divided into eight groups. The meetings are ensured by the teachers in charge of the discipline. The answers are built from the questions collected on the on-line form (www.medatice-grenoble.fr). The fourth week is dedicated to the tutorials. The tutorials are directed by third year tutor medicine student. These meetings are intended for QCMs training. QCMs are validated by the teachers, corrected and explained during the meeting by the tutor students. These students are trained and supervised by the teachers in charge of the disciplines and the contest. The meetings are held the week following the questionsanswer meeting.

Recording of the multi-media courses Recordings are carried out by the Stendhal University Audio-Visual Service. The multimedia resources consist of animated slides commented and recorded by the teachers.

Specificities of the medical profession Given the teacher’s specific statute, the Faculty of Medicine offers more freedom. Most teachers of Medical School are not subject to a quota of hours before the students. A modification in manner of delivering of the courses will not be a problem due to their teacher statute.

Evaluation of the reform The evaluation of the project lies on statistical data describing the correct working of the project as well as on satisfaction surveys carried out weekly. At the end of each tutorial session, students are questioned. The information collected concerns the evaluation of the lesson and training methods.

Grenoble specificities This project is in the continuity of previous reforms made for the years following the fourth year (2002-2003), and

1363

J. M. Renard et al. / E-learning for Students in their First Year: A French Experimentation at the Medical School of Grenoble

Agenda for the project The Table 1 reports the agenda for the project. Table 1 - Agenda of the project Date

Event

November 2005

Realization of the preliminary study

December 2005 to January 2006

Redaction of the performance specification

February 2006

Validation of the project by the Université Joseph Fourier’s council

March 2006 to September 2006

Production of the multi-media records

July2006

Opening of the web designed for the students in their first year

August 2006

Production of the DVD-Rom number 1 (2000 pieces)

September 2006

start of the curriculum – distribution of the DVD-Rom N°1 to students

Figure 1 – self-learning rich media The questions-answers sessions The students send their questions through an on-line form. The questions must be related to the courses delivered during the previous week and are collected from eight independent groups of students. At the end of the first cycle (the first four weeks), 900 students have used the system and asked 7484 questions which were marked "pertinent" 38 879 times by other students. The satisfaction level for that system was 85%. The session with teachers was organized in small amphitheatres. Every student followed two sessions per week. Each session lasts two hours The ratio questions-answers sessions over self-learning time is two hours of questionsanswers for ten hours of self-learning time.

Results Numbers of students and teachers The teaching of the first year of the medical curriculum in Grenoble concerns 1290 students, 40 teachers and 8 disciplines for the year 2006-2007.

The number of questions per teacher varies between 200 and 300. The quality of the question is qualified as good by teachers. The satisfaction level of the student is between 55% and 75%.

Self learning on DVD-ROM

Tutorial session

There was 230 sessions corresponding to 460 hours of recordings. The slides are in flash-R, mp3 and xml format. The materials are equivalent to 220 hours of listening by the students.

The 1290 students are divided into 40 small groups. Every student follows two sessions per week. Each session lasts two hours. 96 hours per student and per year are envisaged. The tutors are older students (120 students in their third year of medical curriculum). A session is animated by two students (always the same two students). The two students are supervised by teachers and spend two hours for the preparation of a session which content is a set of QCM.

Every six months, a new DVD-ROM is distributed to the students. The DVD-ROM contains 4, 7 Go of data (The mean size of one course is about 26Mo). DVD-ROMs are pressed in 2000 specimens. Each DVD-ROM cost 1,60 euro.

1 000 000 QCM are planned to be written over the year.

The courses are also available on the web site dedicated to students under the form of Podcast using Open source Xvid file format [7].

The mean satisfaction rate is 94%.

The user assistance for DVD-ROM usage only received 16 calls and all of them were solved by the assistance. The hot-line was overloaded by calls coming from students registered in Universities other than Grenoble to get the DVD-ROM. Students will fill an electronic form after each course. Every week 1050 students will fill the evaluation form. 36 000 forms will be delivered during the year.

Figure 2 – The four activities of the learning cycle: self-learning on multimedia supports (Etude de cours), preparation of questions for teachers (FLQ), questionsanswers session with teachers (ie 26/9 08h-10h) and tutorials (ie 2/10 18h-20h).

The students’ satisfaction rate ranges between 75% and 85%

1364

J. M. Renard et al. / E-learning for Students in their First Year: A French Experimentation at the Medical School of Grenoble

Logistics

Conclusion

A 700 places amphitheatre was converted into a multimedia room. It was adapted with 130 desktop computers, electrical connectors for personal laptop computers and with WIFI network connexion.

"The doctor of the 21th century must also have teaching competences in technologies which must be acquired at the time of his training" (Thierry KARSENTI).

This room can accept 248 students who work in twos. It is open from 8h-20h and from Monday to Saturday

The MedaTICE project showed that motivation and conviction of a team make it possible to carry out a joint and innovative project to it success.

There is a low cost rental service for laptop computers and, in some cases, laptops can be lent. Other rooms were adapted with computers and dedicated to questions-answers sessions as well as tutorials sessions.

Undoubtedly, the action taken by the Faculty of Medicine of Grenoble will be the starting point of a great scale disruption and will lead other faculties to become involved in similar projects.

During the first week, only 5 students used the 700 place amphitheatre. During the first month, the maximum number of students was 30.

It will be very important to follow on a monthly basis the statistics on the project and to do a complete analysis at the end of this first year.

The Web Site

Acknowledgements

It started in June 2006 and is reserved to first year students (www.medatice-grenoble.fr). It provides information on the organization of the curriculum. It gives access to the forms planned for their asking questions to the teachers. It is an entry point to access to the podcasts.

Our warm thanks to Michèle Jacquemart for her assistance in preparing this paper.

References [1] Apollon D. L'enseignement supérieur face au numérique en Europe : Tradition, innovation et résistance. Université de Bergen, 2006. [2] Basque J. Influence de béhaviorisme, du cognitivisme et du constructivisme sur le design pédagogique.1999. www.teluq.uquebec.ca/~jbasque.montréal. [3] Bernatchez PA. Vers une nouvelle typologie des activités d'encadrement et du rôle des tuteurs. Université de Montréal, 2001. [4] Mayer, C. When learning is just a click away: Does simple interaction foster deeper understanding on multimedia messages?. Journal of educational psychology 2001: 390397. [5] Bernartchez PA, Marchand L. Encadrement médiatisé par ordinateur. Distances 2005 : Vol. 7. http:// cqfd.teluq.uquebec.ca/distances/v7n1_b.html. [6] Bellier S. Le E-learning. Editions Liaisons, RueilMalmaison, 2001. [7] Xvid, Open source MPEG-4 Video Codec; Wikipedia Nov 2006. http://en.wikipedia.org/wiki/Xvid

The average number of daily hits is 875 and the maximum 1200. In October, there has been 21 899 visits and 448 932 pages loaded.

Figure 3 - The web site reserved to first year students

Address for correspondence

Discussion

Jean-Marie RENARD CERIM, EA-CNRS-2694 1, place de Verdun 59045 Lille Cedex mail: jean - marie . renard @ univ - lille2 . fr

The first result of satisfaction from students must be monitored to confirm the efficiency of this organization. An element that needs to be confirmed is the uselessness of the multimedia amphitheatre.

Daniel Pagonis Faculté de Médecine de Grenoble Domaine de la Merci BP 170 LA TRONCHE 38042 GRENOBLE CEDEX 9 mail: Daniel . Pagonis @ imag . fr

A curriculum based on a four week cycle requires strong motivation from the teachers, how it will evolve in time is the question to ask. The efficiency of a learning cycle lead on the sequence: self-learning on multi-média materials, questions-answers with teachers and tutorials with senior's student must be the subject for further analysis.

1365

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

E-learning at Porto Faculty of Medicine. A Case Study for the Subject ‘Introduction to Medicine’ Cristina Costa-Santosab, Ana Coutinhoc, Ricardo Cruz-Correiaab, Ana Ferreirabc, Altamiro Costa-Pereiraab a

Department of Biostatistics and Medical Informatics, Faculty of Medicine, Univ. of Porto, Portugal b Centre for Research in Health Technologies and Information Systems – CINTESIS, Faculty of Medicine, Univ. of Porto, Portugal c Department of Informatics, Faculty of Medicine, Univ. of Porto, Portugal

sible professor and lecturers of the same subject [3]. The first interface, developed using PHP and HTML programming languages and a relational database, namely Postgres, in a LINUX server, was used to store and manage information. In the academic year of 2005/2006 an e-learning tool (i.e. Moodle) was introduced. This was done in order to use information systems’ technology to provide for automatic students’ work correction and evaluation.

Abstract The main objective of the Introduction to Medicine (IM) subject of the first year of the Medical Course at the Faculty of Medicine of the University of Porto is to provide students with a first contact with the areas of Biostatistics, Medical Informatics, Bioethics and the History of Medicine in the belief that they will be better prepared to learn, research, evaluate, share and decide within their practice. This paper presents a case study that describes how the subject IM is organized and how the b-learning tool (Moodle) is used to correct and grade the students’ work. From the 239 students registered to attend the Introduction to Medicine subject 12% failed. The average grade among the successful students was 16 (out of 20). In the previous academic year only 2% of the students failed. However, among the successful students, the average grade was inferior (15 out of 20). The e-learning model that was described in this paper was successful because the results show that the students that made use of the Moodle got better grades.

This paper presents a case study that describes how the subject of Introduction to Medicine is organized and how the ‘b-learning’ tool is used to correct and grade the students’ work in the academic year of 2005/2006. Motivation for the use of an e-learning tool A vast majority of first year Medical students are not interested in learning basic scientific subjects. They probably think that their content is not important for the clinical work ahead of them, and for which they have motivation. The introduction of technologies such as e-learning tools that bring new interactive methods for communication and simulation can help the lecturers to motivate the students and facilitate their learning process. Several case studies support the use of Web technologies in order to teach undergraduate Medical students [4].

Keywords: education, medical faculty, e-learning

Initial expectations

Introduction

Although some of the material from previous academic years already included interactive means to communicate and learn (e.g. forums, chats, placard) the main objective to introduce Moodle was to use the specific module that allows students to make exercises and tests online, which are corrected automatically. With this module, the lecturers wanted the students and themselves to have a better understanding of the evolution of the students’ learning results along the year with the main goal to improve the students’ final results at the end of the year. This module eliminates the waiting time that was needed for the lecturers to correct the exercises and tests made by the students therefore simplifying the process of correcting and grading the weekly correction of 230 tests.

The practice of Medicine requires the use of methods in order to acquire, store, process, analyse, transmit, evaluate and assess information as well as medical knowledge [1]. The study of scientific areas such as Biostatistics, Medical Informatics, Bioethics and the History of Medicine can contribute to improve the practice of Medicine and make it well supported by ethical principles and, therefore, socially effective. The main objective of the Introduction to Medicine subject of the first year of the Medical Course at the Faculty of Medicine of the University of Porto (FMUP) is to provide first year medical students with a first contact with the areas described above in the belief that, as future doctors, the students will be better prepared to learn, research, evaluate, share and decide within their practice [2]. Although there was some existing material on the Internet, it was only in October 1999 that an intranet for this subject was designed and implemented by the respon-

Objectives The main objective of this case study was to create an elearning platform that allows the students and lecturers to

1366

R. Cruz-Correia et al. / E-learning at Porto Faculty of Medicine. A Case Study for the Subject ‘Introduction to Medicine’

follow the progress of students’ work along the year. This helps the students to understand how they need to change or not their learning process with the real-time feedback they get.

B-Learning - integrating e-Learning components within the teaching process The Biostatistics and Medical Informatics modules are taught with theoretical lessons (where the main concepts are presented to the students) and practical lessons (where the lectures support the students in their continuous selflearning process).

Other objectives include facilitating students’ access to the lectures’ material as well as other interactive material and online communication means between students and lecturers that help and motivate the students to study and learn.

It is within the practical lessons that the integration of the theoretical concepts with the e-learning component available at the Moodle platform is made (Figure 1).

During the academic year, the lecturers noticed that the objectives were being achieved because the students were using regularly the material described above. This could be seen by consulting the usage statistics module within Moodle.

The practical lessons have the duration of 2 hours and a half every week and the students have access to their own personal computer. The lecturers start by discussing with the students the concepts given at the theoretical lessons and then these concepts are practised within the e-learning platform. The lecturers assess students’ acquired knowledge (all the 239) individually with online mini-tests (Figure 2) at the end of every lesson. The mini-tests have a maximum duration of 5 minutes and include 2 or 3 multiple or numeric questions. These questions are selected randomly from a pool of questions that is created by the lecturers. The pool about a specific issue is big enough so that when students start the test they have most probably different questions to answer in their test or at least setup in a different order from all the other tests, so that copies among the students is almost impossible. All the information that is available within the Moodle can be accessed at anytime from any computer that has Internet connection. However, the mini-test can only be accessed during the practical lessons, only in some selected PCs where the lecturers need to insert a key so that the students can enter the right interface. This feature avoids problems with counterfeit of mini-tests because their results are taken into account for the students’ final grade. When the 5 minutes expire the mini-test is corrected automatically and the final grade is showed to the student. If the student does not submit his answers before the mini-test finishes, the answers given until that moment are automatically submitted and the grade given accordingly.

Materials and methods Structure and grading system Introduction to Medicine is a subject from the first year of the Medical Course of the Faculty of Medicine of the University of Porto that integrates four modules regarding the following scientific areas: Biostatistics, Medical Informatics [5], Bioethics and the History of Medicine. In the academic year of 2005/2006, 239 students were registered in order to attend this subject. The Bioethics and History of Medicine modules are only taught with theoretical lectures and graded with a written examination. Each module weights 2 out of 20 for the final subject grade. The other two modules, Biostatistics and Medical Informatics are taught with theoretical lectures, an e-learning component and practical lectures. Each one of these modules weights 5 out of 20 in the final classification (2 for the written examination and 3 for the work students do during the year at the practical lectures). The 6 values necessary to complete the maximum grade of 20 come from a group project that is compulsory for all the students to complete at the end of the academic year. The group projects are supervised by the lecturers that help the students to understand and organize their work. In addition, students can attend seminars that give important information for the development and implementation of the group project.

Figure 2 - Example of a weekly mini-test presented to the students, with automatic correction and timing

Figure 1 - Moodle e-learning platform that is used within the Introduction to Medicine subject

1367

R. Cruz-Correia et al. / E-learning at Porto Faculty of Medicine. A Case Study for the Subject ‘Introduction to Medicine’

The average of all the results from the mini-tests can contribute to a maximum of 6 out of 20, which is the final grade. Within the Moodle e-learning platform the content is organized on a weekly basis regarding each week of the academic year, in a total of 33 weeks. Different types of content are included: • •

•

•

•

•

•

SLIDES – the presentations made within the theoretical lessons; FORUM – used to exchange and share information about the subject between the students and the lecturers; EXERCISES – exercises to apply the theoretical concepts are made within the practical lessons and they allow the students to use real data from clinical studies. The tables with the real data that does not affect ethical issues related with the patients are also available in the Moodle. SELF-ASSESSMENT – the students can practice what they have learnt with self-assessment exercises with automatic correction; MINI-TESTS – these are exercises that count for the students’ final grade. They are timed and corrected automatically by the Moodle; GROUP PROJECTS – the group projects are submitted by the students and commented by the lecturers within the Moodle, during 3 review phases; LINKS TO OTHER ONLINE MATERIAL: – MEDSTATWEB [6] - http://medstatweb.med.up.pt/ - the biostatistics interactive manual (figure 3) was developed by the lecturers and explains the basic biostatistics theoretical concepts with the use of practical examples and simulations (figure 4) – MEDICAL INFORMATICS - http://im.med.up.pt/ - the medical informatics manual was developed by the lecturers and explains theoretical concepts of Medical Informatics (see figure 5).

Figure 3 - Biostatistics interactive manual

Figure 4 - Example of a simulation within the Biostatistics manual that explains the concept of regression. The user can move or insert points. The regression line is the ANOVA table and the relation coefficient automatically updated according to the points

Advantages and disadvantages Without an e-learning platform like Moodle it would be very difficult to make all the exercises and corrections that are done now because the number of students is too large compared with the number of lecturers. This model allows students and lecturers to search and update the contents anytime, anywhere via Internet. It also permits the lecturers to monitor each student individually and the preliminary as well as final results of the subject and all the students in a generic way by allowing the analysis of the accesses that were made to its contents. A disadvantage to consider can relate to the privilege that students with access to the internet at home may have in relation to the ones that do not have. To minimize this issue the students are encourage to use the computer laboratories within the Centre for Informatics at the FMUP that provide the Internet connection.

Figure 5 - Medical informatics manual Structure and implementation The implementation and integration of the Moodle e-learning tool was done in collaboration with the Centre for Informatics at the FMUP. The contents have been developed, improved and updated by the lecturers during the past 10 years. In the academic year of 2005/2006 a group including 7 lecturers, the professor responsible for the sub-

1368

R. Cruz-Correia et al. / E-learning at Porto Faculty of Medicine. A Case Study for the Subject ‘Introduction to Medicine’

deviation of 4) in the final exam for this same module. Those who got an average grade between 10 and 15 in the mini-tests got an average of 10 (standard deviation of 4) in the final exam. The students that got an average grade equal or superior to 15 in the mini-tests got an average of 14 (standard deviation of 4) in the final exam. These differences have a statistic significance (p 8

Regular Assessment

Focus groups

Do not Screen

The objective of the two focus groups was to gain an understanding of the knowledge, attitudes, and beliefs of APN students regarding the use of a PDA-DSS. A purposive sample of 6 first and 6 fourth semester pediatric APN students were invited to participate in a 1-hour focus group. A 10-question, investigator-developed, interview guide was used to facilitate the discussion. The questions were developed from discussions with a panel of doctorally-prepared experts in informatics and qualitative research methods, as well as from a review of the qualitative literature on the attitudes of physicians [33-34] regarding PDA use in the clinical setting.

Document A Reason

Screen

Provide List of Reasons Not to screen

Assess parental depression/family history of depression Screen S-MFQ Total >/= 8 Screen for suicide risk Preceptor Validation

Total < 8 Screen for suicide risk

Screen complete Provide Diagnosis

Figure 2 - Application algorithm Usage

At the beginning of the group, each participant was given a copy of the interview guide. An experienced psychiatric clinical nurse specialist (PB) and an experienced pediatric APN (JCH) each conducted and facilitated one of the focus groups. An observer (RJ) took notes and audiotaped the discussion. During group discussion, the facilitator put forth a question and participants responded while interacting with one another. Probes were used to gain more detailed information and personal accounts. Data were content analyzed to extract major themes related to knowledge, attitudes, and beliefs.

A total of 25 students were randomized into depression screening with 8 being fourth semester students and 17 being first semester student. In the fourth semester group, all students screened at least one child or adolescent for depression with an overall screening rate of 44.2%. The first semester students’ overall screening rate was 37% with 23.5% doing no screening for depression. Students who failed to screen tended to be in specialty sites, ED follow-up clinic, or in private practices. Of the 376 encounters eligible for pediatric depression screening, only one third (n=124) were screened for depression. If the student did not screen the child, they documented reasons for not screening. Over 81% cited other and about 8% noted a medical emergency as a reason for not screening. The remaining reasons included retardation (3.2%), learning disability (2.7%), patient refusal (2%), agitation (1.2%), other cognitive disorder (1.1%), and amnesic disorder (0.8%). Further explanation of the reasons for not screening in the other category is discussed in the qualitative data analysis.

Results Depression screening instrument After a careful review of the literature [30-32] and discussion among research team members, the team selected the Short Mood and Feeling Questionnaire (SMFQ) to measure risk for mood disorders rather than a longer diagnostic instrument. An addition four questions were added: two

1383

R. John et al. / Development and Evaluation of a PDA-based Decision Support System for Pediatric Depression Screening

Sociodemographic characteristics of patients in encounters in which screening versus no screening are shown in Table 1. Almost 80% of those screened were either Hispanic or Black.

depression screening. Many of the preceptors were not familiar or desirous of initiating screening. Attitudes about depression screening. There were five related patterns centering on the theme of support that emerged during the focus groups. The students were concerned about a lack of time, knowledge, referral sources, comfort, and preceptor experience. The students expressed their need for more support from the site, preceptor, referral system, appointment system, and educational system to improve their knowledge base and comfort levels. The student felt it was inappropriate to do screening in emergency rooms or specialty clinics and wanted screening in primary care sites.

Table 1 - Sociodemographic Characteristics in Screened vs. Not Screened (n=376)

Ethnicity Hispanic Black, not Hispanic origin Asian or Pacific Islander White, not of Hispanic origin Other or unknown American Indian or Alaska Native Sex Female Male Age 8-10 11-19

% Screened

% Not Screened

63.1 16.3 2.4 17.1 1.1 0

59.7 25.8 3.2 10.5 0.8 0

51.6 48.4

52.4 47.6

27.4 72.6

11.3 88.7

There was also a use and usability theme related explicitly to the PDA-DSS. Most students felt it interfered with the therapeutic relationship by creating a barrier between the patient and the student. One student commented, “The screening did not stimulate discussion. When I am done, there is not a transitional step.” In terms of suicide screening, there was a similar theme of support. The students identified discomfort and apprehension regarding a positive screen. “You can ask the questions, but you must know what your next step is.”

Overall, 20% of the patients screened were at risk for a mood disorder with 4% having a risk for a mood disorder and suicide. In the group at risk for a mood disorder, mean age was 13.1 years and 18.2% of patients were between 810 years. This was similar to the 13.4 years in the group at risk for suicide and a mood disorder. The latter group was totally female. No one screened was positive for suicide and negative for a mood disorder.

There were also cultural concerns raised since the students raised the issue of whether or not the patients felt comfortable with the idea of sharing their feeling. One student who was in a practice of recent Chinese immigrants commented, “Talking about feelings is not acceptable in this population.” Beliefs about barriers and benefits of depression screening. There was a discrepancy about the students’ beliefs about depression screening and the barriers to depressions. Table 2 summarizes the themes identified.

Within the group at risk for mood disorder, 35.3% had a family history of depression while 50% of those at risk for a mood disorder and suicide had a positive family history. In terms of associated diagnoses, those identified as at risk of mood disorder had also had a high incidence of a psychiatric, behavioral or developmental disorders (n=44). The diagnoses with highest frequency were Attention Deficit Hyperactivity Disorder (ADHD), violence risk, and coping impairment. In the group at risk for a mood disorder, 68.2% had a behavioral or developmental disorder and only 31.8% had a diagnosis consistent with a physical disorder. However, of 467 diagnoses listed in the group not screened not screened for depression, only 6.6% had a psychiatric, behavioral, or developmental disorder diagnosis. The diagnosis with the highest frequency in the not screened group was well child (8.4%).

Table 2 - Barriers and Benefits to PDA-Based Depression Screening

Focus group

Discussion

Focus group discussions enabled exploration of students’ knowledge, attitudes and beliefs about depression, depression screening, and the PDA-DSS. The major themes extracted for each of these areas are described in the following paragraphs.

Clearly, there is a discrepancy between what the students believe are the benefits and their actual behaviors. Even though the students felt that screening should take place in a well visit, in 8.4% of the encounters in which no screening occurred, the diagnosis was a well child. In addition, only one third of eligible children or adolescents were screened. While recognizing that depression needed to be identified, the barriers overpowered the benefits. It is not clear whether the difference in the behavioral, develop-

Knowledge. Three main themes emerged during the focus groups: (1) limited, but some exposure to the problem of pediatric depression, (2) lack of a protocol within clinical site, and (3) lack of preceptor knowledge and support for

1384

R. John et al. / Development and Evaluation of a PDA-based Decision Support System for Pediatric Depression Screening

mental, psychiatric diagnoses was a result of the greater awareness of the behavioral problems brought out by the depression screening or whether these children were presented with behavioral problems, cueing the students to screen.

References [1] Wells KB, Kataoka SH, Asarnow JR. Affective disorders in children and adolescents: addressing unmet need in primary care settings. Biol Psychiatry 2001;49(12):1111-20 [2] Pfeffer CR, Conte HR, Plutchick R, Jerrett I Suicidal behavior in latency age children: an empirical study. J Am Acad Child Adolesc Psychiatry 1979;18:670-692. [3] Elliott GR Smiga S. Depression in the child and adolescent. Pediatr Clin N Am 2003;50:1093-1106. [4] Besseghini VH. Depression and suicide in children and adolescents. Ann NY Acad Sce 1997;816:94-8. [5] Lewinsohn PM, Rohde P, Seeley JR. Major depressive disorder in older adolescents: prevalence, risk factors and clinical implications. Clin Psychol Rev 1998;18(7):765-94. [6] Demler O, Jin R Koretz D, Merikangas KR, Rush AJ, Walters EE, Wang PS; National Comorbidity Survey Replication The epidemiology of major depressive disorder: Results from the National Comorbidity Survey Replication (NCS-R). JAMA 2003;289:3095-105. [7] Costello EJ, Mustillo S, Erkanli A, Angold. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry 2003;60(8):837-44 [8] Kessler RC, Berglund P, Demler O, Jim R, Merikangas KR, Walters EE. Lifetime prevalence and age of onset distributions of DSM-IV disorders in the National Comorbidity Survey Replications. Arch Gen Psychiatry 2005;62(6):593-602. [9] Pompili M, Mancineli I, Girardi P, Ruberto A & Tatrelli r. Childhood suicide: a major issue in pediatric health care.Issues Compr Pediatr Nurs 2005;28(1):63-8. [10] Lewinsohn PM, Rohde P, Seeley JR, Klein D N, Gotlib IH (2003) Psychosocial functioning of young adults who have experienced and recovered from major depressive disorder during adolescence. J Abnorm Psychol 2003;112:353-363. [11] Arborelius L, Owens MJ, Plotsky PM, Nemeroff CB. The role of CRF in depression and anxiety disorders. J Endocrinol 1999;1600:1-12. [12] Charmandari E, Kino T, Souvatzoglou E, Chrousos GP. Pediatric stress: hormonal mediators and human development. Horm Res 2003;59(4):161-79. [13] Jellinek MS, Snyder JB. Depression and suicide in children and adolescents. Pediatr Rev 1998;19:255-64. [14] Zalsman, G, Brent DA., Weersing VR. Depressive disorders in childhood and adolescence; an overview epidemiology, clinical manifestation and risk factors. Child Adolesc Psychiatric Clin N Am 2006;15:827-841. [15] Greydanus DE, Patel DR. Substance abuse in adolescents: a complex conundrum for the clinician Pediatr Clin N Am 2003; 50:1179-1223. [16] 2Lewinsohn PM, Rohde P, Seeley JR, Klein DN, Gotlib IH Psychosocial functioning of young adults who have experienced and recovered from major depressive disorder during adolescence. J Am Acad Child Adolesc Psychiatry 1999;38:56-63. [17] Pine DS, Cohen P, Gurley D, Brook J, Ma Y. The risk for early adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders 1998;34:566-578. [18] Apter A King RA. Management of the epressed, suicidal child or adolescent. Child Adolesc Psychiatric Clin N Am 2006;15:999-1013. [19] Eaton DK, Kann L, Kinchen S, Ross J, Hawkins J, Harris WA, Lowry, R, McManus R, Chyen D, Shanklin S, Lim C, Grunbaum J,Wechsler H. Youth risk behavior surveillance United States, 2005. MMWR. 2006 Jun 15; / 55(SS05):1-

This study has educational, clinical, and research implications. First, the students need to be empowered to intervene in children at risk for a mood disorder. Although teaching and counseling interventions are included in the PDA-DSS, students need further education about developing a therapeutic relationship and intervening with a single encounter. Educating preceptors about depression screening and interventions in a variety of settings may also promote screening. The students need to improve their use of the PDA in clinical settings through strategies such as sharing the PDA screening with patients and entering initial information before entering the room. Further research is needed to explore the patient’s reactions to the PDA-based depression screening. It would also be important to compare different cultural group’s reaction to the depression screening tools in paper versus PDA format. Further research is needed to assess whether the identification of depression helps students to identify other behavioral, developmental or psychiatric issues.

Conclusion The data support that risk for mood disorder is fairly common in the population of primarily Hispanic and Black children and adolescents screened using the PDA-DSS and that there are missed opportunities for depression screening in a variety of settings. There are site, preceptor/ student, and usability issues preventing depression screening. The organizational challenges are particularly difficult because APN students practice in different clinical sites with different resources and protocols. Since pediatric depression screening is relatively new, even experienced preceptors may lack the education to be effective primary contacts for the screening and identification of children at risk for a mood disorder or suicide. Informatics innovations cannot be effectively implemented in practice without understanding the context of use. The triangulation of quantitative and qualitative methods was useful in helping the research team understand the screening behavior of students using the PDA-DSS and the associated organizational barriers in clinical settings. Moreover, these data provide direction for curricular changes and communication with preceptors regarding the use of the PDA-DSS. Acknowledgments This research is sponsored by a grant from the National Institute for Nursing Research (R01 NR008903). The authors thank the other members of the Mobile Decision Support for Advanced Practice Nursing for their advice and assistance. The authors also acknowledge the assistance of Dr. S. Ledlie in the design of the interview guide.

1385

R. John et al. / Development and Evaluation of a PDA-based Decision Support System for Pediatric Depression Screening

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

108 Available from http://www.cdc.gov/mmwr/preview/ mmwrhtml/ss5505a1.htm Pfeffer CR, Klerman GL, Hurt SW, Leser M, Peskin JR & Siefker CA. Suicidal children grow up: Demographic and clinical risk factors for adolescent suicide attempts. J Am Acad Child Adolesc Psychiatry 1991;30:606-616. Pfeffer CR, Klerman GL, Hurt SW, Kakuma T, Peskin JR, Siefker CA Suicidal children grow up: rates and psychosocial risk factors for suicide attempts during followup. J Am Acad Child Adolesc Psychiatry 1993;32:106-113. Kelleher KJ, Campo JV, Gardner WP. Management of pediatric mental disorders in primary care; where are we now and where are we going? Curr Opin Pediatr 2006;18:649-653. Weitzman CC, Leventhal JM. Screening for behavioral health problems in primary care. Curr Opin Pediatr 2006; 18:641-648. Williams J, Klinepeter K, Palmes G, Pulley A, Foy, JM. Diagnosis and treatment of behavioral health disorders in pediatric practice. Pediatrics. 2004; 114:601-606. Sayal K, Taylor E. Detection of child mental health disorders by general practitioners. Br J Gen Pract 2004;54:348-352 Brugman E, Reijneveld SA Verhulst f, Verloove-Vanhorick SP. Identification and management of psychosocial problems by preventive child healthcare. Arch Pediatr Adolesc Med 2001;155:462-469. Reijneveld SA, Brugman E, Verhulst F, Verloove-Vanhorick SP. Identification and management of psychosocial problems among toddlers in Dutch preventive childhealthcare. Arch Pediatr Adolesc Med 2004; 158:811-817. Simonian S, Tamowsi K. Utility of the Pediatric symptom checklist for behavioral screening of disadvaged children. Child Psychiatry Hum Dev 2001;31:269-278.

[29] Haughey MT, Claderon Y, Torres S., Nazario, S, Bijur P, Identification of depression in an inner city population using a simple screen. Acad Emerg Med 2005;12(12):1221-1226. [30] Dahlberg LL, Toal SB, Swahn M Behrens CB. Measuring violence-rated attitudes, behaviors, and influences among youths: a compendium of assessment tools, 2nd ed., Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, 2005. Available from http://www.cdc.gov/ncipc/pub-res/pdf/YV/ YV_Compendium.pdf [31] Myer K, Winters N Ten-year review of rating scales. II: Scales for internalizing disorders. J Am Acad Child Adolesc Psychiatry. 2002;41(6):634-659. [32] Angold A, Costello E, Messer S. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int J Meth Psychiatric Res 1995;5:237-249. [33] McAlearney AS, Schweikhart SB, Medow MA. Doctor’s experience with handheld computers in clinical practice: qualitative study. BMJ. 2004; May 15;328(7449):1162 [34] McAlearney AS, Schweikhart SB, Medow MA. Organizational and physician perspectives about facilitating handheld computer use in clinical practice: results of a cross-site qualitative study. J Am Med Inform Assoc2005;12:568-575.

Address for correspondence Ritamarie John, DrNP, CPNP Columbia University Mailbox 6 New York, NY 10032 Telephone: 1-212-305-5542 Fax: 1-212-305-6937 e-mail: [email protected]

1386

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Data Mining Results from an Electronic Clinical Log for Nurse Practitioner Students Patricia Trangensteina, Elizabeth Weinera, Jeffry Gordona, Ryan McNewa a

Frist Nursing Informatics Center, School of Nursing, Vanderbilt University, Nashville, TN, USA

dent, or across students for a given semester or program because of the non-digital nature of the media Furthermore, it was difficult for students to examine their practice over time and for faculty to create reports and track student’s progress.

Abstract Traditional techniques for collecting data on clinical experiences have been greatly flawed. Data cannot be easily collected in real time to make programmatic or placement changes “on the fly”. Furthermore, it is difficult to look at data across students, specialty areas, and years because the data is typically not in a digital format. In response to this problem, the Vanderbilt University School of Nursing has created a web/PDA based clinical log to document the kinds of clinical experiences the students are having. Since our initial report, three years ago, we have collected three years worth of data, over 220,000 different patient encounters. This past year the data has been very complete, giving a full picture of the types of experiences the students are having. Our faculty have begun to analyze the data in the clinical log to examine the kind of experiences the students are having and to make programmatic changes and placement adjustments in real time. In general, the results supported that students in the various specialties managed patients and performed services appropriate to their specialty. Patients varied in ages, ethnic groups, payment sources, and medical diagnoses. Students did progress from an observer role to a more independent role in either a linear fashion or in a more biphasic mode with an increase in the observer role at the start of a new semester

A real-time method of data collection and analysis would allow for changes during, rather than after, the semester of clinical practicum. Faculty also thought that it was appropriate to map current evaluation methods against competencies being developed by the various specialty groups as well as the National Organization of Nurse Practitioner Faculty (NONPF). In short, the faculty wanted to re-design their clinical evaluation tool for the students in their specialty while preserving the ability to make comparisons across specialties in an aggregate format for purposes of accreditation. A review of the literature revealed a scarcity of information with only one article describing the content and quality of a precepted clinical experience for nurse practitioner students. [1] As a result, the informatics team was invited to work with the specialty directors in the design, implementation, and evaluation of an online web based clinical log with PDA data input capabilities.

Methods

Keywords:

In 2004 VUSN began implementing an electronic (web/ PDA based) clinical log for nurse practitioner students. By 2006, the clinical log had been used by 200 students per year in seven different nurse practitioner programs for the entire length of their clinical experience. This article presents the results from the analysis of over 89,000 records collected during this past academic year. The implications for nurse practitioner education, curriculum revision, and student clinical portfolios for future employment are substantial. Not only was the log data used for program evaluation through data mining techniques, the students also used the log to create an electronic portfolio of their experiences and have, in some cases, secured employment as a direct result of their log entries.

education, nursing, graduate; nurse practitioners professional practice; students, nursing; computers, handheld/utilization; education, nursing, graduate/ methods; nursing education research; preceptorship

Introduction Traditional clinical education of nurse practitioner (NP) students has used preceptors for direct clinical supervision with faculty responsible for student progression, student evaluation and attainment of course objectives as well as curriculum revision. Nursing faculty collected data from nurse practitioner students to document their clinical experiences via on-site evaluation, discussions with the preceptor, paper logs and scan sheets completed at selected intervals during the course of study. These models led to a significant delay in receiving the information from students and providing appropriate feedback. In addition, it was very difficult to aggregate this data for a given stu-

Description of the system The electronic clinical log was initially designed for data entry through the web. However, students raised the issue that they were entering data twice (once on paper patient side and again that evening as they transferred the data to

1387

P. Trangenstein et al. / Data Mining Results from an Electronic Clinical Log for Nurse Practitioner Students

the log) a PDA interface was designed in response to their concerns. Students were instructed to synch their PDA with their desktops every night to avoid data loss. They were further instructed to upload their data once each week to the server based database. The records in the database were accessible via a web browser by students, faculty, and preceptors. The students could see only their own data, faculty and preceptors could see their students’ data. Data collected included patient’s age, gender and ethnicity; type of insurance, services rendered (professionally recognized criteria designed by the National Organization of Nurse Practitioner Faculty (NONPF)), ICD9 or DSM-IV codes and self assessment of the student’s responsibility in patient management. Students and faculty could view the records and export logs to a spreadsheet for aggregation and graphing. Because of the unique nature of the Psychiatric/Mental Health Nurse Practitioner program, the PDA component for this specialty will be discontinued. Instead, next year, the PDA will be used to actually audio record the patient encounter, uploaded to the log, and made available for listening and comment by the faculty member or preceptor.

centage of males versus females except for Family Nurse Practitioner (FNP) and Psychiatric/Mental Health Nurse Practitioner (PMHNP) students who saw almost twice as many females as males, and of course the Nurse Midwifery students saw only female patients. While the nurse practitioner students in all programs cared for many ethnic groups, the majority of the patients were Caucasian (49 – 84 percent across all programs), African Americans (11-24 percent across all programs) or Hispanic (1.5 – 24 percent across all programs). The majority of patient seen by Adult Nurse Practitioner (ANP) and Acute Care Nurse Practitioner (ACNP) students used either Medicare or private insurance as their source of payment, while the majority of patients seen by Family Nurse Practitioner (FNP), Nurse Midwifery, Pediatric Nurse Practitioner (PNP) and Pediatric Acute Care Nurse Practitioner (PNPAC) students had private insurance or TennCare (replacement for the state of Tennessee’s Medicaid program). The log allowed the students to enter up to four ICD9 codes per record. DSM codes were supported for the PMHNP students. Tables 1-3 describe the top ranking ICD9 or DSM-IV codes for student encounters across programs. While there were some similarities, the data revealed that students were caring for patients that were appropriate for their specialty. For example, Acute Care Nurse Practitioner (ACNP) students managed more CHF patients, and a significant portion of the Family Nurse Practitioner (FNP) students encounters were routine infant or child health checks.

All electronic clinical logs (N = 114,206) collected during the entire year of study were subjected to data cleaning prior to analysis. If a student did not have data for the entire program of study all their records were eliminated in this analysis. After data cleaning a total of 89,401client encounters for 200 students remained for the purpose of this study. SPSS was used for all an analyses.

Results Each time a student interacted with a patient it was treated as a single encounter. Therefore, if the student saw the same patient multiple times, as is common in the Psychiatric/Mental Health Nurse Practitioner specialty, each visit counted as one encounter.

Table 1 - Rank order of ICD-9 Codes for ACNP, ANP and FNP ICD-9

The total number of patient encounters per nurse practitioner program was astounding and ranged from 2,072 for Pediatric Acute Care Nurse Practitioner students (PNPAC) to 34,700 for Family Nurse Practitioner (FNP). On average, the data showed there were 10 cases per week for each Acute Care Nurse Practitioner student and 21.4 cases per week for each Family Nurse Practitioner student. The mean number of encounters per student ranged from a low of 250 for the entire year for Acute Care Nurse Practitioner students to a high of 550 for Nurse Midwifery students. It should be noted that most of the specialties have an 8 month clinical rotation; however, the Nurse Midwifery program had a 13 month clinical experience. In general, Acute Care Nurse Practitioner, Pediatric Acute Care Nurse Practitioner and Psychiatric/Mental Health Nurse Practitioner students had fewer patients but longer encounters.

ANP

ACN P

FNP

250, Diabetes

1

2

3

401, Essential HTN

2

5

401.1, Benign essential HTN

3

1

272.4, Other hyperlipidemia,

4

3

272, Disorders of lipoid metabolism

5

428, Heart failure,

Acute Care Nurse Practitioner students and Adult Nurse Practitioner students (ANP) students saw older patients with a mean age of 55 while students in the Family Nurse Practitioner (FNP), Nurse Midwifery and Psychiatric/ Mental Health Nurse Practitioner (PMHNP) programs saw middle aged patients (average age = mid 20’s to early 30’s). Students in all programs saw an almost equal per-

1388

1

4

401.9, Unspecified essential HTN

2

V20.2, Rout infant or child health check

4

477.9, 461.9 Allergic rhinitis or sinusitis, 465.9, Acute URI

5

P. Trangenstein et al. / Data Mining Results from an Electronic Clinical Log for Nurse Practitioner Students

Table 2 - Rank order of ICD-9 Codes for NMW and PNP ICD-9 Codes

NM W

Table 4 – Types of service rendered as a percentage of encounters for acute care specialties

PNP Services Rendered

ACNP

PNPAC

V22.1, Supervision of other normal pregnancy

1

ID & documented actual pt prob

71.9

68.3 53.1

2

Analyzed all pharmacological agents

68.9

V22.2, Pregnant state, incidental

Collaborated with others

65.4

69.6

Obtained a H&P

63.8

58.9

ID & documented potential pt prob

57.7

63.9

Provided pt and family education

56.0

79.5

Developed an individualized Rx plan

54.4

69.5

Documented H&P

52.5

58.5

Rank Order

ID expected outcomes ind’l to the pt

51.2

59.2

296.9Mood disorder

1

Developed & documented list of differential diagnoses

48.5

57.6

309.81PTSD

2 Utilized evidence based practice

47.7

52.4

V20.2, Routine infant or child health check

1

465.9, Acute upper respiratory infections of unspecified site

2

382.9, Unspecified otitis media

3

460, Acute nasopharyngitis [common cold]

4

Table 3: Rank order of DSM-IV codes for PMHNP students DSM-IV Codes

314.01 ADHD combined subtype

3.5

296.33 Major depression without Psychotic features

3.5

296.8Bipolar

5

304.8 Polysubstance Dependence

6

Documented ongoing eval of Rx plan

55.2

Table 5 – Types of service rendered as a percentage of encounters for primary care specialties

The faculty members were also interested in the types of services their students provided to their patients. The services rendered were derived from the NONPF content competencies and were customized by specialty. Only those programs with some commonality with other specialties were reported here. Acute Care students (ACNP and PNPAC) provided an average of 10 services per encounter while primary care student provided an average of 3 services per encounter. Tables 4 and 5 describe the highest ranking services provided as a percentage of the encounters. For example, Acute Care Nurse Practitioner

Services Rendered

ANP

FNP

PNP

Focused/episodic exam

84.6

85.7

68.3

Prescription

66.0

73.3

37.7

Health Education

47.6

71.9

55.7

Labs

45.0

35.6

18.5

Complete H&P

34.1

Developmental screening

25.2

The faculty members also wanted to evaluate the level of a student’s responsibility in patient management, increasing from observational roles in early clinical experiences to more independent roles later in their clinical experiences. The student responsibility ranged from Observer, to Novice, to Beginner, to Advanced Beginner and had descriptors associated with each level for reliability purposes. Most of the students’ clinical experiences transpired over two semesters. In order to determine if there was a natural progression from observer to more independent roles, the date of the encounter was converted to the month of the year indicating a progression in clinical experiences.

(ACNP) students identified and documented actual patient problems almost 72% of the time while Pediatric Acute Care Nurse Practitioner (PNPAC) students provided patient and family education nearly 80% of the time.

1389

P. Trangenstein et al. / Data Mining Results from an Electronic Clinical Log for Nurse Practitioner Students

Correlations between students’ responsibilities and the month were statistically significant for all of the programs at the .001 level. The correlations ranged from .192 for the Pediatric Nurse Practitioner (PNP) students to .396 for the Acute Care Nurse Practitioner (ACNP) students indicating that students did indeed progress from observer to advanced beginner during their clinical experiences.

Discussion NP faculty members have the ultimate responsibility for the supervision and evaluation of nurse practitioner students and for oversight of the clinical learning environment. [2] It is their responsibility to develop, evaluate and revise NP curricula. Students wish to secure the most favorable position possible and wish to make their case for such placement using data and evidence collect through the log. The NP faculty at the Vanderbilt University School of Nursing (VUSN) in conjunction with members of the Frist Nursing Informatics Center and with significant student input, developed electronic clinical logs (ECLS) for seven different nurse practitioner programs. Data presented represent a full program of clinical experiences for the students.

While the correlations were statistically significant, they were low, indicating significant variability. In order to visualize where the variability existed histograms were constructed for each of the specialties. Figures 1 and 2 indicate two different paths from observer to advanced beginner that were noted.

In general, the results supported that students in the various specialties managed patients and performed services appropriate to their specialty. Patients varied in ages, ethnic groups, payment sources, and medical diagnoses. Students did progress from an observer role to a more independent role in either a linear fashion or in a more biphasic mode with an increase in the observer role at the start of a new semester. Figure 1 - Histogram of FNP students’ responsibility by month of clinical experience

Conclusions This article demonstrated the effectiveness of an electronic clinical log (ECL) for documenting the content and quality of a precepted clinical experience for NP students. Students entered data either by uploading from their PDA or by completing a web-based form. The records in the database were accessible via the internet by students, faculty, and preceptors. Data collected included patient’s age, gender and ethnicity; type of insurance, services rendered (professionally recognized criteria designed by the National Organization of Nurse Practitioner Faculty (NONPF), ICD9 or DSM-IV codes and self assessment of the student’s responsibility in patient management. Students and faculty could view the records and export logs to a spreadsheet for aggregation and graphing. Using informatics tools over 89,000 patient encounters for 200 NP students were analyzed.

Figure 2 - Histogram of PNPAC students’ responsibility by month of clinical experience

Proposed revisions of the electronic clinical log (ECL) would include the creation of report dashboards to allow faculty members to determine “on the fly” whether the clinical experience was providing the right environment for the student. This would allow the faculty member to discuss with a preceptor the type of experiences the student was participating in as well as the quality of work the student was demonstrating. Currently a running total of the number of clinical hours has been well received by both students and faculty as a first piece of a dashboard. Given the volume of patient encounters the entire dashboard process would need to be automated with inadequacy flags automatically sent to the faculty member. We also plan on exploring the ICD9 data in detail to generate a culleddown list that would be more manageable for the students. We believe this shorter list will be far more useful to the

As can be seen from these two figures, the Family Nurse Practitioner (FNP) students showed a more orderly progression from observer to advanced beginner. On the other hand, the Pediatric Acute Care Nurse Practitioner students (PNPAC) students showed a more biphasic progression with an increase in the observer role in the fifth month. This corresponded to the beginning of a new semester and may reflect a change in preceptors, sites or performance expectations with the introduction of new skills. In any case, the faculty of the Pediatric Acute Care Nurse Practitioner students (PNPAC) program will look at the data to see if this trend continues and needs to be addressed or if it was an anomaly.

1390

P. Trangenstein et al. / Data Mining Results from an Electronic Clinical Log for Nurse Practitioner Students

students who won’t have to query through over 14,000 ICD-9 different codes.

References [1] Longworth, JCD, and Lesh, D. Development of a student nurse practitioner computerized clinical log. J. of the American Academy of Nurse Practitioners 2000:12 (4): 117-122. [2] National Organization of Nurse Practitioner Faculties.

Students have used the data to create a “clinical portfolio” listing their skills and activities across all of their clinical practice sites in their educational program and have analyzed their data using Excel. These advanced practice nurses will be participating in research and data collection/ analysis throughout their careers. A major benefit of our approach was that these students learned how to use Excel and demonstrated spreadsheet analysis techniques. The faculty were pleased that their students were as proficient with Excel as they are with Word.

Criteria for the evaluation of nurse practitioner programs: A report of the national task force on quality nurse practitioner education. National Organization of Nurse Practitioner Faculties, 2002. Address for correspondence [email protected]

An ECL will allow faculty to individualize learning based on the identification of gaps in the student’s clinical experience. Faculty members can track the student’s progress and types of patients seen. Data from the ECL can also be used to modify the curricula and provide documentation for grants and accreditation.

1391

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Phase I Implementation of an Academic Medical Record for Integrating Information Management Competencies into a Nursing Curriculum Carole A. Gassert, PhD, RNa, Katherine A. Sward, MS, RNa a

University of Utah School of Nursing, Salt Lake City, UT, U.S.A.

mendations by federal agencies that informatics content needs to be included in nursing curricula, information and information technology competencies have been slow to become part of the nursing curriculum [8] and many nurses may not be adequately prepared to manage information using technology [4, 9, 10]. A recent study of university students found that students’ self-reports of information management activities were not an accurate predictor of their actual health information competencies [11]. Information management competencies should be assessed by observing actual use of an information system. Education regarding health information management should be conducted in the context of “real-world” applications and behaviors; that is, in the information environment where clinicians work, and should incorporate concepts such as confidentiality, systems thinking, and knowledge-resource evaluation [12].

Abstract This paper is the report of the first phase of a case study from the University of Utah to help students and faculty integrate electronic information management into the nursing curriculum. Cerner AES, a live-production clinical information system with an academic overlay, has been implemented into the first semester of an undergraduate nursing program. A consortium of schools that use Cerner AES collaborate in the design and implementation of forms used by students. The consortium also allows members to share strategies for using the system. By using the system students are developing needed informatics competencies for beginning level nurses. The paper discusses the implementation strategies used and initial results of this project. Plans for expanding the project throughout the nursing curriculum are also presented. Keywords:

One initiative that aims to increase the use of technology in nursing is the TIGER (Technology Informatics Guiding Education Reform) initiative [13]. This initiative, spearheaded by nursing and informatics leaders in the United States, has developed a 3-year plan to integrate informatics seamlessly into nursing.

electronic health record, nursing education, nursing informatics competencies, clinical information system, simulation

Introduction

Materials and methods

Sustainable development can be defined as "development that meets the needs of the present without compromising the ability of future generations to meet their own needs" [1]. This concept has been adapted in the context of health care to define the capacity of a health system to continue over time without major interruptions [2].

This paper describes the first phase of a case report at the University of Utah to incorporate information management competencies throughout the nursing curriculum by using a live-production application that simulates a clinical information system titled UCARE AES. The acronym stands for Utah Clinical Academic Record Excellence and is the name of the Cerner Academic Education Solution system installed for use in first semester undergraduate courses in May 2006. This on-going project is used to introduce curricular changes necessary to ensure that nursing students are adequately prepared to contribute to sustainable health systems. UCARE AES enables faculty to address many of the informatics competencies needed for beginning nurses identified by Staggers, Gassert, and Curran in their work. Examples of beginning level competencies that are taught in first semester undergraduate courses using UCARE AES are presented in Table 1.

Basic resources of sustainable health systems include data and knowledge, and the skills of the health workforce. Improving the safety, quality, and efficiency of health care systems requires ubiquitous access to complete patient information and decision support—that is, electronic health systems [3]. Information has become a capital good [4]. Nurses and other health professionals need skills that will allow them to more effectively manage information technologies. These skills must be learned from the beginning, during professional education [3, 5]. Staggers, Gassert & Curran [6, 7] identified information management competencies for nurses at multiple levels of practice. Despite recom-

1392

C. A. Gassert et al. / Implementation of an Academic Medical Record for Integrating Information Management Competencies

Table 1 - Examples of beginning level competencies obtained through UCARE AES Competency

Activity Example

Uses administrative applications for practice management

Searches for patient

Uses sources of data that relate to practice and care

Locate “patient data” in UCARE AES

Accesses, enters, and retrieves data for patient care

Charts on class activities in UCARE AES

Uses an application to document patient care

Uses UCARE AES to document results of class activities and simulated patients

Uses an application to plan care for patients to include discharge planning

Uses UCARE AES to document plan of care

Uses networks to navigate systems Describes patients’ rights as they pertain to computerized information management

Uses UCARE AES (hosted remotely)

Identifies basic components of the current information system

Knows components of UCARE AES

for teaching the first level concepts of nursing, patient assessment and clinical care courses met, in spite of weekly snow storms, with the undergraduate program director and two informatics faculty in charge of the Cerner project to form an implementation team. The weekly meetings were held in a room that allowed access to and visualization of Cerner AES with an overhead projector. Minutes recorded decisions made by the team for later reference. The first task was for the entire implementation team to learn about AES and plan the structure for Utah. The team selected the name UCARE AES for the system and Swoopes Medical Center for our username. Units in the medical center were named for the focus of the semester, e.g., medical-surgical 1, medical-surgical 2, pediatrics, maternity, etc. and are created to accommodate the number of clinical groups assigned to the semester. For example, there are nine clinical groups in first semester so units Medical-surgical 1A through Medical-surgical 1I were developed. Students are then admitted as patients to their assigned “unit” and given a password to access the system.

Retrieves demographics

Creates plans of care in UCARE AES

To help students and faculty become more familiar with UCARE AES a number of “play patients” were admitted to the Utah Start Unit. Nicknamed the “Olympians” because their fictitious names reflect Olympic events, UCARE AES users were encouraged to chart anything on these patients, whether it made sense medically or not. Case studies for student use during the semester were created and admitted to a folder titled “UCARE 1.” These patients were “off limits” for charting. When the semester began students located information needed for their assignments from the case studies, they charted the information requested on themselves as patients. To help students learn the principles of data security and confidentiality, they are warned that faculty could track where they have been on the system and if they are looking at fellow students charting.

Learns HIPAA regulations through UCARE AES

Cerner academic education solution An academic version of an electronic medical record is available through Cerner Academic Education Solution (AES), an application that simulates a clinical information system using Cerner’s PowerChart. AES has an academic overlay that provides students with prompts and evidencebased practice information as they learn to document assessments and simulated patient events.

Forms needed by students to complete their assignments were moved from the large repository of forms maintained by Cerner to specific folders designated by faculty in the first semester. They chose to have 5 folders – assessment, patient care, plan of care, clinical prep, and competencies. Most activities required forms from the first three folders. To make it easier for students to find needed forms, the Course Coordinator for the clinical course in first semester included instructions in her course syllabus. Examples are listed here:

AES was first introduced at Kansas University [14, 15] and later at University of Missouri-Kansas City. With the introduction of UCARE AES at University of Utah, the three schools have formed a consortium to share design and management ideas and issues about AES. The consortium is a great forum for exploring new ways to use AES. Although the three schools share a domain server and databases that are hosted at Cerner headquarters in Kansas City, each school has a unique username and established structure. Many forms are shared as each school contributes their wisdom to the AES development process. All three schools are evaluating student outcomes. Monthly virtual meetings, held to discuss ideas and issues, are chaired by Cerner personnel.

• •

•

UCARE AES implementation It is interesting that each of the schools in the consortium has implemented the AES system differently. At the University of Utah planning for the phased implementation began in January 2006. Undergraduate faculty responsible

•

1393

Document the following on UCARE AES Vital signs: Located in Assessment Folder/Adult Vital Signs Results of incentive spirometer use: Located in Assessment Folder/Adult Respiratory Assessment/Incentive Spirometer (Left margin) Pulse oximeter results: Located in Patient Care Folder/ Adult Vital Signs Teaching patient to TCDB: Located in Patient Care Folder/Adult Vital Sign/Oxygen Therapy/Document under “other.”

C. A. Gassert et al. / Implementation of an Academic Medical Record for Integrating Information Management Competencies

•

Use of O2: Located in Patient Care Folder/Adult Vital Signs To allow the system built to be completed by Cerner in a timely manner, all design development was completed by mid-April for the May 15th go-live. First semester students and faculty were oriented to UCARE AES on the go-live day. Base-line data for first semester students’ experience with and knowledge about information systems was collected at their orientation.

plaints about documenting on forms instead of free-text notes, and did not recognize the importance of teaching information management concepts, as reflected in comments such as "UCARE does not match the charting used at my clinical site". Other faculty recognized UCARE as a tool to teach information management, and commented that it was "easy to learn" and "an effective way to teach and learn terminology." Our observations of nursing students concur with findings that student self-report of skills is likely not accurate. Incoming students rate their technology skills and knowledge as being low, and indicate they do not know how to find and manage information in an electronic health record, yet rate their information management skills as moderate. After one semester of use, the students reported higher technology skills and knowledge, and greater ability to find and manage information in an electronic record. However, these same first semester students also indicate that they want more guidance and direction with using the system.

The system was implemented with accelerated baccalaureate students (students who hold a baccalaureate degree in another field and wish to become nurses) on their first day of clinical classes. Implementation team members and Cerner representatives were available during the first two days of clinical classes to help students sign onto the system and chart information required for class. The only problems encountered during the go-live were related to student IDs entered into the system, all problems were quickly resolved. Assistance was available in the Computer Lab to work with faculty and students as needed. During the planning phase, some faculty became champions for UCARE AES and were an essential part of the successful implementation of the system. It is also important to note that we had the full support of the Dean of the College of Nursing for implementing UCARE AES.

The students were aware that their faculty were not yet comfortable with the system. Student comments were mixed, ranging from "busy work", "I don't get the point" and "it did not match the system at my clinical site"; to comments such as "Even though my clinical site has a different setup for their electronic records it still helped me" and "It was difficult at first to use not because of the system but because of my lack of nursing knowledge"; to comments such as "It was very cool to know about before starting clinical and to be able to say I know what to do and how to get on and navigate the system."

Since the implementation team is quite small, a decision was made to implement UCARE AES one semester at a time. As soon as the go-live was completed, second semester faculty responsible for clinical courses were asked to join the implementation team to plan for Phase II of the implementation. The plan is to have all four undergraduate semesters implemented by May 2007. Therefore, UCARE AES will have been implemented in the entire undergraduate curriculum in 13-14 months, an ambitious plan.

Implementation status The UCARE AES is an ongoing project that will eventually include undergraduate and graduate nursing, medicine and pharmacy students. Nursing is responsible for the initial design and has begun implementation at the undergraduate level. We recognize that repeated exposure is necessary to achieve a level of comfort with information systems and believe it will be crucial for both faculty and staff to continue regular interactions with the system; therefore we are incorporating use of the system into every semester of our undergraduate curriculum before moving into the graduate program with UCARE AES.

Results Phase I student and faculty response During the planning for implementing Phase I of UCARE AES, first semester faculty decided to tie the learning activities of the three courses more closely together. Timing of content was adjusted so students were learning the same concepts in all three classes. Faculty also began to standardize language used to teach concepts to students. During the design work, only one new form was created and one form was revised. Consortium members adopted the form that was developed for Utah. It seems that schools can use forms designed for others and such action will standardize some of the language used to teach concepts to undergraduate students across schools of nursing.

To date, it has been implemented into the curriculum for 3 (of 4) semesters of the undergraduate nursing program, although only Phase I is the focus of this paper. First semester students use UCARE AES to learn how to document individual “patient” assessments. Second semester students use the system to locate data in a medical record, synthesize information from multiple sources, and make clinical decisions based on the synthesized data. Third semester students currently use the system for pediatrics and maternity nursing. In pediatrics students assess scenarios on patient simulators (manikins) and obtain the “patient’s” history from UCARE AES. In maternity students use UCARE AES to review a case study, prepare care plans, enter nursing orders, and practice reading fetal monitoring strips. All third semester students use a form

Data were collected at the beginning and end of the students’ first semester. Data were also collected from the faculty at the end of the first semester. Specific data will be reported as aggregated data when more phases of the project have been completed. In general, faculty responses were mixed after the first semester of use; faculty reported feeling only moderately comfortable with how to use the UCARE system. Some faculty did not appear to grasp the purpose of learning to use an electronic record; with com-

1394

C. A. Gassert et al. / Implementation of an Academic Medical Record for Integrating Information Management Competencies

called SBAR (situation, background, assessment, and recommendation) to practice communication with nurse and physician providers. The implementation team is presently meeting with fourth semester faculty and mapping their curriculum to UCARE AES. As stated, all semesters of the undergraduate program will be live by May 2007.

[2] Madani S and Aronsky D. Factors Affecting the Sustainability of Information Technology Applications in Health Care. AMIA Annual Symposium Proceedings, 2003: 922. Available online at http://www.pubmedcentral.nih.gov/ articlerender.fcgi?artid=1479905 [3] Yasnoff WA, Humphreys BL, Overhage JM, Detmer DE, Brennan PF, Morris RW, Middleton B, Bates DW, and Fanning JP. Consensus Action Agenda for Achieving the National Health Information Infrastructure. J Am Med Inform Assoc. 2004;11:332–338. [4] Gassert CA. The Challenge of Meeting Patients' Needs with a National Nursing Informatics Agenda. J Am Med Inform Assoc. 1998; 5(3): 263-268. [5] Kinder G, Cashman SB, Seifer SD, Inouye A and Hagopian A. Integrating Healthy Communities concepts into health professions training. Public Health Rep. 2000 Mar–Jun; 115(2-3): 266–270. [6] Staggers, N, Gassert, CA, and Curran, C. Informatics Competencies for Nurses at Four Levels of Practice. J of Nurs Educ. 2001; 40(7): 303-316. [7] Staggers, N, Gassert, CA, and Curran, C. A Delphi Study to Determine Informatics Competencies for Nurses at Four Levels of Practice. Nurs Research. 2002; 51(6): 383-390. [8] Ornes, LL and Gassert, CA. Computer Competencies in a BSN Program. J of Nurs Educ. 2007; 46(2): 75-78. [9] Gugerty, B. Nursing at a Crossroads – Education, research, training, and informatics. J of Healthcare Information Management, 2007, 21(1): 12-14. [10] McDowell, DE and Xiping, MA. Computer literacy in baccalaureate nursing students during the last 8 years. CIN, 2007; 25(1): 30-36. [11] Ivanitskaya L, O'Boyle I, and Casey AM. Health Information Literacy and Competencies of Information Age Students: Results From the Interactive Online Research Readiness Self-Assessment (RRSA). Med Internet Res. 2006 Apr–Jun; 8(2): e6. Published online 2006 April 21. doi: 10.2196/jmir.8.2.e6 at http://www.pubmedcentral.nih.gov/ articlerender.fcgi?tool=pmcentrez&rendertype= abstract&artid=1550696 [12] Staggers N, Gassert CA, and Skiba DJ. Health Professionals' Views of Informatics Education: Findings from the AMIA 1999 Spring Conference.J Am Med Inform Assoc. 2000; 7(6): 550-558. [13] TIGER. The TIGER Initiative: Evidence and informatics transforming nursing: 3-year action steps toward a 10-year vision. Available online March 20, 2007 at http:// www.tigersummit.com. [14] Warren, JJ, Connors, HR, and Weaver, C. Clinical Information Systems and Critical Thinking: Outcomes of an Innovative Education Strategy. NI2003 Proceedings. 2003. [15] Warren, JJ, Connors, HR, Weaver, C. and Simpson, R. Teaching Undergraduate Nursing Students Critical Thinking: An Innovative Informatics Strategy. NI 2006 Proceedings. 2006: 261-265.

Acute care nurse practitioner faculty are anxious to incorporate UCARE into their curriculum. They will be added as soon as the baccalaureate curriculum implementation is complete. For the nurse practitioner students, curricular content will be matched to the “experienced nurse” competencies of Staggers, Gassert, and Curran. The School of Pharmacy has already expressed a desire to be added to UCARE AES. The discussions with pharmacy are pointing to the use of collaborative and interdisciplinary case studies that focus on pharmacotherapeutics. Faculty are becoming more comfortable using the system as their experience increases. In general, faculty are beginning to recognize the value of teaching electronic information management concepts and are continually verbalizing how they think they can expand the use of UCARE in their curriculum. To help faculty adopt more of their learning activities to UCARE AES, we anticipate adding an informatics student in the role of a teaching assistant (TA) to the implementation team to work with clinical teaching staff to assist them in using the system. The TA will also be available to work with students who may be uncomfortable using the system. Midway through the first semester we added a Systems Analysis position to the team. That individual helps to interface with Cerner to resolve any problems that arise and to request needed changes to the system.

Conclusions Since UCARE AES has been added to the undergraduate curriculum it is clear that students are gaining competence in using information technology and in doing electronic data management. This will help with the development of sustainable health information systems. In addition undergraduate students are becoming aware of nursing informatics as a field and some have expressed interest in specialization in this area at the graduate level. We look forward to being able to report data that is being collected throughout the project as the remaining phases of the implementation are completed.

References [1] WCED: UN World Commission on Environment and Development. Our Common Future. Oxford: Oxford University Press, 1987. Available online at http:// www.are.admin.ch/are/en/nachhaltig/ international_uno/unterseite02330/

Address for correspondence Carole Gassert can be contacted at [email protected]

1395

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

ROC van Twente: Nursing Education in Care and Technology William TF Goossena,c Anneke TM Goossen-Baremansb, Laura Hoftec, Bert de Kreyc b

a Results 4 Care, Amersfoort, the Netherlands Acquest Research, Development and Consulting, Koudekerk aan den Rijn, The Netherlands c ROC van Twente, Enschede

standards. Thus, new kinds of healthcare emerge and for health care professionals there is an imperative to make sensible decisions about what technology to introduce in the care environment. The integration of technologies and the changes in health care delivery depend more and more on standardisation and quality assurance.

Abstract The ROC van Twente offers nursing education at the diploma level (MBO), and is innovating the program to include a major /minor structure for education about care and technology. In order to achieve this, a new position was created: the Master Docent, Care and Technology. The task of the master docent includes development of education for nursing about technology, multidisciplinary cooperation, and service to health care institutions among others. The first development concerns a module about electronic patient records, standards, and semantic interoperability for continuity of care. The module is delivered to nursing students and to students from the information technology department, who work jointly in ‘development teams’. This paper describes the background, the development of the educational material and program, and the core content of the module. The core content are the care information models that link clinical materials with health care information standards. The program has started end November 2006. At the Medinfo 2007 conference the results of the course for the first group of about 40 students will be presented.

This paper discusses how this approach to development of electronic messages and EPR is chosen as the topic for nursing education and for the education of application developers. A new educational program is introduced in this area, currently offered by the school of nursing and the school of technology of the ROC van Twente (Regional Educational Centre of Twente).

Background In the Netherlands, the activities of the National ICT institute for Health Care (www.nictiz.nl) [1] lead to the emergence of standards for electronic message exchange and development of electronic patient records [2]. The use of the EPR will be required by law in the near future. Thus there is a need for all health professions to learn using it, and to be able to support the development of EPR that address the clinician’s needs.

Keywords:

The Netherlands have based their ‘information for health care’ strategy on the message paradigm, applying the international Health Level 7 version 3 (HL7 v3) standard for the safe exchange of patient information to authorized users via a national infrastructure. HL7 v3 has been used in about 20 projects now as a method to determine (clinical) user needs, modelling these needs, and implement clinical content in electronic messages. In addition, several vendors successfully base their electronic patient record systems on the HL7 v3 models.

health informatics education, HL7, vocabulary, standards, continuity of care, electronic patient records

Introduction National infrastructures for health care information exchange present a challenge to clinicians to adopt many standards. However, clinicians usually did not get education about standards, and even in this age, the curricula of schools of health professionals usually do not include such topics in the overcrowded programs, despite the growing need for it.

A key part of the developments include the HL7 v3 messages for continuity of care: this is the care provision domain [3]. This care provision message is meant for referral, acceptance, record exchange, discharge summaries and so on. It is a generic structure covering a standard way to identify sender and receiver, the patient, the purpose of the message and the expression of supporting clinical data. Clinical details are expressed in the ‘care statements’ or ‘clinical statements’. The care statements themselves can vary significantly, but the way they are

Ageing populations and increasing numbers of people with chronic diseases change the focus of health care in such a way that the application of technology becomes essential. Example technologies applied include home care technology, video surveillance, smart homes, and information and communication technologies such as telehealth and electronic patient records (EPR). Increasingly, these technologies are integrated with each other, for instance based on internet

1396

W. T. F. Goossen et al. / ROC van Twente: Nursing Education in Care and Technology

included in the message is consistent over different clinical domains. This has lead to the development of care information models [4, 5] that standardize clinical content in such a way it can be used and re-used in the HL7 v3 care provision messages [5]. The purpose of this approach is to realise semantic interoperability between health care information systems and technologies. Semantic interoperability is considered such that professionals receiving patient data electronically, clearly understand the meaning of the message and can adequately continue the required care.

The care and technology module The student groups participating in the minor care and technology include about 25 students from the school of nursing and about 15 students of the application developers program of the school of technology of the ROC van Twente. The teacher team is a multidisciplinary team of nurse educators and information technology educators. An overall goal for the module that started November 23 is that both student groups, each from their own perspective, understand the process of determination of information requirements, standardization and development and implementation of electronic patient record systems and electronic messages. The care information models are intended as a framework that bridges the often existing communication gap between system users and system developers.

The school of nursing in Twente is timely with this approach due to the fact that the Twente region is a national pilot for the electronic patient record [7]. Health care providers in this region have requested more education in the area of care and technology, specifically about the use of information and communication technology. The program described below has been developed with input from representatives of the health care providers. The program is considered a try out for both education within the school of nursing, and for continuing education of the existing nursing workforce. These students from the school of nursing of ROC van Twente will eventually work as nurses, but with additional knowledge and experiences with the EPR.

The minor program serves as a differentiation within the nursing and within the technical education. Therefore it is assumed that students do know the basics of nursing care and have experiences in traineeships before entering the minor program. For the technical students, an equivalent background in systems life cycle and methods applied is expected. Specific learning objectives / required competencies include: • • • • •

Multidisciplinary cooperation Communication and active participation Analysis of the need for care (nursing students) Analysis of information needs (application developers). Development of a care information model that includes purpose, description of variables, codes, HL7 v3 model, and technical data specification. • Development of functional requirements for a electronic patient record system for continuity of care. In total, the program consists of 12 weeks of education. A total of 4 contact hours per week is presented in small working groups with a mix of nursing and technology students. In addition, the students need an average of 4 hours a week for reading and preparation of the teamwork. The program is presented in Table 1.

Design of the educational program The master docent, care and technology The ROC van Twente is positioned in the east of the Netherlands. Twente is the front runner for the national implementation of the Nictiz spearhead projects: the medication record and the general practitioner to general practitioner record exchange. Therefore there is a perfect situational context for education. Students will be confronted with the developments in their practical / clinical traineeships. However, for a school of nursing it is difficult to gain immediate expertise to start participation. Therefore a new teacher role was established: the first master docent for diploma level education. The master docent has responsibilities to innovate the education, in this example about care and technology, to bring in knowledge and experience, in this example built up on many projects in health standards [7], to involve the teachers, in this example via a project team, and to deliver service to health care facilities in the Twente region. Service to health institutions is delivered for instance via participation in requirements gathering workshops and traineeships.

The first five weeks are about the need for electronic patient records and messages, and the content of the messages. Then the application developers fall back on their normal program, and use time in between to work on the functional requirements and system design. In the meantime the nurse students have a clinical traineeship. For six of the nursing students, the traineeship involves participation in a nursing system development. In this particular setting, the ROC van Twente, a home care agency, and a vendor work together to create a new traineeship.

To start the developments, a choice was made to keep it simple in the beginning, but at the same time take an example technology that is innovative and that relates to the national and regional developments of information and communication technology in health care. Thus, semantic interoperability was chosen as the leading principle to start the developments. Health care agencies in the region agree that they require nurses with skills and knowledge to participate in development of patient record systems and messages.

After about 16 weeks, both the nursing students and the application developers come back to school and continue another 7 weeks of education on this module. These 7 weeks deal with the subjects presented in Table 1. They include continuity of care and requirements for electronic patient records and messages. Further, the development of a care information model is a core element. The students

1397

W. T. F. Goossen et al. / ROC van Twente: Nursing Education in Care and Technology

finish the program with a presentation of requirements, design and examples. Teaching materials include a module and reading materials, based on the work for the national information and communication infrastructure.

Week 6. Continuity of care record Prepare a continuity of care record for stroke patients: contents, standards, sequence diagrams and functional design.

Software: stroke care record system Another teaching tool is the electronic stroke care record system by Portavita [8]. Portavita is a vendor that agreed to have their software for stroke systems, still under development, made available for the students of ROC van Twente. This is a cost neutral arrangement, where both parties benefit: ROC van Twente gets access to a professional clinical information system for education, the vendor gets exposure and feedback on the system, and educational materials developed around the system become available for clients.

Week 7. Review existing materials Select all relevant care information models from the repository www.zorginformatiemodel.nl Week 8. Classifications and codes Apply coding from standardised nursing and health vocabularies Week 9. Evaluate design against existing system

Table 1 – Overview of the program for Care and Technology

Compare the functional design with existing system for stroke care

Week 1. The need for the electronic patient record (EPR) and standards.

Week 10. Functional Design

Getting acquainted with each other. Nurses: describe what nursing needs in the EPR. Technicians: support the nurses and apply methods for requirements gathering and functional design. Determine how to cooperate for 12 weeks.

Discuss the functional design Technology student presents to working group and makes final adjustments. Week 11. Preparation Prepare a presentation for the whole class and for teachers Criteria include: 1) Agenda and minutes, 2) Even distribution of the work in the small group, 3) The subject of the presentation has been negotiated with and approved by the teacher, 4) Apply presentation software, 5) Include the following: care information model, functional requirements, evaluation of group work, functional design

Week 2. Analyse information in care Nurses: explain which data are required in a stroke care record system. Technician: interview the nurses to get the requirements for a system. Week 3. Care information model 1 Study the care information model structure and start with making one example. Nurses the clinical and terminology part, technicians the model and technical specification.

Week 12. Final assignment

Week 4. Care information model 2

Present as workgroup the results in public

Develop a care information model, including clinical, terminology, HL7 v3 model and technical specifications.

Evaluation

Traineeship intermezzo Nurse student

Evaluate the course and work

Care information models as a core topic

ICT student

In order to have concrete materials available that are manageable for the students, the use of the care information models [5] is taken as the lead during the minor program. The care information models serve as a reusable building block within the framework of HL7 v3 Care Provision messages [2, 3, 5].

Study the existing paper based Develop a functional methods for continuity of care design for a nursing during traineeship record system Week 5. Continuity of care for stroke

Care information models combine different standards materials and create valuable content for intelligent semantic interoperability [5]. They function as a communication bridge between clinicians and technicians and

Study needs for continuity of care and describe processes, professionals involved, roles, tasks and activities, information to be exchanged and apply UML modelling.

1398

W. T. F. Goossen et al. / ROC van Twente: Nursing Education in Care and Technology

facilitate inputs into the technical development of electronic messages and EPR systems.

modelling and design of a system (component) that meets nurses’ requirements.

The document structure for the care information models consists of meta-information, detailed description of the clinical instrument, and the reason for its application in practice [5]. It specifies clinical care using professional evidence, uses standardized terminology and coding, uses standard (HL7 v3) information models, and specify at the detailed level the technical requirements for the clinical content. Thus, the technical implementation according the HL7 v3 message and data specification are included via mapping tables, which are useful for EPR development as well. In most documents, one for every item of clinical activities, observation, or instruments, the current components include, in a recently revised format, the following components [5, 9]:

Future plans Currently the module is taught to the two student groups. However, once the module is delivered, more work for the master docent and the project group is waiting. The following new developments are on the agenda for 2007 and 2008. 1. ROC van Twente wants to integrate the module to other health care providers’ educational programs. 2. The content and assignments will be put into an electronic learning environment, thus making it available to students wherever they are. 3. The minor program will be made available for continuing education for the health care agencies in the region Twente. 4. Other ROC’s (diploma schools) have joined or will join and a wider spread of the materials and the education on these subjects is in preparation.

1. Version management and authorship 2. Explanatory introduction about the use of care information models 3. Aim of the instrument, index, scale, act, or observation 4. Scientific foundations / evidence base or other foundation such as guidelines 5. Description of variables / data items / values 6. Working instructions for practice 7. Interpretation guidelines for the results 8. Information on the topic relevant for care process 9. References / acknowledgements 10. An example of the instrument (when available) 11. HL7 v3 message model and description 12. Mapping table from domain to standardized terminology and to HL7 v3 domain message model 13. XML message example (extensible markup language) 14. Copyright issues, such as licensing of source materials, allowed use of care information models 15. Screen designs / screen shots for the instrument 16. Remarks, e.g. if a Dutch version is different from English version of an instrument 17. Contact information: how to contact the author(s) 18. Disclaimer A current overview of the 90 care information models, in the earlier – less complete – format, is available at the website: www.zorginformatiemodel.nl. [4]

Discussion and conclusion National and regional developments of information and communication technology in healthcare, such as electronic patient records and electronic messages, are emerging in order to deal with the changes in the health situation of the Dutch population. Due to results of standardisation efforts, useful materials become available for education. The ROC van Twente decided to develop a new role, the master – docent care and technology – to assist in establishing an innovative program for nursing students and technology students. The innovation includes several challenging areas. The first is a new kind of content: the sensible use of technology in the care environment. Secondly, the application of clinical, vocabulary, message and technical standards for exchange of information for continuity of care, based on a well established format of care information models. Thirdly, the use of electronic record software for stroke patients currently under development. Fourth includes the multidisciplinary student groups: nurses and technical developers, handling the same problem each from their own perspective. Fifth, a minor program within the existing education, indeed with a traineeship for currently few nurse students with a focus on system development. Finally, a project group from teachers, representatives of health agencies, and experts working together to achieve this.

Students work together in small groups to complete one (draft version) of a clinical relevant topic and specify it according to the above format for the care information model and complete the HL7 v3 specification.

It is an exciting area of developments and we will be proud to present the results of the first course during the conference.

Exam The exam of the module consists of a presentation by the students of the design of a system for continuity of care, based on the professional and information standards. Nursing students must underpin their requirements from a patient care perspective. Application development students must substantiate their part with the analysis,

Acknowledgments We want to thank the participation of all involved teachers and heads of departments of ROC van Twente, and the members of the project team from the health institutions in the Twente

1399

W. T. F. Goossen et al. / ROC van Twente: Nursing Education in Care and Technology Region. Nelleke Plaisir of Acquest for her contributions to the module and reading materials.

[7] Goossen W. Representing clinical information in EHR and message standards: analyzing, modelling, implementing. HINZ Primary Care and Beyond: Building the e-Bridge to Integrated Care. Eds: Christopher Peck & Jim Warren. Publisher: Health Informatics New Zealand (HINZ) & Health Informatics Society of Australia Ltd (HISA), 2006. [8] Portavita (2006). www.portavita.nl [Webdocuments. Accessed November 2006]. [9] Van der Kooij J, Goossen WTF, Goossen-Baremans ATM, Plaisier N. Evaluation of Documents that Integrate Knowledge, Terminology and Information Models. In: Park HA, Murray P, Delaney C, (Eds). Consumer-Centered Computer-Supported Care for Healthy People. Proceedings of NI2006. Amsterdam etc. IOS Press, 2006: 519-522.

References [1] NICTIZ Nationaal ICT instituut in de Zorg. NICTIZ website. Available at www.nictiz.nl. Accessed November 2006. [2] Goossen WT, Jonker MJ, Heitmann KU, et al. Electronic Patient Records: Domain Message Information Model Perinatology. Int J Med Inf. 2003; 70 (2-3):265-276. [3] Health Level 7. Message standards. WWW documents. http://www.hl7.org/ Accessed November 2006. [4] NICTIZ, Acquest and Portavita. Care Information Models. www.zorginformatiemodel.nl. [Webdocuments, accessed November 2006]. [5] Goossen WTF. Intelligent semantic interoperability: integrating knowledge, terminology and information models to support stroke care. In: Park HA, Murray P, Delaney C (Eds). Consumer-Centered Computer-Supported Care for Healthy People. Proceedings of NI2006. Amsterdam etc. IOS Press, 2006: pp. 435-439. Studies in Health Technology and Informatics, volume 122. [6] Information Technology in de Zorg in Twente (2007). www.izit.nl [Web documents accesed March 2007]

Address for correspondence William Goossen, Results 4 Care, De Stinse 15 3823 VM Amersfoort, the Netherlands, [email protected], [email protected]

1400

MEDINFO 2007 K. Kuhn et al. (Eds) IOS Press, 2007 © 2007 The authors. All rights reserved.

Multiple Measures of Provider Participation in Internet Delivered Interventions Thomas K. Houstona,c, Ellen Funkhouserb,d, Jeroan J. Allisonc,d, Deborah A. Levinea,c, O. Dale Williamsc, Catarina I. Kiefec,d a

Division of General Internal Medicine, University of Alabama at Birmingham b School of Public Health, University of Alabama at Birmingham c Division of Preventive Medicine, School of Medicine, d Birmingham VA Medical Center the intervention.11 Thus, investigators might want to assess frequency of use per month enrolled in a longitudinal intervention. A combination of measures may best represent actual participation.

Abstract Evaluation of Internet-delivered continuing education for health care providers requires appropriate consideration of their level of participation. To fully assess participation requires multidimensional measures, including factors such as the volume of participation (page views), frequency (visits), variety (components accessed by each provider), and duration (months of activity). We calculated crude and refined (adjusted for study design) measures and then compared these measures across three longitudinal Internet-delivered continuing education to health care providers (N = 429). We found that participation varied across study, varied by factor and varied by specific measure. Correlation between crude and refined measures within a factor and across factors differed significantly. Participation assessment of internet-delivered interventions varies by the selection of measure and factor. Further research assessing the potential for these measures to predict intervention effectiveness is needed.

Few previous studies of Internet-delivered continuing education have measured differences in participation. Also, it is unknown how different measures of participation are correlated. As a first step toward developing a scientific approach to measuring participation, we propose a fourdimensional model. The general factors include volume, frequency, variety of components accessed, and duration of access. Within this model, we developed crude and refined measures of each factor and then compared these measures using data from three longitudinal Internet-delivered continuing education interventions. Our goal was to determine how the measures varied by study and to assess the correlation of crude and refined measures within factor and across factor.

Methods

Keywords:

Study design

internet, continuing education, educational measurement, quality of health care

Our group is conducting three separate group-randomized trials of Internet-delivered educational interventions to improve practice patterns of health care providers. Funded by the U.S. National Institutes of Health (NIH) and Veterans Affairs Health Services Research & Development (VA), these trials have similar designs and objectives but different target populations of providers and disease foci. The National Heart, Lung and Blood Institute (NIH) MIPlus, conducted in two southern states, and VA MI-Plus, a national study, are parallel interventions targeting different provider populations: Medicare primary care providers in two Southern states and clinic-based primary care VA providers, respectively. Both MI-Plus studies seek to improve care for ambulatory post-myocardial infarction patients with multiple co-morbidities. Funded by the National Institute on Drug Abuse (NIH) DentalTobaccoControl.net (DTC) targets dental providers and seeks to improve tobacco cessation practice patterns in dentistry.

Introduction The Internet has promise as a new tool to increase translation of research into clinical practice.1 If proven effective, low-intensity Internet-delivered continuing education interventions have a high potential for dissemination. These interventions seek to improve provider knowledge, motivation, and behavior and are often multi-modal with access to case-based education, decision support, and patient education materials. Many such interventions have the ultimate goal of improving quality and enhancing safety. Although evaluations of Internet-delivered continuing education for healthcare providers are increasing2-8, no standard method of evaluating these interventions exists. To more fully understand the impact of these interventions, investigators need to measure participation, including tracking of all provider “encounters” with the program.

We prospectively tracked participation by 429 healthcare providers enrolled in the intervention arms of these trials. Users (private practice physicians, nurse practitioners, physician assistants, dentists, and hygienists) were recruited to the websites through mailings, phone calls, and emails and enrolled over multiple months ranging

Tracking provider-intervention encounters, however, is challenging. Web tracking software most frequently tracks page views and visits.9, 10 However, all page views may not be of equal value. A frequent goal of website developers is to create “stickiness” - a measure of repeat usage of

1401

T. K. Houston et al. / Multiple Measures of Provider Participation in Internet Delivered Interventions

from six to twelve, depending on the study. For this analysis, tracking data for each study was collected through a specific calendar month in 2006 (March for NHLBI MIPlus, August for VA MI-Plus, and September for DTC). As enrollment was rolling, total months of enrollment varied for individual providers (mean months of enrollment was 14 (SD 5) for NHLBI MI-Plus, 11 (SD 4) for VA MI-Plus, and 12 (SD 4) for DTC). Once enrolled, all providers received scheduled email reminders, including notifications of new content, to encourage participation. All providers received continuing education credits specific to their specialty (medical or dental) and a certificate of appreciation for their participation. Additional incentives were provided in the NHLBI MI-Plus study (access to online journals and a textbook) and VA MI-Plus (subscription to the Medical Letter). Each study was approved by the appropriate Institutional Review Board.

cent of enrolled months (refined), defined as the proportion of enrolled time known to be active (months active/months enrolled) because of the variation in potential enrollment across studies. We defined five categories of providers (private practice physicians, VA physicians, VA nurse practitioners/physician assistants, dentists and hygienists).

Intervention descriptions

We calculated the means and standard deviations of each measure. We assessed differences in participation measures by type of provider using t-tests. Because of the multiple comparisons, we chose a significance level of = 0.01. Then, we evaluated participation in each intervention component by provider group and assessed differences between provider groups using chi-square tests. Finally, using participants in all three studies, we assessed pair-wise correlations between measures within and across factors using Spearman’s rank correlation coefficient. We then repeated this analysis for each individual study again with = 0.01.

Other measures of participation could be calculated. Specifically, our measures focused mostly on counts, not session time, or time per webpage. Using session time is challenging as our providers rarely logged off, but would simply close the site so that session time continued indefinitely. Also, frequent outliers for session time existed (over five hours of activity in a single visit) suggesting that providers would just leave the page open and go to another task. Thus, we have not used session time in this analysis. Statistical analysis

The core of all three interventions was case-based educational programs using interactive, web-based modules with tailored feedback based on responses to questions. During the tracking periods, the content and number of cases varied by study (6 for MI-Plus, 8 for VA MI-Plus, and 3 for DTC). All three studies had an accompanying “toolbox” with practice tools and patient educational materials that could be downloaded. The two MI-Plus studies included 1) a literature watch segment updated at intervals with reviews of the literature and 2) a guidelines component with summaries of current guidelines applicable to post-myocardial infarction patients. The literature watch and guidelines of the MI-Plus studies were analogous to the headlines and library components of the DTC study. Feedback of performance data with peer comparisons was provided to NHLBI MI-Plus providers. In the DTC study, testimonials of provider’s success in encouraging smoking cessation were included. For this analysis, we focused on intervention components that were consistent across the three studies.

Results We recruited 429 providers from 344 practices. These included 108 private practice primary care physicians in the NHLBI MI-Plus study, 193 VA primary care providers (125 physicians and 68 physician assistants or nurse practitioners) in the VA MI-Plus study, and 128 private practice dental providers (68 dentists and 60 hygienists) in the DTC study. Across studies, the mean number of months since enrollment was 12.2 months (SD 4.5).

User authentication was required for all providers as they logged onto the interventions. Thus, we used server tracking logs linked to visit to calculate the measures of participation. The log included an individual user identification number and was tagged with date and time.

Results by study Overall, across measures of volume, frequency, variety, and duration of participation, values tended to be higher for the VA providers, both physician and non-physician, compared with private practice physicians and dental providers (Table 1). The mean visits per month among VA physicians, 0.62 (SD 0.52), was twice that of private practice physicians, 0.29 (SD 0.22), P = 0.001; the mean visits per month for VA Nurse-practitioners/Physician Assistants was even higher, 0.85 (SD 1.5), compared with private practice physicians, P = 0.0002. The point estimate for page views per month was highest among dentists in the DTC study (19.5) but the standard deviation was quite wide (106.6), thus this estimate was not significantly different than the estimates amongst the other providers.

Measures of participation As noted above, we propose a four-dimensional model to evaluate participation. The four factors are 1) volume, 2) frequency, 3) variety of components accessed, and 4) the duration of activity. For each factor, we developed crude and refined measures for each factor. Volume measures included total number of page views (crude) and the refined measures (number of page views per visit and number of pages per month). Frequency measures included total number of visits (crude) and number of visits per month (refined). Variety measures included number of components accessed (crude). Because the central component of the three interventions was the cases, we also created variety measures specific for the cases: number of case modules completed (crude), and mean percent of case modules completed (refined). Duration measures included 1) number of months actively participating (crude), defined as months from first to last logon, and 2) mean per-

Variety of access as measured by number of components was lowest among the dental providers [1.5 (SD 1.2) for dentists and 0.97 (SD 0.89) for hygienists] compared with the other studies, but as measured by proportion of cases completed, participation in the DTC study was similar to the VA MI-Plus study. Duration of participation ranged from a mean of 7.9 months (SD 4.1) in NHLBI MI-Plus (the longest running study) to 2.8 months (SD 4.2) for hygienists.

1402

T. K. Houston et al. / Multiple Measures of Provider Participation in Internet Delivered Interventions

Table 1 - Mean Measurements (Standard Deviations) of Participation for Providers in Three Internet-delivered Intervention Studies* NHLBI MI-Plus

VA MI-Plus

_

DentalTobaccoControl.Net

Private Practice Physicians (n = 108)

VA Physicians (n = 125)

VA NP/PA‡ (n = 68)

Dentists (n = 68)

Hygienists (n = 60)

Mean (SD)

Mean (SD)

Mean (SD)

Mean (SD)

Mean (SD)

60.4 (51.4) a,b,c,d

100.3 (74.5) a,e,f

89.2 (73.3) b,h

35.1 (33.0) c,e

24.5 (25.0) d,,h

Number of Page Views Per Visit

16.4 (8.8) c,d

15.6 (8.2) e, f

15.6 (9.6)

9.7 (7.11)c, e

9.5 (6.0) d,f

Number of Pages Per Month

4.5 (3.8) a,b,d

9.3 (8.1) a

13.6 (30.4) b

19.5 (106.6)

2.6 (3.8) d

Total number of Visits

3.8 (3.2) a,b,d

6.7 (5.5) a,e,f

5.9 (5.4)

3.0 (3.7) e,i

2.5 (2.4) d,f,i

Number of Visits Per Month

0.29 ( 0.22) a,b

0.62 (0.52) a

0.85 (1.5) b

1.00 (4.0)

0.25 (0.27)

2.3 ( 1.3) a,b,c,d

2.8 (1.2) a,e,f

2.7 (1.3) b,g,h

1.5 (1.2) c,e,h

0.97(0.89) d,f,g

1.7 (1.6)

3.5 (2.5)

3.2 (2.7)

1.5 (1.3)

1.3 (1.2)

Participation Measures* A. Volume Total number of Page Views

B. Frequency

C. Variety Number of Components Number of Cases Mean Percent of Cases completed

28%(27)

a,c,d

44% (32%)

a

40% (33%)

50%(44%)

c

43%(41%) d

D. Duration Months from First to Last logon

7.9 (4.1) c,d

7.0 (5.4) e,f

6.2 (5.5)

4.8 (4.9)c,e

2.8 (4.2) d,f

Mean Percent of enrolled months

51%(38) d

56% (37) e,f

53 (40)%

39% (38) e

24% (35) d,f

* §

Participation measured in four dimensions – volume, frequency, variety, and duration of access. Cells in the same row with the same superscript are statistically significantly different, p