Barriers and Biases in Computer-Mediated Knowledge Communication
COMPUTER-SUPPORTED COLLABORATIVE LEARNING VOLUME 5
Series Editor:
Pierre Dillenbourg, Swiss Federal Institute of Technology, Lausanne, Switzerland Editorial Board:
Michael Baker, CNRS & Universite'Lumi2re Lyon, France Carl Bereiter, Ontario Institute for Studies in Education, Canada Yrjo Engestrom, University of Helsinki, Finland Gerhard Fischer, University of Colorado, U.S.A. H. Ulrich Hoppe, University of Duisburg-Essen, Germany Timothy Koschmann, Southern Illinois University, U.S.A. Naomi Miyake, Chukyo University, Japan Claire O'Malley, University of Nottingham, U.K. Roy Pea, SRI International, U.S.A. Clotilde Pontecorovo, University 'La Sapienza', Italy Jeremy Roschelle, SRIInternational, U.S.A. Daniel Suthers, University of Hawaii, U.S.A.
The Computer-Supported Collaborative Learning Book Series is for people working in the CSCL field. The scope of the series extends to 'collaborative learning' in its broadest sense; the term is used for situations ranging from two individuals performing a task together, during a short period of time, to groups of 200 students following the same course and interacting via electronic mail. This variety also concerns the computational tools used in learning: elaborated graphical whiteboards support peer interaction, while more rudimentary text-based discussion forums are used for large group interaction. The series will integrate issues related to CSCL such as collaborative problem solving, collaborative learning without computers, negotiation patterns outside collaborative tasks, and many other relevant topics. It will also cover computational issues such as models, algorithms or architectures which support innovative functions relevant to CSCL systems. The edited volumes and monographs to be published in this series offer authors who have carried out interesting research work the opportunity to integrate various pieces of their recent work into a larger framework.
The titles published in this series are listed a t the end of this volume.
R. Bromme F.W. Hesse H. Spada (Editors)
Barriers and Biases in Computer-Mediated Knowledge Communication And How They May Be Overcome
@ - Springer
Rainer Bromme, University of Miinster, Germany Friedrich W. Hesse, University of Tiibingen, Germany Hans Spada, University of Freiburg, Germany
Library of Congress Cataloging-in-PublicationData A C.I.P. Catalogue record for this book is available from the Library of Congress. ISBN 0-387-24317-8
e-ISBN 0-387-24319-4
Printed on acid-free paper.
O 2005 Springer Science+BusinessMedia, Inc.
All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+BusinessMedia, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 9 8 7 6 5 4 3 2 1
SPIN 11161530
CONTENTS Bromme. R., Hesse. F.W. & Spada. H . BARRIERS. BIASES AND OPPORTUNITIES OF COMMUNICATION AND COOPERATION WITH COMPUTERS: INTRODUCTION AND OVERVIEW 1 1. Why "communication and cooperation with computers"? ............................1 2 . Why "barriers" .................................................................................................... 1 3. Why "biases" ....................................................................................................... 2 4 . Why "opportunities"?.......................................................................................... 3 5 . The construction and use of technical artifacts is based on presumptions of barriers and how to overcome them .................................................................... 3 6. Three basic barrier-presumptions related to problems of communication and cooperation.......................................................................................................... 4 6.1. The individual and mutual construction of "meaning" and the exchange of information in groups .............................................................................. 4 6.2. The establishment and maintenance of structure in social interactions........ 4 6.3. The establishment and maintenance of motivation to cooperate and communicate ................................................................................................ 5
7. What is the impact of the computer environment on the mentioned barriers? .... 5 8. The chapters ........................................................................................................ 6 9 . A map for the localization of barriers. biases and opportunities ....................... 12
References .............................................................................................................
14
Weinberger. A., Reiserer. M., Ertl. B., Fischer. F . & Mandl. H. FACILITATING COLLABORATIVE KNOWLEDGE CONSTRUCTION IN COMPUTER-MEDIATED LEARNING ENVIRONMENTS WITH COOPERATION SCRIPTS.................................................................................. 15 1. Collaborative knowledge construction .............................................................. 15
2 . Computer-mediated learning environments ...................................................... 16 3. Facilitating collaborative knowledge construction............................................ 17 3.1 Facilitating social and epistemic activities with scripts .............................. 19 3.2 Social cooperation scripts for collaborative knowledge construction ......... 20 3.3 Epistemic cooperation scripts for collaborative knowledge construction .............................................................................. 21
4 . Implementing social and epistemic cooperation scripts in computer-mediated learning environments ....................................................... 21 5 . Facilitating computer-mediated learning through scripts: Evidence from web-based discussion boards and videoconferencing ....................................... 23 5.1 Study 1: Facilitating collaborative knowledge construction in text-based communication through cooperation scripts ..........................23 5.2 Study 2: Facilitating collaborative knowledge construction in a videoconferencing environment through cooperation scripts ......................29 33 6 . Discussion ......................................................................................................... References .............................................................................................................
35
HOW TO SUPPORT SYNCHRONOUS NET-BASED LEARNING DISCOURSES: PRINCIPLES AND PERSPECTIVES ....................................... 39
I . Learning discourses ........................................................................................... 39 1.1 Prototypical discourse: Face-to-face ........................................................... 40 1.2 Levels of discourse ...................................................................................... 41 2 . Variants of support ........................................................................................ 44 2.1 The augmentation paradigm ........................................................................ 44 2.2 The reduction paradigm .............................................................................. 46 49 3 . A cost-effort framework .................................................................................... 3.1 The costs of grounding ................................................................................ 50 3.2 The utility of grounding .............................................................................. 50 3.3 Deciding to ground ...................... ...... ..................................................... 51 4 . Perspectives ....................................................................................................... 53 References ........................................................................................................
55
Rummel. N., & Spada. H . INSTRUCTIONAL SUPPORT FOR COLLABORATION IN DESKTOP 59 VIDEOCONFERENCE SETTINGS .................................................................... 1. Introduction ....................................................................................................... 59 60 2 . Challenges in computer-mediated collaboration .............................................. 2.1 A first challenge: Problem-solving and learning in collaboration ...............60 2.2 A second challenge: Complementary expertise as a basis for collaboration ............................................................................................... 61 2.3 A third challenge: Collaborating in a computer-mediated setting ...............61 3 . Overcoming the barriers: How to achieve good collaboration ..........................64 3.1 Support during collaboration ....................................................................... 64 3.2 Support prior to collaboration: An instructional approach ..........................65
vii 4 . Assessing collaboration: How to test the effects of support measures ..............67 4.1 Assessing collaborative process .................................................................. 68 4.2 Assessing joint outcome .............................................................................. 74 4.3 Assessing individual knowledge ................................................................. 75 5 . An experiment illustrating instructional measures. experimental paradigm. and assessment methods ..................................................................................... 75 5.1 Task. participants. and setting ..................................................................... 76 76 5.2 Experimental conditions ............................................................................. 5.3 Assessing the dependent variables .............................................................. 77 5.4 Summary and discussion of results ............................................................. 78 6 . Instructional support measures and methods of assessment: 80 Lessons learned ................................................................................................. 6.1 Instructional support measures for achieving good computer-mediated collaboration ............................................................................................... 80 6.2 Methods of assessing process and outcome of collaboration ......................81 6.3 Challenges met. barriers overcome. chances realized? ............................... 83 References ............................................................................................................. 84
Bromme. R., Jucks. R . & Runde. A . BARRIERS AND BIASES IN COMPUTER-MEDIATED EXPERTLAYPERSON-COMMUNICATION ................................................................. 89 1. Two vignettes and a short introduction ............................................................. 89 90 2 . Opportunities and barriers ................................................................................. 2.1. Opportunities for and barriers to distributing and finding specialized medical knowledge via the Internet ........................................................... 90 2.2. Expert's specialist knowledge: Sources of advice - and mutual misunderstanding ....................................................................................... 92 3 . Knowledge differences and mutual understanding in text-based asynchronous computer mediated communication .......................................... 94 3.1. Introducing Herbert H . Clark's communication model .............................. 94 3.2. Some doubts about the cooperative nature of communication ...................96 4 . Expertise and the design of written medical explanations: studies on the use of the community membership heuristic and physical copresence heuristic in computer mediated settings ................... ............................... 97 4.1. Our research questions .................... . ....................................................... 97 4.2. Study 1: Analyzing experts' audience design .......................................... 99 4.3. The other side of the coin: Studies with layperson samples ..................... 106
. . .
5 . Synthesis .........................................................................................................
114
References ........................................................................................................... 116
Anderson. A.H., Mullirz. J., McEwan. R., Bal. J., Carletta. J., Grattan. E . & Brundell. P.
EXPLORING WHY VIRTUAL TEAMWORKING IS EFFECTIVE IN THE LAB BUT MORE DIFFICULT IN THE WORKPLACE.................................. 119 1. Introduction ................................................................................................... 119 120 1.1 Overview of chapter .................................................................................. 121 1.2. Study 1: Lab study of virtual team working ........................................ 1.3. Discussion ................................................................................................ 125
. .
2 . Study 2: Workplace study of virtual teams ................................. ..............127 2.1. Background .............................................................................................. 128 2.2. Observing and analysing virtual meetings ..............................................129 2.3 Observations from virtual team meetings ............................................. 130 3 . Study 3: Simulation study of virtual team meetings ....................................... 132 3.1. Method .....................................................................................................133 135 3.2. Results ...................................................................................................... 3.3. Discussion ................................................................................................ 137 138 4 . Conclusion ...................................................................................................... References ........................................................................................................... 140 Cress.
(I., Barquero.
B., Buder. J . & Hesse. F . W
SOCIAL DILEMMA IN KNOWLEDGE COMMUNICATION VIA SHARED DATABASES .................................................................................... 143 1. Shared databases . knowledge communication of the third kind ...................143 146 2 . Knowledge exchange as public-goods dilemma ........................................... 149 3 . Experimental investigation of the database dilemma ...................................... 3.1. Experimental implementation of the database dilemma ..........................149 3.2. Empirical results ...................................................................................... 153 3.3. Interpretation ............................................................................................ 156 4 . Possible solutions to the communication dilemma ......................................... 157 4.1. Individual solutions .................................................................................. 157 4.2. Structural solutions .................................................................................. 161 164 5 . Summary ......................................................................................................... References ........................................................................................................... 165 Kirschner. P.A. & Kreijns. K.
ENHANCING SOCIABILITY OF COMPUTER-SUPPORTED COLLABORATIVE LEARNING ENVIRONMENTS ......................................169 1. Introduction ..................................................................................................... 169
2 . An educational shift ........................................................................................ 170 171 3. Collaborative learning ..................................................................................... 171 3.1. Enhancing collaborative learning ............................................................ 3.2. The social basis for these approaches .................................................. 173 4 . Social interaction in CSCL-environments.....................................................
175
5 . Affordances ..................................................................................................... 177 5.1. Technological affordances ....................................................................... 178 ................................................. 179 5.2. Educational affordances ..................... . 180 5.3. Social affordances .................................................................................... 5.4. Affordances and useful CSCL-environments..........................................181 6. Operationalising social affordances: Group awareness widgets ..................... 182 182 6.1 Group awareness ....................................................................................... 6.2 History awareness ..................................................................................... 182 6.3 Set of communication media ................................................................... 183 6.4 Group awareness widgets .......................................................................... 184 6.5 A first prototype of the GAW ................................................................... 185 7 . A study of the use of the GAW prototype ....................................................... 186 187 7.1 Method ...................................................................................................... 7.2 Results ....................................................................................................... 187 7.3 Discussion and conclusions.................................................................... 187 References ........................................................................................................... 188 Strube. G., Thalemann. S., Wittstruck. B . & Garg. K .
KNOWLEDGE SHARING IN TEAMS OF HETEROGENEOUS EXPERTS . 193 194 1. Policies of sharing knowledge ........................................................................ 2 . Method ............................................................................................................ 195 3. Web design. our domain of application .......................................................... 195 4 . The structure of a web design project ............................................................. 196 4.1 Overall structure of the web design task ................................................... 196 4.2 Subtask dependencies ............................................................................... 199 5 . Knowledge communication in web design teams ........................................... 199 5.1 Parameter setting: a quasi-experiment .............................................. 200 202 5.2 Examples of design options ...................................................................... 203 6 . Knowledge-level modelling ............................................................................ 6.1 Representing design decisions .................................................................. 204 205 7 . Supporting the web design process ................................................................. 205 7.1 Barriers and biases ..................................... ............................................ 7.2 Interviews.................................................................................................. 207
8. Discussion ....................................................................................................... 209
... References ..................
................................................................................... 211
Fischer. G. & Ostwald. J . KNOWLEDGE COMMUNICATION IN DESIGN COMMUNITIES ..............213 213 1. Introduction ..................................................................................................... 214 2 . Design and design communities ...................................................................... 2.1 Design ....................................................................................................... 214 2.2. Design communities ................................................................................. 219 2.3 Boundary objects ...................................................................................... 224 225 3 . Media in support of knowledge communication ............................................. 3.1 Rich and lean media .................................................................................. 225 3.2 Computer-mediated communication in COPS........................................... 227 3.3 Computer-mediated communication in CoIs ........................................ 229 4 . Lessons learned ............................................................................................... 236 4.1 Barriers and biases .................................................................................... 236 4.2 CoIs: Beyond novices and experts ............................................................ 237 238 5 . Conclusions ..................................................................................................... 239 References ........................................................................................................... Dillenbourg. P . DESIGNING BIASES THAT AUGMENT SOCIO-COGNITIVE INTERACTIONS ................................................................................................ 243 243 1. Introduction ..................................................................................................... 2 . The imitation bias ............................................................................................ 245 247 3 . Augmenting social interaction ........................................................................ 3.1 Persistency creates shared working memory .......................................247 3.2 The context is stored with the message ......................... . . ..............249 3.3 The mirror for the group ........................................................................... 251 3.4 Synthesis .................................................................................................. 255 4 . Augmenting collaborative learning ................................................................. 255 4.1 How to support collaborative learning? ................................................... 256 4.2 Shaping interactions with interfaces ..................................................... 257 4.3 Shaping interaction with scripts ................................................................ 258 4.4 Regulating interactions ............................................................................. 258 References ........................................................................................................... 261
XI
Koschmann, T., Zemel,A-, Conlee-Stevens, M., Young, N.} Robbs, J. & Barnhart, A. HOW DO PEOPLE LEARN?
265
1. Introduction
265
2. Data 2.1 Fragment 1: "What would be the risk?"
268 268
2.2 Fragment 2: "Salicylate toxieity"
275
3. Barriers and biases to understanding computer-mediated interaction
282
References
284
Appendix
287
Suthers, D. TECHNOLOGY AFFORDANCES FOR INTERSUBJECTIVE LEARNING, AND HOW THEY MAY BE EXPLOITED 295 1. Introduction
295
2. Deconstructing the title 2.1 From "Knowledge communication" to an interactional epistemology 2.2 Computer mediation..-.. 2.3 From "Barriers and biases" to "affordances"
295 295 297 298
3. Representational affordances as a central topic for CSCL 3.1 Representational affordances for individual learning 3.2 Representational effects specific to groups 3.3 Representational differences lead to biases 3.4 Experimental evidence of representational affordances
300 300 300 303 304
4. Three approaches to the study of CSCL 4.1 studies of collaboration independent of learning 4.2 Scaffolding collaborative learning to remove barriers..... 4.3 Understanding and supporting processes of intersubjective learning
306 306 307 309
5. Assessing the field 5.1 What are we studying? A call to focus on intersubjective learning 5.2Howdo we study it? Acall for methodological fusion 5.3 What theory drives the research?
310 310 311 313
6. Conclusions
315
References
316
Nameindex
321
Subjectlndex
329
RAINER BROMME, FRIEDRICH W. HESSE & HANS SPADA
BARRIERS, BIASES AND OPPORTUNITIES OF COMMUNICATION AND COOPERATION WITH COMPUTERS: INTRODUCTION AND OVERVIEW
1. WHY "COMMUNICATION AND COOPERATION WITH COMPUTERS"?
New innovative computer-mediated settings open a wide range of possibilities for cooperative learning and work across distance, domain, and level of expertise. Mechanical engineers discuss the repair of a complicated machine over a distance of thousands of miles. Medical specialists located at different hospitals advise a colleague how to treat a rare illness. Students of a distance learning institute learn cooperatively for their next examination taking advantage of computer-mediated communication. A senior citizen asks for help with a computer problem via hot-line. However, the successful use of such computer-mediated settings is not trivial. Cooperative learning and work itself requires special skills and strategies. Furthermore, the technical settings with sometimes restricted, sometimes new possibilities for communication add problems on top of the cooperation itself. As a consequence, computer mediated cooperation has moved not only into the focus of technological and organizational but also psychological and educational research. Relevant findings of this area of investigation are presented in this book. What are the barriers in computer-mediated communication for cooperative learning and work? Which are the most relevant biases in computer-mediated information processing? How is it possible to overcome these barriers and biases to fully gain advantage from the new opportunities? 2. WHY "BARRIERS"? The term "barrier" comes from psychological research on problem solving and creativity. There it refers to the gap between an initial and an end state. In other words, barriers are challenges which have to be overcome in order to attain a goal. They could not have been avoided from the outset but are "natural" difficulties which - in the case of communication and cooperation with computers - can be traced back to features of the software and hardware used, and to the characteristics of the users and settings. Therefore, barriers will be discussed in all contributions to this book with regard to ways of overcoming them. The authors share the conviction that the technical side (hardware and software) is neither the sole cause of - nor the only solution to - the problems which occur with computer-mediated
communication and cooperation. Many of these barriers are rather challenges which are present in all cooperation and communication scenarios. Some of these barriers are aggravated in computer-mediated settings, some are easier to overcome.
3. WHY "BIASES"? This books deals with computer-mediated cooperation and communication scenarios in teaching and learning situations, leisure activities (e.g. laypersons looking for expert information on the internet), and net-based communication at work. Such scenarios will become increasingly important, because in future more people will spend more time in such scenarios. The CSCL (Computer Supported Cooperative Learning) and CSCW (Computer Supported Cooperative Work) research communities also hold this view. However, the computer environment is only part of people's normal environment. People are required to switch between the strategies and skills which they need for computer-based contexts and those (mostly longerestablished and more basic ones) which they have acquired for use in other contexts. Working with computers as tools for communication and cooperation they have to acquire new knowledge and develop new skills. At best, a transfer of knowledge and skills takes place from the non-technical to the technical environment. These skills are then adapted to the altered conditions of communication and cooperation required when working with the computer. However, in the process, also weaknesses and errors in non-technical information processing and communication are transferred to the computer environment. Therefore, the question arises as to how a computer environment affects weaknesses and biases of this kind. For example: the fundamental attribution error is a well-known bias of attribution judgments (Ross & Nisbett, 1991). Observers tend to over-emphasise dispositional factors of the actor, and under-emphasise situational factors. One of the reasons for this bias is that actors and observers often have different access to information about situational circumstances of the observed behavior. But what happens if the access to information about a remote partner's behavior is confined to what is available on the computer screen? Are people more aware about situational circumstances if the judged behavior is represented on a computer? Or is the fundamental attribution error even stronger due to the remoteness of the observed actor (Cramton, 2002)? We therefore need to analyze both empirically and theoretically in what situations CSCL and CSCW environments provoke or augment biases of human information processing and action regulation and when they reduce them. A distinction needs to be made between evocation and augmentation of biases. In the first case errors are involved which are specifically caused by the technical environment and the characteristics of remote communication. In the second, more frequently occurring case, it is a bias which is present in our behavior anyway, but which is augmented in computer environments. Conversely, there is a need to distinguish between preventive and compensatory effects of technical environments on weaknesses and biases of information processing.
INTRODUCT~ON AND OVERVIEW
4. WHY "OPPORTUNITIES"? Computer-mediated settings for remote cooperative learning and work help to overcome many traditional barriers of distance and time. While it is a challenge to use them in an optimal way, to ignore the new opportunities would mean to miss valuable chances. It would be a misunderstanding to see computer-mediated cooperation just as being inferior to face-to face interaction, because of its restricted possibilities for non-verbal communication, transfer of emotional signs, turn taking etc. Most computer mediated settings enable joint activities by means of application sharing technologies. Documents can be viewed simultaneously and jointly edited. Objects can be manipulated in a workspace that is visible and accessible for all participants. Functioning as an external memory shared workspaces can reduce cognitive load during interaction. Some of the chapters of this book show possibilities to represent the social structure of the interaction and to use this information to promote cooperation.
5. THE CONSTRUCTION AND USE OF TECHNICAL ARTIFACTS IS BASED ON PRESUMPTIONS OF BARRIERS AND HOW TO OVERCOME THEM Every technical artifact is based on an assumption about the problems which ought to be solved by using the artifact. Thus, artifacts are based on a presumption about one or more causes of the barriers which precisely the artifact in question is intended to overcome. In the following we will refer to these assumptions as barrierpresumptions. They have a factual content but also contain a theoretical attribution of causality. Hence, as the following example should illustrate, they are not merely descriptive: a ladder serves as an artifact for overcoming differences in height by providing steps. Its construction is based on a barrier-presumption which is helpful for humans though not for dogs and hardly for fluids. In other words the barrier presumption is not only based on the notion of "height" but also on a certain supposition about the causes why humans have difficulties to overcome "height". For dogs a ramp would be a better proposition, and for fluids pipes and pumps are needed. It is, however, rarely as obvious as in this example what presumptions are made to overcome a barrier. While it is generally easy to formulate such barrierpresumptions for everyday artifacts they are not so straightforward in connection with technical environments for remote cooperative learning and problem solving. It is important to reconstruct theoretically which barrier-presumptions are underlying the construction of some specific environments. We will provide a short overview of such presumptions below. Then we will describe the contributions to this book, always asking what aim is to be achieved by means of the setting examined, and which barriers are intended to be overcome.
6. THREE BASIC BARRIER-PRESUMPTIONS RELATED TO PROBLEMS OF COMMUNICATION AND COOPERATION
6.1. The individual and mutual construction of "meaning " and the exchange of information in groups Many contributions to this book deal with the construction of "meaning" when information is exchanged via computers. This holds true for learning scenarios as well as for workplace settings. When, for example, processing and use of the information presented by the teacher or fellow-students via the computer are discussed, the cooperative establishment of "meaning" is viewed as the central challenge. This we term the meaning barrier. The authors of this book are in agreement that in communication information is not simply transmitted from sender to receiver but that meaning is constructed mutually. Most of their contributions follow perspective-taking models: "Perspective-taking models of language use focus on the shared context that communicators must identify or create to produce and comprehend messages" (Krauss & Fussell, 1996, p. 674). This model also contains a more specific barrier presumption, namely the assumption that without adequate common ground, communication cannot succeed (common ground barrier). Also inherent in the perspective-taking model is a cooperative aspect, i.e. the proposition that both the producer and the recipient of an utterance are responsible for the communication being understandable. However, some instructional contexts entail not only mutual generation of meaning but also acquisition of a meaning which is required as learning content. The learner tries to achieve this by means of an active construction. This barrier applies to a lack of knowledge and skill on part of the individual learner, not to a lack of shared meaning between the communication partners. This we term the epistemic barrier. Common ground barrier and epistemic barrier can be considered as more specific variants of the meaning barrier. A group of people working on a problem has the potential advantage that the group as a whole is more knowledgeable than each of its members. However, this is only true if all the unshared knowledge is pooled which is often not the case (Stasser & Titus, 1985). We term this the unshared knowledge barrier. This barrier is relevant in the case of problems which have to be solved by experts from different fields, too.
6.2. The establishment and maintenance of structure in social interactions Computer mediated communication and cooperation is social interaction. It is emphasized in many of the following chapters that social interaction needs structure. In traditional instruction with a teacher standing in front of the class, it is the task of the teacher to establish and maintain structured social procedures (who does when what?). In computer-mediated cooperation scenarios, too, a structure is necessary: we need to determine when the members of a team work on their own, when they exchange information, etc. Such structures can be provided (e.g., by scripts), they
INTRODUCTIONAND OVERVIEW
5
may evolve automatically during cooperation, or they can be established by technical means, if for example the members of a learning group are asked to contribute consecutively. Therefore, the construction of such environments is based on the assumption that a missing or inadequate structure of the interaction represents a barrier, which we term the structure barrier.
6.3. The establishment and maintenance of motivation to cooperate and communicate The learning and cooperation scenarios addressed in this book are mainly concerned with complex learning goals or elaborate tasks that have to be worked on without a strong emphasis on external goal setting and control. Such tasks promote motivation and interest, which are, however, also assumed to be present. Some of the contributions, therefore, also ask what effects the computer environment might have on the users' motivation. They are concerned with overcoming motivational problems, in the following referred to as the motivation barrier.
7. WHAT IS THE IMPACT OF THE COMPUTER ENVIRONMENT ON THE MENTIONED BARRIERS? Meaning and shared knowledge, interaction structure and motivation are critical whenever groups have to cooperate in work or learning contexts, not only in computer-mediated contexts. Nevertheless, problems related to the establishment of meaning, structure and motivation take a different form in computer-mediated contexts. An additional aspect has to be considered. Computers are universal tools. They are powerful tools for the representation and transfer of information, for social distribution (many receivers can be reached who in turn can easily switch from the recipient to the producer role), and for storage and relocation of information. This universality and power of the computer must be set against the everyday experience of many users who have to invest a lot of time and energy when working with computers which do not function in the way they ought to (e.g. programs crashing, incompatible transfer protocols, confusing user interfaces). Such experiences may lead to the impression that barriers in remote communication are mainly caused by the hardware and software, or that a lot of effort is needed to operate this "universal machine" at all adequately. Furthermore, users often think that they themselves are responsible for these barriers. Human information processing is flexible enough to work - within limits - with confused user interfaces, poor transmission rates etc., i.e. to function in spite of inadequate tools. On the other hand, empirical surveys show that quite frequently good features of hardware and software which would be capable of supporting users are not utilized (Aleven, Stahl, Schworm, Fischer, & Wallace, 2003). We will in the following, when introducing the chapters, consider in each case which barrier-presumptions underlie the development or testing of the particular computer-mediated setting for remote communication. We will then outline the assumed impact of the computer environment on the emergence of barriers as well
as ways of overcoming them. In order to highlight the authors' assumptions effectively, we will examine the localization assumptions, i.e. question where (with regard to the computer environments and with regard to the users' prior knowledge, skills, biases etc.) the authors of the individual chapters localize the sources of the barriers. To sum up, the cumulative effect of the chapters taken together is (a) to provide an overview of the types of problems for which computer environments for remote cooperation have been constructed and with which the user is confronted and (b) to present and evaluate measures to solve these problems. 8. THE CHAPTERS
Weinberger, Reiserer, Ertl, Fischer, and Mandl discuss the possible impact of scripts on collaborative learning. A script is a tool made for the facilitation of collaborative knowledge construction. The authors compare two kinds of scripts, one kind being the so-called epistemic script. Here questions about the learning task guide the learner to make relevant contributions and put them on the communication platform. A second kind of script is the social cooperation script. Here the script allocates certain roles to learners, e.g. the role of a tutor. The implementation of the epistemic scripts is based on a meaning barrier presumption (more specifically the epistemic barrier). The implementation of the social cooperation script is based on an interaction structure barrier presumption. How do Weinberger et al. see the impact of computer environments on the meaning and structure barrier? The starting point of their chapter is the observation that learners in CSCL scenarios are quite often overtaxed. They have to fulfill too many demands which distract them from actual work on the learning material. They have to adjust to the learning environment, which delays the system's reactions. Weinberger et al. localize some causes for barriers in the differences between faceto-face interaction and computer-mediated interaction. They argue that computermediated interaction generates problems due to a reduction of information which does not occur in face-to-face interaction. In addition, Weinberger et al. suggest a further localization assumption which also can be found in other chapters of this book: learners in their settings under study have little experience and practice with computer-supported cooperative learning. Such learners should be instructed clearly what their role is, when they should contribute, and how to indicate what they are referring to. These structure barriers are not only found with computer-mediated environments. Structures for social interactions have to be established anew in every (!) instruction scenario if and when new groups form. Thus the question arises whether the scripts being examined here will remain necessary when computermediated cooperative learning has become an everyday occurrence, like group work in the classroom. Pfister too tests a special variant of scripts in a text-based learning environment. His technical environment is intended to be used when a group of learners discusses an external representation (a text, drawing or illustration) with the assistance of a tutor. Scripts (referred to as learning protocols) define the didactic functions of statements,
INTRODUCTION AND OVERVIEW
7
i.e. every participant must classify hislher type of utterance as a question, answer etc. The software also makes it possible for participants to refer their contributions explicitly to earlier statements. Referencing new contributions to earlier information is made graphically visible. The empirical investigation examines the effect of three types of protocols which were designed to support coordination and formation of coherence: the explicit reference of contributions, their type classification, and deciding on an order in which learners have to contribute. Hence Pfister's learning protocols are seen as an appropriate measure to overcome the structure barrier. He links this structure barrier presumption with a meaning barrier presumption: only if participants understand how new contributions refer to the contributions made earlier common ground can be established. How does Pfister perceive the impact of the computer environment on the meaning barrier and the structure barrier? Just as the aforementioned authors, he localizes the source of the difficulty in establishing common ground in the features which in asynchronous and text-based communication impede the rapid repair of misunderstandings. Co-presence, instantaneousness, and simultaneity are the missing elements. To this extent it is a computer-oriented localization: the specific features of the computer environment create common ground barriers. However, the reduction of communication in such settings is not only the source of problems. Pfister localizes the solution there, too. He maintains that it is the reduction in the degrees of freedom which helps to overcome the common ground and structure barriers. From this point of view the computer environment both creates and compensates for the barriers of meaning and structure.
Rummel and Spada deal with a computer mediated setting in which a complex task has to be solved cooperatively on the basis of complementary domain knowledge. The collaborating partners work in a synchronous cooperation environment and they are able to see and hear one another via an audio-video link and can use a shared text editor. Advanced students of psychology and medical students work together on a case that contains psychological as well as medical aspects. They have to contribute their complementary expertise and agree on joint problem-solving strategies. The central topic of this chapter is the acquisition of competence in interdisciplinary cooperation, the question of how one can best learn to cooperate: via observational learning (participants watch a video showing a best practice example of working on a task demanding knowledge from different domains), via a script (similar to but more detailed than the organizational script which Weinberger et al. use) or by means of trial and error (unscripted problem solving). The authors discuss the meaning barrier, the unshared knowledge barrier as well as the structure barrier. The emphasis in the empirical analyses is on learning effects when confronted with problems of interaction structure: how do participants decide when to work on their own and when together? Learners in the script condition and learners in the observational learning condition receive assistance in order to overcome all three barriers. The instructions are expected to reduce the time and energy spent on coordination and consequently lower the structure barrier. Additionally, participants learn how to use their complementary levels of expertise
in interdisciplinary work. The impact of the computer environment on the occurrence of such problems and coping with them is discussed in detail. Rummel and Spada emphasize that establishing common ground in interdisciplinary communication is difficult by definition. Hence they localize the source of the meaning barrier in the interdisciplinary collaboration and in its unfamiliarity for the learner (cf. Weinberger et al.). In addition, Rummel and Spada examine computerbased localizations: the video recording is not perfectly synchronous and the visual space is limited compared to what would be visible for participants in a face-to-face setting. It is also pointed out that the complexity of the task combined with the demands of computer-mediated communication cause intra- and inter-subject coordination problems and cognitive overload and consequently contributes to problems of establishing an interaction structure.
Bromme, Jucks, and Runde focus in their contribution on net-based health advice. In their scenarios medical experts reply to enquiries mainly received from laypersons. Due to the qualitatively different prior knowledge (medical expert knowledge vs. naive lay theory about medicine), communication requires exceptional efforts on the part of the expert to adapt to the level of the layperson. In this empirical study the authors also examine the influence of external representations. They ask whether the co-presence of an expert illustration possibly leads to an "illusion of evidence" on part of the experts, i.e. leads to the erroneous assumption that everything visible to everyone can also be understood by everyone. Starting point of the project is the common ground barrier proposition. The authors examine whether heuristics which serve to establish common ground in face-to-face interaction still "work" when computer-mediated communication is involved. In this investigation the different prior knowledge of the communication partners (experts and laypersons) also is of great importance because it can contribute to the difficulties during the establishment of common ground. In this respect the localization assumptions are not concerned solely with the computer environment since communication between experts and laypersons is very difficult in face-to-face interactions, too. However, in this chapter it is also assumed that in text-based asynchronous communication barriers to establishing shared meaning are raised. Moreover, Bromme et al. argue that it is computer-mediated communication which makes the barriers they are examining relevant in the first place. Since computers have become widespread, and expert information has become readily available to non-experts on the internet, experts increasingly have to answer queries from laypersons they do not personally know, e.g. via hotlines. Without the internet this could hardly occur because laypersons normally would not consult doctors they are not known to as patients. We can say, therefore, that the spread of computers is, in a sense, responsible that the scenario under investigation can be found outside research settings. One could call this the "quantitative enabling effect" of technology. It is not the technical feature of a technology per se but its widespread use which makes such a barrier part of social reality though it would in principle also exist without computers.
INTRODUCTION AND OVERVIEW
9
Anderson, Mullin, McEwan, Bal, Caerletta, Grattan, and Brundell report on a series of field studies and laboratory experiments with regard to the effect of new communication technologies on cooperative work. They deal with the size of groups, access to interfaces, and the organizational status of the communication in question. In the first study three groups are compared, each of which has the task to work out a route which has been drawn in on the so called instructor's map but not on the other group members' maps. VMC (Video Mediated Communication) is compared between groups of different sizes and a face-to-face condition. The aim of the cooperation is to achieve optimal understanding. This is the case when the participants have worked out the exact "meaning" set by the instructor. This contribution is therefore concerned with meaning and structure barriers. Another field study analyzed the mediated communication between two firms, one being a supplier (of tires), the other a manufacturer of the product in question (cars). The third study compared a communication condition in which several persons on each side share communication facilities, with a situation where each person has a computer to him- or herself. The main concern is about structure barriers which could impede smooth turn taking. Compared with the contributions described earlier quite different localization assumptions are involved: the number of cooperation partners and their status (in terms of power and responsibility for success) in the cooperation are not cognitive but organizational causes of barriers, when establishing structure and shared meaning. Furthermore, this chapter opens up a new field of enquiry: in two studies, teams cooperating on a computer-mediated task consist, on each side of the remote connection, of groups which communicate with one another face to face. So these scenarios contain both mediated inter-group communication and non-mediated intra-group communication. Situations of this kind occur relatively often (e.g. in the classroom, when several pupils share a computer). However, there has been very little research so far in this area. It remains to be clarified in what way challenges of meaning and structure are met and overcome when mediated and face-to-face communication coincide. Cress, Barquereo, Buder, and Hesse deal with conditions under which people are willing to put information into data banks. The scenario of their experiments is modeled on using databanks in a company. Participants are required to use data from a databank to complete their task, and put their results in the bank for other people's use. Research questions are asked relating to various payoff functions for putting data in the data bank. Therefore, the experiment is concerned with a motivation barrier. The authors examine for example whether the importance of the information, the amount of the reward, and the size of the group are relevant for the behavior observed. This is what is known as the "public good dilemma", a wellresearched phenomenon in social psychology. Here computer mediation does not really represent a specific cause for the motivation barrier being investigated. Nevertheless, computer mediation is an indispensable precondition for the public good dilemma with data banks. Only since the spread of computer networks in the workplace has the emergence and use of such databanks become possible. On the
other hand, there are also various factors (e.g. instant individual feedback) which only occur with computer systems. As with the computer hotlines for laypersons mentioned above, this could be called the quantitative enabling effect of computer technology. The problems of knowledge communication analyzed here are not limited to computer applications, but it is the computer which turns them into social reality. Furthermore, Cress et al. mention another indirect cause of the motivation barrier: when people use a data bank they first have to realize that a communication process is involved. This is an important point, which is also relevant for other environments. It is by no means a matter of course that computer-mediated interactions are perceived as inter-personal communication.
Kirschner and Kreijns describe in their introduction the barriers of meaning, of structure and of motivation. They suggest the term affordances sensu Gibson for the theoretical description of the impact of the computer environment on the barriers. They illustrate the effect of affordances by means of different technical artifacts. This contribution deals with scenarios of self-directed learning in school and university. Learners must be assisted with the development of a content structure and a social structure for the interactions. Since such cooperative forms of learning do not develop by themselves, even when not computer-mediated, here too the cause of barriers is not the technical environment but the unfamiliarity with the learning objectives. The authors call these barriers to cooperative and independent learning social and educational affordances. Kirschner and Kreijns also emphasize ergonomic aspects. The utilization of many computer environments is not a trivial task, and the interface may be more or less well suited to the various users. This is a further important computer-based localization of barriers but it is here discussed from a different point of view. The authors introduce a group awareness widget, a tool which produces a graphic illustration of data obtained from the social interaction of the group members during their work. The goal of using this graphic representation is to overcome motivational and structure barriers. Particularly when the group has no prior history it is important that participants receive information about group processes. In this way, participants can be motivated to take part in group communication. It is hoped that the group awareness tool will compensate for missing personal proximity. Consequently Kirschner and Kreijns localize the cause of the motivation barrier in the computer environment. Due to the missing social presence of communication partners, motivation to take part is reduced, i.e. the remoteness is responsible for the barrier. The group awareness widget compensates for motivation and structure barriers by supplying adapted information which on the other hand could not be acquired (so easily) in face-to-face interactions. Strube, Wittstruck, Thalemann, and Garg analyze communication in teams of heterogeneous expertise. Their analysis deals with the cooperation between different types of specialists producing web-page layouts. The cooperating participants with varying functions need to know how the relevant knowledge is distributed in the group. The experts involved make default assumptions, and there is in fact very little
INTRODUCTION AND OVERVIEW
11
explicit communication about already existing common ground. Hence the authors assume a meaning barrier proposition. They localize potential communication problems in the false assumptions that partners make about the prior knowledge of the other participants. Finally, the authors outline possible kinds of technical support which might compensate for such false assumptions. Computer modeling of previous implicit assumptions about task-relevant parameters could help to correct false default assumptions. This is very much in line with the previous chapter with regard to compensating for barriers by improved awareness. However, in this case it is not the awareness of group processes, but the "objective" parameters of the joint task which is targeted by the authors when discussing tools which might improve collaborative work on web design tasks.
Fischer and Ostwald examine the utilization of computer environments for cooperative work and design problems. By "design" they mean all constructive problem solutions which are concerned with the planning of artifacts. Prototypical designers are town planners and architects but also software developers. The chapter offers a basis for the classification of barriers which occur in design work. The authors distinguish between spatial, temporal, and technical dimensions, thereby providing an interesting conceptual framework for distinguishing between different types of computer environments for net-based cooperation. The authors chiefly discuss expertise of communication partners as a precondition for constructive problem solving, but also as a potential cause of communication problems caused by the unshared knowledge barrier and the common ground barrier (cf. Rummel & Spada; Bromme, Jucks & Runde, and Strube, Wittstruck, Thalemann & Garg in their chapters). They distinguish between communities of practice (COP) and communities of interest (COI). COPS are groups of practicians with the same background of knowledge who work on the same problems. They meet at intervals and exchange relevant experiences. COIs are groups with heterogeneous expertise (laypersons and experts of different expertise domains) who have come together to solve specific problems. The authors presume there to be barriers of meaning particularly with the COIs but also deal with barriers of structure and motivation. Fischer and Ostwald see the causes of barriers in characteristics of computer environments and in heterogeneous nature of the expertise. As a solution for overcoming the barrier of meaning they suggest the concept of boundary objects. These are external representations serving as points of reference in order to enable reciprocal communication. Central to their case study are domain oriented design environments, i.e. integrated systems which support communities of practice working together on designs. Design environments make it possible to create boundary objects and contextualize information. Dillenbourg's contribution deals with the localization of barriers in connection with the utilization of computer environments for communication and cooperation. He criticizes the assumption, widely held in the CSCL and CSCW community, that the greater the similarity of computer communication with face-to-face communication, the better computer environments can be utilized. He provides evidence that a
reduction in transmitted information (in contrast to the abundance of information which can be transmitted face to face) can be very useful. He also claims that some characteristics of asynchronous and written communication offer advantages which cannot be provided by video-supported or "direct" communication. His examples refer to the meaning, the structure and the motivation barriers. From a research strategy point of view, he recommends investigating the enabling side of computer technology systematically. His examples are awareness tools similar to those introduced in Kirschner and Kreijns' contribution. The tools which he and his colleagues have developed allow graphic illustrations of social interaction and the linking of participants' arguments. The system supplies such illustrations continually during work in the collaboration environment and feeds them back to the participants. Such software is used to solve the problem of lack of participation and is, therefore, based on a motivation barrier presumption. However, the technical solution - as Dillenbourg points out quite clearly - does not answer the normative and psychological questions which have to be answered in practical terms when constructing and using such awareness tools. The question arises whether the reduction in the individual participants' privacy enhances their readiness to cooperate and if this is even desirable. Hence technology can help to make implicit normative standards explicit, e.g. with regard to balancing the contributions of the various group members. Thus Dillenbourg's chapter emphasizes the dependency of barrier solutions on those standards. The contribution of Koschmann, Zemel, Conlee-Stevens, Yound, Robbs, and Barnhart also starts out from an assessment of present research activities with regard to computer-mediated learning. Koschmann et al. discuss and illustrate the usefulness of ethno-methodological approaches for the analysis of interaction in learning groups, concentrating on the meaning barrier. They examine problembased discussions in small-group tutorials held during medical training. They compare two discussions, one face-to-face and one via CSCW-software. The authors claim that the main problems which emerge from an ethno-methodological view are relatively independent of the medium employed. Considerable efforts are consistently required to make explicit the problems inherent in the subject that students are supposed to be learning. Above all, the process of transfer from an individual to a collective approach to a problem proves to be difficult. It also emerges that the methods which the participants apply in the computer-mediated environment are similar to those used in face-to-face situations. The methodological "message" of this contribution is that localization of the cause of meaning barriers can only be achieved by a detailed analysis of the constitution of meaning in the discussion process.
9. A MAP FOR THE LOCALIZATION OF BARRIERS, BIASES AND OPPORTUNITIES It has become obvious that the causes of difficulties arising from computer use in remote communication and cooperation are manifold. Taken together, the
INTRODUCTION AND OVERVIEW
13
contributions to this book provide a detailed map of the places where barriers and biases may have their source. It has also become apparent that the localization of difficulties invariably depends on theoretically based assumptions concerning the nature of barriers and biases. Working with new technologies is often difficult, simply because they are new, and because individual routines and social routines have to be established in using them. Additionally, the use of these technologies is difficult because they are not just alternative tools for dealing with old conventional problems but they are also expected to help with meeting new challenges, e.g. new forms of self-directed learning, a new way of participation by ordinary citizens, or new ways of interdisciplinary collaboration. That is why, in fact, most of the barriers and biases discussed in this book have multiple causes. Only knowing the "places" and causes of barriers and biases allows to develop successful strategies to overcome them and to gain advantage from the new technological possibilities. Computer problems are commonplace for ordinary and expert users alike. There is at present no other technical domain in modern industrial societies where errors and problems play so large a role in the public mind as computers. Older readers of this introduction will remember that a few decades ago it was similarly commonplace for users of the technical system "car" to have an understanding of its technical problems and to be able to do some small repairs themselves, probably carrying a nylon stocking in the boot in case the V-belt needed replacing. Nowadays anybody using a computer could tell a lot of stories about problems concerning histher experience with computer related problems, has some "nylonbelts" at hand in order to fix some of these problems and of course has implicit theories about the reasons for the difficulties shethe is experiencing. The combination of rapidness of technological developments and the new challenges they offer, contribute to the fact that our experience and knowledge about the nature of barriers remain mostly implicit. However, it is equally important for users, designers, and researchers that assumptions about the nature of such difficulties and their sources are made explicit and discussed. This book is intended to contribute to the explicit discussion of such tacit assumptions and to shed light on reliable measures to promote effective computer-mediated cooperative learning and work. In case of the technical system "car" such problems were solved by means of technical solutions. Remote computer-mediated communication will also be improved by technical progress but psychological and educational research will have a major impact, too. The editors hope that the contributions to this book are giving evidence of this claim. ACKNOWLEDGEMENTS We are grateful to the editor of the CSCW book series, Pierre Dillenbourg (Lausanne), who supported the concept of this book from the beginning. We would like to give special thanks to two anonymous reviewers who have provided very helpjid and detailed comments on all chapters. Many thanks also to Regina Jucks (Miinster) for comments on a draft of this introductory chapter and to Katrin Sommer (Miinster)for her support of the editorial work and for the final formatting of this book.
REFERENCES Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73 (3), 277-320. Cramton, C. D. (2002). Attribution in distributed work groups. In P. Hinds & S. Kiesler (Eds.), Distributed work (pp. 191-212). Cambridge, MA: MIT Press. Krauss, R. M., & Fussell, S. R. (1996). Social psychological models of interpersonal communication. In E. T. Higgins & A. Kruglanski (Eds.), Social psychology: Handbook of basic principles (pp. 655701). New York: Guilford Press. Ross, L., & Nisbett, R. E. (1991). Tlze person and the situation. New York: McGraw-Hill. Stasser, G., & Titus, W. (1985). Pooling of unshared information in group decision making. Biased information sampling during group discussion. Journal of Personality and Social Psychology, 48, 1467-1478.
bromme@ uni-muenster.de friedrich. heme @mi-tuebingen.de spada @psychologie.uni-freiburg.de
ARMIN WEINBERGER, MARKUS REISERER, BERNHARD ERTL, FRANK FISCHER, & HEINZ MANDL
FACILITATING COLLABORATIVE KNOWLEDGE CONSTRUCTION IN COMPUTER-MEDIATED LEARNING ENVIRONMENTS WITH COOPERATION SCRIPTS
Abstract. Collaborative knowledge construction in computer-mediated learning environments poses difficulties regarding what tasks learners work on and how learners interact with each other. Learners who collaboratively construct knowledge in computer-mediated learning environments sometimes construct inadequate conceptions of a subject and rashly build consensus regarding these conceptions. Collaborative learning tasks can be structured through cooperation scripts. It is unclear, how cooperation scripts could be designed for different tasks and different technologies for computer-mediated communication. In this chapter, two studies with a 2x2-design will be reported that applied social and epistemic cooperation scripts in computer-mediated learning environments based on web-based discussion boards and videoconferencing technologies. Results show that social cooperation scripts substantially foster the processes of collaborative knowledge construction as well as learning outcomes. Epistemic cooperation scripts facilitate the processes of collaborative knowledge construction, but have no or negative effects on learning outcomes.
1. COLLABORATIVE KNOWLEDGE CONSTRUCTION Current approaches of learning and instruction emphasize the relevance of collaborative learning environments (see Greeno, Collins, & Resnick, 1996). In these approaches collaborative learning is often both method and aim of instruction. First, collaborative learning can facilitate knowledge building processes by requesting students to engage in activities beneficial for learning when cooperatively solving a problem task or discussing and elaborating text material (see Slavin, 1995; Webb, 1989). Second, working in small groups should prepare learners for life-long learning activities, which are largely embedded in social contexts. In this way, collaborative learning should result in specific learning outcomes that are beyond what could be achieved in individual settings. The socio-cognitive perspective is probably the most elaborated theoretical framework in order to highlight and explain the benefits of collaborative learning environments (see Slavin, 1996; Webb, 1989). According to this framework, when working in small groups, learners construct knowledge by actively participating in discussion and sharing knowledge with their learning partners. From this perspective, cooperative learning aims at fostering processes of what we call collaborative knowledge construction (Fischer, Bruhn, Grgsel, & Mandl, 2002). Students ideally actively engage in learning processes when jointly working on a learning task. This is done by mutually explaining the learning contents, giving
16
WEINBERGER ET AL.
feedback to contributions of their teammates, asking and answering questions etc. There is a broad understanding that the specific benefits of collaborative knowledge construction are strongly linked to these specific processes. Numerous studies indicate, however, that learners do not spontaneously engage in productive processes of collaborative knowledge construction, and consequently, the desired effects often fail to emerge (see Cohen, 1994; Mandl, Gruber, & Renkl, 1996). These difficulties can be particularly found on two major process dimensions of collaborative knowledge construction. On one hand, learners' difficulties in collaborative knowledge construction can be related to a social process dimension. A social process dimension of collaborative knowledge construction refers to the interactions of learners with their learning ,partners. Social interactions may be suboptimal with respect to the goal of knowledge acquisition. For instance, discussions remain at a superficial level (Coleman, 1995; Linn & Burbules, 1993), and learners may try to quickly come to a consensus rather than critically refer to each others' contributions (Teasley, 1997). On the other hand, problems can be linked to an epistemic process dimension. An epistemic dimension refers to the tasks learners are confronted with, e.g., categorizing or defining new concepts (Fischer et al., 2002). That means, that learners appear to have difficulties in dealing with the learning task. They may disregard important aspects of the learning material and try to make sense on grounds of their prior knowledge only, instead of applying new concepts to the problem task (Hogan, Nastasi, & Pressley, 2000; Salomon & Globerson, 1989). Based on Vygotsky's (1978) perspective of collaborative knowledge construction as an internalization of processes on a social level, the epistemic activities during the social processes may have effects on how knowledge is acquired collaboratively. Both social and epistemic process dimensions need to be considered in order to analyze and facilitate collaborative knowledge construction. 2. COMPUTER-MEDIATED LEARNING ENVIRONMENTS Distant learners work together on tasks and communicate through computer-based media in order to individually acquire knowledge. For instance, learners are expected to contribute their individual perspectives and resources, as well as to comment on each others' perspectives in a shared workspace, which they can access via the internet. In computer-mediated learning environments ideas and questions of learners can be represented in a central database (Scardamalia & Bereiter, 1996). Computer-based media may therefore build a specific context for collaborative knowledge construction. Computers provide different communication modes with various technical and non-technical delays (Weinberger & Mandl, 2003). Information may be conveyed as text or as picture, for instance. Some computerbased media can be categorized as text-based (e.g., email, chat), whereas others are audio-visual (e.g., videoconferencing). Messages in computer-mediated communication can be recorded and stored for later retrieval. Therefore, some forms of computer-mediated communication enable so called asynchronous communication. The discussants are not expected to interact at the same time, but a non-technical delay between the individual discourse
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
17
activities may take place. This means that discussants receive and record messages, and respond to them at a later, more convenient time. Computer-mediated communication nowadays suffers only little technical delay. Messages are sent off and are almost at the same time received. This enables a discourse, which has been described as synchronous., i.e. the communicants are supposed to participate in textbased or audio-visual computer-mediated communication at the same time. In this chapter, we will present studies that aim to analyze and facilitate asynchronous, text-based communication built on web-based discussion boards as well as synchronous, audio-visual communication built on videoconferencing in computer-mediated learning environments. On one hand we aim to overcome specific barriers of computer-mediated communication for collaborative knowledge construction. The barriers we focus on are the reduction of exchanged information and the increase of coordination demands in computer-mediated communication in comparison to face-to-face communication. Therefore, learners may aim to reach consensus more quickly in computer-mediated communication than in face-to-face communication and only superficially deal with the learning tasks. On the other hand, we aim to exploit the potentials of different computer-based media to support processes of collaborative knowledge construction. Specific features of computermediated communication may foster the quality of collaborative knowledge construction. In particular, learners may have the chance to participate more actively and better reflect upon text-based communication than may be possible in face-toface classroom talk (Scardamalia & Bereiter, 1996). In order to appropriately use these potentials, we will systematically consider both social and epistemic process dimensions of collaborative knowledge construction in the context of different computer-mediated learning environments. 3. FACILITATING COLLABORATIVE KNOWLEDGE CONSTRUCTION
Dillenbourg (2002) distinguishes two different ways to facilitate collaborative learning. Taking a condition-oriented approach, teachers can indirectly influence the effectiveness of collaboration by arranging basic conditions like the group size, the group task or the communication media. In contrast, process-oriented approaches aim at directly influencing the interactions of group members by giving appropriate instructions. There are a number of disadvantages of condition-oriented approaches compared to process-oriented approaches. First of all, condition-oriented approaches may be more dz..cuZt to design. Condition-oriented approaches aim to facilitate the processes of collaborative knowledge construction indirectly. The rationale of this approach is, that when the basic conditions are set, the relevant processes of collaborative knowledge construction will emerge. The number of basic conditions relevant to collaborative knowledge construction, however, may be high, and mutual dependencies between these conditions are complex (cf. Dillenbourg, 1999). For instance, the effects of incentive structures on collaborative knowledge construction particularly depend on the complexity of the learning task, with the complexity of the learning task
18
WEINBERGER ET AL.
influencing what kind of processes are beneficial to knowledge construction (cf. Cohen, 1994). Therefore, it may be complicated to arrange all the conditions optimally to foster collaborative knowledge construction. Second, condition-oriented approaches may be more costly. For instance, prior knowledge and experience in collaboration has been identified as a central basic condition of collaborative knowledge construction and thus, has been subject to cooperation training. Some of these training programs, however, take more time than the actual collaboration of learners (cf., Weinberger & Mandl, 2003). These costs may make the application of condition-oriented approaches less likely and less useful in educational practice. Furthermore, some basic conditions of collaborative knowledge construction cannot be arranged at all. Mandl and colleagues (1996) note, for instance, that examination regulations typically disregard knowledge and competencies particularly fostered by collaborative knowledge construction. Typically, students need to memorize theoretical concepts in order to pass exams rather than to reflect and defend multiple perspectives on a complex subject matter. Consequently, students aim to avoid the costs of collaborative knowledge construction since its specific benefits are not requested in conventional examinations. Therefore, process-oriented alternatives to facilitate collaborative knowledge construction may need to be considered. Process-oriented approaches may be more feasible, because they apply during the collaborative processes and because they can aim to directly facilitate specific activities and interactions of learners. Still little is known with respect to how process-oriented facilitation can be applied. Some process-oriented approaches, e.g., moderation of collaborative processes, may require complex skills and their success depends highly on the quality of the individual facilitator (cf. Clark, Weinberger, Jucks, Spitulnik, & Wallace, 2003). Cooperation scripts, however, have been regarded as a qualitatively consistent possibility to directly facilitate collaborative learning activities (cf. O'Donnell, 1999). Cooperation scripts aim at facilitating processes of collaborative knowledge construction by suggesting a structure to learners' collaboration. Cooperation scripts specify, sequence, and assign activities to collaborative learners. Specifying activities should help learners to produce activities which are beneficial to collaborative knowledge construction and to avoid activities which may be detrimental. Typically, a teacher specifies activities, which are believed to facilitate knowledge construction, prior to a collaborative phase of learners. For instance, teachers introduce students to the collaborative learning strategy of question asking. Subsequently, learners are expected to engage in the specified activities in the collaborative phase. Furthermore, sequencing of activities supports students in engaging in the specified activities. The specified activities may be beneficial for collaborative knowledge construction only when they are applied at specified times. In this way, interactions of learners may be organized to build sensible discourse structures. For instance, after question asking, the sequence of a script may suggest to answer questions as the next step. Therefore, sequencing may support learners to better relate to each other and support critical discourse. Assigning activities aims to warrant that the specified activities are carried out by all learners. This typically includes that learners are expected not only to engage in one specific activity, but to
SCRIVTS FOR COMPUTER-MEDIATED LEARNING
19
take turns in assuming responsibility for various specified activities. For instance, one learner may be assigned the activity to ask questions regarding one specific problem and another learner may be expected to answer those questions. Then, these learners may switch their roles to work on a subsequent problem. Cooperation scripts have been studied extensively in face-to-face contexts. Recently, cooperation scripts have gained more and more importance for the design of computer-mediated learning environments, for which the disadvantages of condition-oriented approaches particularly apply. The computer-mediated learning environment builds a specific context in which distant learners cannot easily be prepared for online collaboration without giving up the idea of online and distance learning. This contribution highlights central assumptions and empirical findings of this field of research in educational psychology in order to utilize these findings for computer-mediated learning environments. Moreover, we will describe two studies we recently conducted in order to analyze the effects of social and epistemic cooperation scripts in different computer-mediated learning environments. 3.1 Facilitating Social and Epistemic Activities with Scripts Cooperation scripts are instructional approaches that aim at facilitating the processes of collaborative knowledge construction (O'Donnell & Dansereau, 1992). Despite this common goal, cooperation scripts can be designed in very different ways, based on various approaches, and aim at various process dimensions. Typically, cooperation scripts focus on several different process dimensions at the same time. Apart from social and epistemic activities, prototypical scripts aim, for example, to facilitate affective, elaborative, as well as meta-cognitive activities. Early attempts to disentangle the confounding of several dimensions of collaborative knowledge construction have been made with varying outcomes (Larson et al., 1985; see also O'Donnell, Dansereau, Hall, & Rocklin, 1987). Larson et al. (1985) compared effects of an elaborative and a meta-cognitive cooperation script on the quality of processes and results of collaborative knowledge construction. This comparison showed diverging effects on processes and outcomes of collaborative knowledge construction. The meta-cognitive cooperation script of this study produced a positive effect on processes, but was detrimental for individual outcomes of collaborative knowledge construction. The elaborative cooperation script, in contrast, only facilitated outcomes, but impeded processes of collaborative knowledge construction. Various studies indicate that social and epistemic processes are particularly important for specific aspects of collaborative knowledge construction (Fischer et al., 2002; O'Donnell, 1999). As outlined above, specific difficulties regarding social and epistemic dimensions of collaborative knowledge construction have been discovered. Learners appear to have problems regarding the learning task as well as regarding productive social interactions. Starting from these specific difficulties, cooperation scripts can be designed that facilitate the social and epistemic processes.
WEINBERGER ET AL.
3.2 Social Cooperation Scriptsfor Collaborative Knowledge Construction In order to overcome deficits of learners' social interactions, social cooperation scripts can help learners to structure discourse according to successful interaction patterns of knowledge construction. Successful interaction patterns can include refining knowledge through mutual critique (e.g., Doise & Mugny, 1984). The design of social cooperation scripts profoundly depends on the underlying theoretical assumptions regarding the social processes of collaborative knowledge construction and can be based on different approaches that have been developed and investigated in educational-psychological research over the last twenty years. Important approaches in this field are, for example, reciprocal teaching (Brown & Palincsar, 1989; Palincsar & Brown, 1984) and scripted cooperation (O'Donnell & Dansereau, 1992). These approaches aim to structure the interactions in groups in order to enhance the quality of learning. The instructions include the assignments of roles, which are associated with special learning tasks. In addition, the instructions prescribe a sequence of phases in which different learning activities should be applied by the learners. For example, reciprocal teaching takes up text comprehension research, which stresses the relevance of cognitive and meta-cognitive processes for effectively learning from texts. In reciprocal teaching, groups of two are built, learners are then provided with a structure for comprehending text material. This structure contains several activities in a specific sequence, which are modeled by the teacher. These activities include specific text comprehension strategies that the learners are expected to apply, namely questioning, summarizing, clarifying, and predicting. First, learners read the beginning section of a text. Subsequently, one learner takes the role of the "teacher". The "teacher's" task is to ask questions on the text that should be answered by the "student". Then, the "teacher" tries to summarize the main ideas of the text. If necessary the "student" completes missing aspects. Thereafter the "teacher" identifies difficult passages of the text and tries to clear them up in collaboration with the "student". Finally, the two learners try to predict the contents of the following text passages. Learners change "teacher" and "student" roles for further text passages in order to ensure equal involvement of all learners in collaborative knowledge construction. The adopted strategies enhance learning by facilitating the learners to engage in effective processes of knowledge construction. Similar to reciprocal teaching, scripted cooperation aims at fostering learning of students who collaborate in dyads (O'Donnell & Dansereau, 1992). Scripted cooperation typically contains the activities of reiterating, providing feedback and elaborating. By collaboratively engaging in the tasks suggested by the script, learners should construct knowledge better than "unscripted" groups when learning from texts or working on other learning tasks. In this contribution we further build on the idea of cooperation scripts that are implemented in computer-mediated learning environments.
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
3.3 Epistemic Cooperation Scripts for Collaborative Knowledge Construction Epistemic scripts provide structures not by referring to interactions of learners but by structuring the learning tasks. Epistemic cooperation scripts aim at facilitating processes of collaborative knowledge construction by assisting the learners to focus on the main topics and ideas when collaboratively discussing and constructing knowledge. These scripts often provide some kind of external representation, such as a diagram or a table that contain central, yet abstract characteristics of the content discussed during collaboration. The most prominent approaches that include epistemic scripts are, for example, guided reciprocal peer teaching (King, 1999) or content schemes (Brooks & Dansereau, 1983). In guided reciprocal peer teaching, one of the learners is supposed to supervise the collaboration with the help of prompt cards. This learner possesses several prompt cards with clauses like "What inferences can be drawn from ...?'which the learners should complete in their discourse. In this way, reciprocal peer teaching is supposed to be guided by meaningful questions about the subject matter. Suthers and Hundhausen (2001) use the concept of representational guidance, in order to explain the effectiveness of content schemes. They assume that epistemic scripts facilitate collaborative knowledge construction by focusing the learners' attention on the material to be learned. Epistemic scripts can support the group in structuring the contents to be discussed and can provide "anchors" for each learner to integrate the new knowledge.
4. IMPLEMENTING SOCIAL AND EPISTEMIC COOPERATION SCRIPTS IN COMPUTER-MEDIATED LEARNING ENVIRONMENTS There is a large body of empirical data that gives evidence for the effectiveness of the use of cooperation scripts concerning face-to-face collaboration (e.g., Rosenshine & Meister, 1994). In contrast, research in the context of computermediated environments has not only had a short tradition, but also has faced theoretical shortcomings. The latter are associated with the variety of applications of communication technologies for the design of computer-mediated learning environments. In the context of computer-mediated learning, cooperation scripts can have quite different characteristics, e.g., depending on the communication mode (synchronous vs. asynchronous) they consider and the time periods (from one hour to a semester) they cover (Dillenbourg, 2002; Weinberger & Mandl, 2003). When considering cooperation scripts in the context of computer-mediated learning (see Rummel & Spada, chapter 4 this volume for different knowledge communication contexts), a more specific question needs to be raised: How are the instructions of the cooperation scripts presented and to what degree are the learners coerced to follow a structure given by the script? Scripts can be realized in computer-mediated learning environments through interface design. This approach argues that no medium was genuinely designed for collaborative knowledge construction and thus, the interface design of the media
22
WEINBERGER ET AL.
could be modified and improved for specific collaborative knowledge construction scenarios (e.g., Baker & Lund, 1997). Media can therefore be adapted to foster collaborative knowledge construction by technically implementing support into the virtual learning environment. The development and experimental research of interface design to support collaborative knowledge construction has many practical implications. Typically, interface design strives to reduce the deficits of computermediated communication as compared to face-to-face communication. Many interfaces have been designed, for instance, to reduce coordination disadvantages of computer-mediated communication (e.g., Baker & Lund., 1997). Interfaces may also be designed to foster specific interactions found to be beneficial for collaborative knowledge construction in educational psychology (e.g., Hron, Hesse, Reinhard, & Picard, 1997). Oriented towards specific process dimensions of collaborative knowledge construction, scripts might improve online learning beyond what could be achieved even by well-coordinated collaborative learners. The implemented scripts for computer-mediated learning may have different degrees of coercion (or degrees of freedom). For instance, learners may be guided through a virtual learning environment along prescribed paths. Hron et al. (1997) sequenced the interaction of learners by alternately prompting learners to propose correction of the learning partner, explain the correction, and obtain agreement from the learning partner. Only when both partners reached agreement could they successfully access the interface and make the correction. Scripts may also be realized with prompts displayed on the communication interface. Learners are supposed to respond to these prompts and thereby, engage in the intended activities (see King, 1999). Thus, specific activities can be suggested by interface design; these suggestions may guide intended learning activities without disrupting natural interactions of learners. Scardamalia and Bereiter (1996) implemented the idea to guide collaborative knowledge construction with prompts for their ComputerSupported Intentional Learning Environment (CSILE). In this environment, learners are expected to assign different given categories, such as "problem", "what I already know", "new learning", and "my theory" to their individual messages. These message types aim to foster specific collaborative task strategies. In this way, instructional support is implemented into the computer-mediated learning environment and learners are led to engage in specific discourse activities when they collaboratively construct knowledge online. Several researchers continuously built on this idea to apply scripts in computer-mediated learning environments with the help of prompts. Baker and Lund (1997) pre-structured interactions of learners in a detailed manner by providing buttons for specific speech acts in a text-based computermediated communication interface of a virtual learning environment. The buttons are labeled with speech acts, such as "I propose to ...," "Ok," "Wait!" etc., that could be pasted into the interface and eventually completed by the user. Learners were expected to use those buttons to reduce typing demands. Some speech acts would also improve socio-cognitive knowledge construction processes and grounding. Nussbaum, Hartley, Sinatra, Reynolds, and Bendixen (2002) provided learners with a number of prompts called note starters, e.g., "My theory is ...." or "I need to understand," which students could choose when starting to write a message in text-
based computer-mediated learning environments. These note starters are implemented into the text window, which discussants use to formulate messages in online debate. The findings of this study show that note starters could encourage students to explore and discuss alternative viewpoints in comparison to discourse without structure, which is induced by interface design in text-based computermediated learning. Thus, it can be said, that prompts can have a positive effect on collaborative knowledge construction in text-based computer-mediated communication (Nussbaum et al., 2002). Examples of groupware systems show that not any kind of structure endorses interaction (Flores, Graves, Hartfield, & Winograd, 1988). For instance, tacit interaction processes may not require additional structure. The question therefore is, at what process dimensions of collaborative knowledge construction scripts should aim at? In the following sections we will describe two studies on social and epistemic cooperation scripts based on prompts in computer-mediated learning environments. The first study analyzes a text-based computer-mediated learning environment and the second analyzes a videoconferencing learning environment.
5. FACILITATING COMPUTER-MEDIATED LEARNING THROUGH SCRIPTS: EVIDENCE FROM WEB-BASED DISCUSSION BOARDS AND VIDEOCONFERENCING Based on the outlined framework, we arranged and analyzed the effects of different computer-based learning environments, which made use of social and epistemic cooperation scripts as described above. In these learning environments we applied two different communication technologies: (1) web-based discussion boards supporting asynchronous text-based communication and (2) videoconferencing which allowed synchronous communication based on sound and picture. In both of the studies we focused on the question, how can processes of collaborative knowledge construction be facilitated through social and epistemic cooperation scripts?
5.1 Study 1: Facilitating Collaborative Knowledge Construction in Text-Based Communication through Cooperation Scripts Text-based computer-mediated communication enables new collaborative knowledge construction scenarios. Distant learners may participate in asynchronous collaborative knowledge construction (for synchronous text-based computermediated learning environments see Pfister, chapter 3 this volume). The main idea of collaborative knowledge construction in text-based computer-mediated communication is, that learners engage in more active, reflective, and socially supported knowledge construction (Clark et al., 2003; Scardamalia & Bereiter, 1996). Therefore, text-based computer-mediated communication may be a suitable context for learners to jointly explore complex problems by contributing their individual perspectives in order to acquire knowledge. There are indications,
24
WE~NBERGER ET AL.
however, that collaborative knowledge construction in text-based computermediated communication may need additional support. The medium does not appear to be an efficient tool for complex learning tasks (Kiesler, 1992; Straus & McGrath, 1994). Text-based computer-mediated communication poses additional efforts for learners (e.g., navigating in a computer-mediated environment, typing, spelling, waiting for an answer), which in turn reduce the learners' capacity for actual knowledge construction activities (see Kiesler, 1992). First, these disadvantages of the medium could compromise the quality of collaborative knowledge construction with respect to how peers interact with each other. For example, learners may try to quickly come to a (possibly false) consensus rather than engage in cognitive conflict, which has been regarded as a crucial indicator for the quality of collaborative knowledge construction (Doise, 1990). Conflict orientation can facilitate the development of new knowledge structures by suggesting alternative perspectives, focussing learners on the task, and receiving new information (Doise & Mugny, 1984). Second, the disadvantages of the medium may impair how theoretical concepts are being applied. Learners may disregard important aspects of the learning material and try to make sense only on grounds of their prior knowledge, instead of applying theoretical concepts to the problem task (Hogan et al., 2000). Based on Vygotsky's (1978) perspective of collaborative knowledge construction as an internalization of processes on the social level, the application of theoretical concepts to the problem task during the processes may have effects on how knowledge is acquired collaboratively. Study 1 aimed to analyze and facilitate processes and outcomes of collaborative knowledge construction. Therefore, a social and an epistemic cooperation script have been implemented in a text-based computer-mediated learning environment with the help of prompts that pre-structured the discourse of small groups of three learners (Scardamalia & Bereiter, 1996; Weinberger, Fischer, & Mandl, 2002; 2003). The research questions of study 1 with the two scripts independently varied in a 2x2-design are: 1. What are the effects of a social cooperation script and an epistemic cooperation script on the processes of collaborative knowledge construction regarding the application of theoretical concepts to problems as well as conflict orientation in a text-based computer-mediated learning environment? With respect to this first question of the study, we assume that the epistemic cooperation script fosters the application of theoretical concepts to problems whereas the social cooperation script facilitates conflict orientation. 2.
What are the effects of a social cooperation script and an epistemic cooperation script on the learning outcome of collaborative knowledge construction in a text-based computer-mediated learning environment?
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
25
On the grounds of the theoretical framework on collaborative knowledge construction outlined above, both cooperation scripts should enhance learning outcomes.
5.1.1 Sample and design of study 1. Ninety-six students in their first semester of Pedagogy from the University of Munich participated in this study. The students participated in an online learning session about attribution theory (Weiner, 1985), which is a standard part of the curriculum. Laboratory room 1
I
Task information and
Laboratory room 2
Laboratory room 3
Case
Cnfo
ion
J Orientation map of the learning environment
Prompts of the scripts in the discussion area
Figure 1: The experimental setup with a learning group of three participants in separate rooms (upper section of the figure) and the computer-mediated learning environment with a web-based discussion board (lower section of the figure).
26
WEINBERGER ET AL.
Students had to work together in applying theoretical concepts to problems that were presented online as a text, and jointly prepare an analysis for the problems by communicating via web-based discussion boards (figure 1). They were asked to discuss three problems using the attribution theory and to jointly compose at least one final analysis for each problem. The problems portrayed typical unfavorable attribution patterns of university students, e.g., a student interpreting his failure on an important test as a lack of a talent. All groups collaborated in three web-based discussion boards - one for each problem. The web-based discussion boards provided a main page with an overview of all message headers. In this overview, answers to original messages appeared in outline form. The learners could read the full text of all messages, reply to the messages, or compose and post new messages. In the replies, the original messages were quoted with ">" as in standard newsreaders and e-mail programs. The social cooperation script aimed to foster conflict-oriented interactions in order to avoid quick and false consensus. For this reason, each student in the social cooperation script condition was assigned two roles: (a) analyst for one of the problems and (b) constructive critic for the other problems. Role (a) included taking over the responsibility for the preliminary and concluding analysis of one problem and responding to criticism by the learning partners. In their role (b) as a constructive critic, the learners had to criticize the analyses of the other problems presented by the learning partners. These activities were supported by the prompts of the social cooperation script (see table I), which were automatically inserted into the critics' messages and into the analyst's replies in order to help learners successfully take over their roles. Students were given a time limit for each of the required activities. Table I : Prompts of the social script of study 1. Prompts for the constructive critic These aspects are not yet clear to me:
We have not reached consensus concerning these aspects: My proposal for an adjustment of the analysis is: Prompts for the case analyst Regarding the desire for clarity:
Regarding our difference of opinions: Regarding the modification proposals:
The epistemic cooperation script aimed at facilitating how the learners worked through the learning task. With the help of prompts, learners were suggested to
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
27
apply theoretical concepts to problems. When composing a new message that represented the initial contribution to a discussion thread, epistemic prompts prestructured the input window (see table 2), i.e., the learner's message already contained prompts. These prompts were questions about the problem and aimed at supporting the learners' identification of the relevant problem information, application of the concepts of Weiner's (1985) attribution theory to problem information, and predictions and proposals for pedagogical interventions regarding the problem. Table 2: Prompts of the epistemic script of study 1.
Case information, which can be explained with the attribution theory: Relevant terms of the attribution theory for this case: Does a success or a failure precede this attribution? Is the attribution located internally or externally? Is the cause for the attribution stable or variable? Does the concerned person attribute himselflherself or does another person attribute himlher? Prognosis and consequences from the perspective of the attribution theory: Case information which cannot be explained with the attribution theory:
5.1.2 Processes of collaborative knowledge construction. In order to investigate the activities of the learners in the collaborative phase, the written discourse of the learners and their individual analyses have been analyzed with a multi-level coding system. With the help of this coding system, the learners' discourse and their analyses have been segmented and classified. We measured in particular, how well the learners collaboratively applied theoretical concepts to problems, to what extent they engaged in conflict-oriented interactions, and how much knowledge they acquired (learning outcome). 5.1.3 Application of theoretical concepts to problems. The task of the learner was to analyze and discuss problem cases. With respect to the application of theoretical concepts to problem cases, relations between theoretical concepts and case information have been analyzed. On the grounds of expert solutions, correct and central relations between theoretical concepts and case information have been identified within the discourse of the learners. For instance, the case information of a student who failed a test and said "I am simply not talented for it at all" needed to be explained by the subjects with the theoretical concepts of a
28
WEINBERGER ET AL.
stable and internal attribution according to Weiner's (1985) attribution theory. A subject who categorized the case information "No talent" as a stable, internal attribution applied theoretical concepts to the problem case. 5.1.4 Conflict orientation
Any response of the learners during their collaborative work on the problem cases which explicitly declined or modified statements of the learning partners was rated as conflict orientation. Thus, individual conflict-oriented segments are indicated by explicit rejections ("I think you are wrong in that"), replacements (A: "The attribution of the teacher is de-motivating."; B: "The attribution of the teacher is beneficial!"), modification (A: "The attribution of the parents is positive, because it liberates Michael of his feelings of guilt"; B: "You are right, this attribution is positive; but the reason why it is positive is that his parents continue to support Michael"), or endorsement (A: "The teacher motivates Michael by ascribing his bad performance to laziness"; B: "The teacher motivates Michael also by evaluating the attributions of his parents"). 5.1.5 Learning outcome.
Data regarding the learning outcome have been collected in a post-test, in which learners had to analyze problems individually. Similar to the application of theoretical concepts to problems as a process variable, the relations between theoretical concepts and case information in the learners' case analyses have been coded with respect to an expert solution. The sum of all correct relations between theoretical concepts and problem case, the learners constructed in the individual post-test, is taken as indicator of the learning outcome. 5.1.6 Results.
The results of study 1 (see also Weinberger et al., 2002) are that scripts may produce varying successes regarding processes and outcomes of collaborative knowledge construction. With respect to learning processes, the results show that both cooperation scripts could foster the processes of collaborative knowledge construction as was expected. Only the social cooperation script proved to support the conflict-orientation of learners substantially. The learners have been encouraged to confront their ideas with their partners' perspectives and reflect on the differences of perspectives. The epistemic cooperation script could support the application of theoretical concepts during the collaborative knowledge construction phase as expected. With respect to the learning outcome, however, only one of the scripts could facilitate collaborative knowledge construction, whereas the other produced significantly detrimental effects. 5.1.7 Results discussion.
The results show that the social cooperation script could facilitate the individual learning outcome. The epistemic script, however, had negative effects on the learning outcome, i.e. learners in this condition acquired significantly less
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
29
knowledge than learners of the control group. This can be ascribed to several reasons. While the epistemic cooperation script may have supported the participants in solving problems during the collaborative phase, it might not have fostered the internalization of concepts since important processes of learning failed to take place. The epistemic cooperation script may have limited the processes of reflective thinking about the problems. Like a checklist, it may have facilitated the identification of sub-problems and application of theoretical concepts as long as the cooperation script was available to the learners, but did not support the subjects in developing their own conceptual understanding. The results generally show that important processes and outcomes of collaborative knowledge construction can be influenced positively as well as negatively by cooperation scripts implemented into a computer-mediated learning environment with the aid of prompts that structure the learning discourse itself. 5.2 Study 2: Facilitating Collaborative Knowledge Construction in a Videoconferencing Environment through Cooperation Scripts Videoconferencing enables synchronous forms of collaborative distance learning, which are required when learners need to frequently interact. Despite these conveniences, videoconferencing so far does not play a prominent role for the design of computer-mediated learning environments. One reason, of course, are the technical demands users have to face when using systems which need high bandwidth. Another reason is the lack of concepts for distance learning based on videoconferencing. Therefore, we first conceptualized a design for a learning environment based on videoconferencing. Results from earlier studies show that peer-tutoring/-teaching settings can be realized with videoconferencing (Geyken, Mandl, & Reiter, 1998; Guzley, Avanzino & Bor, 2001). These settings are characterized by situations in which a peer-tutor directly interacts with the tutee or student when the latter faces a learning problem and therefore needs assistance. The tutor's tasks are to give explanations, or feedback, when needed, but also to ask questions in order to help the partner to finish the learning task. Peer-teaching through videoconferencing may be a particularly effective method of collaborative knowledge construction when more experienced tutors guide tutees through multiple aspects of learning material. Student tutors often lack the skills to elaborate learning material together with the tutee, however, and concentrate on only conveying theoretical concepts. There are indications that videoconferencing can further encumber peer-teaching, because learners may dedicate their attention towards the learning material in a sub-optimal way (Weinberger & Mandl, 2003). In videoconferencing scenarios, learners appear to have more difficulties to coordinate communication (Anderson et al., 1997). This impediment of coordination may be particularly problematic for peer-tutors who need to convey a coherent representation of learning material and simultaneously meet the demands of the new medium. Taking up these considerations, in study 2 we investigated a peer-teaching setting in which the learning partners collaborated via a videoconferencing system.
30
WEINBERGER ET AL.
We analyzed and facilitated processes and outcomes of collaborative knowledge construction independently varying social and epistemic cooperation scripts in a 2x2-design. The research questions of study 2 are: 1. How do a social cooperation script, an epistemic cooperation script, and their combination influence processes of collaborative knowledge construction in a videoconferencing peer-teaching setting? 2. How do a social cooperation script, an epistemic cooperation script, and their combination influence outcomes of collaborative knowledge construction in a videoconferencing peer-teaching setting?
5.2.1 Sample and design of study 2. Eighty-six students in their first semester of Pedagogy from the University of Munich took part in this experiment. A high-end videoconferencing system including audio and video and a shared application to support the dyads' knowledge construction allowed participants to verbally communicate and jointly create text material in a shared text editor at the same time (figure 2). The experiment was conducted in one session that consisted of two main phases. During an individual text acquisition phase, one learner of each dyad read a text which contained a description of the theory of genotypeenvironment-effects (Scarr & McCartney, 1983), which is part of the standard curriculum of Pedagogy. In the following cooperative learning phase this learner took the role of a tutor. Correspondingly the other learner took the role of a tutee during collaboration. Laboratory room 1
Laboratory room 2
Figure 2: The experimental setup of the videoconferencing setting with a learning group of two participants in separate rooms.
In the social cooperation script condition, the text document included instructions about the tutor- and tutee-role in order to effectively direct the learners' interactions. Learners received a pre-structured text document. This text document included a short description of the roles of tutor and tutee and directed the learners' interactions during the collaborative learning phase by defining four steps of interaction (see table 3):
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
31
Table 3: Sequences and learning activities of the social cooperation script of study 2.
Step I
step Step 3
Step 4
Tutor
Tutee
Explaining the text material
Asking comprehension questions
Explaining and typing the inSupporting the learner's activities formation received in the shared text document Elaborating on text information individually Discussing generated ideas with Discussing generated ideas with the partner and writing the results the partner in the shared text document
(1) explaining the text material (tutor) and asking comprehension questions (tutee), (2) typing the information received (tutee) and supporting the learner (tutor), (3) generating own ideas concerning the theory (tutor and tutee individually), and (4) discussing (tutor and tutee) and writing down the results of the discussion (tutee only). In the condition with epistemic cooperation script, the text document was structured in a way that it included several content-related prompts (see table 4). The epistemic cooperation script was divided into four sections consisting of two prompts each. The different sections stressed important aspects including concepts and main ideas of the theory, empirical findings, consequences and individual judgements regarding the theory. Participants were asked to generate answers to all questions and write them down in the text document. Neither theory text provided any information concerning the prompts regarding the consequences and the individual judgement. By responding to these prompts, the participants were expected to draw conclusions that go beyond the scope of the texts. Table 4: Prompts of the epistemic script of study 2. Theory
Empirical Findings
What are the most important concepts of the How was the theory examined? theory? What were the results of the empiWhat are the main ideas of the theory? rical studies? Consequences
Individual Judgement
Which pedagogical interventions can be concluded from the theory?
What do I likeldislike about the theory?
Which limits of pedagogical interventions can be concluded from the theory?
Which of my own experiences supportldo not support the theory?
32
WEINBERGER ET AL.
5.2.2 Processes of collaborative knowledge construction. In order to get insights into the processes of the collaborative situation, we analyzed the written contributions the learners typed in the shared text editor during the collaborative learning phase. For this analysis we first segmented the shared text documents in propositions, each consisting of a meaningful statement related to the learning contents. In a second step each identified unit was assigned to one of the three following categories, which followed the design of the epistemic script: (1) theory (units referring to the theory of genotype-environment-effects), (2) empirical findings (units referring to empirical evidence of the discussed theory) and (3) elaborations (units regarding consequences and individual judgements). 5.2.3 Learning outcome. We measured the individual outcome of collaborative knowledge construction on the basis of a cued recall test which covered the main contents of the read theory text about genotype-environment-effects (Scarr & McCartney, 1983). 5.2.4 Results. The results of study 2 (see also Ertl, Reiserer, & Mandl, 2002) show clear effects of both treatments with regard to processes and outcomes of collaborative knowledge construction. Concerning processes of collaborative knowledge construction, the learners generated external representations showing clear differences dependent on the experimental condition. With respect to learning processes, learners who worked with the social cooperation script obviously focused more on theoretical concepts, whereas dyads who worked with the help of the epistemic cooperation script also took into consideration the other content fields as they were outlined by the epistemic script. The differences concerning the category empirical findings were not significant. Furthermore, dyads who worked with the help of the epistemic cooperation script produced significantly more written elaborations than the learners in the other conditions. Concerning the learning outcome the social cooperation script showed the expected effects. Learners in the socially scripted conditions on the average gained higher test scores. These findings indicate the benefit of the developed social script for fostering collaborative knowledge construction in videoconferencing. In contrast, no outcome effects could be found concerning the epistemic cooperation script. 5.2.5 Results discussion. Regarding the process of collaborative knowledge construction, the effects of the epistemic cooperation script correspond with the concept of "representational guidance" as previously described. The prompts of the epistemic cooperation script seemed to assist the learners to consider not only theoretical concepts but also empirical findings and own judgments regarding the discussed theory. One important question concerns the discrepancy of process and outcome effects of the epistemic cooperation script. An explanation could be that additional
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
33
challenges induced by instructional support may distract learners from actually internalizing learning material. It is possible that the epistemic script in the videoconferencing scenario poses additional demands on the learners. Learners may rather focus on technical aspects of the epistemic script, such as filling out the text document, than to engage in the epistemic activities themselves. Epistemic activities, however, may be more directly linked to knowledge construction than social interactions. An alternative explanation might be the inadequate fit between the prompts implemented in the epistemic cooperation script and the demands required to answer the knowledge test (i.e. the outcome measure). In this way, the epistemic cooperation script was able to guide the learners to more frequently discuss empirical findings and to produce elaborations on the text material. This kind of elaboration, however, did not lead to improved individual outcomes as expected. 6. DISCUSSION
The two studies reported in this article have conceived social and epistemic cooperation scripts to facilitate collaborative knowledge construction in computermediated learning environments. Rather than arranging the basic conditions (e.g., group size), these cooperation scripts aimed to support the processes of collaborative knowledge construction directly (cf. Dillenbourg, 2002). We have analyzed how specific processes and outcomes of collaborative knowledge construction can be facilitated with specific cooperation scripts that are realized with different communication media and learning tasks. What can be learned about applying scripts to computer-mediated learning environments? First, the two studies investigated the effects of different cooperation scripts in computer-mediated learning environments. The learning environments of the studies differed not only concerning the communication media (e-mail vs. videoconference), but also in reference to the learning task (problem solving vs. text comprehension). Despite these differences the two studies had in common that they investigated similar approaches to collaborative knowledge construction and instructional supports, which were adapted to the characteristics of the respective learning task: (1) social cooperation scripts that aimed to facilitate how learners interacted with each other and (2) epistemic cooperation scripts that structured what learners discussed to handle the learning task. Our results led to similar conclusions in both of the studies despite the mentioned differences. Taken together, the results of the two studies indicate that cooperation scripts may facilitate as well as impede processes and outcomes of collaborative knowledge construction. The findings indicate in particular that in both computer-mediated learning environments the social scripts were able to enhance the processes and outcomes of collaborative knowledge construction, as was intended. Thus, social cooperation scripts may enable learners to actually exploit the aforementioned advantages of collaborative knowledge construction. In contrast, the epistemic cooperation scripts of both studies did not show any positive effect on individual knowledge acquisition. In study 1 the epistemic script actually hampered the learning outcome. Positive effects
34
WEINBERGER ET AL.
of the latter treatment, however, appeared on the level of collaborative processes. It is possible that the epistemic cooperation scripts were not designed in an optimal manner in order to guide learners to the expected outcomes. The epistemic cooperation scripts provided an approved, correct structure of categories, which implied a specific model, in which individual theoretical concepts could be collated. The learners were able to use theoretical concepts adequately with the help of the epistemic cooperation scripts. Therefore, epistemic scripts may be an interesting approach to facilitate work processes like problem-solving. Epistemic scripts might, however, hamper the construction of a coherent cognitive representation of the theoretical concepts. The process-outcome-discrepancies may have occurred due to the lack of internalization of the activities and sequences, as they were suggested by the epistemic cooperation script. This would explain why the learners could make use of the epistemic cooperation scripts as long as these scripts were available to the learners, but did not acquire knowledge individually. As a consequence, epistemic scripts may not be generally recommendable for facilitating knowledge construction. Second, it may be possible to improve the scripts to better foster collaborative knowledge construction. On one hand the cooperation scripts have proven to be one approach that is able to facilitate computer-mediated learning. Therefore, scripts may be an instructional approach towards overcoming the biases and barriers of computer-mediated collaborative knowledge construction. On the other hand, some scripts may impede internalization of important processes of collaborative knowledge construction. In this case, fading of the cooperation script could improve internalization processes. The instructional support that the cooperation scripts provide can be successively reduced and the learners are expected to adopt the suggested activities. The epistemic script has successfully supported learners' problem-solving. For the goal to support knowledge acquisition, however, facilitation of epistemic activities may be further improved by fading. Instead of being provided with an approved, correct task strategy, learners could be prompted to construct a conceptual model themselves. In this line of thought, cooperation scripts sometimes may need to make tasks more difficult for learners (cf. Reiser, 2002). This appears to be particularly relevant in computer-based learning environments. Learners may apply cognitive resources in order to coordinate the computer-mediated environment instead of focusing on the learning task and constructing knowledge together. Computer-based cooperation scripts therefore need to facilitate the actual participation in the collaborative construction of knowledge, rather than to reduce the cognitive demands of the learning task. Third, as this and other contributions (e.g., Pfister, this volume) have shown, computer-mediated learning environments are a suitable context for scripting interactions of learners. Clearly, there is further need to examine beneficial applications of cooperation scripts for computer-mediated collaborative knowledge construction. If scripts should be applied, for example, in virtual seminars, we need to understand more clearly how scripts apply over longer periods of time. Therefore, cooperation scripts may need to be applied with care. Under certain conditions, scripts may bring forward overscripting effects, impeding processes and outcomes of collaborative knowledge construction (Dillenbourg, 2002). Overscripting can mean that the instructional support of scripts may ease the learning task in an exaggerated
SCRIPTS FOR COMPUTER-MEDIATED LEARNING
35
manner, reducing the complexity of learning tasks, and hampering productive discourse of learners. This appears to have been the case regarding the epistemic cooperation scripts. Furthermore, cooperation scripts for computer-mediated learning environments may have a higher level of coercion compared to traditional instructions (Dillenbourg, 2002). Cooperation scripts in face-to-face contexts, for instance, make explicit the teacher's expectation regarding the desired interactions, but do not warrant the application of the induced strategies by the students. Cooperation scripts that are implemented into computer-mediated learning environments, however, may constrain and/or afford specific activities directly in order to facilitate knowledge construction. Some of these scripts may leave learners no choice but to follow the specific instructions, e.g., because they limit the time or the admission rights to the computer-mediated learning environment. Scripts for computer-mediated learning environments may be particularly useful, however, when they flexibly assist learners of different learning prerequisites. There are indications that scripts based on prompts in computer-mediated learning environments have a lower degree of coercion than face-to-face scripts and are typically used in a flexible manner (see Veerman & Treasure-Jones, 1999). Therefore, an important question for future research of computer-mediated learning environments is how scripts can be designed that leave sufficient degrees of freedom for learners in order to allow flexible application of the script instructions. The design of scripts for collaborative knowledge construction in computer-mediated learning environments is therefore not only a question of what specific activities the script requires, but also what the script does not instruct. REFERENCES Anderson, A. H., O'Malley, C., Doherty-Sneddon, G., Lanton, S., Newlands, A., Mullin, J., Fleming, A. M., & Van der Felden, J. (1997). The impact of VCM on collaborative problem solving: An analysis of task performance, communicative process, and user satisfaction. In K. E. Finn, A. J. Sellen, & S. B. Wilbur (Eds.), Video-mediated communication (pp. 51-74). Mahwah, NJ: Erlbaum. Baker, M. & Lund, K. (1997). Promoting reflective interactions in a CSCL environment. Journal of Computer Assisted Learning, 13, 175-193. Brooks, L. W. & Dansereau, D. F. (1983). Effects of structural schema training and text organization on expository prose processing. Journal of Educational Psychology, 75,811-820. Brown, A. L. & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. B. Resnick (Ed.), Knowing, learning, and instruction. Essays in the honour of Robert Glaser (pp. 393-451). Hillsdale: Erlbaum. Clark, D., Weinberger, A., Jucks, R., Spitulnik, M., & Wallace, R. (2003). Designing effective science inquiry in text-based computer supported collaborative learning environments. Infernational Journal of Educatiortal Policy, Research & Practice, 4(1), 55-82. Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of Educational Research, 64, 1-35. Coleman, E. B. (1995). Learning by explaining: Fostering collaborative progressive discourse in science. In R. J. Beun, M. Baker, & M. Reiner (Eds.), Dialogue and instruction: Modeling interaction in intelligent tutoring systems (pp. 123-135). Berlin: Springer. Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 6191). Heerlen: Open Universiteit Nederland.
36
WEINBERGER ET AL.
Dillenbourg, P. (1999). Introduction: What do you mean by "collaborative learning"? In P. Dillenbourg (Ed.), Collaborative Learning. Cognitive and computational approaches (pp. 1- 19). Amsterdam: Pergamon. Doise, W. (1990). System and metasystem in cognitive operations. In M. Carretero, M. L. Pope, P. R. J. Simons, & J. I. Pozo (Eds.), Learning and instruction: European research in an international context (pp. 125-139). Elmsford, NY: Pergamon. Doise, W. & Mugny, G. (1984). The social development of the intellect. Oxford: Pergamon Press. Ertl, B., Reiserer, M. & Mandl, H. (2002). Kooperatives Lernen in Videokonferenzen [Cooperative learning in videoconferences]. Unterrichtswissenschaft, 30,339-356. Fischer, F., Bruhn, J., Grisel, C., & Mandl, H. (2002). Fostering collaborative knowledge conshuction with visualization tools. Learning and Inshuction, 12,213-232. Flores, F., Graves, M., Hartfield, B., & Winograd, T. (1988). Computer systems and the design of organizational interaction. ACM Trans. on Information Systems, 6(2), 153-172. Geyken, A., Mandl, H., & Reiter, W. (1998). Selbstgesteuertes Lernen mit Tele-Tutoring [Self-guided learning through tele-tutoring]. In R. Schwarzer (Ed.), Multimedia und TeleLearning [Multimedia and telelearning] (pp. 181-196). Frankfurt am Main: Campus. Greeno, J. G., Collins, A. M., & Resnick, L. B. (1996). Cognition and learning. In D. C. Berliner (Ed.), Handbook of educational psychology (pp. 15-46). New York, NY: Macmillan. Guzley, R. M., Avanzino, S. & Bor, A. (2001). Simulated Computer-Mediated I Video-Interactive Distance Learning: A Test of Motivation,Interaction Satisfaction, Delivery, Learning & Perceived Effectiveness. Journal of Computer Mediated Communication, 6. Hogan, K., Nastasi, B. K., & Pressley, M. (2000). Discourse patterns and collaborative scientific reasoning in peer and teacher-guided discussions. Cognition and Instruction, 17(4), 379-432. Hron, A., Hesse, F. W., Reinhard, P., & Picard, E. (1997). Strukturierte Kooperation beim computerunterstiitzten kollaborativen Lernen [Structured cooperation in computer-supported collaborative learning]. Unterrichtswissenschaft, 25(1), 56-69. Kiesler, S. (1992). Talking, teaching, and learning in network groups: Lessons from research. In A. Kaye (Ed.), Collaborative learning through computer conferencing. The Najaden Papers (pp. 147-165). Berlin: Springer. King, A. (1999). Discourse patterns for mediating peer learning. In A. M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 87-1 15). Mahwah, NJ: Lawrence Erlbaum Associates. Larson, C. O., Dansereau, D. F., O'Donnell, A. M., Hytecker, V. I., Lambiotte, J. G., & Rocklin, T. R. (1985). Effects of metacognitive and elaborative activity on cooperative learning and transfer. Contemporary Educational Psychology, 10,342-348. Linn, M. & Burbules, N. C. (1993). Construction of knowledge and group learning. In K. Tobin (Ed.), The practice of constructivism in science education (pp. 91-119). Washington, DC: American Association for the Advancement of Science (AAAS). Mandl, H., Gruber, H., & Renkl, A. (1996). Communities of practice toward expertise: Social foundation of university instruction. In P. B. Bakes & U. Staudinger (Eds.), Interactive minds. Life-span perspectives on the social foundation of cognition (pp. 394-41 1). Cambridge: Cambridge University Press. Nussbaum, E. M., Hartley, K., Sinatra, G. M., Reynolds, R. E., & Bendixen, L. D. (2002, April). Enhancing the quality of on-line discussions. Paper presented at the Annual meeting of the American Educational Research Association, New Orleans, LA. O'Donnell, A. M. (1999). Structuring dyadic interaction through scripted cooperation. In A. M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 179-196). Mahwah, NJ: Erlbaum. O'Donnell, A. M. & Dansereau, D. F. (1992). Scripted cooperation in student dyads: A method for analyzing and enhancing academic learning and performance. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interactions in cooperative groups. The theoretical anatomy of group learning (pp. 120-141). Cambridge, MA: Cambridge University Press. O'Donnell, A. M., Dansereau, D. F., Hall, R. H., & Rocklin, T. R. (1987). Cognitive, sociallaffective, and metacognitive outcomes of scripted cooperative learning. Journal of Educational Psychology, 79(4), 431-437. Palincsar, A. S. & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and monitoring activities. Cognition and Instruction, 1, 117-175.
SCRIPTS FOR COMPUTER-MEDIATED
LEARNING
37
Reiser, B. J. (2002). Why scaffolding should sometimes make tasks more difficult for learners. Paper presented at the Computer Support for Collaborative Learning: Foundations for a CSCL Community, Boulder, CO. Rosenshine, B. & Meister, C. (1994). Reciprocal teaching: A review of the research. Review of Educational Research, 64,479-630. Salomon, G. & Globerson, T. (1989). When teams do not function the way they ought to. International Journal of Educational Research, 13(1), 89-99. Scardamalia, M. & Bereiter, C. (1996). Computer support for knowledge-building communities. In T. Koschmann (Ed.), CSCL: Theory and practice of an emerging paradigm (pp. 249-268). Mahwah, NJ: Erlbaum. Scam, S. & McCartney, K. (1983). How people make their own environments: A theory of genotypeenvironment effects. Child Development, 54,424 - 435. Slavin, R. E. (1996). Research for the future. Research on cooperative learning and achievement: What we know, what we need to know. Contemporary Educational Psychology, 21.43-69. Slavin, R. E. (1995). Cooperative learning: theory, research, and practice (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall. Straus, S. G. & McGrath, J. E. (1994). Does the medium matter? The interaction of task type and technology on group performance and member reactions. Journal of Applied Psychology, 79(1), 8797. Suthers, D. D. & Hundhausen, C. D. (2001). Learning by constructing collaborative representations: An empirical comparison of three alternatives. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), European perspectives on computer-supported collaborative learning (pp. 577-592). Maastricht, NL: University of Maastricht. Teasley, S. (1997). Talking about reasoning: How important is the peer in peer collaboration? In L. B. Resnick, R. SIlj6, C. Pontecorvo, & B. Burge (Eds.), Discourse, tools and reasoning: Essays on situated cognition (pp. 361-384). Berlin: Springer. Veerman, A. L. & Treasure-Jones, T. (1999). Software for problem solving through collaborative argumentation. In P. Coirier & J. E. B. Andriessen (Eds.), Foundations of argumentative text processing (pp. 203-230). Amsterdam: Amsterdam University Press. Vygotsky, L. S. (1978). Mind in society. The development of higher psychological processes. Cambridge: Harvard University Press. Webb, N. M. (1989). Peer interaction and learning in small groups. International Journal of Educational Research, 13,21-39. Weinberger, A., Fischer, E, & Mandl, H. (2002). Fostering computer supported collaborative learning with cooperation scripts and scaffolds. Paper presented at the Conference on Computer Support for Collaborative Learning (CSCL), Boulder, USA. Weinberger, A,, Fischer, F., & Mandl, H. (2003). Gemeinsame Wissenskonstruktion in computervermittelter Kommunikation: Wirkungen von Kooperationsskripts auf den Erwerb anwendungsorientierten Wissens? [Collaborative knowledge construction in computer-mediated communication: Effects of cooperation scripts on acquisition of application-oriented knowledge]. Zeitschrift fiir Psychologie, 21 1(2), 86-97. Weinberger, A. & Mandl, H. (2003). Computer-mediated knowledge communication. Special Issue: New Media in Education. Studies in Communication Sciences, 81-105. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92,548-573.
a. weinberger@ iwm-krnrc.de markus.
[email protected] ertl @emp.paed.uni-muenchen.de
[email protected] [email protected] H A N S - R ~ I G E RPFISTER
HOW TO SUPPORT SYNCHRONOUS NET-BASED LEARNING DISCOURSES: PRINCIPLES AND PERSPECTIVES
Abstract. In this paper, the potential of synchronous net-based learning discourses as a special case of computer supported cooperative learning (CSCL) is analysed. Discourses among learners, or among learners and tutors, can significantly improve understanding of complex subject matter. However, netbased discourses are often suboptimal, since technical restrictions constitute barriers to an efficient exchange of knowledge. Support of net-based discourses may proceed either on the macrolevel of the overall discourse structure, or on the microlevel of single contributions. As a central microlevel activity the grounding of contributions is identified. Two approaches to assist learners to ground the discourse are discussed, and the learning protocol approach is illustrated in detail as a method to foster sufficient grounding on the microlevel in net-based learning discourses. As a generalization, a cost-benefit framework is proposed which portrays grounding activities as trade-off decisions between the conflicting goals of minimizing effort and maximizing understanding. Finally, some conclusions for the construction of learning environments designed for cooperative learning by discourse are suggested.
1. LEARNING DISCOURSES A common assumption in educational psychology maintains that learning by discourse among a group of learners is appropriate and beneficial when the learning goal goes beyond the acquisition of simple facts or skills, i.e., when a deeper understanding of complex subject matter, a critical reflection and evaluation of principles, arguments, and theories is aimed at. As far as a learning discourse goes beyond mere informal and casual discussion, it can be characterized by features such as number of participants, type of participants, time constraints, the knowledge domain, and, especially, by a set of conventions and rules according to which the discourse is supposed to proceed. The study of learning discourses, especially the problem of how to support discourses to become more efficient, has been intensified with the emergence of computer-technology, especially with the general availability of net-based online communication tools. On the one hand, there has been much hope that sophisticated communication technologies will not only augment but actually enhance or even revolutionize online learning, particularly with respect to computer supported collaborative learning (CSCL), as launched by Koschmann (1996). On the other hand, the actual usage and success of such learning scenarios has been slightly disappointing, concerning acceptance of its users, i.e., learners and teachers, as well as concerning educational outcomes. In this paper, some possible prerequisites as well as some barriers of successful net-based learning are discussed, and a framework which provides some guidelines on how to overcome these barriers will be proposed.
PFISTER
1.1 Prototypical Discourse: Face-to-Face In the following, a few helpful and common distinctions will be introduced. First, we will focus on cooperative learning (the term 'collaborative' will be used synonymously) as opposed to individual learning. Cooperative learning can be loosely defined as a learning method involving a group of learners who exchange knowledge andlor solve a problem together and interdependently, i.e., under a common learning goal. Whenever the participants of a cooperative learning group are physically present at the same place and time, this is called face-to-face learning, be it with or without the use of computers and networks. Whenever the participants are separated physically, and communication is mediated by some sort of computer technology (primarily: the internet), this is called net-based learning. With respect to time, a common distinction is between synchronous communication with participants communicating at the same time (e.g., video conference), and asynchronous settings with participants exchanging information at different times (e.g., email), though this is rather a continuum and not a dichotomy. Finally, with respect to synchronous collaboration, a combination of constructive and discursive processes is usually employed; we will focus on the discursive aspect and will not deal with constructive aspects such as building an artifact together. Implicitly or explicitly, face-to-face cooperative learning serves as a standard of comparison in the evaluation of net-based cooperation (Clark & Brennan, 1991; Dennis & Valacich, 1999). The rationale why discursive learning is beneficial and advantageous compared to individual learning, however, remains unchanged when moving to the net-based situation. The basic assumption is that knowledge will be more elaborated when learners need to make their knowledge (also, their lack of knowledge) explicit, and when they need to negotiate knowledge via arguments and justifications with other participants during a discourse. As a result, more elaborated knowledge will lead to deeper understanding, to better retention, and is more easily transferred and applied (Anderson, 2000; Fischer, 2002; King, 1999; Slavin, 1995). A shortcoming of this elaboration principle is the restriction on individual cognitive changes that might result from cooperative learning, it does not specifically address cognitive processes of the group as a whole. Hence, though the method of cooperative learning concerns the interaction among several persons, the main target is on the individual's cognitive structure and performance; for approaches which deal with the group as a unit see Wegner (1987) or Hinsz, Tindale, and Vollrath (1997). Unfortunately, efficient learning discourses among students usually do not occur spontaneously (see Kirschner & Kreijns, this volume). Without assistance, just learning together, i.e., talking together, will not trigger elaborative processes automatically, and thus will generally not improve learning; this holds for face-toface discussions (King, 1999), and even more so for net-based chat (Herring, 1999). A discourse setting provides an opportunity, but needs further systematic support to promote better learning. So the question is how to support groups of learners to communicate efficiently. From a technological point of view, Jermann, Soller, and
SUPPORTING NET-BASED DISCOURSES
41
Miihlenbrock (2001) distinguish three types of support systems: 'mirrors', which reflect the users' actions directly on the interface; 'monitors', which present highly aggregated interaction data such as communication patterns; and 'advisors', which give explicit advice to the users how to proceed according to a theoretical model. From a didactical perspective, we propose a similar distinction between enabling, supporting, and advising systems, yielding increasing support for learners. Enabling systems simply provide technological opportunities, i.e., tools, which learners may use or may not use. A familiar example is Microsoft NetmeetingTM,which offers a set of tools such as chat, shared whiteboard, and video conferencing, but without any didactical guidance (see also Roseman & Greenberg, 1996). Supporting systems provide some kind of pre-designed guidance derived from didactical principles, either focusing on formal aspects of the discourse process, or focusing on structuring the subject matter itself; this is either achieved via instruction or via implementation (Baker & Lund, 1997; Hron, Hesse, Cress, & Giovis, 2000). Advising systems try to provide competent recommendations for learners based on an analysis of ongoing interactions (Ayala & Yano, 1998), or try to adapt in an intelligent way to the learning status of the participants. We will concentrate on supporting systems for learning discourses, and discuss the prerequisites of such systems to achieve efficient learning outcomes. 1.2 Levels of Discourse In order to support learning discourses, face-to-face as well as net-based, one needs to establish structure, i.e., systematic regularities that guide and constrain the ongoing discourse and suppress other irrelevant interactions, thus causing more elaborative activities and accomplishing better learning outcomes. Structure may be introduced on many levels, but by and large, a macrolevel and a microlevel can be distinguished. 1.2.1 The Macrolevel Looking at a discourse as a whole, even naturally occurring learning discourses divide into typical phases. As a simple example take the typical phases a tutor might follow to introduce a well-defined topic: Starting with an introduction, he or she might present an overview, define the central concepts, followed by questions and explanations, and finally finish with a closing summary. Detailed analyses of such segmentations yielded a number of discourse types, which appear useful for specific didactical scenarios. In scripted cooperation (O'Donnell, 1999; O'Donnell & Dansereau, 1992), participants typically work in pairs to comprehend a piece of text about a scientific topic. The script starts with one learner reading the text, then recalling and summarizing it, followed by the other learner adding missing points and criticizing imprecise issues. Eventually, both learners elaborate on the topic and obtain a common understanding. Roles then alternate, and another section of text is read. King (1999) proposes a set of discourse patterns based on a sequence of predefined question types. For example, a comprehension discourse pattern proceeds
42
PFISTER
from comprehension questions to comprehension statements, definitions, or descriptions, to corrections and details. Students are trained to utilize these question types and then follow a number of stages defined by their question types (King, 1998, 1999). Other approaches are reciprocal teaching (Brown & Palincsar, 1989) or scaffolding (Collins, Brown, & Newman, 1989; Fischer & Mandl, 2001). Generally, any such approach defines a succession of phases, defined by specific learning goals, or tasks, and the group of learners works through these stages, possibly with changing roles (see Weinberger et al., this volume). Although it is still somewhat unclear which knowledge domains fit to which of these structuring methods, empirical evidence clearly shows that structuring is superior to unstructured discourse in most cases (Derry, 1999). 1.2.2 The Microlevel A discourse can be analyzed as consisting of atomic elements, i.e., the utterances of its participants, which must be constructed, formulated, and understood. To describe the functioning of single utterances, we draw on Clark's theory of grounding (1996). Very briefly, grounding according to Clark (1996; Clark & Brennan, 1991; Clark & Schaefer, 1989) is the process of collaboratively establishing common ground during communication. Common ground is a shared basis of knowledge among participants, which they mutually take for granted in an ongoing discourse (see Bromme, Jucks, & Runde, this volume). The basic component of grounding is a 'contribution', a two-part exchange of utterances. First, in the presentation phase, the speaker presents some statement, expecting the addressee to respond with evidence that he or she has understood. Second, in the acceptance phase, the addressee provides evidence that she or he has understood, or, otherwise, initiates a new contribution to clarify what has not been understood, eventually trying to establish common ground. Though several mechanisms are available to ground a contribution - such as acknowledgements, repairs, non-verbal signals -, it is not guaranteed that grounding is successful each time in any situation.
1.2.3 The Grounding-First Principle With respect to a successful learning discourse, macrolevel and microlevel are not independent. A discourse designation which characterizes the type of discourse on the macrolevel, for example, specifying a discourse as centered mainly around explanations or around argumentation, to a large extent defines the types of contributions on the microlevel. In an discourse centered around explanations, contributions such as question-explanation adjacency pairs are substantial; in an argumentation discourse, contributions such as claim-critique adjacency pairs will dominate. More important, the discourse type defines what Clark (1996) calls the grounding criterion, i.e., the degree or strength of grounding necessary for current purposes. A casual exchange of opinions might imply a lower criterion than does the explanation of a complex issue. Depending on the grounding criterion set on the
SUPPORTING NET-BASED DISCOURSES
43
macrolevel, grounding activities on the microlevel will be more or less intense and elaborated (see section 3 for details). This argument implies that macro-/microlevel dependency is asymmetric, i.e., discourse phases on the macrolevel will finish successfully if and only if grounding on the microlevel succeeds sufficiently. As the microlevel is essentially concerned with grounding, the macrolevel is concerned with structure. However, since cooperative learning involves successful transfer of knowledge or construction of knowledge among several learners, this implies that cooperative learning is equivalent to the accumulation of common ground. We call this the grounding-first principle: Supporting a learning discourse by imposing structure on the macrolevel will only be accomplished if grounding of the elementary contributions on the microlevel of a discourse is warranted. This applies to face-to-face as well as to netbased discourse settings. An important difference between face-to-face and net-based communication, however, hinges on the relation of macro- and microlevel. In face-to-face discourse, grounding (microlevel) occurs spontaneously and automatically, and without substantial deliberate effort. Even though computers are these days almost everywhere, the average person nevertheless is habituated to face-to-face discourse as the standard of everyday communication. Mechanisms for grounding, such as background acknowledgements ("mh, mh, ..."), deictic gestures, attention signals (e.g., eye contact), facial expressions, and many others, are well learned and operate autonomously. These mechanisms, unfortunately, do not transfer to net-based communication. Net-based communication, from simple email to highly advanced video conferencing tools, suffers from a number of shortcomings, which are usually attributed to the technical particulars of the communication device. Three major features, due to which grounding proceeds naturally in face-to-face communication, are either missing or only incompletely realized using net-based technologies (Bromme & Jucks, 2001; Clark & Brennan, 1991; Dennis & Valacich, 1999): 1. Copresence: In face-to-face discourse, speakers, addressees, their contributions, as well as the most of the objects the discourse is about (textual symbols, pictorial presentations, etc.), are physically present, i.e., perceptible and evident for all participants. This makes referencing, e. g., pointing to an object or to a symbol on a blackboard in order to indicate the meaning of what one is saying, a seamless and natural process. In a netbased setting such as a chat room, however, only the textual utterances are copresent, i.e., visible for everybody, but any person or artifact a speaker is referring to is out of sight. 2. Instantaneousness: Speakers are immediately aware of positive or negative feedback concerning their contributions, i.e., there is no time lag between presentation and acceptance phase. The technical distinction between synchronous and asynchronous communication tools does not completely map instantaneous versus delayed feedback; even in synchronous settings such as chat rooms or video conferencing, a temporal or spatial interval
PFISTER
3.
usually exists between the utterance of one participant and the corresponding utterance of the addressed person. If this interval is nonempty, i.e., filled with other informative events, the speaker's attention might miss the utterance corresponding to his previous presentation completely, causing high amounts of ungrounded contributions. Simultaneity: More than one communication channel can be active at one time in face-to-face communication, which implies that a discourse can proceed with several symbol systems in parallel. For example, a tutor can show a movie, talk about the movie, and point to elements of the movie all simultaneously. Depending on the technology used, in net-based scenarios there is either only one channel (chat), or several channels which often are not precisely synchronized (audio-video conferencing), or a set of tools is available (chat plus whiteboard), but can only be dealt with in an alternating fashion.
All net-based technologies currently available have to cope with deficiencies concerning copresence, instantaneousness, and simultaneity. As a consequence, grounding cannot proceed smoothly and spontaneously as in face-to-face discourse. According to the grounding-first principle, any approach to support net-based learning discourses has to make sure that grounding does occur sufficiently on the microlevel, prior to imposing structure on the macrolevel. We distinguish two basic paradigms that aim at an enhancement of grounding in learning environments: First, the augmentation paradigm is shortly discussed but discarded as a feasible solution, second, the reduction paradigm is outlined, and one prototype from our own research is described in more detail. 2. VARIANTS OF SUPPORT
Taking the grounding-first principle as a starting point, the question is: How can we support and foster grounding processes in net-based learning environments? If, as outlined above, grounding occurs spontaneously in face-to-face conversation, one direction should be to augment net-based environment towards more similarity with face-to-face situations. On the other hand, a second direction could be to establish grounding processes which do not occur spontaneously by control mechanisms built into the learning system itself. 2.1 The Augmentation Paradigm
The augmentation approach aims at extending the learning environment in such a way that it resembles a face-to-face situation as closely as possible. This does not necessarily mean to achieve full-fledged virtual reality environments, which at present still is a technological fiction. The objective is to implement a user-interface metaphor which is functionally equivalent to those activities in face-to-face situations that are relevant for successful grounding. One example in this direction is the virtual room metaphor (Greenberg & Roseman, 1998; Pfister, Schuckmann,
SUPPORTING NET-BASED DISCOURSES
45
Beck-Wilson, & Wessner, 1998; Pfister, Wessner, Beck-Wilson, Miao, & Steinmetz, 1998), as will be illustrated with the VITAL, system (Pfister, Schuckmann, et al., 1998; Pfister, Wessner, & Beck-Wilson, 1999). In VITAL, some essential properties which guide behavior in real rooms are mapped to virtual rooms: (a) boundaries: entering a room, and leaving a room, respectively, indicate joining and leaving a learning session, (b) social awareness: direct perception of indicators of all participants who currently are in a room, (c) object persistence: any artifacts in a room persist over time independent of any person's presence, (d) 'same room same view' property: people in a room at the same time have the same view of artifacts, activities, and communications occurring in that room, (e) distinctive room types: special types of rooms such as discussion rooms or auditoriums encourage special types of learning activities, such as listening, discussing, or reading. The implementation of these properties in the prototype VITAL is illustrated in Figure 1. For example, the boundaries of a room are defined as separate windows; awareness is provided by pictograms showing all participants currently in a room, including their roles, as well as by tele-pointers indicating the current focus of attention; distinctive room types are indicated with a label and functionally differentiated by access rights for different resources depending on one's role, for example, who is allowed to speak and to control turn-taking.
Figure I . The cooperative learning environment VITAL.
The maior drawback of this kind of system, however, remains even if virtual reality is achieved: as a simple enabling system, the genuine learning discourse is not supported. It is completely up to the participants how to ground and how to structure the learning process, if at all (see Kirschner & Kreijns, this volume). Hence, insufficient grounding, implying insufficient coherence of dialogues and deficient knowledge transfer, is not automatically overcome with augmented learning environments.
PFISTER
2.2 The Reduction Paradigm In this section the reduction paradigm will be outlined in more detail. Instead of augmenting the learning environment towards reality, the rationale is to reduce the learning environment to its primary purpose and function, i.e., to support an efficient learning discourse. We now aim at an explicit and intentional grounding largely controlled by the system itself. This, on the one hand, implies to reduce the system's interface to only those functions relevant for discourse, but, on the other hand, it also means to implement additional functionalities that guide and regulate the learners' communicative behavior on the microlevel. This line of thinking is similar to the idea of not trying to mimic face-to-face learning, but to exploit the specifics of the medium to overcome its own barriers (see Dillenbourg, this volume). An instantiation of the reduction paradigm is the learning protocol approach (Pfister & Muhlpfordt, 2002; Pfister, Muller, & Muhlpfordt, 2003), which aims at the implementation of structured discourses at the macro- as well as at the microlevel. Learning protocols are defined as implemented scripts for cooperative learning, i.e., a set of rules and constraints for performing learning discourses in a net-based environment, enforced by the system. Presently, the technical realization is based on synchronous text-based (chat) tools (Pfister & Muhlpfordt, 2002), though the approach can be generalized to asynchronous applications as well. Given a group of learners which are able to communicate via chat, a learning protocol controls the discourse in a way to ensure sufficient grounding and, as a result, to increase the likelihood of successful knowledge acquisition. On the macrolevel, a learning protocol is a specific instantiation of a cooperation script or, more precisely, of a discourse type suitable for some learning goal. For example, if the learning goal is to acquire basic knowledge of a new domain, and to understand central concepts and relations, an 'explanation protocol' might be appropriate. During an explanation discourse, learners will ask questions and provide explanations for each other, or, possibly, a tutor might be involved and give special explanations (Plbtzner, Dillenbourg, Preier, & Traum, 1999). In a 'summarizing protocol', to give another example, a group of learners will read a piece of text, one learner will summarize the text, others will add to and correct the proposal, and eventually the group will converge on a consensus summary (O'Donnell & Dansereau, 1992). As can be seen from these examples, on the macrolevel a learning protocol is just an instantiation of a didactically valid method to regulate and structure a learning discourse; the main difference is that structural aspects such as a sequence of phases, special roles assigned to the participants, and a typical learning goal with subgoals for the discourse phases, are supported or even largely controlled by the system itself, instead of being imposed by a teacher, a moderator, or by the presentation of instructional text. On the microlevel, however, learning protocols include two supporting mechanisms for grounding. First, participants in the learning discourse are required to explicitly reference their contributions, i.e., to indicate what the referred to element of the current contribution is. This might be a previous contribution, some
SUPPORTING NET-BASED DISCOURSES
47
part of a previous contribution, a single word, a phrase, or some part of additional information material, such as text, images, or diagrams accessible through the system. With respect to user interaction, this requires to indicate with the mouse the referred to object, to mark the contribution, or a fragment of the contribution, and the system then automatically visualizes the referential relation with an arrow pointing from the new contribution to the referred to element. As a result, the relationship of a contribution to previous contributions can be directly perceived on the screen, by simply following the referring arrows. This guarantees, at least on a syntactic level, that there are no loose ends in the discourse and that any contribution is coherently embedded into the context of the ongoing discourse, which constitutes an essential precondition of grounding. As a second mechanism for grounding, participants need to explicitly indicate the type of their contributions (Baker & Lund, 1997; Soller, Goodman, Linton, & Gaimari, 1998). For example, if a learner asks a questions, she or he will select the type 'question' from a menu of available contribution types, and then write the specific formulation of the question; or, if someone wants to justify an earlier statement, she will select the type 'justification' from a menu prior to entering the textual formulation. On the screen, all participants can see the type labels just preceding the textual input, and, together with the reference indicator, it will be clear what has been contributed as well as to what it refers. From a grounding perspective, both functions imply that contributions will be formally closed (Clark, 1996), though, .of course, these mechanisms will not guarantee that closure and feedback are semantically sound.
Figure 2. User interface of the explanation learning protocol.
Taken together, a learning protocol can be formally characterized by a set of roles assigned to the participants, by a set of contribution types, by a referencing function, and by a set of rules which determine the sequence of permissible contributions or contributors, respectively (see Table 1).
PFISTER
Table I. Definition of the explanation learning protocol EXP. EXP := {ref(con), T, R, S, C], with
ref(con) a function mapping con 3 r; con = contributions, r = {contribution, partial contribution, partial textlobject); T := set of permissible types = {explanation, question, commentary} R := a set of roles = {Tu: tutor, Lr: learner}; S := a set of rules for turn-taking (n: contribution index, m: learner index) = (IF conn(Lrm)and T=question THEN c,+,(Tu) and T=explanation}; {IF conn(Lrm) and T=explanationlcommentary THEN cn+,(Lm+l)and T=question~explanation~commentary ); C := a set of external constraints (n of participants, maximum duration, ...). By specifying the parameters of the protocol, e.g., by defining the set of available types and the set of rules, numerous special variants of discourse types can be realized. For example, Table 1 defines an explanation protocol, i.e., a discourse that centers around asking questions and giving explanations. The referencing function maps contributions to other (partial) contributions and text sections, the permissible types are explanation, question, and commentary, the roles are tutor and learner, and the rules for turn-taking specifiy, for example, that if the nth contribution was a question from a learner, than the (n+l)th contribution must be an explanation from the tutor. Changing the set of types or the turn-taking rules would generate another type of learning protocol. Based on the grounding-first principle, the definition of the macrolevel structure follows from the microlevel specifications: typed contributions and explicit referencing ensure grounding, and the specific collection of available types as well as the specific referencing function serve as input for the set of rules which define the sequence of contributions, and, hence, the unfolding of the discourse on the macrolevel. From the user point of view, a learning protocol is just a conventional chat tool enhanced by some additional functionalities. As can be seen in Figure 2, the user interface consists of a middle pane showing the ongoing text contributions, i.e., the basic chat pane; the left pane shows additional information such as text or images, and the right pane provides information on the participants' roles and turntaking. The referencing function is displayed by a simple arrow, for example, going from the current to a previous contribution. The available contribution types can be selected from a menu which opens before the textual input is written into the input pane at the bottom of the chat window. Note that all supporting functions are mandatory: Learners and tutors can contribute if and only if it is their turn as prescribed by the protocol rules, and if it is their turn they have to contribute something; furthermore, any contribution has to be referenced (no isolated contributions are permitted), and for any contribution a type has to be selected (as a residual category, the type 'commentary' may be used). In more advanced protocols changing roles or canceling the turn-taking rules during the discourse might be
SUPPORTING NET-BASED DISCOURSES
49
beneficial for acceptance; also, the obligation to choose a type before generating the text should be relaxed, since this might lead to mismatches between type-declaration and content (Winograd & Flores, 1986). First empirical results concerning learning protocols are reported in Pfister and Miihlpfordt (2002) and in Pfister et al. (2003). It was found that under certain conditions learning protocols yield better learning performance than chat-based discourses without learning protocol support. Generally, the efficiency of learning protocols increases with group size, and performance as measured by pre-lpostknowledge tests is only improved for certain knowledge domains: we found improvement for learning a science topic (mechanism of earthquakes) but not for learning a philosophical topic (knowing and believing). Furthermore, testing a learning protocol without the referencing function yielded no significant improvement, hence, referencing seems to be a necessary component for learning protocols (Pfister et al., 2003); latest experiments indicate that referencing might even be sufficient. These findings agree with results from Weinberger, Fischer, and Mandl (2002) who report positive effects of a comparable implementation of scripted cooperation (see Weinberger et al., this volume). The findings are also in accord with a line of research trying to support online-discourses by structuring the discourse either on the macro- (Hron et al., 2000), or on the microlevel (Baker & Lund, 1997; Soller & Lesgold, 2000) by providing so-called sentence openers to trigger certain types of contributions.
3. A COST-EFFORT FRAMEWORK Given the general premise that grounding improves cooperative learning, it is still largely unclear which cognitive mechanisms are involved, and, hence, which conditions need to be satisfied to improve learning by discourse in net-based settings. In this section, a framework is proposed which draws a great deal on previous work by Baker, Hansen, Joiner, and Traum (1999), Clark and Brennan (1991), Traum and Dillenbourg (1998), Traum (1998), and Paek and Horvitz (1999). In these studies it is assumed that grounding is not an all-or-nothing process, but operates in different degrees. How much grounding should be accomplished in a given situation is defined by a grounding criterion (Clark, 1996; Clark & Brennan, 1991), i.e., the degree of grounding sufficient for current purposes. A second assumption is that grounding is an effortful activity for discourse participants, who have a propensity to minimize this effort, what Clark (1996) calls the principle of least joint effort. From these assumptions it follows that grounding essentially implies a trade-off between the objective to learn, i.e., to increase the grounding criterion, and the tendency to minimize effort, i.e., to decrease the grounding criterion. The degree of grounding attained can be modeled as a decision process under uncertainty with conflicting goals, as first proposed by Traum and Dillenbourg (1998) and Paek and Horvitz (1999). Though the original formulation of the grounding criterion and the principle of least effort (Clark & Brennan, 1991; Clark & Schaefer, 1989) are fairly vague and underspecified (Traum, 1998, 1999),
50
PFISTER
we will keep to this line of argument and try to formulate a more detailed model of grounding for learning as a decision process, involving distinct cognitive operations.
3.1 The Costs of Grounding The principle of least joint effort (Clark, 1996) claims that participants jointly try to minimize the effort to reach the grounding criterion. In decision theoretic terms, effort is one constituent, possibly the most important one, of the overall costs associated with a grounding activity. Cost might be of different kinds, such as cognitive effort, motor-perceptual actions, time delays, and motivational efforts. In net-based scenarios, these costs are tightly linked to interactions with the specific user interface (Clark & Brennan, 1991). Whatever unobservable cognitive processes occur, eventually some action needs to be taken. For example, in plain chat, the costs of a contribution that go beyond those of face-to-face communication consist of entering the utterance as text via the key board, i.e., a psycho-motoric effort, of accepting the time-delay of typing as well as to take the uncertainty into account that the addressee might not perceive the contribution. In structured chat, such as a learning protocol interface, the costs of grounding can also be clearly mapped to distinct interactions: selecting the reference of a contribution, marking the referred to words, searching for and selecting a contribution type, matching one's formulation with the previously declared type, and entering the text via key board. Imagine a learning protocol which does not enforce all the actions to ground a contribution as outlined above, but gives the learner the opportunity to choose those grounding functions he or she thinks are appropriate for their current purpose. This leads to a kind of paradox, since the objective to provide support for grounding at the same time requires to exert more effort when actually using such functionalities. Assuming that the basic cognitive processes such as processing and understanding someone's contribution, and generating one's own response, are basically invariant for face-to-face and for computer-mediated communication, the additional cost of net-based discourse should be a function of the required human-computer interactions. In other words, it is assumed that the cost of grounding is primarily a function of the user interface and could be approximately measured by observable action units (clicking, typing, selecting from a menu, etc.; see Shneiderman, 1998). 3.2 The Utility of Grounding
Why should a learner accept the costs of grounding, or, what is the benefit of grounding one's own and others' contributions during a learning discourse? To answer this question, the grounding criterion concept needs clarification. Suppose a speaker has started a contribution by presenting the statement p. If the addressee understands p instantly and completely, she or he will close the contribution by a simple acceptance signal or by immediately initiating a new contribution; assume that grounding in this case is perfect, i.e., y has become part of their common ground. However, understanding of p might not be perfect, but the addressee might still signal acceptance. This suggests that she is satisfied with only a partial
SUPPORTING NET-BASED DISCOURSES
51
understanding of y, obviously sufficient with respect to a given grounding criterion, and anticipated costs are too high to warrant additional grounding. Now assume the addressee does not understand y at all; she will now consider to initiate some action y to ground y, for example, to ask for a repetition of y, or to start a repair subcontribution in order to clarify the meaning of y (Cahn & Brennan, 1999; Clark, 1996). The action y to be chosen should be such that it is expected to achieve the level of understanding set by the grounding criterion, but it should as well be that action among all available actions that will accomplish the grounding criterion with the least costs. If grounding succeeds, the new information contributed has become part of the common ground, i.e., shared knowledge. The importance of some piece of information to be an element of common ground depends on the goal guiding the discourse. In casual small-talk, the goal really is not exchange of information at all, but social organization; the grounding criterion will be close to a minimum, and acceptance signals without understanding will be frequent. In learning discourses, however, acquisition of knowledge is the primary goal, and social aspects are less important. Especially in net-based settings, the social function of communication is even less significant (Kiesler & Sproull, 1992) and typically dominated by taskoriented communication. Here, the grounding criterion will generally be close to a maximum, particularly when learners expect some kind of test or examination at the end of a learning session. If the grounding criterion is high, learners will try hard and exert much effort to attain an understanding of contributions, as long as they consider the contribution to convey valuable knowledge.
3.3 Deciding to Ground Taken together, a net-based learning discourse can be portrayed as a sequence of contributions, each one conveying knowledge to be learned, i.e., knowledge becoming part of common ground. Each contribution can be portrayed as a sequence of decisions pertaining to the grounding of the knowledge exchanged with that contribution. Each decision will take into account the current grounding criterion, the grounding actions available, and the costs associated with each action as determined by the user interface currently in operation.
Figure 3. Processing tree of grounding decisions.
Figure 3 depicts the basic decisions involved as a processing tree (Riefer & Batchelder, 1988), i.e., as a probabilistic sequence of binary choices to activate a specific cognitive process or not, the branch parameters indicating the probability of each process taking place. Hence, observed behaviors such as accepting the explanation or doing nothing might result from different paths of unobservable cognitive processes. Take as an example the situation that during execution of a learning protocol as described above, an explanation is provided by some learner or tutor (starting node in Figure 3). The learner to whom the explanation is addressed first decides with a probability u that she or he understands the explanation. Factors affecting u will be, among others, the currently attained common ground, how strongly the explanation is rooted within the current common ground, and the complexity of the explanation per se. If the addressee decides that she or he has understood, the next decision will be to supply an acceptance signal with probability a, thereby closing the contribution, or not to provide such a signal. If, on the other hand, the explanation has not been understood (with probability 1-u), the decision now is either to initiate a grounding process or to refrain from grounding. The probability g of grounding will be affected, as outlined above, by the costs of grounding and by the grounding criterion. Following a positive grounding decision, subsequent decisions will concern the details of grounding, i.e., which of the mechanisms available in the user interface will be applied. In Figure 3, for example, the choice is to provide a reference (with probability r) in order to specify which part of the explanation has not been comprehended, or to select another grounding mechanism currently available (this part of the tree might be much more complex). If, finally, the decision has been not to ground the explanation, though it has not been understood, there will still be the decision to send an acceptance signal or not. Social factors such as politeness might affect the probability a' to accept an
SUPPORTING NET-BASED
DISCOURSES
53
explanation which has not been understood. The crucial decision node is to ground or not to ground given the explanation has not been understood. Assume for simplicity that there is only one grounding action y available. Following Traum and Dillenbourg (1998), a basic decision model can be formulated which states the expected utility EU(y+y) of an action y to ground a statement y as a function of the probability that y will lead to sufficient grounding of p (given a grounding criterion GC), times the utility of y as part of common ground (given a learning goal LG), minus the costs to execute y:
Now assume that a set of different grounding actions yi is available. The learner might select the activity with maximum expected utility, or, probabilistically (Luce, 1959), the probability of using action yi can be expressed as the ratio of its expected utility to the sum of utilities of all available actions yj:
From this framework, grounding in net-based learning environments is regarded as a probabilistic decision process, determined primarily by features of the user interface and by the current grounding criterion; in other words, by the costs and benefits associated with a grounding activity. In the context of real learning sessions, it is reasonable to assume a grounding criterion which is invariably high for all participants. Hence, the most important factor which influences grounding decisions are the costs entailed by enacting the grounding functions via the user interface. 4. PERSPECTIVES As others have done before (Baker et al., 1999; Bromme & Jucks, 2001), it has been argued that support for grounding is an essential requirement for net-based learning environments which provide tools and methods for cooperative learning. Especially when supporting learning discourses among a group of learners, one needs to distinguish between the macrolevel of distinct stages that organize the overall structure of the discourse, and the microlevel concerning the performance of elementary utterances and the coherence among the contributions of different participants. It turned out that successful performance on the microlevel as conceptualized by the notion of grounding is a precondition for useful macrolevel structuring. Taking the learning protocol approach as an example of the reduction paradigm, it has been shown how grounding mechanisms such as referencing and typed contributions can be implemented to support structured learning discourses. Grounding in the context of learning discourses means to add knowledge to the common ground, which in essence is the gist of cooperative learning: going from unshared to shared information. A closer look at the cognitive processes possibly
54
PFISTER
involved in grounding showed that grounding is an activity which can be conceptualized as a decision with conflicting goals. Even if incentives to ground are high, participants in a learning course will not always and automatically ground contributions to the full extent, since grounding involves costs of different kinds. According to the principle of least joint effort, individuals have a tendency to minimize their joint effort to ground their conversations, which conflicts with the learning goal to acquire knowledge. Hence, there is a trade-off between the costs of grounding and the benefits of understanding, and, as a result, not each piece of communicated information will be grounded and increment shared knowledge. As a major source contributing to costs the specific features of the user interface available for grounding have been identified. As a perspective, some tentative conclusions will be drawn that might follow from this framework, particularly with respect to the design of net-based learning environments. Taking the processing tree from Figure 3 as a starting point, the first process usually is to decide if a contribution has been understood; if so, extended grounding becomes unnecessary anyways. To increase the probability of prior understanding, the degree of common ground attained at any point in time should be as high as possible. This could be achieved by providing comprehensive learning and discourse histories, such as discourse-logs, intermediate results, design histories of preceding problem solving attempts (Reimann & Zumbach, 2001), and any kind of documentation of past contributions to shared knowledge. Grounding involves acceptance of a presentation, i.e., some kind of feedback that one has understood. Frequently, addressees neglect this obligation and omit the acceptance signal, possibly because even this minimal effort is considered to be too high. A discourse system should provide easily accessible means to give positive feedback, or even to enforce explicit acceptance in case of understanding. Actually, in human-computer interaction, it has become a standard feature in user interface design that the system informs the user about a successful operation (for example, in email communication the sender is usually informed that the 'message has been sent successfully'). What seems much more difficult is to prevent participants from accepting an explanation when the explanation is actually not understood (see lower branch in Figure 3). The contributing person then is falsely informed that her or his presentation has been understood; if such false acceptance signals accumulate during a learning discourse, an 'illusion of acceptance' might ensue and form a barrier to critical questions and deeper reflection. Hence, the provision of simple acceptance functions (such as pressing an OK-button) should be avoided, and some kind of connection to the content presented should be elicited. The referencing function described above in the learning protocol approach is an attempt to enforce substantial links with previous contributions. However, two drawbacks of such enforced grounding actions can be identified. Enforced grounding also implies some amount of effort, and even though a participant in this situation may not be able to trade-off effort and expected gain, the effort invested will not be available for other cognitive activities. Cognitive load theory (Sweller, van Merrienboer, & Paas, 1998) clearly defines the constraints of a human learning system given the limited capacity of working memory. If cognitive load is high when processing a complex subject matter, additional cognitive load
SUPPORTING NET-BASED DISCOURSES
55
imposed by external requirements such as interacting with a computer interface to ground one's contributions might exceed the available capacity and cause detrimental effects on learning performance. A potential solution to this dilemma could be what we call 'optional learning protocols' (as opposed to the mandatory version described here), which provide all the grounding opportunities (referencing, typing, etc.), however, their use is not enforced by the system but it is up to the participant to activate one or more such functions when she or he considers it appropriate. Further research will show if this is a viable implementation of grounding in net-based learning environments. ACKNOWLEDGEMENTS This work was supported by the Deutsche Forschungsgemeinschaft (DFG) under research grant PF330/1-I to Hans-Riidiger Pfister as part of the special priority program "Net-based Knowledge Commutzication in Groups". I would like to thank Werner Miiller (University of Applied Sciences, Liineburg), Martin Miihlpfordt (Fraunhofer IPSI, Darmstadt), and Jorg Haake (Distance University Hagen), for valuable discussions.
REFERENCES Anderson, J. R. (2000). Cognitive psychology and its implications (5" ed.). New York: Freeman. Ayala, G., & Yano, Y. (1998). A collaborative learning environment based on intelligent agents. Expert Systertw with Applications, 14, 129-137. Baker, M., Hansen, T., Joiner, R., & Traum, D. (1999). The role of grounding in collaborative learning tasks. In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and coriiputational approaches (pp. 3 1-63). Oxford: Elsevier Science / Pergamon. Baker, M. J., & Lund, K. (1997). Promoting reflective interactions in CSCL environments. Journal of Cornputer Assisted Learning, 13, 167-193. Bromme, R., & Jucks, R. (2001). Wissensdivergenz und Kommunikation: Lernen zwischen Experten und Laien im Netz. In F. W. Hesse & H. F. Friedrich (Eds.), Partizipation und hteraktion irn virtuellen Serninar (pp. 1-23). Miinster: Waxmann. Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. B. Resnick (Ed.), Knowing, learning, and instruction. Essays in honor of Robert Glaser (pp. 393-452). Hillsdale, NJ: Erlbaum. Cahn, J. E., & Brennan, S. (1999). A psychological model of grounding and repair in dialog. In S. E. Brennan, A. Giboin, & D. Traum (Eds.), Proceedings qf the AAAI Fall Symposiurir on Psychological Models of Cornrilunication in Collaborative System (pp. 25-33). Menlo Park, CA: AAAI Press. Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press. Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine & S. D. Teasley (Eds.), Per.spectives or1 socially slzared cognition (pp. 127-149). Washington, DC: APA. Clark, H. H., & Schaefer, E. F. (1989). Contributing to discourse. Cognitive Science, 13,259-294. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, arid instructiorl (pp. 453-494). Hillsdale: Erlbaum. Dennis, A. R., & Valacich, J. S. (1999). Rethinking media richness: Towards a theory of media synchronicity. In R. H. Sprague (Ed.), Proceedings of the 32th Annual Hawaii International Conference on Systems Sciences [CD-ROM] (pp. 1-10). Los Alamitos, CA: IEEE Computer Society. Deny, S. J. (1999). A fish called peer learning: Searching for common themes. In A. M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 197-21 1). Mawah, NJ: Erlbaum. Fischer, F. (2002). Gemeinsame Wissenskonstruktion - Theoretische und methodologische Aspekte. Psychologisclte Rundsclzau, 53, 119-134.
Fischer, F., & Mandl, H. (2001). Facilitating the construction of shared knowledge with graphical representation tools in face-to-face and computer-mediated scenarios. In P. Dillenbourg, A. Eurelings & K. Hakkarainen (Eds.), Proceeding of the European Conference on Computer-Supported Collaborative Learning EuroCSCL-2001 (pp. 230-236). Maastricht: The Netherlands. Greenberg, S., & Roseman, M. (1998). Using a roorn rnetaphor to ease transitions in groupware. Research Report 9811 1/02, Department of Computer Science, University of Calgary, Canada. Herring, S. (1999). Interactional coherence in CMC. Journal of Conzputer-MediatedCornrnunication [Online], 4(4). Available: http:llwww.ascusc.org/jcmclvol4lissue4Pning.html. Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of groups as information processors. Psychological Bulletin, 121.43-64. Hron, A., Hesse, F. W., Cress, U., & Giovis, C. (2000). Implicit and explicit dialogue structuring in virtual learning groups. British Journal of Educational Psycltology, 70,53-64. Jermann, P., Soller, A., & Muehlenbrock, M. (2001). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. In P. Dillenbourg, A. Eurelings & K. Hakkarainen (Eds.), Proceedings of the European Conference on Computer-Supported Collaborative Learning EuroCSCL-2001 (pp. 324-331). Maastricht, The Netherlands. Kiesler, S., & Sproull, L. (1992). Group decision making and communication technology. Organizational Behavior and Hurnart Decision Processes, 52,96-123. King, A. (1998). Teaching effective discourse patterns for small group learning. In R. Stevens (Ed.), Teaching in American sc1tool.s (pp. 7-17). Upper saddle River, NJ: Prentice-Hall. King, A. (1999). Discourse patterns for mediating peer learning. In A. M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 87-1 15). Mahwah, NJ: Erlbaum. Koschmann, T. (Ed.) (1996). CSCL: Ttteory and practice of an ernerging paradigm. Mahwah, NJ: Erlbaum. Luce, R. D. (1959). Individual choice behavior. New York: Wiley. O'Donnell, A. M. (1999). Structuring dyadic interaction through scripted cooperation. In A. M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 179-196). Mahwah, NJ: Erlbaum. ,O'Donnell, A. M., & Dansereau, D. F. (1992). Scripted cooperation in student dyads: A method for analyzing and enhancing academic learning and performance. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups. Tlie rtteoretical anatorny of group learning (pp. 120-141). New York: Cambridge University Press. Paek, T., & Horvitz, E. (1999). Uncertainty, utility, and misunderstanding. In S. E. Brennan, A. Giboin, & D. Traum (Eds.), Proceedings of the AAAI Fall Syrizposiurn on Psychological Models of Cornrnunication in Collaborative Systerns (pp. 85-92). Menlo Park, CA: AAAI Press. Pfister, H.-R., & Muhlpfordt, M. (2002). Supporting discourse in a synchronous learning environment: The learning protocol approach. In G. Stahl (Ed.), Computer Support for Collaborative learning: Foundationsfor a CSCL Cornniunity. Proceedings of CSCL2002, Boulder, Colorado (pp. 581-589). Hillsdale, NJ: Erlbaum [On-Line]. Available: http://newmedia.colorado.edu~cscl/l78.pdf. =ster, H.-R., Muller, W., & Muhlpfordt, M. (2003). Lernprotokollunterstiitztes Lernen - ein Vergleich zwischen unstrukturiertem und systemkontrolliertem diskursivem Lernen im Netz. Zeitsckrif? fir Psychologie, 21 1,98- 109. Pfister, H.-R., Schuckmann, C., Beck-Wilson, J., & Wessner, M. (1998). The metaphor of virtual rooms in the cooperative learning environment CLear. In N. Streitz, S. Konomi & H.-J. Burkhardt (Eds.), Cooperative buildings. Integrating inforrnation, organization, and architecture. Lecture notes on computer science (pp. 107-113). Berlin: Springer. Pfister, H.-R., Wessner, M., Beck-Wilson, J., Miao, Y., & Steinmetz, R. (1998). Rooms, protocols, and nets: Metaphors for computer-supported cooperative learning of distributed groups. In A. Bruckman, M. Guzdial, J. L. Kolodner & A. Ram (Eds.), Proceedings of ICSL 98, International Conference of tlte Learning Sciences 1998 (pp. 242-248). Charlottesville, VA: Association for the Advancement of Computing in education (AACE). Pfister, H.-R., Wessner, M., & Beck-Wilson, J. (1999). Soziale und kognitive Orientierung in einer computer-gestutzten Lernumgebung. In U. Arend, E. Eberleh & K. Pitschke (Eds.), SqftwareErgonornie '99. Design von Inforrrlation.swelter1 (pp. 265-274). Stuttgart: Teubner. PIBtzner, R., Dillenbourg, P., Preier, M., & Traum, D. (1999). Learning by explaining to oneself and to others. In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 103-121). Oxford: Pergamon.
SUPPORTING NET-BASED DISCOURSES
57
Reimann, P., & Zumbach, J. (2001). Design, Diskurs und Reflexion als zentrale Elemente virtueller Seminare. In F. Hesse & F. Friedrich (Eds.), Partizipation und Interaktion in1 virtuellen Seminar (pp. 135-163). MUnchen: Waxmann. Riefer, D. M., & Batchelder, W. H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95,3 18-339. Roseman, M., & Greenberg, S. (1996). TeamRooms: Network places for cooperation. Proceedings of the ACM 1996 Conference on Cornputer Supported Cooperative Work (CSCW96), 325-333. Shneiderman, B. (1998). Designing the User Interface (3rded.). Reading MA: Addison-Wesley. Slavin, R. E. (1995). Cooperative learning: Theory, researck and practice (2nded.). Needham Heights, MA: Allyn and Bacon. Soller, A., Goodman, B., Linton, F., & Gaimari, R. (1998). Promoting effective peer interaction in an intelligent collaborative learning environment. Proceedings of the Fourth International Conference on Intelligent Tutoring Systenu. Sun Antonio, Texas, 186-195. Soller, A., & Lesgold, A. (2000). Knowledge acquisition for adaptive collaborative learning environments. In M. Bauer & C. Rich (Eds.), Learning how to do things. Papers front the AAAI Fall Syrnposiunl (pp. 57-60). North Falmouth, MA. Technical Report. Sweller, J., van Merricnboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10,251-296. Traum, D. R. (1998). On Clark and Schaefer's contribution model and its applicability to humancomputer collaboration. COOP'98 Workshop # I, The Use of Herbert H. Clark's Models of language Use for the Design of Cooperative Systems, On-line Working Notes, 1998. http://citeseer.nj.nec.com~traum98clark.html. Traum, D. R. (1999). Computational models of grounding in collaborative systems. Working Notes of AAAI Fall Synlposiuni on Psychological Models of Communication, 124-131. Traum, D. R., & Dillenbourg, P. (1998). Towards a normative model of grounding in collaboration. Proceedings of the ESSLLI98 workshop on Mutual Knowledge, Cornn~on Ground and Public Information, 1-5. Wegner, D. M. (1987). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior (pp. 185-208). New York: Springer. Weinberger, A., Fischer, F.,& Mandl, H. (2002). Fostering computer supported collaborative learning with cooperation scripts and scaffolds. In G. Stahl (Ed.), Computer Support ,for Collaborative Learning: Foundations,for a CSCL Community. Proceedings o f CSCL 2002 (pp. 573-574). Hillsdale: Erlbaum (e-document). and cognition: A new ,foundation ,for Winograd, T., & Flores, F. (1986). Understanding co~nputer.~ design.Norwood: Ablex.
pjister @finon.de.
NIKOL RUMMEL & HANS SPADA
INSTRUCTIONAL SUPPORT FOR COLLABORATION IN DESKTOP VIDEOCONFERENCE SETTINGS How It Can be Achieved and Assessed
Abstract. Innovative computer-mediated settings can help to overcome many traditional barriers to knowledge communication and collaborative work. However, successful collaborative learning and tasksolving can be a challenge, which is aggravated in the case of complementary expertise of the collaborating partners. To meet these challenges, we propose providing support for computer-mediated collaboration by instructional methods. The goal is to improve the collaborative skills and knowledge of the people jointly working on a task. We outline different instructional approaches using the example of a situation in which experts from different fields are required to jointly solve psychiatric cases communicating via a desktop videoconference. In addition, we discuss assessment methods for evaluating the effects of these approaches on three levels: collaborative process, joint outcome, and individual knowledge. Finally, we summarize an experiment in which instructional support methods were tested by assessing their effects on all three of these levels.
1. INTRODUCTION Innovative computer-mediated settings can help to overcome many traditional barriers to knowledge comniunication, opening up a wide array of opportunities for collaboration across distance, domain, and expertise. In working contexts such as remote surgery, web design, or the assessment of cases in international law firms, the use of technology for remote collaboration is on the increase (Leskovac, 1998; Nardi, Kuchinsky, Whittaker, Leichner & Schwarz, 1997; Whittaker, 1995). Unfortunately, implementing computer-mediated settings for collaboration is far from straightforward. Establishing and maintaining common ground, pooling unshared knowledge, and coordinating collaboration can present considerable difficulties. These problems are compounded by the fact that the possibilities for exchanging nonverbal information remain restricted. Without the necessary support, collaborating partners often fail to complete their joint task or find that it requires too much time and effort. The development of effective support methods goes hand in hand with the analysis of what constitutes good collaboration in a computermediated setting and how such collaboration can be achieved. In this chapter, we discuss (1) challenges encountered in particular computer-mediated collaboration settings, (2) different approaches to meet these challenges, and (3) methods to evaluate the support approaches by assessing their effects on process and outcome of the collaboration. In our research (Rummel & Spada, accepted), we provide collaboration support by promoting through instruction the collaborative skills of the
people involved. We summarize our ideas and findings with regard to these support measures and look at the methods that can be used to analyze their effects. 2. CHALLENGES IN COMPUTER-MEDIATED COLLABORATION
Picture the following scenario: A medical doctor and a psychologist are asked to collaborate on solving a complicated clinical case. As the case involves both a physical illness and psychopathological symptoms, the assessment requires the two persons to make use of their complementary expertise. Their joint task is to formulate a report that includes a detailed diagnosis for the patient and a proposal of a suitable therapy. The two experts are not able to meet in person and decide therefore to take advantage of a desktop videoconferencing system that has recently been implemented at both of their institutions. The desktop videoconference enables them to see and hear each other while discussing the case at hand. Moreover, the system includes a shared workspace (text editor) as well as individual text editors, which they can use to develop their joint report. The characteristics inherent in this scenario can be seen in terms of three main barriers that have to be overcome in order for the collaborative endeavor to succeed: the challenge of solving a complex task collaboratively, and on the basis of complementary domain knowledge of the collaborating partners, is combined with the challenges of communication in a computer-mediated setting. However, such challenges can also be regarded as representing promising resources for the solution of the task. The collaboration of two people who bring complementary expertise to the task can provide a powerful way of solving complex problems that could not be tackled by one person alone. Similarly, under the right circumstances a computermediated setting can provide great affordances for collaboration. We argue that the potential inherent in the three characteristics outlined for the above scenario will only be unlocked if they are approached in the right way - which is why we call them as challenges. 2.1 A First Challenge: Problem-Solving and Learning in Collaboration Possible goals of collaboration, such as the exchange of unshared information, the joint solution of complex and ill-structured problems by experts from different fields, and collaborative learning in a new domain, require well-coordinated collaborative activities. Extensive research has shown that the success of collaborative efforts does not occur by itself (Diehl & Stroebe, 1991; Johnson & Johnson, 1992; McGrath, 1984; Slavin, 1995; Salomon & Globerson, 1989). Without systematic support, people can differ greatly in the way in which they collaborate, depending on a variety of interacting conditions such as group size, group composition, collaborative task, or the media used for communication (Dillenbourg, Baker, Blaye & O'Malley, 1995). Although some groups collaborate quite efficiently and with a good level of success even when left to their own devices, the majority of collaborations only succeed with adequate support.
INSTRUCTIONALSUPPORT FOR COLLABORATION
2.2 A Second Challenge: Complementary Expertise as a Basis for Collaboration
In the scenario outlined above, a second challenge and potential opportunity arises from the situation of "complementary expertise" of the collaborating partners. In other words, the partners in the collaboration complement one another in that each possesses a relevant part of the unshared knowledge. Each of the partners is a "novice" in the other's domain and an "expert" in his own. This can provide a promising basis for collaborative problem-solving. In fact, in many domains the enormous and rapid growth of domain knowledge, in conjunction with an ever-increasing specialization of this knowledge, results in a growing need for interdisciplinary collaboration. Experts from different fields of expertise need to work together in order to successfully solve the tasks at hand. In addition, interdisciplinary collaboration is considered to be the key to a successful exploration of complex phenomena, where only considering one perspective would be insufficient (Gibbons et al., 1994; Kneser & Plotzner, 2001; Platzner, Fehse, Kneser & Spada, 1999). For example, the collaboration of psychologists and medical doctors is increasingly regarded to be of great importance and potential for the well-being of patients. A successful treatment is only possible if a correct diagnosis has been deduced from the symptoms of a patient. However, some symptoms can indicate both a somatic and a psychological diagnosis. Sleep disorders, for instance, could be a symptom of depression, but they could also be a side effect of medicines prescribed to a patient to treat another problem. A psychologist treating a patient who is taking medication for another medical condition would certainly be advised to consult a medical doctor before diagnosing depression. A further reason for the importance of interdisciplinary collaboration between psychologists and medical doctors is the high comorbidity of psychological and somatic disorders, particularly among in-patients. In spite of its great potential, interdisciplinary collaboration is not an easy undertaking (Lewis & Sycara, 1993; Bromme 2000). Problems known to be symptomatic for collaborative learning and problem-solving in general apply to an even greater extent to interdisciplinary collaboration (Thompson Klein & Porter, 1990; Stasser, Stewart & Wittenbaum, 1995). Establishing and maintaining common ground (Clark & Brennan, 1991) and pooling unshared knowledge (Stasser & Titus, 1985) present significant challenges in this kind of collaboration. Interdisciplinary collaborators bring with them different domain knowledge, which comprises different concepts, methodological approaches, and cultures of thinking. Moreover, interdisciplinary collaboration is often burdened by naive theories and prejudices about the partner's domain. A further well-known problem is that collaborators might fear that their competence could be undermined.
2.3 A Third Challenge: Collaborating in a Computer-Mediated Setting The situation of remote, computer-mediated collaboration, in our scenario via a desktop videoconference with shared and individual workspaces (text editors), has
important consequences for communication and the collaborative problem-solving process. On the one hand, it can be perceived as an additional challenge: in computer-mediated collaboration, the challenge of working collaboratively is aggravated by the constraints of an environment that has restricted possibilities for particular aspects of communication (Clark & Brennan, 1991; O'Conaill & Whittaker, 1997). On the other hand, such settings also open up opportunities to collaborate in new, more enriched ways. In a desktop videoconference, partners at different locations each sit at their personal computer and communicate via an audio-video connection. On the computer screen they can see video pictures from the remote sites, e.g. of the remote partners, often in the form of "talking heads". Each video picture is captured by a small camera sitting on top of the computer screen or placed directly to the side of the screen. A continuous audio channel provides the possibility to speak with the remote partners. Application sharing tools can be added to this setting and enable joint manipulation of objects, data, or text (e.g. Word documents, Excel spreadsheets) in a workspace that is visible and editable simultaneously for all collaborators (Dillenbourg & Traum, 1999; Giirer, Kozma & Millin, 1999; Whittaker, Geelhoed & Robinson, 1993). Thus, shared workspaces support the externalization and visualization of contents during collaborative interactions. The framework proposed by Clark and Brennan (1991) allows the characterization of different media for collaboration along eight dimensions (p. 141): copresence, visibility, audibility, cotemporality, simultaneity, sequentiality, reviewability, and revisability. Following this framework, desktop videoconferences at least partly fulfill all of these dimensions. Visibility: The remote partners are able to see each other. This adds several important aspects to the spectrum of information available during communication compared to other forms of computer-mediated communication such as chat: facial expressions provide nonverbal information; being able to see provides information about the availability and receptiveness of the partner. However, the visual contact possible in desktop videoconference settings is in most cases limited to the face or upper body of the partner; eye contact is not usually possible, and nor is gaze awareness (Angiolillo, Blanchard, Israelski & Mane, 1997; Joiner, Scanlon, O'Shea, Smith & Blake, 2002). Audibility: The remote partners are able to communicate by speaking. The possibility of talking to and hearing the collaborating partners enables highfrequency, natural interaction. The modulation of the voice additionally transports paraverbal information. In contrast, in communication limited to text, the exchange of verbal information is impeded as writing text to express one's ideas and opinions in a dialog is generally found to require much more effort. One reason for this is of course that the act of handwriting or typing itself takes more time and effort. Another reason can be found in the fact that written language is different, usually more formal, than spoken language. Moreover, in text-based communication, the exchange of para- and nonverbal information is not possible in the usual way, which has severe effects on basic communication processes such as turn-taking, referencing or feedback mechanisms.
INSTRUCTIONAL SUPPORT FOR COLLABORATION
63
Reviewability and revisability: Shared applications add these important dimensions to the setting. While the contents of a discussion fade, the actions that were performed in the shared workspace remain and can be reviewed by all partners. Serving as an external memory for the collaborative problem-solving process, shared workspaces can therefore reduce cognitive load during the interaction (VanBruggen, Kirschner & Jochems, 2002). Furthermore, the stored contents can be revised by all partners. Thus the shared workspace facilitates the collaborative construction of a joint work product. In addition, the workspace can also be used to share individual contributions of the partners, in order to store them for later reviewability by other group members or to make them available for discussion and integration into a joint work product. The latter advantage is particularly salient if a shared workspace is combined with individual workspaces, as in the scenario described above. It should be noted, however, that shared workspaces also place additional demands on the coordination of the computer-mediated collaboration. The use of the workspace (who types or revises what and when) needs to be coordinated in order to avoid chaos and to prevent double the amount of work. Copresence: One criticism leveled at videoconference systems is that they do not support joint awareness of and attention towards objects in the physical environment (Kato et al., 2002). This is indeed the case for classical videoconferences, but shared applications provide a new aspect of copresence: a workspace that is visible and can be manipulated by all the collaborators. Cotemporality and simultaneity: Desktop videoconferences fulfill the dimension of cotemporality in part: utterances of the speaker are immediately received by the listener; however, depending on the quality of the audio and video transmission, delays in the transmission of sound and picture may at times cause some breaks and overlaps in the dialog structure (Angiolillo et al., 1997). With regard to simultaneity, the collaborating partners are able to receive and send messages or perform actions at once and simultaneously. For example, one partner can nod while the other explains something. The possibility of seeing and speaking with each other, together with the cotemporality of the performed utterances, forms the basis of the simultaneity of the setting. If the cotemporality is limited by the system's transmission delay, simultaneity is also affected. Shared applications contribute to the dimensions of cotemporality and simultaneity. Actions that one partner performs in the shared workspace become visible immediately, and actions can be performed simultaneously by both partners. Sequentiality: The speakers take turns in communicating, and these turns are not separated by other intervening turns. At times the sequence of turns in a videoconference can be affected by the aforementioned problems caused by delays in the transmission of video and audio. However, even with these delays, messages build on one another, and the interaction sequence is not disrupted by intervening messages, as Clark and Brennan (1991) describe for communication in asynchronous communication settings (e.g. emails, letters, answering machines). In fact, O'Conaill and Whittaker (1997) found that collaborators in video-mediated interactions tend to communicate in a more "lecture-like" fashion (e.g. handing over turns in a formal way by using questions or naming the next speaker) compared to face-to-face collaborators.
In sum, a desktop videoconference with shared workspace and individual workspaces undeniably offers great resources for remote collaboration. The opportunity to talk to and see the partner - even with the limitations mentioned allows for synchronous verbal interactions and immediacy of responses. This is particularly important for collaboration on a task like the one outlined above, which requires the negotiation of concepts as well as individual ideas in discourse. In addition, the shared and individual workspaces (here: text editors) support the development of a joint work product from both individual text contributions and collaborative discourse. However, such a collaboration setting can also be expected to exert high demands on the collaborating partners due to the various issues raised above: visual contact limited - no eye contact, no gaze awareness; cotemporality and simultaneity of utterances sometimes limited by delays in audio-video transmission; use of shared and individual text editors requires careful coordination. In order for the collaboration to benefit from the opportunities provided by the setting, adequate coordination of the joint proceeding is vital. 3. OVERCOMING THE BARRIERS: HOW TO ACHIEVE GOOD COLLABORATION
Having identified the challenges that need to be tackled in order to collaborate successfully in a computer-mediated collaboration scenario of the type described above, it is necessary to consider how collaborators can overcome these barriers. The different support approaches that we propose (see also Rummel & Spada, accepted) draw on well-researched strategies for supporting face-to-face collaboration and various other forms of computer-mediated collaboration. Two general approaches to supporting computer-mediated collaboration can be distinguished: measures of support can be realized prior to or during the collaborative session.
3.1 Support During Collaboration One way of directly influencing interactions during an ongoing collaboration might be achieved by designing the environment (e.g. by configuring the interface in a specific way or by making available certain communication channels and shared applications). Second, several approaches have been directed at fostering fruitful collaboration by externally structuring the interaction process. Two of the most well-known structuring techniques are reciprocal teaching (Palincsar & Brown, 1984) and scripted cooperation (0' Donnell & Dansereau, 1992). These techniques share a common feature in that both prescribe work phases involving specific cognitive activities for the interaction of the collaborating partners ( 0 ' Donnell, 1999). For example, the reciprocal teaching technique designates roles to the collaborating learners, which include activities such as questioning, summarizing, clarifying, and predicting (Palincsar & Brown, 1984). Evidence for the effectiveness of scripting techniques in supporting face-to-face collaboration (O'Donnell & Dansereau, 1992; O'Donnell, 1999; Rosenshine & Meister, 1994), as well as
INSTRUCTIONAL SUPPORT FOR COLLABORATION
65
collaboration in computer-mediated settings (Baker & Lund, 1997; Hron, Hesse, Reinhard & Picard, 1997; Reiserer, Ertl, & Mandl, 2002; see also Weinberger, Reiserer, Ertl, Fischer & Mandl, this volume and Pfister, this volume), has been found in numerous studies. The main idea behind the use of cooperation scripts in computer-mediated environments is to focus the collaborative process on the most important subtasks and thus reduce the costs of coordination. In such settings, cooperation scripts are often implemented by giving step-by-step instructions. Further, in this context, scripts are often embedded in the structure of the collaborative environment (e.g. Baker & Lund, 1997; Hron et al., 1997; Weinberger et al., this volume; Pfister, this volume). This ties in with the aforementioned property of desktop videoconference systems that they can make available shared workspace. The relevant idea here is that shared workspaces can be prestructured by embedding script information that can guide the collaborators and enhance content-specific negotiation in the workspace ("representational guidance"; cf. Suthers, 2001). Particularly in computer-mediated collaboration, this prestructuring of the communication interface enables the integration of both interface design and scripting of the collaborative process (Bruhn, Fischer, Grhel, & Mandl, 2000). 3.2 Support Prior to Collaboration: An Instructional Approach
Support implemented prior to the collaboration aims at promoting the collaborative competences of the people involved. In other words, collaboration strategies are taught in order to facilitate subsequent collaboration. We propose an instructional approach, under the assumption that long-lasting, "sustainable" effects can be achieved by instructionally promoting the collaborative skills of the people involved. But which instructional strategy is able to adequately convey the relevant aspects of good collaboration to people, thereby promoting collaborative skills for subsequent interactions? Direct instruction or training (Cannon-Bowers & Salas, 1998) would be the most obvious example of this approach. However, a successful transfer of trained behavior to new application situations is hard to achieve (Mayer & Wittrock, 1996; Renkl, Mandl, & Gruber, 1996). It might be more promising to follow a situated learning perspective (Collins, Brown & Newman, 1989; Greeno and MMAP, 1998; Lave & Wenger, 1991) and introduce collaborators to the "craft of collaborating" by immersing them in a corresponding environment, in other words by involving them in instructionally meaningful collaborative activities. The idea of a situated approach is that the learning situation should resemble the application situation as closely as possible. But how should we design a situation that allows people to learn to collaborate in a collaborative context that closely resembles the later application situation? 3.2.1 Unsupported (Unscripted) Collaborative Problem-Solving One possibility is to involve people in collaborating on a task similar to the ones they will be confronted with later. In this way, collaborating partners can gain experience on at least three different levels: experience in performing the steps
necessary to solve tasks of this type (problem-solving); experience in jointly working with the specific partner (collaborative problem-solving); and experience in communicating and working with the desktop videoconference system (computermediated collaborative problem-solving). Such collaborative problem-solving without any additional help can be regarded as the most natural but also the most restricted form of situated learning. It is, however, doubtful that relevant learning processes actually occur in such a situation without any help. Cognitive load theory (Sweller & Cooper, 1985; Sweller, VanMerrienboer & Paas, 1998) strongly suggests that the demands of problem-solving in such a complex situation might cause cognitive overload and lead to failure of both the problem-solving and the learning process. The present situation is particularly likely to cause such overload, since the demands of solving the problem at hand are aggravated by the difficulties of working collaboratively and in an interdisciplinary constellation, and by the challenges of computer-mediated interaction. Learning in this way is likely to lead to impasses and be very time-consuming. 3.2.2 Observational Learning from a Worked-out Example of Collaborative Problem-Solving A promising instructional strategy is to have people observe a model of a successful computer-mediated collaborative solution of a problem. We call such a model a "worked-out collaboration example". While they are observing the model, people are encouraged to reflect on the solution steps of the worked-out example and on the behavior of the collaborating partners. This should enable them to learn what aspects they need to pay attention to when collaborating. Why do we expect a worked-out collaboration example to be effective in promoting collaborative skills? And how should such a model be designed in order to provide optimal opportunities for learning? Reimann (1997), Renkl (1997), VanLehn (1996) and others have emphasized that individual learning from worked-out examples presents a successful way of acquiring cognitive skills. This type of learning is primarily based on the selfexplanation of the solution steps (Chi, Bassok, Lewis, Reimann & Glaser; 1989). Sweller and Cooper (1985) have provided evidence that learning from worked-out examples is often more effective than learning by problem-solving due to cognitive overload caused by the demands of the latter. In sum, the strengths of worked-out examples lie in reducing cognitive load, focusing the learner's attention on relevant aspects of the problem-solving process, and fostering the acquisition of adequate problem-solving schemas (VanLehn, 1996). How do these results on individual learning with worked-out examples transfer to our scenario? Why do we expect a worked-out example to be effective in promoting collaborative problem-solving skills? Our assumptions are supported by research showing that observational learning is of particular value in the context of dialog and discourse. Stenning and colleagues (1999) have provided empirical evidence that the observation of dialogs supports the acquisition of dialog competence. Along the same lines, a study by Cox,
INSTRUCTIONAL SUPPORT FOR COLLABORATION
67
McKendree, Tobin, Lee and Mayes (1999) analyzed the effect of reading the content of a tutor-student dialog, and found positive results on subsequent dialog. Furthermore, in industrial settings, a behavior modeling approach (Goldstein & Sorcher, 1974) based on observational learning (Bandura, 1977) has been shown to be an effective training method for the acquisition of complex behavioral skills (Latham & Saari, 1979; Meyer & Raich, 1983), similar to the collaborative skills we wish to convey. Combining these different strands of research, it is plausible that observing the worked-out example of a well-structured computer-mediated collaboration and reflecting on the solution steps and on the behavior of the collaborating partners constitutes a promising method for learning relevant aspects of what constitutes a good collaboration in the present scenario and for acquiring collaborative skills with long-term effects.
3.2.3 Learning from Scripted Collaborative Problem-Solving A second instructional approach that we propose relates to a measure for supporting collaboration that has already been introduced: cooperation scripts. So far, cooperation scripts have mostly been implemented as short-term "online" interventions directed towards the achievement of immediate effects in a single collaboration. But what are the long-term effects of cooperation scripts? Is it possible to script collaboration over many sessions? From an instructional point of view the central question is whether the effects of cooperation scripts extend beyond the experimental session in which they were provided, by promoting the skills of collaboration. The idea is to consider cooperation scripts as an opportunity for learning and to investigate their effects beyond the scripted session. While externally scripting collaboration over longer periods of time might lead to motivational losses (Bruhn, 2000; Kollar, 2001), learning from scripted collaboration might be an effective instructional measure. 4. ASSESSING COLLABORATION: HOW TO TEST THE EFFECTS OF SUPPORT MEASURES With the goal of empirically assessing the effects of instructional support measures like the ones introduced above, the important question that arises is which dependent variables provide appropriate evidence? A criticism of most studies investigating the effects of support measures on computer-mediated collaboration is that they have concentrated either only on the collaborative process or only on the outcome (Anderson et al., 1997). We believe that in order to fully evaluate the impact that support measures have, it is necessary to include data both from the collaborative process itself and from its outcome. Moreover, assessing both sources of data allows insights to be gained into the relationship between process characteristics and the quality of the outcomes of the joint work. This should promote the development of a theory of good computer-mediated collaboration. In addition, data on the level of declarative individual knowledge should also be collected, e.g. by means of a posttest on knowledge of what constitutes good collaboration.
In order to be able to assess both process and outcome effects of instructional support measures, we propose (Rummel & Spada, accepted) an experimental paradigm, which comprises two phases: a learning phase and a subsequent application phase (see Table 1). During the learning phase, the instructional support measures are implemented (=experimental variation). The selection of data sources for testing the effects of these measures is guided by the following considerations: If an instructional measure is successful in the learning phase, participants acquire knowledge of aspects of good collaboration and consequently develop collaborative skills. The knowledge and skills acquired should then become evident in the subsequent application phase (which is the same in all conditions: computermediated collaboration without any further instruction or help) and in an individual knowledge posttest. The acquired collaborative skills should result in a better collaborative process during the application phase and thereby yield a better outcome Cjoint solution). Improved explicit knowledge about aspects of good collaboration and about the solution of the task should be a further instructional effect. Table I . Experimental paradigm and data sources to test the effects of support measures Learning phase
I
Implementation of support measures
I Assessment of effects
Application phase
+ Data on collaborative process + Data on outcome Cjoint solution) I + Data on individual knowledge
I
In the following paragraphs, we describe methods of assessing performance on all three levels: (1) the level of the collaborative process, reflecting the effects of collaborative skills acquired in the learning phase; (2) the level of the joint solution, representing the outcome of the collaborative process; and (3) the level of individual knowledge acquired in the learning phase. We have developed and applied the methods described as part of an experimental study, which will be summarized in the next section. As proposed in the experimental paradigm, the assessments were applied to the unsupported collaboration in the application phase. We use the scenario that formed the basis of our study (and that was introduced at the beginning of this chapter) as a framework with which to describe the assessment methods we propose. However, the methods are not limited to this scenario, but may be transferred to other computer-mediated collaborative settings with the same generic features.
4.1 Assessing Collaborative Process In order to assess the collaborative process, we integrated empirical findings from different strands of research in an attempt to define aspects of good collaboration. We propose a distinction between three levels: (1) coordination of joint work; (2)
INSTRUCTIONAL SUPPORT FOR COLLABORATION
69
communication: the way in which new content is introduced or requested, feedback is given, and turn-taking is orchestrated; and (3) domain-related content and quality of the dialog. The assessment methods developed for the coordination and communication levels are not restricted to any specific domain. The domain-specific assessment obviously has to be adjusted if collaboration in another content domain is being analyzed. In the following, we outline the theoretical background of the three levels of assessment. The systems of criteria for assessing these levels of the collaborative process will then be introduced and discussed in detail. In a computer-mediated collaboration setting, the coordination of the joint work is of great importance (Barron, 2000; Olson, Malone & Smith, 2001; Malone & Crowston, 1990). In this context, coordination has to serve several goals: managing time constraints, dividing the task into subtasks, dividing labor between the partners, balancing individual and joint work phases, and integrating individual contributions. Particularly in the case of complementary expertise of the partners - as in the scenario at hand - the question of joint and individual working phases has to be considered. What does each partner have to prepare on the basis of their individual knowledge before it can be integrated into the joint solution? Which elements of the preliminary joint solution require disciplinary reflection and revision? A wellbalanced proportion of individual and joint work phases is crucial for a successful collaboration. Recent studies have shown that individual work is often neglected in computer-mediated collaboration of the type analyzed in the present study (Hermann Rummel & Spada, 2001). Hence, the amount of individual work is of great interest to our analysis. Aspects of the communication that we are particularly concerned with here include the establishment of mutual understanding, the pooling of unshared information, the tailoring of explanations to the partner, and the handling of turntaking. Developing and maintaining mutual understanding is a constant challenge during collaboration, a phenomenon widely known as "grounding" in communication (Clark & Brennan, 1991; Baker, Hansen, Joinier & Traum, 1999). When collaborating partners come from different disciplinary backgrounds, the establishment of a common ground and convergence on central concepts (Roschelle, 1992) is particularly important but at the same time considerably difficult. In order to avoid misunderstandings, it is vital to give feedback of one's understanding and use the partner as a source for clarifications by asking appropriate (comprehensible and relevant) questions. Asking questions is also of central importance in fostering the exchange of unshared information. The pooling of unshared information (accessible only to individual members of the group) is one of the crucial aspects of successful collaborative problem-solving and decision-making (Stasser & Titus, 1985; Larson, Christensen, Franz & Abbott, 1998). The failure of collaborating partners to pool their unshared knowledge resources is fatal in a situation where the group members are mutually depending on one another's knowledge to successfully complete the group task (Johnson & Johnson, 1992). Such a situation arises in the present scenario through the distribution of complementary expertise in the dyad.
When tailoring one's explanations to the knowledge of the partner, the pitfalls of an "illusion of evidence" (Jucks, Bromme & Runde, 2003) have to be avoided. The importance of adapting the level of questions (information requested) and answers (information provided) is further supported by results summarized by Webb (1989). Only explanations at an appropriate level of elaboration can be of help to the questioner. The way two people regulate turn-taking during their interaction is a further crucial factor for the quality of the collaboration. How do the partners determine who will speak when? How do they regulate transitions, and how explicit (verbal) or implicit (nonverbal) are the turn-taking cues they give to each other (Sacks, Schlegloff & Jefferson, 1974)? For instance, in computer-mediated communication settings, explicitly handing over a turn can be a good solution to compensate for the reduced possibilities of transmitting nonverbal information. Last but not least, for a successful collaboration the domain-related content of the dialog during collaboration has to be assessed and judged in terms of its quality. In order to come to a good joint solution, particular topics have to be addressed during the interaction. The way in which these topics are addressed differentiates between collaborations. In what breadth and depth was a topic discussed? Were the statements made correct and adequate with regard to the topic's relevance for the task solution? To gain information about the collaborative process at the levels described above, we propose two approaches. First, we suggest an analysis on the basis of logfile data, which provides information about a central aspect of the coordination of the collaboration: the distribution of individual and joint phases of work. Second, video recordings and transcripts allow a detailed, albeit laborious, analysis of the dialogs with regard to aspects of all three levels: coordination, communication and the domain-related verbal interactions of the partners. 4.1.1 Analysis of Log-file Data On the basis of log-files taken during the application phase, the activity patterns of the collaboration can be analyzed. The log-files we used in our research recorded what the collaborating partners were doing minute by minute: whether they were talking with each other, whether they were using the personal or shared text editors, and whether text segments were being exchanged. An example of an activity pattern produced from log-file data is given in Figure 1. The diagram illustrates the collaboration of dyad 18 from our study. It documents the sequence of activities over time: the collaboration pattern is depicted from 0 to 120 minutes from left to right. For each minute, the diagram shows which activities took place. The upper three lines represent activities of the student of psychology (exchanging notes, writing in the personal text editor, reading). Correspondingly, the bottom three lines represent the activities of the medical student (in reverse order: reading, writing in the personal text editor, exchanging notes). The two center lines show joint activities of the partners (dialog and writing in the shared text editor).
INSTRUCTIONAL SUPPORT FOR COLLABORATION
71
Model condition, dyad 18 (indivdualwork: 58 min.; joint work: 61 min.) ~1.Exdanginat6xt~owch~
I
I
Figure I . Example of activity patterns as extracted from log-file data
From the activity patterns recorded in the log-files, individual and joint phases of work can be identified. Then, for example, the total amount of individual and joint work (in minutes) can be calculated. This enables the assessment of one aspect of the collaborative process on the coordination level: the balance of individual and joint work. In addition, if a given collaborative task allows a normative, exemplary collaborative process to be outlined with regard to the proportion of individual and joint work, the deviation of the collaboration from this exemplary collaboration during the application phase can be analyzed. 4.1.2 Analysis of Dialogs A second approach to the collaborative process can be taken by analyzing the dialogs with regard to coordinative, communicative, and domain-related aspects. These analyses can be performed on different sources of data and for different units of analysis. Video recordings of the collaboration are one possible source of data. Video data is not suited for fine-grained analyses, e.g. at the individual turn level. As the video tape displays the collaborative dialog in real time, the analyses performed on this data source require a coarser unit of analysis, e.g. a minute. From the video recordings, transcripts can be made, enabling more fine-grained analyses. However, producing a transcript is very time-consuming. In the following paragraphs, we describe in detail the dialog analyses that we have developed and implemented in our experimental study (Rummel & Spada, accepted). With the exception of the domain-related ratings, it would be possible to apply the analyses to dialog data from other computer-mediated collaborative settings with similar generic features. Coordination and communication analyses. For the analyses of the coordination and communication aspects of the dialogs, systems of criteria were developed that drew on the empirical findings of good collaboration as discussed above. The criteria allow the assessment of relevant elements of the collaboration from the dialog. Tables 2 and 3 give an overview of the categories assessed at the two levels (see also Grolj Ophoff, 2003). At the coordination level (see Table 2), time management (1) is assessed both globally (e.g. whether partners map a plan for the general proceedings and arrange a timetable), and locally (e.g. whenever the partners refer to time, monitor the state of their work, and rearrange their timetable if necessary). Second, special attention is devoted to the assessment of good work coordination (2). Coordination encompasses here the division of labor with regard to both person and content: whose role (2b) is
to do what (2a). In addition, discussion of the technical coordination of work (for example, when person A asks person B to go ahead and copy her individual notes on the diagnosis in the shared text editor) is assessed in a separate category (2c). Finally, explicit reference to the situation of complementary expertise in the dyad is assessed (3). This embraces the distribution of both domain knowledge and text material. Parts of this system relate to the categories described by Bruhn and colleagues (Bruhn, Grasel, Fischer & Mandl, 1997). Table 2. System of criteriafor assessment of coordination
1) Time management (global and local) 2) Coordination of work a) division of labor: content b) division of labor: roles c) technical coordination 3) Reference to distribution of knowledge or material (complementary expertise)
Communication analysis. The system of criteria that was developed for the communication-level analysis (see Table 3) assesses the communicative function of domain-related utterances: when the collaborators ask questions (la); when explanations are formulated (lb); and when partners give each other feedback in the sense of showing agreement, disagreement, or demanding further explanation (Ic). In addition, the turn-taking behavior during the collaboration is assessed. It is noted when the partners talk simultaneously, thereby interrupting each other (2a), and when they explicitly hand over a turn (2b). Table 3. System of criteriafor assessment of communication
1) Function of domain-related utterances a) asking the partner about a new content (elicitation) b) explaining a new content to the partner (explication) c) giving feedback agreement disagreement further inquiry / clarification 2) Turn-taking a) simultaneous talk (interruption) b) explicit handover It should be noted that both levels of analysis include only categories unrelated to domain-specific content. Domain-related aspects are assessed separately.
INSTRUCTIONAL SUPPORT FOR COLLABORATION
73
Domain-related analysis. The focus of the domain-related analysis was on "topics" arising within the dialog. With the term "topics", we denote short, identifiable thematic segments within a dialog. A specific symptom corresponding with a chosen diagnosis is an example of such a topic. For example, for the diagnosis "depressive episode", the topic "symptom of constant fatigue" was sometimes discussed in terms of whether it resulted from the patient's medication or was an indicator of a depressive episode. As part of the domain-related analysis, the dialog of a given dyad is scanned to identify such topics on the basis of a predefined list of topics. Each topic is further classified with regard to its general relevance for the solution of the case at hand, the adequacy of the way in which it was discussed, the correctness of the statements, and the depth of the discussion (see Table 4). Table 4. System of ratingsfor topics
1 = irrelevant 2 = possible
3 = relevant 1 = adequate 2 = inadeauate 1 = incorrect 2 = incomplete
Relevance topic that does not contribute to the solution of the case topic that does not directly relate to the solution of the case, but might contribute to the understanding of the case topic that needs to be brought up during discussion to allow for a successful solution of the task Adequacy the topic is discussed in a way that contributes significantly to the advancement of the task solution the t o ~ i cis discussed in an insignificant wav
at least one of the partners makes an incorrect statement due to the incompleteness of a statement, the discussion cannot be regarded as correct only correct statements are made during the discussion, or 3 = correct incorrect statements are corrected Depth of discussion 1 = superficial repeating or enumerating of factual information from the case description or text material without adding new aspects 2 = simple simple explanations or interpretations 3 = elaborated detailed and elaborated exvlanations or intervretations Mutual understanding 1= the listening partner shows signs of not understanding the no understanding speaker (verbally, nonverbally, obvious misunderstanding) or misunderstanding does not attend to the speaker (e.g. reads in his own material while partner is explaining, does not react to the partner) 2 = unclear content of dialog does not allow a determination of whether or not understanding has occurred 3= listening partner explicitly signalizes understanding (verbally, understanding nonverbally or by relating to what was said in his next utterance)
For all topics, their relevance for the solution of the specific case at hand was defined beforehand. Finally, it is inferred from the dialog whether the partners reach a mutual understanding when discussing a topic. The domain-related analysis scheme was developed as part of the diploma thesis of one of our students (Schornstein, 2003). A summary of the levels of dialog analysis, their data source, and unit of analysis is given in Table 5. As all three types of dialog analyses require a great deal of work, in most cases it will be necessary to perform them on a restricted sample and only for a selected part of the collaboration. To analyze dialog at the communication level and the domain-related level, fine-grained analyses are required (looking at turns or topics), which can only be performed on transcribed dialog. For example, in order to analyze the communication with regard to turn-taking, it is necessary to be able to identify individual turns as well as interruptions. The analysis of the coordinationrelated utterances does not necessarily require transcribed dialog, but can be performed on the video recording of an interaction of a dyad if a coarser, more easily identifiable unit of analysis is chosen. In our study, the coordination-level dialog analysis was performed minute-by-minute, similar to the log-file analysis: each minute of dialog was classified for the occurrence of the coordination categories. A minute could be classified for containing utterances on every category. If utterances crossed minute boundaries, they were counted in each minute of their occurrence. Table 5. Data sourcesfor different levels of dialog analysis Unit of analysis
Minute of dialo Topic
4.2 Assessing Joint Outcome Assessing the outcome of computer-mediated collaborative work involves the analysis of a joint solution and the evaluation of its quality. In our type of scenario, the assessment of performance often comprises a content analysis of freely formulated text. We took the approach of defining elements that denote a good solution. After a system of quantitative criteria was developed by experts in the area of psychotherapy, the solution of each dyad was then analyzed for such elements. The sum of the elements was taken as an indicator for the quality of the joint solution. The elaboration of the diagnosis (justification of the diagnosis from case material) and the quality of the planned therapy were analyzed separately. To justify a particular diagnosis, participants were expected to extract symptoms in support of their diagnosis from the case description and relate them to the diagnostic criteria listed in the ICD (International Classification of Diseases; World Health
INSTRUCTIONAL SUPPORT FOR COLLABORATION
75
Organization, 1993). A good therapy plan required goals of the therapy to be specified, therapeutic measures to be planned, and potential problems to be discussed. The therapeutic measures were expected to include both psychological and medical treatments.
4.3 Assessing Individual Knowledge Instructional support measures should have the additional effect of increasing people's knowledge about aspects characteristic of a good collaboration and a good solution of the task. Such effects can be assessed in an individual posttest. The posttest results can be very interesting as they add a different dimension to the results. While the measures both of the collaborative process and of its outcome (the joint solution) assess collaborative skills, the posttest requires explicit verbalization of knowledge about what constitutes a good collaboration and task solution. In our experimental study, the posttest on individual learning effects contained two subscales: (a) knowledge about central aspects of a good collaboration, and (b) knowledge about important elements of a therapy plan. Subscale (a) refers to some macro and micro characteristics of a good collaboration in the given type of scenario. Participants were asked to describe important aspects that needed to be taken into account when collaborating in the present scenario. They were expected to name aspects such as the importance of continuously ensuring mutual understanding, of using the partner as a resource for clarification, and of explicit coordination and division of work. Subscale (b) relates to a facet of the domainspecific demands: the development of a therapy plan. Participants were asked to describe what needed to be included in a thorough therapy plan. They were expected to name elements such as the necessity of specifying the goals of a therapy before thinking about concrete measures, the importance of considering both psychotherapy and pharmacological treatments, and the importance of discussing expected difficulties, such as possible resistance of the patient, relapse, or failure of the therapy. Since most of the analyses described above comprise an analysis of freely formulated text, the reliability of the scoring procedure was safeguarded by having a second, independent judge score parts of the material.
5. AN EXPERIMENT ILLUSTRATING INSTRUCTIONAL MEASURES, EXPERIMENTAL PARADIGM, AND ASSESSMENT METHODS The purpose of this section is to provide an example of an experimental study that we conducted (see Rummel & Spada, accepted) to test the effectiveness of the two instructional support measures introduced above: (1) observational learning from a worked-out collaboration example, and (2) learning from scripted collaboration. By describing this particular study we also wish to exemplify the attempt to assess effects of such support measures on all the three levels described in the previous section: collaborative process, joint outcome, and individual knowledge.
In the experiment, the two instructional support measures were compared to (3) learning from unsupported computer-mediated collaborative problem-solving (unscripted condition), and (4) a control condition without a learning phase. The experimental design is shown in Table 6. Table 6. Experimental Design Model Condition
Script Condition
Unscripted Condition
Control Condition
Learning from unscripted computerNo learning mediated phase collaborative problem-solving
5.1 Task, Participants, and Setting As introduced above, the collaborative task comprised the interdisciplinary solution of psychiatric cases with combined psychological and physical pathology. Dyads, each consisting of a medical student and a student of psychology, were asked to jointly diagnose the patients described in the cases and to develop a suitable therapy plan making use of their complementary expertise. In each condition, 9 dyads were administered. Two psychiatric cases were utilized in the experiment: case 1 for the learning phase and case 2 for the application phase. In both cases a psychological disorder coincided with some physical illness. Thus both cases made it necessary to take advantage of the complementary domain knowledge represented in each dyad. In order to make the correct diagnosis and map out an adequate therapy plan, both medical and psychological aspects had to be considered. Throughout the collaboration, dyads communicated via a desktop videoconferencing system (VCON, ViGO professional) including audio and video connection, personal text editors and a shared text editor (WordPad shared with MS NetMeeting). The scenario supported synchronous verbal communication and joint activities (e.g. editing of the joint solution) as well as individual work phases.
5.2 Experimental Conditions The experimental variation during the learning phase (see Table 6 ) was implemented in the following way:
INSTRUCTIONAL SUPPORT FOR COLLABORATION
77
Participants in the model condition observed scenes of collaborative "model" problem-solving between a student of psychology and a medical student who are collaborating on the first psychiatric case (case 1). The scenes were presented in a multimedia presentation via audio supplemented by animated text, which showed how the model collaborators developed the joint solution. Thus, the worked-out collaboration example modeled the solution steps necessary to solve the case and illustrated the joint solutions for the diagnosis and the therapy plan. To facilitate elaboration and learning, the model collaboration was accompanied by instructional explanations. In addition, self-explanation activities were promoted by the prompting of collaborative self-explanation phases in the course of the model presentation. Dyads in the script condition were provided with a detailed script prescribing specific phases for their interaction. The script was structurally equivalent to the worked-out collaboration example, meaning that participants in this condition actively engaged in the same collaborative phases that were presented to the participants in the model scenes. To control for learning effects of collaborative problem-solving without instructional guidance, dyads in the so-called unscripted condition collaborated in the learning phase without additional support. The control condition was restricted to collaboration in the application phase. These dyads had no opportunity to gain experience in collaborating during the learning phase. The activity during the application phase was the same in all four conditions: computer-mediated collaborative problem-solving. Dyads collaborated via the desktop videoconference system in order to formulate a diagnosis and to work out a therapy plan for the patient introduced in the second case. No further instruction or help was provided in any of the conditions. Participants in the instructional conditions were expected to outperform their uninstructed counterparts on all dependent measures: the variables reflecting the quality of the collaborative problem-solving process during the application phase, the quality of the joint solution, and the individual knowledge posttest. A slight advantage was expected for the model condition compared to the script condition due to motivational problems associated with the cooperation script. With regard to the other two conditions, the unscripted condition should not significantly outperform the control condition, due to the high cognitive demands of unsupported collaboration in the learning phase.
5.3 Assessing the Dependent Variables The dependent variables were assessed as described in the previous section. Table 7 gives an overview of the analyses, the data sources, and sample sizes of the analyses. For the outcome of the collaboration, the joint solutions on diagnosis and therapy plan of all 36 dyads were analyzed. The knowledge posttest was answered individually, resulting in a sample size of 72 for this variable. An activity pattern like the one shown in Figure 1 was extracted from log-file data for each of the 36
dyads. These activity patterns were analyzed with regard to the amount of individual work. To elucidate the collaborative process in greater depth than could be achieved with the log-file analysis, our system of criteria for assessment of the coordination was applied to the video recordings of the dialog of all 36 dyads, although only on the diagnosis part of the dialogs. The communication categories were applied to the transcribed dialogs (diagnosis part) of a restricted sample of 8 dyads. This sample comprised two dyads from each condition: one "successful" and one "unsuccessful" dyad as defined by the performance on outcome and posttest. The same restricted sample was used for the analysis of the domain-related dialog (topics). Table 7. Overview of dependent variables and their assessment
I
De~endentvariable outcome
I
Data source joint solution text on diagnosis and therapy plan
Samule 36 dyads
knowledge about good 72 individuals answers to posttest collaboration and solution collaborative process: activity patterns extracted 36 dyads from log-files amount of individual work collaborative process: video recordings of dialog 36 dyads coordination collaborative process: 8 selected dyads transcribed dialog communication collaborative process: 8 selected dyads transcribed dialog topics 5.4 Summary and Discussion of Results
In the following, we give a short overview of the central results on the different levels of assessment. For more details see Rummel & Spada (accepted). 5.4.1 Outcome
The results for the quality of the joint solution can be summarized in that dyads in the instructional model condition outperformed their counterparts in the unscripted and control conditions in terms both of diagnosis and therapy plan. The dyads in the script condition yielded outstanding results only for the therapy plan. There were no differences between the unscripted collaboration and the control condition without learning. 5.4.2 Knowledge Posttest
The posttest results revealed a clear superiority of the instructional conditions (model and script) on both subscales of the posttest. This implies that participants in the instructional conditions were not only able to profit from the instruction they received during the learning phase in terms of their subsequent collaboration (as
INSTRUCTIONAL SUPPORT FOR COLLABORATION
79
evident in the results on the process - see below - and outcome variables), but also benefited with regard to the explicit knowledge they had acquired about important aspects of a good collaboration and a good therapy plan.
5.4.3 Collaborative Process Activity patterns. Summarizing the results of the log-file analysis of the activity patterns during the application phase, it was apparent that both of the instructional conditions - and especially the model condition - showed a substantial amount of individual work. In comparison, dyads in the unscripted and the control condition on average showed an insufficient amount of parallel individual work. A particularly interesting finding is that the variances of the amount of individual work differed remarkably between the four conditions. Dyads in the unscripted and the control conditions in particular showed an enormous amount of variance in the way in which they collaborated. This confirms a phenomenon well known from the literature on collaborative problem-solving and learning: without support, people differ greatly in the way in which they collaborate (for examples see Johnson & Johnson, 1992; Slavin, 1995; Salomon & Globerson, 1989). The result also supports findings of Hermann et al. (2001), who showed that there is a risk of individual work being neglected in desktop videoconference settings. In accordance with their findings, in our study the average amount of individual work in all four conditions was lower than the amount of individual work in the instruction, and particularly so in the conditions without instruction. Dialog analysis: coordination of joint work. In comparing the results of the dialog analyses of the coordination between the four conditions, a particularly notable result was that the number of minutes that contain utterances within these categories was consistently very low under the unscripted condition compared with the three other conditions, which did not show any remarkable differences. Assuming that more coordinative dialog is needed when collaborating for the first time, this result for the unscripted condition is not surprising. The participants in this condition had already collaborated before and had therefore already gone through the process of coordinating their joint work once. It is natural that less coordinative activity would be expected during their second collaboration compared to.dyads in the control condition, who are collaborating for the very first time during the application phase. But why did dyads in the instructional conditions (model and script) show as many coordinative utterances as those in the control condition? Participants in these experimental conditions had been instructed that coordinative dialog is necessary for good time management and good coordination of work. Thus, the activities that we see in these conditions would be expected to be of different and better quality than the activities in the control condition. But can we tell quantity and quality apart with our analysis of the coordination? The answer is: not really. We will come back to this issue in the concluding section.
Dialog analysis: communication and domain-related aspects. As the analysis of the communication and the domain-related content were performed only on 8 selected dyads (one "successful" and one "unsuccessful" dyad from each condition), the data does not allow for a comparison of the four conditions. Descriptively comparing the results of the successful dyads with the unsuccessful dyads did not reveal any systematic differences. However, before extending this time-consuming analysis to the entire sample, the benefits that might be gained need to be weighed against the time expenditure. We will come back to this point in the following section. 6. INSTRUCTIONAL SUPPORT MEASURES AND METHODS OF ASSESSMENT: LESSONS LEARNED
In this section, we will discuss the results gained from the reported experimental study in terms of two main questions. First, what do they tell us about the effects of our instructional support measures on computer-mediated collaboration? And second, how appropriate were the methods used to assess process and outcome of the collaboration?
6.1 Instructional Support Measures for Achieving Good Computer-Mediated Collaboration In sum, both of the instructional methods that were developed and empirically tested showed the capability to improve collaboration in the given scenario. Dyads who received instruction by observing the worked-out collaboration example during the learning phase (model condition) as well as dyads who were led through the collaboration in the learning phase step-by-step by a cooperation script (script condition) outperformed their counterparts in the unscripted and the control condition on many of the aspects assessed with regard to the subsequent collaborative process, its joint outcome, and an individual posttest. These results lead us to the following conclusions: The poor results of the unscripted condition in terms of the subsequent collaboration indicate that learning from unsupported computer-mediated collaboration is not very effective in promoting skills relevant for this kind of collaboration. The reason for this might be that the situation is so demanding (the collaborative work on the psychiatric case itself, the interdisciplinary communication, and the technical setting) that it prevents people from paying attention to and reflecting on the critical aspects of the collaborative process. Comparing the collaborative work of these dyads between the "learning" and the application phase - both without instructional support in this condition (see Table 6) - yielded an interesting result. The collaborative process of the individual dyads was quite similar for both collaborations, e.g. with regard to the distribution of individual and joint work. In other words, dyads collaborating in disadvantageous ways were not able to improve their collaboration.
INSTRUCTIONAL SUPPORT FOR COLLABORATION
81
Observational learning from a worked-out collaboration example can obviously be a successful way of promoting subsequent collaboration. If such an example is well conceived, it can function as a model for the people observing the collaboration. However, in spite of all their strengths, worked-out examples may lead to superficial processing of the example features and, in consequence, a mere illusion of understanding (Renkl, 1997). This danger could be counteracted by promoting the elaboration of the example, particularly by eliciting self-explanations (Renkl, Stark, Gruber & Mandl, 1998) and providing instructional explanations (Renkl, 2002). Similar to research on worked-out examples, in the context of behavior modeling training (Goldstein & Sorcher, 1974), Decker (1984) has shown the importance of "learning points" - instructional explanations accompanying the model's behavior. It follows that when observing the worked-out collaboration example, participants should be guided to reflect upon what they see and hear. Cooperation scripts can trigger learning about collaboration. Partners who work jointly on a problem-solving task following a cooperation script acquire collaborative skills and knowledge that also improve the collaboration and outcome in a subsequent task. In future research, cooperation scripts should be considered more closely as a promising instructional measure and not only a means of providing support during an ongoing collaboration (Hron et al., 1997; Reiserer et al. 2002). It should not be overlooked, however, that cooperation scripts may cause motivational problems since they often regulate the interaction in a strict manner. Following the motivation theory of Deci and Ryan (1985), which identifies selfdetermination as a major constituent of motivation, negative motivational responses of participants are to be expected. Kollar (2001) has presented preliminary results in support of this assumption. Motivational reactance towards the cooperation scripts might impede the internalization of the script as a standard for subsequent collaborative work. To counteract possible motivational problems caused by cooperation scripts, the collaborating partners should be guided to reflect on the relevant features of the script in order to understand their usefulness. In our current research we are addressing the issue of elaboration support discussed for both instructional measures: observational learning from a worked-out collaboration example and learning from scripted collaboration. By conducting an experimental study comparing the effects of a model and a script with elaboration support and without such support, our aim is to assess the importance of guiding meta-reflection about the instructional support measures. We are using the same experimental paradigm as in the study described in this chapter.
6.2 Methods of Assessing Process and Outconze of Collaboration In evaluating the proposed methods of assessment, our focus is on the process analyses. The analysis of activity patterns from the log-file data has been shown to provide information about a central aspect of the collaborative process: the distribution of individual and joint phases of work. This information is particularly valuable because a balanced proportion of joint and individual work (Hermann et al., 2001)
with a substantial amount of individual work (Rummel & Spada, accepted) has proven to be an important factor of successful computer-mediated, interdisciplinary collaboration. However, it is not only the overall amounts of individual and joint work that can be deduced by analyzing the activity patterns. Many other variables describing surface features of the collaboration, such as time spent by each partner on writing in personal text editors or reading the material, can be assessed from this data source. From a practical point of view, this type of analysis has proved to be very feasible compared to the high expenditure of time required for the dialog analyses. When evaluating the dialog analysis with regard to coordination it is important to note that these scores were gained through a time-sampling procedure: each minute of dialog was classified for the occurrence of the coordination categories (see Table 2). In other words, what was being assessed was the quantity - or intensity of dialog activity with regard to these aspects. The quantity of coordinative dialog activity is of interest, yet its relation to the quality of the collaborative work is complicated. A substantial amount of coordinative dialog is necessary to structure the collaboration in an optimal way, particularly when working jointly for the first time. Too much coordinative dialog, on the other hand, reduces the amount of time spent on the task itself, for domain-related dialog, and for writing. In short, a consideration of time management or a coordination of the division of labor is relevant, but what is decisive is the quality of this interaction. However, counting the minutes of this type of dialog does not give clear information about the quality of the time management and the coordination of work. In the light of this criticism it would be desirable to complement the quantitative analysis by assessing the quality of the coordinative dialog. A promising step in that direction has been taken by a qualitative content analysis of the dialog that was performed as part of a student's diploma thesis (Sosa y Fink, 2003). First, a thorough literature review was conducted to identify additional aspects relevant for a successful collaboration. In addition, the transcripts of four collaborative dialogs from the unscripted condition were examined in an extensive analytic procedure following the qualitative methodology of Mayring (2003). From these analyses, seven dimensions were identified to be constitutive of a good collaboration: (1) task alignment, (2) goal conformity of the partners, (3) self-presentation, (4) coordination, (5) construction of a common knowledge base, (6) social competence in conflict, (7) structure of the problem-solving process. Subsequently, these dimensions were applied to the video recordings of the remaining 5 dyads in the unscripted condition and the entire 9 dyads in the control condition in a rating procedure. The rating revealed promising results: a substantial correlation between the seven dimensions and the quality of the joint solution of these 14 dyads was found (r=0.62). The dimensions seem to be adequate to distinguish successful vs. less successful collaborations. Currently, the system is being revised and cross-validated by applying it to another sample of dyads from a second experiment. As discussed for the dialog analysis with regard to coordination, we should keep in mind that counting the occurrence of utterances on the communication level in the way it was done in our approach does not give clear evidence about the quality of
INSTRUCTIONAL SUPPORT FOR COLLABORATION
83
these utterances. Our system of criteria for the communication was developed to serve as an assessment tool that is not domain specific, but "content-free". Perhaps also for the assessment of the characteristics of the communication, a more qualitative, content-related approach of analysis would be better suited. After all, we should question whether the communication features of an interaction are in fact such a significant indicator for the success of a collaboration. It might well be that the relevance of this level for the collaboration has been overestimated. Basic features such as turn-taking, giving feedback (especially in the sense of backchanneling), asking for information, or explaining something to somebody are characteristic of everyday communication. If problems occur at this level, they may have more to do with long established individual differences in communicating than with new problems arising from the computer-mediated collaboration and our experimental conditions. Moreover, our participants were certainly well aware that they were collaborating in an "experimental", new situation that requires specific attention to be paid to these aspects. The assessment of individual knowledge about central aspects of a good collaboration and about one aspect of the domain, namely important elements of a therapy plan, did not pose problems. Due to the complexity of the domain, the assessment of the quality of the joint solution (the diagnosis and therapy plan) was not as easy. Cases like those used in our experiment can be examined in terms of different aspects. Our system of criteria for evaluating the joint diagnosis and the planned therapy was developed by experts. However, the application of the criteria to the particular solutions by the dyads was not always easy and straightforward
6.3 Challenges Met, Barriers Overcome, Chances Realized? To summarize our results, it can be stated that particularly dyads in the instructional conditions of the experiment were able to meet the challenges of working in collaboration, with complementary expertise, and in a desktop videoconference setting. Some of the dyads without instructional support also worked quite efficiently, but many had problems. The technical setting allowed work to be carried out jointly and individually, but predominantly invited the partners to work jointly. However, when the participants in our experiment were speaking to each other during the collaboration, it was difficult for them to concentrate sufficiently on their own field of expertise and to reflect upon the jointly developed solutions against the background of their own discipline. The poor results of unsupported computermediated collaboration of the dyads in the unscripted condition indicate that the challenges described at the beginning of this chapter are real barriers if no support is given. Compared to support implemented during the ongoing collaboration, our instructional approach aims to achieve long-lasting, sustainable effects. Up to now we have been able to demonstrate effects on one subsequent collaboration. In order to test long-lasting effects, an assessment of the collaboration for much longer periods of time would be required. Thus at the moment the claim of potentially longlasting effects is nothing more than a hypothesis - but it is a promising hypothesis.
When analyzing the video data of the collaboration we gained the impression that the visibility of the face of the partner was not of central importance for the participants. Indispensable were the features of audibility (talking with each other) and reviewability and revisability (working with a shared workspace). These features go hand in hand with cotemporality and simultaneity, both of which are very important for a lively collaboration. What about the opportunities of the investigated scenario? Remote collaboration of experts in desktop videoconference settings can achieve good results. Instructional support helps the collaborators to apply their complementary knowledge to the problem at hand. The ability for the collaborators to talk to each other provides a natural way to discuss a problem and a shared workspace adds the dimension of a copresent work product. Spontaneity meets reviewability and revisability. ACKNOWLEDGEMENTS The present research was supported by the Deutsche Forschungsgemeinschaft (DFG; [German Research Foundation]) with a project grant awarded to the second author and Prof F r a u Caspar (contract Sp 251/16-1). We thank Prof Franz Caspar for his work on the clinical aspects of the project. Many thanks also go to our students Jana GroJ Ophofi Stefanie Sosa y Fink and Katrin Schornstein who have completed their diploma thesis as part of the project, and who have helped with many practical aspects of the project such as data collection, scoring, and statistical analysis. REFERENCES Anderson, A. H., O'Malley, C., Doherty-Sneddon, G., Langton, S., Newlands, A., Mullin, J., Fleming, A. M. & Van der Felden, J. (1997). The impact of VMC on collaborative problem solving: An analysis of task performance, communicative process, and user satisfaction. In K. E. Finn, A. J. Sellen & S. B. Wilbur (Eds.), Video-mediated cornmunicatiort (pp. 133-156). Mahwah, NJ: Lawrence Erlbaum Associates. Angiolillo, J. S., Blanchard, H. E., Israelski, E. W. & Man& A. (1997). Technology constraints of videomediated communication. In K. E. Finn, A. J. Sellen & S. B. Wilbur (Eds.), Video-mediated comn~unicatiort(pp. 5 1-74). Mahwah, NJ: Lawrence Erlbaum Associates. Baker, M., & Lund, K. (1997). Promoting reflective interactions in a CSCL environment. Journal of Cornputer Assisted Learning, 13(3), 175-193. Baker, M., Hansen, T., Joinier, H. & Traum, D. (1999). The role of grounding in collaborative learning tasks. In P. Dillenbourg (Ed.), Collaborative Learning. Cognitive and conzputational approaches (pp. 3 1-63). Amsterdam: Pergamon. Bandura, A. (1977). Social learning theoryLEnglewood Cliffs, NJ: Prentice Hall. Baron, B. (2000). Achieving coordination in collaborative problem-solving groups. The Journal of the Learning Sciences, 9,403-436. Bromme, R. (2000). Beyond one's own perspective: The psychology of cognitive interdisciplinarity. In P. Weingart & N. Stehr (Eds.), Practicing Interdisciplinarity (pp. 115-133). Toronto: Toronto University Press. Bruhn, J. (2000). Forderung des kooperativen Lernens uber Conlputernetze: Prozess und Lernetfolg beirn dyadischen Lernen rnit Desktop-Videokonferenzen. Frankfurt a. M.: Peter Lang.
INSTRUCTIONAL SUPPORT FOR COLLABORATION
85
Bruhn, J., Fischer, F., Gdsel, C., & Mandl, H., (2000). Kooperatives Lernen mit Mapping-Techniken. In H. Mandl & F. Fischer (Eds.), Wissen siclztbar nzachen. Wissensmanagenzent mit MappingTechniken, 119-133. Gottingen: Hogrefe. Bruhn, J., Grasel, C., Fischer, F. & Mandl, H. (1997). Kategoriensystem zur Erfassung der gerizeinsarizen Wissenskonstruktion irrz Diskur.~:Unvertiffentlichtes Manuskript. Ludwig-Maximilians-Universittit Munchen: Institut fur Empirische Ptidagogik und Rdagogische Psychologie. Cannon-Bowers, J. A., & Salas, E. (1998). Team performance and training in complex environments: Recent findings from applied research. Current Directions in Psyc/zological Science, 7(3), 83-87. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P. & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182. Clark, H. H. & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine & S. D. Teasley (Eds.), Perspectives on socially shared cogrlition (pp. 127-148). Washington, DC: American Psychological Association. Collins, A., Brown, J. S. & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and instruction (pp. 453-494). Hillsdale, NJ: Lawrence Erlbaum Associates. Cox, R., McKendree, J., Tobin, R., Lee, J. & Mayes, T. (1999). Vicarious learning from dialog and discourse, Instructional Science, 27,43 1-458. Deci, E. L. & Ryan, R. M. (1985). Irztrinsic motivation and self-deterrrilation in l~wnanbehavior, New York: Plenum. Decker, P. J. (1984). Effects of different symbolic coding stimuli in behavior modeling training. Persorznel Psychology, 37(4), 71 1-720. Diehl, M. & Stroebe, W. (1991). Productivity loss in idea-generating groups: Tracking down the blocking effect. Journal of Personality and Social Psychology, 61,392-403. Dillenbourg, P. & Traum, D. (1999). Does a shared screen make a shared solution? In C. M. Hoadley & J. Roschelle (Eds.), Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference (pp. 127-135). Mahwah, NJ: Lawrence Erlbaum Associates. Dillenbourg, P., Baker, M., Blaye, A. & O'Malley, C. (1995). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.), Learning in lzunzans and nzaclzines: Towards an interdisciplinarylearning science (pp. 189-211). Oxford: Elsevier/Pergamon. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S. Scott, P. & Trow, M. (1994). The new production of knowledge. The dyrzarnics of science and research in conterizporary societies. London: Sage. Goldstein, A. P. & Sorcher, M. (1974). Changing supervisor behavior, New York: Pergamon Press. Greeno, J. G. and the Middle School Mathematics through Applications Project Group. (1998). The situativity of knowing, learning and research. Aazerican Psyckologist, 53, 5-26. GroR Ophoff, J. (2003). Elemente einer erfolgreichen Kooperation. Analyse zum Zusammenhang von Kooperationsprozess und -produkt bei netzbasiertem kooperativen Problemltisen unter komplementtirer Expertise. Unveriiffentlichte Diplomarbeit, Albert-Ludwigs-Universitlt, Freiburg. Giirer, D., Kozma, R. & Millin, E. (1999). Impact of shared applications and implications for the design of collaborative learning environments. In S. Lajoie (Ed.), Proceedings for the ninth world corlference on artificial intelligence in education (AI-ED99, pp. 439-445). Amsterdam: IOS Press. Hermann, F., Rummel, N. & Spada, H. (2001). Solving the case together: The challenge of net-based interdisciplinary collaboration. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), Proceedirzgs of the First European Confererzce on Computer-Supported Collaborative Learning (pp. 293-300). Maastricht: McLuhan Institute. Hron, A,, Hesse F. W., Reinhard, P. & Picard, E. (1997). Strukturierte Kooperation beim computergestutzten kollaborativen Lernen. Ur~terricl~t.swi.ssenscl~afi, 25(1), 56-69. Johnson, D. W. & Johnson, R. T. (1992). Key to effective cooperation. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups. The theoretical anatorizy qf group learning (pp. 174199). New York: Cambridge University Press. Joiner, R., Scanlon, E., O'Shea, T. Smith, R. B. & Blake, C. (2002). Evidence from a series of experiments on video-mediated collaboration: Does eye contact matter? In G. Stahl (Ed.), Proceedings of the Corizputer Support for Collaborative Learning (CSCL) 2002 Conference (pp. 371-388). Mahwah, NJ: Lawrence Erlbaum Associates.
Jucks, R., Bromme, R. & Runde, A. (2003). Audience Design von Experten in der netzgestiitzten Kommunikation: Die Rolle von Heuristiken uber das geteilte Vorwissen. Zeitschriftfiir Psychologie, 211(2), 60-74. Kato, H., Yamazaki, K., Suzuki, H., Kuzoka, H., Miki, H. & Yamazaki, A. (2002). Designing a videomediated collaboration system based on a body metaphor. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL 2: Carryingforward the conversation.Mahwah, NJ: Lawrence Erlbaum Associates. Kneser, C. & Pldtzner, R. (2001). Collaboration on the basis of complementary domain knowledge: Observed dialogue structures and their relation to learning success. Learning and Instruction, 11,5283. Kollar, I. (2001). Gewissheits- und Ungewissheitsorientierung beirn kooperativen Lernen rnit Unverdffentlichte Videokonferenzen - der Einfluss verschiedener StrukturierungsrizaJnaI~~iler~~ Magisterarbeit. Ludwig-Maximilians-UniversitltMiinchen: Institut fur Empirische Padagogik und Pldagogische Psychologie. Larson, J. R., Christensen, C., Franz, T. M. & Abbott, A. S. (1998). Diagnosing groups: The pooling, management, and impact of shared and unshared case information in team-based medical decision making. Journal of Personality and Social Psychology, 75(1), 93-108. Latham, G. P. & Saari, L. M. (1979). Application of social-learning theory on training supervisors through behavior modelingJourna1 of Applied Psychology, 64,239-246. Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation, Cambridge, U K : Cambridge University Press. Leskovac, H. (1998). Distance learning in legal education: Implications of frame relay videoconferencing. Albany Law Journal of Science & Technology, 8, 305-335. Lewis, C. & Sycara, K. (1993). Reaching informed agreement in multispecialist cooperation. Group Decision and Negotiation, 2, 279-299. Malone, T. W. & Crowston, K. (1990). What is coordination theory and how can it help design cooperative work systems? Proceedings qf the Conference on Conzputer-Supported Cooperative Work, 357-370. Mayer, R. E. & Wittrock, M. C. (1996). Problem-solving and transfer. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of Educational Psychology. New York: Macmillan. Mayring, P. (2003). Qualitative Inhaltsanalyse. Grundlagen und Techniken (8. Aufl.). Weinheim: Beltz. McGrath, J. E. (1984). Groups: Interaction andpei$ormance, Englewood Cliffs, NJ: Prentice Hall. Meyer, H. H. & Raich, M. S. (1983). An objective evaluation of a behavior modeling training program. Personnel Psychology, 36,755-761. Nardi, B. A,, Kuchinsky, A., Whittaker, S., Leichner, R. & Schwarz, H. (1997). Video-as-data: Technical and social aspects of a collaborative multimedia application. In K. E. Finn, A. J. Sellen & S. B. Wilbur (Eds.), Video-mediated conzrnunication (pp. 487-517). Mahwah, NJ: Lawrence Erlbaum Associates. O'Conaill, B. & Whittaker, S. (1997). Characterizing, predicting, and measuring video-mediated communication: A conversational approach. In K. E. Finn, A. J. Sellen & S. B. Wilbur (Eds.), Videomediated conzrnunication (pp. 107-132). Mahwah, NJ: Lawrence Erlbaum Associates. O'Donnell, A. M. & Dansereau, D. F. (1992). Scripted cooperation in student dyads: A method for analyzing and enhancing academic learning and performance. In R. Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups. The theoretical anatomy of group learning (pp. 120-141). New York: Cambridge University Press. O'Donnell, A. M. (1999). Structuring dyadic interaction through scripted cooperation. In A. M. O'Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 179-196). Mahwah, NJ: Lawrence Erlbaum Associates. Olson, G. M., Malone, T. & Smith, J. (Eds.). (2001). Coordination theory and collaboration technology. Mahwah, NJ: Lawrence Erlbaum Associates. Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1(2), 117-175. Pldtzner, R., Fehse, E., Kneser, C. & Spada, H. (1999). Learning to relate qualitative and quantitative problem representations in a model-based setting for collaborative problem-solving. The Journal q f the Learning Sciences, 8, 177-2 14. Reimann, P. (1997). Lernprozesse beim Wissenserwerb nus Beispielen: Analyse, Modellierung, Fiirderung; Bern: Huber.
INSTRUCTIONAL SUPPORT FOR COLLABORATION
87
Reiserer, M., Ertl, B. & Mandl, H. (2002). Fostering collaborative knowledge construction in desktop videoconferencing. Effects of content schemes and cooperation scripts in peer-teaching settings. In G. Stahl (Ed.), Proceedings of the Computer Support for Collaborative Learning (CSCL) 2002 Conference (pp. 379-388). Mahwah, NJ: Lawrence Erlbaum Associates. Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21, 1-29. Renkl, A. (2002). Worked-out examples: Instructional explanations supplement self-explanations. Learning and Instruction. 12,529-556. Renkl, A., Mandl, H. & Gruber, H. (1996). Inert knowledge: Analyses and remedies. Educational Psychologist, 31, 115-121. Renkl, A., Stark, R., Gruber, H. & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Conterizporary Educational Psychology, 23, 90108. Roschelle, J. (1992). Learning by collaborating: Convergent conceptual change. The Journal of the Learning Sciences, 2(3), 235-276. Rosenshine, B., & Meister, C. (1994). Reciprocal teaching: A review of the research. Review of Educational Research, 64(4), 479-530. Rummel, N., & Spada, H. (accepted). Learning to collaborate: An instructional approach to promoting collaborative problem-solving in computer-mediated settings. Journal of the Learning Sciences. Sacks, H., Schlegloff, E. & Jefferson, G. (1974). A simplest systematics for the organization of turntaking in conversation. Language, 50,696-753. Salomon, G. & Globerson, T. (1989). When teams do not function the way they ought to. International Journal of Educational Research, 13, 89-99. Schornstein, K. (2003). Kooperation lernen: Entwicklung eines Ausbildungselementes zur FBrderung der Kooperation zwischen Studierenden der Psychologie und der Medizin. UnverBffentlichte Diplomarbeit, Albert-Ludwigs-Universitlt, Freiburg. Slavin, R. E. (1995). Cooperative learning: Theory, research, andpractice (2nd ed.). Boston, MA: Allyn & Bacon. Sosa y Fink, S. (2003). Merkmale gelungener Kooperation. Eine qualitative Analyse netzgestiitzter Zusarnmenarbeit. UnverBffentlichte Diplomarbeit, Albert-Ludwigs-Universitat, Freiburg. Stasser, G., & Titus, W. (1985). Pooling of unshared information in group decision making: Biased information sampling during group discussion. Journal of Personality and Social Psychology, 48, 1467-1478. Stasser, G., Stewart, D. D. & Wittenbaum, G. M. (1995). Expert roles and information exchange during discussion: The importance of knowing who knows what. Journal of Experimental Social Psychology, 31, 244-265. Stenning, K., McKendree, J., Lee, J., Cox, R., Dineen, F. & Mayes, T. (1999). Vicarious learning from educational dialog. In C. Hoadley & J. Roschelle (Eds.), Proceedings of the Conzputer Support for Collaborative Learning (CSCL) 1999 Conference (pp. 341-347). Mahwah, NJ: Lawrence Erlbaum Associates. Suthers, D. (2001). Collaborative representations: Supporting face to face and online knowledge-building discourse. Proceedings of the 34th Hawai'i International Conference on the System Sciences (HICSS-34), January 3-6, 2001, Maui, Hawaii (CD-ROM), Institute of Electrical and Electronics Engineers. Sweller, J. & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2,59-89. Sweller, J., VanMemenboer, J. J. G., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, @,25 1-296. Thompson Klein, J. & Porter, A. L. (1990). Preconditions of interdisciplinary research. In Ph. H. Birnbaum-More, F. A. Rossini & D. R. Baldwin (Eds.), International research management. Studies in interdisciplinary methods froriz business, government and academia (pp. 11-19). New York: Oxford University Press. VanBruggen, J. M., Kirschner, P. A. & Jochems, W. (2002). External representation of argumentation in CSCL and the management of cognitive load. Learning and Instruction, 12, 121-1 38. VanLehn, K. (1996). Cognitive skill acquisition. Annual Review of Psychology, 47, 513-539. Veermann, A. & Veldhuis-Diermanse, E. (2001). Collaborative learning through computer-mediated communication in academic eduaction. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.),
Proceedings of the First European Conferenceon Cornputer-SupportedCollaborative karning (pp. 625632). Maastricht: McLuhan Institute. Webb, N. M. (1989). Peer interaction and learning in small groups. Journal of Educational Research, 13, 21-39. Whittaker, S. (1995). Rethinking video as a technology for interpersonal communication: Theory and design implications. lnternational Journal of Hutnun-Computer Studies, 42,501-529. Whittaker, S., Geelhoed, E. & Robinson, E. (1993). Shared workspaces: How do they work and when are they useful? International Journal of Man-Machine Studies, 39,813-842. World Health Organisation: Tenth Revision of the International Classification of Diseases, Chapter V (F): Mental and Behavioural Disorders. Diagnostic Criteria for Research. World Health Organisation, 1993.
rummel @psychologie.uni-freiburg.de spada @psychologie.uni-freiburg.de
RAINER BROMME, REGINA JUCKS & ANNE RUNDE
BARRIERS AND BIASES IN COMPUTER-MEDIATED EXPERT-LAYPERSON-COMMUNICATION An overview and insights into thefield of medical advice
1. TWO VIGNETTES AND A SHORT INTRODUCTION The layperson seeking medical advice. Mr. Smith has watched a television program on health issues and is now wondering whether, since he has reached the age of 52, it is really necessary for him to give up butter in order to avoid a heart attack. At the next opportunity he asks his doctor. The doctor takes a blood sample but the results are inconclusive. As the doctor is pressed for time his explanations are not very clear and rather difficult for Mr. Smith to understand. So Mr. Smith is none the wiser. This is all the more puzzling for him because a friend of his of about the same age who has a similar life style received some very clear advice from his - different doctor. However, this advice unfortunately was exactly the opposite of what Mr. Smith thought his doctor had told him. At this point Mr. Smith decides to get some more information from the Internet to help him solve his problem. He finds a mass of information and learns something about the complex links between life-styles, risk factors, diets, cholesterol and cardiovascular disease. Some of the material on offer is aimed directly at the layperson. However, Mr. Smith does not know whether the information can be relied upon, since it has clearly been compiled by firms that are no doubt interested in making money from people who take their advice. In his search he also comes across web pages of serious journals that have obviously not been formulated with the layperson in mind. These seem to contain interesting information much of which, however, he does not understand properly. After a while he comes to feel that his new knowledge helps him to realize the complexity of the problem, but still does not offer a conclusive answer. Finally he finds a web site that allows him to put his own individual question to an expert and receive an answer. As he is familiar with computers he includes a graphic in his query that he has come across in a journal during his earlier searches. Though he did not fully understand it he thought the illustration would help to clarify his query. The expert giving medical advice. Dr. Muller has taken on a new position as an expert working on a medical hotline. The web site is financed by a public institution and is intended to contribute to health education. Dr. Muller worked as a doctor in a hospital before taking up her new post. She had enjoyed advising patients and their
relatives. By watching people's gestures during her conversations with them she soon developed a feeling for how much of what she said to her patients had actually been understood. Particularly when there was little time for lengthy explanations she thought it important to adapt to the speech partners' level of understanding. In her new job she finds herself wondering how much the people whose enquiries she is reading already know on the subject, and how much the medical layperson will understand of her advice. In fact, she has to reply to questions from people she knows virtually nothing about. How much use will her previous experience be in her new work? On the other hand, she is pleased to see how much lower the threshold is for people to write in rather than go to a doctor or hospital for health advice. The scenarios illustrate a function that the Internet increasingly have in our knowledge society: helping laypersons to make informed decisions. A patient must, for example, give his permission for an operation, and should know to some extent the alternatives and the risks involved. In this chapter we will discuss the opportunities for and barriers to computermediated communication between experts and laypersons. In the subsequent subsection (2.1) we will begin by looking at the opportunities and barriers that result from computer mediation in the scenarios which we have described above. Medical advice is a special case of expert-layperson-communication and as such it has characteristic problems of its own, which will be discussed next (2.2). These problems result from the deep and systematic difference between the everyday amateur knowledge of laypersons and expert knowledge. We will argue that the problematic impact of such knowledge differences might be augmented by a computer as the medium of communication. Next, in section 3.1. and 3.2., such communication problems will be reconstructed more theoretically, based on a recent debate about theoretical accounts proposed by Clark (1996) and especially about the (non-) cooperative nature of interpersonal communication. We will argue that experts might suffer from an illusion of evidence when communicating with laypersons in computer-mediated settings. This hypothesis is explicated with regard to the use of graphical representations of specialist knowledge (4.1.). In sections 4.2. and 4.3. interrelated studies are outlined in order to illustrate how barriers and biases in expert-laypersoncommunication could be theoretically reconstructed and tackled empirically. Having summarized our findings (4.4.) we conclude with a synthesis (5.). 2. OPPORTUNITIES AND BARRIERS 2.1. Opportunities for and barriers to distributing and finding specialized medical knowledge via the Internet. The Internet is becoming a more and more important medium where laypersons can ask for and obtain experts' medical advice (Beredjiklian, Bozentka, Steinberg, & Bernstein, 2000). By the end of 1999, Gemini Consulting (2001) had found over 600.000 Internet pages on the subject of health in German-speaking areas alone. The
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
91
number of users searching the Internet for information on health is rising considerably faster than the number of general users (Pricewaterhouse Coopers, 2001). It is an important medium for health communication on prevention issues. It will not of course replace personal interaction between doctors and patients and their relatives. However, Internet information can complement such interaction and help patients to understand the advice given to them by medical personnel face to face (ftf), or to comprehend other relevant health information that they have obtained from other sources. In addition, anonymity that is possible when sending in a query can be regarded as an advantage. Information that laypersons have obtained via the Internet may make it easier for them to actually consult a doctor. Since compliance is an important therapeutic factor and since it is based on a full understanding of the case on the part of relatives and patients, all sources of information that foster such an understanding are useful. Thus the Internet can contribute to the participation of laypersons in expert knowledge, something which was hitherto not easily accessible to non-experts (see also Fischer & Ostwald on the participative effects of computer systems in this book). This is the enabling aspect of computer mediation in the search for medical advice. It is based on the availability of a huge amount of data, on the availability of search engines that make specific inquiries possible, on the openness of the Internet that allows room for controversies, and on the possibility of finding very up-to-date information. Every one of these enabling features, however, is accompanied by features that contain barriers to gathering information. The most important barrier concerns the quality of the information available (quality control barrier). A consequence of the openness of the Internet is the fact that anyone can put unchecked information on the Internet. This is a dramatic difference compared with other ways and means by which laypersons can obtain relevant health information. Doctors who give advice have to pass examinations before they are allowed to practice. Articles in journals only appear after they have been reviewed by peers etc. There are no comparable mechanisms on the Internet to safeguard the quality of contributions. In the meantime, numerous case studies and a few systematic studies have been undertaken regarding the expert quality of medical information on offer. Deficits in the quality of information have been revealed (e.g. Li, Irvin, Guzman, & Bombardier, 1998; Eysenbach, Powell, Kuss, & Eun-Ryoung Sa, 2002; Suarez- Almazor, Kendall, & Dorgan, 2001). As a reaction to these findings attempts at some kind of certification have been made (HON, 1997). However, there are different competing approaches, and most of the available information is uncertified. It is therefore up to the layperson to decide whether helshe can trust the information on offer (Wittwer, Bromme & Jucks, 2004; Eysenbach & Diepgen, 1999). The vast amount of information available on the Internet not only makes it difficult to judge its quality but also makes it hard to find relevant information. One can get lost quite easily while searching (orientation barrier). Quite a bit of practice is needed to be able to use the search engines effectively to find specific information. The openness and sheer size of the Internet are also sources of barriers when searching for specific information. A third barrier concerns the question of comprehensibility of the information. In contrast to the barriers just mentioned, the comprehensibility of specialist
92
BROMME,JUCKS & RUNDE
information for laypersons and the quality of expert-layperson advice have received little attention in Internet research. However, as has been illustrated in the introduction, the presentation of information on health matters available to laypersons on the Internet is adapted to laypersons' needs in very different ways. On the one hand, some information available in electronic versions of scientific journals is, strictly speaking, only intended for scientific discourse between experts. Nevertheless it can be assumed that laypersons, too, access these periodicals via the Internet, by searching through www.thelancet.com, for example. On the other hand there is information explicitly intended for laypersons, though not tailored to the requirements of specific individuals. For example, in sections like "Frequently asked Questions" (FAQs), probable questions are answered. Furthermore, there are also personalized expert answers to specific user inquiries, which are either later put on the Internet for general use, or remain a kind of private correspondence. Such advice on health matters is mainly given by writing answers to written inquiries. Thus communication is text-based as in email-conversations and follows an asynchronous pattern with delayed or without feedback. These examples have at least one feature in common: it is almost impossible for experts providing information to explore recipients' prior knowledge at great length. Instead, experts have to rely on their assumptions about laypersons' knowledge and preconceptions in the domain they are going to talk or write about, and to formulate their explanations accordingly. In our studies it is precisely the consequences of these features that we are dealing with: How does one achieve reciprocal understanding between experts and laypersons in these circumstances? This question is all the more acute since communication between experts and laypersons is difficult anyway, even outside computer-mediated contexts. 2.2. Expert's specialist knowledge: Sources of advice -and mutual misunderstanding. From the viewpoint of laypersons who have to rely on expert advice or explanations, experts often fail to express themselves in a comprehensible way. If a physician overloads a patient with specialist terminology, or if a computer expert explains technical details that a person seeking advice cannot possibly understand, a problem of deficient recipient design is obviously present. Problems of understanding between experts and laypersons can have a wide range of causes. Sometimes experts make no effort to communicate understandably, either because they want to enhance their image by using complicated technical terminology or because this kind of language enables them to avoid dealing directly with emotionally stressful topics. We are not going to tackle such motivational barriers here, but will concentrate instead on difficulties of mutual understanding (meaning barrier). As a source of the meaning barrier we will in the following look more closely at the systematic differences in knowledge between experts and laypersons. These differences are the reason for the communication naturally (laypersons seek experts' advice because of their specialist knowledge). However, at the same time they can be a source of difficulties in mutual understanding (see also Rummel & Spada; Strube et al., and
NET-BASEDEXPERT-LAYPERSON-COMMUNICATION
93
Fischer & Ostwald, in this volume, for similar approaches to the difficulties of mutual understanding among experts of different domains). The process of training for a qualified profession over many years should be understood psychologically as a growing into a community of experts (enculturation). The training and accumulation of experience not only include the acquisition of knowledge, but also methods of thinking and problem-solving, forms of perception, modes of communication, etc. These are factors which one is not always entirely aware of. They form part of a person's unexamined perception of the world. Training and experience mould people's perception of things which are important in their work so that they view them in a certain way. Consequently it is difficult to imagine how people who do not possess the expert perspective of a doctor, architect or computer expert would see these things (an x-ray, the user interface of a computer or the design of a building). In these cases experts might become the victims of their own professional perspective. This was characterized very well as 'the curse of expertise' by Hinds (1999, see also Hinds, Patterson & Pfeffer, 2001). In other words, experts do not just know more than laypersons, they also have a different way of structuring their domain-specific knowledge. It is not only experts who are immersed in their knowledge and might therefore have difficulties in imagining how others perceive the issues they know about. Laypersons also have ideas about the nature of the issues that are the subject of their questions to experts. They could also be immersed in such ideas. Good examples of this are the so-called subjective theories about diseases. Some of them are very comprehensive systems of experience, explanations and ideas about symptoms, causes of diseases and the courses they take. People have such subjective theories nearly in all areas of life in which they have to make decisions and where they need expert advice. In this respect the interaction between experts and laypersons is similar to the interaction between teachers and pupils. Pupils have elaborate preconceptions for many subjects based on everyday experience and prior experiences with other teachers and with instruction on similar topics. Research on teaching has shown the importance of such preconceptions for mutual understanding in classrooms and for successful learning (Bruer, 1994). It is helpful for teachers to take such preconceptions into account when explaining new subject matter. Similarly, experts' explanations will be understood by laypersons against the background of their subjective theories about the topic in question. If experts do not bear this in mind when formulating their explanations for laypersons misunderstandings are inevitable. As well as this similarity between expert-layperson and teacher-student interaction, there are also important differences. In most professions experts have not received any specific training in explaining their expert knowledge in a comprehensible way. They are dealing with patients, clients and customers who are much more heterogeneous with regard to their prior knowledge and learning capability than pupils and students, who have after all been graded according to their age, pre-knowledge etc. However, the most important difference is the fact that there are no essentially pedagogical intentions on either side. Nevertheless, understandable explanations are necessary because laypersons need to make informed decisions, as has already been pointed out. Experts and laypersons form a
temporary problem-related 'community of interest' as described by Fischer and Ostwald in this volume. 3. KNOWLEDGE DIFFERENCES AND MUTUAL UNDERSTANDING IN TEXT-BASED ASYNCHRONOUS COMPUTER MEDIATED COMMUNICATION
3.1. Introducing Herbert H. Clark's communication model In the following we will refer to Herbert H. Clark's communication model (summarized 1992, 1996) because we assume that it is helpful to conceptualize the impact of knowledge (and knowledge differences among interlocutors) on mutual understanding. It has to be emphasized that this is an extension of Clark's work, which is primarily concerned with the interdependencies among utterances, not with the structures and processing of knowledge by those who make the utterances. Clark and his colleagues treat communication as joint activity of the speech partners (Clark, 1996; Clark & Wilkes-Gibbs, 1986). To ensure mutual comprehension, common ground has to be established between the speech partners. Knowledge presumed to be common ground does not have to be reiterated. At the beginning of an interaction between strangers common ground is limited and leaves little room for automatic understanding. To a certain degree common ground can be established explicitly, e.g. by means of an explanation related to a certain topic which can be accepted by the interlocutor, thus increasing the common ground with respect to this issue. But it is usually impossible to establish common ground from scratch. There is always a huge stock of knowledge that has to be taken for granted. Problems of mutual understanding occur when speakers' assumptions about this stock of common knowledge are misleading. When a speaker takes into account a partner's level of knowledge when planning and carrying out utterances, Clark and Murphy (1982) call this audience design while Sacks, Schegloff and Jefferson (1974) refer to recipient design. Audience design may fail because speakers have no or false assumptions about their partners' background knowledge or it may fail because speakers are not able to design their conversational contributions accordingly. How do speakers know how much their speech-partner already knows about the subject matter, i.e. how much common ground can be taken for granted? Clark and Murphy (1982) assume that speakers rely on three heuristics in order to estimate what a partner already knows and also in order to process utterances by putting them into an assumed context of meaning. The first heuristics is based on the perceived group membership of the communication partner (community membership heuristics), criteria being age, sex, nationality and, very important in our context, the perceived status of expertise (Isaacs & Clark, 1987; Clark & Marshall, 1981). If utterances can be related to items that are within both speech partners' field of vision, such as objects or illustrations, speech partners can make use of the physical copresence heuristics. Note that the term 'copresence' refers to shared objects and not to the visibility of the communication partner. Objects, that are visible to one partner can be assumed to be also visible to the other, provided the appropriate physical conditions are met, and be directly referred to in the interaction. These two
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
95
heuristics are particularly relevant for our studies described below, but the third heuristics should be mentioned too: the linguistic copresence heuristics, i.e. the fact that something which has been mentioned once can subsequently be presumed to be known if there is no indication from the speech partner that he did not understand. Like the other heuristics, this assumption also allows experts to deal economically with their communication contributions (in accordance with Grice's quantity maxim, 1989), since content which can be presumed to be known does not have be explained again. As far as interactions are concerned, where the communication partners do not have large and systematic differences in knowledge and where there are ample opportunities for feedback to signal understanding or misunderstanding (e.g. by facial expression and gestures) these heuristics are some of many tools for a successful recipient design of utterances. However, when asynchronous and textbased communication is involved, it is more difficult to establish common ground (Clark & Brennan, 1991, Brennan, 1998). Here, the heuristics are more important than in face-to-face interaction because there is no immediate feedback and because the paralinguistic information of face-to-face communication (facial expression, gestures, tone of voice) is missing in text-based communication. (In some net communities, e.g. chat forums, there is some kind of compensation via text layout, deviations from the usual syntax and spelling, and the use of a special dictionary, i.e. a group-specific slang. However, the use of information like this presupposes a degree of shared familiarity with the conventions that is not present in the scenarios between experts and laypersons. In the light of this it can be claimed that in our context paralinguistic information is, in fact, missing and the heuristics in question should be regarded as particularly important.) From what has been said so far, it can be assumed that difficulties of a certain kind are to be expected in written asynchronous communication between experts and laypersons: Hence, heuristics for the assessment of the common ground are particularly important for the design of the written explanations because the opportunities for subsequent repairs are limited (at least when compared with ftf). The application of these heuristics may, however, cause many errors because of the systematic differences in knowledge between experts and laypersons described above. It has to be emphasized that this is a hypothetical assumption that has to be elaborated further and then tested empirically. Clark's theoretical account is frequently used at present to explain the special features of net-based communication compared with ftf-interaction, although so far this extension of his approach has not been put to many empirical tests (Clark & Brennan, 1991; Brennan, 1998; Brennan & Ohaeri, 1999; Bromme & Jucks, 2001). Empirical examination of the viability of Clark's concept is very important, all the more so since Clark's basic theoretical assumption, that communication is a joint and cooperative activity, has been attacked recently. Net-based communication is not involved in this debate, which is taking place in the context of psycholinguistic basic research. The communication scenarios deployed in the key experiments are far less complex than the communication between experts and laypersons as dealt
with above. Nevertheless, the controversy is very interesting for the scenarios discussed here.
3.2. Some doubts about the cooperative nature of communication There is empirical evidence that assumptions about speech partner's prior knowledge are used in formulating utterances. In several studies Susan R. Fussell and Robert M. Krauss examined the effect of preconceptions about the interlocutor on the design of verbal explanations (Fussell & Krauss, 1991, 1992). The authors demonstrate that speakers' assumptions about knowledge distribution are relevant, i.e. speakers take into consideration the anticipated knowledge of the recipient when formulating their explanations, and in doing so employ audience design. Hence, audience design in written communication reflects a cooperative behavior of the author. However, Fussell and Krauss also show that a considerable portion of listener-orientation is derived from direct feedback in ftf-communication. This is also confirmed by Schober and Clark's findings (1989): if a speaker knows that there is no way of receiving any feedback he will employ audience design more frequently. On the other hand, Buhl (1996), Speck (1993) and Roljnagel(1995), only report minor use of audience design by speakers. Furthermore, there are experimental findings indicating that assumptions about speech partner's prior knowledge are not as significant in the planning and realization of communicational explanations as Clark and his colleagues assume (Nickerson, 1999; Horton & Keysar, 1996; Keysar, Ban, Balin, & Paek, 1998; Brown & Dell, 1987). Keysar (1994) has demonstrated experimentally that it is very difficult for speech partners to distinguish between their own knowledge of a particular subject and what the other speech partner already knows about the subject. Quite often there is privileged information that is known only to the speaker but not to the recipient. Utterances seem quite comprehensible to the speaker in the light of his privileged knowledge, but without this the recipient is unable to understand them as the speaker intended. Keysar has proposed the term illusory transparency of intention for this case. However, Keysar's experiments (1994; Horton & Keysar, 1996) are not undisputed. Gerrig, Brennan and Ohaeri (2000) argue that his communication scenarios were misleading for research participants (see also Polichak & Gerrig, 1998). This is not the right place or context to describe this controversy in detail or pass a final judgment. Despite the criticism of Keysar's experiments, the basic idea behind his investigations is very interesting in the present context. His assumption that participants have difficulties distinguishing between privileged and shared information, and that these difficulties have an impact on the design of utterances, may be especially true for experts. The above-mentioned 'curse of expertise', as Hinds (1999) has put it, describes precisely the difficulty that experts have when it comes to being aware of the privileged nature of their own specialist knowledge. In a series of studies we have established that there are - under certain conditions systematic biases in experts' estimations about laypersons knowledge (Bromme, Rambow, & Nuckles, 2001; Rambow, 2000; Jucks, 2001). In our context of textbased communication it is also of interest that similar misleading effects of
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
97
privileged knowledge have been reported in experiments on writing and revising of texts (Lumbelli, Paoletti, & Frausin, 1999; Jucks, 2001). It has already been emphasized that the above-mentioned studies on the issue of audience design did not treat computer-based communication. Therefore it needs to be empirically investigated, whether audience design and the use of the heuristics are also deployed in net-based communication among experts and laypersons, i.e. in a scenario with systematic knowledge differences between experts and laypersons. In this scenario both sides are aware of their roles (and knowledge status), but have no further possibilities to explore the relevant knowledge background of their interlocutor in detail. Though one could argue that writing is a individual activity (just like or even more than speaking) we think that the produced written artifacts are the result of a cooperative activity where the author considers hisfher addresses' needs. We have established a series of experiments that analyze the use of the above-mentioned heuristics in text-based computer-mediated-communication. The first of these experiments and two subsequent studies will be outlined in the following section. They focus on community membership heuristics and physical copresence heuristics. 4. EXPERTISE AND THE DESIGN OF WRITTEN MEDICAL EXPLANATIONS: STUDIES ON THE USE OF THE COMMUNITY MEMBERSHIP HEURISTIC AND PHYSICAL COPRESENCE HEURISTIC IN COMPUTER MEDIATED SETTINGS Our scenario for the experiment described below: An expert on health topics get a health related e-mail inquiry from a (fictitious) recipient. This inquiry demands a complex expert answer. The expert formulates hisfher answer computer-mediated and presses the 'send it back'-button in order to transfer it to the recipient. The question of barriers and biases regarding this scenario is in the following portrayed from both sides of the coin. First we describe a study on experts' audience design in web-based communication.' Afterwards we analyze whether experts' explanations can be understood by prospective laypersons. In this issue we report on two studies with layperson samples. One focuses on the subjective assessment of text-comprehensibility. The other uses a classical text-comprehension-method to identify differences between our experimental conditions.
4.1. Our research questions In order to examine the community membership heuristics we have manipulated inquirer's expertise status. In the first instance the inquirer is a layperson, in the second a co-expert from a related domain (here: medicine). We assume that explanations to the medical co-experts will be shorter, at the same time, however, they should contain more technical terms and generally take more for granted than the explanations to laypersons. We assume that the experts under investigation will expect more shared knowledge on the part of their co-experts, i.e. the medical people, than on the part of the laypersons (CM hypothesis). Though such differences
are to be expected - consistent with Clark's theoretical approach - it is an open question which needs to be answered empirically. What changes does the expert make to his explanations in order to adapt them to the recipient's knowledge status? The physical copresence heuristics refers to assumptions about the mutuality of knowledge of the environment within the field of vision of all speech partners. In Clark's case studies (1992) the environment consists of real objects of the physical surroundings. With net-based communication it is external representations, i.e. illustrations or graphics which are available in shared workspaces to the communication partners on their respective monitors. With respect to the (erroneous) application of the physical copresence heuristics we expect experts to succumb to an 'illusion of evidence'. As mentioned above, Keysar (1994) termed the effect which privileged information has on the production of utterances 'illusory transparency of intention'. By analogy we use this term for the erroneous assumption that everything which is within everyone's field of vision can be regarded as shared knowledge. We assume that experts' explanations are less recipient-oriented when both the expert and the layperson have a technical illustration to hand, than when there is no illustration copresent (PC hypothesis). Many patients have experienced in their daily lives medical experts using x-rays or ultrasound pictures to back up their explanations, yet leaving out of account the importance of prior knowledge for the layperson to be able to interpret the illustrations. Theoretically this effect is well described by the erroneous application of the physical copresence heuristics, but there has been little research on it so far. This experiment investigating illusion of evidence is intended to contribute to the debate about the significance of external representations in computer-mediated communication. The easy availability and manipulability of external representations are the augmenting features of computer environments (see Dillenbourg; Kirschner & Kreijns; Fischer & Ostwald, this volume). Alpay, Giboin, & Dieng (1998), for example, report on the supportive effects of shared external representations on the communication between speech partners of different expertise. There are various examples in cooperative computer-mediated teaching-learning scenarios demonstrating the use of external representations to support communication among pupils and between teachers and pupils (Suthers & Hundhausen, 2002). However, results have not always been positive for communication and learning success (van Bruggen, Boshuizen & Kirschner, 2003). Problems arise, for example, when the external representations are not readily accessible to the user and when he doesn't receive any assistance with the deciphering (Lowe, 1989, 1996). Such problems are expected to be even greater outside instructional contexts. There, experts have not been trained to discern the difficulty of 'reading' diagrams. Laypersons on the other hand do have access to illustrations which cannot be 'read' without help from experts. The effect of both hypotheses on audience design has been extensively investigated experimentally in face-to-face interaction (see Clark, 1992, 1996). However, the subject matter which participants were asked to use in their interaction was always simple. The application of PC heuristics, for example, involves referential descriptions of simple objects which are in the field of vision of the communication partners. Whether these heuristics are also relevant in net-based,
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
99
asynchronous communication with more complex subject matter has not yet been clarified empirically. Since, as a consequence, the complexity of scenarios to be investigated increases considerably, and experimental control options of effecting factors are reduced, we have limited ourselves to only one 'turn' in the experiment described below, i.e. the audience design of the reply to one inquiry without further questions. 4.2. Study I: Analyzing experts 'audience design 4.2.1. Design and Materials of Study I Students of pharmacy in the final phase of studies at different universities worked with the inquiry from a fictitious layperson on the topic of laxatives, and with the inquiry from a fictitious medical expert on the same topic. The topic of laxatives and especially their misuse is frequently the subject matter of communication between pharmacists and their patients and can therefore be seen as relevant for expertlayperson communication in health related areas. The inquiry from the fictitious medical expert was identical with regard to the content, but more specialized terminology was used than in the inquiry from the layperson, so that the text of the inquiry matched the information about his expertise status. After receiving the inquiry, participants were asked to explain to the respective addressees the link between the use of laxatives and potassium deficiency and its effects. Experimental design. The application of both the community membership and the physical copresence heuristics was examined in a 2 x 3 factorial design, whereby the factor 'addressee' was carried out in a within-subject-design (2 levels) and the factor external representation in a between-subject-design (3 levels). The specialist illustration - used under the appropriate experimental conditions illustrated the relationship between the use of laxatives and potassium deficiency. It was necessary for it to be clear enough for the expert to identify the relevant information. On the other hand we had to ensure that laypersons without prior knowledge did not profit from it unduly. This was tested in a preliminary experiment. Experts for their part assumed the illustration to be helpful for explaining the subject matter to a layperson. Figure 1 shows the specialist illustration. As a further experimental condition experts were given a list of key words as an external representation. The list contained the same terms - in the same sequence starting with Obstipation - as those depicted in the illustration. This enabled us to produce a control condition without a graphical representation, ensuring at the same time that the specialist contents relevant for the explanation task were salient in a similar way to the other experimental conditions; an experimental condition without any form of external representation would have been unsuitable. The empirical comparison between the explanations produced on the basis of the illustration and those produced using the list of key words should also give some indication of the impact of the representational formats (graphical versus purely text-based) on the formulation of expert explanations.
Figure I : Specialist illustration
4.2.2. Dependent Variables and Hypotheses The explanations given by the experts were analyzed by means of a content analysis. Linguistic structural characteristics of the texts served to ascertain the audience design of the experts. In doing so, we based our work on variables that had been used in the psycholinguistic studies mentioned at the beginning. We also included variables on the basis of pedagogical-psychological findings. It can be assumed that these structural characteristics are relevant for recipients' text understanding (e.g. using examples, cf. Reimann, 1997). As in other studies of our own concerning expert communication, we distinguish between content-unspecific and contentspecific indicators of audience design (cf. Jucks 2001). Content-unspecific indicators. The number of times the expert addressed his explanation directly to the (fictitious) recipient was counted. We refer to this use pronouns as "use of direct address". Furthermore, the number of words used by experts in the explanation task should also function as an indicator for audience design. With regard to these two content-unspecific indicators we expect to see a more detailed explanation given to laypersons compared to the explanation given to medical experts (application of the community membership heuristics). When experts operating under the condition 'illustration copresent' make less use of the content-unspecific indicators than under the condition 'illustration not copresent', this is seen as confirmation for the use of (inappropriately applied according to the illusion of evidence hypothesis) physical copresence heuristics. Explicit reference to the copresent illustration was analyzed, too. This involves expressions like 'in the upper left corner of the picture'. This indicator will give us
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
101
information about the utilization of the illustration, but a comparison with other conditions is not possible since no such references can occur there. Content-spec@ indicators. The use of examples is analyzed as an indicator for the writer regarding some subject matter as requiring more detailed explanation for some recipients. We classify examples as either requiring a high or a low level of prior knowledge. An example is termed low level if it can be understood without prior knowledge. In this case, for example, the term 'fibre' would be explained by referring to certain food items such as fruit, vegetables, whole-meal bread, etc. An example is considered to be high level if it requires knowledge about medicalpharmaceutical matters in order to understand it. If, for example, the liver is described as a detoxification organ, then the expert presupposes knowledge about the functions of organs. There is evidence of audience design if the experts use more examples generally, above all examples which require low prior knowledge, in their explanations to laypersons than in explanations to medical experts. There would be an indication for the presence of illusion of evidence if experts used less examples under the condition 'specialist illustration copresent' compared with the condition 'specialist illustration not copresent' because - as explained above - the illustration, on account of its specialized character, does not contribute to the understanding of the explanation per se and thus does not compensate for the lack of missing examples. Selection of contents. The explanation task was the same for both addressees. Nevertheless, experts had some scope with regard to the choice of contents for the reply to the inquiry. In total, 28 themes were identified which could be divided into three categories: 13 themes were classified as behavioral tips, 9 themes as closely related to the explanation task and the remaining 6 themes as broadly related to the explanation task. Thus, information about a chemical laxative being better than a vegetable laxative is considered a tip, while information about potassium being important for preventing obstipation is classified as an argument closely related to the explanation task. On the other hand, information that laxatives help food to pass through the digestive system quickly, while not allowing the body to absorb potassium, is considered to be only broadly related to the explanation task. In the case of the CM-hypothesis, fewer broadly related themes and more behavioral tips are expected to be used in the advice to laypersons than to medical experts. As for the themes closely related to the explanation task, which represent the core of the task and as such are of vital importance, we expect no difference in use by the experts resulting from their view of the competence of the addressees. If experts who are working with a copresent illustration present fewer arguments than experts working without a copresent illustration, this would indicate an inappropriate use of the physical copresence heuristics, i.e. it would be an indication for an illusion of evidence.
Use of specialist terms. A list was made of the 106 explanations produced by the 53 experts containing all the specialist terms used. There were 43 terms in all, 21 of which were used in the explanations to both laypersons and medical experts, with an average occurrence rate of at least 10 %. We analyzed how often the terms were used in the explanations to both groups of addressees. For further differentiation, these 21 terms were distinguished with respect to specialist terms already contained in the material and specialist terms which were not contained in the material. 12 terms were identified as already present in the material. These appeared in the external representations and/or in the inquiries. Experts introduced the remaining 12 terms of their own accord. Pharmaceutical specialist terms represent a greater obstacle for laypersons than for medical experts. Hence, according to the CMhypothesis there should be less specialist terms used in explanations to laypersons than to medical experts. As far as the physical copresence heuristics is concerned we should expect to see more specialist terms being used under the condition 'illustration is copresent' than under the condition 'illustration not copresent'. In fact, this applies to both the specialist terms not contained in the material and those that are. In this case, the illusion of evidence occurs if experts regard the specialist terms already contained in the material as part of common ground, and if they presuppose that any additional specialist terms are known to the recipient.
4.2.3. Results Testing of the CM-hypothesis is carried out using the within subjects design. The testing of the PC-hypothesis is achieved by comparing the two conditions in which the illustration is either copresent or not copresent. The influence of the format of the external representation is assessed by a comparison of the conditions 'illustration not copresent' and 'list of key words'. The analysis of the main effects regarding the factor 'external representation' and appropriate post-hoc tests reveals the two distinctions made with respect to the external representations mentioned above (PChypothesis - format of the external representation). In the following the results are described on a general level, leaving statistical aspects aside (see Jucks et al., 2003, for this information). With regard to the dependent variables, the results are presented separately for the contents-unspecific and contents-specific indicators of audience design. Contents-unspecijic indicators of audience design. Direct address was used more frequently in the explanations to laypersons than in the explanations to medical experts. Furthermore, a significant impact of the external representation emerged. The interaction effect addressee X external representation was also significant. Differences between addressees appeared smaller in the condition 'illustration copresent' than in the condition 'list of keywords'. Experts in the condition 'illustration copresent' used less direct address than experts in the condition 'specialist illustration not copresent'. Explanations to the laypersons were more detailed, with an average of about 180 words, than explanations to the medical experts (mean of about 100 words). There was no main effect with regard to the external representation. The interaction effect
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
103
addressee X external representation was significant. Experts in the condition 'illustration copresent' used more words in their explanations to laypersons than experts in the condition 'specialist illustration not copresent'. This difference did not occur in explanations to medical experts. Of the 19 experts who had a specialist illustration to hand while working on the explanation task, only four participants referred to it explicitly. One of them did so both in the explanation to the layperson and to the medical expert. The remaining three experts only used explicit references in their explanations to the layperson. Content-specific indicators of audience design. More low-level and more high-level examples were used when addressing laypersons than when addressing medical experts. Furthermore, under the condition 'illustration not copresent' significantly less low-level examples occurred than in the condition 'list of key words'. The interaction addressee X external representation was also significant regarding the use of high-level examples. Experts in the condition 'illustration not copresent' included fewer high-level examples in their explanations to laypersons than did experts in the condition 'list of keywords'. These differences did not occur in the explanations to medical experts. There were also no differences between the conditions 'illustration copresent' and 'illustration not copresent' with regard to the explanations to laypersons. Altogether, more low-level than high-level examples were used in explanations to laypersons, there was no such difference in explanations to medical experts. There were less specialist terms in explanations to laypersons than to medical experts. This applies to both the specialist terms already included in the materials and those not included. There were differences in the use of specialist terms depending on the format of the external representation. Post hoc analyses show that more specialist words already in the text material were used in connection with the copresent illustration than with the not copresent illustration. The interaction addressee X external representation was also only found to be significant with regard to the specialist terms already included in the materials. There are fewer differences in explanations to the addressees when the specialist illustration is copresent than in the other two conditions. In general, in explanations to both laypersons and medical experts, more specialist terms already included in the text material were used than specialist terms which were not contained in the text material. In a multivariate analysis of the variables 'themes closely related to the explanation task', 'themes broadly related to the explanation task' and 'behavioral tip themes' both main effects were significant. No interaction-effect occurred. With respect to the layperson addressees fewer of the six broadly-related arguments available and more of the behavior-related themes were used compared to the medical expert addressees. There were no differences for the closely-related themes. Post hoc analyses show that differences occurred regarding the factor external representations on account of significantly more behavior-related arguments and less closely related arguments being used under the condition 'list of keywords' than
under the condition 'illustration not copresent'. There were no differences between the conditions 'illustration copresent' and 'illustration not copresent'. 4.2.4. Discussion Was the community membership heuristics applied? The results show that participants adapted their explanations significantly, according to the knowledge they expected from a layperson or a medical expert. When giving an explanation to a layperson they used fewer specialist terms than when addressing a medical expert. Besides, in their replies to laypersons experts explained issues in more detail, and used more direct address and illustrative examples than in their explanations to medical experts. Moreover, the examples they used in their replies to laypersons were mainly of the low knowledge requirement type rather than the high knowledge type. As this was not observed with the medical experts, this can be interpreted as evidence for experts' adaptation to the recipient. Altogether explicit references to the copresent illustration did not occur very often, and where more frequent in explanations to laypersons. Concerning the selection of topics experts also showed adaptation to the (fictitious) recipient: Laypersons received more behavior-related information and less additional specialist information about bio-chemical processes than medical experts. As expected, no distinction was made between laypersons and medical experts with respect to the use of themes closely related to the task itself. These themes formed the core of the explanation task and left, therefore, little 'room for manoeuvre'. From these results it can be concluded that the community membership heuristics definitely played a role in the formulation of the replies. This is in accordance with Clark's theory, which frequently uses the expertise variable to explain the basic idea of audience design (e.g. Clark, 1992), even though the relevant experiment in this connection (Isaacs & Clark, 1987) used a trivial expertise domain (knowledge of New York landmarks) and the simple task of making references to them (on postcards). In this respect our findings add to the results gained so far. At the same time, our findings show that when experts reply to an inquiry the community membership heuristics is used from the outset if no further feedback is expected. This also needs to be stressed especially considering the background of criticism of Clark's concept mentioned at the beginning (Keysar, Ban, Balin, & Paek, 1998).
Was the physical copresence heuristics applied and did it lead to the illusion of evidence? For a number of indicators there were no differences between the explanations formulated under either the condition 'illustration copresent' or the condition 'illustration not copresent: The explanations did not differ, either in length or in the choice of content, i.e. reference to themes. The use of examples (assuming high or low level of prior knowledge) did not vary, either, independent of whether the illustration available was copresent or not copresent. However, the (fictitious) reader was addressed less often in a direct manner with a copresent illustration than with a not copresent illustration. This applied particularly when addressing the layperson recipients, as shown by the interaction effect.
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
105
As far as the use of specialist terms was concerned, the findings were heterogeneous. With respect to the terms not contained in the text material no differences occurred between the two conditions. To this extent, the assumption of an illusion of evidence with regard to the experts could not be confirmed. Furthermore, when the illustration was introduced as copresent, terms already contained in the text material were used more often than when the illustration was not copresent. As the interaction effect shows, the distinction made between medical experts and laypersons as addressees when applying the community membership heuristics decreased when the illustration was copresent. This is an indication for the application of the physical copresence heuristics. The specialist terms contained in the illustration are, in our view, presumed by the experts to be common ground and are thus used more often. The findings seem, therefore, to point in the direction of an illusion of evidence, as the frequent use of these specialist terms is a characteristic mainly attributed to experts rather than laypersons. Furthermore, the data do not provide any clues as to whether the experts used the copresence of the specialist illustration in their explanations to laypersons in a didactic sense. Thus, despite the more extensive use of specialist terms, no more examples were used in the copresence condition which could have served to illustrate them. The fact that there were so few references to the illustration may also serve as an indirect indication that participants presumed that the recipient would also be able to find the information contained in the explanation in the illustration. The low number of explicit references thus provides an indication of an illusion of evidence.
What effects did the external representation have on the experts' explanations? In contrast to the inconsistent findings with respect to the application of the physical copresence heuristics, the comparison of the two conditions 'illustration not copresent' and 'list of key words' provided considerably clearer results - much to our surprise. This comparison permits us to make a statement about the impact of the representation format on explanation behaviors. There is a considerable impact of the external representation format on the content referred to. Experts who had a specialist illustration available when replying to the inquiries (even though the illustration was not available to the recipient!), used significantly fewer examples than experts who had a list of key words to work with. In addition, more 'themes broadly related' to the explanation task were addressed when experts had an illustration at hand rather than a list of key words. This suggests that experts in the condition 'illustration not copresent' quite clearly turned to the illustration when selecting content, 'ticking off the items of information one by one. In contrast, experts working with a list of key words included more behavior-related information. Hence, the representation format had a massive effect on the explanations, even when the explanation task remained the same. Nevertheless, it cannot be said that the illustration itself was more demanding, thus requiring more effort on part of the experts (in terms of cognitive load) than the purely verbal presentation of the key words: comparing the two conditions 'illustration not copresent' and 'list of key words' there were no differences between the use of broadly related themes and use of specialist terms (containedhot contained in the
text material). The contents-unspecific variables 'use of direct address' and 'length of explanations', too, indicated no differences between the various external representation formats.
4.3. The other side of the coin: Studies with layperson samples Clark's empirical studies (more often case studies of transcriptions) are predominantly conversation-analytically oriented. For example, Clark analyzes how speech partners communicate about the referential meaning of certain concepts and clear up ambiguities as they go along. The level of analysis are individual speech acts and the links between them. The variables used in the experiments explore for example reference to proper nouns, deictic formulations, coordination of turn-taking, number of supplementary inquiries, non-verbal fillers (er, mmm, etc.), selfcorrections, etc. Such analyses can only be undertaken empirically if the content of interactions is limited to relatively simple issues. In Isaacs and Clark's study mentioned earlier concerning the effect of expertise differences on the establishment of common ground, only one simple referential task was used. Basically, these investigation paradigms are not about understanding (i.e. individual learning gains on the part of the participants) but avoiding misunderstandings which might impede the natural flow of communication. However, in the context of net-based advice we believe that a more strongly cognitive-oriented understanding of recipient orientation would be more useful. We therefore propose using comprehensibility of more complex written expert utterances as an indicator for recipient-orientation. This entails defining variables of text comprehensibility and testing these experimentally. Additionally, features will be included in the measures testing recipientorientation which up to now have not been dealt with in the research on text comprehensibility, but which are specific for specialist texts, such as labeling and paraphrasing of specialist terms. Hence, another way to address the question of differences in experts' explanations is to analyze laypersons' point of view regarding this issue. Therefore we tried to work out whether experts' explanations for laypersons vary in comprehensibility and lead to different degrees of comprehension according to the underlying experimental conditions under which they were formulated. We restrict our new investigations to the explanations provided for laypersons and do not therefore supply evidence for the application of community membership heuristics. The reason for this is that the illusion of evidence was only hypothesized to occur in the explanations for laypersons. The questions addressed in the following studies are: are the explanations that were produced with a copresent illustration found to be less comprehensible, and do they result in less comprehension of the subject matter addressed, than explanations that were produced under the other conditions? These questions lead to two sub-questions that have to be answered in separate experimental designs: the question of text-comprehensibility addresses the subjective assessment from laypersons' points of view, and the question of factual comprehension requires objective measurement (Jucks, 2001).
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
4.3.1. Study 2a: Subjective rating of text-comprehensibility and experts' audience design Study 2a uses two groups of dependent variables to ascertain the extent of textcomprehensibility. We look at four dimensions of text-comprehensibility (according to Langer, Schulz von Thun & Tausch, 1973, 1993; Groeben, 1982): (1) simplicity, (2) structure-organization, (3) brevity-shortness, and (4) interest-liveliness. We also look at the emotional content of the texts, e.g. laypersons' feelings regarding the text. This factor, too, plays an important part in processing informational texts. Bartels (1992) was able to demonstrate that texts which were formulated taking the recipient into account were rated by participants as more comprehensible, and also increased performance in comprehension and memorizing. Schiirer-Necker (1991) reports similar results: Texts of 'objectively' equal difficulty appear differentially comprehensible depending on their emotional content. The more emotional version is more comprehensible and can be remembered better while the neutral version was thought to be less comprehensible, more abstract and more confusing. If the informational text alludes to possible problems and emotions of the reader, he will sense that efforts are being made on his behalf, and form the impression that he understands the text better. Adapting to the reader, i.e. adopting the reader's perspective is what we are testing in the studies reported on here. We ask the participants of our study for example in how fare the author has put effort in making himself understandable to a lay audience. Since 'emotional' aspects of texts affect the reader's achievement, we measure the perceived audience design. '
Materials and Procedure. After correcting spelling and grammar mistakes each explanation was printed using the same font and size. One of the 53 explanations was given to each layperson-participant. If the explanation was written in the condition 'illustration copresent' the illustration was printed on the left hand side of the questionnaire booklet (sized DIN A4). After having read the text laypersons assessed text-comprehensibility and the amount of experts' audience design that they considered to be revealed by the text. At the end of the questionnaire laypersons provided information about their personal data. The assessment of text-comprehensibility was carried out using a 24-item questionnaire covering the four dimensions of text-comprehensibility. This questionnaire has been deployed in other studies (cf. Clark, Weinberger, Jucks, Spitulnik & Wallace, 2003; Jucks, 2001). The amount of audience design was rated using 23 items that addressed (1) the perceived audience design, i.e. the extent to which laypersons had the impression that experts wanted to explain the issue and tried to take into account a layperson's perspective (10 items), (2) subjective assessment of ones own comprehension (5 items), (3) the assessment of the experts' specialized knowledge (4 items), and (4) the assessment of the experts' commitment to writing on this specific issue (3 items). One additional item asked how much the layperson enjoyed reading the text. Each item was rated on a five-point scale, a small number indicating a negative assessment of text comprehensibility, i.e. perceived poor audience design.
Participants. 106 university students assessed the comprehensibility of the 53 explanations for laypersons given by our 53 expert-participants. Hence, each text was assessed twice. Data of 3 persons had to be excluded from further analyses because of many missing values. 80 % of our participants studied psychology (diploma), 82 % being female. Participants' age was between 19 and 45 years (M = 22.87, SD = 3.80). With regard to the three different conditions under which explanations had been produced, the layperson sample showed no significant differences regarding age, gender, pre-knowledge about laxatives, word-processing and Internet competencies. Table 1 provides the descriptive data for the subjective rating of textcomprehensibility and experts' audience design. Table I : Means and standard deviation of text-comprehensibility and audience design ratings (a small mean indicates a small amount of text-comprehensibility or rather perceived audience design (jive-point scale)
I
Illustration copresent
I
I Text comvrehensibilitv
I
interest Audience Design audiencedesign own - comprehension - expert's knowledge commitment enjoy of reading -
1
2.92 3.22 3.96
----
-
I
I
M
I
S
D
List of key words
Illustration not copresent I
M
I
S
D
I
M
I
S
D
1 0.90 0.78
3.69 4.08
0.90 0.74
3.39 3.92
0.89 0.81
0.63 0.85 1.20
3.87 3.12 2.76
0.67 0.83 1.18
3.89 3.17 3.03
0.63 0.86 1.19
Results. A multivariate analysis showed significant effects for the four dimensions of text comprehensibility with regard to the condition under which the explanations were produced, F(4,97) = 3.49, p < .05. The univariate analyses yielded a significant effect only for the dimension simplicity, F(2,99) = 3.80, p < .05; all other F(2, 99) < 1.27, ns. Post hoc analyses showed that the effect occurs because the explanations that were produced under the condition 'illustration copresent' had been rated as less simple than the explanations under the condition 'illustration not copresent'. Figure 2 illustrates the means for all four dimensions of textcomprehensibility.
NET-BASED EXPERT-LAYPERSON-COMMUNICATION co-present illustration H not co-present illustration list of key words
3
2
low
,
Figure 2: Rating of text-comprehensibility regarding the four dimensions
Rating of the perceived audience design was subjected to another multivariate analysis. Five dependent variables were used: the four scales on audience design and the additional item (enjoyment of reading). The multivariate analysis showed significant differences with regard to the condition under which the texts were produced, F(5,96) = 2.40, p < .05. The univariate analyses yielded a tendency only for the sub-scale perceived audience design, F(2,99) = 2.44, p < . l o ; all other F(2, 99) < 0.47, ns. Figure 3 illustrates the means regarding all five dependent variables. Post hoc analyses showed that the effect occurs because the explanations that were produced under the condition 'illustration copresent' were perceived as less audience-designed than the explanations under the condition 'illustration not copresent', T(66) = -2,16, p < .05. Hence, both groups of indicators reveal significant differences between the two conditions that had access to the scientific illustration. Nevertheless, within both groups of indicators most of the variables did not show differences between conditions. Note that using a between-subjects-design that asks independent participants to assess the explanations given by the experts, it is much harder to show differences between conditions while there is a lot of variance within cells. The results can be summarized as follows: Explanations that had been produced under the condition 'illustration copresent' were perceived as less simple and less audience designed than explanations that had been produced under the condition 'illustration not copresent'.
110 high
BROMME, JUCKS & RUNDE
5
co-present illustration not co-present illustration
0list of key words
low
1
Figure 3: Rating of the perceived audience design using four scales and the single item enjoy
4.3.2. Study 26: Measuring text comprehension using Cloze procedure In study 2b one of the established techniques for measuring comprehension is used, i.e. the Cloze Procedure (Taylor, 1953): Following certain rules, words are removed from the text, which participants are asked to replace. According to Taylor's basic assumption (1957), the more comprehensible a text, the better it can be understood, even though some words are missing. This method provides a fairly objective approach, as the value of any (missing) word can be calculated for each reader. In a text containing many words and expressions which are familiar to the reader, it will be easier for himlher to find the right words. On the other hand, if the reader is unfamiliar with the language used in the text and knows very little about the subject, it will be difficult for himlher to find the missing words, which shows that the text contains a lot of new information for the reader. Bormuth (1968) showed correlations of Cloze test values with other methods measuring text difficulty ranging from r = .90 to r = .96. These results prove that Cloze tests are valid indicators for the comprehensibility of text materials under consideration. Taylor (1953) originally suggested leaving out every nth word as a standard technique from which the standard-standard technique of leaving out every fifth word resulted This distance between gaps proved to be particularly suitable, as 20 % of missing words do not make a text incomprehensible, yet enough of the information is missing to be
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
111
able to test comprehension. However, the distance between two gaps should not be smaller than four words so not to impair the surrounding context. There are several options for marking Cloze tests. The most common procedure is only to give a score to the identical word, the procedure thus being both objective and at the same time very economical. In his analysis of several different options Bormuth (1968) comes to the conclusion that this is the most valid way of evaluating comprehension. Taking alternative answers into account would only lead to marginally higher correlations with the test values. These findings were confirmed by Brown (1980). Taking alternative answers into account, although considerably more costly, does result in higher Cloze values (Green, 1965). In most of these studies the words filled in are analyzed with regard to their semantic similarity to the original word and whether they have been used correctly with respect to their syntax (Kobayashi, 2002). However, with this procedure it is difficult to define what can be accepted as a synonym. One option would be to have experts judge similarity of meaning (Kobayashi, 2002). Piper and McEacern (1988) used a dictionary of synonyms. In our study we report both, one narrow calculation that only counts the identical word as correct and one wide calculation that takes alternative answers into account. Materials and Procedure. Each fifth word in each explanation (revised in the way mentioned above, e.g. with the spelling mistakes corrected) was deleted and replaced by a line. Laypersons received a questionnaire that started with questions about their computer knowledge and interest in the subject matter (laxatives). After that they were given one of the 53 explanations. Again, if the explanation had been written in the condition 'illustration copresent' the illustration was printed on the left hand side of the questionnaire booklet. The participants had been instructed to read each single sentence first and then fill in the word that had most probably been deleted. At the end of the questionnaire laypersons provided personal data. Participants. 159 university students were asked to fill in the gaps in the 53 explanations for laypersons given by our expert-participants. Hence, each explanation was presented three times. The data of 5 persons had to be excluded from further analyses because there were too many missing values. None of the participants of study 2b has participated in study 2a. 82 % of our participants studied psychology (diploma), 87 % being female. Participants were between 19 and 44 years old (M = 22.44, SD = 3.37). With regard to the three different conditions under which the explanations had been produced, the layperson sample showed no significant differences regarding age, gender, self-assessed pre-knowledge about laxatives and medicine in general, and interest in as well as ascribed relevance to the topic. Results. To analyze whether the inserted words match the original word used by the expert we decided on two different indices. The narrow method was as follows: Each word that a participant put in was counted as correctly inserted if it was
identical with the word that was deleted. This method also included variations in numbers (singular vs. plural forms of words) and grammatical differences from the original word (e.g. there are three words for "the" in German language "die", "der" and "das"), mostly occurring because of mistakes in other gaps. InterraterCorrelation was r. = .99. The wide method additionally counted input as correctly inserted if the meaning of the respective sentence remained comparable to the original sentence. Therefore, for example synonyms (e.g. enough / sufficient) and words that have a comparable stem (e.g. in German Ernlhrung / Nahrung which means nutrition / food) were counted as correctly inserted. To shed light on interrater agreement each word that was counted as correctly inserted was noted and a dictionary of synonyms (Duden 8, 1997) was consulted. Interrater-Correlation was r = .98. The percentage of correctly inserted words was calculated by dividing the amount of (narrowly/widely) correct insertions by the total amount of gaps. Table 2 provides the corresponding descriptions. Table 2: Means and standard deviation of correctlyjilled in gaps
Illustration copresent
Illustration not copresent
List of key words
Two univariate analyses were conducted that yielded marginally significant differences between the three experimental conditions under which the explanations had been written, for the narrow method, F(2, 151) = 2.75, p = 0.07; for the wide method, F(2, 151) = 2.42, p = 0.09. Post hoc analyses showed that both effects relate to the difference between both illustration-conditions. A smaller percentage of gaps had been correctly filled under the condition 'copresent illustration' than under the condition 'illustration not copresent', T(1,102) = 2.11, p < .05 for the narrow score and T(l, 102) = 1.9 1, p < .05 for the wide score. Figure 4 shows the results.
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
70
,
co-present illustration not co-present illustration 0 list of key words
Figure 4: Percent of correctly inserted gaps using two different methods
As the figure clearly shows, the wide scoring resulted in approximately 8% more correctly filled gaps. Both scores are highly correlated, r = .94, p < .001.
4.3.3. Discussion The subjective method that was used to identify text-comprehensibility and perceived audience design and the objective method that was used to assess text comprehension showed an almost identical picture. Both methods identified differences between the two sets of explanations that had either been written under the 'illustration copresent' condition or the 'illustration not copresent' condition. Whenever differences occurred this was in benefit for the explanations that had been written in the 'illustration not copresent' condition. Hence, the results provide evidence that experts succumb an illusion of evidence: Being aware of the copresence of the scientific illustration experts produced texts that laypersons assessed to be less simple and clear and less audience designed and that furthermore didn't lead to as much comprehension than explanations did that had been produced under the 'illustration not copresent' condition. Nevertheless it has to be stated that these differences only occurred with respect to some of the aspects in question. Only one of the four dimensions of textcomprehensibility and one of the five indicators for perceived audience design revealed differences. The observed differences in the assessment of simplicity can be related to the use of specialist terms in experts' explanations. The six items that form the simplicity scale explicitly address the issue of scientific jargon. The correlation between the means of the simplicity scale and the sum of scientific terms is r = .43,p < .001. Regarding the different indices used to assess Cloze values it can be concluded that the more costly analysis of allowing alternative answers - though it resulted in a higher amount of filled in gaps - didn't provide further insights. Using the narrow method identified the differences in text comprehension just as much.
5. SYNTHESIS Our research studies simulated a text-based communication situation between experts and laypersons. External representations (specialist illustration, list of key words) have proved to play an important role in determining type and quality of experts' explanations. It can be concluded that external representations take center stage, so to speak, because they are - in contrast to the addressees - quite literally present on the computer screen. A content analysis of the explanations provided evidence that experts produced their replies by closely following the external representations. This is remarkable, since the content of the reply should be independent of the visualization which - at most - should influence only the form and the didactic process of the explanation. However, this is not the case. The illustration visualizing interrelationships possesses what one could call a power of its own, and thus influences the selection of the explanation content and the use of specialist terms. The generally quite considerable difference between the two recipients, and consequently the expert's adaptation to the layperson's prior knowledge and his perspective - quite different from the expert's perspective decreases when a specialist illustration is available even if it is not copresent. In addition, informing the expert of copresence alters the explanations and influences the selection of content. Items of information contained in the illustration were selected as themes, purely because they were present in the illustration, whether they were necessary for the reply to the layperson's or the co-expert's inquiry or not. On the basis of these findings, it would seem that specialist illustrations have a negative impact with respect to net-based expert-layperson communication. At least in this study, specialist illustrations rather seemed to act as a hindrance when selecting content and taking the recipient into account. Suthers & Hundhausen (2002) report a similar observation with groups learning in a netenvironment with external representations. The different external representations available influenced the participants' contributions in thematically identical tasks. Suthers & Hundhausen call this the 'salience' of external representations which molds communicative behavior. To sum it up it can be said that the presence of an illustration in our scenario influenced the quality of the experts' explanation in an unfavorable way or - in other words - lead to an illusion of evidence. As the results show, the copresence of illustrations tempts experts to use less audience design. In contrast to study 1 where the most clearly results touch upon the differences between the impact of the representation format on explanation behaviors, study 2a and 2b identify differences within both groups working with an illustration at hand. Hence it has to be stated that content analysis of experts' explanations and their results on laypersons do not provide fully overlapping results. Further research has to be done to illuminate the differences and connections between those different research methods. Nonetheless results of studies 2a and 2b showed that the differences founded in study 1 also affect laypersons ratings of text comprehensibility and performance. Especially the comparison between 'illustration copresent' and 'illustration not copresent' showed a general difference between this
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
115
two conditions. Participating laypersons rated as well as performed worse under the condition 'illustration copresent'. One can speculate if the laypersons perceived the poor didactical embedding of the illustration as a problem of simplicity and less audience design. As mentioned before, earlier studies showed that the illustration was rated as not helpful for understanding the facts. Furthermore the bad performance in the cloze procedure task strongly indicates an underlying illusion of evidence on the part of the experts. Further studies should clarify whether a more effectively interactive design of the communicational situation would reduce the dominance of the external representation over the task requirements. We need to examine whether the reported findings would also emerge in an advice scenario constructed in this way. We will also investigate empirically whether the strong influence of graphical external representations on the form of communicational contributions can be reduced if, in addition, experts are explicitly requested to anticipate the layperson's prior knowledge. Just as technical external representations can affect communication negatively, it is conceivable that other representations can also affect it positively. Representations with a degree of difficulty adequate to laypersons may support the communication quite extensively. In further studies, information with respect to group membership (and thus the expertise status) will be supplemented by taking into account the style in which inquiries are written. It remains to be seen what kind of impact this has on the ensuing contributions to the communicational partner seeking advice. A series of studies which is in preparation at present (cf. Bromme & Jucks, 2002) investigates among other things the application of the linguistic copresence heuristics (Clark & Murphy, 1982) in asynchronous expert-layperson communication. We hope that this will contribute to identifying barriers in net based health advice and consequently can lead to reduce them in everyday expert-layperson-communication. ACKNOWLEDGEMENTS The research that is reported here has been funded by the DFG (BR 11260-1). The authors are thankful to Tina Becker, Tanja JanJ'en, Ulrike Schmidt, Annika Schneider, Petra Schulte-Lobbert, Maria Senokozlieva, Katrin Sommer and Verena Vogel for collaborating on the issue and to Ingrid Speight and Thomas Wagnerfor improving our English
NOTES
'
This study has already been published in German (Jucks, Bromme, & Runde, 2003). As it is important for the whole issue we report some of our earlier findings in section 4.2.
REFERENCES Alpay, L., Giboin, A., & Dieng, R. (1998). Accidentology: an example of problem solving by multiple agents with multiple representations. In M. W. Van Someren, P. Reimann, H. P. A. Boshuizen, & T. de Jong (Eds.), Learning with multiple representations (pp. 152-174). Amsterdam: Pergamon. Bartels, M. (1992). Personzentrierte Gestaltung von Texten und Auswirkungen auf Verstehen und Behalten. Jahrbuch personzentrierte Psychologie und Psychotherapie, Band 3, 144-161. Beredjiklian, P. K., Bozentka, D. J., Steinberg, D. R., & Bernstein, J. (2000). Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome. The Journal of Bone and Joint Surger, 82-A(l I ) 1540-1543. Bormuth, J. (1968). Cloze test readability: criterion reference scores. Journal of Educational Measurement, 5, 189- 196. Brennan, S. E. (1998). The grounding problem in conversations with and through computers. In S. R. Fussell & R. J. Kreuz (Eds.), Social and cognitive approaches to interpersonal conzmunication (pp. 201-225). Mahwah, New Jersey: Lawrence Erlbaum. Brennan, S. E., & Ohaeri, J. 0. (1999). Why do electronicconversations seem less polite? The costs and benefits of hedging? Paper presented at the International Joint Conference on Work Activities, Coordination, and Collaboration, San Frnacisco. Bromrne, R., & Jucks, R. (2001). Wissensdivergenz und Kommunikation: Lernen zwischen Experten und Laien im Netz. In H. F. Hesse & F. Friedrich (Eds.), Partizipation und Interaktion im virtuellen . Seminar (pp. 81-103). Munster: Waxmann. Bromme, R., & Jucks. R. (2002). Rezipientenorientierung bei der netzgestiihten, schriftlichen Komrnunikation rwischen Experten und Laien. Unverbff. Projektantrag an die DFG. Westf. Wilhelms-Universitit Munster. Bromme, R., Ramhow, R., & Ntickles, M. (2001). Expertise and estimating what other people know: the influence of professional experience and type of knowledge. Journal of experimental psychology: Applied, 7(4),317-330. Brown, J. D. (1983). A closer look at cloze: validity and reliability. In J. W. Oller (Eds.) Issues in language testing research (237-251). Rowles, Massachusetts: Newbury Publishers. Brown, P. M., & Dell, G. S. (1987). Adapting production to comprehension: The explicit mention of instruments. Cognitive Psychology, 19,441-472. Bruer, J. T. (1994). Schools for Thought: A Science of Learning in the Classroom von John T. Bruer: Bradford. Buhl, H. M. (1996). Wissenserwerb und Raumreferenz. Tubingen: Niemayer. Clark, D., Weinberger, A., Jucks, R., Spitulnik, M., &. Wallace, R. (in press). Designing Effective Science Inquiry in Text-Based Computer Supported Collaborative Learning Environments. International Journal of Research in Politics and Education. Clark, H. H. (1992). Arenas of language use. Chicago: The University of Chicago Press. Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press. Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine & S. D. Teasley (Eds.), Perspectives on socially shared cognition. (pp. 127-149). Clark, H. H., & Marshall, C. R. (1981). Definite references and mutual knowledge. In A. K. Joshi, B. L. Webber, & I. A. Sag (Eds.), Elements ofdiscourse understanding (pp. 10-63). Cambridge: University Press. Clark, H. H., & Murphy, G. L. (1982). Audience design in meaning and reference. In J.-F. Le Ny & W. Kintsch (Eds.), Language and comprehension. (pp. 287-299): North Holland. Clark, H. H., & Murphy, G. L. (1982). Audience design in meaning and reference. In J. F. LeNy & W. Kintsch (Eds.), Language and cornprehension (pp. 287-299). Amsterdam: North-Holland Publishing Company. Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22, 1-39. Duden Band 8 (1997). Die sinn- und sachverwandten Worter. Mannheim: Dudenverlag. Eysenbach, G., & Diepgen, T. L. (1999). Patients looking for information on the internet and seeking teleadvice. Archives of Dermatology, 135, 151-156. Eysenbach, G., Powell, J., Kuss, O., & Eun-Ryoung, S. (2002). Empirical studies assessing the quality of health information for consumers on the World Wide Web. JAMA, 287(20), 2691-2700.
NET-BASED EXPERT-LAYPERSON-COMMUNICATION
117
Fussell, S. R., & Krauss, R. M. (1991). Accuracy and bias in estimates of others' knowledge. European Journal of Social Psyclzology, 21,445-454. Fussell, S. R., & Krauss, R. M. (1992). Coordination of knowledge in communication: Effects of speakers' assumptions about what others know. Journal of Personality and Social Psychology, 62(3), 378-391. Gemini Consulting (2001). Patient Relationship Management. Die Rolle des Patienten in der LifeSciences-Industrie. Abgerufen am 19. MBrz 2002, von http://www.de.cgey. corn/servlet/PB/menu/ 1001426/ index.htm1 Genig, R. J., Brennan, S. & Ohaeri, J. 0. (2000). Illusory transparency revisited. Discourse Processes, 29, 137-159. Greene, B. (2001). Testing reading comprehension of theoretical discourse with cloze. Journal of Research in Reading, Volume 24,82-98. Grice, H. P. (1989). Studies in the way of words. Cambridge: Harvard University Press. Groeben, N. (1982). Leserpsychologie: Textverstdndnis - Textverstiindliclzkeit.MUnster: Aschendorff. Hinds, P. J. (1999). The curse of expertise: The effects of expertise and debiasing methods on predictions of novice-performance. Journal of Experimental Psyclzology: Applied, 5(2), 205-221. Hinds, P. J., Patterson, M., & Pfeffer, J. (2001). Bothered by abstraction: The effect of expertise on knowledge transfer and subsequent novice performance. Journal of Applied Psychology, 86(6), 12321243. HON (1997). HON Code of conduct for medical and health web sites, 1997, from http:/kon.chlConduct.html Horton, W. S., & Keysar, B. (1996). When do speakers take into account common ground? Cognition, 59, 91 - 117. Isaacs, E. A., & Clark, H. H. (1987). References in conversation between experts and novices. Journal of Experimental Psyclzology: General, 116(1), 26-37. Jucks, R. (2001). Was verstehen Laien? Die Verstandlichkeit von Fachtexten aus der Sicht von Computer-Experten.Munster: Waxmann. Jucks, R., Bromme, R. & Runde, A. (2003). Audience Design von Experten in der netzgestutzten Kommunikation: Die Rolle von Heuristiken Uber das geteilte Vorwissen. Zeitschriftfir Psychologie, 211, (2). 60-74. Keysar, B. (1994). The illusory transparency of intention: Linguistic perspective taking in text. Cognitive Psychology, 26(2), 165-208. Keysar, B., Barr, D. J., Balin, J. A., & Paek, T. S. (1998). Definite reference and mutual knowledge: Process models of common ground in comprehension. Journal of Memory and Language, 39, 1-20. Kobayashi, M. (2002). Cloze tests revisited: exploring item characteristics with special attention to scoring methods. The Modern Language Journal, 86,571- 586. Langer, I., Schulz v. Thun, F., & Tausch, R. (1993). Siclz verstandlich ausdriicken. Miinclzen; Basel: Ernst Reinhard Verlag. Langer, I., Schulz v. Thun, F., Meffert, J. & Tausch, R. (1973). Merkmale der VerstBndlichkeit schriftlicher Informations- und Lehrtexte. Zeitsckrift fur experinzentelle und angewandte Psychologie, 20, 269-286. Li, L., Irvin, E., Guzman, J., & Bombardier, C. (1998). Surfing for back pain patients: The nature and the quality of back pain information on the internet. Spine, 26(5), 545-557. Lowe, R. K. (1989). Search strategies and inference in the exploration of scientific diagrams. Educational Psychology, 9(1), 27-44. Lowe, R. K. (1996). Background knowledge and the construction of a situational representation from a diagram. European Journal of Psychology and Education, 11(4), 377-397. Lumbelli, L., Paoletti, G., & Frausin, T. (1999). Improving the ability to detect comprehension problems: From revising to writing. Learning and Instruction, 9(2), 143-166. Nickerson, R. S. (1999). How we know - and sometimes misjudge - what others know: Imputing one's own knowlege to others. Psychological Bulletin, 125(6), 737-759. Piper, T. & Mc Eachem, W. R. (1988). Content bias in cloze as a general language proficiency indicator, English Quaterly, 21 (I), 41- 48. Polichak, J. W., & Gerrig, R. J. (1998). Common ground and everyday language use: Comments on Horton and Keysar (1996). Cognition, 66, 183-189. Pricewaterhousecoopers (2001). Gesundlzeitsportale 2001. Abgerufen am 07. Oktober 2002, von http://www.medical-communities.de/pdf/p~ouse.pdf
Rambow, R. (2000). Experten-Laien-Kommunikation in der Architektur. MBnster: Waxmann. Reimann, P. (1997). Lernprozesse beirn Wissensenverbmit Beispielen. Bern: Huber. Robnagel, C. (1995). Ubung und Htirerorientierung beim monologischen Instruieren. Zur Differenzierung einer Grundannahme. Sprache & Kognition, 14, 16-26. Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A simplest systematics for the organization of turntaking for conversations. Language, 50(4), 696-735. Schober, M. F., & Clark, H. H. (1989). Understanding by addressees and overhearers. Cognitive Psychology, 21(2), 21 1-232. Schiirer-Necker, E. (1991). Der Einflub des emotionalen Gehaltes eines Textes auf seine Versti4ndlichkeit Zeitscltriftfur experimentelle Psycttologie, 38(1), 63-75. Speck, A. (1993). Textproduktion irn Dialog. Unvertiffentlichte Dissertation, FU Berlin. Stehr, N. (1994). Arbeit, Eigentum und Wissen: Zur Tlteorie von Wissensgesellsclzaften. Frankfurt: Suhrkamp. Suarez-Almazor, M. E., Kendall, C. J., & Dorgan, M. (2001). Surfing the net - information on the World Wide Web for persons with arthritis: patient empowerment or patient deceit? Journal of Rheunuztology, 28(1), 1-2. Suthers, D. D., & Hundhausen, C. D. (2002). The effects of representation on students' elaborations in collaborative inquiry. In G. Stahl (Ed.), Computer support for collaborative learning: Foundations for a CSCL community (pp. 472-480). Hillsdale, NJ: Lawrence Erlbaum. Taylor, W. L. (1953). Cloze Procedure: a new tool for measuring readability. Journalisnt Quarterly, 30, 415-433. Taylor, W. L. (1957). Cloze readability scores as indices of individual differences in comprehension and aptitude. Journal of Applies Psychology, 41, 19- 26. Van Bruggen, J. M., Boshuizen, H. P. A,, & Kirschner, P. A. (2003). A cognitive framework for cooperative problem solving with argument visualization. In P. A. Kirschner, S. J. Buckingham Shum & C. S. Carr (Eds.), Visualizing arguntentation: Sqfhvare tools for collaborative and educational sense-making (pp. 25-47). London: Springer. Wittwer, J., Bromme, R. & Jucks, R. (2004). Kann man dem Internet trauen, wenn es um die Gesundheit geht? Die Glaubwtirdigkeitsbeurteilung medizinischer Fachinformationen im Internet durch Laien. Zeitscltriftfur Medienpsycltologie,2,48-56.
bromme @mi-muenster.de jucks @mi-muenster.de arunde @ uni-muenster.de
ANNE H. ANDERSON, JIM MULLIN, RACHEL MCEWAN, JAY BAL, JEAN CARLETTA, EDWARD GRATTAN & PAT BRUNDELL
EXPLORING WHY VIRTUAL TEAMWORKING IS EFFECTIVE IN THE LAB BUT MORE DIFFICULT IN THE WORKPLACE. Abstract. In this chapter we report a series of studies on the use of multimedia communication technologies. These studies in the lab and in the workplace have explored how people in distributed teams communicate and collaborate. We have identified several aspects of the process of communication and collaboration process which multimedia technologies can support very effectively and others that are more problematic for distributed virtual teams. The investigations we conducted explored the communication process in detail, examining aspects of the turn taking process, the patterns of interactions among team members and how these related to the way the communication technologies had been implemented. The results indicated that the way facilities are implemented and factors such as status and organizational relationships can have noticeable impacts on the behaviour of virtual teams. Suggestions will be made about how to exploit multimedia technologies to deliver real benefits for virtual team working.
1. INTRODUCTION To compete in the global marketplace many organizations are investigating new ways of working. Many of these exploit the potential of communication and information technologies in general and the use of computer supported collaborative working (CSCW) in particular. Large economic advantages could accrue if information and communication technologies could be used successfully to support real time computer supported collaborative working. Several authors have described the possibilities offered by virtual teaming (Grenier & Metes, 1995; Lipnack & Stamps, 1997; Snow et al., 1996). Although others (DeMeyer, 1991; Nohria & Eccles, 1992) are more sceptical and have stressed the importance of various aspects of collocated working, see for example the review by Olson & Olson (2000). Researchers have responded to the growth in new technologies and new ways of working which these can facilitate by exploring the behaviour of technologically supported groups in the laboratory and in the work place. So what does the existing research literature suggest are the barriers that have to be overcome before the benefits of virtual team working are realised? What are the biases that we might anticipate would emerge if we compare these new ways of working with traditional face-to-face interactions? There is wide spread optimism that the technological barriers of providing access to effective information and communication technologies of increasing sophistication can be overcome. So many organisations will be able to provide their staff with access to tools such as desktop videoconferencing which allows person-to-person connections often with access to shared tools, ranging from simple shared whiteboards through to more complex shared software applications. If a company wished to experiment or
120
ANDERSON ET AL.
implement virtual teaming there will be cost implications for equipment and training for staff. Although the technologies are accessible the cost implications of different types of technological support mean that organisations have to decide how to implement virtual team working and the choices made in terms of the equipment provided may produce biases in team behaviour. As well as the technology itself, are there other potential barriers to effective virtual team working? Recent laboratory studies of groups collaborating via technologies suggest that there may be social aspects of distributed working that may be difficult. Researchers have explored virtual teams' attitudes and interactions style (e.g Bos et al., 2002; Bradner & Mark, 2002; Potter & Balthazard, 2002). These lab studies seem to suggest that though the technologies can support similar interaction and interaction styles to face-to-face interactions, they can have subtle impacts on the participants so that people are more likely to distrust or deceive remote collaborators and are slower to trust them. Although there is considerable interest in the role of virtual teams in organizations, there are relatively few studies that report evidence of the way such teams communicate during their virtual meetings. O'Conaill et al. (1993) found that work teams were more formal and less interactive in videoconferences than in faceto-face meetings. Tang & Isaacs (1993,4) report that teams were more task focused and held briefer interactions in videoconference meetings. Herbsleb et al. (2000) noted patterns of information exchange in international virtual teams and how the information exchanges were perceived. They found virtual team members felt that they were equally helpful to their local and remote fellow team members but perceived the remote team members as less helpful to them. In a study of the work place from our own group, France et al. (2001) found that when virtual team meetings were observed, the impacts of technology interacted with status. Managers were found to dominate (i.e. say more than more junior staff) in face to face team meetings but this effect was much greater in audio-conference meetings. This contrast with reports from earlier studies of the organizational use of e-mail, such as Kiesler & Sproull (1992), who found e-mail to have an equalizing effect on corporate patterns of communications. So there may be biases in the way that virtual teams operate. These may reflect social or organisational factors such as how people interact with their remote collaborators, how friendly and helpful they perceive them to be and how egalitarian or hierarchical are their patterns of communication. 1.1 Overview of Chapter
In this chapter we report briefly on a number of studies that we have conducted to explore the impact of multimedia communications technologies on collaborative working in small groups. We combine laboratory experiments and field studies in the workplace in an attempt to obtain an overall picture of the impacts of these new virtual team working technologies. One way to assess how effectively virtual teams operate is to assess how well they communicate and how effectively they perform their joint tasks. We adopt this particular research focus on detailed analyses of the
BARRIERS AND BIASES IN VIRTUAL TEAMS
121
communication processes within groups, because we believe these provide very sensitive indicators of how effectively the group is operating and of the impacts of the use of communication technologies. In the first study we conducted a lab based study using a fairly large number of comparable teams of participants who were unfamiliar with one another and who all tackled identical problems under controlled and comparable conditions. This study provides a rigorous test of the impact of use of multimedia communications technologies on the performance and patterns of communication of distributed problem solving groups. In the second study we studied how virtual teams in the workplace communicated during their technology-supported meetings. We are again interested in the impacts of multimedia communications technologies on patterns of communication in distributed groups. These workplace observations provided as with very rich data from a small number of virtual teams. These interactions were complex not just because the teams were tackling real work problems but also because the participants had existing social and organisational relations with one another. We therefore had to consider the impacts not of only of the introduction of new technology but also how this interacted with factors such as status and organisational role. From our observations in the workplace we identified a number of interesting phenomena which we wished to explore further. These concerned the impact of organisational status and technology sharing on the patterns of communication in virtual team meetings. To do this we conducted what we called a simulation study. This involved a study in the lab of a real work-based problem that was tackled by several comparable teams of participants. The teams in the lab consisted of individuals with relevant industrial experience who role-played the different organisations we had observed in the workplace. We seek to address the following questions. How well to virtual teams perform tasks compared to face-to-face groups? Does the size of the group impact on task performance or patterns of communication? How does the introduction of virtual team working interact with social relations in the team such as status? Does the way the technology is implemented make a difference to group interactions? Can we suggest ways to overcome the possible biases of virtual team working? 1.2. Study 1: Lab Study of Virtual Team Working In study 1 we used a standard laboratory collaborative problem-solving task. This provides an objective measure of task outcome and allows many detailed analyses of the associated dialogues that can be directly compared across a large number of different speakers. This kind of multifaceted approach to evaluating the impacts of
122
ANDERSON ET AL.
IT technologies has been advocated by several researchers, e.g. (Monk et al., 1996). This kind of study allows us to make an initial exploration of the impacts of use of communications technology where there are no complicating factors such as the social relations between the participants or their personal or professional agendas. When such technologies are introduced in the workplace such factors clearly have to be considered and later in this chapter we report on studying just such complex interactions of technologies and social and organisational relations. In our explorations of technology supported collaborations in the laboratory we used a collaborative problem solving task, the Map Task (Brown et al., 1984), which elicits spontaneous yet comparable dialogues from different speakers and which we have used in previous research, (e.g. Anderson et al., 1997; Doherty-Sneddon et al., 1997). The task has been found to be sensitive to the effects of communicative medium. We have conducted a large number of earlier studies on two-party faceto-face interactions which forms a back drop and a source of comparison for our studies of technology supported and small group interactions. interactions (e.g. Anderson, et al., 1991; Boyle at al., 1994; Anderson & Boyle, 1994). In this first lab study of virtual team working we address the following questions: -
How well do small groups perform a collaborative task when supported by multimedia communications technologies?
From the previous experimental literature on technology-supported dyads, our hypothesis was that task performance would be equal in technology-supported and face-to-face interactions. -
How do three party multimediated interactions differ from two?
Our hypothesis was that arriving at a common understanding and problem solution would require more communicative effort in three person groups. We predicted longer dialogues in these conditions. -
How do these multiparty multimediated interactions differ from similar face-to-face interactions?
Our hypothesis was that face-to-face interactions provide more visual cues and a better sense of mutual understanding than can be provided by videoconferencing technologies. We predicted that communication would be easier and shorter in faceto-face interactions. 1.2.1. Participants One hundred and forty eight undergraduates were paid •’5 sterling for participating. For the video-mediated conditions (VMC), undergraduates were recruited at both the University of Glasgow and the University of Nottingham. In the face-to-face condition all participants were recruited at the University of Glasgow.
BARRIERS AND BIASES IN VIRTUAL TEAMS
1.2.2. Task The task used in this study was the Map Task (Brown et al., 1984). This is a collaborative problem-solving task which elicits spontaneous yet comparable dialogues from different speakers. In this study two different versions of the Map Task were used. Participants each have access to a copy of a schematic map, either on screen or on paper. The maps bothfall show a start point and a number of named landmark features. Some of these features are identical on each participant's map, whilst some differ. The number of landmarks and the number of discrepant features are the same in both map tasks. The instructions giver(s)' map also shows a route. In three party versions of the task both instruction givers have the identical route but again some of the landmark features differ between their copies of the map. The object of the task is for the instruction giver(s) to communicate how to complete this route to the instruction follower. 1.2.3. Design Sessions were run in blocks for the 2 party VMC, 3 party VMC and 3 party face-toface conditions (in order to reduce reconfiguration requirements on the computers used in the VMC conditions). Participants attempted two versions of the Map Task, swapping between the instruction giver (IG) and the instruction follower (IF) role, on trials one and two in the two party VMC. In the three party conditions, two of the three participants swapped between instruction giver and instruction follower roles, while one remained an instruction giver in trials 1 and 2. None of the participants knew one another before the trials. Table 1 shows the number of trails in each condition. Subjects were randomly assigned on their first task to the instruction giver or instruction follower roles. Audio recording were made of all sessions and the detailed word level transcripts were made of the tape recordings. Table I . Study I: Trials in each Condition. Condition Number of Trials
1 2 Party VMC 1
46
1 3 Party VMC 1
36
1 3 Party Face-to-Face 1
32
1.2.4. Procedure Face-to-Face conditions: In the 3 party face-to-face conditions, participants sat at different sides of a table, with low screens placed between them to prevent them from seeing one another's maps, to ensure that communication had to take place verbally as in the VMC conditions. Participants' faces were fully visible to one another and paper maps were used. Video Mediated Conditions: All the participants were novices at videoconferencing, and so were shown how the camera captured and relayed their images. They were also shown how to use the mouse to draw the route on screen. Participants were reminded that the other (or one of the other) participants was at a remote site (approximately 300 miles away).
ANDERSON ET AL.
1.2.5. Instructions to Subjects Subjects were told that they each had a map of the same place but that, as the copies had been drawn by different explorers, there were some differences in the landmarks shown on the maps. Only the instruction giver(s)' maps had a route on it, and their task was to tell the instruction follower how to reproduce the route accurately on their map. In VMC conditions participants were told to look into the camera when they wished to look at the other person. 1.2.6. Equipment In the VMC conditions each participant sat at a workstation which displayed an image of the map (6.3" X 7.85") and video window(s) of the other participant(s). A camcorder was trained on the local participant. The workstations processed and sent video images across the network to the other workstation(s). An Asynchronous Transfer Mode (ATM) network directly connected the workstations using IP over ATM. Connections between Glasgow and Nottingham were made via the SuperJANET ATM network. Delivery of video images across the network averaged over 24.9 fps in every session. 1.2.7. Task Outcome As we report in detail in Anderson et al. (1999), task outcome was evaluated by comparing the map drawn by the instruction follower with the original being described by the instruction giver(s). The area of deviation between the two routes was measured and these scores were compared across conditions. Overall task performance does not differ significantly in two or three party mediated interactions and these do not differ from those achieved in three party face-to-face interactions. The deviation scores (in square centimeters) for the three conditions are shown below, (higher deviation scores indicate less accurate map routes and so poorer task performance). The performance achieved was as good as that found in our earlier studies of two-party face-to-face interactions. The results are: VMC 2 Party: 60.1 (MSE 6.0); VMC 3 Party: 49.0 (MSE 6.7); Face-to-Face 3 Party: 62.9 (MSE 7.1). 1.2.8. Communication Analyses To explore the interactions a number of analyses were conducted. First the overall lengths of the dialogues were examined and the number of words spoken in each dialogue was totalled. These data were subjected to a 2 (Trial 1 vs. Trial 2) X 3 (Condition: 3 Party Face-to-Face, 3 Party VMC, 2 Party VMC) Mixed ANOVA, with Condition as a between subjects variable and Trial as a within subjects repeated measure. This analysis showed only a significant main effect of Condition, F (2,54) = 3.27, p 5: internal hosting of the web site.] At 12:18:20, after some pieces of software to be used have been discussed, the programmer embarks on a new parameter: miissen wir uns schon gedanken um die navigation machen - z.b. eine fixe linke hauptnavi und eine sich entsprechend andernde kopfnavi ... oder andersrum vijllig egal !!!? [we must think about navigation, e.g., a fixed main menu on the left and a context-dependent menu at the top ... or the other way around, doesn't matter!!!?] At 12:18:55 the graphics designer agrees: Linke Hauptnavi find ich angenehm [I like a fixed main menu on the left]. But the project manager, at 12:19:09, defers the issue: Nein, wie gesagt, erst muss die Struktur stehen, das ist jetzt unser Hauptaugenmerk [No, as I have said before, we must first agree on the structure, that is our main point that needs attention].
In total, about half of the task-relevant chat exchanges (47.6 percent) dealt with design parameters either explicitly (i.e., naming the parameters, 20.6 percent), or implicitly (27.0 percent). The other half of the discussion focused on how to arrange application-specific concepts only vaguely familiar to the team members. (For this
202
STRUBE ET AL.
study, the domain of application chosen was financial services, especially insurance products.) These data show that an essential part of knowledge communication in web design is organized around, or even organized by, design parameters. These parameters imply design decisions, i.e., characteristics of the future product (the website) where designers have several options. We might also say, therefore, that the communication of knowledge focuses on choice points, organizing issues of economy, but also design principles, or even personal preferences. In the terminology developed by Fischer and Ostwald (this volume), design parameters take on the functional role of boundary objects. They can be accessed from the perspective and domain of expertise of each team member, while other knowledge associated with them (like how to implement a certain design decision), may well remain in the individual expertise of only one team member.
5.2 Examples of design options In the following, we briefly describe some of the dozens of design parameters, and the values they may take: Parameter: Corporate Identity (CI). Value: List of CI items (logo, font, etc.) and their format (digital or print media); default value: (non-digital logo). Relevant to: Project Manager (PM), Screen Designer (SD). - The SD needs to know whether CI material, especially a company's logo, are available (or will be made available by a graphics designer in the team). If there is CI material, the PM needs to know whether it is already digitized or must be digitized. If no CI material has been made available by the customer (empty list as value of the parameter CI), the SD has to design it. The default value recognizes that most customers already have CI designs, e.g. letterheads with the company's logo. Parameter: Multilingual Site. Value: List of languages; default value: (native language). Relevant to: PM, SD, HTML programmer (HP). International companies and organizations usually require a multilingual site, e.g., for their customers in different countries. The PM must know whether the materials are available in all languages required. The HTML programmer must install the necessary links. The SD bears most of the load, having to design several sets of pages in parallel. The latter is necessary because of different conventions affecting usability, e.g., users speaking languages that are written right-to-left usually start at the top right corner, whereas speakers of European languages start reading a page at the top left. Parameter: Purpose of Entertainment. Value: yes or no; default value: no. Relevant to: PM, SD. - The SD needs to know whether the site is purely information-centered, or should be entertaining as well. While purpose of entertainment and information-centeredness are independent parameters in principle, some means of entertainment (e.g., an animated introduction with
music) may interfere with a user's information needs (e.g., getting the information quickly, without a lengthy introduction). Basically, it's the customer who decides. Parameter: Size of Site. Value: minimum screen size in pixels, and scrollable or not; default value is 760 by 520 pixels, non-scrollable. Relevant to: SD, HTML programmer. - The HTML programmer has to decide this according to statistics about screen sizes of different user groups. Because of this dependency on the site users, the PM also has a word in this. The default acknowledges that VGA is already a standard of the past, and that many users fail to notice a part of the document that becomes visible only by scrolling. 6. KNOWLEDGE-LEVEL MODELLING
During production, experts work with minimal communication in parallel to construct the website prototype. The distribution of work as depicted in Figure I, which shows the task decomposition and the overall sequence of work-steps during the prototype production phase, together with the results of the subtask dependency analysis and the financial project plan, would be sufficient to construct a workflow model of web design. A model like that, however, would not be able to capture the knowledge that drives the design process, i.e., the design decisions that result in parameter settings. Therefore, the next logical step is to develop a model of the relevant knowledge. Expert work, in all domains, is certainly a knowledge-intensive process, and there exists a vast literature on how to formalize task-specific knowledge (e.g., Stefik, 1995). However, we are interested in the group processes and not the individual processes. Therefore, we do not aim at modelling each expert's taskrelevant knowledge (as it has been attempted in research on expert systems, e.g., Breuker & Wielinga, 1985). To the contrary, our present work ignores individual expertise because it is private and largely irrelevant for the team, although it is instrumental for the resulting product. What is of importance, and what our model should account for, is knowledge shared by all (or at least, by several) team members, and the processes that are responsible for its deployment. To summarize, shared knowledge in web design comprises:
-
-
common background knowledge, e.g., the two upper levels of the task decomposition and dependencies between subtasks, together with metaknowledge about how expertise is distributed (a feature focused on in the transactive memory approach by Wegner, 1987), shared knowledge about the design decisions that must be taken in order to specify each expert's part of the design.
In addition, we are interested in how the design decisions are negotiated (the process by which shared knowledge about the specifications is generated); this last issue is, however, beyond the scope of this chapter.
204
STRUBE ET AL.
The common background knowledge deals with the overall and standardized procedure of web design, and may well be integrated with existing approaches for workflow systems. This leaves open the question of how to represent the design decisions, i.e., the values of the design parameters. All team members have complete knowledge about the parameters that are relevant to their own work, and about possible constraints among parameter values that affect their own work.4 Additionally, everyone has at least some knowledge about parameters that are relevant to the other team members, but it is not complete, and may be ignorant of subtle constraints and dependencies among parameter values.
6.1 Representing design decisions We propose to model the knowledge pertaining to a given role in the web design team as a kind of frame, or software object (according to object-oriented programming, e.g., Khosafian & Abnous, 1990). In the following, we call this structure a role schema, or schema for short. Schemas have been used widely in psychology for knowledge representation, e.g., of common social events like visiting a doctor, or eating at a restaurant (so-called scripts: Schank & Abelson, 1977). Task definition
5 Project manager
1
mmaatc WIW yln
mfurmaHonenwed rhn fP*f Idigltd W d l y l "In nlvltlliyual rite dlxurrlon plvum ddSbds~POiiuton
"I" "I" "In
4 Screen Designer
4 HTML Programmer
Figure 3.Part of the knowledge-level model for collaborative web design, showing schemas for the roles of project manager, screen designer, HTML programmer, and software programmer. Only the headers of those schemas are visible to other team members. The schema headers contain the parameters relevant to the role, as well as the values of parameters that result from design decisions (and default values where no design decision has been taken).
A schema has a header and a body. In our case, the body contains all the taskrelevant expert knowledge 'owned' by the role. This knowledge is private, i.e., not visible to the other team members. It consists of a (presumably large) number of modules for executing the subtasks associated with each role, but even the kind and
number of the subtasks is not known to the other team members. The modules, however, need to be parameterized. For instance, graphics designers must know whether they can use a logo provided by the client, or whether they have to develop a logo. The screen designer, on the other hand, does not need to know about the source of the logo, or how to produce one, but needs to know where it should be placed on the screen, and how large it should be. The HTML programmer, in turn, does not need to know any of these parameters, but instead wants to know how the logo and other graphics materials are encoded (PEG, GIF, or whatever). In sum, the schema structure can be used here to encapsulate the knowledge that is private and not shared. The header is visible to the other team members. It contains the parameters needed by the routines in the body of the schema. These parameters are set as the result of design decisions, which may be met in group discussions. (Usually, there is one team meeting at the end of the preparatory phase where many of these issues are discussed, as already described above.) Schemas may also contain default values of parameters (see the examples given above). This obviates the need for explicitly setting all parameters. On the other hand, default values are a possible source of misunderstandings in the web design process if they happen to be interpreted differently by different team members. Of course, there are many formal alternatives to represent knowledge (see Stefik, 1995, for an overview). But the schema-based approach outlined here has the advantage of representing the accessibility of the knowledge involved more clearly than other conventional formalisms. 7. SUPPORTING THE WEB DESIGN PROCESS We have seen that in web design, communication between the team members is kept to an economic minimum, and is highly efficient. On the other hand, IT magazines are full of stories about web design projects that have failed. Is that just because many firms of the 'New Economy' are not well-grounded financially? Or do errorprone communication and coordination problems play a role as well?
7.1 Barriers and biases First, we can easily identify barriers in web design teams. Their very strength most knowledge being individual expertise, and only a little background and metaknowledge being shared - is a potential threat to the following areas which constitute a collaborative task:
7.1.1 Task coordination. As pointed out earlier, web design is a complex task divided into many different subtasks to be executed by several different experts. In order to make this divisionof-labour scheme work, effectively shared meta-knowledge about roles and responsibilities is needed to establish the group's transactive memory system (Wegner, 1987). Problems may be due to different factors. One of these, likely to
206
STRUBE ET AL.
occur in real companies, is fluctuation. Another one is team size, still another is geographical distribution.
7.1.2 Information distribution. As the division-of-labour scheme requires the coordination of task responsibility, it also requires an appropriate distribution of information. This is another aspect of Wegner's (1987) transactive memory model, stating that experts are responsible for keeping track of information belonging to their area of expertise. It follows that if the other group members get some pertinent piece of information, the have to pass it on to the right expert. This kind of information concerns design parameters as well as information belonging to private expert knowledge. Inappropriate knowledge distribution, therefore, emerges as another potential barrier in expert groups like web design teams. One reason for such an inappropriate distribution could be the 'CISbias' (Stasser, 1992): Empirical studies show that in group discussions, people prefer to discuss information already known to everyone. This tendency can become critical if relevant, but private information fails to get communicated. Concerning the situation in web design teams a potential bias in information distribution could be that important information is not communicated to the expert responsible for it. Another potential error is implicit in our present knowledge model, since it is hidden in the private expertise of the role schemas. This issue concerns the subtle interdependencies of parameter values, not all of which are known to us at present. For example, only the HTML programmer knows that if the 'mouse over' technique for activating hyperlinks has been chosen, the only choice in today's state of technology is to use Java Script instead of other options. It may well be that the screen designer, caring about website usability, advocated the choice but was ignorant of the consequences. If the HTML programmer was present, but not attentive at the time when the 'mouse over' option was chosen, he might have agreed to using some other scripting language, only to find out later - too late, in fact! - that there was a conflict between two design decisions. So, another barrier is that even if information regarding design decisions is communicated to everyone, resulting consequences might not be, and conflicting decisions might be detected too late. 7.1.3 Grounding. As web design is done in multi-expert teams, a third potential barrier is a lack of common ground (Clark, 1996). Expertise usually comes with its domain-specific terminology, and even the same terms are not understood equally by different professions. If one of the team members has a model of another member that is too 'na'ive' (in the sense of Lewis & Sycara, 1993), misinterpretations most probably result. To counteract such errors, instances of explicit 'grounding' will be likely to occur (and did so, cf. Section 5.1) - although such grounding of the dialogue detracts from the objective to be economic. But this procedure requires that the misunderstanding be detected. Yet another potential source of error are terms or design parameters understood in essentially the same way by all, but with different default values. Default values
are what biases an interpretation - if a design option has not been discussed explicitly, each member will automatically assume the default value. But the defaults are not necessarily shared. They may be influenced by personal preferences (as in our example above, where the project leader seems to detest static HTML, and prefer dynamic HTML), but also by technical progress. In the case of static versus dynamic HTML, static HTML has been available for a long time, whereas dynamic HTML needs a database server, considerably more computing power, and certainly was the exception five years ago. Here we have a point where again, economy of communication may jeopardize understanding.
7.1.4 Documentation. People usually rely on external representations to extend their individual memories. In a complex task like web design, in which many design parameters have to be set and multiple experts collaborate, it seems highly probable that documentation of design parameters and other materials such as meeting protocols is necessary. Therefore, a last barrier can result from a lack of, or an inappropriate choice of documentation. As we are interested in supporting knowledge communication in expert teams like the web design teams in our example, our aim is to identify the requirements which need to be fulfilled in order to make the communication of knowledge work. Starting from analyses of the potential barriers and biases listed above, we can infer that teams need meta-knowledge to coordinate their tasks and to distribute information effectively, a body of shared knowledge concerning at least the design parameters and an effective documentation. Before designing a support system which meets all these requirements, it seems necessary to explore in more detail how the barriers and biases emerge in real teams, and how companies attempt to prevent or overcome these barriers. To that end, we conducted the interviews described in the next section. 7.2 Interviews We conducted semi-structured interviews in four different web design companies (A, B, C and D), located in three different German cities. Two of the companies (B,D) were small (4 employees) and the other two (A, C) were of medium size with about 15 to 35 employees. The interviews contained several questions concerning the potential barriers identified above and lasted about one to two hours. Results are described in the following section.
7.2.1 Task coordination Assigning tasks to the different experts did not seem to be a problem in the web design teams under study. Tasks are assigned according to the expertise required to perform them. If there is more than one person qualified for a specific task, the actual workload of each expert is considered before distributing the task. This procedure works well in all of the companies. Only one company (A) reported that
208
STRUBE ET AL.
team members often performed more than one role (e.g., screen design and project management), which resulted in work overload. Another important aspect of task coordination is the timing of the projects. This works well in most companies, too. Projects are planned with deadlines that can be met under normal circumstances. Only company A reported problems with deadlines because they embark on many small projects, which are only roughly planned. A more common source of problems, mentioned by nearly all of the companies (A, B, C), were delays due to customers being late in providing important material. 7.2.2 Distribution of Information We were surprised that none of the companies reported problems concerning the distribution of information. Usually the project managers are responsible for passing information to the other team members. This is often done during team meetings. Important private information is not lost during this process and most of the time, all team members are sufficiently informed. Problems concerning the implementation of design parameters were only reported in one company (A). These were not due to insufficient communication, however. The recipients just seemed to not be taking the information seriously. There is no common structure of information distribution. In some teams the internal, as well as the external communication is mostly 'done' by the project manager (B, D) whereas in others (A, C), team members also communicate informally within the team as well as with customers. As project managers mostly function as links to the customers, it is essential that they are always well-informed and available. In one company (A) problems arose because project managers were sometimes not available when important decisions had to be taken. This resulted in them not being aware of the project's current state when they were communicating with customers. 7.2.3 Grounding Misunderstandings due to a lack of common ground can be distinguished on four different levels. Within the team. None of the companies reported misunderstandings because of a lack of common ground within the team. Companies A and D reported that they tried to prevent misunderstandings in the extensive meetings at the beginning of a project, and by adapting to the language usage of the other experts and relying on an already existing high amount of shared knowledge. With new team members. The designer of company A stated explicitly that the common ground within the teams is acquired over time, so that a common language has to be established with new team members. In other companies (B, D) misunderstandings with interns were reported as well. With customers. Two companies (A, B) reported severe misunderstandings with their customers, who seemed to have difficulty in understanding what web designers meant and in describing the kind of solution they wanted to have. In one company (A), these problems were so severe that projects became both expensive and nerve-
wracking. The other two companies (C, D) reported only minor problems due to a lack of common ground. Their customers are normally well-informed about technical solutions, and in a first meeting, project managers introduce customers to the project. In one company (D), a special checklist was developed in order to gather all the necessary information and to clarify all important issues during the first meeting. With freelancers. No severe misunderstandings emerged in the communication with freelancers, i.e., external experts enlisted only for one project. This was due to the following reasons: All companies tend to cooperate with the same people over a long period of time (not only for one project, but rather regularly), and freelancers get a workplace within the company (B, C). Freelancers are very motivated to clarify everything concerning their task, as they are not paid unless their work fits the requirements (D). Furthermore, freelancers are provided with a detailed task description (C). 7.2.4 Documentation What is documented always varies, depending on the project and the company. In most companies, project documentation is stored on a central server with a clear filestructure (B, C, D). Commercial software programs are not used. Instead, the companies have developed proprietary systems on their own, or state that they are looking for appropriate software to buy. In general, project documentation is clear and therefore, no problems were reported. Only in company A, the documentation is not stored centrally so that it cannot be accessed by everyone, and the file structure was sometimes so inconsistent that only a single person knew it. Some material has even been lost because of these inconsistencies.
7.2.5 R b u m t In general, the results of our interviews show that the communication of knowledge functions well in most of the companies we interviewed. The hypothesized barriers and biases do not seem to play an important role within real web design teams. Only company A reported problems concerning task coordination, information distribution and documentation. We therefore asked our interview partner what kind of support system she could imagine. The only thing she deemed useful was a support system for consistent documentation, as well as checklists for different tasks. A system for task management that would, for instance, remind users of their tasks and deadlines via e-mail was something that she considered to be a nuisance rather than a help. 8. DISCUSSION
Our results from the area of web design seem puzzling: With a really complex task, involving different roles and all kinds of expertise, we observed a style of knowledge communication that could hardly be sparser. The plausible conclusion that this should lead to a host of problems was not borne out. Of the four companies interviewed, only one reported frequent and serious problems - and even that
210
STRUBE ET AL.
company did not report problems of a lack of common ground within a team. Instead of building a sophisticated IT solution for support, a system combining workflow and project administration components and enriching these with knowledge-based components that would control the setting of design parameters, we found, to our surprise, that some simple checklists would do the job. How can this be explained? We can only offer a post hoc explanation. After a period of about five years, when (at least in Germany) the so-called New Economy flourished and an incredible amount of money was available to be invested in web design projects, a crisis followed that drove the majority of web companies out of business. Only a small percentage remained, and the companies interviewed by us were among those successful few. What these four companies have in common is their lack of fluctuation. The web design teams there consist of people that have been working together for at least three years. We can be sure that they have acquired shared knowledge far beyond the amount necessary for doing a web design project. They have established firm social relationships. Therefore, a first explanation for our results is that our studies underestimated the amount of shared knowledge present. (As discussed in Section 1, minimal communication at execution time is possible when all relevant knowledge is shared.) A further point concerns communication means within teams. We observed that at least in one team, members wrote emails to other members in the same room, and even to themselves just for the sake of documentation. But we also observed that working in the same office is the rule rather than the exception for web design teams. Companies B and C, for example, want even the occasionally hired freelancer to work with them in the same office. We may therefore assume that communication within the team is face-to-face more often than not. A final observation is that web design companies have developed a style of their own over the years. This implies a common understanding of default values of design parameters, ruling out biases in interpretations. It also provides artifacts (i.e., web design cases, including documentation) as standard examples - another instance of a boundary object, since those examples are shown to potential customers in the acquisition phase, and also provide a model for possible novices, interns, and freelancers. This means that a company's style has benefits for the acquisition of shared knowledge, and for learning in general. It also enables extremely parsimonious discussions among team members: whatever is in accord with the style as a quasi-standard, does not need to be discussed at all. We cannot be sure how far our results can be generalized beyond the domain of web design. It seems plausible, however, to assume that relatively stable teams will acquire more than the minimal amount of shared knowledge, and that more shared knowledge will increase the stability of their collaboration. For design tasks, examples, models, and a recognizable style provide a kind of implicit 'scripting' (not unlike the 'epistemological scripting' of Weinberger et al., this volume), and the same holds for checklists of design parameters. Design decisions and the options available (in our terminology: the design parameters and their values) have been identified in other design domains as pivotal elements for the communication of knowledge (Fischer and Ostwald, this volume). It seems that although we chose a
special area of application, the phenomena observed and the explanatory constructs that we found useful may claim some generality. We thank the Deutsche Forschungsgemeinschaft for funding the work reported here (DFG STR 301/10-I), and half a dozen web services companies for their cooperation. We gratefully acknowledge the help of Karen KoJler and Benjamin Maier for some of the interviewing, for running subjects, and for preparing the graphics, and thank Sonja Stracke and Kristen Drake for their language assistance.
NOTES I
Why is project acquisition separated from the project proper? Presumably because in order to persuade a customer of one's ability to design the website exactly how he or she has it in mind, and then some, is an ability not often found, and usually absent in those people that do a web design project. However, important design decisions occur at this early stage, and problems engendered there can lead to severe problems later on. 2
For instance, the much acclaimed book by Goto and Cotler (2001) prescribes - 'one process fits all' - a strict sequence of '(I) defining the site, (2) developing the site structure, (3) visual design, (4) production.. .' This is not what web design teams do, and probably not what they should do. As one critic remarked: 'There's a relentlessly detailed approach to identifying and documenting every smallest feature of the client's business and requirements. The problem with this approach is that it would cost an enormous amount in person-hours just to gather the information, irrespective of how useful it proved to be. I wonder how many consultancies could afford to factor in such workloads without fear of pricing themselves out of the contract.' (Johnson, 2002). 3
Communications via electronic media are not only fast, but have the additional advantage of being self-documenting. It is also common for team members to send emails to themselves for better documentation. 4
For instance, when a design decision has been taken to activate links just by positioning the mouse cursor on them, 'mouse over', this implies that the HTML programmer must use Java Script.
REFERENCES Breuker, J. & Wielinga, B. (1985). KADS: structured knowledge acquistion ,for expert systems. Proceedings of 2nd Banff Conference on Expert Systems and Their Applications. Burdman, J. (1999). Collaborative web development. Strategies and best practices for web teanzs. Reading, MA: Addison-Wesley. Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making. In Castellan, J. Jr. (Ed.), Individual and group decision making. Current issues. Hillsdale, NJ: Lawrence Erlbaum. Clark, H. H. (1996). Using Language. Cambridge: Cambridge University Press. Dawson, P. J. (1997). Web design and HCI: Problerns and promises. Retrieved Feb. 6, 2001 from http://www.monroe.lib.in.us/-bpmchi/scholarship/dawson.html. DOmer, D. (1976). Problerrzlosen und Infonnationsverarbeitung.Stuttgart: Kohlhammer. Fischer, G. (1994). Domain-oriented design environments. Auto~rzatedSofnyare Engineering, 1 , 177-203. Goto, K., & Cotler, E. (2001). Web ReDesign: Workflow that works. Indianapolis, IN: New Riders Publishing. Khosafian, S., & Abnous, R. (1990). Object orientation. New York: Wiley. Kolodner, J. (1993). Case-based reasoning. San Mateo, CA: Morgan Kaufmann.
212
STRUBE ET AL.
Johnson, R. (2002). Web ReDesign [Book Review]. Retrieved June 8, 2003, from http:llwww.mantex.co.uWreviews/goto.htm Lewis, C. M., & Sycara, K. P. (1993). Reaching informed agreement in multispecialist cooperation. Group Decision and Negotiation, 2,279-299. Moran, T., & Carroll, J. (Eds.). (1996). Design rationale: concepts, techniques, and use. Mahwah, NJ: Erlbaum. Niickles, M., & Bromme, R. (2002). Internet experts' planning of explanation for laypersons: a web experimental approach in the internet domain. Experimental Psychology, 49, 292-304. Rice, J. C., Merrill, P. F., & Hawkins, C. L. (1996, October). Procedures for creating useful web sites. WebNet 96. Retrieved Feb. 16, 2002 from http://curry.edschool.virginia.edu/aace/conf/webnet~ htmlll8l .html. Rummel, N., & Spada, H. (submitted). Learning to collaborate: an instructional approach to pronzoting collaborative problern-solving in computer-mediated settings. Schwabe, D., Rossi, G., & Barbosa, S. (1996). Structured web site design. Retrieved from http://www.cs.bgsu.edu/hrweb/papers/schwabe/index.html. Schank, R. C., & Abelson, R. P. (1977). Scripts, plans, goals, and understanding. Hillsdale, NJ: Erlbaum. Shneiderman, B. (1997). Designing information-abundant websites: Issues and recornnzendations. Retrieved Nov. 30, 2002 from http:Nwww.cs.umd.edu~hciVmembers/bshneiderman/ijhcs/main.html. Simon, H. A. (1973). The structure of ill-structured problems. Artificial Intelligence, 4, 181-200. Stasser, G. (1992). Pooling of unshared information during group discussion. In S. Worchel, W. Wood & J. A. Simpson (Eds.), Group process andproductivity (p. 48-67). Newbury Park, CA: Sage. Stefik, M. (1995). Knowledge system. San Francisco: Morgan Kaufmann. Strube, G. (1996). Knowledge-based systems from a socio-cognitive perspective. Behaviour & Information Technology, 15, 276-288. VoB, A,, Bartsch-SpOrl, B., Hovestadt, L., Jantke, K. P., Peterson, U., & Strube, G. (1996). FABEL. Kiinstlicke Intelligenz, 10(3), 70-76. Wegner, D. M. (1987). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior (pp. 185-208). New York: Springer.
[email protected] thaleman @cognition.uni-freiburg.de
GERHARD FISCHER & JONATHAN OSTWALD
KNOWLEDGE COMMUNICATION IN DESIGN COMMUNITIES
Abstract. Design is a rich domain in which to investigate barriers and biases in computer-supported communication because it involves many different modes of communication in social-technical contexts. This chapter briefly describes different design approaches. It analyzes the biases and barriers of two different types of design communities: communities of practice and communities of interest. To address the communication challenges between diverse design communities, boundary objects are needed to establish common ground and shared understanding in the context of complex design tasks. We explore the unique possibilities that computational media have to support our conceptual framework. Our work is based on the fundamental belief that there is no media-independent communication and interaction-that tools, materials, and social arrangements are always involved in some way in these activities. The possibilities and the practice of design are functions of the media with which we design. We present examples of such environments from our work.
1. INTRODUCTION Design is a rich setting in which to study computer-mediated communication. Large and complex design projects cannot be accomplished by any single person, and they often cut across different established disciplines, requiring expertise in a wide range of areas (Ernesto G. Arias, Eden, Fischer, Gorman, & Scharff, 2000). Software design projects, for example, involve designers, programmers, human-computer interaction specialists, marketing people, and user participants. Design projects are unique (Rittel & Webber, 1984), and therefore each design project requires learning and produces new knowledge in the form of understanding as well as artifacts. Complexity in design arises from the need to synthesize stakeholders' different perspectives of a problem, the management of large amounts of information relevant to a design task, and understanding the design decisions that have determined the long-term evolution of a designed artifact. Successful projects must overcome many barriers to communication and shared understanding. Media change the nature of learning and communication in design. Ideally, new media will improve both individual and collaborative design by augmenting the cognitive abilities of designers and allowing them to transcend some of the barriers that have limited knowledge creation and sharing in design. This chapter characterizes design as a human activity and discusses knowledge communication in several design contexts, including individual as well as collaborative design. It focuses on design communities as key loci of collaborative design, and their respective biases and barriers for knowledge communication. Specifically, this chapter analyzes communities of practice (Cops) and communities of interest (CoIs); the latter addressing the challenges of collaborative design involving stakeholders from different practices and backgrounds by promoting
constructive interactions among multiple knowledge systems (Fischer, 2001). Our approach to coordinating the various perspectives of different communities for a shared design task relies on boundary objects to mediate knowledge communication. We present several major system developments that employ boundary objects in support of knowledge communication within design communities. The chapter concludes by discussing how some of the barriers and biases in computer-mediated communication, specifically in the context of design, can be overcome by new media that support design communities. 2. DESIGN AND DESIGN COMMUNITIES 2.1 Design
Design is a ubiquitous activity that is practiced in everyday life as well as in the workplace by professionals (Cross, 1984; Donald A. Schtin, 1983; Simon, 1996). It is not restricted to any specific discipline, such as art or architecture, but instead is a broad human activity that pursues the question of "how things ought to be", as compared to the natural sciences, which study "how things are" (Simon, 1996). It is a fundamental activity within all professions: architects and urban planners design buildings and towns, lawyers design briefs and cases, politicians design policies and programs, educators design curricula and courses, writers design novels and technical documentation, psychologists design experiments, and software engineers design computer programs. Designers solve problems. But apart from problems in school, most problems in real life are encountered, not given. For these problems, understanding the problem is the problem. Real-life problems must be framed, a process in which the important objects are determined and desired outcomes are defined. Many problem-solving methodologies assume that problems can be clearly framed a priori, before any attempt at a solution is made. Design problems are typically, however, "ill-defined", or "wicked" (Rittel & Webber, 1984), creating the following dilemma: (1) one cannot understand the problem without information about it; (2) one cannot gather information meaningfully unless the problem is understood; and (3) one cannot understand the problem without having a concept of the solution in mind. In real life, as opposed to the classroom, problems are moving targets requiring an integration of problem framing and problem solving, such that the work in progress suggests ways to proceed and the development of a solution causes the framing of the task to grow and change. Emphasizing the integration of problem framing and problem solving casts design as a search for a problem space rather than just within a problem space. It brings into question all design methodologies that are founded on a separation of analysis and synthesis. Furthermore, it emphasizes the importance of problem owners (those for whom an artifact is designed) as stakeholders in the design process because they have the authority and the knowledge to reframe the problem as the problem space becomes better understood. Our research in design integrates the task of problem framing with that of problem solving by stressing the importance of externalizations that enable
designers to represent both tasks. In this sense, externalizing ideas is not a matter of emptying out the mind but of actively reconstructing it, forming new associations, and expressing concepts in external representations while lessening the cognitive load required for remembering them: "Externalization produces a record of our mental efforts, one that is 'outside us' rather than vaguely 'in memory'. ... It relieves us in some measure from the always difficult task of 'thinking about our own thoughts' while often accomplishing the same end. It embodies our thoughts and intentions in a form more accessible to reflective efforts." ((Bruner, 1996), p. 23)
Designers engage in a cyclic process of action (the creation or modification of an externalization) and reflection (Donald A. Schbn, 1983). Action is governed by nonreflective thought processes and proceeds until it breaks down. A breakdown occurs when the designer realizes that an action has resulted in unanticipated consequences. Designers engage in a conversation with their materials by listening to the "back-talk of the situation". In collaborative design, Schiin's metaphor of "conversation with the situation" takes on new meaning. The design situation now includes other designers as well as external representations, and conversation occurs between designers as well as between designers and design representations. The external design situation serves as context for communication between designers as well as between individual designers and the design situation. A common notion about interpersonal communication is that knowledge is transmitted from one person to another. This assumption seems to hold in unproblematic communication, such as that between people who share a common background. But when we think of the difficulties in communication between people with different backgrounds, or in communicating a complex or vague idea, it is evident that "The phenomenon of communication depends on not what is transmitted, but on what happens to the person who receives it. And this is a very different matter than transmitting information" (Maturana & Varela, 1987).
2.1.1 Differentiating Design Approaches Design processes involve stakeholders (often coming from different disciplines) who create artifacts. For many design activities, one can distinguish between design time (when the artifact is being designed) and use time (when the artifact is being used). At design time, a major challenge is to imagine how users will experience artifacts, whereas at use time the users are actually experiencing the artifacts.In professionally dominated design, professional designers (such as architects, software developers, urban planners, and teachers) engage in design methodologies founded on the belief that they understand the users' needs. At design time, they create artifacts with which users "have to live" at use time. In professionally dominated design, the "experts" see the creation of artifacts as their primary tasks (e.g., architects build buildings, software developers create software systems, urban planners design cities, and teachers develop courses); and understanding and communicating with other stakeholders are seen as secondary tasks representing
extra work and thereby taking resources away from the primary task (Rambow & Bromme, 2000). Participatory design approaches (Fischer & Ostwald, 2002a; Schuler & Namioka, 1993) seek to involve users more deeply in the process as co-designers by empowering them to propose and generate design alternatives themselves. Participatory design supports diverse ways of thinking, planning, and acting, thus making work, technologies, and social institutions more responsive to human needs. It requires the social inclusion and active participation of the users. It is a response to the theoretical argument that design problems are ill-defined and wicked and therefore cannot be delegated to experts. Instead, all stakeholders who are owners of problems must have a voice in the design process and they must participate in the framing of the problem. Communication processes between designers and clients in participatory design face two barriers: (1) clients may not know exactly what they want; and (2) stakeholders lack a common language that allows them to educate each other, propose new visions, understand and critique these proposals, and come to a shared understanding of how things should be. Developers are often biased toward working in their own language and formalisms, which is a barrier for users, who are forced to express their knowledge in the developer's vocabulary. Communication breakdowns occur when developers and users do not have a shared context. The challenge for communication is to establish a shared context that allows for communication and the accumulation of shared understanding. Despite the best efforts at design time, designed artifacts need to be evolvable at use time to fit new needs, account for changing tasks, and incorporate new technologies. However, design approaches (whether done for users, by users, or with users) have traditionally focused primarily on activities and processes taking place at design time and have given little emphasis and provided few mechanisms to support systems as living entities that can be evolved by their users (see Table 2). Meta-design approaches (Fischer & Scharff, 2000; Giaccardi, 2003) characterize objectives, techniques, and processes for creating new media and environments that allow the owners of problems to act as designers (Fischer, 2002). A fundamental objective of meta-design is to create socio-technical environments that empower users to engage in creating knowledge rather than being restricted to the consumption of existing knowledge. Meta-design extends the traditional notion of system design beyond the original development of a system to include an ongoing process in which stakeholders become co-designers-not only at design time, but throughout the whole existence of the system (Morch, 1997; Rummel & Spada, 2004). A necessary, although not sufficient, condition for users to become co-designers is that software systems include advanced features that permit users to create complex customizations and extensions. Rather than presenting users with closed systems, meta-design approaches provide them with opportunities, tools, and social reward structures to extend the system to fit their needs. Meta-design shares some important objectives with user-centered and participatory design, but it transcends these objectives in several important dimensions and it changes the processes by which systems and
content are designed. Meta-design shifts control over the design process from designers to users and empowers users to create and contribute their own visions and objectives at use time as well as at design time. Meta-design is a useful perspective for projects for which 'designing the design process' is a first-class activity, meaning that creating the technical and social conditions for broad participation in design activities (in both design time and use time) is as important as creating the artifact itself (Wright, Marlino, & Sumner, 2002). Table 2 summarizes the role of the user in professionally dominated, participatory, and meta-design approaches. Only meta-design views the users as active participants and designers throughout the lifecycle of a designed artifact.
+
Table 2: The Role of Users in Different Design Approaches
Design Approach
Design Time
Use Time
Professionally-
Users have no voice
Users have to live with artifacts designed by others Users are consumers of artifacts designed with their input, but artifacts cannot be evolved to serve unforeseen needs Users can act as designers and evolve the artifact tofit new needs
Users are active participants; systems are designed as complete systems artifacts
I Users are active participants; systems are designed as seeds; design is focused on design for participation, as well as use
I
2.1.2 Dimensions of Computer-Mediated Communication in Design computer-mediated comhunication (specifically in complex design activities) can be differentiated along the following dimensions:
-
-
spatial (across distance), requiring networks (B. Nardi & Whittaker, 2002; Olson & Olson, 2001); temporal (across time), requiring support for asynchronous, indirect, longterm communication (Fischer et al., 1992; Mprrch & Mehandjiev, 2000); technological (between persons and artifacts), requiring knowledge-based, domain-oriented systems (Fischer, 1994; Terveen, 1995); and social (across different communities of practice), requiring support for common ground and shared understanding (Fischer, 2001; Resnick, Levine, & Teasley, 1991).
Many research efforts in computer-mediated communication have focused on collaborative activities across time and space, in which media support is not a luxury but a necessity. Spatial Dimension. Even though communication technology enables profoundly new forms of collaborative work, Olson and Olson (Olson & Olson, 2001) have
found that closely coupled work can still be difficult to support at a distance. In addition, critical stages of collaborative work, such as establishing mutual trust, appear to require some level of face-to-face interaction. Brown and Duguid (John Seely Brown & Duguid, 2000) present a similar argument: "Digital technologies are adept at maintaining communities already formed. They are less good at making them" (p. 226)."
In contrast, distributed teams of collaborators are able to carry out effective work, and indeed evolve totally new ways of working that have a great impact on their activities (Olson & Olson, 2001). Open source software communities provide an example of successful collaboration on a large scale mediated by computational media (Raymond & Young, 2001; Scharff, 2002). Temporal Dimension. Design processes often take place over many years, with initial design followed by extended periods of evolution and redesign. In this sense, design artifacts (including systems that support design tasks, such as reuse environments (Ye & Fischer, 2002)) are not designed once and for all, but instead they evolve over long periods of time. For example, most computer networks are enhanced and updated, rather than redesigned completely from scratch, when a new device or technology emerges. Much of the work in ongoing design projects is done as redesign and evolution, and the people doing this work are often not members of the original design team. But to be able to do this work well, or sometimes at all, requires "collaboration" with the original designers of the artifact. A special case of this collaboration is reflexive computer-supported cooperative work (CSCW) supporting the same individual user, who can be considered as two different persona at points of time that are far apart (Thimbleby, Anderson, & Witten, 1990). In ongoing projects, longterm collaboration is crucial for success yet difficult to achieve. This difficulty is due in large part to individual designers' ignorance of how the decisions they make interact with decisions made by other designers. A large part of this, in turn, consists of simply not knowing what has been decided and why. Long-term collaboration requires that present-day designers be aware of the rationale (Moran & Carroll, 1996) behind decisions that shaped the artifact, and aware of information about possible alternatives that were considered but not implemented. This requires that the rationale behind decisions be recorded in the first place. Closed systems thus present a barrier to rationale capture by not providing opportunities for designers to add rationale for their decisions. As argued before, designers are biased toward doing design but not toward putting extra effort into documentation. This creates an additional rationale-capture barrier for longterm design. Another barrier raised by long-term design projects is the ability to modify a system's functionality. During the lifecycle of a ongoing design project, the environment in which the artifact functions may change in ways that were not anticipated by the original designers. If the system cannot be adapted to its changing
environment at use time, it will cease to be useful. One way to view this need for adaptation is to think of the lifecycle of a system as an ongoing design process, sometimes called design-in-use to emphasize that design of a system happens alongside use (Henderson & Kyng, 1991).
Technological Dimension. Design can be described as a reflective conversation between designers and the designs they create. Designers use materials to construct design situations, and then listen to the "back-talk of the situation" they have created (Donald A. SchBn, 1983). Unlike passive design materials, such as pen and paper, computational design materials are able to interpret the work of designers and actively talk back to designers. For example, critiquing mechanisms embedded in domain-oriented design environments can alert designers when they violate design principles and then deliver relevant information to help designers understand how to improve their designs. Social Dimension. Design communities are increasingly characterized by a division of labor, comprising individuals who have unique experiences, varying interests, and different perspectives about problems, and who use different knowledge systems in their work. Shared understanding (Resnick et al., 1991) supporting collaborative learning and working requires the active construction of a knowledge system in which the meanings of concepts and objects can be debated and resolved. In heterogeneous design communities, such as those that form around large and complex design problems, the construction of shared understanding requires an interaction and synthesis of several separate knowledge systems. Our own research efforts have focused on supporting communication across two conceptual dimensions: (1) the expertise gap between experts and novices within a particular practice; and (2) the conceptual gap between stakeholders from different practices. In the following section, we analyze these dimensions of communication in the context of design communities. 2.2. Design Communities Design communities are social structures that enable groups of people to share knowledge and resources in support of collaborative design. Different communities grow around different types of design practice. Each design community is unique, but for the purposes of this discussion, we identify two stereotypical kinds of design community-the community of practice (COP) and the community of interest (Co1)-and discuss their respective barriers and biases for knowledge creation and sharing in collaborative design. 2.2.1 Communities of Practice Cops (Wenger, 1998) consist of practitioners who work as a community in a certain domain undertaking similar work. For example, copier repair personnel who work primarily in the field but meet regularly to share "war stories" about how to solve
the problems they encountered in their work make up a COP (Orr, 1996). Learning within a COP takes the form of legitimate peripheral participation (LPP) (Lave & Wenger, 1991), which is a type of apprenticeship model in which newcomers enter the community from the periphery and move toward the center as they become more and more knowledgeable (depicted in Figure 4). Sustained engagement and collaboration lead to boundaries that are based on shared histories of learning and that create discontinuities between participants and non-participants. Highly developed knowledge systems (including conceptual frameworks, technical systems, and human organizations) are biased toward efficient communication within the community at the expense of acting as barriers to communication with outsiders: boundaries that are empowering to the insider are often barriers to outsiders and newcomers to the group.
Figure 4. Learning in COPS At the center are knowledgeable members and knowledge systems. Members enter the community from the periphery and move toward the center over time through participating in the community.
A community of practice has many possible paths and many roles (identities) within it (e.g., leader, scribe, power-user, visionary, and so forth) (Ye & Kishida, 2003). Over time, most members move toward the center, and their knowledge becomes part of the foundation of the community's shared background. 2.2.2 Communities of Interest "Innovations come from outside the city wall."
CoIs bring together stakeholders from different COPSand are defined by their collective concern with the resolution of a particular problem. CoIs can be thought of as "communities of communities" (John S. Brown & Duguid, 1991) or a community of representatives of communities. Examples of CoIs are: (1) a team
interested in software development that includes software designers, users, marketing specialists, psychologists, and programmers, or (2) a group of citizens and experts interested in urban planning, especially with regard to implementing new transportation systems, as illustrated later in this chapter by the Envisionment and Discovery Collaboratory (EDC).
Figure 5: Cols - Bringing Different COPSTogether Cols bring together stakeholders from different COPS(represented by the different colored circles). The ragged edge of the bounding shape depicts that the boundaries of the problem and the community are not well established, particularly at the beginning of a project.
Stakeholders within CoIs are considered as informed participants (J.S. Brown, Duguid, & Haviland, 1994) who are neither experts nor novices, but rather both: they are experts when they communicate their knowledge to others, and they are novices when they learn from others who are experts in areas outside their own knowledge. As a model for working and learning in CoIs, informed participation (Fischer & Ostwald, 2002b) is based on the claim that for many (design) problems, the knowledge to understand, frame, and solve these problems does not already exist, but must be collaboratively constructed and evolved during the problem-solving process. Informed participation requires information, but mere access to information is not enough. The participants must go beyond the information that exists to solve their problems. For informed participation, the primary role of media is not to deliver predigested information to individuals, but to provide the opportunity and resources for social debate and discussion. In this sense, improving access to existing information (often seen as the major advance of new media) is a limiting aspiration. A more profound challenge is to allow stakeholders to incrementally acquire ownership in problems and contribute actively to their solutions (Florida, 2002). Communication among stakeholders is difficult because they come from different COPS, and therefore use different languages, different conceptual knowledge systems, and perhaps even different notational systems. In his book, "The T w o Cultures" (Snow, 1993), C. P. Snow describes these difficulties through an analysis of the interaction between literary intellectuals and natural scientists, who (as he had observed) had almost ceased to communicate at all. He writes,
"there exists a profound mutual suspicion and incomprehension, which in turn has damaging consequences for the prospects of applying technology to the alleviation of the world's problems" and "there seems to be no place where the cultures can meet."
The fundamental barrier facing CoIs is that knowledge distribution is based on an asymmetry of ignorance (or knowledge) (Rittel, 1984), in which each stakeholder possesses some, but not all, relevant knowledge, and the knowledge of one participant complements the ignorance of another. This barrier must be overcome by building a shared understanding of the task at hand, which often does not exist at the beginning, but is evolved incrementally and collaboratively and emerges in people's minds and in external artifacts. Members of CoIs must learn to communicate with and learn from others (Engestrijm, 2001) who have different perspectives and perhaps a different vocabulary for describing their ideas. In other words, this symmetry of ignorance must be exploited.
2.2.3 Comparing COPSand Cols Learning through informed participation within CoIs is more complex and multifaceted than legitimate peripheral participation (Lave & Wenger, 1991) in CoPs. Learning in Cops can be characterized as "learning within a single knowledge system", whereas learning in CoIs is often a consequence of the fact that there are multiple knowledge systems. CoIs have multiple centers of knowledge, with each member considered to be knowledgeable in a particular aspect of the problem and perhaps not so knowledgeable in others. In informed participation, the roles of "expert" or "novice" shift from person to person, depending on the current focus of attention. Table 2 characterizes and differentiates CoPs and CoIs along a number of dimensions. The point of comparing and contrasting CoPs and CoIs is not to pigeonhole groups into either category, but rather to identify patterns of practice and helpful technologies. People can participate in more than one community, or one community can exhibit attributes of both a CoI and a COP. Our Center for LifeLong Learning and Design ( L ~ D is ) an example: It has many characteristics of a COP (having developed its own stories, terminology, and artifacts), but by actively engaging with people from outside our community (e.g., other colleges on campus, people from industry, international visitors, and so forth), it also has many characteristics of a CoI. Design communities do not have to be strictly either Cops or CoIs, but they can integrate aspects of both forms of communities. The community type may shift over time, according to events outside the community, the objectives of its members, and the structure of the membership.
"
Table 3: Differentiating COPSand CoIs
Dimensions Nature of roblems Knowledge
I CoPs
I Different tasks in the same
I domain
I Refinement of one knowledge system; new ideas coming from
I Common task across multiple domains Synthesis and mutual learning I through the integration of multiple knowledge systems Shared understanding, making all voices heard I Lack of a shared understanding Social creativity; diversity; making all voices heard Stakeholders (owners of problems) from different
I Major objectives
coverage Group-think Shared ontologies
I People
Beginners and experts; apprentices and masters
Learning
Legitimate peripheral participation
Informed participation
Both forms of design communities exhibit barriers and biases. COPS are biased toward communicating with the same people and taking advantage of a shared background. The existence of an accepted, well-established center (of expertise) and a clear path of learning toward this center allows the differentiation of members into novices, intermediates, and experts (see Figure 4). It makes these attributes viable concepts associated with people and provides the foundation for legitimate peripheral participation as a workable learning strategy. The barriers imposed by Cops are that group-think can suppress exposure to, and acceptance of, outside ideas; the more someone is at home in a COP, the more that person forgets the strange and contingent nature of its categories from the outside. A bias of Cols is their potential for creativity because different backgrounds and different perspectives can lead to new insights (Bennis & Biederman, 1997; Campbell, 1969). CoIs have great potential to be more innovative and more transforming than a single COP if they can exploit the asymmetry of ignorance (Rittel, 1984) as a source of collective creativity. A fundamental barrier for CoIs might be that the participants failed to create common ground and shared understanding. This barrier is particularly challenging because CoIs often are more temporary than CoPs: They come together in the context of a specific project and dissolve after the project has ended. CoPs are the focus of approaches such as CSCW: They provide support for work cultures with a shared practice (Wenger, 1998). The lack of a shared practice in CoIs requires them to draw together diverse cultural perspectives. Computer-mediated knowledge communication in Cops is different from that in CoIs. CoIs pose a number of new challenges, but the payoff is promising because they can support pluralistic societies that can cope with complexity, contradictions, and a willingness to allow for differences in opinions.
2.3 Boundary Objects Boundary objects (Bowker & Star, 2000; Star, 1989; Wenger, 1998) are
externalizations of ideas that are used to communicate and facilitate shared understandings across spatial, temporal, conceptual, or technological gaps. In design communities, boundary objects help to establish a shared context for communication by providing referential anchoring (Clark & Brennan, 1991). Boundary objects can be pointed to and named, helping stakeholders make sure they are talking about the same thing. Grounding communication with external representations helps to identify breakdowns and serves as a resource for repairing them. In CoPs, boundary objects represent the domain concepts and ontologies that both define and reflect the shared practice. They might take the form of documents, terminology, stories, rules, and unspoken norms. For example, the boundary objects in our community of researchers include research papers, dissertations, and a conceptual framework that encompasses the individuals and work done within the community. In CoIs, boundary objects support communication across the boundaries of different knowledge systems, helping people from different backgrounds and perspectives to communicate and to build common ground (see Figure 6).
Boundary
Objects
Figure 6: Boundary Objects as Bridges between COPS Boundary objects should be meaningful within the conceptual knowledge systems of at least two communities of practice. The meaning need not be the same-in fact, the differences in meaning are what lead to the creation of new knowledge.
Boundary objects allow different knowledge systems to communicate by providing a shared reference that is meaningful within both systems. Computational support for CoIs must therefore enable mutual learning through the creation, discussion, and refinement of boundary objects that allow the knowledge systems of different CoPs to interact. In this sense, the interaction between multiple knowledge systems is a means to turn the asymmetry of ignorance (Rittel, 1984) into a resource for learning and social creativity (Fischer, 2000). Boundaries are the locus of the production of new knowledge. They are where the unexpected can be expected,
where innovative and unorthodox solutions are found, where serendipity is likely, and where old ideas find new life. The diversity of CoIs may cause difficulties, but it also may provide unique opportunities for knowledge creation and sharing (Ernesto G. Arias et al., 2000). Importantly, boundary objects should be conceptualized as evolving artifacts that become understandable and meaningful as they are used, discussed, and refined (Ostwald, 1996). For this reason, boundary objects should be conceptualized as reminders that trigger knowledge, or as conversation pieces that ground shared understanding, rather than as containers of knowledge. The interaction around a boundary object is what creates and communicates knowledge, not the object itself. Humans serving as knowledge brokers can play important roles to bridge boundaries that exist across or within communities. For example, within design communities that develop around complex software systems, members who are interested and inclined to learn about the technologies may develop into power-users (also known as "local developers" and "gardeners" (B. A. Nardi, 1993)) who are able to make modifications and customizations. By making needed changes to a system on behalf of the community, or by teaching others how to do so, power-users help others to transcend the boundary that exists between using a system as it is and modifying it.
3. MEDIA IN SUPPORT OF KNOWLEDGE COMMUNICATION There is no media-independent communication and interaction: tools, materials, and social arrangements always mediate activity. The possibilities and the practice of design are functions of the media with which we design. We explore here the unique possibilities that computational media can have on design. Cognition is shared not only among minds, but also among minds and the structured media within which minds interact (Resnick et al., 1991; Salomon, 1993). In this section, we briefly differentiate among various kinds of media and then provide examples of the sociotechnical environments that we have developed to support design in different design communities.
3.1 Rich and Lean Media "You cannot use smoke signals to do philosophy. Its form excludes the content". ((Postman, 1985), p. 7)
Our research is grounded in the basic belief that computer-mediated communication is not the opposite of face-to-face communication, but that face-toface communication can be effectively supported with computational media. We distinguish between rich media and lean media (see Figure 7). Rich media exploit all communication channels (face-to-face interactions being a prime example). They are highly interactive and highly malleable, and provide rich knowledge structures (including boundary objects), but they are often very costly to realize. We are in search of defining media ecologies (B. Nardi & Whittaker, 2002) that support matching appropriate and effective media to specific tasks. Although cost-effective
solutions are important, our primary interest is in maximizing the creativity of all stakeholders in design (National-Research-Council, 2003; Shneiderman, 2002). task ill-defined
well-defined lean
rich
media
Figure 7: Matching Media to Tasks
Our basic assumption is that complex tasks (such as creating an initial understanding of a new public transportation system in urban planning) require rich media, whereas for well-specified tasks (such as finding out how far different people are willing to walk to a bus stop or how long they are willing to wait for a bus), lean media are sufficient. Inappropriate uses of media occur when lean media are used to address complex tasks, and ineffective uses occur when rich media are used to deal with well-specified problems (see Figure 7). In the course of solving complex design problems, d~fferentphases occur (see Figure 8): -
-
-
what: deciding what the problem is by framing it; how: determining how the problem can and should be decomposed; subtasks: doing the work on the subtasks from decomposing the original problem; restructurirzg: restructuring the pieces and reassembling them into a whole; and reframirzg: taking stake, evaluating an initial solution, and reframing the problem.
rich (o.g., faco.wfacc
lean (a& omnil)
time
Figure 8: Covering a Wide-Spectrum of Activities with Media Integration Computer-mediated interaction and collaboration should use many different kinds of media to fit the specific tasks to be achieved.
Rich media (such as face-to-face collaborations) enable us to leverage our native modes of communication. This is particularly important at the boundaries of domain knowledge, where problems are ill-defined, only partially understood, and difficult, if not impossible, to express explicitly (Rummel & Spada, 2004). In such situations, the full range of communication capabilities and facilities is required to aid different people's natural abilities to construct shared understanding through detection and repair of communication breakdowns. Once problems are more fully understood, they can be more meaningfully expressed by using lean media because they can be explicitly described. The creation and evolution of boundary objects requires rich media, but once the evolution is over, these objects can be represented and referred to by using leaner media. An advantage of lean media could also be that they force designers to abstract, decontextualize, and avoid a premature focus on being too explicit about low-level details (e.g., architects prefer in early stages of design "fat pencil" technologies rather than tools like Autocad that impose a counterproductive level of detail) (Dillenbourg, 2004). Solving complex design problems requires that stakeholders engage in all of these activities. Therefore integrated socio-technical environments (as described in the following sections) should offer a variety of media to support the whole spectrum of different activities. They should support COPSand CoIs by allowing them to think previously unthinkable thoughts, to do previously undoable actions, and to explore previously unfeasible questions. 3.2 Computer-Mediated Communication in COPS Domain-oriented design environments (DODEs) (Fischer, 1994) are a class of integrated systems that support design in a particular domain by a COP. They explicitly represent the domain-specific knowledge structures developed by the COP, including abstractions, domain models, tools, design methodologies, and so forth;
they embody the Cops intellectual history; and they have theories and basic assumptions built into them. Moreover, users accept the built-in theories and assumptions when when they use these tools. DODEs support COPS by providing cognitive economy to a particular professional community, but they are of little or no use outside of this community.
Figure 9: A DODE for Computer Network Design
Figure 9 shows a screen image of a DODE supporting a COP of computer network designers who create local area networks in the co~zstructionspace (see Pane 2 ) by using a palette of components (see Pane 3). A specification component (see Pane 4 ) allows the designers to articulate high-level intentions for their projects, such as ranking of priorities, that are not explicit in the worksheet. The DODE contains a group memory (see Pane 1 ) (Lindstaedt, 1998) that holds information collected from previous projects, email communication archives, and other textual information. The catalog (see Pane 5 ) contains example networks that can be used to see how a similar problem was solved, to understand the evolution of the particular network being designed, or as a starting point for a new design, thereby supporting case-based reasoning approaches (Kolodner, 1993). DODEs help users to be reflective practitioners (Donald A. Schon, 1983) by providing support for "reflection-in-action." The action space (i.e., construction workspace) is linked with the reflection space (i.e., group memory and catalog)
through critics, which are codified domain knowledge in the form of design rules. Embedded critics (Fischer, Nakakoji, Ostwald, Stahl, & Sumner, 1998) enable DODEs to (1) increase the "back-talk" of a design situation by monitoring the actions of users as they work and informing them about breakdowns; (2) increase the user's understanding of problems to be solved; (3) point out the need for information that might have been overlooked; and (4) locate relevant information in very large information spaces. Embedded critics use partial constructions and partial specifications as implicit queries over information spaces. This enables the system to automatically find relevant information, rather than requiring the user to explicitly search for it. A primary focus of DODEs was to support the technological dimension of computer-mediated communication by bridging the communication gap between a computational environment and an individual user, who may be a domain expert but not typically a computer expert. Domain-oriented objects, embedded critics, and rich information spaces allow users to communicate with the design situation and the problem domain, rather than with the computer per se, thereby supporting human problem-domain interaction in addition to human-computer interaction. In this sense, the interactive objects and mechanisms provided by DODEs are boundary objects that mediate communication between the user and the domainoriented knowledge contained in the system. DODEs support meta-design by providing end-user modifiable components (Girgensohn, 1992) that enable these boundary objects to be changed and modified at use time. The result of this approach is that the boundary objects can evolve and acquire new meanings as users discover new design rules, or new applicability conditions for existing rules, in the course of solving design problems. DODEs support the temporal dimension of computer-mediated communication by enabling users to communicate indirectly through artifacts and group memories. This indirect channel of communication has several advantages within CoPs, including the fact that interactions can be captured and associated with the artifacts to which they refer. We found (Reeves, 1993), however, that this was not a sufficiently rich channel to support the kind of communication and learning required by CoIs, especially in the early stages of framing a problem (see Figure 8).
3.3 Computer-Mediated Communication in Cols Communication in CoIs requires boundary objects to address the unique challenges of allowing people from different CoPs to establish common ground and mutual understanding. This section describes two different contexts in which boundary objects are used to mediate conceptual gaps among stakeholders in CoIs. The first context is informed participation in support of collaborative decision making. The second context is informed participation in software development.
3.3.1 Collaborative Decision Making: The Envisionment and Discovery Collaboratory The Envisionment and Discovery Collaboratory (EDC) (Ernesto G. Arias et al., 2000) attempts to maximize the richness of communication between stakeholders in face-to-face interaction, mediated by both physical and computational objects. The EDC supports CoIs by empowering all stakeholders to (1) engage in informed participation, (2) create shared understanding, (3) contextualize information to the task at hand, and (4) create boundary objects in collaborative design activities. Whereas, DODEs primarily support COPS within a specific domain, the EDC supports CoIs by providing boundary objects that all stakeholders can understand and manipulate, as well as by providing underlying computational support for trying out alternative solutions, accessing information relevant to the task at hand, and capturing information and design rationale from the design process. The EDC approach is based on our experience in building and using physical simulation games to support decision making and critical thinking in the participatory design of physical environments (E.G. Arias, 1995). In a simulation game for urban neighborhood planning, game pieces representing structures, such as houses or commercial buildings, are placed on a game board representing the streets and lots. The game pieces and their placement on the board allow neighborhood residents to create and evaluate possibilities for changing their environment. The game pieces form a language for the stakeholders to use as they explore areas of conflict and consensus in planning the neighborhood. The language of the game pieces is a vehicle for interactions between players, including neighbors and design professionals. Design games aim to integrate design and communication. The objects used to express the design situation are also the means for communication. The goal of the language is to support shared understanding and critical thinking - not to get in the way by introducing unneeded complexity. The language of the game pieces should "integrate the requirements of relevancy, flexibility, transparency, and above all, simplicity" (E.G. Arias, 1995). The EDC extends the physical simulation game approach by integrating computational environments and (computationally enriched) external physical worlds with mechanisms capturing the larger (often unarticulated) context of what users are doing. Like DODEs, the EDC is grounded in Schon's "reflection-inaction" problem-solving approach (Donald A. Schon, 1983). Stakeholders using the EDC convene around a computationally enhanced table that serves as the action space. Currently realized as a touch-sensitive surface, the action space allows users to manipulate a computational simulation projected on the surface by interacting with the physical objects placed on the table. The simulation is an interactive model of the design problem that allows users to propose and explore alternative solutions in a complex design space. The table is flanked by another touch-sensitive (vertical) surface that serves as the reflection space. The reflection space displays information that is relevant to the context as defined by the simulation in the action space.
Figure 10: The Envisionment and Discovery Collaboratory In the action space (foreground), stakeholders use physical objects to interact with an underlying computational simulation environment. In the reflection space (background), stakeholders interact with an information space, in which they access information, fill out surveys, and add new information. The EDC framework is applicable to different domains; our initial effort has focused on the domains of urban planning and decision making, specifically transportation planning and community development. In Figure 10, neighbors are filling out a Web-based transportation survey associated with the simulation being constructed. Boundary Objects in the EDC. In the EDC, a design functions as a communication artifact around which stakeholders from different Cops, coming together as a CoI in the context of a specific problem, can negotiate their contributions, their positions, and their alignments. Action space objects are domain-oriented - they represent objects in the problem domain in terms of both visual appearance and behavior within the simulations. These objects and their behaviors are meaningful to all stakeholders who have familiarity with the problem domain. However, the precise meanings of the objects and the implications of these meanings for design decisions for each stakeholder may not be shared initially among them. The objects serve as boundary objects by providing a common starting ground for stakeholders to identify and explore the differences in their understandings and to build new understandings that bridge the boundaries.
For example, in the domain of transportation planning, stakeholders include transportation engineers and neighborhood residents who will work together to improve the design of bus routes in their neighborhood. In the action space, they use domain objects, such as buses, bus stops, neighborhoods, and streets to explore different facets of the problem. An engineer might think of a bus stop in terms of its capacity to serve a certain size of neighborhood, whereas a resident might think of a bus-stop in terms of its convenience to his house, or maybe in terms of its afterdark safety. The bus stop object in the EDC is a boundary object for engineers and residents to build a shared understanding of the "bus-stop" concept in terms of the importance and implications for the particular design. This process is enhanced by the action space simulation, which helps stakeholders to explore alternatives, and the reflection space, which provides background that informs each perspective. Human-Computer Interaction Support for Boundary Objects in the EDC. In the original version of the EDC, the game board was biased toward single-user interaction due to limitations in the underlying SmartBoard technology. This bias resulted in the following barrier: parallel interactions, which were often attempted by users unfamiliar with this restriction, resulted in unpredictable effects. The single-user limitation of the SmartBoard could not simply be "programmed around" because the device accepted simultaneous presses as a normal single input occurring halfway between the two presses. This limitation for acting in parallel combined with the existence of only a single cursor led to frequent "mode" errors (for example, a user might attempt to delete an object when the "add mode" was active). The limitation imposed by a single cursor required that an explicit association be made between the physical cursor and the current virtual object of interest. In addition, users had to take an explicit action to associate a physical object with the underlying simulation by firmly pressing the object onto the touch screen rather than just placing the object at the desired location. We observed that users coming from COPSwith little experience or interest in computers per se frequently failed to make this association, which resulted in an operation other than that intended being erroneously applied to an object. Taken together, these limitations required users to have an abstract mental model of how the SmartBoard technology works, in addition to a model of how the object being manipulated behaves. Although experienced users acquire an understanding of the SmartBoard interaction model as they worked with the system, participants who had limited exposure to the system may have experienced confusion that significantly degraded their engagement with the system. Such situations are a barrier for collaborative design because they (1) break the built-up context of a partial solution, (2) force stakeholders to focus on the interface rather than on the problem, and (3) reduce the emergence of boundary objects that all stakeholders can deal with in a natural way. To remove these barriers in the SmartBoard technology, we are currently developing a new game board technology called the Participate-in-the-Action Board (PitA-Board) (Eden, 2003) that allows multiple users to interact with the virtual environment directly and simultaneously, leading to more engaging forms of
interaction (see Figure 11). Because the interface objects will behave more like the domain objects they represent, their potential to serve as boundary objects is greatly enhanced.
Figure 11: Parallel Interactions in the PitA-Board EDC The PitA-Board EDC eliminates barriers of the SmartBoard EDC and facilitates the creation and evolution of boundary objects.
3.3.2 SofhYare Development: The Evolving Artifact Project The Evolving Artifact (EVA) project (Ostwald, 1996) explores the use of boundary objects in support of participatory software development (Schuler & Namioka, 1993). The EVA approach is based on the claim that the knowledge required to design software systems cannot be acquired simply through interviews, observations, and other types of analysis. Instead, it must be constructed in an evolutionary and participatory manner, driven by the creation and refinement of boundary objects that mediate knowledge communication between users and developers (Ehn, 1988). In the EVA approach, boundary objects serve as a bridge between stakeholders and their respective knowledge systems. The development process is driven by the creation, discussion, and refinement of boundary objects, which are collected in a single information space (EVA) that reflects the shared understanding constructed by the stakeholders. The approach was applied to design a new system to support service representatives at a regional telephone company. The EVA project focused on knowledge communication between stakeholders from two COPS (Greenbaum & Kyng, 1991): -
developers, who prefer to think and communicate in terms of programming
languages and object-oriented design, and
-
service representatives, who conceptualize their practice in terms of "screens" and "orders" and have difficulty verbalizing some of their practice and skills because this knowledge is tacit and therefore difficult to put into words.
The EVA project explored the use of several types of boundary objects for communicating design knowledge, including rich pictures, scenarios, and prototypes (Ehn, 1988). Rich pictures were created mainly to express ideas about the existing service-provisioning domain, scenarios were used primarily to express ideas of what new computational tools could change the way service representatives worked, and prototypes expressed how these changes could take place.
Customer
Training
Order Rep
I
@ ---. ce ---.---, . N e w product flyers
p&i
Paper n o t e s o n desk
---. Handbooks
Figure 12: Rich Picture used in the EVA Project This rich picture (from (Ostwald, 1996))depicts the many information sources, including databases and paper-based documentation, with which a representative must interact while simultaneously speaking with a customer by telephone. This boundary object was created by developers early in the project tofacilitate design communication about how the information sources should be integrated in a new system. Rich pictures are a powerful type of boundary object that combine text and graphics (see Figure 12). Rich pictures do not have a formal syntax, but they do make use of symbols and diagrammatic conventions to represent a particular situation in a manner that is explicit and understandable by all stakeholders. Rich pictures give users the opportunity to identify important aspects of their work, and to correct missing elements and incorrect terminology. Additionally, rich pictures serve to identify well-defined aspects of the current domain, and to understand these
aspects in domain practices. Rich pictures thus help stakeholders to build a shared understanding of the current domain. Scenarios were used in the EVA project to build a shared vision of how a new system might change the practice of users. Task-based scenarios were built as an evolutionary step in the development process by using terminology and concepts made explicit in rich pictures. Because they were task-based, scenarios allowed users to think about what they would like to do with a new system, rather than to articulate system requirements in an abstract context. Scenarios proved to be a powerful type of boundary object because they combine aspects of both reflective and experiential artifacts (Norman, 1988). Scenarios are reflective in that they involve an explicit context, and experiential in that they allow users to imagine or act out an activity. The emphasis of traditional scenario approaches is to help system builders understand the user's requirements. In these approaches, scenarios help to uncover hidden implications and ambiguities in the requirements document. In EVA, scenarios are also used to provide a context for prototypes, which are concrete, interactive representations that constrain what a user can do (unlike scenarios, which provide a more flexible context for improvisation). In this use, scenarios might specify what tasks will be performed, and the prototype determines how the tasks can be performed. As boundary objects, prototypes were used in a manner similar to the rich pictures: to express what developers knew and what they did not know, and to provide an opportunity for users to articulate their knowledge. When developers lacked the domain knowledge to implement a particular piece of functionality, they would implement their "best guess" at what the functionality should be, and ask the users to critique the prototype. Prototypes that are grounded in the tradition of the domain bring the users' skill and practical knowledge to bear. Users are experts in their traditions, even though this type of knowledge may be "literally indescribable in linguistic terms" (Ehn, 1988). Prototypes are essential boundary objects because they let users directly experience possible new ways of working, thus going beyond scenarios in allowing users to envision the future. Boundary objects in EVA followed an evolutionary trajectory from rich picture to scenario to prototype. The shared understanding gained from discussion and interaction with each object was used to guide the next step in the trajectory. Because all the boundary objects, as well as the conversations around them, were collected in the evolving artifact, the final product of the development process contained a history of the design process, including design rationale for the decisions made along the way. In this way, EVA bridges the temporal gap to support indirect communication between stakeholders at design time and those seeking to modify the system at use time. The use of boundary objects in EVA can be contrasted with traditional software development approaches (based on waterfall models (Rittel, 1984)) that are based on the idea of transformation from one representation to another. The representations in these transformation-based approaches must be complete because ambiguities in one representation are carried over into the next one through the transformation process. In EVA, representations need not be complete. Instead, ambiguities in boundary objects are considered as opportunities for activating knowledge and creating new
understandings, serving as the driving force for communication and the construction of shared understanding. 4. LESSONS LEARNED
4.1 Barriers and Biases Our research over the last decade has developed conceptual frameworks and sociotechnical environments to support design and design communities. This research was driven forward by analyzing the barriers and biases inherent in specific approaches; subsequent approaches were aimed at overcoming the limitations and shortcomings of earlier approaches. Table 4 provides an overview of the biases and barriers discussed in this chapter. Table 4: Overview of Barriers and Biases
I Biases Design Approaches I Professionally dominated design Participatory design Meta-design
Focus on design time; ignores the needs of the users Focus on design time Create context only
Design Communities COP Group-think
CoI Design Media DODEs
I Barriers
I
I
I Innovations from beyond the city
I Divergent thinking
I Creation of a shared understanding
I walls
I Codified domain knowledge; I Innovations beyond the boundaries
intelligent design support Face-to-face interaction mediated by tangible boundary objects Face-to-face interaction EVA mediated by boundary objects Forms of Computer-Mediation S~atial I Face-to-face s u ~ ~ o r t s maximal bandii'dth I Communication through Temporal I artifacts
of the domain Limited support for parallel interactions; conversation between participants is lost Users may have limited ability to create boundary objects.
EDC
I
I
Technological Social
Users have to deal with professionally conceived solutions Closed systems Difficulty in envisioning "unknown" futures"
I Focus on what is technologically doable Focus solely on communication
I Face-to-face limits number of
I participants
I Inherent difficulty of collaboration I between people who do not know
I
cach other
I Requires formalization Requires shared understanding
The specific biases and barriers associated with boundary objects used in the EDC and in EVA are summarized in Table 5. Textual descriptions and graphics are well suited for describing and understanding the tradition of the domain and the context in which the new system will be embedded. They are useful for activating existing domain knowledge, and for envisioning how the tradition of the domain should be changed. Scenarios are good for imagining the tasks that a new system might support as well as the steps necessary to accomplish tasks. They are weak, however, in allowing stakeholders to actually experience the situations they are designing. Prototypes are strong at allowing users to experience what work might be like using new systems, but they run the risk of being misinterpreted (Atwood et al., 1995). Games encourage collaborative, critical thinking and informed participation (Ernesto G. Arias et al., 2000), but the support of the EDC is required to capture the knowledge generated through the interactions. Computational simulations allow stakeholders to ask "what-if' questions, but unless a meta-design approach is supported, they may provide no support for specific important questions to be explored. Table 5. A Spectrum of Barriers and Biases Associated with Different Kinds of Boundary Objects Representations
Biases
Barriers
Text and graphics
Expressive (i.e., lack of syntax), easily modified
Scenarios
Envisioning, focusing
Prototypes
Experiential cognition
Limitations of verbal descriptions; often not part of design artifact Fictional, and not part of design artifact Can be misinterpreted and otherwise misused Cannot capture knowledge in reusable form, lack of realism
Games
Collaborative, critical thinking
Computational simulations
Dynamic behaviors
Role in Collaborative Design To make knowledge explicit A context for experiencing and envisioning
A vehicle for expressing
ideas about, and experiencing visions. A perspective on design as a cooperative game involving many participants and grounded I by design artifacts I To observe and understand G ~ between D sinhation and real I emergent behavior
4.2 Cols: Beyond Novices and Experts As argued before, the complexities of real problems transcend the boundaries of a COP and require the collaboration of stakeholders from different domains. Practitioners from several domains (including architecture, engineering, management, psychotherapy, and town planning) were studied by Schtin (D.A.
Schon, 1987), yielding the interesting conceptual framework of the "reflective practitioner", which we have used and extended in our research (Fischer & Nakakoji, 1992) and to which we have referred on several occasions in this chapter. Rambow and Bromme (Rambow & Bromme, 2000) have analyzed SchBn's work and suggest that Schon's reflective practitioner could learn by communicating with "laypersons" (e.g., clients, customers, patients, or users). They argue that the knowledge of laypersons is not merely an incomplete version of the knowledge of the expert, and therefore they should not be considered as students or apprentices, but rather as experts in their own right, albeit with a different expertise than that of professional designers (Bromme, Jucks, & Runde, 2004). These observations become obvious in our framework, which contrasts CoPs and CoIs. In addition, our framework clarifies the following issues: It introduces a symmetry between representatives of different Cops by postulating a asymmetry of ignorance, rather than referring to one person as an "expert" and the other person as an "layperson ". It illustrates that legitimate peripheral participation (implying that the learner will eventually learn what the expert knows) is a concept belonging to CoPs, whereas informed participation is a better characterization for CoIs. It shows that a primary objective of Cops is "learning when the answer is known" (by the expert), whereas the primary objective of CoIs is "learning when the answer is not known" (e.g., the answer to a unique design problem). It emphasizes that in CoIs (as they try to solve complex design problems), being a "1earner1novice" or "teacherlexpert" is an attribute of a context, and not of a person (e.g., some of our computer science students know considerably more about specific programming environments than we do. It provides evidence that the specific languages and ontologies used by stakeholders in one COP(e.g., the diagrammatic representations used by architects, or the formal system descriptions used by computer scientists) will in many cases not serve as boundary objects for stakeholders coming from other CoPs.
5. CONCLUSIONS Design is a ubiquitous activity. The complexity of design problems requires communities to address them. We have presented a conceptual framework based on different design approaches and we have identified different design communities. Communities of interest (CoIs) bring different communities of practice (CoPs) together to cope with the complexities of real-world design problems. CoIs provide unique opportunities to bring social creativity alive by transcending individual perspectives. To create a shared understanding and common ground between different CoPs requires boundary objects.
In the past, most computational environments focused on the needs of individual users. Our research has evolved from (1) empowering individuals to (2) supporting COPS with domain-oriented design environments to (3) creating shared understanding among CoIs with the Envisionment and Discovery Collaboratory and the Evolving Artifact approach. In this journey, we have not abandoned earlier themes, but we have widened our focus. Moreover, by analyzing biases and barriers of earlier systems, we have learned how different computational and conceptual knowledge systems fit together and complement each other. ACKNOWLEDGEMENTS The authors thank the members of the Center for LifeLong Learning & Design at the University of Colorado, who have made major contributions to the conceptual framework described in this chapter, with special thanks to Ernesto Arias, Hal Eden, and Eric Scharfi the primary developers of the EDC. We thank Anders Morch, Bonnie Nardi, and Yunwen Ye, who provided valuable feedback to an earlier version of this chapter. The research was supported by ( 1 ) the National Science Foundation, grants (a) REC-0106976 "Social Creativity and Meta-Design in Lifelong Learning Communities", and (b) CCR-0204277 "A Social-Technical Approach to the Evolutionary Construction of Reusable SofhYare Component Repositories"; ( 2 ) SRA Key Technology Laboratory, Inc., Tokyo, Japan; and (3) the Coleman Initiative, Sun Jose, CA. REFERENCES Arias, E. G. (1995). Designing in a Design Community: Insights and Challenges. In G. M. Olson & S. Schuon (Eds.), Proceedings qf Designing Interactive Systerizs (DIS'95) Synzposium: Processes, Practices, Methods, & Techniques (pp. 259-263). New York: ACM Press. Arias, E. G., Eden, H., Fischer, G., Gorman, A,, & Scharff, E. (2000). Transcending the Individual Human Mind--Creating Shared Understanding through Collaborative Design. ACM Transactions on Cornputer Hunuln-Interaction, 7(1), 84-1 13. Atwood, M., Bums, B., Girgensohn, A,, Lee, A,, Tumer, T., & Zimmermann, B. (1995). Prototyping Considered Dangerous Paper presented at the Fifth International Conference on Human-Computer Interaction (Interact '95) Lillehammer, Norway (June 27-29). Bennis, W., & Biederman, P. W. (1997). Organizing Genius: The Secrets of Creative Collaboration. Cambridge, MA: Perseus Books. Bowker, G. C., & Star, S. L. (2000). Sorting Tlzings Out - Class$cation and Its Consequences. Cambridge, MA: MIT Press. Bromme, R., Jucks, R., & Runde, A. (2004). Barriers and Biases in Computer-Mediated ExpertLayperson-Communication. In R. Bromme, F. Hesse & H. Spada (Eds.), Barriers and Biases in Computer-Mediated Knowledge Conznzunication (pp. (to appear)). Dordrecht, Netherlands: Kluwer Academic. Brown, J. S., & Duguid, P. (1991). Organizational Learning and Communities-of-Practice: Toward a Unified View of Working, Learning, and Innovation. Organization Science, 2(1), 40-57. Brown, J. S., & Duguid, P. (2000). Tlze Social Life of Information. Boston, MA: Harvard Business School Press. Brown, J. S., Duguid, P., & Haviland, S. (1994). Toward Informed Participation: Six Scenarios in Search of Democracy in the Information Age. Tlze Aspen Institute Quarterly, 6(4), 49-73. Bruner, J. (1996). The Culture of Education. Cambridge, MA: Harvard University Press.
Campbell, D. T. (1969). Ethnocentrism of Disciplines and the Fish-Scale Model of Omniscience. In M. Sherif & C. W. Sherif (Eds.), Interdisciplinary Relationships in the Social Sciences (pp. 328-348). Chicago: Aldine Publishing. Clark, H. H., & Brennan, S. E. (1991). Grounding in Communication. In L. B. Resnick, J. M. Levine & S. D. Teasley (Eds.), Perspectives on Socially Shared Cognition (pp. 127-149). Washington, DC: American Psychological Association. Cross, N. (Ed.). (1984). Developnzents in Design Metlzodology. New York: John Wiley & Sons. Dillenbourg, P. (2004). Designing Biases that Augment Socio-Cognitive Interactions. In R. Bromme, F. Hesse & H. Spada (Eds.), Barriers and Biases in Computer-Mediated Knowledge Cornn~unication ( p p (to appear)). Dordrecht, Netherlands: Kluwer Academic. Eden, H. (2003). Conceptual and Technological Support for Social Creativity in Face-to-Face Collaboration. Ph.D. Dissertation (forthcoming), University of Colorado at Boulder, Boulder, CO. Ehn, P. (1988). Work-Oriented Design o f Conzputer Arttfacts (second ed.). Stockholm: Arbetslivscentrum. EngestrBm, Y. (2001). Expansive Learning at Work: Toward an Activity Theoretical Reconceptualization. Journal of Education and Work, 14(1), 133-156. Fischer, G. (1994). Domain-Oriented Design Environments. Autorimted Software Engineering, 1(2), 177203. Fischer, G. (2000). Social Creativity, Symmetry of Ignorance and Meta-Design. Knowledge-Based Sy.sterns Journal (Special I.ssue on Creativity & Cognition), Elsevier Science B. V., Oxford, UK, 13(78), 527-537. Fischer, G. (2001). Comnzunities of Interest: Learning tlzrough the Interaction of Multiple Knowledge Systena. Paper presented at the 24th Annual Information Systems Research Seminar In Scandinavia (IRIS124),Ulvik, Norway. Fischer, G. (2002). Beyond 'Couch Potatoes': From Consunzers to Designers and Active Contributors, in from FirstMonday (Peer-Reviewed Journal on the Internet), http://firstmonday .org/issues/issue7~12/fischer/ Fischer, G., Grudin, J., Lemke, A. C., McCall, R., Ostwald, J., Reeves, B. N., et al. (1992). Supporting Indirect, Collaborative Design with Integrated Knowledge-Based Design Environments. Hunmn Conzputer Interaction, Special Issue on Conzputer Supported Cooperative Work, 7(3), 28 1-314. Fischer, G., & Nakakoji, K. (1992). Beyond the Macho Approach of Artificial Intelligence: Empower Human Designers - Do Not Replace Them. Knowledge-Based Systems Journal, Special Issue on A1 in Design, 5(1), 15-30. Fischer, G., Nakakoji, K., Ostwald, J., Stahl, G., & Sumner, T. (1998). Embedding Critics in Design Environments. In M. T. Maybury & W. Wahlster (Eds.), Readings in Intelligent User Interfaces (pp. 537-559). San Francisco: Morgan Kaufmann. Fischer, G., & Ostwald, J. (2002a). Seeding, Evolutionary Growth, and Reseeding: Enriclzing Participatory Design with Infornzed Participation. Paper presented at the Proceedings of the Participatory Design Conference (PDC'02), Malmij University, Sweden. Fischer, G., & Ostwald, J. (2002b). Transcending the blforrnation Given: Designing Learning Environnzents for Infornzed Participation. Paper presented at the Proceedings of ICCE 2002 International Conference on Computers in Education, Auckland, New Zealand. Fischer, G., & Scharff, E. (2000). Meta-Design-Design ,for designer.^. Paper presented at the 3rd International Conference on Designing Interactive Systems (DIS 2000), New York. Florida, R. (2002). The Rise of the Creative Class and How Itk Tran:fornzing Work, Leisure, Community and Everyday Life. New York: Basic Books. Giaccardi, E. (2003). Principles of Metadesign: Processes and Levels o f Co-Creation in tlze New Design Space, Ph.D Dissertation. CAiiA-STAR, School o f Computing, Plymoutlz, UK, from http://x.idat.org/-eg/research.htm Girgensohn, A. (1992). End-User Modifiability in Knowledge-Based Design Environments. Ph.D. Dissertation, University of Colorado at Boulder, Boulder, CO. Greenbaum, J., & Kyng, M. (Eds.). (1991). Design at Work: Cooperative Design of Computer Systems. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Henderson, A., & Kyng, M. (1991). There's No Place Like Home: Continuing Design in Use. In J. Greenbaum & M. Kyng (Eds.), Design at Work: Cooperative Design of Computer Systems (pp. 219240). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Kolodner, J. L. (1993). Case-Based Reasoning. San Mateo, CA: Morgan Kaufmann Publishers, Inc.
Lave, J., & Wenger, E. (1991). Situated Learning: Legitinlate Peripheral Participation. New York: Cambridge University Press. Lindstaedt, S. N. (1998). Group Memories: A Knowledge Medium for Cornrnunities of Interest. Ph.D. Dissertation, University of Colorado at Boulder, Boulder, CO. Maturana, H. R., & Varela, F. J. (1987). The Tree of Knowledge: The Biological Roots of Hurilan Understanding. Boston, MA: Shambhala. Moran, T. P., & Carroll, J. M. (Eds.). (1996). Design Rationale: Concepts, Techniques, and Use. Hillsdale, NJ: Lawrence Erlbaum Associates. Morch, A. (1997). Three Levels of End-User Tailoring: Customization, Integration, and Extension. In M. Kyng & L. Mathiassen (Eds.), Cornputers and Design in Context (pp. 51-76). Cambridge, MA: MIT Press. Morch, A. I., & Mehandjiev, N. D. (2000). Tailoring as Collaboration: The Mediating Role of Multiple Representations and Application Units. Computer Supported Cooperative Work, 9(1), 75-100. Nardi, B., & Whittaker, S. (2002). The Role of Face-to-Face Communication in Distributed Work. In P. Hinds & S. Kiesler (Eds.), Distributed Work: New Research on Working across Distance Using Technology (pp. 83-1 12). Cambridge, MA: MIT Press. Nardi, B. A. (1993). A Small Matter of Progranznzing. Cambridge, MA: The MIT Press. National-Research-Council. (2003). Beyond Productivity: Infornzation Technology, Innovation, and Creativity. Washington, DC: National Academy Press. Norman, D. A. (1988). The Psychology o f Everyday Things. New York: Basic Books. Olson, G. M., & Olson, J. S. (2001). Distance Matters. In J. M. Carroll (Ed.), Human-Conzputer Interaction in tlze New Millenniunz (pp. 397-417). New York: ACM Press. Orr, J. (1996). Talking about Machines - An Ethnography o f a Modern Job. Ithaca, NY: ILR Press/Cornell University Press. Ostwald, J. (1996). Knowledge Construction in Software Development: The Evolving Artifact Approach. Ph.D. Dissertation, University of Colorado at Boulder, Boulder, Colorado. Postman, N. (1985). Anzusing Ourselves to Death-Public Discourse in tlze Age of Show Business. New York: Penguin Books. Rambow, R., & Bromme, R. (2000). Was Schons 'reflective practitioner' durch die Kommunikation mit Laien lernen ktinnte? In G. H. Neuweg (Ed.), Wissen - Konnen - Rejlexion: Ausgewahlte Verlzaltnisbestinznzungen (pp. 245-263). Innsbruck: Studienverlag. Raymond, E. S., & Young, B. (2001). The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. Sebastopol, CA: O'Reilly & Associates. Reeves, B. N. (1993). Supporting Collaborative Design by Embedding Conznzunication and History in Design Artifacts. Ph.D. Dissertation, University of Colorado at Boulder, Boulder, CO. Resnick, L. B., Levine, J. M., & Teasley, S. D. (Eds.). (1991). Perspectives on Socially Shared Cognition. Washington, DC: American Psychological Association. Rittel, H. (1984). Second-Generation Design Methods. In N. Cross (Ed.), Developments in Design Methodology (pp. 317-327). New York: John Wiley & Sons. Rittel, H., & Webber, M. M. (1984). Planning Problems Are Wicked Problems. In N. Cross (Ed.), Developments in Design Methodology (pp. 135-144). New York: John Wiley & Sons. Rummel, N., & Spada, H. (2004). Sustainable Support for Computer-mediated Collaboration. In R. Bromme, F. Hesse & H. Spada (Eds.), Barriers and Biases in Conzputer-Mediated Knowledge Conznzunicatior~(pp. (to appear)). Dordrecht, Netherlands: Kluwer Academic. Salomon, G. (Ed.). (1993). Distributed Cognitions: Psychological and Educational Considerations. Cambridge, UK: Cambridge University Press. Scharff, E. (2002). Open Source Software, a Conceptual Framework for Collaborative Artifact and Knowledge Construction. Ph.D. Dissertation, University of Colorado at Boulder, Boulder, CO. Schon, D. A. (1983). The Rejlective Practitioner: How Professionals Think in Action. New York: Basic Books. Schon, D. A. (1987). Educating tlze Reflective Practitioner. San Francisco, CA: Jossey-Bass. Schuler, D., & Namioka, A. (Eds.). (1993). Participatory Design: Principles and Practices. Hillsdale, NJ: Lawrence Erlbaum Associates. Shneiderman, B. (2002). Leonardo:~Laptop - Hurnarz Needs and the New Contputing Technologies. Cambridge, Mass: MIT Press. Simon, H. A. (1996). The Sciences oftke Artificial (third ed.). Cambridge, MA: The MIT Press. Snow, C. P. (1993). The Two Cultures. Cambridge, UK: Cambridge University Press.
Star, S. L. (1989). The Structure of Ill-Structured Solutions: Boundary Objects and Heterogeneous Distributed Problem Solving. In L. Gasser & M. N. Huhns (Eds.), Distributed Artz3cial Intelligence (Vol. 11, pp. 37-54). San Mateo, CA: Morgan Kaufmann Publishers Inc. Terveen, L. G. (1995). An Overview of Human-Computer Collaboration. Knowledge-Based System Journal, Special l.ssue on Human-Coritputer Collaboration, 8(2-3). 67-81. Thimbleby, H., Anderson, S., & Witten, I. H. (1990). Reflexive CSCW: Supporting Long-Term Personal Work. Interacting with Contputer.s, 2(3), 330-336. Weuger, E. (1998). Contrnunities q f Practice - Learning, Meaning, and Identity. Cambridge, UK: Cambridge University Press. Wright, M., Marlino, M., & Sumner, T. (2002, May). Meta-Design qf a Coritritunity Digital Library, DLib Magazine, Volurtte 8, Number 5, from http://www.dlib.org/dlib/may02/wright/05wright.html Ye, Y., & Fischer, G. (2002). Supporting Reuse by Delivering Task-Relevant and Personalized Information. In Proceedings of 2002 International Conference on Software Engineering (ICSE'02) (pp. 513-523). Orlando, FL. Ye, Y., & Kishida, K. (2003). Toward an Understanding of the Motivation of Open Source SoShyare Developers. Paper presented at the (to appear) Proceedings of 2003 International Conference on Software Engineering (ICSE'03), Portland, OR.
[email protected] [email protected]. edu
PIERRE DILLENBOURG
DESIGNING BIASES THAT AUGMENT SOCIOCOGNITIVE INTERACTIONS
Abstract. This chapter questions the assumption that the best environment for computer-supported collaborative learning is the one that most closely reproduces the features of face-to-face collaboration. Empirical studies have failed to establish the superiority of group interaction with richer media. Instead, the chapter explores media features that do not exist in face-to-face interactions and explains how these features might augment group cognition. The first feature, the persistency of the information display, turns the medium into a shared working memory. The second feature, storing the context in which the message is emitted, should enhance the construction of a shared understanding. The third feature, the display of a graphical summary of group interactions, is expected to facilitate group regulation. In these three examples, the medium is more than a neutral wire. It constitutes a functional component within the distributed cognitive system formed by the learners and the collaborative environment.
1. INTRODUCTION How can I dare speaking about computer-supported collaborative learning (CSCL)? Collaboration software is often so frustrating that I should instead talk about despitecomputer collaborative learning! Undoubtedly, computer-mediated communication (CMC) is less rich than face-to-face interactions. From Daft & Lengel's (1984) viewpoint, the richness of text-based media (email, forums, chat systems) is lower since they do not mediate facial expressions, voice intonations, body language etc. Even video-based interaction suffers from several drawbacks compared to face-toface dialogues, such as eye contact problems, the reduced mobility of the camera, the loss of peripheral vision and, subsequently, contextual cues, and so forth. This loss of richness may not be important for tasks that do not require high quality interactions, for instance following a routine process, circulating factual information or applying formal procedures. Conversely, these effects will be important when the full subtlety of social interactions is needed. For instance, electronic mail is not an optimal tool at the initial phase of a project when partners have to align their visions and goals (Hansen et al., 1999) and to mutually assess their degree of commitment to the project. Even when face-to-faceICMC losses are minor, they may become visible when repeated over long time periods. Let's imagine that one splits a team into two geographical locations, Geneva and London, and provides them with phones and all existing CMC tools, including videoconferencing. Five months later, the Geneva guys will produce sentences such as "you know, these are the guys from London" and vice-versa. The accumulation of small but continuous losses in interaction will turn progressively one team into two distinct teams. Formal procedures will have to be set up to guarantee the collaboration among them.
These examples confirm my alignment with the theme of this book: YES, computer-mediated communication is less rich than face-to-face interaction; YES, it is important for CSCL research to investigate the many drawbacks and constraints of CMC, to measure their effects and to understand their mechanisms. However, I question the implications that are often derived from these statements. The unquestionable difference in richness often leads to a more questionable postulate, the imitation bias: the more a system would be able to reproduce face-to-face interactionfeatures, the better it would be! This postulate was questioned by Hollan and Stornetta more than a decade ago (Hollan & Stornetta, 1992) but is still implicitly present in the educational technology literature. Many discourses convey an implicit hierarchy that parallels the bandwidth levels: text-based is "less" than audio, which is "less" than video. In section 2, I argue that empirical studies have not found evidence of this hierarchy. Instead, I recommend exploring CMC features that are different from what exist in face-to-face. "Requiring one medium to imitate the other inevitably pits strengths of the old medium against weaknesses of the new. At the same time, to the extent that the goal is imitation, one will not be led to exploit the distinctive strengths of the new medium" (Hollan & Stornetta, 1992, p. 121). I investigate some of these features in section 3 and relate them in section 4 with studies on collaborative learning If CMC was only "less" than face-to-face, the only rationale for CSCL would be to bridge distance, i.e, simply to let people learn with peers they cannot meet physically. In other words, CMC would be only better than no communication at all. Should I only use CSCL when students cannot meet? Or do I see CSCL as a possibility to improve collaborative learning even if the students are collocated? For instance, a tool like Belvedere (Suthers et al., 2001) was originally designed for improving the argumentation process of two students sitting in front of a single machine. This chapter argues in favour of the second position: the purpose of CMC tools is not to perform better than face-to-face interactions but to augment social interactions (in the sense of augmented reality). What 'augmentation' means is of course specific to each task: what facilitates one task may not be useful for another. Hence, in our context, the 'augmentation' should reflect what I know on the effectiveness of collaborative learning (section 4). This contribution sheds the light on new features of CMC tools or CSCL environments. These features are quite appealing, but they do not guarantee sociocognitive effects. Tools do not determine the way they are used. For instance, I observed (Zeller & Dillenbourg, 1997) that the same hypertext led to completely different navigation patterns according to the activities set up by the teacher. This contribution analyzes tool affordances, not effects. I still do not have much experimental evidence of their effects. Users are extremely adaptive; they repurpose software tools; they invent conventions and other tricks to cope with the limitations of CMC. Hence, this contribution does not present tools for themselves but as instances of a thesis: CMC tools offer affordances that cannot be derived from the imitation of face-to-face interactions.
2. THE IMITATION BIAS We could say that, although the design of planes is somewhat inspired by the morphology of birds, planes do not flap their wings! E-Learning is still in a stage of design-by-imitation. based on the nayve assumption that the richer a medium is, the better it will be. Here follow three simple counter-examples of this intuitive law. A first counter-example is the comparison of WAP and SMS services on mobile phones. The WAP displays graphically rich documents -of course smaller than standard web pages-, while SMS are limited to short text messages. In terms of media richness, the WAP should have been a killer application compared to SMS. Statistics show the opposite1: in 2000, 1053 million SMS were exchanged in Switzerland versus only 6 million WAP hits. The richer media is not always the most appropriate. A second counter-example is the turn taking behaviour in a MOO, which is a synchronous text-based environment (a chat) enriched with a spatial metaphor (Dillenbourg & Traum, 1999). In this study, we observed many interwoven dialogues. For instance B answers to A's last utterance and then to A's before-last utterance. From an outsider's view point, these dialogues are incoherent while, as Herring (1999) pointed out, they seem to be fine for dialogue participants. To have a n estimate of regularity in turn taking, we computed an index of complexity. This index ranges from 0 if the speaker sequence is completely regular (ABABABA...) to 1 if there is no way to predict the emitter of the next utterance. Twenty pairs used the system for an average of 123 minutes. Subjects included both novices and advanced users. The average index complexity was 0.89 (SD =.06), i.e. very close to complete irregularity. More importantly, this index was not related to measures of the quality of collaboration such as the rate of acknowledgement (which percentage of A's utterances are acknowledged by B) or the percentage of redundant problem solving actions carried out by the peers. In the same spirit, Phillips (2000) showed that chat users who are forced to imitate face-to-face turn taking rules reach lower performance both in recall and brainstorming tasks. These results confirm Herring's hypotheses: the irregularity of turn taking does not indicate communication problems but the natural difference between face-to-face and MOO dialogues. Users invent smart ways to dialogue effectively in a MOO, using features that do not exist in face-to-face. As Herring stated, "Afterall, if CMC was seriously incoherent, users would notflock to the Internet so enthusiastically" (Herring ,1999, p. 1) A third counter-example is the generally expected superiority of video-based communication over audio-communication. Actually, most studies failed to confirm this expected superiority (Carles, 2001). Olson, Olson & Meader (1995) studied 36 groups of three professionals involved in a design task. The groups were placed in one of the three following conditions: 1) remote with audio-only and a shared editor, 2) remote with a shared editor, plus audio and video links, 3) co-located with the shared editor. The video preserved spatial relations and enabled eye contact. The study revealed nearly no impact of these conditions on quality in this professional design task. However, groups rated the audio-only condition as having a lower
quality and reported more difficulties communicating. The authors concluded that perceptions suffer when video is not available, and that work is accomplished in a slightly different manner, but that the quality of the work suffered very little. The lack of difference in performance between audio-only and audio+video has also been found by Anderson et al. (1997) and Fussel et al. (2000). In most cases however, the experiment subjects declared a more positive attitude in audio+video situations than in audio-only situations. One may expect that these differences in attitude could, in longer experiments, have an impact on group performance. To make a first step in this direction, we hypothesized that video communication would lead group members to build a more accurate representation of their partners' emotional states (Glaus, 2002). Twenty groups of three students had to argue an emotionally rich social issue (authorising adoption by homosexual couples) by using a shared concept map tool (TeamWaveTM).Ten groups had only an audio connection and ten groups had audio plus video. The video communication was provided by a simple web cam. In the audio+video condition, the screen was divided in three sections. The main part was devoted to the concept map window (24 X 24 cm). In the remaining part appeared the image of each team-mate (8.5 X 8.5 cm). In the audio condition, this part was left empty so that the size of the concept map was equal in each condition. At the end of the experiment, each subject had to describe his or her own emotions with a questionnaire including 18 items (anxious, enthusiastic, proud, bored, hostile, ...) expressed on a Likert Scale. Then, each subject had to describe the emotions of his or her team-mates with the same questionnaire. The hypothesis was that facial expressions would produce a better perception of emotions. Therefore, we expected that the correlation between self-described emotions and other-described emotions would be higher in the video condition than in the audio-condition. The results were opposite: the average correlation among self-description of emotion and the description of others' emotions was .62 (SD .28) with the audio+video condition and .71 (SD .19) in the audio condition. If we take the intra-team correlation as the dependent variable for an Anova, the superiority of the audio-only group was almost significant (F=3.4, p: 0.065), which is the contrary of our hypothesis! One explanation for these surprising results is that seeing what one's partner sees (shared tool) has more impact than seeing each other, as reported by Gaver et al (1993) or Anderson et al. (2000). Other explanations relate to the quality of sound and images, but, as Hollan and Stornetta phrase it: "It's tempting to think that with perhaps a little more screen resolution, a little more fidelity in the audio channel, [...I telecommunications systems will achieve a level of information richness so close to face-to-face that for most needs it will be indistinguishable" (Hollan & Stornetta, 1992, p. 120). These results do not discard the potential value of video communication. They do not question the existence of examples in which richer media occur to support effective collaboration. Our counter-examples only reject the claim that media richness per se makes a medium more effective. I thereby question the intuitive trend to consider face-to-face settings as the ideal model for CMC design. Instead, I encourage designers to explore CMC functionalities that do not exist in face-to-face
AUGMENTING INTERACTIONS
247
interactions. The next section presents CMC features that differ from face-to-face situations. 3. AUGMENTING SOCIAL INTERACTION
Hollan & Stornetta (1992) emphasized some features of CMC that offer new opportunities that do not exist in face-to-face settings: asynchronous communication, anonymous communication and archival of communication. I report three other features that were revealed by the design and experimentation of CSCL environments. In the following examples, the technology 'augments' computermediated interactions. However, advances in mixed reality technologies should also make them applicable to face-to-face dialogues, which would fit better with the very idea of 'augmentation'. 3.1 Persistency creates shared working memory Is a dialogue in a chat system worse than a phone conversation? It is certainly much slower, does not convey intonation cues and does not follow usual turn taking rules. I previously explained that different turn taking rules do not necessarily mean worse interactions. Chat systems also offer some advantages. Despite being synchronous, they leave time for reflection: one can reread previous utterances, check our understanding or refer to it, etc. This advantage is due to the fact that chat messages are more persistent than audio utterances. Most chat systems include an interaction history. This fairly simple and trivial feature has an important impact on conversations. A user can simultaneously participate in multiple conversations because the cognitive load of remembering 'who said what' is off-loaded by the system. Turn taking rules get modified since an utterance does not need to refer to the last uttered sentence but to any phrase uttered before recently. We observed that the average acknowledgment delay in a chat (MOO) environment used for collaborative problem solving was 48 seconds (Dillenbourg & Traum, 1999). In our daily use of a MOO environment as tool for collaboration and communication with students, I sometimes have a 2 hour long discussion during which a sentence is uttered every 15 minutes. Even if these chat systems are referred to as 'synchronous' systems, the interaction trace supports a high degree of freedom compared to the timing of voice dialogues. Therefore, the actual interactions are rather semisynchronous (Hollan & Stornetta, 1992). The persistency of the medium refers to how long this information can be perceived without additional action: voice is not persistent (unless recorded or counted in microseconds), chat windows are semipersistent (the information scrolls slowly up the window) while whiteboards (real or virtual ones) are quite persistent (the information remains there unless somebody erases it). The importance of persistency was revealed by our previously mentioned study on collaborative problem solving in a virtual environment (Dillenbourg & Traum, 1999). Two subjects had to solve an enigma (similar to the Cluedo game) by interacting via a chat (MOO) and a whiteboard. The subjects had to collect
information among suspects and infer who had committed a crime. We expected that subjects would use the whiteboard to draw schemata that disambiguate chat interactions. We did not observe this expected contrast between graphical and textual information. The whiteboard was mainly used to store and organize information (see figure 1). When we analyzed the text pieces transmitted via the chat and the whiteboard, the main difference was not their format (text versus graphics) but the persistency of the information content. The persistency of information refers to how long it remains true or useful for the subjects. For instance, "suspect A claims he was the lover of the victim" or "I found a ski jacket in her room" are persistent pieces of information. Conversely, sentences such as "let's ask this question to this suspect" or "have you been in the bar yet?'do not convey persistent information since they won't be true after a while. In this experiment, less-persistent information was transmitted in the chat while persistent information was mediated through the whiteboard. In others words, the subjects even though many of them were using such tools for the first time in their life spontaneously matched the persistency of the medium with the persistency of the information, exchanging the more persistent information via the more persistent medium and vice-versa. This persistent storage of information acts like a group memory. Both users added the information that had to be remembered. More precisely it acted as a group working memory: the set of collected information represented the state of the problem. Solving the problem was not only done by adding information pieces, but also by reorganizing them (e.g. gathering in the same area pieces of information that concern the same suspect), annotating them (e.g. adding a red cross on the top of information about a suspect that can be discarded) and erasing them. This collaboratively maintained representation of the problem fulfils a cognitive function that goes beyond the simple flow of information between users.
Figure I : Shared problem representation in a mystery-solving task. The two subjects add simple text notes on the whiteboard and use the graphics to organize them.
AUGMENTING INTERACTIONS
249
This point illustrates our main argument. The collaboration medium is more than a wire; it contributes to maintaining a joint representation of the problem state. It thereby supports the construction of a problem solving strategy. Computer-mediated technologies could be improved by exploring not only their limitations but also these affordances.
3.2 The context is stored with the message The virtual environment plays a role in defining the conversational context. For instance, in the experiment reported above, the chat interaction occurred in a MOO environment, which is a text-based virtual reality: rooms are described with words. However, we observed cases where the two users meet in a virtual room and one says "He lies" to his partner while "he" has not been grounded in a previous utterance. However, as the two participants are in a virtual room in which only one suspect is located, each subject may assume that "he" refers to this single referable person. In other words, the virtual room narrows down the conversational context. As rooms define a discrete space, we wondered whether the same behaviour would exist in a continuous space. The MOO conversational rule "I don't need to make explicit what I refer to if it's the same virtual room" would in a 3D virtual space become something like "I don't need to make explicit what I refer to if it's close to me in virtual space". Another study (Ott & Dillenbourg, 2002) confirmed that this also happens in a 3D virtual space: the smaller the distance between the emitter and the object he is referring to, the faster the receiver finds out which object is referred to by the emitter. One may exploit these results by designing virtual spaces that support the contextualisation of communication: the virtual surroundings of the message emitter provide the receiver with contextual cues, i.e. the receiver receives more information than what is explicitly stated in the message. These contextual cues remain of course less rich than those provided by physical co-presence. The previous examples concern environments where synchronicity creates an implicit link between the message received at time T and what the receiver perceives as the context at time T. Now, what if the receiver reads the message later on? An original feature of CMC is the ability to store the context in which the message has been emitted. A simple example is when one replies to an email by inserting the emitter's message into one's response. A more elaborate example is the Sticky Chat environment (Churchill et al., 2000), where two people collaboratively edit an electronic text. A small chat session, that has more or less the format of a sticky note, is attached to an anchor in the edited text and will move up or down and disappear as users scroll the document. The same approach has been developed for asynchronous communication. In the DocMeeting environment (Dillenbourg et al., 2002; Corti, to appear), arguments are anchored into the web document that students are arguing about. The black window serves to initiate a discussion forum that is anchored to a specific place (red triangle) of a web document. The argument window offers several options in setting up the discussion, mainly who is invited to join it.
f
travel site.
We wefen? sure what to enoect from hiicrcr
PI. 1.f.
....,... 3
(Expeila '3:')
I.""&
a,.,
s -. .
-
..-....-.
.
.
.
..- ..
.
But Mlcrosofl Expedla surprised us, turning out belter lhan we expected. ~ J O e s p l l ea i e r glaring errors in the ~ n t apager, l most Expeda pages made far a smooth, easy buying supenence. L S w l r h a few mprovemenll, ErrEdla could evolve Into a lruly excellent e-commerce slle The gaal of our Expedia test was to buy a round-tlip plane ticket from New Yo* to Seattle.
Figure 2: The DocMeeting environment (Corti, to appear)
While documents are usually joined as attachments to the message that refers to them, here the relationship is inverted; the messages are attached to the document. Storing the link between an object and the interaction about this object is a very simple feature. Nevertheless, it 'augments' communication in the sense that the message is enriched with contextual information that has otherwise to be maintained in long term memory by the receiver for correctly interpreting the message.Of course, the StickyNote and DocuMeeting anchors do not precisely define the scope of the context: the user who places an anchor in a paragraph may be referring to the word following the anchor, to the sentence, to the paragraph or even to the page. The document-interaction links stored in the two examples correspond to deictic gestures: they help to relate an interaction to the object being referred-to in this interaction. Placing an anchor is like any deictic gesture: it does not produce a definite reference (Clark & Wilkes-Gibbs, 1986). What is being pointed to needs itself to be negotiated. This deictic relationship is only one dimension of what defines the context of a message. The context is a socially constructed frame that may include a variety of events, values, stories, etc. Could we design systems that are able to collect the information about this broader context and attach it to the message? Actually, most information useful for defining the context is already
AUGMENTING INTERACTIONS
25 1
available on-line. Personal organizer software may re-construct the context from information about meetings ("This message was sent shortly before the meeting". .. hence A understands that B did not know that.. ..), about travel ("This message was sent just after A's trip back from Hong-Kong") or from the to-do list ("This answer was sent while A had to produce a report by the next day"). Many of the events occurring in society ("This message was sent two hours after G.W. Bush declared that.. .") are available in newspaper web sites. The messages that the user received or sent are known by the system. Moreover, his activity with a variety of software can be used to infer his recent activities. There is an infinite amount of data that can be used to describe the context in which a message was emitted. Of course, this raises obvious privacy issues. However, to develop this approach, the designers' challenge is not to find available contextual information, but to filter relevant contextual information, i.e. to decide which peripheral information needs to be mutually known by the emitter and the receiver to foster mutual understanding of the message. 3.3 The mirror for the group
Another way in which electronic communication can be augmented is by providing a group with a representation of their interactions. We developed a CSCL script, called ArgueGraph, which triggers argumentation by forming pairs with conflicting views (Jermann & Dillenbourg, 2003). In order to form these pairs, the students answered an individual questionnaire reflecting their views of the domain. Based on these answers, the system plotted the student names on a graph (Figure 3). The role of this representation was to form pairs by selecting students who where located far from each other. However, even before making the groups, the simple fact of showing this social map always generated many comments among students. Showing a group an image of itself generates a lot of remarks, jokes and questions among students. These observations were informal. We experimentally investigated the effect of this script on the argumentation process (Jermann & Dillenbourg, 2003), but we have not systematically investigated the group reaction to the display phase per se. Therefore, I can only refer to potential effects of these dynamically generated group representations that we call group mirrors. If our informal observations were confirmed, these mirrors would constitute a simple way to achieve a very difficult goal: raising individual engagement and group dynamics in a distance learning environment. Most recent software for group interactions, such as community portals and learning management systems, include procedures for voting and displaying results that enable the same approach. The two main features of group mirrors are what is displayed by the mirror (content of display) and how it is displayed (mode of display). The ArgueGraph mirror displays opinions. Other input data could be used such as the interactions among individuals or their actions in the learning environment. Figure 4 shows a MOO client where the users' actions and interactions are visualized. Dialogues in a MOO are structured by virtual rooms; each user 'hears' conversations in his room and messages from other rooms that are addressed to her.
As a complement, Figure 4 provides users with a global picture of what's going on in the whole MOO environment.
r
ISCOVERY
TERCHING
Figure 3: Graph representing individual opinions in the ArgueGraph script. (Names have been erased)
The graphical representation displays one line per user. Time runs from left to right. The two zoom lines at the bottom enable the user to broaden or narrow the period being considered. The intensity of activity is displayed vertically. The actions displayed include interactions (messages) and other actions (e.g. moving between virtual rooms). The former represents the vast majority of the recorded actions. The size of the bar is the number of actions performed per time unit. The coloured lines represent periods where the user was not logged into the environment. In the left hand pane, users could define subgroups to be visualized, e.g. a specific project team.
.
?-.-. .............. .............................. ..:,... Lectures i concer~tsi Derllos i Grllle I ( ~ r l l l e2 i Grille 3 ?
:I
: .....-",.
-
11
i--
'1
::.Tom..............................................................................................L.. .......... ............... NO....... .. :jlWinset, ....... ,, . . . ,.&,lkrrh,., ..u... : ...................No .... ....,,,, ......... .- .......... ;,P'ri * ............ ..................... -:.NO .- ........T... ...... j i :,wan.- . . ....... "........ ,............................. :.....--.No ...-.... i . . . J ,._ i: ............. NO .....-..... ..... :. .......,.
.:-. I-.
'
Figure 4. The representation of interaction in VisualMoo (Dillenbourg et al., 2003)
This simple representation provides the user with a quick overview of who is connected or not, from how long and whether or not team mates have been very active. This information is not very rich per se. For instance, a low level of activity in the MOO does not give precise information regarding to the availability of the partner, who might be away from his keyboard or busy with other software, etc. The interest is to provide a global appraisal of the group activity, for instance the covariation of the activity rate of several group members. As in any visualisation tool, our eye may view more than what is in the data. The design of collective mirrors raises an interesting design challenge: to invent grammars that turn social information into visual 'gestalt'. I compare the VisualMoo grammar with the grammars used in three other chat environments. 1.
2.
In Babble (Erickson et al., 1999), time is also represented by distance but in two dimensions: in their so-called 'social proxy', active users are near the centre and drift slowly towards the periphery. Colours represent the users' identity. In Chat Circles (Viegas & Donath, 1999), time is represented by gradually shrinking the size and fading the colour of the circles where messages are
displayed. The clustering of circles into groups represents conversations, i.e. who was able to read the messages of whom. Colours represent the users' identity. Conversation Landscape (Donath & Viegas, 1999) uses a different grammar based on the same elements. The picture is built perpendicularly to our VisualM00: time is represented vertically, from top to bottom, and vertical bars represent users. The length of chat utterances determines the length of horizontal bars. The role of these visual grammars is to aggregate information. One cannot expect that users devote a lot of attention or effort to analytically read these social representations. Users are supposed to work, not to carefully watch these pictures. In a glance they must get an idea of what's happening in the distributed group. When I participate in a co-present meeting, I maintain a rough representation of who is talking more, who has been speaking recently, who was arguing with whom, etc. This representation may not be very detailed and accurate but is still useful to grasp the group dynamics. The group mirrors aim to provide a similar snapshot for mediated interactions. The same idea has also been applied to asynchronous interactions. In this case, the initial information that can be interpreted is richer: the system knows which message has been sent to whom and answered by whom. LOOM (Donath, Karahalios & Viegas, 1999) represents discussion groups. Time is represented horizontally from left to right and each user is plotted on a horizontal line. Dots represent individual postings and conversational threads are displayed as lines between dots. In a glance, these representations give an appraisal of the structure and the intensity of interactions in the discussion group. The technology is available to build representations which are more elaborated than the examples so far. Social network analysis techniques (Scott, 1991) can be used to represent the social dynamics of distributed groups (leaders, outsiders, etc.). Latent semantic analysis (Landauer et al., 1998) can be added to take into account the content of the interactions: for instance, it may be interesting to display the fact that two people may be discussing similar topics without talking to each other. The development of those systems is however hampered by several obstacles. The main issue is inherent to all so-called 'awareness tools' (Gutwin & Greenberg &, 1999). Informing A about what B is doing automatically generates privacy problems. Not surprisingly, most of the systems illustrated above have been used with university teams or informal communities where privacy is not a major concern. When I attempted to apply this approach to support teamwork in a Geneva bank, the privacy problem was unbearable, ranging from "unacceptable" to "illegal". Users declared they were interested in knowing what others were doing in the environment but were not willing to show them what they were doing themselves. This functionality was in conflict with their corporate culture, and, to some extent, with data protection laws. The second obstacle is the normative strength of social representations. The pictures are not neutral. Most of them enable each individual to compare oneself to others and to the group. Naturally, these pictures may reinforce social norms such as
AUGMENTING INTERACTIONS
255
"being in the centre of the group", "being above the others", "being different from others" , ... These norms, carried out by group culture, could be reinforced by the representations. Since humans are highly adaptive, this normative approach will inevitably induce 'cheating', i.e. fake behaviour produced only for influencing the representation. In a class or in a conference, there is always a person who asks many questions to make himself prominent. The same person, once connected to an environment enriched with a group mirror, might for instance post quasi-useless messages just to appear as very active on the picture! Nevertheless, as educational designers, we can use this bias! If, by their normative virtue, group mirrors influence the users behaviour, the design of mirrors becomes an educational engineering issue. The point is not only to build a grammar that nicely matches input information (message sender, time, length, ...) with graphical objects (colour, size, position, ...) but to influence the group interaction towards an improved way of interactions. The meaning of "improved" is specific to the environment. In the context of this book, what is aimed to be improved is the effectiveness of learning collaboratively. Hence, in section 4, I relate this approach to the research on computer-supported collaborative learning. This functional role of collective mirrors raises the issue of experimentation. I don't know of any controlled experimental studies on collective mirrors. Most experiments with them were informal evaluation studies, except the one presented in section 4.4. 3.4 Synthesis
This section reviewed features of computer-mediated communication which do not exist in face-to-face dialogues: the persistency of previous interactions, the possibility to store contextual elements with the message and the possibility to provide in real time a visual summary of the group interactions. These three features are presented as examples of our claim: computer-mediated communication is definitely less rich than face-to-face interaction but also possesses interesting features worth exploring. Now, the question is whether these CMC-specific features offer any added value in the context of collaborative learning. 4. AUGMENTING COLLABORATIVE LEARNING In some learning situations, the group members cannot or can rarely meet physically because they live far from each other, because they have non-overlapping time slots, because they have a reduced mobility and so forth. In this case, technology offers the obvious advantage of enabling interactions. Face-to-face meetings being impossible or rare, computer-mediated interactions will in any case be more productive than no interaction at all. The real challenge for CSCL designers is to create added value even when face-to-face meetings are possible. In 'augmented reality' (Azuma, 1993), overlapping direct world experience with digital information is expected to increase performance in the real world. To what extent could the
CSCL environment overlap the group's activities with functionalities that would lead to higher learning gains? To investigate this question, I need to discuss what makes collaborative learning effective. 4.1 How to support collaborative learning?
The research on collaborative learning has led to controversial results with respect to its effectiveness. The effectiveness varies according to the conditions in which collaborative learning occurs: whether the groups include 2, 3, 5 or 20 members; whether they mix males and females or high and low performing students, whether the group members freely select each other or the groups are formed by the teacher; whether the learning object is something that can be discussed or not; whether regulation plays an important role in the target skills, ... These conditions of effectiveness concern the group composition, the task and the medium. Not only are these conditions numerous, but they also interact with each other in such a complex way that it is impossible to specify the 'ideal conditions for a specific target group and specific learning objectives. Hence, scholars are not focusing on the initial conditions of collaborative learning, but on the collaborative process itself (Dillenbourg et al., 1995). The effects of collaborative learning depend on the quality of interactions among peers. Since the teacherlresearcher cannot a priori control the conditions that might guarantee that these interactions occur, helshe may attempt to directly shape the interactions that occur in the group. I speak here about interactions in an abstract way (including argumentation, negotiation, explanation, ...) because each learning theory has its own way to describe them and to articulate them with learning: interactions are analyzed as ways to solve a socio-cognitive conflict in the piagetian school (Doise & Mugny, 1984), as the social version of the self-explanation effect identified in cognitive science (Chi et al., 1989), as ways to internalize mutual regulation into self-regulation (Blaye, 1988), as ways to build common grounds in psycholinguistics (Clark & Brennan, 1991), or as cognitive tools in socio-cultural approaches. I do not claim here that these theories are equivalent, but that the approach described below can be adapted to each perspective: To 'augment' collaborative learning - in the sense of 'augmented reality' - means to increase the probability that 'rich' interactions occur, 'rich' being dependent on the learning theory behind them. Interactions can be shaped in different ways:
-
-
Proactively, by structuring the collaborative process in order to favour the emergence of productive interactions, either by - the design of specific communication interfaces (section 4.2) - the design of scripts that specify how collaboration should occur (section 4.3). Retroactively, by regulating group interaction if the desired interactions do not occur or if undesirable interactions occur frequently. CSCL tools may help the teacher monitor group activities (section 4.4.)
AUGMENTING INTERACTIONS
257
I now relate these three modes of support with the notion of augmented interaction.
4.2 Shaping interactions with interfaces Two types of interfaces have been designed to shape interactions: semi-structured textual communication tools and graphical representations. Semi-structured communication interfaces (Baker & Lund, 1996) implicitly postulate a relationship between the cost of dialogue acts and their frequency. These acts are built into (often synchronous) communication tools with dialogue buttons such as "I disagree with you" or sentence openers such as "Please tell me why ... " that the user completes with text input. The latter case still allows the user to type whatever he wants after the sentence opener. These tools are referred to as 'semistructured' interfaces: they do not force the user to perform specific dialogue acts but encourage her to do so. Usually, the user can still use a free text zone. The incentive to use buttons is mainly the reduction of typing costs. However, the time necessary for a user to select a button with the mouse and then put her hands back on the keyboard to type the remainder of the sentence is not significantly smaller than the time necessary to type the first words of the sentence. Interactions can be further structured by tools that activate and deactivate dialogue buttons according to dialogue rules that specify well-formed sequences such as [explanation request] [explanation] - [acknowledgement]. So far, the research on this subject has not produced strong evidence of the effectiveness of these interfaces (Veerman & Treasure-Jones, 1999), namely because users are very talented at 'repurposing' the available dialogue buttons to their natural dialogue style instead of adopting the desired patterns of interactions. The graphical communication tools seem to have encountered more positive results. Roschelle (1992) emphasized that graphical objects on the screen cannot be bias-free but must be designed as tools that support the construction of a shared understanding. Suther's work on successive versions of the argumentation tool Belvedere (Suthers et al., 2001) and his empirical comparisons of alternative representational notations (graph, text, matrix) led him to rephrase the notion of 'representational bias' into the more positive notion of 'representational guidance' (Suthers & Hundhausen, 2003). They found for instance that, with visually structured representations (graph and matrix), users more often revisit issues they had previously discussed. They conclude that "the choice of representational notations for collaborative learning applications can have significant effects on the learner's interaction and may differ in their influence on subsequent collaborative use of the knowledge being manipulated." (p. 213). These findings illustrate my main point. These communication tools produce 'barriers and biases', as referred to in the title of this book. Indeed, they constitute purposely-designed biases. Their design rationale is to bias the communication in a sense that is expected to enhance learning: produce explanation; express disagreement, etc. The CSCL challenge is not to avoid biasing communication but to
determine which interactions are desired and how they can be induced by the interface.
4.3 Shaping interaction with scripts The second way of structuring collaborative learning is to embed group activities with a script that increases the probability that rich interactions occur. A collaboration script (O'Donnell & Dansereau, 1992) is a sequence of activities performed with different forms of grouping and different distributions of roles (Dillenbourg, 2002). I do not develop the notion of scripts here since they are illustrated in the chapters by Rummel & Spada and Weinberger. The idea of scripting collaborative learning came before the use of computers in collaborative learning. The added value of technology relies in the enforcement and the logistics of CSCL scripts. Instead of presenting the script to students as a list of instructions to be memorized and applied by the students, the script is enforced by the environment: the timely delivery of activity products, the distribution of information among groups and the distribution of roles within groups, the competition or collaboration between groups, etc. The environment also manages the flow of data between phases. Referring again to the topic of this book, these environments are purposely setting up barriers (or constraints): team members are less free since they have to play defined roles; time constraints are defined for subtasks; access to information is sometimes limited, etc. Various CSCL scripts have been defined to foster specific classes of interactions: in a JIGSAW-like script (Aronson et al., 1978), the information is divided into complementary sets of knowledge in order to trigger explanations; in a reciprocal script (Palincsar and Brown, 1984), the task is split into cognitive and metacognitive layers in order to trigger regulation interactions; in the ArgueGraph script (Jermann & Dillenbourg, 2003), group are forms with students with different opinions in order to generate conflict solving interactions.
4.4 Regulating interactions The two previous approaches, as semi-structured communication tools and collaborative scripts, aim to increase the probability that productive interaction occurs, but without any guarantee that they do. For instance, it may happen that one group member dominates interactions and does not involve his or her team mates in the decision process or that the discussion remains very superficial and does not include the knowledge to be acquired. In these cases, the tutor has to intervene and regulate interactions. Regulating collaborative learning is a subtle art. The tutor has to provide prompts or cues without interfering with the social dynamics of the group. Moreover, human tutoring is quite time-consuming: the tutor has to monitor group interactions. In a classroom, experienced teachers manage to grasp group dynamics from a very short observation period, then intervene and walk to the next group. They may even get a
first appraisal of what's going on from remote cues such as noise level and postural attitudes. When interactions are computer-mediated, this quick-and-intuitive appraisal of the quality of group interactions is difficult to build. The cost of tutoring multiple groups in a CSCL environment might be very high. Several solutions are explored. Balancing load.
In individual learning, tutoring is not required continuously, but only when students face a novel difficulty (Delibvre, 2000). If these results are confirmed with group tutoring, CSCL scripts could start with a time offset in such a way that teams pass through the same phase at different times and hence require intense tutoring at different times.
Automated tutors. Several researchers attempt to automate group regulation (Inaba & Okamoto, 1996; Barros & Verdejo, 2000; Constantino-GonzAlez, Suthers & Icaza, 2001). To avoid parsing natural language dialogues, the analysis of collaboration is focusing on the task-level actions and the comparison of the actions performed in the private problem space and in the joint problem space (GaBner, 2001). Of course, any combination of automated regulation and human regulation can be designed. Group self-regulation. Group mirrors provide the groups with a representation of its own processes or interactions. Their purpose is to help the group regulate itself. As explained in section 3.3, mirrors are not neutral. For instance, a graph that indicates who speaks more in the team conveys the assumption that an even distribution of interactions is suitable. Referring again to the 'biases' in CSCL, if one can't design group mirrors that do not implicitly convey norms, let's use this bias for helping the group to adapt its interactions to the desired model of interactions. Teacher's cockpit. The synthetic descriptions of group interactions, used as group mirrors, can also be used as tools for the tutors who regulate teamwork. These displays do not provide a detailed description of interactions, but raise awareness of a problem. The teacher would then look more closely at what's going on in a specific group.
Jermann (2002) investigated the group self-regulation approach. Pairs of students were using a piece of simulation software: their task was to tune the duration of traffic lights in order to minimize the time necessary for cars to cross a city. He found that successful pairs interacted more than others beforelafter tuning lights. Hence, a very simple indicator - the tuneltalk ratio - was predictive of group performance. This ratio was displayed in the group mirror (see figure 9). The experiments revealed that groups using the tool enriched with this display modified their behaviour accordingly: the tuneltalk ratio in the condition with the mirror was higher than in the condition without the mirror (Jermann, 2002).
Figure 5. Group Mirror showing a normative ratio of talking (right hand side) and turning (Left hand side) during joint problem solving. (Jermann, 2002)
A mirror is a prosthesis for group regulation, not a substitute. The mirrors provide a real time feedback that is only expected to facilitate regulation. Nickerson (1993) proposed that a group could only be described as a distributed cognitive system if it has its own metacognitive process at the group level. Is the group regulation more than the sum of individual regulation processes? The work reported here does not answer this question but proposes new ways to investigate it. Finally, let me mention that the analysis of group interactions can also be performed a posteriori, by reading a log file of one's own interactions (Gassner et al., 2003).
5. CONCLUSION Yes, computer-mediated collaboration is a pain. In many ways, it can only be defended as a substitute when face-to-face interaction is not possible. But, on the other hand, it offers some interesting features that I summarize in one sentence: these environments turn communication into substance. What is exchanged among users is in the system to be transferred from one point to another, but the system can do many other things with this substance. What can be done is limited by factors such as privacy and the low performance of natural language understanding. I illustrated simple examples of processing, such as storing links between an utterance and the context in which it was emitted or producing graphical summaries of behaviour and interactions. The main bottleneck here is our imagination: it is difficult to design features that do not exist in volatile face-to-face interactions. Hence, this domain is still largely unexplored.
AUGMENTING INTERACTIONS
26 1
The processing of interactions goes beyond mere communication; it aims to contribute to the group processes. For the distributed cognition perspective (Salomon, 1993), the communication tool in a component is a distributed cognitive system that includes the users and other tools. Jermann's mirror has been designed to become a metacognitive component of the problem solving system and the MOO whiteboard was used by our subjects as a group working memory. These examples illustrate my thesis: The wire is more than a simple wire: it's an agent within the distributed cognitive system. The substance processed in the examples illustrated here is textual interaction. The application of the same techniques to audio communication, via a phone or in face-to-face, raises technical problems that are non-trivial, but that do not question the fact that the same approach could be used in face-to-face dialogues, leading to a notion of 'augmented collaboration' close to the way head-mounted displays augment vision in augmented-reality systems. The main implication of our approach does not concern the design of distance learning environments, nor the 'blended learning' approach (i.e. the mere juxtaposition of face-to-face and computer-mediated activities). The challenge is the deep integration between co-present activities and technology-based support. In integrated learning, face-to-face interactions are available, both among students and with the teacher. The value of computer-mediated communication is then clearly not to reproduce available face-to-face interactions, but to support types of interactions that do not exist in face-to-face or to shape face-to-face interactions in a pedagogically desirable way. ACKNOWLEGEMENTS Most of the work illustrated here has been carried out by or with Patrick Jermann, Thomas Wehrle, Yvan Bourquin, Diego Corti, David Ott, Markus Glaus, Daniel Schneider and David Traum, while I was at the University of Geneva. NOTES
REFERENCES Anderson, A. H., O'Malley, C., Doherty-Sneddon, G., Langton, S., Newlands, A,, Mullin, J., & Fleming, A. M., & Van der Velden, J. (1997). The impact of VMC on collaborative problem solving: An analysis of Task Performance, communicative process, and user satisfaction. In K.E. Finn, Sellen, A.J. et al. (Eds.). Video-mediated conanunication: Computers, cognition, and work, 133-155. Mahwah, N.J.: Lawrence Erlbaum Associates, Inc., Publishers. Anderson, A. H., Smallwood, L., MacDonald, R. Mullin, J. Fleming, A. and O'Malley, C. (2000): Video Data and Video Links in Mediated Communication : What do Users value ? international Journal of Human-Con~puterStudies, 52, 165-187.
262
DlLLENBOURG
Aronson, E., Blaney, N., Sikes, J., Stephan, G., & Snapp, M. (1978). The Jigsaw Classroom. Beverly Hills, CA: Sage Publication. Azuma, R. (1993). "Tracking Requirements for Augmented Reality." Communications of the ACM 36 (7): 50-51. Baker, MJ. & Lund, K. (1996) Flexibly structuring the interaction in a CSCL environment. In P. Braa, A. Paiva & J. Self (Eds), Proceedings of the European Conference on Artificial Intelligence in Education. Lisbon, Portugal, Sept. 20 - Oct. 2, pp. 401-407. Barros, B. & Verdejo, F. (2000) Analysing student interaction processes in order to improve collaboration: The DEGREE approach. Journal ofArtificial Intelligence in Education, 11, 211-241. Blaye, A. (1988) Confrontation socio-cognitive et resolution de problemes. Doctoral dissertation, Centre de Recherche en Psychologie Cognitive, Universit£ de Provence, France. Carles, L. (2001) Benefits and limits of Video Channel on Mediated Interactions. Internal Report, Geneva Interaction Lab. Chi M.T.H., Bassok, M., Lewis, M.W., Reimann, P. & Glaser, R. (1989) Self-Explanations: How Students Study and Use Examples in Learning to Solve Problems. Cognitive Science, 13,145-182. Churchill, E.F, Trevor, J, Bly, S., Nelson, L. & Cubranic D. (2000). Anchored Conversations. Chatting in the Context of a Document. In Proceedings of CHI 2000 Conference on Hwnan Factors in Computing Systems, ACM Press, pp. 454-461. Clark, H.H. & Brennan S.E. (1991) Grounding in Communication. In L. Resnick, J. Levine & S. Teasley (Eds.), Perspectives on Socially Shared Cognition (127-149). Hyattsville, MD: American Psychological Association. Clark, H.H. & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22:1-39. Constantino-Gonzales, M. A., Suthers, D., Icaza, J. (2001). Designing and Evaluating a Collaboration Coach: Knowledge and Reasoning. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.) Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future (lOth International Conference on Artificial Intelligence in Education), Amsterdam: IOS press, May 19-23, San Antonio Texas, pp. 176-187. Corti, D. (to appear). Digital Annotations: Collaboration Patterns and Common Ground. Journal of the ACM. Daft, R.L. & Lengel, R.H. (1984). Information richness: a new approach to managerial behavior and organizational design. In: Cummings, L.L. & Staw, B.M. (Eds.), Research in organizaüonal behaviorö, (191-233). Homewood, IL: JAI Press. DeLievre , B. (2000). Etude de l'effet de quatre modalites de tutorat sur l'usage des outils d'aide dans un dispositif informatise d'apprentissage ä distance, These de doctorat non publiee. Universite de MonsHainaut, Belgique. Dillenbourg, P. & Traum, P. (1999). Does a shared screen make a shared understanding ? C. Hoadley et J. Roschelle (Eds), Proceedings of the Third Computer-Supported Collabomtive Leaming Conference, pp. 127-135, Stanford, Dec. 1999. Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed). Three worlds ofCSCL Can I support CSCL (pp. 6191). Heerlen, Open Universiteit Nederland. Dillenbourg, P., Baker, M., Blaye, A. & O'Malley, C. (1995) The evolution of research on collaborative learning. In E. Spada & P. Reiman (Eds) Leaming in Humans and Machine: Towards an interdisciplinary leaming science. (pp. 189-211) Oxford: Elsevier. Dillenbourg, P., Ott, D., Wehrle, T., Bourquin, Y., Jermann, P., Corti, D. & Salo, P. (2002). The sociocognitive functions of community mirrors. In F. Flückiger, C. Jutz, P. Schulz and L. Cantoni (Eds). Proceedings ofthe 4th International Conference on New Educational Enviwnments. Lugano, May 811,2002. Doise, W. & Mugny, G. (1984) The social development ofthe intellect. Oxford: Pergamon Press. Donath, J. Karahalios, K. & Viegas, F. (1999) Visualizing Conversations. Proceedings of HICSS-32, Maui, HI, January 5-8, 1999. Erickson, T., Smith, D.N., Kellogg, W.A., Laff, M.R., Richards, J.T. & Bradner, E. (1999). Socially Translucent Systems: Social Proxies, Persistent Conversation, and the Design of 'Babble'. In Proceedings of CHI'99 Conference on Human Factors in Computing Systems. New York: ACM Press. Fussell, S. R., Kraut, R. E. , & Siegel, J. (2000). Coordination of Communication: Effects of Shared Visual Context on Collaborative Work . Pwceedings ofCSCW2000 (pp. 21-30).
GaBner, K (2001). Architecture of a Cooperative Discussion Environment based on Visual Languages. In P. Dillenbourg, A. Eurelings, & Kai Hakkarainen (Eds) Proceedings of the European Conference on Couputer-Supported Collaborative Learning (Euro-CSCL 2001). pp. 261-268. Maastricht, The Netherlands, March. GaBner, K., Jansen, M., Harrer, A., Hemnann, K. & Hoppe, H.U. (2003). Analysis methods for collaborative models and activities. In Wasson, Ludvigsen and Hoppe (Eds). Designing for change in networked learning environments. Proceedings of the International Conference on Computer Supported Collaborative Learning (CSCL2003). (pp. 269-378). CSCL Series. Kluwer Academic Publishers, Dordrecht Gaver, W. & Sellen, A. & Heath, C. & Luff, P. (1993): One is not enough: Multiple Views in a Media Space. Proceedings of INTERCHI'93, ACM. Glaus, R. (2002) Apport de la vid6oconfCrence sur la reprksentationd es Cmotions et de l'implication a la tlche dans une situation de travail collaboratif. Unpublished master theses. TECFA, University of Geneva. Available under http:lltecfa.unige.ch/staflstaf-g/glaus/s~~5/memoire~sta~2.pdf Gutwin, C. and Greenberg, S. (1999). The Effects of Workspace Awareness Support on the Usability of Real-Time Distributed Groupware. ACM Transactions on Con~puter-HunlanInteraction, 6 (3), 24328 1, September. Hansen, T, Dirckinck-H olmfeld, L, Lewis, R & Rugelj, J. (1999) Using telemtics for Collaborative Knowldge Construction. In P. Dillenbourg (Ed) Collaborative learning: Cognitive and Computational Approaches (pp. 169 - 198) Oxford: Pergamon. Herring, S. C. (1999). Interactional coherence in CMC. Proceedings of the 32"' Hawai'i International Corlference on System Sciences. IEEE Computer Society Press. Hollan, J. & Stornetta, S. (1992). Beyond being there. Proceedings of the International Coqference on Couputer-Human Interaction (CH1'92).pp. 119-1 25. Inaba, A. & Okamoto, T (1996) Development of the intelligent discussion support system for collaborative learning. Proceedings of Ed-Telecom '96. (pp 494-503), Bostoo. Jermann, P. & Dillenbourg, P. (2003) Elaborating New Arguments Through A CSCL Script. In Andriessen, G., Baker, M. and Suthers D. (Eds) Arguing to learn: Confronting Cognitions in Computer-Supported Collaborative Learning environments. CSCL Series, Kluwer. Jermann, P. (2002) Task and interaction regulation in controlling a traffic simulation. In G . Stahl (Ed.) Computer Support for Collaborative Learning. Proceedings of CSCL 2002, Boulder (pp. 601-602). Lawrence Erlbaum, Hillsdale, NJ. Landauer, T. K., Foltz, P. W., Laham, D. (1998). An introduction to Latent Semantic Analysis . Discourse Processes, 25,259-284. Nickerson, R.S. (1993) On the distribution of cognition: some reflections. In G. Salomon. (Ed). Distributed cognitions. Psychological and educational considerations (pp. 229-262) Cambridge, USA: Cambridge University Press. O'Donnell, A. M., & Dansereau, D. F. (1992). Scripted cooperation in student dyads: A method for analyzing and enhancing academic learning and performance. In R. Hertz-Lazarowitz and N. Miller (Eds.), Interaction in cooperative groups: The theoretical anatomy of group learning (pp. 120-141). London: Cambridge University Press. Olson, J. S., Olson, G. M., & Meader, D. K. (1995). What mix of video and audio is useful for remote real-time work. Proceedings of the Conference on Hurruzn Factors in Cornputing Systenzs (pp.362368). Denver, CO: Academic Press. Ott, D. & Dillenbourg P. (2002). Grounding through proximity in a 3D Collaborative Environment. In F. Fliickiger, C. Jutz, P. Schulz and L. Cantoni (Eds). Proceedings of the 4" International Conference on New Educational Environments. Lugano, May 8-1 1,2002. Palincsar A.S. and Brown A.L. (1984) Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities. Cognition and Instruction, vol. I , no.2, pp. 117-175. Philips, B. (2000) Should I take Turns ? In Proceedings of the CHI2000 Conference on Human Factors in Computing Systenzs ( pp. 341-342). Roschelle, J. (1992) Learning by Collaborating: Convergent Conceptual Change. Journal of the Learning Sciences, 2, 235-276. Scott, J. (1991). Social Network Analysis: A Handbook. Newbury Park, CA: Sage Publications. Suthers, D., and Hundhausen, C. (2003). An Empirical Study of the Effects of Representational Guidance on Collaborative Learning. Journal of the Learning Sciences, 12(2), 183-219.
Suthers, D., Connelly, J., Lesgold, A., Paolucci, M., Toth, E., Toth, J., and Weiner, A. (2001). Representational and Advisory Guidance for Students Learning Scientific Inquiry. In Forbus, K. D., and Feltovich, P. J. (2001). Smart machines in education: The coming revolution in educational technology. Menlo Park, CA: AAAIIMit Press, pp. 7-35. Veerman, A. L., & T. Treasure-Jones (1999). Software for problem solving through collaborative argumentation. In P. Coirier & J. E. B. Andriessen (Eds.), Foundations of argumentative text processing (p. 203 - 230). Amsterdam, Amsterdam University Press. Viegas, F. & Donath, J. (1999). Chat circles. In Proceedings of the CHI 1999 Conference on Hurnan Factors in Cornputing Systems. New York, ACM Press, pp. 9-16 Zeller, P. & Dillenbourg, P. (1997) Effet du type d'activit6 sur les stratkgies d'exploration d'un hyperdocument. Sciences et techniques t!ducatives,4 (4), p. 413-435.
TIMOTHY KOSCHMANN, ALAN ZEMEL, MELINDA CONLEE-STEVENS, NATA P. YOUNG, JULIE E. ROBBS & AMBER BARNHART
HOW DO PEOPLE LEARN? Members' Methods and Communicative Mediation
Abstract. We are concerned with how learning and instruction are accomplished as interactional achievements, that is the practical details of how participants actually go about doing learning and instruction on a moment-to-moment basis. We focus in this chapter on what we term a problernatizing move, that is a form of social action that has the effect of calling something previously held to be so into doubt. Drawing conceptually and methodologically on Ethnomethodology and Conversation Analysis, we examine problematizing moves in two settings-a problem-based learning (PBL) tutorial meeting conducted face-to-face (F2F)and a distributed PBL (dPBL) meeting mediated through a chat interface. We note that even with the constraints on communication imposed by the mediating technology, the methods employed by members to problematize a problem resemble those seen in F2F meetings. We argue that contrasting members' methods across settings employing different forms of communicative mediation can be instructive with regard to understanding both the effects of the mediation and the nature of the methods themselves.
1. INTRODUCTION
A book recently published by the National Research Council (Bransford, Brown, & Cocking, 1999) bears the hopeful title, "How People Learn" and provides a summary of research in psychology and education over the last century. Missing from this account, however, is a careful description of how learning is actually accomplished as an interactional achievement within instructional settings, that is, how participants in such settings actually go about doing learning. Presumably, competent participants in such social scenes (i.e., members) have practical procedures (i.e., methods) by which they recognize and display that learning is being done. Ethnomethodology (EM) is a discipline that takes up such matters as its topic of study. The eponymous methods studied by ethnomethodologists might be described as "sense assembly procedures" (Garfinkel & Sacks, 1970, p. 343) in that they are the means by which members make sense of the world around them and their role in it (Garfinkel, 2002; Heritage, 1984). Members' sense-making methods constitute a repertoire of otherwise taken for granted social practices. They provide the basis for our common sense understandings of the world and are the means by which the actions of others acquire their definite sense. Because they are taken for granted, however, they have tended to fall outside the purview of traditional educational inquiry. This oversight, we argue, has served as a barrier to understanding the interactional practices through which instruction is produced and has biased our conceptualisation of learning itself, treating it as an exclusively psychological matter. In a project that has now spanned more than a dozen years, we have been engaged in studying how members do learning in a particular setting,
266
KOSCHMANN ET AL.
problem-based learning (PBL) tutorials in an undergraduate medical education curriculum. Problem-based learning has been described as a "collaborative, casecentered, and learner-directed method of instruction" (Koschmann, Kelson, Feltovich, & Barrows, 1996, p. 96). Though this description nicely specifies the instructional principles upon which the method is based, it provides little insight into the practicalities of how PBL is actually produced in the nonce. Our research has been designed to address the question, what methods do members use to constitute an activity recognizably and accountably as PBL. Numerous reports have come out of this project (cf., Koschmann, Glenn, & Conlee, 1997, 2000; Koschmann & LeBaron, 2002; Koschmann, Zemel, Conlee-Stevens, Young, Robbs, & Barnhart, 2003; Glenn, Koschmann, & Conlee, 1999; Glenn & Koschmann, in press) and our descriptive approach has evolved over time. Early on, we focused on the talk leading up to the production of learning issues (Koschmann et al., 1997). When students come to recognize a deficiency in their understanding of some matter related to a case, they may take up the matter as a learning issue (LI) to be independently researched following the tutorial meeting (Barrows, 1994). Learning issues are recorded on a whiteboard, so it is a relatively easy matter to trace backwards through the stream of interaction to locate where the recorded issue was initially raised. We termed the intervals of talk leading up to the production of a learning issue Knowledge Assessment Segments (KASs). In communication studies, a segment refers to an intermediate-level unit of discourse, larger than a two- or three-utterance sequence such as the extensively documented IRE sequences (c.f., Sinclair & Coulthard, 1975; Mehan, 1978; Cazden, 1988; Lemke, 1990; Fox, 1993; Wells, 1999) and smaller than episodes of talk such as lessons, classes, or tutorial meetings. We defined a KAS as, "a topic-delimited segment of instructional discourse in which participants raise a topic for discussion and one or more members elect to display their understanding of that topic" (Koschmann et al., 1997, p. 2). The production of a LI is predicated on three conditions: "there must be a recognizable knowledge deficiency, the students must see the missing knowledge as relevant to or necessary for the eventual practice of medicine, and, finally, there must be consensus about the timeliness of undertaking the study" (pp. 1-2). There remains, of course, the task of providing a more practical account of how this is routinely accomplished in interaction and this was our focus of initial study. As we became more deeply involved in this work, however, we began to find our initial analytic strategy to be too restrictive and our labelling to be unsatisfactory. Though the production of learning issues is a crucial aspect of the method, there is much, much more going on in these meetings than simply producing a list of items for later study. We also began to find the notion of assessment to be a bit problematic. Though something very much like assessment is entailed in producing a LI, it doesn't necessarily always occur whenever participants display an understanding with respect to some matter. The term assessment also has two
H O W DO PEOPLE LEARN?
267
usages potentially relevant to tutorial discourse. In educational psychology, assessment usually denotes measurement of a subject's understanding using some pre-defined instrument. In communication studies, on the other hand, assessment is used to describe action, vocal or otherwise, that is treated interactionally as a display of judgement with regard to some matter (cf., Pomerantz, 1984a; Goodwin & Goodwin, 1987). To dispel any possible ambiguity and also to mark a broadening of the scope of our analytic interests, in Koschmann et al. (2000), we adopted the label Knowledge Display Segments (KDSs) to describe our object of study. In so doing, we no longer limited ourselves to studying intervals of talk that resulted in the production of a LI, but instead sought out fragments in which one member raised a topic for discussion and others elected to display their understanding(s) with regard to the that matter. We focused in particular on the tutor's role in facilitating the production of KDSs. In related work (Glenn et al., 1999; Glenn & Koschmann, in press), we examined how diagnostic theories were occasioned within tutorial meetings. The two activities, learning issue production and theory generation, both involve a display of understanding and both are a collaborative achievement. In Koschmann and LeBaron (2002), we introduced the notion of learner articulation as an analytic category that brings together all of these elements (i.e., knowledge assessment and theory generation, knowledge display and the integration of understandings across participants). Following on Koschmann et al. (1996), we defined learner articulation as having two aspects: "the act of giving utterance ... to force a cohesive explanation" and "the action of jointing or interrelating of concepts and relationships" (p. 93). The second aspect of learner articulation highlights the interactional nature of knowledge display-the need to produce collective understandings by negotiating expressed individual understandings. We argued that learner articulation is not simply a lexical matter but also an embodied phenomenon and we extended our analysis to include the ways that participants used their hands, bodies, and material environments as resources in joint sense-making. Our analytic strategy was one of identifying instances in which speakers employed gestures coupled with talk to display their understanding and create connections to the displayed understandings of others. Learner articulation provides a broad and useful means of conceptualising what participants in PBL tutorials are doing, but it falls short of serving as a member method by which the activity of PBL is recognizably produced. Learners gather in tutorials not necessarily to articulate understandings so much as they gather to engage in a form of joint problem solving. While learner articulation may be occurring, therefore, it is not the object of the activity per se from the perspective of the participants. To understand the activity as problem solving, therefore, we needed to go beyond learner articulation to understand how problems themselves are produced as interactional achievements. Koschmann, Kuutti, and Hickman (1996) observed that many current theories of learning assume a precipitating breakdown in understanding as a starting point for learning to occur. For the American pragmatist philosopher John Dewey, for example, learning is initiated in situations that have turned problematic. Dewey (193811991) defined inquiry as the process whereby the problematic aspects of the situation are ultimately resolved. He made clear, however, that he was not talking
268
KOSCHMANN ET AL.
about problems as posed instructional exercises. Dewey wrote, "A problem is not a task to be performed which a person puts upon himself or that is placed upon him by others" (p. 111). Instead, it is a matter for learners themselves to discover. For that reason, Dewey described inquiry itself as "a progressive determination of a problem and its possible solution" (p. 113). Koschmann (2001) argued that a key contribution of PBL is that it affords a setting rich in opportunities for doing inquiry in precisely the sense described by Dewey. Dewey's useful conceptualisation, however, does not explain how the "progressive determination of a problem" is actually accomplished interactionally. In more recent work (Koschmann et al., 2003), we have attempted to address this issue by documenting how some matter comes to be treated as problematic by participants in a PBL tutorial meetings. We focused in particular on what we termed the problematizing move, that is a form of social action (e.g., utterance, gesture, facial expression) that has the effect of calling into doubt some matter previously taken as so. Problematizing moves perform two kinds of work: they direct attention to some potentially problematic matter and, at the same time, project some form of collective action with regard to that matter. A problematizing move, however, does not in and of itself produce a problem in the sense described by Dewey until it is taken up as such by all the parties to the conversation. In prior writing (with the exception of Koschmann et al., 2003) we have focused on the methods members employ in face-to-face (F2F) tutorial meetings to negotiate mutually-endorsed understandings and evaluate the adequacy of these understandings. In our most recent work, we have begun to examine alternative models of PBL, models in which the constraints that tutorial participants convene in the same place or even at the same time are relaxed. Such approaches are sometimes described as "distributed" (Koschmann, 2002) versions of PBL to differentiate them from the more conventional, F2F variety. To do distributed PBL (dPBL), some form of mediating technology is inevitably entailed. A logical extension of our work done to date might be to ask how is problematizing accomplished in non-F2F PBL tutorial meetings? In this chapter, therefore, we will carefully examine the methods employed in problematizing some matter in a dPBL meeting mediated through a computer-based, textual (chat) interface. Before doing so, however, we will re-examine a fragment of interaction in a F2F tutorial meeting that has been described in a number of earlier publications (Koschmann et al., 1997, 2000; Koschmann, 2001; Koschmann & LeBaron, 2002) this time focusing on the methods employed to problematize some matter. We will conclude by discussing how all of this might be related to overcoming some of the biases and barriers to better understanding how learning occurs in settings of collaboration and the role of technology therein. 2. DATA 2.1 Fragment 1: "What would be the risk?" 2.1.1 Setting PBL involves small teams of students (five or six is considered optimal) working with a faculty tutor who learn in the process of working through a collection of
HOW DO PEOPLE LEARN?
269
clinical teaching cases. Within tutorial meetings, data are gathered, hypotheses generated and tested, and conclusions drawn in an interactive manner similar to that used by medical practitioners. Curriculum designers use various means to simulate encounters with patients. Trained patient surrogates, known as "standardized patients" (Barrows, 1987), are used for some cases. For others, a paper-based simulation, the Problem-Based Learning Module (PBLM), was designed to allow for free inquiry, providing responses for any question, examination, or laboratory test a physician might request for an actual patient (Distlehorst & Barrows, 1982). In the implementation of PBL that we studied, the tutor guided students through their discussions of teaching cases by modeling reflective reasoning Information and ideas that emerged during discussion were organized and recorded on a whiteboard by a student designated the scribe. The whiteboard was sectioned to hold information of specific types--emerging evidence with regard to the case (Data), hypotheses of underlying cause (Ideas), matters for further study (Learning Issues), and developing plans for future inquiry about the patient (Actions). In the fragment analysed here, the medical students (all identified by pseudonyms) were enrolled in the second-year of the problem-based learning track. The tutor ("Coach") was highly experienced and widely recognized for his skill in teaching in collaborative settings. During the segment of talk represented below, these students were in the end of a Gastrointestinal/Endocrine/Reproductiveunit that occurred late in their second year of medical school. The group was introduced to this case by performing a history and physical on a standardized patient. The interval of talk analysed here took place during a follow-up meeting and the students were engaged in the final stages of processing the patient problem. They sat at a table containing notes, textbooks, and materials relevant to the case under discussion. The group used a white board to record notes pertaining to the patient's status, diagnostic theories, and learning issues. At the time the fragment begins, the students have collected most of the information usually derived from the patient history and physical and they are discussing possible lab tests.
2.1.2 The Case The case was authored by a medical school faculty member and was based on actual clinical records. It involved a 32-year old female patient (L.M.) who presented with a complaint of abdominal pain of two weeks duration worsening in the last two days. This was associated with chills, nausea, and malaise. On questioning, the patient also revealed experiencing a burning sensation on urination and a small amount of yellowish vaginal discharge. An abdominal exam revealed right lower quadrant tenderness and the pelvic examination indicated cervical tenderness. During the pelvic examination, cultures of the cervix were taken which revealed infection with Chlamydia, a sexually transmitted bacteria that can commonly cause pelvic inflammatory disease (PID). Pelvic inflammatory disease is a consequence of untreated sexually transmitted diseases of the lower genital tract. It is postulated that the infection ascends to the upper genital tract (uterus, fallopian tubes and ovaries), causing clinical symptoms of fever and abdominal pain. Advanced cases of PID can result in abscesses in the fallopian tubes or ovaries; these may not respond to
270
KOSCHMANNET AL.
antibiotics and may require surgical drainage of the abscess. An aspect of the final sequelae of PID is that scarring may occur after infection and inflammation resolves, which can produce infertility. As the fragment begins, the group was discussing how to further investigate L.M.'s abdominal pain using radiographic imaging, given their hypothesis of PID. Computerized axial tomography (CT) scans are often performed on patients with abdominal pain to look for causes of the pain. In patients with PID, CT scans are useful to look for complications such as abscesses, and reactive inflammation of adjacent organs. CT scans employ a thin beam of X-rays that are generated to pass through the patient's body; the radiation that passes through is picked up on the other side by an electronic radiation detector. Since the x-ray tube and the detector are mounted on opposite sides of a circular gantry, rotating the gantry as the measurements are being made every few degrees and using a computer to reconstruct that image will result in a cross-sectional image of that part of the patient. The amount of radiation patients are exposed to in an abdominal CT scan is around 10 mSv (millisieverts). The millisievert is a scientific unit of measurement for radiation dose. This is around 3 times the amount of radiation the average person in the U.S. receives each year from naturally occurring background radiation (from radon, cosmic radiation etc.). This is much more than the amount of radiation exposure from one standard plain film chest x-ray (0.lmSv). Different tissues and organs have varying sensitivity to radiation exposure, for example, organs that contain more rapidly dividing cells or the rapidly dividing cells in an embryo or fetus are more sensitive to harm from radiation exposure. In this case, there is a question of whether or not L.M. might be pregnant. The amount of radiation from an abdominal CT scan might be harmful to a developing fetus, and most physicians would prefer to avoid exposing the fetus to that risk if at all possible. A glossary of some of the medical terminology related to the case discussion can be found in Appendix D. 2.1.3 Analysis In describing the activity seen here as problem-based learning we must be clear about what we are taking to be "the problem." There are, in fact, an assortment of problems to consider. There is the problem that originally brought the patient into the physician's office, that is the patient problem. As a case, this encounter with the patient represented a clinical problem for the attending physician. When the case is selected and used for instructional purposes, it becomes a teaching problem. As the tutorial members work through the details of the case, they may display incomplete, discrepant, or otherwise inadequate understandings which, when taken up, are treated as problems of understanding. There are forms of interactional work involved, however, in producing problems of understanding and we were interested in documenting the methods members use to accomplish this. Ethnomethodology defines a topic of study (i.e., members' methods) and a locus (i.e., within practical activity), but does not specify any particular methodology by which it should be studied. Conversation Analysis (CA) is an area of specialization within ethnomethodological research that focuses on the methods members use in
HOW DO PEOPLE LEARN?
27 1
interaction to make sense of each other and to, in turn, be seen and heard as sensible (Heritage, 1984). CA furnishes a methodological framework for the rigorous study of unfolding interaction (c.f., Psathas, 1995). Studies of member methods always begin with extensive fieldwork. It is difficult, however, to study the details of conversation in situ as it is being produced. Observational notes, compiled in the field, are often not an adequate basis on which to construct an analysis. Conversation analysts, therefore, usually work with detailed transcripts built from field recordings, either audio or video. In this study, we have both incorporated findings from the CA literature and employed CA methods and transcription conventions. Our analytic strategy in this study was to collect examples of what appeared to be problematizing moves, that is actions that have the demonstrated effect of (1) indexing some matter as potentially problematic and, at the same time, (2) marking it as actionable, i.e. projecting some form of collective action with regard to the matter at hand. Analysis of the materials gathered in the field was done in phases of individual and group effort. Preliminary analyses were presented in collaborative "data sessions" to extend and enrich the analysis presented here. In describing what might count as "ethnographically adequate description," McDermott, Gospodinoff, and Aron, (1978) argued that analysts must provide readers with sufficient ethnographic material to enable the reader to independently evaluate the proposed analytic claims. To this end, we provide in Appendix B a transcript of the full exchange upon which our analysis was based. Prelude to a problematizing. The fragment begins with one of the students, Joel, making a proposal based on prior reading for how the case should be assessed. Patrick immediately offered a concern regarding the proposal: Excerpt 1.1 1 Joel: 2 3 4 5 Patrick: 6 7 Jackie: 8 Patrick:
They did talk about doing a CT along with along with it if you feel there are abscesses but it's low yield unless you feel there are abscesses= =Think you can [get can a lot of risks doing a CT 1= L ~ n dan ultrasound can be used for that reason] = to the pelvis.
Patrick's utterance indexes a potential problem ("a lot of risks") but lacks specificity. Offered as a question, Joel i.esponded to it as such and then asked Patrick to better articulate his concern ("What would be the risk?'). Jackie (lines 13, 15) interjected that there would only be a risk under certain circumstances. Patrick reformulated his "risks of doing a CT" to the "danger of X-raying the ovaries and that." There was difference of opinion, therefore, as to whether there was a safety concern at all and, if so, under what circumstances. If it constituted a problem of understanding, however, it was not one for which the group had yet established a sense of collective ownership. At this point, Coach intervened:
KOSCHMANNET AL. Excerpt 1.2 20 Coach: 21 Jackie: 22 23 Patrick: 24 Joel: 25 Jackie: 26 Patrick: 27
Is there a ?risk to CT? OHmO (0.9) Sure. l t l s an ?x-[ray. L~erah h t ' s an X-ray, there's always a risk to an X-ray.
Coach's question was produced with an accompanying gesture similar to that of a crossing guard stopping traffic. His question had a similar effect on the flow of the conversation-it momentarily arrested the discussion and set it in a particular direction. Because of his role as tutor, his question elevates Joel's concern to a matter warranting the attention of the collective. But the nature of the danger alluded to by Patrick had yet to be articulated. In the discussion that followed, Coach probed the students to get them to articulate their respective understandings, focusing on the differences between CT scans and X-rays. Joel, in formulating a position with respect to this matter, raised a question of his own: Excerpt 1.3 40 Joel: 41
.
?What is the dosage (0.4) relative ( ) from uh normal X- ray to a C T i
Joel's question reshaped the issue under discussion, from that of some largely unspecified risk associated with CT scans to the amounts of radiation received from CT scans and conventional X-rays. Jackie (lines 47-48) indicated that she did not remember the "relative dosage." After some additional discussion on the part of Coach, Jackie, Joel, and Patrick, Joel offered an answer to the question he had raised in the prior excerpt: EXCerDt 1.4 75 Joel : 76
I understand that the CT is ?just about equivalent to an X-Jray.
Joel's position, carefully qualified ("I understand that"), constitutes a display of understanding. In producing this formulation, Joel set the scene for a problematizing move. Problematizing the problem. Though Joel's display of understanding was performed by an individual speaker, its production was a collaborative effort. It built upon Patrick's original expressed concern and Coach's leading questions. Most directly it tied back to his own question in line 40-41. The problematizing move was also produced collaboratively: ExCerDt 1.5 75 Joel: 76 77 Coach:
I understand that the CT is ?just about equivalent to an X-Jray. Is it?
HOWDO PEOPLE LEARN? 78 79 80 81 82
Joel:
That's what my understanding &is I- I'm not ) r ~ * just m saying ( Melissa: LWhy don't we just put it up as a learning issue. Joel: >Let throw that [up< Jackie: L~eah.
Coach's query directs attention to Joel's expressed understanding. This move has particular force owing to Coach's recognized expertise and his role as a tutor. It assesses Joel's understanding without providing an explicit assessment. Schegloff, Jefferson, and Sacks (1977) described how participants in a conversation display a preference for speaker self-repair over listener correction. In this case, Coach's question creates a space for Joel to do self-repair without specifying exactly what needed to be repaired or offering an alternative formulation. Joel's response acknowledges that his prior formulation has been challenged as potentially problematic, but rather than offer an alternative formulation instead reiterates his qualifying condition, "That's what my understanding is." This is an example of telling "how I know" (Pomerantz, 1984b). By formulating this as only an "understanding," Joel explicitly marks his proposal as defeasible. Coach's query is seen and heard as directed to Joel. This is made visible by the fact that Coach turns to look at Joel while issuing the question. It is also evident from the adjacent positioning of the query and the "cohesive" (Halliday & Hasan, 1976) deployment of anaphora. Despite the fact that the question is so marked as Joel's to answer, Melissa responds in overlap with Joel (line 80) and proposes that the matter be taken up as a LI. Melissa's "it" is subtly different from Coach's. Coach's "it" refers to Joel's displayed understanding, Melissa's to the underlying issue to which Joel's display constituted a response. Melissa's proposes uptake of this matter as a collective concern. It treated the matter as actionable and proposed a specific course of action. Joel and Jackie both ratified this proposal. Postscript to a problematizing. In response to Melissa's proposal, Alice rose from her seat and recorded "CT vs. x-ray risks" on the board as a LI. By this action, the group formally agrees to address the problematic issue by investigating it after the meeting. Though this would appear to resolve the matter for the purposes of the current discussion, Coach pushed it a bit further: Excerpt 1.7 83 Coach: 84 85 Joel: 86 87 Joel: 88 89 Coach: 90 91 Joel:
>I was going to say< how sure are you on a scale of zero to [ten. Lnot (0.6) Three? (0.8) Think we oughta make a ?learning issue out of it. r ~ h e hheh heh t ~ a(maybe we ought to)
Coach again withheld assessment of the correctness or incorrectness of Joel's original formulation (lines 75-76). Instead he spoke to the way in which Joel
274
KOSCHMANN ET AL.
qualified his answer ("my understanding is") and pressed Joel to quantify his level of certainty. Joel confessed that his level of certainty with regard to this matter was low. The students had now displayed an understanding (such as it was), had acknowledged that their understanding was incomplete, and had resolved to redress that deficiency. One might think that it was time to move onto new things, but Coach was apparently not quite finished: EXCerRt 1.8 92 Jackie: 93 Melissa: 94 Jackie: 95 Coach: 96 Jackie: 97 Coach: 98 Jackie: 99 Coach: 100 101 102 103 Jackie:
But we
know [that ultrasound isn' t radiation L (Joel) this is from the ( 1
Right?= =No, h l trasound isn ' t radira: tion. L~verybodyknows that. But if you're gonna get [hesitate what I ' m gonna I'say, doing a scan, because the woman might be preqnant then I think you oughta know if the CT scan is to be of concern or ?not when doing it on a pregrnant woman. L~i::~ht.
LSO
In this final excerpt, Coach provided a justification for his original question, "Is there a risk to CT?'(line 20). He offered an argument that the matter would be of importance to the students in their later clinical work. In particular, Coach is modelling a form of professional practice-whenever a clinician makes a decision based on some belief, it is necessary to assess the certainty with which the belief is held. 2.1.4. Discussion In Koschmann et al. (1997), we described this same fragment in terms of the work required to produce a learning issue. In Koschmann et al. (2000), we documented the tutor's role in this process. In Koschmann and LeBaron (2002), we examined the ways in which the participants in the 'What would be the risks?' fragment used gesture in articulating their understandings. Finally, in Koschmann (2001), this fragment was analysed as an example of Deweyan inquiry enacted. Here, we revisit it once again to document the work involved in producing a problem in understanding. Collaborative effort was required to build toward a display of understanding and collaborative work was needed to produce a problem of understanding. Coach's query (line 77) implicitly places Joel's expressed understanding in doubt. Like Patrick's initial expressed concern, however, Joel's problematized understanding was still an individual matter, since there was no evidence of group ownership. Melissa's proposal that it be taken up as a learning issue transformed it into a collective concern which was immediately ratified by the other members of the group. We defined the problematizing move as an action that has the "demonstrated effect" of (1) indexing some matter as potentially problematic and (2) projecting some form of collective action. In this case, the problematizing move was accomplished jointly by Coach and Melissa and evidence of uptake was provided by the other students through their endorsement of the matter as a learning
H O W DO PEOPLE LEARN?
275
issue. We now turn our attention to a problematizing move performed in a CMC discussion. 2.2 Fragment 2: "Salicylate toxicity" 2.2.1 Setting The students participating in this PBL tutorial were third-year medical students doing a clerkship rotation in Family and Community Medicine. The six-week clerkship begins with a short orientation and training period at the main campus, followed by a five-week preceptorship experience in which each student is assigned to a practitioner in the community. The students provide patient care in both ambulatory and hospital settings under the direct supervision of the preceptor. The preceptorships occur in different communities scattered across the state. Because most students enrolled in the clerkship are located off campus, it would be inconvenient to convene F2F tutorial meetings during their clerkship. 2.2.2 Mediating Technologies Students in the Family and Community Medicine clerkship are supplied with laptops and have dial-up access to the Internet at their preceptor sites. Weekly PBL tutorial meetings are conducted using off-the-shelf CMC software (Microsoft NetMeetingTM). NetMeeting offers a chat facility as well as a shared whiteboard. Groups of four to six students interact with a faculty tutor to explore selected patient cases and identify learning issues. Between weekly meetings, students independently investigate these issues from their preceptor sites. Students post information gathered about the learning issues to a discussion forum using another off-the-shelf collaboration tool, WebCT. When the group reconvenes the following week, the learning issues are reviewed, the case is closed, and a new case is opened. Interaction among participants was constrained by the way the technology was organized for use. Typical steps in posting a message were as follows: (1) A participant composed a message by typing it into the message field provided by the chat software. (2) Once a participant completed a message, he or she posted the composed text to a server for distribution to recipients. (3) Once the message was received by the server, it distributed the message to recipients at different locations. The posting and distribution of messages in this manner created certain interactional difficulties for participants. For example, a message distributed by a server may have been received at a participant's computer while the participant was engaged in typing his or her own message, while reading a previously posted message or while doing anything else, as long as the computer was connected to the chat session. Because participants did not see messages as they were being composed, something like an overlap could occur between the receipt of a sender's message and a recipient's actions. In such cases, the sender of a message did not have access to the activities of recipients while the message was being composed or sent, or knowledge of when or if recipients actually received his or her message. How participants oriented to and responded to the arrival of a new message is an important interactive
276
KOSCHMANN ET AL.
feature of chat room interactions (cf., Herring, 1999). In a recent paper, Garcia and Jacobs (1999) observed that the organization of turn taking in computer-mediated communication is different from that in spoken conversation. They note, for example, that the notion of "transition relevance place," so central to the Sacks, Schegloff, and Jefferson's (1977) "simplest systematic" for coordinating turns at talk is lost in what Garcia and Jacobs describe as "quasi-synchronous CMC" (p. 339). They further contend that the meaning of pauses and the methods available to participants for self-repair are also different in this mode of discourse. We made certain presumptions about the messages posted and received in these tutorial meetings. Previously posted messages were resources to which all participants in the chat had access. This does not presuppose that all messages were in fact relevant to each recipient or were resources to which recipients necessarily oriented in the production of their own messages. It only suggests that posted messages constituted a corpus that participants could use as resources in the production of subsequent messages. Each posted message was designed to be intelligible to recipients. This may seem a bit obvious, but we expand the sense of intelligibility to include, in addition to specific content, some sense of the intended recipient(s) as well as the relation a given message may have had to previously posted messages. From this presumptive base, it is possible to inspect posted messages as they were received at the server and draw some tentative conclusions. As the following excerpts show, even in a quasi-synchronous electronic chat environment, clinical problems and learning issues, which are central to problembased learning approaches (Barrows, 1994), are not given ready made but are jointly negotiated and socially-organized achievements. 2.2.3 The Case The case under consideration concerned a patient, R.W., a 55 year-old farmer, who was brought to the Emergency Department by ambulance after his wife found him wandering, cold and confused, outside their home earlier that morning. In the ambulance, an intravenous line was started, R.W. was given oxygen, and his blood sugar level was checked and found to be elevated. R.W. had no prior history of diabetes (high blood sugars). On physical examination, he was noted to be breathing with very deep breaths, despite not complaining of any shortness of breath or difficulty breathing. This type of breathing is termed "Kussmaul's respirations" and is associated with conditions that cause an imbalance in the body's pH. The students ordered a laboratory test, termed arterial blood gases (ABG), that measures the pH, oxygen and carbon dioxide content in the arterial blood. Two such tests had been done on R.W., one when he arrived at the emergency room and another a few hours later. Since R.W. was noted to have high blood sugar levels (glucose) and had Kussmaul's respirations, students had initially hypothesized he might have a condition called diabetic ketoacidosis (DKA). This is a complication of uncontrolled diabetes that results in the disruption of normal aerobic metabolism, and a switch to anaerobic metabolism. Normal pH is around 7.4. Anaerobic metabolism, however, produces ketone bodies which can ultimately turn the blood pH acidic. An ABG reading made when the patient was admitted to the hospital, however, showed a
HOWDO PEOPLE LEARN?
277
slightly alkalotic pH of 7.44. R.W. had been taking large amounts of prescription and over-the-counter painkillers. The students, therefore, also ordered a toxicology screen to see if R.W. had toxic levels of acetaminophen or aspirin (salicylates). The test indicated that R.W. had toxic levels of salicylates in his bloodstream. An overdose of aspirin may initially cause the blood pH to appear normal because of respiratory alkalosis and a metabolic acidosis. As the toxicity progresses, however, a patient with salicylate toxicosis is unable to compensate for the metabolic acidosis by lung or kidney mechanisms and severe acidosis results. Indeed, this was what was seen in the second arterial blood gas measures for R.W. It is at this point that we begin our analysis of the interaction. While there was agreement among participants that a severe acidosis was occurring, one participant remained uncertain as to whether the change was due to primarily metabolic or respiratory mechanisms. Clinically, it is important to understand which mechanism is responsible for the body's acid-base imbalance because treatment depends on first normalizing the pH, then on correcting the primary cause of the imbalance. Because of this, the plans for treating R.W.'s problem would differ depending on the primary mechanism involved. 2.2.4 Analysis As in the 'What would be the risk?' fragment, our interest here is in members' methods for problematizing a matter, though in this case in a computer-mediated tutorial meeting. The same analytic strategy was employed for collecting examples of problematizing moves, that is actions that have the demonstrated effect of (1) placing some matter into doubt and (2) projecting some form of projective action with regard to that matter. Interaction is automatically transcribed in CMC, simplifying the work of the analyst. The transcript so produced, however, is not like a conventional CA transcript, such as the one found in Appendix B. Conventional CA transcripts capture details of delivery (e.g., timing, intonation) not relevant to CMC. Other aspects of a conversation analytic approach to studying interaction do remain relevant, however. A shared objective for all CA studies is to give an account of the order and placement of each contribution to a conversation. An omni-relevant question is, why this and why here? Sequential organization is a crucial resource for meaning making for members and analysts alike. This is true not only of F2F conversation, but also of computer-mediated discourse. The meaning of any particular utterance/contribution/message can be displayed, acknowledged, and, in some cases, shaped in the unfolding sequence that follows it. As Heritage (1984) described it: [C]onversational interaction is structured by an organization of action which is implemented on a turn-by-turn basis. By means of this organization, a context of publically displayed and continuously up-dated intersubjective understanding is systematically sustained. (p. 259)
This is also true of messages produced in computer-mediated exchanges. Because of this, the methods for studying the sequential organization of talk in F2F conversation are also useful in studying computer-mediated exchanges. Other
KOSCHMANNET AL.
278
researchers have already employed CA methods in studying CMC (cf., Arnseth, Ludvigsen, Wasson, & Morch, 2001; Garcia & Jacobs, 1999). A transcript of the complete fragment can be found in Appendix C and a glossary of some of the medical terms and abbreviations can be found in Appendix D. Putting forward a candidate matter of concern. In the first excerpt, a participant put forward a matter of concern to her and solicited consideration of this matter from other participants in the PBL tutorial. As is evident from the transcript, participants resisted its uptake as a matter of concern for the group. Excerpt 2.1 1 3:40:58 2 3
T:
S1: S2 :
S5 :
S4 :
I was impressed with the depth and sources you had for your posted learning issues. What did you think of what others posted? nice job everyone. Did everyone find mine? everyone did a good job but I still can't really figure out why the initial ABG was alkalotic- can anyone help me out I think everyone did a good job . . . I certainly enjoy reading posts like Sl's, where the info is related back to the case. Strong work Sl! lab error, S2 ;)
In lines 7 through 10, S2 used the tutor's invitation to evaluate previously posted learning issues (lines 1 through 4) as a way of introducing a matter for the group to consider. S2's message oriented to the tutor's invitation by offering a cursory evaluation of previously posted learning issues followed by an initial formulation of a matter as problematic. S2's actual formulation of the problematic matter is notable for a number of reasons. First, S2 localized the problem as a deficit in her knowledge or understanding: "I still can't really figure out why the initial ABG was alkalotic" (lines 7 through 9). In asking for assistance from the group as a whole, she constituted members of the group as resources that were to be called upon to help her, i.e. "can anyone help me out" (lines 9 and 10). This formulation of the query implied that recipients were able to understand the nature of the problem presented by S2, assess the extent and limits of their own and S2's understanding of the problem based on the formulation of the query, and respond either to resolve the issue or to take up the matter for further investigation for how it might lead to the formulation of a clinical problem. Second, the laboratory finding is problematic because the group has previously committed to a diagnosis of DKA, but S2 did not explicitly implicate this diagnostic theory in her message. This excerpt ends with S4's post at line 15 which displayably oriented to S2's request since it was explicitly addressed to S2 and constituted the first identified uptake of S2's problem. However, the formulation of the post suggests that S4 was actually downgrading the significance of S2's message by invoking "lab error" as
HOW DO PEOPLE LEARN?
279
the explanation and then offering the gestural "wink" marker. S4's message was framed as a possible reason for "why the initial ABG was alkalotic" but the wink marker, i.e. ;), suggested that the response was not to be taken seriously. This "joking" response was in the form of a candidate resolution of the difficulty. However, the wink marker indicated to S2 and to other recipients that the post was not to be taken as a serious response to the query but was to be seen as a comment on the query itself, implying that S2's query was of a sort that warranted this kind of non-serious response. S4's downgrade also could be seen to imply that responses from other group members were not warranted. S4's downgrading response treated S2's problem as a problem that did not warrant uptake by the group. Thus we see that uptake of a member's issue by the group as a whole is not always a simple matter. Two possible explanations for the group's reluctance to take up the matter are suggested by the postings. First, the fact the S2 tried to divert the group from attending to its own accomplishments could be seen as abrupt and unsympathetic to the effort others expended in producing previously posted resolutions of learning issues. The resistance of the tutorial group to abandon its concern with its own accomplishments also suggests that these accomplishments, incremental or partial though they may have been, were very important to the way the group understood itself. It could have been very difficult for this group to maintain itself as a group without an acknowledged sense of accomplishment. A second reason that might account for the group's unwillingness to take up S2's question in a serious way, and one that constituted S4's response, may be that an anomalous lab result will not always warrant further scrutiny. Perhaps S2 had not provided enough information about the possible consequentiality of the anomaly to pique the interest of other members of the group or make evident its nature as a problem. We hold as a matter of course that for an anomaly to be taken up as a problem, it needs to be recognized by the participants themselves as significant, relevant and problematic for the entire group. In other words, the fact that a tutorial member did not understand an anomalous laboratory result did not necessarily mean that the laboratory result was problematic for the group. The problem could have been attributable to the tutorial member herself. Of course, a combination of these reasons could also account for the observed reluctance of members to take up S2's request for help. The problematizing move. The response S2 received suggests that a matter of concern to an individual participant was not always considered a matter of concern for the group. Thus, in order for S2's concern to be taken up by a group of this sort, additional work needed to be done. In the next excerpt, we see evidence of the work done to transform S2's initial query into a matter for the entire tutorial group to investigate. This involved a reformulation of the initial query and its subsequent uptake by other members of the group. As the transcript demonstrates, the tutor's endorsement of the matter contributed in a large way to the group's uptake of the matter.
KOSCHMANNET AL.
S1: S1: S2 : S1: S1: T: S5:
you points S5. you won points does everyone think we're dealing with DKA? I thought S6 may have been excessive yes, DKA I posted the initial and repeat ABGs, what do all think of S2's first and second question? I'm a little confused by that pH as well S2.
When there was no subsequent uptake of her initial inquiry by any other members of the tutorial group, S2 produced a second query. S2's second question makes explicit the matter being placed in doubt (the diagnosis of DKA). By addressing her question to "everyone," S2 elevates the matter to one of collective concern. Her message was formulated in a way that called on recipients to weigh in either in support of or in opposition to the proposed candidate cause (see Pomerantz, 1984b). 2.4.3 Uptake of the matter by the group. Uptake is an artful, interactional achievement and is often delicately done, especially if it had been resisted previously by members of a group. In this case, we can see how uptake was done in ways that also displayed specific alignment with S2. Excerlst 2.3 22 3:44:18 23 24
I posted the initial and repeat ABGs, what do all think of S2's first and second question? I'm a little confused by that pH as well S2. salicylate toxicity causes a=resp alkalosis initially he has over compensated. now is entering resp. alk. I sort of think this is a mixture of ASA toxicity and DKA . . .can that be possible? So is it mixed metab acidosis and resp alkalosis? yes yes yes ...and salicylates can cause this
As in the previous fragment we see an example of the faculty member pursuing a "teachable moment." The tutor here first oriented the attention of the group to the anomalous lab finding, posted on the NetMeeting whiteboard. By introducing additional clinical data, the tutor endorsed Sl's uptake of S2's problem of understanding, thus indicating this was a matter for the group to take up.
HOW DO PEOPLE LEARN?
28 1
Once the tutor had endorsed S2's queries by posting additional clinical data, she then called on all participants for an evaluation of S2's queries (lines 22 to 24). S5 addressed a message explicitly to S2 (lines 25 to 26) in a way that specifically aligned with S2 and the tutor's endorsement of the anomaly S2 had identified. By addressing his message to S2, S5 may have been doing work to explicitly display alignment with and to further endorse the uptake of S2's queries for how they stood for his own problem of understanding the matter and thus pointed to possible problems for the group to take up, especially given the difficulties S2 had encountered with respect to getting the group to take up her queries. Other members of the tutorial group took up discussion of the anomaly once these endorsements from F and S5 were made available to the group. S4, S1 and S5 engaged in an interchange in which various ways of accounting for the clinical anomaly were put forward (lines 27 through 34). With these possible candidate causes for the observed anomaly now available to members of the group, the tutor called on the group to weigh in with respect to a particular candidate cause that would account for the observed anomaly (lines 33 and 34). What follows is a series of alignments in lines 35 through 37. S4 did more than just align with the tutor's position, however. S4 expanded on that alignment by citing a specific basis, i.e. "salicylates", for the candidate cause put forward by the tutor. Building on this, the tutor queried the group to describe salicylate toxicity and this became a new topic of discussion.
2.2.5 Discussion There are many differences between the 'What would be the risk?' fragment and the 'Salicylate toxicity' fragment and not all are due to the mediational circumstances under which the described events were conducted. In the latter fragment, the students are at a later stage in their training and the discussion is more sophisticated. More is left to inference and the matter in question is much more complex, having to do with dynamic processes in multiple, unfolding systems. In the first case, considerable work went into formulating a display of understanding before the problematizing was undertaken. In the second fragment, the matter being placed in doubt was first implied and only named as the problematizing move was produced. In 'What would be the risk?', the matter problematized became a learning issue for the group; in 'Salicylate toxicity,' the problem of understanding became a catalyst for a different form of collective action-the reformulation of a diagnostic theory. In both cases, however, a similar form of work was required to produce a problem of understanding. That is, in both cases, some matter being called into doubt was referenced by the participants and some form of collective action was projected and undertaken. As the analyses suggest, even in highly mediated forms of interaction that are constrained by the affordances of the technology used, members of PBL tutorials need to enlist the interest, involvement and support of other members of the group to elevate a matter from a personal or idiosyncratic problem of understanding to a problem of understanding for the group as a whole. As with the face-to-face interaction examined earlier, the work done to get participants in distributed learning to take up a matter as a collective problem of understanding involved calling on
282
KOSCHMANNET AL.
participants to take positions with respect to the matter at hand. When taking positions, members displayed the scope and limitations of their understanding of or knowledge about the matter at hand. The surprise finding, therefore, is that even with the documented barriers to communication imposed by the mediating technology (e.g., the lack of "coherence" [Herring, 19991, the absence of prosodic and gestural cues, disruption of ordinary turn-taking strategies as documented by Garcia & Jacob [1999]), the interactional methods employed by members to problematize a problem (at least with respect to the initial steps of establishing a problem) resemble those seen in F2F meetings. This is an important finding and one with important implications for instruction. It should, for example, come as welcome news for those who would like to pursue the use of PBL in distributed environments. As with the face-to-face interaction examined earlier, this interaction showed a clear orientation among participants to learner-directed instruction. In fact, it was a student who put forward a matter of concern to the group and it was the same student who persisted in putting the matter forward when other students were reluctant to address the matter. The tutor in this instance worked with the students to allow them to work out for themselves the relevance of the matter. An important feature of the tutor's work involved withholding explicit evaluation of student responses to S2's queries. By withholding her evaluation of the responses to S2's queries, the tutor provided the students with the opportunity to develop their own positions with respect to matter S2 put forward. By explicitly calling on students to develop their own positions with respect to S2's queries and Sl's response, the tutor could be seen to be implicitly calling on the other students to offer their own evaluations. Rather than enact a conventional IRE sequence (c.f., Sinclair & Coulthard, 1975; Mehan, 1978; Wells, 1999) in which the teacher puts forward an inquiry, the student responds, and the teacher evaluates the response, it was the students themselves who produced all three components. By making students accountable for the evaluation of the responses they themselves put forward, the tutor displayed an orientation to the interaction as learner-directed instruction. Likewise, by accepting that accountability within their interaction, both the students and the tutor constituted their work as learner-directed instruction.
3. BARRIERS AND BIASES TO UNDERSTANDING COMPUTER-MEDIATED INTERACTION We begin from the premise that the greatest barriers to understanding instructional interaction are conceptual and methodological. This applies both to instruction carried out F2F and under distributed and technologically-mediated conditions. The disabling bias in educational research has been and continues to be in treating learning and instruction as exclusively psychological matters. Instruction is produced through the joint interactional work of teachers and learners. It stands to reason then that if instruction is, first and foremost, an interactional matter, then it should be studied as such. Further, if learning is something that is produced in and by instructional discourse, then evidence that learning is being done should be made available within the same discourse. Herein lies the importance of documenting
HOWDO PEOPLE LEARN?
283
members' methods of sense making in instructional settings, for it is in and through these methods that instruction and learning are both achieved and produced. The PBL literature consists almost exclusively of outcome studies designed to determine whether or not PBL works (Albanese, 2000; Colliver, 2000). These assessments of efficacy, however, take the treatment variable (i.e., the instructional intervention) as a given. No one to date has undertaken the difficult task of documenting the actual practices by which PBL is recognizably and accountably produced (Hak & Maguire, 2000). The work described here endeavours to remedy this omission. As such, it represents a departure from prior research on PBL. As our studies shift from PBL in F2F settings to distributed implementations of PBL, we have begun to examine members' methods for producing PBL in tutorial meetings mediated by technology. In this way our work speaks not only to what it means to do PBL, but also represents a novel approach for studying the effects of communicative mediation. Rawls in her introduction to Garfinkel's Ethnomethodology's Program (2002), wrote, "methods used by persons to create the orderliness of ordinary social occasions are constitutive of those occasions" (p. 6). She went on, "Constitutive practices are those which can only meaningfully exist if they are made recognizable by those who practice them" (p. 6). Garfinkel, in establishing EM as a new field of inquiry, took understanding of the orderliness of ordinary social occasions as the central question. Rawls, in fact, defined Ethnomethodology as, "the study of the methods people use for producing recognizable social orders" (p. 6). Documenting the methods by which recognizable orderliness is produced is by no means a simple matter, however. As the overview of our earlier work provided in the introduction would attest, an adequate characterization of members' constitutive practices can be elusive. The evolution of our descriptive framings from Knowledge Assessment Segments to problematizing moves reflects our ongoing struggle to produce a usable account of what participants in a PBL tutorial are actually doing. Numerous challenges lay ahead. As we have described, the task of producing adequate description of the interactional practices by which members produce instruction is neither trivial nor straight-forward. We also have yet to develop a practical methodology for applying the findings of descriptive research to pedagogical design work (c.f., Koschmann, Zemel, & Stahl, 2004). Nonetheless, it is only when we become more articulate about the methods members use to produce what recognizably passes as instruction that we will be able to make meaningful advances toward designing artefacts to support collaborative forms of instruction. As the analysis presented here demonstrates, contrasting members' methods across settings employing different forms of communicative mediation can be instructive with regard to understanding both the effects of the mediation and the nature of the methods themselves. Such studies may also contribute in the end toward answering the more basic question, how do people learn.
KOSCHMANN ET AL.
ACKNOWLEDGEMENTS Support for the development of the distance learning program described here came through a grant from the U.S. Dept. of Health & Human Services, Bureau of Health Professions. Partial support for the first author while preparing this paper was provided through a grant from the National Science Foundation (EHR 01-261 04).
NOTES 1 A summary of the PBL process as practiced at this institution can be found in Koschmann et a1.1996). An expanded description can be found in Barrows (1994). The problem-based learning track was discontinued in the fall of 2000.
2
These transcription conventions, developed by Gail Jefferson, are summarized in Appendix A. 3
By "demonstrated effect" we mean that through an analysis of the unfolding sequence, an utterance (or succession of utterances) can be demonstrated to do the work of problematizing in the way described here. This is never based solely on semantic content of the utterance, since evidence must be provided of uptake of the matter by the speaker's interlocutors (see Sacks [I9921 on the analysis of sequence in talk). 4
Digitzed video for this fragment can be found on the CD-ROM that accompanied Koschmann and LeBaron (2002).
5 Though not necessarily novel in other respects. There have been a number of studies carried out from an ethnomethodological perspective in instructional settings (e.g., Fox, 1993: Lynch & Macbeth, 1998; Ford, 1999) and there is a strong tradition of doing ethnomethodologically-informed research in work related to CSCW (c.f., Button & Sharrock, 2000; Clarke et al., 2003). We know of no other work, however, that focuses specifically on the details of how instructional interaction is produced under different communicative circumstances.
REFERENCES Albanese, M. (2000). Problem-based learning: Why curricula are likely to show little effect on knowledge and clinical skills. Medical Education, 34,729-738. Arnseth, H., Ludvigsen, S., Wasson, B., & Morch, A. (2001). Collaboration and problem solving in distributed collaborative learning. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.), Proceedings of the First European Corzference on Computer-Supported Collaborative Learning (pp. 75-82). Maastricht: Maastricht McLuhan Institute. Barrows, H.S. (1994). Practice-Based Learning: Problem-Based Learning applied to rnedical education. Springfield, IL: Southern Illinois University School of Medicine. Bransford, J., Brown, A., & Cocking, R. (1999). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press. Cazden, C. (1988). Classroom discourse: The language of teaching and learning. Portsmouth, NH: Heinemann. Colliver, J. A. (2000). Effectiveness of problem-based learning curricula: Research and theory. Academic Medicine, 75,259-266.
H O W DO PEOPLE LEARN?
285
Dewey, J. (199111938). Logic: The theory of inquiry. In J. A. Boydston (Ed.), John Dewey: The Later Works, 1925-1953, Vol. 12. Carbondale, IL: SIU Press. Distlehorst, L.H., & Barrows, H.S. (1982). A new tool for problem-based self-directed learning. Journal of Medical Education, 57, 466-488. Fox, B. (1993). The human tutorial dialogue project. Hillsdale, NJ: Lawrence Erlbaum. Garcia, A., &Jacobs, J. (1999). The eyes of the beholder: Understanding the turn-taking system in quasisynchronous computer-mediated communication. Research on Language and Social Interaction, 32, 337-368. Garfinkel, H. (1967). Studies in ethnonzethodology. Cambridge, U.K.: Polity Press. Gartinkel, H. (2002). Ethnonzetltodology's program: Working out Durkheirn's aphorism. Lanham, MD: Rowman & Littlefield Publishers. Garfinkel, H. & Sacks, H. (1970). On formal structures of practical action. In J.C. McKinney & E. Tiryakian (Eds.), Tl~oreticalsociology: Perspectives and developrrzents (pp. 337-366). NY: Appleton-Century-Crofts. Glenn, P. & Koschmann, T. (in press). Learning to diagnose: Production of diagnostic hypotheses in problem-based learning tutorials. To appear in M. Maxwell, D. Kovarsky, & J. Duchan (Eds.), Diagrlosis as cultural practice: An account of the power of language in diagnosis. The Hague: Mouton. Glenn, P., Koschmann, T., & Conlee, M. (1999). Theory presentation and assessment in a problem-based learning group. Discourse Processes, 27, 119-133. Goodwin, C. & Goodwin, M. (1987). Concurrent operations on talk: Notes on the interactive organization of assessments. Papers in Pragmatics, 1, 1-52. Hak T., & Maguire, P. (2000). Group process: The black box of studies on problem-based learning. Acadentic Medicine, 75,769-72. Retrieved October 27, 2000 as: http://www.academicmedicine.org/cgi/content/ Halliday, M.A.K. & Hasan, R. (1976). Cohesion irz English. London: Longmans. Heritage, J. (1984). Gat-finkelarzd Etlzrto~rtetl~odology. Cambridge, MA: Polity Press. Herring, S. (1999). Interactional coherence in CMC. Journal of Computer-Mediated Conznzunicatiort, 4(4). Retrieved April 15,2003 as: http:Nwww.ascusc.org/jcmc/vol4/issue4/heing.html Koschmann, T. (2001, March). Dewey's contribution to a standard of problem-based learning practice. First European Conference on Computer-Supported Collaborative Learning (EuroCSCL), Maastricht, Netherlands. Retrieved April 5, 2001 as: http://www.mmi.unimaas.nl/euro-cscVPapers/9O.pdf Koschmann, T. (Ed.)(2002). Studying collaboration in distributed PBL environments. Distance Education, 23(1). Koschmann, T., Glenn, P., & Conlee, M. (1997). Analyzing the emergence of a learning issue in a problem-based learning meeting. Medical Education Online, 2(2). Retrieved on October 15,2002 as: http://ww~v.n~ed-ed-onlinc.ordrcs00007.ht1n Koschmann, T., Glenn, P., & Conlee, M. (2000). When is a problem-based tutorial not tutorial? Analyzing the tutor's role in the emergence of a learning issue. In D. Evensen & C. Hmelo (Eds.), Problern-based learning: A research perspective on learning interactions (pp. 53-74). Mahwah, NJ: Lawrence Erlbaum Assoc. Koschmann, T., Kelson, A.C., Feltovich, P.J., & Barrows, H.S. (1996). Computer-supported problembased learning: A principled approach to the use of computers in collaborative learning. In T.D. Koschmann (Ed.), CSCL: Theory and practice of an ernerging paradigm (pp. 83-124). Hillsdale, NJ: Lawrence Erlbaum Associates. Koschmann, T., Kuutti, K., & Hickman, L. (1998). The concept of breakdown in Heidegger, Leont'ev, and Dewey and its implications for education. Mind, Culture, and Activity, 5, 2 5 4 1 . Koschmann, T. & LeBaron, C. (2002). Learner articulation as interactional achievement: Studying the conversation of gesture. Cognition & Instruction, 20, 249-282. Koschmann, T., Zemel, A., Conlee-Stevens, M., Young, N., Robbs, J., & Barnhart, A. (2003). Problematizing the problem: A single case analysis in a dPBL meeting. To appear in B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.), Designing for change. Amsterdam: Kluwer Academic Publishing. Koschmann, T., Zemel, A,, & Stahl, G. (2004, June). The video analyst's manifesto (or The implications of Gartinkel's policies for the development of a program of video analytic research within the learning sciences). In Y. Kafai, W. Sandoval, N. Enyedy, A. Nixon, & F. Herrera, Proceedings qftlte Sixth International Cortfererzce qf the Learning Sciences (pp. 278-285). Mahwah, NJ: Lawrence Erlbaum Associates.
286
KOSCHMANN ET AL.
Lemke, J. (1990). Talking science: Language, learning, and values. Norwood, NJ: Ablex. McDermott, R., Gospodinoff, K., & Aron, J. (1978). Criteria for an ethnographically adequate description of concerted activities and their contexts. Semiotica, 24, 245-275. Mehan, H. (1978). Learning lessons. Cambridge, MA: Harvard University Press. Pomerantz A. (1984a). Agreeing and disagreeing with assessments: Some features of preferredldispreferred turn shapes. In J.M. Atkinson & J. Heritage (Eds.), Structures of social action: Studies in Conversation Analysis (pp. 57-101). Cambridge, UK: Cambridge University Press. Pomerantz, A. (1984b). Giving a source or basis: The practice in conversation of telling 'how I know'. Journal ofPragnzatics, 8,607-625. Psathas, G. (1995). Conversation analysis: Tlie study of talk-in-interaction. Thousand Oaks, CA: Sage. Sacks, H. (1992). Lectures on conversation. Oxford, U.K.: Blackwell. Sacks, H., Schegloff, E., & Jefferson, G. (1974). A simplest systematics for the organization of tumtaking for conversation. Language, 50,696-735. Schegloff, E., Jefferson, G., & Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53,361-382. Sinclair, J.M. & Coulthard, M.C. (1975). Towards an analysis of discourse; The English used by teachers and pupils. NY: Oxford University Press. Wells, G. (1999). Dialogic inquiry: Towards a sociocultural practice and theory of education. NY: Cambridge University Press.
tkoschrnann @siurned.edu
HOWDO PEOPLE LEARN?
Appendix A: Transcription conventions
I
Timing
I Marks the beginning and end of temporal overlap among
Brackets Equal sign
-
Timed silence
(1.8)
Mcropause
(.)
I
utterances produced by two or more speakers. Indicates the end and beginning of two sequential 'latched' utterances that continue without an intervening gap. Measured in seconds, a number enclosed in parentheses represents intervals of silence occurring within (i.e., pauses) and between (i.e., gaps or lapses) speakers' turns at talk. A timed pause of less than 0.2 sec.
Period
No.
I
Indicates a fallinu pitch or intonational contour at the
Question mark
No?
I
Rising vocal pitch or intonational contour at the
Exclamation point Comma
No!
Delivery
I conclusion of a t i h constructional unit (TCU).
I conclusion of a TCU. An inverted question mark (i) I represents a half rise. I Marks the conclusion of a TCU delivered with emphatic and animated tone. Indicates a continuing intonation with slight upward or downward contour, as in the enunciation of an item in a not yet completed list, occurring (generally) at the end of
no,
a - TCLJ .
Hyphen Colon(s)
no:
Greater than/ Less than signs Degree signs
z c:
Capitalization
NO
Underlined text
S
Arrows Breath sounds
hhh
Double parentheses
"no
? no
I I I I
An abrupt (glottal) halt occurring within or at the conclusion of a TCU. A colon indicates sustained enunciation of a syllable vowel, or consonant. Longer enunciation can be marked using two or more colons. Portions of an utterance delivered at a noticeably quicker (> c) or slower (c >) pace than surrounding talk. Marks speech produced softly or at a lower volume than surrounding talk. Represents speech delivered more loudly than surrounding talk. Underscoring indicates stress on a word, syllable or sound. Marks a rise (1')or fall (J)in intonation. Audible expulsion of breath (linguistic aspiration) as in laughter, sighing, etc. When aspiration occurs within a
KOSCHMANN ET AL.
288
Appendix B: "What would be the risk?" Tape #95-001.1 (1:55:25 to 1:57:46)
Joel :
Patrick: Jackie : Patrick: Joel : Joel : Jackie: Patrick: Jackie:
(?I Patrick: Coach: Jackie:
They did talk about doing a CT along with along with it if you feel there are abscesses but it's low yield unless you feel there are abscesses= =Think you can rget can a lot of risks doing a CT L ~ n dan ultrasound can be used for that reasonj = to the pelvis. (0.5) No why. (2.5) What would be the rjsk. W u h only if it was [ectopic. L~eah ( . ) (but probly) rror if she was =nantl
I=
LL (
)
J=
=Well even even ( . ) well would you have ( . ) danger of X-raying ovaries Oand that0 Is there a ?risk to CT? OHmO
(.)
the
HOWDO PEOPLE LEARN? Patrick: Joel : Jackie: Patrick: Coach: Patrick: Coach: Patrick: Joel : Joel : Joel : Patrick: Jackie :
Coach: Jackie: Coach : Jackie: Coach: Jackie: Joel : Coach: Joel : Jackie : Coach : Patrick: Joel : Jackie: Joel : Jackie: Coach: Joel : Coach : Joel : Melissa: Joel :
(0.9) Sure. ltfsan ?x-rray. L~erah
Lit's an X-ray, there's always a risk to an X-ray. (0.5) I mean ( . ) what is the risk of a CT is there a difference between X ( . ) uh CT and an ordinary X-ray? (0.2) Yea:h (0.2) uh C-tee 1:s ( . ) urn:: ( . ) in uh p1a:ne (0.2) Yu:h So: I would think that the CT (1.0) would be: (0.6) instead of just a plain fi:lm (0.4) would be more X-rays being used .hhhh (0.5) *What is the dosage (0.4) relative ( . ) from uh normal X- ray to a CTi [~CTis *serial CT* Ois ito serial X-rays >is it not