Human error and system design and management

Lecture Notes in Control and Information Sciences Editor: M. Thoma 253 Springer London Berlin Heidelberg New York Ba...

Author: P. Elzer | Rainer Kluwe | Badi Boussoffara

211 downloads 1994 Views 5MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Control and Information Sciences Editor: M. Thoma

253

Springer London Berlin Heidelberg New York Barcelona Hong Kong Milan Paris Santa Clara Singapore

Tokyo

P.F. Elzer, R.H. Kluwe and B. Boussoffara (Eds)

Human Error and SystemDesign and Management With 55 Figures

~ Springer

Series Advisory

Board

A. Bensoussan • M.L Grimble • P. Kokotovic • A.B. Kurzhanski • H. Kwakernaak • LL. Massey • M. Morari

Editors P.F.Elzer, MSc, P h D I n s t i t u t e for P r o c e s s a n d P r o d u c t i o n C o n t r o l T e c h n o l o g y , T e c h n i c a l U n i v e r s i t y o f Clausthal, J u l i u s - A l b e r t - S t r . 6, D-38678 Clausthal-ZeUerfeld, G e r m a n y R.H. Kluwe, D r phil, D r h a b i l

Institute for Cognitive Research, University of the Federal Armed Forces, Hamburg, Holstenhofweg 85, D-22043 Hamburg, Germany B. Boussoffara, PhD Institute for Process and Production Control Technology, Technical University of Clausthal, Jttlius-Albert-Str. 6, D-38678 Clausthal-Zellerfeld, Germany

ISBN 1-85233-234-4 Springer-Verlag London Berlin Heidelberg British Library Cataloguing in Publication Data Human error and system design and management. - (Lecture notes in control and information sciences; 253) 1. Human-computer interaction - Congresses 2.Human-machine systems - Congresses I.Elzer, P. (Peter) II.Kluwe, Rainer H. III.Bousoffara, B. 629.8 ISBN 1852332344 Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. © Springer-Verlag London Limited 2000 Printed in Great Britain The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Typesetting: Camera ready by contributors Printed and bound at the Athenmum Press Ltd., Gateshead, Tyne & Wear 69/3830-543210 Printed on acid-free paper SPIN 10746187

Preface "Human Error and System Design and Management" contains a collection of contributions presented at an international workshop with the same name held from March 24 - 26, 1999, at the Technical University of Clausthal, Germany. The workshop included participants from Germany, Great Britain, the Netherlands, Norway and the USA. It was supported by the "Volkswagen Stiftung". Its purpose was to discuss the results of a research project investigating the "Influences of Human-Machine-Interfaces on the Error-proneness of Operator Interaction with Technical Systems" in a broader context. Therefore experts from academia and from industry had been invited to discuss the theoretical as well as the practical aspects of the subject matter. The book contains the latest state of this discussion and can be regarded as a supplement to engineers and researchers who are active in the area of human machine interfaces. In particular it is important that interaction between humans and machines can not be discussed in isolation as the relation "operator(s) - machine". The correctness of identification and assessment of the state of a technical system and the (eventually) necessary operator actions also depends on the construction proper of the technical system and on the organisational structures ("management") surrounding it. Topics of the workshop also included more recent considerations concerning "Multimedia" and "ecological interfaces" as well as "situation awareness". Proper training of operators and modern principles in the design of control rooms were discussed from an industrial viewpoint. Finally, one of the original assumptions of the project was confirmed, i.e. that in the particular area less experimental results exist than is generally assumed. Therefore, it appeared appropriate to intensify experimental work at the border-line of cognitive science and technology.

Outline The book is organized into five parts and preceded by a State-of-the-Art lecture. The parts including full and short papers as well as summaries of discussions are: I

Research in Human Error, Operator Behaviour and Operator Models

II

Influence of Human-Machine-Interfaces in Technical Systems

III

Technical Aspects

IV

Management Issues and Operator Training

V

Final Discussion

Acknowledgements The editors are grateful to:

The Volkswagen Foundation, Hanover, Germany for supporting the project (Ref.# I/69 886) and sponsoring the workshop. The authors want to thank all the workshop contributors for contributing to this book project. Finally, Mrs. Alison Jackson at Springer-Verlag London should be thanked for editorial suggestions and for helping us with general publishing questions.

Clausthal, September 1999

Peter F. Elzer Badi Boussoffara

Hamburg, September 1999

Rainer H. Kluwe

Contributors Alty, J. L., IMPACT Research Group, Loughborough University of Technology, Great Britain. Borys, B-B., Systems Engineering and Human-Machine Systems, University of Kassel, Germany. Boussoffara, B., Institute for Process and Production Control Technology, Technical University of Clausthal, Germany. Bubb, H., Chair of Ergonomics, Technical University Munich, Garching, Germany. DeVries, J.W., Nuclear Power Plant Borssele, Netherlands. Dr~ivoldsmo, A., OECD Halden Reactor Project, Institute for Energy Technology, Halden, Norway. Elzer, P. F., Institute for Process and Production Control Technology, Technical University of Clausthal, Germany. Endsley, M. R., SA Technologies, Marietta, USA. Flach, J. M., Psychology Department, Wright State University, Ohio, USA. Geb, T., Seamans Center for the Engineering Arts and Sciences, University of Iowa, USA. Grams, T., University of Applied Science, Fulda, Germany. Harrison, M. D., Dept. of Computer Science, University of York, York, Great Britain. Heinbokel, T., Institute for Cognitive Research, University of Federal Armed Forces, Hamburg, Germany. Hollnagel, E., Graduate School for Human-machine Interaction,University of Link6ping,, Sweden. Kanse, L., Human Reability Associates HRA-NL, Breda, Netherlands. Khalil, Ch. J., IMPACT Research Group, Loughborough University of Technology, Great Britain. Kluwe, R. H., Institute for Cognitive Research, University of Federal Armed Forces, Hamburg, Germany. Lee, J. D., Seamans Center for the Engineering Arts and Sciences, University of Iowa, USA. Marrenbach, J., Department of Technical Computer Science, RWTH Aachen, Germany.

viii

Pollack, E., Seamans Center for the Engineering Arts and Sciences, University of Iowa, USA. R6se, K., Institute for Production and Automation, University Kaiserslautern, Germany Sch6bel, M., Research Center System Safety, University of Technology Berlin, Germany. Sheridan, T. B., Human-Machine System Laboratory, Massachusetts Institute of Technology, USA. Stolze, Peter, Institute for Safety Technology GmbH, Garching, Germany. Str~iter, O., Chair of Ergonomics, Technical University Munich, Garching, Germany. Sttirenburg, H-G., Kraftwerksschule e.V. Essen, Germany. Szameitat, S., Research Center System Safety, University of Technology Berlin, Germany. Trimpop, R., Friedrich-Schiller University of Jena, Germany. van der Schaaf, T. W., Safety Management Group, Eindhoven University of Technology, Netherlands. Zinser, K., Asea Brown Boveri, Utility Automation GmbH, Mannheim, Germany.

Contents H C I in Supervisory Control: Twelve Dilemmas .................................................. 1

T. B. Sheridan I

Research in Human Operator

Error, Operator

Behaviour

and

M o d e l s ....................................................................................... 13

1

Errors in Situation Assessment: Implications for System Design .............. 15 M. R. Endsley

2

Errors and E r r o r Recovery ........................................................................... 27 T. W. van der Schaaf, L, Kanse Analysis and Prediction of Failures in Complex Systems: Models & Methods .......................................................................................................... 39 E. Hollnagel

4

Scenarios, Function Allocation and H u m a n Reliability .............................. M. D. Harrison

43

Experience Feedback and Safety Culture as Contributors to System Safety .............................................................................................................. 47 M. Sch~bel, S. Szameitat 6

O p e r a t o r Modelling and Analysis of Behavioural Data in H u m a n Reliability Analysis ......................................................................................... 5 l O. Str~iter

D i s c u s s i o n Session I .............................................................................................. 55

II Influence of Human-Machine-Interfaces

in Technical

S y s t e m s ........................................................................................

57

A Project Overview: Influence of Human-Machine-Interfaces on the Error-Proneness of O p e r a t o r Interaction with Technical Systems ................. 59 P. F. Elzer, B. Boussoffara Attributes of the Interface Affect Fault Detection and Fault Diagnosis in Supervisory Control .................................................................................. 65 T. Heinbokel, R. H. Kluwe

Evaluation of Interfaces by Means of Experiments: what's behind Taxonomy? ......................................................................................................

79

B. Boussoffara, P. F. E l z e r

Human Performance and Interface-Design - Some Remaks based on Experiments ....................................................................................................

89

P. Stolze, O. Str~ter

4

Putting the Normative Decision Model into Practice .................................. 99 T. G r a m s

Discussion Session II ..........................................................................................109 III

Technical Aspects .................................................................

111

Multimedia Interfaces and Process Control: The Role of Expressiveness ..............................................................................................

113

J. L. Alty

2 Ecological Interface Design: Some Premises .............................................. 125 J. M. Flach 3

Ecological Interface Design (EID) and the Management of Large Numbers of Intelligent Agents ..................................................................... 137 J. D. Lee, T. Geb, E. P o l l a c k

4 Operator Support in Technical Systems ..................................................... 153 H. B u b b 5

Interfaces for Every Day Things ................................................................. 163 K. R 6 s e

6

Operator Process Interfaces - A R e t r o s p e c t i v e V i e w of the "90s ............. 167 K. Zinser

7 Acceptance of new Technology and Layout in Control Rooms ................ 173 J. W. de Vries

8 Advanced User Interface Design for Aircraft Cockpit Devices ................ 177 J. M a r r e n b a c h

AMEBICA- An Auto Adaptive Multimedia Environment Based on Intelligent Collaborating Agents .................................................................181 C. J. Khalil

xi

IV

Management

Issues and Operator

T r a i n i n g ..................... 187

1 Safety Culture ............................................................................................... 189 R. Trimpop 2

Study of Errors by Means of Simulation and Training ............................. 201 A. Dr~ivoldsmo

3 Operator Training and Implication for the Practice ................................. 207 H-G. Sttirenburg Function Distribution between Man and Machine: Experiments p e r f o r m e d in FANSTIC I I ........................................................................... 211 B-B. Borys

V

F i n a l D i s c u s s i o n ....................................................................

217

Discussion Session V .......................................................................................... 219

HCI in Supervisory Control: Twelve Dilemmas

Thomas. B. Sheridan Massachusetts Institute of Technology, 3-346 Cambridge, MA 02139, USA e-mail: [email protected]

Abstract: Increasingly in recent decades the computer has become a mediator

between the human operator and the physical system being controlled. This occurred first in aviation, then in process control, manufacturing and military systems. More recently it happened in hospitals, trains, automobiles and home appliances. This new form of control is often called supervisory control, where the role of the human operator becomes more like a manager: planning, setting goals and constraints, diagnosing failures, intervening in control as necessary, and learning from experience. The hope has been to relieve the operator of tedious work and make the system both more efficient and more reliable. Unfortunately human-computer interaction (HCI) and reliability of supervisory control have not been as positive as had been hoped. This paper reviews twelve dilemmas that have emerged from experience. They have implications for human error and system reliability.

1. The imperative of automation: How to moderate the rush? Technology development clearly drives the appetite for more technology. The fact that a system can be automated encourages developers to claim that the system should be automated. Why not automate whatever can be automated and thereby save the human operator or worker the tedium of doing the task? Does not automation not only relieve human effort but also prove more economical, more reliable and safer in the long term than having people perform those same tasks? There is now sufficient experience with automated systems to give an unequivocal answer: no. Of course automation is proving itself increasingly effective and trustworthy across a wide spectrum and heretofore human-controlled tasks. But we have also experienced a number of failures, where poorly designed automation has led to more load on the human, more human error, automatic actions which are not appropriate to the circumstances, and in the end greater cost and compromise with safety. In most complex systems where human safety or economic costs are at risk, the human is not removed from the system but is asked to play the role of supervisor: to

2


monitor the automatic operations, to detect whether the automation is failing in any respect, and to fix whatever has gone wrong immediately. However, humans are known to be poor monitors, who become bored and inattentive if nothing much is happening and who are therefore likely to miss the signs that serious trouble is occurring. So supervisory control is not in itself a solution. It must be done properly. Sheridan [9] details some of the pitfalls and the methods of solution. In any case, in spite of many recent efforts, improving on the Fitts List by rational means to allocate functions between human and machine has not been easy. The proper level of automation must be selected, and this level is likely to be very different for different stages of a process: information acquisition; information analysis; action decision; and action implementation [11]. This is suggested in Figure 1.

Stages: acqAire information] people stored records sensors times places strategies

analyze & ] display

decide action

algorithms criteria past states actions present states future states confidences interpretations display formats

]

I

implement action

control methods speed precision resources used

Levels: 1. The computer offers no assistance: the human must do it all. 2. The computer suggests alternative ways to do the task. 3. The computer selects one way to do the task, and 4 ..... executes that suggestion if the human approves, or 5 ..... allows the human a restricted time to veto before automatic execution, or 6 ..... executes automatically, then necessarily informs the human, or 7 ...... executes automatically, then informs the human only if asked. 8. The computer selects, executes, and ignores the human.

Figure 1: Four stages of a process, and eight levels of automation.


3

2. Increasing situation awareness AND having time to act: Do they conflict? Currently there is great interest in ways to improve situation awareness by the human operator. But the more we insist on improving situation awareness the more time and stress constraint we are likely impose on taking action. Automation is supposed to relieve the need for situation awareness and the preparedness for taking action based upon that awareness. Because of the time or effort resources it demands it may do neither. Particularly in situations where the automation is failing and the human is expected to know what is going on and to step in to fix it, the human may come up short. In such situations the human may have been relatively inattentive and be suddenly faced with a transient in mental workload to understand what is going on and to act. Workload transients are likely to be particularly difficult to cope with. One approach to analysis of this problem is as follows [10]. It is generally conceded that the human requires some time to "become aware of the situation", though the quantitative measure of this is not well established. The analysis works just as well if we are dealing with some resource other than time, such as mental workload capacity. Hypothetically, insofar as more of the available time (or other resource) is used for attending to the "situation" (we will call that fraction x), a smaller fraction of that resource capacity is available for attending to decision-making and execution of appropriate control action. So there is an obvious trade-off.

j

x

~ { A G - AL]= (G-L)

ideal operating point, d'= ~

x=~

A Total gain G

information acquisition effort, x 2a: marginal SA gain and control loss

0

Total IossL ~

max

x=0 2b: ROC plot

Figure 2: Compromise of situation awareness and time stress for decision/action. Assume that AG(x) is the probability density for whatever incremental gain accrues from spending some incremental bit Ax of time (resource) x in order to acquire

4

HCI in Supervisoo, Control: Twelve Dilemmas

more situation awareness, and that AL(x) is the probability density of the incremental cost of having to control with the residual time (resource), where both densities are distributed as a function of x as shown in Figure 2a. Then the expected net gain as a function of x is the integral of (AG - AL), the heavy line in 2a. Figure 2b shows a cross plot of the component integrals G and L, a plot that superficially resembles the ROC curve of signal detection theory. In this case the quantity (G-L) is the height of the curve above the diagonal, and the optimum operating point x* is the point where the maximum (G-L) is obtained. Obviously one can come closer to the ideal of combined greatest gain from situation awareness and least cost from unavailable residual attention (represented at the upper left corner of 2b) when the AG and AL density functions do not overlap, or are most spread apart. In that case there is a value of x* for which the integral G is maximum and the integral L is zero.

3. Decision aids: How much to trust them? While automation is often not trusted by the user, management or public, having a human operator available provides a sense of comfort. To assist the human operator a decision aid (expert system, advisory system, predictor display, etc.) is often proposed, but the sense of comfort remains because, after all, the human need not take the advice of the decision aid. However, if a decision aid is very trustworthy the operator is likely to take its advice uncritically. If the decision aid unexpectedly fails he is likely to be unprepared. If the decision aid is unreliable he will simply pay no attention to its advice. These are limiting circumstances, mostly unavoidable. Between these limits is a range of circumstances for which the operator may be unsure of how much to trust or distrust the decision aid. The subject of trust is a relatively new one for human factors and seemingly irrelevant for systems engineering, and in either case one that is not particularly tractable to scientific investigation. But in fact there have been several recent investigations in which trust has been measured experimentally as a function of failure rates of automation and other properties of systems [6],[8]. Validating a decision aid is in some sense an impossibility, since it requires that some optimum advice be known for all possible situations which might be encountered. First, all possible situations can never have occurred in any finite time, and second, even if they did it would be impossible to say what optimum advice might be for that circumstance, since that depends on the relatively unknown ability of the operator to understand the message and be able to act upon it. One might assert that if optimum advice were available for all possible circumstances there would be no need for the human operator, since the system could then be automated reliably without need for a human supervisor. Reality poses a very different situation, for most decision aids are quite good for those circumstances which are predictable and have been experienced, but are of little or no use otherwise.

HC! in Supervisory Control: Twelve Dilemmas

5

The challenge then is to provide decision aids which convey to the user some estimate of how confident they are in their advice, and how much confidence the user should place in them. Decision aids which make use of fuzzy rules, for example, carry along in the computation a membership function, which itself is a measure of self-confidence based on the membership of the degree of relevance of the fuzzy rules employed in relation to the input data provided.

4. Ultimate authority: Who should be on top when? It is comfortable to assert that the human should always have final authority. Realistically, however, there are many situations where engineers have given to the automation the authority to override human commands. The empirical fact is that humans have neither the computational speed nor the response speed sufficient for many situations, the latter largely due to distractions or diminished alertness. For example in a nuclear power plant when core neutron flux reaches a certain level it is imperative that moderating boron control rods be dropped into the reactor core instantaneously. There is no time to wait for human attention and deliberation to catch up. The same may be said for an aircraft which is about to stall at some combination of altitude, airspeed and attitude if a pilot command were to be followed; in this case the computer inhibits the command from being executed. In the nuclear power industry in the U.S. there is a "tenminute rule", which recommends that the human should never be the ultimate authority when action must be taken within ten minutes. In Europe there is a corresponding "20 minute rule"! Most of the time, when the human is available, and if the required response time is not too short (i.e., where there is time for the human to be called to attend to some situation by an alarm or other means, when there is time for the human to read and interpret the display, where there is additional time for the human to contemplate, decide upon and then execute some action) it is best that the human be in charge.

5. Naturalistic decision-making and traditional decision theory. Can they be compatible?

normative

Many cognitive engineers are asserting that traditional quantitative decision theory is inappropriate for characterizing real human decision-making. Qualitative theories for naturalistic, recognition-primed, ecological, and situated, decision-making are becoming popular, stemming from the ideas of philosopher Heidegger and the psychologist J. Gibson. Naturalistic decision-making has been espoused by Zsambok and Klein [ 13] and refers to the seemingly intuitive and non-rational ways that people arrive at decisions.

6


Recognition-primed refers to the idea that decision and action are automatically triggered by recognition of appropriate action opportunities or necessities in the environment, a basis for much of Gibson's theories of action. Gibson called such opportunities or necessities affordances. Ecological is a term used by Vicente and Rasmussen and other to mean perceptions and actions which correlate naturally from understanding, i.e., from a good mental model. Situated is a term attributed to Suchman in describing how naive users of mechanical devices explore action alternatives in a context of what they assume and what they know. These researchers for the most part have asserted that normative models of decisionmaking simply do not fit the empirical facts and therefore should be rejected. This author [11] agrees that much of human decision making, especially when under time stress or in familiar situations, does not fit the available normative theory, in the sense that all the alternative alternatives are not initially and explicitly considered, together with their corresponding probabilities and utilities, and best expected value decision made therefrom in a rational way, the rejection of normative decision theory is misguided. However, quantitative decision theory is needed for engineering. The current naturalistic, recognition-primed, ecological and situated theories are purely qualitative and provide little that is useful for prediction. Current normative decision theory asserts what be decided based on well defined assumptions. It is definitely not a descriptive theory. Much research has been done to show how people deviate from these norms, and that work must continue. Much work must also be done to show how people take short-cuts based on what has been learned and what works in a satisficing way [7], a way that produces decisions which are satisfactory to the point that further effort to improve the decision is not worthwhile. The available normative decision theory is a solid base from which to build a theory of how humans actually decide. There are no alternatives in sight.

6. Error attribution: Can it be done in closed-loop systems? The operator and the machine being operated form a closed loop, Figure 3a. In closed- loop systems attribution of cause and effect are not possible since the effect of one of the two elements is the cause of behaviour in the other, and that in turn causes behaviour in the first, and so on. Attribution (isolation of cause) of system error (a change in some variable to an unacceptable value) cannot be made to A or B in this case.

3a: AB interaction

3b:

reference

input

Figure 3: Problem of error attribution for interacting systems.


7

Only where a known transient signal is independently introduced into the loop (Figure 3b) can transfer functions be determined, and failure (or equivalently, parameter variations) be isolated. If A and B are linear differential equation transfer functions, then for a reference input r, a/r = A/(I+AB) and b/r = AB/(I+AB). Then by taking a ratio of the latter to the former one gets a/b=A. Since a and b were empirically determined, A is determined. If A fails, its transfer function changes. In this way, i.e., by injecting independent reference inputs any change in either A or B can be determined. With non-linear systems the determination is more difficult.

7. Designer predictability versus free will and creativity: How to make the trade-off? System designers prefer to have predictability in their systems. That is the basis of all engineering models. Yet human operators allegedly possess free will and creativity, which are not predictable behaviours. As tasks become more complex, the cognitive demands on the human operator are ever greater when the automation fails or is not up to the task, which poses ever greater demands on the operator's free will and creativity. How should the designer treat this dilemma? The answer must lie in compromise. With a maximum of human free will and creativity there is no predictability and the system designer can do nothing. The design is a random process. On the other hand, with no free will and creativity there is hardly any purpose served in having a human in the system. Or at least in this case the human is acting like a robot, an automaton, performing in completely predictable (and some would say inhuman) way. One purpose in having humans in systems is to allow for improvisation and learning when unexpected circumstances occur. Learning and adaptation require some variation in behaviour, plus some means to evaluate the benefits and costs that result from one or another behaviour. This principle is essentially the same as what Darwin called requisite variety in the species to allow for its own improvement or survival in a competitive and predatory world. A reasonable guideline is that there be sufficient invariance of response that no harm is done, but sufficient tolerance for variation in response that the human can make reasonable explorations and experiments on how to improve performance or adapt to new circumstances. This necessarily involves a compromise.

8


8. Belief/trust/valuation: A plethora of inconsistent methods? Whatever the task, and however well designed the control system, much will depend upon the operator's beliefs, trusts, subjective probabilities, and relative valuations of the external circumstances and response options. The problem is that we have a plethora of mostly incommensurable methods for dealing with belief and valuation. Von Neumann utility theory and Bayesian probability estimation are the traditional quantitative methods. Fuzzy logic and Dempster-Shafer belief theory are more recent quantitative approaches. Of course, most elicitation of belief from groups of people is done in by simple questionnaires with qualitative categories of response form which respondents may choose. Appendix I provides an example of a simple questionnaire [11] which includes four belief elicitation techniques: conventional qualitative categories, probability estimates, fuzzy scale, and Dempster-Shafer scale. Different answers would necessarily result from these different techniques. It comes down to the fact that if you ask a different question (or ask the same question in different ways) you get a different answer.

9. Gap in modelling HCI and supervisory control: Is there any hope? While useful mathematical models have become available for psychophysics, signal detection, simple information processing, vision, hearing and simple motor skill, no generally acceptable model incorporating the essential elements of supervisory control has yet emerged. Perhaps this is because supervisory control necessarily includes the mix of cognition and behaviourai complexity as well as automation and system complexity. The GOMS model of Card et al [2] and the PROCRU model of Baron et al [1] useful, but are not sufficiently robust or comprehensive to incorporate all the elements of supervisory control planning, teaching, monitoring, intervening and learning).

10. Human-centred design: The emperor's new clothes? The phrase human-centred design is currently very fashionable. Generally the term means that considerations of the human operator or user are kept in the forefront of the design process, an appropriate principle which is frequently ignored. However,

H C I in Supervisory Control: Twelve Dilemmas

9

b e y o n d this g e n e r a l m e a n i n g there is m u c h a m b i g u i t y w h e n m o r e specific m e a n i n g is a t t e m p t e d . In fact m a n y d i f f e r e n t m e a n i n g s h a v e b e e n e m p l o y e d in the literature, m o s t o f w h i c h p o s e p r o b l e m s o f o n e kind or a n o t h e r . T a b l e 1 lists ten a l t e r n a t i v e m e a n i n g s . A f t e r e a c h t h e r e is a p h r a s e s u g g e s t i n g w h y this m e a n i n g is i n a p p r o p r i a t e in s o m e cases, that is, use o f the t e r m h u m a n - c e n t r e d d e s i g n m a y p r o v e i n c o n s i s t e n t or c o n f u s i n g u n d e r s o m e c i r c u m s t a n c e s . 1. Allocate to the human the tasks best suited to the human, allocate to the automation the tasks best suited to it. Unfortunately there is no agreement on how best to do this. 2. Make the operator a supervisor of subordinate automatic control system(s). For many tasks direct manual control may prove best. 3. Keep the human operator in the decision and control loop. Humans can handle only control tasks of bandwidth below one Hz, a demonstrated fact from much research in manual control. At the same time, if the signals to be observed are changing too slowly the human is not likely to maintain awareness of such changes, the well-known result from vigilance experiments. 4. Maintain the human operator as the final authority over the automation. This is not always the safest way. There are many systems where the human is not to be trusted. 5. Make the human operator's job easier, more enjoyable, or more satisfying through friendly automation. Operator ease and enjoyment are nice to have, provided system performance is not compromised. 6. Empower the human operator to the greatest extent possible through flexibility of interface or through automation. The operator may feel a false sense of empowerment by having many controls and displays, and great flexibility in how these may be configured. This is the wellknown peril, leading to complex and unreliable interaction and mode errors. 7. Support trust by the human operator With too much trust there may be a tendency to follow the advice of a decision aid without concern for whether the advice is appropriate, or to abandon responsibility. The challenge is to engender the right amount of trust, not too little or too much. 8. Give the operator information about everything he or she should want to know. The problem here is that too much information will overwhelm. 9. Engineer the automation to minimize human error and response variability. Error is a curious thing. Darwin taught us about requisite variety many years ago. A good system tolerates some reasonable level of "error" and response variability.

10. Achieve the best combination of human and automatic control, where best is defined by explicit system objectives. Don't we wish we always had explicit system objectives. Table 1: Ten alternative meanings of human-centred automation, and some reasons to question

11. Individual alienation to technology: How to deal with it? T h e r e are m a n y aspects o f c o m p u t e r m e d i a t i o n in s y s t e m s that are a l i e n a t i n g to o p e r a t o r s a n d s y s t e m users. O f t e n t h e s e are s u b l i m i n a l f e e l i n g s that are not readily a d m i t t e d to. T a b l e 2 lists a n u m b e r o f these.

I0

HCI in Supervisory Control." Twelve Dilemmas

Those who worry about these aspects of technology are often labeled "doomsayers" by the proponents of the new technology, and are seen as impeders of progress. But such worrying cannot and should not be ignored. While in the short term the worries often do not manifest themselves, over the long term these alienating effects can become very troublesome, in some cases resulting in total abandonment of the technology. For this reason it is important to appreciate that these feelings can be overcome by means of education, more enlightened management, or a more gentle pace of introduction of the technology. 1.

Threatened or actual unemployment

2.

Erratic mental workload and work dissatisfaction

3.

Centralization of management control and loss of worker control

4.

Desocialization

5.

Deskilling

6.

Intimidation of greater power and responsibility

7.

Technological illiteracy

8.

Mystification and misplaced trust

9.

Sense of not contributing

10. Diffusion and abandonment of responsibility 11. Blissful enslavement Table 2: Alienation factors of automation.

12. The world as a multi-HCl supervisory control system: An enlargement of our responsibility? As more sensors and computing elements are added to any one system the complexity necessarily increases. As communication technology allows systems to be interconnected at long distances the complexity increases still further. The internet has already connected information systems world-wide. The information systems themselves are beginning to be interfaced with sensors and effectors (e.g., video cameras, robot arms and vehicles, etc.). Much of this vast interconnectivity supports goals such as education and commerce. Others activities being supported are military systems and espionage, developed of course with the stated goal of security and peace, but capable at a moment's notice to kill and destroy. In any case, for a variety of reasons system complexity is continuing to increase. It is possible now for one person to perform some action anywhere on earth, including the deep ocean and space, from anywhere else. The person or group that has the greatest access to the system becomes very powerful. The person or group


11

w h o has little access b e c o m e s less powerful, and this inequity naturally creates serious p r o b l e m s o f jealously, intimidation, and fear. W e see it already o c c u r r i n g in international relations. A d d to this the fact that c o m p l e x i t y leads to unpredictability, and c o n f u s i o n about what cause p r o d u c e d what effect. This makes it easy for an amateur hacker or a serious terrorist can produce trouble on a grand scale, g i v e n only that they have access to the net. The challenge, then, is to ensure security of c o m m u n i c a t i o n channels by m o n i t o r i n g and by design.

13. Conclusion C o m p u t e r mediation and associated human supervisory control offer great advantages in control o f c o m p l e x systems which are highly a u t o m a t e d at l o w e r levels. H o w e v e r , the h u m a n - c o m p u t e r interaction remains fraught with p r o b l e m s which m u s t be appreciated and dealt with. T w e l v e such d i l e m m a s are described, w h e r e h u m a n designers and operators must m a k e and c o m p r o m i s e s .

Appendix L Questionnaire on Degree of Belief among Given Set of Propositions, Comparing Four

Different Measurement Methods The purpose of the experiment is to explore the differences which emerge when asking people in alternative ways about their belief regarding a given proposition. Admittedly any one way of asking a question about a complex issue will pose problems -- hence the reason to compare people's answers for the four different ways of questioning belief about the same proposition. The proposition is as follows: Within the next five years, somewhere in the world, there will occur a nuclear power plant accident with some degree of core melt. Method 1. Standard Ouestignnairr

Please check the one of the two possible answers which most closely fits your belief: True

False

Method 2. Subiective Prob;tbllit u

Next to each answer write the probability (as a decimal number from zero to one) you would assign to that answer. The two decimal numbers must add to 1. True False Sum = 1? (check) Method 3. F~zz3, Logic This time there are five phrases each of which may characterize to some degree your belief in the above proposition. For each phrase write a decimal number between 0 and 1 which specifies the degree to which that set of words characterizes your belief (1 is maximum degree, 0 is none). In this case the numbers need not add to 1. Surely t r u e

Probably t r u e

Ambivalent__ Probably f a l s e

Surely f a l s e

12


Method 4, Demp~ter-Sh~fer Thc0rV of Evidence In this case please use a decimal number between 0 and I to indicate your degree of belief for each of the three given answers. Again, for this method, the numbers must add to one. True

False

(No idea which is t r u e ) _ _ Sum = 1?

(check)

References [1]

[2] [3] [4] [5] [6] [7] [8]

[9] [10]

[11]

[12]

[13]

Baron, S., Zacharias, G., Muralhidaran, R., and Lancraft, R. (!980). PROCRU: a model for analyzing flight crew procedures in approach to landing. In Proc. 8'h IFAC Congress, Tokyo, 15: 71-76. Card, S., Moran, T. and Newell, A. (1983). The Psychology of Human-Computer Interaction, Mahwah, N J: Erlbaum. Gibson, J.J. (1979).The Ecological Approach to Visual Perception, Boston, MA: Houghton Mifflin. Heidigger, M. (1962). Being and Time. Macquarrie, J. and Robinson, E. (translators), Harper and Rowe. Landauer, T.K. (1995). The Trouble with Computers. Cambridge, MA: MIT Press, pp. 246, 307, 389. Lee, J.D. and Moray, N. (1994). Trust, self confidence, and operators' adaptation to automation. International Journal of Human-Computer Studies, 40, 153-184. March, J. G., and Simon, H. A. (1958). Organizations. Wiley. Muir, B.M. and Moray, N. (1994). Trust in automation, Part 11. Experimental studies of trust and human intervention in a proces control simulation. Ergonomics, 39 (3). 429-461. Sheridan, T.B. (1992). Telerobotics, Automation and Human Supervisory Control. Cambridge, MA: MIT Press. Sheridan, T.B. (1993). How far to commit to open loop action: a probabilistic decision approach with analogies to signal detection theory, IEEE Trans. on Systems, man and Cybernetics, Vol. 23, No. 3, May/June. Sheridan, T.B. (1998). Technical safety and social safety. Proc. Seminar on Humans and Technology, Institute of Nuclear Safety Systems, Mihama, Fukui, Japan, 21-22 September. Vicente, K.J. and J. Rasmussen (1992). Ecological interface design: theoretical foundations. IEEE Trans. on Systems, Man and Cybernetics, Vol. SMC-22, No. 4, July/August, pp. 589-606. Zsambok, C.F. and Klein, G. (1997). Naturalistic Decision-Making. Mahwah, NJ: Erlbaum.

Errors in Situation Assessment: Implications for System Design

Mica. R. Endsley SA Technologies, Inc., Marietta, Georgia, USA

1. Introduction In complex and dynamic environments, human decision making is highly dependent on situation awareness (SA) - - a constantly evolving picture of the state of the environment. SA is formally defined as a person's "perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future" [1]. It encompasses not only an awareness of specific key elements in the situation (Level 1 SA), but also a gestalt comprehension and integration of that information in light of operational goals (Level 2 SA), along with an ability to project future states of the system (Level 3 SA). These higher levels of SA (Levels 2 and 3) have been found to be particularly critical to effective functioning in complex environments, such as the cockpit, air traffic control, driving, medicine and control rooms. The failures of human decision making are frequently cited in investigations of error in a wide variety of systems. In aviation mishaps, failures in decision making are attributed as a causal factor in approximately 51.6% of all fatal accidents and 35.1% of non-fatal accidents, of the 80-85% of accidents which are attributed to human error [10]. While some of these incidents may represent failures in actual decision making (action selection), a high percentage are actually errors in situation awareness. That is, the aircrew makes the correct decision for their picture of the situation, but that picture is in error. This represents a fundamentally different category of problem than a decision error - - in which the correct situation is comprehended, but a poor decision is made as to the course of action to take - - and indicates different types of remediation attempts. Problems with SA were found to be the leading causal factor in a review of military aviation mishaps [9]. In a study of accidents among major airlines, 88% of those involving human error could be attributed to problems with situation awareness as opposed to problems with decision making or flight skills [4]. Based on a review of literature on human information processing and cognition, a taxonomy for

I6

Errors in Situation Assessment," Implications for System Design

classifying and describing errors in SA was developed [5]. The taxonomy, presented in Table 1, incorporates factors affecting SA at each of its three levels. Loss of Level 1 SA - Failure to correctly perceive the situation (80.2%)

Loss of Level 2 SA - Failure to correctly comprehend the situation

(16.9%) Information not available (11.6%) i, system & design failures 2. failure of communication 3. failure of crew to perform needed tasks Information difficult to detect (11.6%) 9 poor runway markings 9 poor lighting 9 noise in the cockpit. Information not observed (37.2%) 9 omission from scan 9 attentional narrowing 9 task related distractions 9 workload 9 over-reliance on automation 9 stressors 9 other distractions Misperception of information (8.7%) 12, negative interference from prior expectations 13. task distractions Memory error (11.1%) 9 disruptions in routine 9 high workload 9 distractions

Lack of/incomplete mental model (3.5%) 9 Automated systems 9 unfamiliar airspace Incorrect mental model (6.4%) 9 Mismatching information to expectations of model or model of usual system Over-reliance on defaults values in the mental model (4.7%) 9 General expectations of system behaviour Other (2,3%)

Loss of Level 3 SA - Failure to correctly project situation (2.9%) Lack of/incomplete mental model (Off_@) Qver-projection of current trends (0 6%) Other (2.3%)

Table 1: Causal Factors related to Errors in Situation Awareness in Aircraft Operations (Jones & Endsley, 1995).


17

2. A T a x o n o m y o f Errors in Situation Awareness The Author [4] applied the SA Error Taxonomy to an investigation of causal factors underlying aircraft accidents involving major air carriers in the United States based on National Transportation Safety Board (NTSB) accident investigation reports over a four year period. Of the 71% of the accidents that could be classified as having a substantial human error component, 88% involved problems with SA. Of 32 SA errors identified in these accident descriptions, twenty-three (72%) were attributed to problems with Level 1 SA, a failure to correctly perceive some pieces of information in the situation. Seven (22%) involved a Level 2 error in which the data was perceived but not integrated or comprehended correctly, and two (6%) involved a Level 3 error in which there was a failure to properly project the near future based on the aircrew's understanding of the situation.

2.1 Causal Factors Associated with SA Errors in Aircraft Operations Jones and Endsley [11] further applied this taxonomy to a more extensive study of SA errors based on voluntary reports in NASA's Aviation Safety Reporting System (ASRS) database. The analysis included 111 incidents, involving pilots. This analysis provides some indication of the types of problems and relative contribution of causal factors leading to SA errors in the cockpit, as shown in Table 1. Level 1 - Failure to correctly perceive the situation. At the most basic level, important information may not be correctly perceived. In some cases, the data may not be available to the person, due to a failure of the system design to present it or a failure in the communications process. This factor accounted for 11.6% of SA errors, most frequently occurring due to a failure of the crew to perform some necessary task (such as resetting the altimeter) to obtain the correct information. In other cases, the data is available, but is difficult to detect or perceive, accounting for another 11.6% of SA errors in this study. This included problems due to poor runway markings and lighting and problems due to noise in the cockpit.

Many times, the information is directly available, but for various reasons, is not observed or included in the scan pattern, forming the largest single causal factor for SA errors (37.2%). This is due to several factors, including simple omission - - not looking at a piece of information, attentional narrowing and external distractions that prevent them from attending to important information. High taskload, even momentary, is another a major factor that prevents information from being attended to. In other cases, information is attended to, but is misperceived (8.7% of SA errors), frequently due to negative interference from prior expectations. Finally, in some cases it appears that a person initially perceives some piece of information but then forgets about it (11.1% of SA errors) which negatively effects SA as it relies on keeping information about a large number of factors in memory. Forgetting was

18


found to be frequently associated with disruptions in normal routine, high workload and distractions. Level 2 SA - Failure to comprehend the situation. In other cases, information is correctly perceived, but its significance or meaning is not comprehended. This may be due to the lack of a good mental model for combining information in association with pertinent goals. 3.5% of SA errors were attributed to the lack of a good mental model, most frequently associated with an automated system.

In other cases, the wrong mental model may be used to interpret information, leading to 6.4% of the SA errors in this study. In this case, the mental model of a similar system may be used to interpret information, leading to an incorrect diagnosis or understanding of the situation in areas where that system is different. A frequent problem is where aircrew have a model of what is expected and then interpret all perceived cues into that model, leading to a completely incorrect interpretation of the situation. In addition, there may also be problems with over-reliance on defaults in the mental model used, as was found for 4.7% of the SA errors. These defaults can be thought of as general expectations about how parts of the system function that may be used in the absence of real-time data. In other cases, the significance of perceived information relative to operational goals is simply not comprehended or several pieces of information are not properly integrated. This may be due to working memory limitations or other unknown cognitive lapses. 2.3% of the SA errors were attributed to miscellaneous factors such as these. Level 3 SA - Failure to project situation into the future. Finally, in some cases, individuals may be fully aware of what is going on, but be unable to correctly project what that means for the future, accounting for 2.9% of the SA errors. In some cases this may be due to a poor mental model or due to over projecting current trends. In other cases, the reason for not correctly projecting the situation is less apparent. Mental projection is a very demanding task at which people are generally poor. General. In addition to these main categories, two general categories of causal factors are included in the taxonomy. First some people have been found to be poor at maintaining multiple goals in memory, which could impact SA across all three levels. Secondly, there is evidence that people can fall into a trap of executing habitual schema, doing tasks automatically, which render them less receptive to important environmental cues. Evidence for these causal factors was not apparent in the retrospective reports analyzed in the ASRS or NTSB databases. SummaoLOverall, this analysis indicates that the primary cause of human error in complex systems is not poor decision making, but poor situation awareness. The factors underlying those errors are myriad, and include solutions at both the system design and training levels. Particular problems with situation a~careness are


19

indicated at the level of attention and working memory, and can be tied to system designs that over-stress these human limits. 2.2 Causal Factors Associated with SA Errors in Air Traffic Control Although operational errors are very rare in traffic control, of those that occur, problems with situation awareness appear to be a critical factor. SA problems have also been well documented in air traffic control operations. In 1997, the three levels of situation awareness were added to the Federal Aviation Administration's (FAA) forms for investigations of operational errors in air traffic control operations. These forms are filled out by FAA quality assurance investigators (also air traffic controllers) following an operational error (separation of aircraft at less than the prescribed limits). Prior to this change, some 57% of operational errors investigated were attributed to problems involving the radar display, with 14% involving misidentification of information and 47% involving "inappropriate use of displayed data, other" [ 13], a catch-all category that indicated many errors were not well accounted for in the analysis form. Following the addition of the three levels of SA to the form, it was found that the quality assurance investigators in the field were able to greatly reduce the number of errors that were regarded as "other". Of the top ten causal factors for operational errors in Tracon air traffic control operations in the United States in 1997, 58.6% were associated with SA problems [12]. In order of frequency the causal factors involved: (lst) Failure to project future (Level 3 SA) - 29.0% (2nd) Failure to detect displayed data (Level 1 SA) - 17.8% (4th) Failure to comprehend displayed data (Level 2 SA) - I 1.8% The numbers were very similar for enroute ATC operations, where 69.1% of the operational errors in air traffic control involved SA problems: (l st) Failure to project future (Level 3 SA) - 32.8% (2nd) Failure to detect displayed data (Level 1 SA) - 19.6% (3rd) Failure to comprehend displayed data (Level 2 SA) - 16.7% Situation awareness problems can be seen to be pervasive in the air traffic control domain, accounting for the vast majority of errors attributed to the human controllers. The causal factors associated with these errors are also well accounted for by the SA error taxonomy. Endsley and Rodgers [8] conducted a study in which 14 operational errors made by air traffic controllers were examined. An analysis of the errors showed that ten (71%) involved Level 1 SA problems. In these cases there was a failure to monitor key situation parameters, most frequently due to distractions by other pertinent tasks (35.7%). There were also misperceptions (21.4%), and memory problems (14.3%) observed, leading to Level 1 SA errors. Level 2 SA problems were found to exist in 21.4% of the errors, and Level 3 SA problems in 28.6% of the errors. (Note: some operational errors involved more than one SA problem). Interestingly,

20


in two-thirds of the cases, the controllers were not even aware at the time that the operational error was occurring. In the study, twenty active duty controllers were asked to watch re-creations of the operational errors and to report on their situation awareness and workload at two stops during the re-creations. In these re-creations, for 10 of the 14 errors (71.4%) there were no subjects (out of 4 observing each operational error) who were aware of the problem at a stop which occurred 2 minutes before the operational error. In 8 of the 14 errors (57.1%), at least one of the four subjects was not aware the error was occurring at the time of the operational error. As the subjects in the study were all highly trained air traffic controllers observing traffic flows in sectors they were familiar with and qualified on, this finding highlights the great difficulty associated with the conditions under which these operations take place. There are significant demands on controllers situation awareness that makes effective SA and performance very difficult. The fact that so many people could experience SA problems when confronted with the same situational challenges highlights the need for more effective design to support the SA processes of these operators.

3. Addressing SA Errors Through System Design An examination of the underlying factors associated with situation awareness and human error provides a more integrated perspective for addressing the system design issue. This perspective focuses on the means by which the human operator maintains an on-going representation of the state of the environment. If designs can be created that enhance an operator's awareness of what is happening in a given situation, decision making and performance should be dramatically better. A situation awareness-oriented design process has been developed that seeks to improve human decision making and performance by optimizing situation awareness. The process includes analysis of SA requirements using a goal-directed cognitive task analysis, the application of design guidelines for enhancing SA, and the validation of resultant designs.

3.1. SA RequirementsAnalysis. Designing interfaces that provide SA depends on domain specifics that determine the critical features of the situation that are relevant to a given operator. A goaldirected task analysis methodology [2] has been used successfully for determining SA requirements in several different domains, including aircraft, air traffic control and remote maintenance control centers. This methodology focuses on the basic goals of operators (which may change dynamically), the major decisions they need to make relevant to these goals, and the SA requirements for each decision. SA requirements are established in terms of the basic data that is needed (Level 1 SA),


21

required integration of the data for a comprehension of system state in light of goals (Level 2 SA), and projection of future trends and events (Level 3 SA). Conducting such an analysis is usually carried out using a combination of cognitive engineering procedures. Expert elicitation, observation of operator performance of tasks, verbal protocols, analysis of written materials and documentation, and formal questionnaires have formed the basis for the analyses. In general, the analysis has been conducted with a number of operators, who are interviewed, observed and recorded individually, with the resultant analyses pooled and then validated overall by a larger number of operators. An example of the output of this process is shown in Table 2.

1.3 Maintain aircraft conformance 1.3.1 Assess aircraft conformance to assigned parameters 9 aircraft at~proceeding to assigned altitude ? 9 aircraft proceeding to assigned altitude fast enough? 9 9 9

9 9

aircraft at~proceeding to assigned airspeed? aircraft proceeding to assigned airspeed fast enough ? 9 9

9 9 9

time until aircraft reaches assigned altitude amount of altitude deviation climb/descent 9 altitude (current) 9 altitude (assigned) 9 altitude rate of change (ascending/ descending)

time until aircraft reaches assigned airspeed amount of airspeed deviation 9 airspeed (indicated) 9 airspeed (assigned) 9 groundspeed

aircraft on/proceeding to assigned route ? aircraft proceeding to assigned route fast enough ? aircraft turning? 9 time until aircraft reaches assigned route/heading 9 amount of route deviation 9 aircraft position (current) 9 aircraft heading (current) 9 route/heading (assigned) 9 aircraft turn rate (current) 9 aircraft heading (current) 9 aircraft heading (past) 9 aircraft turn capabilities 9 aircraft type 9 altitude

22


9 9 9

aircraft groundspeed weather winds (direction, magnitude)

1.3.2 Resolve non-conformance 9

Reason f o r non-conformance ?

9

Verify data 9

Is presented altitude correct?

9 9 9

Aircraft altimeter setting Aircraft altitude (indicated)

Is presented airspeed correct?

9 Aircraft airspeed (indicated) 9 groundspeed 9 winds (magnitude, direction) 9 Is presented position/heading correct?

9 9 9 9

Fix distance to Nav aid range/bearing to Fix track code

Will current behaviour cause a problem?

9 Assess a i r c r a f t s e p a r a t i o n (1.1.1) 9 Assess aircraft/airspace s e p a r a t i o n (1.2.1) 9 Assure m i n i m u m altitude r e q u i r e m e n t s (1.4) 9

Action to bring into conformance ?

9

Provide clearance (2.2)

Table 2: Example of Goal-Directed Task Analysis for En-route Air Traffic Control [7]. The method is significantly different from traditional task analyses in that: 1.) it is not pinned to a fixed timeline, a feature which is not compatible with the work flow in dynamic systems, 2.) it is technology independent, not tied to how tasks are done with a given system, but to what information is really, ideally needed, and 3.) the focus is not just on what data is needed, but on how that data needs to be combined and integrated to support decision making and goal attainment. This last feature, defining comprehension and projection needs, is critical for creating designs that support SA instead of overload the operator with data as many current systems do.

3.2. SA-Oriented Design. The development of a system design for successfully providing the multitude of SA requirements that exist in complex systems is a significant challenge. A set of design principles have been developed based on a theoretical model of the


23

mechanisms and processes involved in acquiring and maintaining SA in dynamic complex systems [5]. These guidelines are focused on a model of human cognition involving dynamic switching between goal-driven and data-driven processing and feature support for limited operator resources, including: 9 Direct presentation of higher level SA needs (comprehension and projection) instead of low level data, 9 Goal-oriented information display, 9 Support for global SA, providing an overview of the situation across the operator's goals at all times (with detailed information for goals of current interest), enabling efficient and timely goal switching and projection, 9 Use of salient features to trigger goal switching, 9 Reduction of extraneous information not related to SA needs, and 9 Support for parallel processing. An SA-oriented design is applicable to a wide variety of system designs, It has been successfully applied as a design philosophy for systems involving remote maintenance operations, medical systems and flexible manufacturing cells. 3.3. Evaluation

Many concepts and technologies are currently being developed and touted as enhancing SA. Prototyping and simulation of new technologies, new displays and new automation concepts is extremely important for evaluating the actual effects of proposed concepts within the context of the task domain and using domain knowledgeable subjects. If SA is to be a design objective, then it is critical that it be specifically evaluated during the design process. Without this it will be impossible to tell if a proposed concept actually helps SA, does not effect it, or inadvertently compromises it in some way. The Situation Awareness Global Assessment Technique (SAGAT) has been successfully used to provide this information by directly and objectively measuring operator SA in evaluating avionics concepts, display designs, and interface technologies [3]. A primary benefit of examining system design from the perspective of operator situation awareness is that the impact of design decisions on situation awareness can be objectively assessed as a measure of quality of the integrated system design when used within the actual challenges of the operational environment. An example of the use of SAGAT for evaluating the impact of new system concepts can be found in [6]. A totally new form of distributing roles and responsibilities between pilots and air traffic controllers was examined. Termed "free flight", this concept was originally described to incorporate major changes in the operation of the national airspace. It may include aircraft filing direct routes to destinations rather than along pre-defined fixed airways, and the authority for the pilot to deviate from that route, either with the air traffic controllers permission or perhaps even

Errors in Situation Assessment: Implicationsfor System Design

24

fully autonomously'[14]. As it was felt that such changes could have a marked effect on the ability of the controller to keep up as monitor in such a new system, a study was conducted to examine this possibility [6].

100

eo 8 0 " O

"~ 30 O

=~ 20 0

Baseline

D~,rect Deviation Deviation Rout(m with Intenl wilho~ Inter.

Baseline

Condition

I~ect Deviation Deviation Routes w~h Intent without Intent Condition

100

50

8O

E3 60

2O ," the properties of the situational conditions under which the task is to be performed, and finally the properties how the interface considers all these circumstances. In our opinion, any statement attempting to find a human properties that allow a design-suggestion independent from these aspects has to fail. This understanding is not new in ergonomics but we observed that many investigations and ergonomic measures either explicit or implicit ignore these dependencies of human performance. We will first describe the experiment performed and then we will justify this conclusion.

2. Preparatory Tasks for the Experiments In order to achieve realistic scenarios for the experiment, operating experience from nuclear facilities was evaluated from the GRS-database (called BVOR/VERA) according to human abilities and problems (see [3] for the procedure of identification and classification of human error events). The database contains more than 4000 events (about 3000 events on PWR - Pressurized Water Reactors). As a basis for the experimental cases, a PWR was selected and consequently we only considered operational events in PWR. 100 of these events were selected and used to find a detailed classification of problem-areas for human interventions. The following classification scheme for distinguishing the complexity of scenarios resulted from this event analysis: >" Class 1: Malfunction or failure of one component >" Class 2: Malfunction of one component due to faulty activation Class 3: Malfunction of a control or limiting system due to failure of a data acquisition unit

Human Performance and Interface-Design - Some Remarks based on Experiments

91

Class 4: Malfunction of one component combined with or happening after the malfunction of another component According to this classification, we defined 6 scenarios with different degree of difficulty: Simple cases (class 1) to complex cases (class 4). The definition of the 6 Scenarios that also represents the event classification is as follows, see Stolze [2] for details about the scenarios:

Simple scenarios Scenario 1: Stuck open failure of one pressurizer spray valve (class 1) Scenario 2: Unexpected opening of a pressurizer relief valve due to faulty activation (class 2) Scenario 3: Failure of a feedwater control unit due to failure of a data acquisition unit (class 3)

Complex scenarios >" Scenario 4: Failure of a main coolant pump and malfunction of a feedwater valve (class 4) ~" Scenario 5: SCRAM due to manual maloperation of the bypass after a turbine trip (class 4) ~" Scenario 6: Malfunction of bypass and malfunction of a steamgenerator safety valve due to decreased opening pressure (class 4) The scenarios were transferred into a fast running simulation model of German PWR by adapting the scenarios to capabilities of the GRS-Analysis-Simulator, the simulator model, and the simulator interface. Several variants for each scenario were calculated (interaction or no interaction of operator, different types of interactions).

2. Design of the Experiment The experiment was performed in the GRS-Analysis-Simulator. The hypotheses build for the experiment were: >" HI: The higher the training level of the operators, the higher is the diagnostic performance (Factor: Qualification) >" H2: The more possibilities a MMI offers, the better is the diagnostic performance; the simpler the MMI, the worse is the diagnostic performance (Factor: Number of functions of MMI) H3: The more complex the scenario, the better is the diagnostic performance of a MMI that offers more possibilities (Factor: Complexity of situation)

92

Human Performance and Interface-Design Some Remarks based on Experiments -

To investigate these hypotheses, several conditions were introduced for the available parts of the interface (Figure 1 illustrates pictures and experimental conditions). SYNOPS: Permanently in condition 1,2, 3 and 4

Bar-Graphs: Called by User in condition 2 and 4 ~_examF)!e!rom a set)

Subsystems: Called by User in condition ,1,2, 3 and 4/example from::~a set

Trends: Called by User in condition 3 and 4 (example from a set)

Figure 1: Overview about some of the pictures available in each condition. Condition 1: ~" An initial picture (always visible): Overview of primary and secondary loop (socalled SYNOPS-picture) Pictures of subsystems (always available by active call of picture): pressurizer, steam generator, feedwater control, turbine, bypass, reactor safety system Condition 2: ~" All pictures from Condition 1 and ~" Bar graph display of the main system parameters (collection of main variables) Condition 3: ~" All pictures from Condition 1 and ~" Trend-Pictures (available by active call of Trend-Pictures) Condition 4: ~" All pictures from Condition 1 to 3


93

Two Groups of probands were used in the experiment: students with basic knowledge about PWRs (Group 1, n=4, called novices) and students with basic knowledge about PWRs plus practical experience on the simulator (Group 2, n=3, called experts). In the GRS-Analysis-Simulator, a block of four 21"-screens (arrangement 2x2) was used. Any picture or trend could be displayed on every screen. The starting picture was permanently displayed on the left screen of the top row. The bargraph display is positioned automatically on the right screen of the top row.

3. Realization of the Experiments Training Phase: Each participant was trained on the use GRS-Analysis-Simulator and on the available pictures. Additionally, a basic knowledge-test performed. Experimental Phase: The scenarios were varied in sequence but every participant had to diagnose every scenario once in every condition (6 scenarios * 4 conditions = 24 trails per operator, random sequence of scenarios and interfaces, each operator performed 2 sessions with 12 trails).The scenarios were demonstrated to the probands up to a predefined halting point. The probands had then been asked to tell their observations by thinking aloud. After the halting point, the probands were asked to give a summary of their observations pointing out malfunctions and failures. At this point they were allowed to use additional pictures and trends if available according to the experimental condition. Furthermore they were asked to give a suggestion for an applicable further measure and to show up the expected consequences of this measure. At last they were asked to tell how they expect the scenario to go on without any measure. Further measurements were: >" Written protocols of diagnostic statements during each session >" Automatic logging of chosen pictures with time-stamp >" Protocols on diagnostic statements after halting point

Post Experimental Phase: After the experiment, the knowledge-test was performed again and a questionnaire on interfaces, strategies, scenarios, and performance in the experiment had to be filled out.

4. Results The results will be described in two steps: First, the results concerning the hypotheses. Second, other observations and evaluations of the experimental data concerning strategies in diagnostic performance.

94

Human Performance and Interface-Design Some Remarks based on Experiments -

4.1. Results Concerning the Hypotheses The following Figures present the results according to the hypotheses mentioned above [2]. In the Figures, "E" means fault correctly recognized, "TW" means fault partly recognized, "NE" means fault not recognized. "VP" means Proband. "Anteil" means portion of success. There is no considerable difference between novices and experts in diagnostic performance. Knowledge about the technical system (here PWR-reactors) seems not to be important to find the correct navigation through the pictures (Figure 2).

..D ~ u ~ Dia.qnosis (vor Haltepun _kt)--

In Total (Gesamte_rgebnis):

Novices (VP1,2,4,5); Experts (VP3,6,7)

Novices (VP1,2,4,5); Experts (VP3,6,7)

Figure 2: Results according to H 1: The higher the training level of the probands, the higher is the diagnostic performance.

Complex scenario:

Simple scenario: S z e r m d o 3: v o r H e l t e p u n l d 80.0%

,,

"~:4~ :

::"

Szen~rto 5: v o r H a ~ p u n l d

"

:"""

- t

=

~,/~.:~..

! :

~!!.~

~':

i'.:/

ruNE

~

20.0%0.0% 1

2

3

4

1

Interface

100.0%

~ . . . . . . .

800.I L :

1

. . . . . . . . .

I!

2

Irdorfsm

3

IntsrM==

3

4

s,nmao 5:Ge=m~rcJa~s

S,~,narlo 3: G e m m t e r g e l o n J s loo.o%

2

.~

~.~.

'i..'i~ '~'2):,~ . ~ " . ?

4

O O .% 1

2 irdoft~o 3

4

Figure 3: Results according to H2: The more possibilities a MMI offers, the better is the diagnostic performance, simple MMI causes worse diagnostic performance


95

Diagnostic performance and complexity of the MMI are depending on each other (Figure 3). Interface condition 1 (SYNOPS-overview about the thermohydraulic flows in the plant) seems to be a sufficient source for identifying an abnormality (in the more simple Scenario 3). In the more complex Scenario 5, the SYNOPSoverview even seems to be superior to the other information sources. It seems that the other possibilities of the MMI are rather distracting the proband from generating a diagnosis than supporting him.

In Total: Szer=do 6:Gemmlrg~nls

During_Diagnosis:

Szenado6:~r Hal1~xmld 100.0%

,

.

I

~

]

!

= ~.:..

1

_,oo. 1

t!,:

18~ ~

........

E

I

2 inliflll~

3

4

Figure 4: Results according to H3: The more complex the scenario, the better is the diagnostic performance using a MMI that offers more possibilities However, in the most complex Scenario 6, the SYNOPS-overview is not sufficient (Figure 4). This Scenario can only be detected, when the small deviation gradient (the decreased opening pressure) is realized. For this, the trend-graphs are superior (Interface 3), but this is - of course - reasonable that a decreasing opening pressure can only be observed when presented and best with a trend-graph.

4.2. Strategies in Diagnostic Performance Successful diagnosis is accompanied by an increased number of pictures investigated by the proband (Figure 5). Probands searching more active in the picture-set of the MMI achieve better diagnostic performance. Failure in Diagnosis:

Successful Diagnosis:

i,A

-

Picture,4~o. - ,//, ~ < : ~ . . " ,,

M,, , ,"

i l ,~ /)f/;.-M,":.~' . ....... i~..f,d~,~.7~,,.:

1

,,,,,/i./,

......

~'/

9

!

./

i

..i

~ S t e p . .le.'.... : .

[

~'ilStepof

of ..---~i Picturecall " [

.......~ I

Average of 5,43 Pictures per Scenario

I

i

/

//

!

.../

i i

~

.....................~!et~m.~!......................'.

I

1

4

I

I

Y

I

I

~

II

11

II

Average of 4,87 Pictures per Scenario

Figure 5: Results according to number of used pictures.

96

Human Perfomlance and Interface-Design - Some Remarks based on Experiments

However, diagnostic performance is not assured, if the correct picture is called during this active search. As Figure 6 shows, failed diagnosis appears by some kind of unmotivated use of the potentials of the MMI (in Scenario 5, the failed proband only called three pictures), or by some kind of unmotivated gambling through the pictures. In Scenario 6, the failed proband went through the pictures by calling nearly every picture without any structure in this proceeding. The more successful probands performed an active search strategy (the sequential lines of pictures are above the more split-angle like line of the failed proband). Scenario 6

Scenario 5 I=] ................................

r ............................................................................

,~

,

/

Picture:No. /

/ ~ //" /

/'/'failed proband

\

Step of r-J=A6 .

,

.

,

~

.

.

call .

I

Step'of __.i

.....

_

r = u t u r e

=

I

.

s

9

Pic,t u r e ~ ! `

.

i

.

1.

.

n

,1

~

~

i ,.

Figure 6: Results according to search strategy and sequence of called pictures.

5. C o n c l u s i o n a n d Perspectives None of the (very simplified) hypotheses could be supported by the experiment. This means, we cannot define a simple rule for supporting diagnostic performance. Moreover, the experimental results show that the information has to be appropriate to what is to be detected (i.e., the task and failure mode). The scenario 6 was found as a good example for this: If we do visualize dynamic aspects like the decreased opening pressure inappropriately (e.g. by using a digital number), we do not have to wonder about the result that human have difficulties with such time aspects. But this statement only means: If we do represent time aspects inappropriately then humans will have problems in dealing with them nothing else. If such tautologies are wrongly used in the design (or assessment process), this will not exhibit an improved design of interfaces. Ergonomic layout of computerized CR is not assured by ,,nice pictures". Rather more, we have to design the pictures according to the possible spectrum of tasks and situational conditions under which the operator has to deal with the interface.

5.1. The importance of compatibility of information to the task It is necessary to link both aspects (cognitive Human properties and ergonomic design properties) together according to the tasks of the operators: If the task is to


97

consider a failure mode with time-aspects, then the layout of the interface has to be according to this aspect. The system ergonomic approach is known as a proposal to perform this linkage [1]. The presentation of the information has to be compatible to the nature of the parameter (absolute value if the this has a certain meaning vs. analog representation if the sequence of values in time or space have a certain meaning). Especially, if one intends to investigate diagnostic performance, the observations and evaluations of experimental data are showing that such aspects concerning the relationship of ergonomic design and human desires have a big influence.

5.2. The importance of the structure of process information pictures Another more important aspect for diagnostic performance is the structure of the MMI-picture, the picture hierarchy. The structure of process information pictures is of considerable importance to support the search-process of operators. Diagnostic performance is unrelated or even in contrast to the theoretically availability information on the MMI as far as the MMI does not provide a linkage to the correct (failure respective) sub-pictures. Process information pictures have to present the required information for a certain diagnostic step at a given time. Therefore, the interface has also to announce important information independent of the current diagnosis path of the operators. At this point, the fundamental difference between conventional and computerized process control room has to be mentioned: In a conventional control room the information is present in parallel. The operator has all the information present and has only to look around to diagnose. In a computerized control room he has to search actively and sequentially in a hidden structure of pictures. Therefore, he has to remember how to access a certain information, which means an additional cognitive effort for him. This is a clear disadvantage of computerized control rooms. To improve computerized control rooms in the future, it is not needed to concentrate on finding human non-abilities to be supported ergonomically but to find out how abilities are used and selected in a given ergonomic layout and a given failure situation.

References [1] [2]

[3]

Bubb, H. (1993) Systemergonomie. In: Schmidtke, H. (Hrsg) Ergonomie. Hanser. M0nchen. Stolze, P. (1999) Konzeptsammlung und systematische Auswertung theoretischer und empirischer Befunde zu StOrfi~llen in Kernkraftwerken. MeMoVis-T-ISTec-09. ISTec, Garching Str~iter, O. (1997) Beurteilung der menschlichen Zuverltissigkeit auf der Basis yon Betriebserfahrung. GRS-138. GRS. KOln/Gerrnany. (ISBN 3-923875-95-9)

Putting the Normative Decision Model into Practice

Timm Grams University of Applied Sciences, Fulda, Germany e-mail: timm.grams @et.fh-fulda.de

Abstract: The proposed normative model of operator behaviour, called decision event tree (DET), is basically a model of multistage decision under risk. Operator errors are defined to be deviations from the so defined norm. By this the methods of technical risk assessment, economic decision theory and psychology can be utilised for an assessment of operator errors. The proposed model reveals that 1. without a precise definition of what the norm is we don't know what an error is, 2. scenario oriented psychological experiments can be misleading, and 3. good tools are needed for the design of normative models.

1. Introduction O v e r a l l goal. W e want to know the cognitive mechanisms and causes of typical operator errors because we want to avoid these errors through a better design of

man-machine interfaces. M e t h o d of investigation. During simulation experiments the subjects are exposed to a series of scenarios. These scenarios are based on rather realistic operator tasks of a coal-fired power station. A test person representing an operator is called subject. The person who has planned and is conducting the experiment will be named experimentalist. The subjects have to solve a diagnosis task: The experimental conditions are given by introducing failures of certain components of the power station. The task is to find the real causes of the malfunctions and to advise the right mitigation measures. The subjects/operators are asked to think aloud while operating the system. These protocols are then analysed by the experimentalist with respect to possible deviations of the subject's mental model from reality and the technical and psychological causes thereof, the human-machine mismatches [ 1].

100


Problems encountered. Such investigations of decision making in complex situations are posing serious difficulties concerning the planning of the psychological experiments and the analysis of the results: The experiments include many hardly controllable extraneous variables. Through complex instructions further extraneous variables like the learning capabilities and the background knowledge of the subjects are introduced. The decision situations are quite complicated and the operator's behaviour cannot be assessed solely from his following a certain predefined linear chain of events. But most important seems to be the experimentalist's risk of falling into the same thinking traps (cognitive deceptions) as the operators and subjects. The experimentalist could overlook or deliberately exclude important failure hypotheses in the evaluation scheme of a scenario. Or he could - according to faulty assumptions upon the frequency or severity of failures - concentrate on a scenario of no significance in reality. To put these mistakes into a generalized setting [2, 3]: We are seeking evidence and confirmation of our assumptions and prejudices thus losing more plausible hypotheses out of sight. By our cognition's tendency towards meaning we often select irrelevant hypotheses on the basis of some rnisleading clues or by applying inappropriate heuristics. Within a taxonomy of operator errors the above thinking traps can be sorted into the classes of inductive reasoning errors and linear cause effect thinking [4].

Proposed solution: normative modelling. To overcome these difficulties it is proposed to base all investigations of operator errors on normative models. A normative model says what the operator should do. An operator error is defined to be a deviation from the normative model. The framework of normative modelling should take into account the normal behaviour of a real process and plant, the various component failures and resulting symptoms, as well as their frequencies and probabilities. There must not only be a simulation model of the plant but also a - perhaps simplified - risk analysis thereof [5], (If we don't know what types of malfunctions we possibly encounter, we cannot know how to act on their symptoms at an as low risk as possible.) The design of an experiment starts with the selection of a scenario following some initiating event, p. e. the breakdown of a coal mill. The symptoms of a malfunction generally cannot be traced back directly to the initiating event because of ambiguity. Other initiating events could result in the same symptoms. The operator should take into consideration all possible causes. Some of the failures taken into consideration may ask for urgent action, others may not. Therefore the normative model should include all known failure hypotheses


101

and their a posteriori probabilities, i. e. the conditional probabilities under the symptoms of the scenario. A short introduction into the normative modelling technique will be given. By an example it will be demonstrated how to put this technique into practice, and what can be learned from its application.

2. The Task: Fault Diagnosis of a Coal-fired Power Station The setting. Let the operator's task be the fault diagnosis of a coal-fired power station. Within a specific scenario the operator is confronted with the following symptom: There is a rising level in one of the tanks in the high pressure preheater (HPP). Possible causes (hypotheses) are 1. a small leak in a feedwater pipe or 2. a spindle breakage. The failure rates attributed to a small leak or a spindle breakage, respectively, are known from a technical risk assessment of the power plant. From this the a posteriori probability of each of the hypotheses can be calculated. The probabilities are 0.05 (leak) and 0.95 (break). Without countermeasures a leak possibly results in a bypass activation. In any case the efficiency of the plant will be reduced considerably. The severity of the consequences of the leak can be reduced by mitigation measures. Likewise a breakage results in a reduced efficiency. The severity can be reduced by appropriate countermeasures. The operator may choose to wait and see and he possibly may miss the right time for countermeasures. Multi-stage decision under Risk. According to these observations and hypotheses the operator may choose among the following alternatives of action: do nothing (default), mitigate the consequences of a leak, mitigate the consequences of a breakage, or defer the decision in favour of further examination. After a time consuming investigation it may be possible to discriminate between the hypotheses. If the leak hypothesis is true, there is a loss of feedwater in the main feedwater circle and the temperature of the overheater is rising. This can be observed within 60 s. If the break hypothesis is true, the system stabilizes after a while due to automatic controllers, there is no loss of feedwater and no rising temperature of the overheater. Each of the actions is accompanied by some risk. The risk of an action can be calculated by means of fault tree or event tree techniques. Without countermeasures the following will happen in case of a leak: The level in the HPP increases while the feed water supply to the boiler decreases. The level controller of the HPP becomes active to drain off water from the HPP. If the level continues to increase an emergency valve opens additionally. If these control actions are not able to reduce the level in the HPP two bypass valves become active.

102


This results in a reduced efficiency. The bypass activation is assumed to occur with probability 0.5. Even if no bypass activation occurs the efficiency will be reduced considerably. The cost of bypass activation and a reduced efficiency is set to the severity value 1. If an action is taken to mitigate the consequences of the leak the severity is equal to 0. Without countermeasures a breakage can result in a reduced efficiency with probability 1. The severity value is assumed to be 1. The loss can be reduced to 0 if appropriate countermeasures are taken. If the operator chooses to wait and see, he possibly may miss the right time for countermeasures. The decision in favour of a further investigation will be accompanied by a penalty (cost) of let us say 1/10 in the case of a leak. In case of breakage there is a loss of 1/100 due to the deferment.

3. The Normative Model Trees. The Normative Model defines what the operator should know of the processes and of the machine: All hypotheses as well as their a priori probabilities, all scenarios following from these hypotheses, all possible observations and their conditional probabilities, the a posteriori probabilities of the hypotheses, and the severity of all consequences. This knowledge can be represented by a so called tree. The basic terms and definitions for trees as well as the fundamental algorithms on trees can be found in the books on data structures and algorithms [6, 7]. Two types of trees are relevant with respect to the given task: 1. decision trees as known from game theory and from the studies of economic behaviour [8], and 2. event trees as known from technical risk assessment [1]. Event trees are the appropriate means for modelling the consequences of decisions. To meet the requirements of the above multi-stage decision situation these two types of trees will be integrated into one model. We will call such a tree a Decision Event Tree (DET).

By means of DETs we can find the optimum decisions with respect to the minimization of risk. Additionally, DETs serve well as a classification scheme of operator errors [4]. Structure of Decision Event Trees (DET): A DET is defined to be a tree with two distinct kinds of non-terminal nodes: decision nodes and event nodes. In the graph of a tree the decision nodes are represented by rectangles, the event nodes by


103

circles. A terminal node or leaf does not need a graphical representation. The connecting line from a parent node to one of its child nodes is called an edge, fig. 1. T I T2

T3 T, T~ Figure 1: Decision event tree with root D~and five terminal nodes (T~..... T~) Adding attributes: An edge may be associated with some cost or loss, called transition loss. These values will be attributed to the child node of the respective edge. For all children of event nodes an additional attribute has to be defined: The (conditional) probability of a transition from the parent node to this node. This probability will be called transition probability. Evaluation of DETs: The calculation of minimal risk can be achieved recursively starting with the terminal nodes. During the evaluation an optimum value will be assigned to all the nodes. Each node is the root of a tree or subtree. The optimum value of a node is defined to be the minimum obtainable risk of the respective tree or subtree. The optimum values of the nodes are defined as follows. 9

Terminal node: zero

9 Decision node: the minimum value of the optimum values of all its subtrees (including the transition loss associated with the edge from the node to the subtree's root) 9 Event node: the mean value of the optimum values of all its subtrees, taking into account the probabilities of the edges leading to the subtrees and the associated transition loss Such an evaluation can directly be implemented by a recursive procedure [6]. Evaluation using spreadsheets. The representation by indentation is a textual alternative to the graphical one [7]. This representation can immediately be transferred into a spreadsheet. The name of a node as well as its attributes (including those of the incoming edge) are the constituent parts of the node record. Each node record is written into one separate row of the table. (The identifiers of all children of a node must be distinct. Identical names are allowed for nodes with different parents.)

104


The cells of a node record are arranged in the following manner:

{void} [TransitionProbability] Name [TransitionLoss] Optiraum Value The meaning of the contents can be seen from the cell type, given in italic letters. Square brackets are used to denote optional cells. If the cell type is enclosed by braces there can be any number of adjacent cells of this type. The number of leading void cells depends on the level of indentation: the children of a node are indented by one more cell than their parent node. The transition probability attribute is only applicable to children of event nodes. By the convention of placing the transition probability - a number - in front of the node name the parent node is unequivocally marked as an event node. The transition loss attribute is applicable to all nodes except the root. The calculated optimum value of a node is given in bold numbers at the end of its record.

4. The DET of the Preheater Diagnosis Table 1 shows the spreadsheet version of the DET of the preheater diagnosis. The framed parts plus the extra root node are event trees. These trees describe the events occurring without further interference by the operator. If the operator would do nothing, the consequences are described by the tree with root RisingLevel.Default. All other event trees result from this tree by simple manipulations. (The full name of a node consists of the sequence of all node names along the path from the root up to this node, written in dot notation: "RisingLevel.Default" denotes the child Default of the node RisingLevel. The latter is the root of the entire tree.) For explanation let us have a look at the event tree with the root node RisingLevel Deferment. The optimum value of the deferment node results from the following calculation: With probability 0.05 the rising temperature event follows. This event entails a transition cost of 0.1 due to deferment. With probability 0.95 the no rising temperature event follows. This event entails a transition cost of 0.01 due to deferment. Since the deferment node is an event node (as can be seen from its children) the optimum value of this node is given by the mean value 0.05*(0.1+0)+0.95"(0.01+0), which is equal to 0.0145.


~.:

....

105

0.0145

Let,'~:: +::

Default

0 0.05 0.95

LeakMilig

BreakMitig

Deferment

0 0.05 0.95 0 0.05

1 Leak 0.5 0.5 Break 1 0.~3 Leak 1 Break 1 0.05 Leak

0.95

0.5 0.5 Break 1

0

0.0145

LeakMitig

0 Bypass Reduction 0 Reduction

1 1 1 1 1

0 NoReduct 0 Reduction

0 0 1 I

0

1

Bypass Reduction 0 NoReduct

1 1 0 0

0

0

I1 BreakMitig

LeakMitig

0 1

0

i1 BreakMitig

0

I'

Leak 1 1 Leak 0.5 0.5

0 0 0

0 0

00

0

0 NoReduct

0 0

0

l

0 Bypass Reduction

1 1 1

0 0

1 0

Reduction

11

0

I

1

I~11~

0

o

0

j

1

Table 1: DET of the Preheater Diagnosis (Spreadsheet). This is also the optimum value of the entire tree, because all other decisions (default, leak mitigation, and break mitigation) yield higher risk values. Under the given conditions the optimum decision is the deferment decision. This can be seen from the calculated numbers and the evaluation rules given above. This result is strongly dependent from the chosen parameters. Let the penalty for deferment in the case of a breakage be 0.05 instead of 0.01. This would result in an increase of the risk of deferment from 0.0145 to 0.0525. Now the deferment decision is no longer the best choice. An immediate break mitigation would be preferable.

5. The Lessons

Learned

Without a precise definition of what the norm is we don't k n o w what an error is. Errors are defined to be deviations from the norm. Operator errors are defined to be behavioural deviations from the known best behaviour described by a decision event tree. Thus the precise representation of the DET is crucial with respect to the assessment of operator behaviour. The normative model defines unambiguously what we must know of the processes and machines the operator has to control:

106

9 9 9


all hypotheses to be included in the model as well as their a priori probabilities all scenarios following from these hypotheses all possible observations (symptoms) and their conditional probabilities, as well as the a posteriori probabilities of the hypotheses

Scenario oriented psychological experiments can be misleading. Sometimes a shorter way to the understanding of behavioural errors is chosen: The actions of the subjects are compared with some predefined optimum actions given by the scenario. By this method optimum decisions could possibly be judged as being faulty. This can be seen from the DET of the preheater diagnosis: Let the penalty for deferment in the case of a breakage be 0.05 instead of 0.01. As shown above the optimum decision under the normative model is an immediate break mitigation. All other decisions are operator errors under the normative model. Let us now consider a scenario following a leak within the HPP. The first observable symptom is identical to that of the breakage scenario (a rising level in one of the tanks of the HPP). Under the narrow view of the scenario only the leak mitigation would be accepted as a correct decision. All other decisions would be classified as an error. But how could the subject expect this very rare scenario? The more likely scenario of a spindle breakage (given the symptom) demands prompt reactions. The mitigation of breakage should be undertaken even if there is a chance of 5 % of a leak instead of a breakage. The useless actions in the view of the currently applied scenario are the rational ones in the view of the normative model. Thus a subject's behaviour during psychological experiments cannot be assessed solely from its following some predefined scenario. Instead, a normative model should be applied during the evaluation process. Good tools are needed for developing normative models. Spreadsheet techniques are error prone. Larger DETs should not be evaluated by means of spreadsheets. An appropriate DET tool should * 9 9 9 9

provide a user interface designed for input error avoidance (for example by plausibility checks) make structural changes of trees easy provide means for easy navigation through the tree offer synoptic representations as well as a zoom facility provide automatic evaluation of DETs


107

References [1] [2] [3] [4]

[5] [6] [7] [8]

Leveson, N. G.: Safeware. System Safety and Computers. Addison-Wesley, Reading, Mass. 1995 Salvendy, G. (Edt.): Handbook of Human Factors and Ergonomics (2nd edition). John Wiley, New York 1997 Reason, J.: Human Error. Cambridge University Press 1990 Grams, T.: Operator Errors and their Causes. In: Computer Safety, Reliability and Security. Proceedings of the 17th International Conference, SAFECOMP '98, Heidelberg, Germany, October 1998 (Herausgeber: W. Ehrenberger). Lecture Notes in Computer Science. Springer-Verlag, Berlin Heidelberg 1998 (S. 89-99) Shooman, M. L: Probabilistic Reliability: an Engineering Approach. Robert E. Krieger Publishing Company, Malabar, Florida 1990 Aho, A. V.; Hopcroft, J. E.; Ullman, J. D.: Data Structures and Algorithms. AddisonWesley, Reading, Massachusetts 1983 Knuth, D.: The Art of Computer Programming. Vol. 1: Fundamental Algorithms. Addison-Wesley 1973 Neumann, J. von; Morgenstern, O.: Theory of Games and Economic Behaviour. Princeton University Press, Princeton, New Jersey 1972

Acknowledgement:Preparation of the manuscript was supported by the ,,Volkswagen Stiftung" in Germany.

Discussion Session II

As planned, this discussion session concentrated upon the MeMovis project and - in particular - on the method used for the experiments. The main part was concerned with the 'de'sign' of experiments. This comprised aspects like complexity of scenarios, task analysis, choice of subjects, etc. With respect to complexity a kind of agreement could be observed insofar as this concept as an isolated measure was of limited meaning. Every experiment inevitably reduces the complexity of reality. The important choices that have to be made relate to content, control variables, constraints, etc. - and they have to be made consciously and carefully. They have also to be based upon experience with older experiments. A key property of experiments has to be their 'representativeness' although this may be difficult to quantify or to measure. As an illustration for the importance of this criterion it was e.g. mentioned that a field study - which is usually regarded as representing 'real life'- that is poorly planned may be much less representative than a thoroughly conceived experiment in a laboratory. Aspects that are of importance for the representativeness of an experiment are e.g.: capabilities of the simulator, realism of interfaces, training of subjects, number and kind of possible diagnostic strategies and action options, etc. Broad room was also given to the topic of 'task analysis'- possibly because from the presentations of the project partners it had not become quite clear how this had been done in the project. It turned out that basically two strategies had been combined. One was the 'scenario based' approach, partially using personal experience from industry and earlier experiments about actual and plausible cases of process irregularities and their detection and partially using an explicit analysis of a large database with recorded undesirable process states and appropriate countermeasures. The other one was based upon a 'cognitive analysis' concerned with the information necessary for operators to correctly diagnose a situation. A third strategy was mentioned but had not been used: 'decomposition' of more global tasks into subtasks. The discussion about the mutual advantages of using either professional operators or non-professionals (like e.g. students) - with some special training - as subjects was also very intense and revealed some new aspects. A very interesting one was that careful evaluation of the experiments allows to normalize the measurements in such a way that the results for both groups of persons are comparable. It also turned o u t that in really novel situations the 'experts' may have to resort to basic knowledge and therefore behave like novices. In various cases it had been observed that professionals had mental reservations against the 'new-fangled' interfaces with which they had to work. So, taking also into account the pragmatic point of view that it costs much more to employ professionals as subjects, a reasonable approach

110

Discussion Session H

appears to be that carefully trained non-professionals a subjects are good choice in most cases, but that the results of such experiments should be cross-checked from time to time by experiments with professionals. Triggered by this topic a discussion ensued about the role of user participation in interface design. It was generally agreed to be good method, but with some drawbacks. So e.g. it has been observed that experienced personnel is used to the existing interfaces in such a way that they simply can not think of basic innovations. There have always to be specialists who propose the innovations - and these have to be tested for acceptability in a dialogue with real users. This latter procedure was also recommended for the taxonomy of interfaces proposed by some project partners. It was widely agreed to be a good starting point for a framework for further conceptual work on interface design but it needed further discussion (like the one mentioned~ and - eventually - extension by a fourth dimension: the discrimination between 'active' and 'passive' interfaces. 'Active' meaning in that context that the operator always has the initiative in searching for and identifying relevant information, 'passive' meaning the predominance of signals and cues that guide and influence his behaviour.

Summarized by Peter F. Elzer

Multimedia Interfaces and Process Control: The Role of Expressiveness

James L. Alty IMPACT Research Group, Department of Computer Science Loughborough University, Loughborough, UK Email: [email protected]

Abstract: The concepts of medium and multimedia interfaces are defined and input and output media are examined. The problem of how and when to use alternative renderings in process control interfaces to maximise operator performance is considered, and the idea of expressiveness of a medium is explored. It is related to an Information Signal to Noise ratio idea and some examples of its use in a process control experiment are given.

1. Why Multimedia Interfaces? Many of today's industrial plants are controlled by supervisory and control systems that increasingly reside in software and hardware. The successful and safe operation of a plant depends upon careful engineering of the hardware and the software that drives it. However, the skill of the operators, their understanding of what constitutes an acceptable system state, and the implications of deviations from such states, are also very important. Their understanding clearly depends upon appropriate training and experience, but an additional key factor is the design of the interface between the operators and the system. Do these interfaces clearly communicate the relevant information whilst at the same time avoiding irrelevant information? Are they designed so that they take advantage of human cognitive characteristics rather than compete with them? Are the various media involved in communicating information to the operators being used appropriately and effectively? Recent developments in interface design have included advances in multimedia technology. The question has therefore arisen as to how such technology can be used in interface design to improve operator performance. The term "multi-media" came into common usage in the late 1980's. It describes a set of techniques and technologies for facilitating communication between computers and human beings through the use of a variety of "media". The term "medium" when applied to interface design essentially means a language (or

114


medium) of communication. Since languages consist of a set of allowable symbols (e.g. words in written English), a set of rules for putting these symbols together (e.g. English syntax), semantics (what the word collections actually mean), and pragmatics (conventions and common usage), media can be described in a similar way. Using such a definition one can show the difference between the media of written and spoken language. We often talk about languages as being "expressive". Expressiveness here means being able to communicate what we need to say, with the required subtlety, in a concise manner. In like manner, some media are rich and expressive, whilst others are sparse and very limited in what they can express (compare written language and semaphore). However, richness, or expressive power, does not always mean bestfor-purpose. The expressiveness of the medium should ideally match that of the task to be carded out at the interface [ 1]. The medium should be complex enough to be able to represent the problem, but not so complex as to hide or obscure the meaning. Human beings have employed a variety of media to communicate with each other since the earliest times. It is the degree of success with which human beings use multiple media that has convinced interface designers of the benefits of multimedia communication. Some researchers believe that multimedia communication is a natural product of evolution and corresponds closely with how the brain has developed. Marmollin [2] has described multimedia as exercising "the whole mind". An extension of this viewpoint sees the human brain as having evolved in a multi-sensory environment, where simultaneous input on different channels was essential for survival. The processing mechanisms of the human brain have therefore been fine-tuned to allow for simultaneous sampling and comparison between different channels. When channels agree, a sense of safety and well-being is felt. When channels degrade, input from one channel can be used to compensate another. Thus, input channel redundancy (within limits) may well be an essential part of human-human communication.

2. T y p e s o f M e d i a Media can be subdivided into input and output media. These can then be further divided according to the sense used to detect them - visual, aural or haptic media. A further subdivision can then be effected (for example, into language and graphics for visual output media, or sound and music for aural media, see Table 1).


Aural

Visual

Haptic

Input Media

Natural Sound Spoken Word Synthesised Sound

Video Camera TextScan Diagram Scan Gesture Eye Tracking

Output

Natural Sound Music Synthesised Sound Spoken Word

Written Text Graphics Animation Still Video, Moving Video

Keyboard Mouse Breathing Tube Data Glove Touch Screen Tracker Ball Data Glove Braille Pad

f

Media

115

Table 1: Some Common Media.

2.1 Output Media Many current output media are well-matched to human capabilities. Media such as text, graphics, pictures, video and sound, map well onto books, pictures, and the world we see and hear. Although normal sized VDU screens do not have quite the same properties as A4 paper, the correspondence is close enough. Most human beings therefore have little trouble in adjusting to understanding well-designed visual or aural output media. The problems of designing effective output using these media are similar (but not the same) as those in traditional media design (for example, publishing). Most computers can support quite sophisticated aural output (e.g. music), but this is rarely utilised except in games. Gaver [3] has suggested the use of Auditory Icons. These are well-known, natural, sounds that have common associations. Blattner [4,5] has suggested the use of structured Earcons (based on simple musical motifs. The author [6] has mappings run-time computer algorithms to music, mapping the internal workings of an algorithm to musical structures. Musical mappings have also been suggested to aid computer program debugging [7,8]. Output text has been extended through the creation of Hypertext structures [9]. Hypertext linkages across communication networks have now become commonplace, an obvious example being the Web, and the term Hypermedia is often used to describe the creation and support of linkages between different media. Elements of text may be linked to photographs, movies or even sound sequences either on local systems or across the communication network. One current problem in Hypertext and Hypermedia structures, however, is navigation. Users can easily become lost in Hypermedia space. Non-dynamic links can also inhibit exploratory learning. This is partially because the hypertext approach has no obvious counterpart in normal human activity.

116


2.2 Input Media Current input media are cumbersome and unnatural, requiring skills (such as keyboard or mouse skills) to be used effectively. Input media often need to be coupled with some form of output medium to be useful. Keyboard input is not effective unless the user receives simultaneous output of what is being typed. In a similar manner, input using a mouse requires visual feedback to be effective. This complicates the analysis of input media. Recently, there has been active research on new input media. Developments have been reported on voice recognition (now beginning to reach acceptable levels of performance), gesture and pointing (where the actual visual gestures are tracked by video cameras and interpreted). Other work involves eye-movement (the actual movement of the eye is tracked and used as a selection device), lip motion (to assist in speech recognition), facial expression and handwriting, and even brain wave input. The research is driven by the current primitive state of input media in contrast to human-human communication. An interesting feature of many input media is their impreciseness. Voice recognition is difficult because of extraneous noise, gesture is often vague and ambiguous, and lip motion is not read accurately by most human beings. Such imprecise media are still useful, because human beings process such media in parallel (for example, gesture and lip movement usually accompany speech, exploiting redundancy across these channels). Human beings therefore compare inputs across the different channels for confirmation, or to seek support for the interpretation in one channel by another. Experiments on input media have involved the combination of speech recognition with lip reading, gesture with speech, and speech with handwriting . One experiment concerned the simultaneous input of lip reading and voice input. The acoustic input performance was measured in clean and noisy environments [10]. When the acoustic input was clean, a word accuracy in excess of 90 % was attained. The Lip reading performance, on its own, varied between 32 and 47% accuracy, and, when used in parallel with the acoustic input had minimal effect on overall accuracy. When the noisy acoustic input was used, however, acoustic recognition on sit own fell to around 50% but with lip-reading added in parallel, performance improved to over 70 %. Thus, adding the lip reading input (which had a relatively poor recognition rate on sit own) boosted the recognition rates of acoustic input in the noisy environment.


117

2.3. The Importance of Alternative Renderings Although an important aspect of interface design is choosing the ,,obvious" or "best" medium for particular communication requirements, the deliberate presentation of information in a ,,foreign" medium can deliver new and interesting insights to a problem. Musical harmony is normally presented through the audio channel, yet new insights into harmonic progressions can be obtained by displaying the harmony in a visual medium. A good example of this is the HarmonySpace application of Holland [11]. This tool offers both experts and beginners the opportunity of exploring musical harmony by allowing them to use spatial attributes in their exploration (e.g. nearness, centrality and shape similarity). Similarly, music can be used to assist in the understanding of computer algorithms or physical processes such as turbulence [12]. This use of unusual media for presenting information can be an aid to problem solving in difficult process control situations. It is well-known that operators often get side-tracked into pursuing false explanations for control problems even when the current facts contradict them. They make strenuous attempts to fit every new fact that emerges into their current framework. A new viewpoint using a different medium might shift operators out of these blind alleys.

3. Media Characterisations The Multimedia design issue is not primarily a technical one. The issue is one of choosing an optimal set of media to achieve a particular goal. In process control interfaces, our goals may be: 9 9 9 9 9

performing the task more quickly performing the task with fewer errors making the task easier making learning and remembering the task easier achieving safer operation

The proponents of Multimedia technology hypothesise that good multimedia design, should assist designers in reaching these goals. What is needed is a characterisation which enables designers to map task characteristics to media properties. Media have different capabilities for communicating different types of information. Figure 1 shows a simple space where some properties of various media are illustrated.

118


The labelled axes are: Real to Abstract (by how far is the representation connected with reality?) Static to Dynamic (What role does time play in this medium?) Quantitative to Qualitative. (How far can the medium handle numbers and ratios?)

.......................................S c h = m l j'lcture

Qualitative

.."

fftative

Keal

9

~

AOstract

Figure 1: A Media Space Contrasting the Properties of Different Media. Text is highly Abstract, Static and can be Quantitative. Moving Video is Qualitative, Real and Dynamic. Trend .Graphs are Dynamic, Abstract and Qualitative. Of course these are just typical examples. Actual designation will depend upon their use in any particular situation. For example, a TrendGraph could be reasonably Quantitative and a Video very Abstract. The figure does highlight differences between media and indicate how they might be effective under different condition.

4. The Characteristics of an Acceptable Representation We can assert that all acceptable representations must have at least the following properties: a) all the information required must be currently available (or accessible) in the representation chosen. b) the information should be presented in such a way as to be readily perceived and understood in the right context by the user. This means that the representation should generate, or match, some form of model in the user's head (an internal representation).


119

c) other information, not relevant to the problem solving process, should be kept to a minimum. Designers manipulate the external representation (that is, the set of media chosen to communicate), to make particular operator tasks as easy as possible. To do this the external representations must generate (or match with) the most effective internal representation to enable the operator to solve the problem. What sort of representations might best be used? An obvious start might be to use reality. In the process control case the designer could show actual pictures of the plant and its components and (perhaps) video data of the process in action. In educational programmes the emphasis would be on real pictures and videos (though this might be difficult in Mathematics). Unfortunately, realistic representations tend to fail points a) and c). This is because most of the information provided by realistic representations is not useful for many purposes and tends to add to the information noise. In Process Control, for example, a schematic diagram of a valve is usually more relevant and useful than a picture of it. In educational situations, schematic diagrams are usually more meaningful than realistic pictures. In spreadsheet and word processor applications, the last thing a user wants is a realistic representation of what is happening inside the computer. Process control designers, therefore, normally use representations that are more symbolic than real. However, this provides an immediate problem. Once designers move away from realistic representations, the choice of representations becomes very large. Sometimes the choice may not be too difficult. There may be a set of standardised symbols for an application (Pipe and Instrumentation Diagrams in Process Plants, for example). At other times the choice may be completely open. This therefore is the key issue in Multimedia Design - How does a designer choose a medium (or media) to represent a problem in such a way as to maximise the operator's capability of solving it? There are obvious examples that illustrate this point. Suppose a group of chemists wish to solve a problem involving the gross topology of a complex molecule. They could be presented with a complete listing of the electron density values at every point in space. Alternatively, they could examine a 3-dimensional representation of electron density values where colour is used to show the distribution of electron density in space. In both cases all the information is there to solve the problem. However, the listing would require hours of work, whereas the 3-D view might answer some questions almost instantaneously. Clearly, for this problem, one representation is more appropriate than another. This example tells us that in trying to measure the "effectiveness" or "expressiveness" [ 13] of a medium for a particular task, we are seeking to calculate something like a signal-to-noise ratio for some medium trying to meet some operator objective. In other words, the critical information is the signal and all the additional irrelevant information provided by the representation is the noise. The

120


calculation is not easy since the definitions of signal and noise will vary with the task, and the capabilities of the user. One might think of it simplistically as an Information signal-to-noise ratio: Essential Information required to solve problem. Total Information presented In the molecule example, it is not hard to see that part of the problem is a surfeit of "noise" raising the value of the denominator. Another way of measuring this signal-to-noise ratio might be to determine the work required by an operator to extract the required information to solve the problem. This is still hard to calculate but it does at least provide a way of measuring it. This would be appropriate for performance problems but may not be appropriate for educational situations. In these cases, a better measure might be concerned with how much knowledge was imparted, additional work being part of the learning process.

5. Examples from Process Control Because of space limitations we will provide two examples from work done in the PROMISE project [14] (supported by the EC in the ESPRIT programme). In this work a set of laboratory experiments were carried out using different combinations of media to render the same problem. The problem was the well-known Crossman's water bath [15], and the different media included text, scrolling text, graphics, dynamic trend graphs, sound and speech. A large number of subjects were evaluated carrying out a set of procedures using these different media. The tasks which subjects performed could be classified at three difficulty levels [16] depending upon the task compatibility, described as levels 1,2, or 3 (3 being the most difficult). One experiment concerned a comparison between textual and graphical representations. Three variables were measured - the time taken to stabilise the system from a disturbed state, the number of actions performed, and the number of times the state variables were outside a defined envelope of behaviour (warning situations). Figure 2 shows the results for graphics and text over the three task difficulty levels. It is interesting to note that, as the task complexity becomes higher, the graphical representations start to provide improved performance in all three measures. At low task difficulty there is no difference in performance. Here the expressiveness of the graphical medium provides no advantage over the simpler textual medium at low complexity. As the task difficulty increases, information which previously was "noise" in the graphics now becomes signal and performance improves. For difficult tasks, the expressiveness of the graphical representation better matches the task requirements.

Multimedia Interfaces and Process Control." The Role of Expressiveness

Time

300

Actions

12

j6

200

100

,

.

0

.

0

-

9

1 2 3 Task Difficulty "'-~"

Textual Graphical

Warning

4

situations

/

3

.: .

121

9

/

1 -

1

i,

9

2

-

m

-

|

3

-

9

1

-

9

2

-

m

3

Category 1 - compatible tasks Category 2 - incompatible tasks Category 3 - incompatible tasks

Figure 2: Effect on Performance of Graphics and Text. Another example concerns the use of Sound. The sound of flowing water was added to give an indication of the speed of filling of the tank, and the current water level. When performance (overall) was compared in the sound versus no-sound conditions, the results marginally favoured the no-sound condition (Table 2). Condition

Completion Time

Number of Actions

Sound No Sound

124 107

6.99 4.73

Number of Warning Situations 0.906 0.5

Table 2: The Effects of Sound. However, when the task difficulty was separated out, an interesting Figure 3 Effect of Task Difficulty on Sound Performance result was obtained (Figure 3). A clear cross over can be seen. It appears that at low task difficulty, sound was part of the information noise, but that at greater task difficulty it became part of the information signal. This Information Signal to Noise concept can therefore be useful for describing the usefulness of different media in varying task situations.

122


Actions

Time

300

200

Warning situations

/

6

~7/ 9

9

"1"

! 1

8

J

100

/

10

"2"

"3"

. !

4

9

. 2

9

.

,

3

2 9

"I "

"2 "

'3

Task Difficulty

............

no sound sound

C a t e g o r y 1 - c o m p a t i b l e tasks C a t e g o r y 3 - incompatible tasks) C a t e g o r y 2 - i n c o m p a t i b l e tasks

Figure 3: Effect of Task Difficulty on Sound Performance.

6. Conclusions The definition of expressiveness of a medium is an attempt to capture the complex process of relating medium choice to task requirements in interface design9 Expressiveness is an elusive quality but it is connected with the levels of abstraction afforded by a medium and its relationship with the task needs. We have reformulated the expressiveness idea as a Signal to Noise Ratio9 Using this approach we have been able to offer an explanation of some Process Control results in terms of the relationship of signal to noise and the movement of information between these labels..

References [I] [2]

[3] 14] [5]

Williams, D., Duncumb, I & Alty J.L., Matching Media to Goals: An Approach based on Expressiveness, Proc HCI'96, People and Computers XI, (Kirby, M.A.R, Dix, A.J., & Finlay, J.E., eds.), ,Cambridge University Press, pp 333 - 347, 1996. MarmoUin, H., Multimedia from the Perspective of Psychology", in Kjelldahl, L., (ed.), Multimedia: Systems Interactions and Applications, Springer-Verlag, Berlin, pp 39 52, 1992. Gaver, W.W., Auditory Icons: Using Sound in Computer Interfaces, in Human Computer Interaction, Vol. 2, No. 1, pp 167 - 177. 1986. Blattner, M., Greenberg, R., and Kamegai, M., Listening to Turbulence: An Example of Scientific Audiolisation", in Multimedia Interface Design, Blattner, M., and Dannenberg, R., (eds.), ACM Press, Chapter 6, pp 87 - 102. 1992. Blattner, M9 Sumikawa, D., and Greenberg, R., Earcons and Icons: Their Structure and Common Design Principles, in Human Computer Interaction, Vol 4, No. 1, pp 11 - 44. 1989.


[6]

[7] [8]

[9] [10] [11]

[ 12]

[13]

[14]

[15]

[16]

123

Alty, J.L., Can we Use Music in Computer-Human Communication ?, in People and Computers X, Kirby, M.A.R., Dix, A.J., and Finlay, J.E., (eds.), Proc. of HCI'95, Cambridge Univ. Press, Cambridge, pp 409 - 423. 1995. Francioni, J., Albright, L., and Jackson, J., Debugging Parallel Programs Using Sound", in ACM SIGPLAN Notices, Vol. 26, No. 12, pp 68 - 75. 1991. Alty, J.L., and Vickers. P., The CAITLIN Auralization System: Hierarchical Leitmotif Design as a Clue to Program Comprehension, Proceedings of 4'h ICAD, (Mynatt, E., and Ballas, J.A., eds.), Santa Fe Institute, ISBN 0-9661289-07, pp 89 - 96. 1997. Nielsen, J., Hypertext and Hypermedia, Academic Press, London, pp 263. 1990. Waibel, A., Tue Vo, Minh., Duchnowski, P., and Manke, S., Multimodal Interfaces, The Artificial Intelligence Review, pp 1 - 23. 1995. Holland, S., Interface Design Empowerment: A Case Study from Music, in Multimedia Interface Design in Education, Edwards, (A.D.N., and Holland, S., eds.), NATO AS! series F, Springer Verlag, Berlin, pp. 177 - 194, 1992. Blattner M., Greenberg, R.M., and Kamegai, M., Listening to Turbulence: an Example of Scientific Audiolisation, in Multimedia Interface Design, (Blattner, M., and Dannenberg, R.M., eds.), Chapter 6, pp. 87 - 102, Wokingham: ACM Press, 1992. Williams, D.M.L., and Alty, J.L., Expressiveness and Multimedia Interface Design, Proc. of Edmedia-98, (Ottman, T., and Tomek, I. Eds.), Freiburg, Germany, pp 1505 1510, 1998. Alty J.L., Bergan, M., Dolphin, C., & Schepens, A., Multimedia and Process Control: Some initial Experimental Results, Computers and Graphics, 17 (3), pp 205 - 218. 1993. Crossman, M., and Cooke, J.E., Manual Control of Slow Response Systems, The Human Operator in Process Control, (Edwards, E., and Lees, F., eds.), London, Taylor and Francis, 1974. Sanderson, P.M., and Verhage, A.G., and Fuld, R.B., State Space and Verbal Protocol Methods for Studying the Human Operator in Process Control, in Ergonomics, Vol 32, No. 11, pp 1343 - 1372, 1989.

Acknowledgements: The author wishes to acknowledge a number of useful discussions with Prof. Peter Elzer of Clausthal University, Germany that helped to clarify the idea of expressiveness particularly in the process control domain. These discussions were made possible by sponsorship from the COPES-PROJECT of the EC, and took place during the summer of 1998.

Ecological Interface Design: Some Premises

John M. Flach Psychology Department, Wright State University, Dayton, OH 45435, USA e-mail:j.flach @desire.wright.edu

Abstract: This chapter presents three premises of an ecological approach to humanmachine systems. The first premise is that human-machine systems are dynamic, closed-loop systems that require a circular view of causality. The second premise is that the behaviour of these dynamic systems can best be understood in terms of the constraints in the functional workspace. These constraints include design intentions (e.g., functional goals), physical laws, organizational structure, and physical process and form. The final premise is that the explicit representation of the workspace constraints in the interface will greatly facilitate performance and will enhance the overall stability of the human-machine system.

Despite incredible advances in the development of automated control systems that are capable of closing many of the inner loops in complex work domains (e.g., energy production, advanced manufacturing, or aviation) human operators are ultimately responsible for controlling these work processes. That is, a human operator must monitor the system, compare the state of the system to normative expectations and functional objectives, and ultimately intervene in a way that will compensate for any deviations that are observed. At some level (more likely at multiple levels) the loop is closed through one or more human operators. Thus, stability of the system depends, in part, on the humans' ability to perceive deviations and to act appropriately to correct those deviations. Thus, whenever a system breaks down, it will almost always be possible to trace back and find that human actions were on the error path. That is, the human made an incorrect action, failed to detect a significant deviation, or failed to diagnose the deviation (i.e., correctly compensate for the deviation). Thus, it is tempting to identify human error as the "cause" in many accidents. However, since error trajectories are often unique, it is difficult, based on analysis of the time histories (causal trajectories) of these events, to draw general principles that will help in the design of safer systems. An ecological approach attempts to take a broader holistic view that looks beyond activities (behavioural trajectories) to consider the landscape (i.e., ecology) that shapes trajectories within a work domain. This chapter will consider some of the premises that motivate an ecological approach to the analysis of work domains and to the design of interfaces. The chapter is organized into three major sections. The first section considers the nature of the coupling between perception and action. The second section discusses the identification of constraints as a fundamental goal of analysis and as the semantic foundation for building interfaces. The third section discusses the specification of constraints within representations as a critical factor for skilled control.

126


1. Perception-Action Coupling Controlled action within complex work domains almost always depends on some degree of feedback. Feedback allows the system to monitor progress with respect to functional objectives and to minimize variations due to disturbances from the environment (e.g., a change in energy demands, a breakdown in equipment, or a change in wind direction). Thus, in complex work domains we are almost always dealing with closed-loop systems. This has several important implications for how we think about these systems. The first issue is the nature of explanation. For example, suppose you were trying to understand why an aircraft suddenly veered away from its normal landing approach path? You could say that this manoeuvre was caused by a particular force to the control stick that was translated into changes in the flight surfaces that altered the aerodynamic forces on the aircraft. You could dig deeper and say that the force on the control stick resulted from a particular pattern of muscle contractions, that in turn were caused by a pattern of electrochemical stimulation, etc. As you follow the causal path deeper into the system it is not clear that deeper insights with respect to the question of "why the aircraft veered suddenly" will result. This analysis helps us to understand "how" the manoeuvre was accomplished, but does not help to answer "why." On the other hand, we could ask about the pilot's intentions. She was trying to avoid collision with another aircraft that had taxied onto the runway. Now we are getting somewhere. Again, we can probe deeper and ask questions about why this manoeuvre and not another? (Is this a satisfactory response to the situation? Is it optimal?) Or we could ask why would she want to avoid a collision (that is what are the costs or values associated with collision compared to alternative outcomes). This line of questioning seems more productive and better suited to the question of why. For physical systems, we are most concerned with "how" the system works. How do the component forces sum to produce the behaviour? But for closed-loop control systems this is not enough. We must also be concerned with the question of "why the system behaves as it does?" This involves consideration of intentions (i.e., goals or purposes) and values (i.e., costs and benefits). This also involves consideration of information and feedback. That is, how can the physical behaviour be compared with goals and values in a way that will result in "error" signals that in turn can be translated into appropriate actions? Thus, we must be concerned about the information coupling that allows co-ordination between system behaviour, intentions, and control actions. This coupling involves both hard/force links (e.g., manipulation of a control stick) and soft/information links (e.g., the problem representation at the interface). Rasmussen [14], [15] provides an important discussion of the need to understand "how" and "why" and of the implications for work analysis. The key point to understand is that information (and the lack of information) is fundamental to system stability (or instability). A second issue, related to the nature of the perception-action coupling, is the very nature of causality. In "physical systems" the future is determined or caused by past


127

events. Thus, it is often possible to trace backward along behavioural trajectories to identify events in the past (e.g., forces, stimuli) that combine to determine behaviour of the physical system (e.g., motion, responses). However, this type of reasoning breaks down when analyzing closed-loop systems. For example, if a servo-mechanism becomes unstable, it is unlikely that the "cause" of the instability will be found by tracing back along the time history of the process to find an event or events that "caused" the instability. Stability in these systems depends on the coordination between information (feedback) and action. Instability is generally caused by global properties of the system (e.g., forward loop gain in relation to the effective time delay). Identifying these "causes" of instability requires a more global analysis of the system. The organization and the coupling of forces and information must be evaluated in terms of global constraints that generally are not evident from analyses of time histories alone. The key point here is that the linear causal reasoning that works well for the analysis of inanimate physical objects (e.g., the motion of planets) will not work as well for closed-loop control systems. Yes, control systems are deterministic systems. Yes, they are constrained by the same physical laws that constrain inanimate physical systems. However, closed-loop systems have a higher level of organization. This organization introduces global constraints on behaviour that require a different logic of causality --- a logic that is sensitive to the circular coupling of perception and action and the resulting ability to adapt to constraints in the environment [2], [3]. Thus, the ecological approach is a challenge to get beyond the logic of simple stimulus-response chains to examine the role of global (systemic or ecological) constraints in shaping system behaviour.

2. Analysis of Constraint The idea of constraint as a causal factor in determining system behaviour and thus as an object for scientific investigation is perhaps most explicitly shown in the emerging field of non-linear dynamics (e.g., chaos theory). Abraham and Shaw's [1] books illustrated the constraints that determine the behaviour of non-linear systems as three-dimensional landscapes. The hills (repellers) and basins (attractors) in these landscapes represent constraints that shape behavioural trajectories of the dynamical systems. Although Gibson's [6], [7] ecological approach to perception and action predates popular interest in non-linear dynamics, there are obvious parallels between the logic of his approach and the logic that is emerging from the field of non-linear dynamics [11], [12], [17]. This is most clearly seen in Gibson and Crook's [8] analysis of driving, where they introduced the construct of "safe field of travel." The "safe" regions represent basins of attraction and the objects of potential collision are repellers on the ecological landscape that shape driving behaviour. The term affordance was introduced as a construct that reflects the "attractive" or "repelling" nature of the behaviour ecology. The affordances are the opportunities for action in the ecology. The challenge for an

128


ecological approach is to understand how these basins of attraction and repelling regions (i.e., affordances) arise from the dynamics of perception and action. It is very important to differentiate between the concept of constraint and of state. The states of a system reflect the dimensionality of the process. In other words the states are the coordinates for all points on behavioural trajectories through the workspace. The landscapes (such as those depicted by Abraham and Shaw) show the impact of external constraints on the workspace, but the sources of the undulations in the landscape are not the state variables themselves, but constraints on those variables. For functional work domains these constraints include goals and values (as might be reflected in the cost functional of an optimal control problem). They include physical laws that constrain motion within the state space (as might be reflected in the process or plant dynamics in an optimal control analysis). Note that to determine the "optimal" control law --- the analyst must have an explicit model of both the physical laws (dynamic equations) and the value system (cost functional). Again, Gibson's ecological approach to perception attempted to go beyond pure "state" descriptions of stimulation to better capture the functional constraints that are very important factors when attempting to account for skilled adaptation (i.e., optimal control) to a task ecology. For simple systems (e.g., a laboratory tracking task) the physical laws are known, the goals are clearly specified (e.g., minimize RMS error), and we have good engineering heuristics for approximating the value systems (e.g., the quadratic cost functional). For these laboratory tasks, the task analysis has been done --- that allows application of analytic control models (e.g., the optimal control model) that can provide important intuitions about human and system performance. It is tempting to generalize to complex work domains using these analytical models that worked in the laboratory contexts. However, an ecological approach is a challenge to look past these simple analytic models to the task analyses that went into their construction. The ecological approach challenges researchers to analyze the complex work domains with the same rigor that was given to the analysis of the laboratory tasks. That is, develop valid models of the work dynamics, identify the functional goals and the values or figures of merit that reflect the cost/benefit tradeoffs associated with system behaviour and control action. The expectation is that the "models" for the complex domains may be very different than the models that worked for laboratory tasks. These models will reflect the particularities of the complex work domains. Thus, the ecological approach takes advantage of the analytic toolbox of systems engineering for developing models of the work ecology, but it is sceptical about generalizations based on analytic models derived from simple laboratory task ecologies. A good example is the work on arm protheses that was done by Stassen's group at Delft [16]. The analysis of the prothesis problem included detailed modelling of the physical constraints, but also intensive field investigations to study the ecology of use. Lunteren and Lunteren-Gerritsen [13] describe one of the field analyses:


129

9 . . an observation method was chosen in which a child with a prothesis is followed over a normal school day from the moment he or she starts dressing until going to bed in the evening. If possible, a day is planned that includes both a physical exercise class and a class involving manual activities. During the day all actions of the child, with or without a prothesis are observed and the ways in which they are executed are noted. This means that the list o f activities is not standardized, which makes a comparison more difficult. However, the only question being asked was f o r what activities do the children use their prosthesis. Moreover, it was felt that in a standardized test situation the child might try to do things differently from his or her normal pattern, i.e., in a way that was thought to please the observer. Besides the functions that were used for a certain action, it was also observed whether the way an action was executed was unobtrusive or involved some unnatural-looking motion (the cosmesis o f using). The way the child moved while not performing any action with the prothesis (the cosmesis of wearing) was also observed (p. 99 - 100)

The work of the Delft group illustrates a commitment to understanding the ecology of use. As opposed to a more common practice were the commitment is to a particular "model," and those aspects of the ecology that don't conform to the model are ignored. Note that a commitment to understanding the ecology does not require throwing away models. Rigorous analytic control models also played an important role in the Delft work. It is simply a matter of priority. Analytic models can be seductive, so that the task analysis reduces to an identification of those aspects of the work domain that correspond to model parameters. An ecological approach gives priority to understanding the domain constraints. This understanding becomes the context for intelligent application of analytic models. Another clear example where priority was given to understanding the task ecology is Hutchins' [10] analysis of ship navigation. While neither Stassen nor Hutchins would typically be identified with an ecological approach --- the commitment to understanding the natural constraints within work domains that their work illustrates is fundamental to an ecological approach. There are important qualitative differences among the kinds of constraints within a work domain that can have important implications for work analyses. These differences, in part, reflect the underlying nature of causality (i.e., the distinction between "how" and "why" discussed in the previous section). For example, the intentional constraints (functional purpose or design purpose) are qualitatively different than the physical constraints (natural laws). Yet, both kinds of constraints interact to determine the underlying topology of the work space. The qualitative differences have implications for both the level of abstraction and the level of decomposition that are most appropriate for analysis and design. Rasmussen [14], [15], [18], [20] has written extensively about these implications and has introduced the Abstraction Hierarchy as a framework that has been useful for analyses of many different work domains. Vicente has utilized the Abstraction Hierarchy very

130


effectively for the development of graphical interfaces [19] and for analysis of expertise [21]. The key point for this section is that an ecological approach includes the constraints within the work ecology as fundamental targets for analysis. This is consistent with control theoretic and general systems approaches to analysis (particularly evident in the analysis of non-linear systems). However, this insight has often been lost as researchers have tended to focus on the analytic models from control theoretic analyses of laboratory tasks, rather than on the standard practices of control analysis.

3. Degree of Specification In the previous section, the argument that constraints are fundamental to understanding work domains was introduced. This is true from the point of view of the analyst and from the point of view of the operator. For operators to perform skillfully in complex work domains there must be information that allows them to recognize the best opportunities for action. Classically, the recognition problem has been modelled as a computational problem. That is, cognitive psychology has generally assumed that the opportunities for action are underspecified and that logical inferential processes must interpret the sparse information to make informed guesses about the situation. For example, it seemed obvious that a two-dimensional retina could at best only provide hints or cues about the true three-dimensional structure of ecological space. Therefore, it has been assumed that there were powerful information processing systems in the visual system that allows animals to infer the three-dimensional structure from the cues provided by our sensory system. Consistent with this view of human information processing, operator support systems have been designed to either support the inferential process (help the operators to integrate information according to logical rules) or to replace the human (who has a limited capacity for following the dictates of logic) with an automated (expert) system with more powerful and reIiable computational engines. Gibson's theory of direct perception challenged the general assumption that the opportunities for action were always underspecified. For example, he argued that the dynamic optical array (structural properties of reflected light available to a moving observation point) provided direct specification of the three-dimensional ecology in a way that supports many skilful interactions (e.g., control of locomotion). He argued that it was "attunement" to the structure in the optical array (not computational inferences) that allowed skilled interaction (e.g., control of locomotion). The idea that structure in the optic array allowed direct closed-loop control of action is clearly expressed in Gibson's [5] description of visual control of locomotion.


131

The center of the flow pattern during forward movement of the animal is the direction of movement. More exactly, the part of the structure of the array from which the flow radiates corresponds to that part of the solid environment toward which he is moving. If the direction of movement changes, the center of flow shifts across the array, that is, the flow becomes centered on another element of the array corresponding to another part of the solid environment. The animal can thus, as we would say, "see where he is g o i n g . " . . . To aim locomotion at an object is to keep the center of flow of the optic array as close as possible to the form which the object projects. (p. 155) The construct of Ecological Interface Design (EID) is in part an extrapolation of Gibson's observations about direct perception to the design of graphical interfaces. The challenge is to design representations that provide as direct a coupling between perception and action as possible. The idea is to allow operators to "see where they are going" as they navigate over the functional landscape of the work domain. Again, this does not simply mean that the "states" of the process are represented, but that the whole landscape is made visible (i.e., specified). That is, the operators must be able to see the states in relation to the functional constraints. The representation must reflect the functional goals and values, the physical laws, the organizational constraints, and the physical process constraints that shape action in the workspace. Vicente's work on the DURESS interface is the most systematic and most thoroughly tested and documented application of Ecological Interface Design. A key feature of the DURESS interface is that the constraints on mass and energy balance are explicitly represented in the interface. Tanabe [17], [22] also has an impressive program to apply Ecological Design principles in the development and evaluation of the next generation control room for the nuclear power domain. However, a simpler example from our work in the aviation domain will be used to illustrate the representation of work constraints within an interface. Controlled flight into terrain (CFIT) is a significant problem for high performance military aircraft [9]. CFIT is a situation where an aircraft collides with the ground where there are no obvious mechanical failures, no medical problems, or no unusual weather conditions that might cause the accident. Thus, CFIT appears to be a clear example of operator error. We are currently preparing to test both a hypothesis about how structural properties of optical flow can be a potential contributing factor to these accidents and a display modification to help prevent these accidents. The hypothesis concerns an ambiguity in optical flow fields. As an observer moves through the environment the light reflected from texture surfaces streams by. The faster the observer moves the faster the texture flows by. However, the rate of flow also depends on the distances to the surfaces. The greater the distance the slower is the flow. Thus, the global optical flow rate is directly proportional to observer speed and inversely proportional to distance to the surface. For an observer who is at a constant distance from the ground, the optical flow can provide reliable information about speed of locomotion. However, when altitude and speed are both

132


varying, global optical flow rate is no longer specific to speed. Thus, a potentially dangerous situation is created when an aircraft simultaneously loses significant altitude and airspeed. The loss of altitude causes an increase in optical flow rate that may mask the loss of airspeed. Loss of airspeed is critical since without adequate airspeed the aircraft looses lift (i.e., stalls). When the pilot pulls up to avoid collision with the ground there is not adequate airspeed and the aircraft stalls and crashes.

Figure 1: The traditional attitude display shows pitch and roll as the relative position of a fixed aircraft icon and a moving horizon. The modification above includes a textured surface with lines that f/ow in a manner similar to stripes on a highway trader a moving vehicle. The rate of flow is proportional to the difference between current air speed and the minimum air speed for controUed flight (i.e., the stall boundary). In most military aircraft, airspeed is displayed either on a linear tape, a rotary dial, or as a digital value. It is not meaningfully configured with any other flight information. A pilot who is confident that airspeed is adequate (due to high optical flow rate) may not consult this instrument and thus may not realize that the aircraft is approaching a stall speed. To help prevent this we have designed a modification to the primary flight display that integrates an indication of airspeed with information about aircraft attitude. Figure 1 shows this primary flight display. The traditional display shows aircraft attitude as the relative position of an aircraft symbol and an indication of the horizon. The standard format in the US is for the plane to be fixed and for the horizon to move in a way that is consistent to the orientation of the aircraft with the horizon. When an aircraft lowers the right wing (banks right). The horizon rotates counter clockwise (banks to the left) until the angle between the plane symbol and virtual horizon matches the orientation of the aircraft and the real horizon. When the aircraft pitches up, the virtual horizon will move down in the display. This is called an inside-out display and was designed to mimic the view through a periscope that has a fixed orientation to the aircraft frame. We have modified the standard flight display to include texture lines on the virtual ground portion of the representation. These line flow downward similar to stripes on a highway flowing


133

under a vehicle. The flow rate is proportional to the difference between the current flight speed and the minimum speed need to maintain flight (i.e., the stall speed). Thus, the rate of flow is proportional to distance from the stall boundary. We think that there are two principal advantages of this modification. First, by configuring the airspeed with attitude information we hope to make the dynamic interactions among the flight dimensions more readily visible. That is, one potential cause of simultaneous loss of altitude and air speed is unusual orientation of the aircraft (e.g., high degree of banking). Thus, by integrating air speed with the attitude information we hope that pilots will be better able to recognize this interaction. Also, since this is the primary flight display, it is hoped that this will increase the probability that indications of air speed will be within the attentional field of the pilot. That is, we believe the change in flow rate within the primary flight display will be more salient than a change of position on a dial or a change in a digital value. The second advantage is that the flow rate is anchored to the stall speed. That is, the rate of speed is directly proportional to distance from the stall boundary. Thus, the relation between the state variable speed is represented in the context of the aerodynamic constraints that determine lift. The hope is that this will allow the pilot to "see" approach to the stall boundary (as opposed to having to infer this based on knowledge of aerodynamics). Certainly, this is not a difficult inference for skilled pilots. However, unless they recognize that they are approaching the stall boundary, there is no reason for them to sample the air speed. Again, by explicitly modifying the representation to reflect stall constraints this information is made more salient and thus it is more likely to contribute to the pilots' situation awareness. This is one of several modifications that are being considered to improve the interface representations available to pilots [4]. The key point is that providing "feedback" to the operators about state variables is insufficient to insure stable control. The state variables must be represented in the context of the work constraints. These constraints can be integrated within computational engines (e.g., expert systems) or they can be integrated within graphical representations. The ecological approach is most concerned with integrating the constraints into graphical representations. We think that this is the most effective way to utilize the unique skills that a human operator brings to the work domain. We fear that if the constraints are buried within computational engines, then the logic of these automated systems will be opaque to the human operator and thus it will be difficult for the human to trust the expert system. Again, it should be clear that there is an important role for expert systems in complex work domains. However, we think that coordination between these expert systems and human operators will be improved if the internal constraints that drives the computations are made explicit in the interface between human and automated system.

134


4. Summary and Conclusions In sum, an ecological approach starts with the recognition that the study of human performance in natural work domains is the study of dynamical control systems that are governed by a circular causality. This class of systems can not be understood based on an analysis of state trajectories alone. A broader analysis of the constraint topology (i.e., the ecology) is required. This analysis requires descriptions at multiple levels of abstraction and decomposition. Ecological Interface Design attempts to explicitly incorporate the designer's understanding of the constraint topology within a graphical representation. The goal is to build a representation where the states are represented together with the domain constraints so that the operators can directly "see" the functional implications of any action or event.

References [1] [2] [3] [4]

[5]

[6] [7] [8] [9] [10] [11] [12] [13]

Abraham, R.H. & Shaw, C..D. (1984). Dynamics: The geometry of behavior. Three volumes. Santa Cruz: Aerial Press. Flach, J.M. (1990). Control with an eye for perception: Precursors to an active psychophysics. Ecological Psychology, 2, 83 - 111. Flach, J.M. (1999a). Beyond error: The language of coordination and stability. In Hancock, P.A. (ed.). Human performance and ergonomics. San Diego: Academic Press. Flach, J. M. (1999b). Ready, fire, aim: A "Meaning-processing" approach to display design. In D. Gopher and A. Koriat (Eds.) Attention and Performance XVII: Cognitive regulation of performance: Interaction of theory and application. (197 - 221). Cambridge, MA: MIT Press. Gibson, J.J. (1955/1982). Visually controlled locomotion and visual orientation in animals. British Journal of Psychology, 49, 182 - 194). Also in E. Reed & R. Jones (Eds.) Reasons for realism. Hillsdale, NJ:Eflbaum. Gibson, J.J. (1966). The senses considered as perceptual systems. Boston: HoughtonMifflin. Gibson, J.J. (1979). The ecological approach to visual perception. Boston, MA: Houghton-Mifflin. Gibson, J.J. & Crooks, L.E. (1938/1982). A theoretical field analysis of driving. American Journal of Psychology, 51, 453-471. Also in E. Reed & R. Jones (Eds.) Reasons for realism. Hillsdale, NJ:Erlbaum. Haber, R.N. (1987). Why low-flying fighter planes crash: Perceptual and attentional factors in collisions with the ground. Human Factors, 29, 519 - 532. Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press. Kelso, (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: M1T Press. Kugler, P.N. & Turvey, M.T. (1987). Information, natural law, and the self-assembly of rhythmic movement. Hillsdale, NJ: Erlbanm. Lunteren, T. van, & Lunteren-Gerritsen, E. van. (1997). In search of design specifications for arm prostheses. In T.B. Sheridan & T. van Lunteren (Eds.) Perspectives on the human controller. Mahwah, NJ: Erlbaum.


135

[14] Rasmussen, J. (1986). Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering. New York: North Holland. [15] Rasmussen, J., Pejtersen, A.M., & Goodstein, L.P. (1994). Cognitive Systems Engineering. New York: Wiley. [16] Sheridan, T.B. & van Lunteren, T. (Eds.) (1997). Perspectives on the human controller. Mahwah, NJ: Erlbaum. [17] Tanabe, F., Yamaguchi, Y. & Rasmussen, J. (1998). Simulator experiments with ecological interface systems. JAERI Research Report. Japan Atomic Energy Research Institute. Thelen, E. & Smith (19). [18] Vicente, K.J. (1999). Cognitive work analysis: Towards safe, productive, and healthy computer-based work. Mahwah, NJ: Erlbaum. [19] Vicente, K.J. (1992). Memory recall in a process control system: A measure of expertise and display effectiveness. Memory & Cognition, 20, 356 - 373. [20] Vicente, K.J. & Rasmussen, J. (1990). The ecology of human-machine systems II: Mediating 'direct perception' in complex work domains. Ecological Psychology, 2, 207 - 250. [21] Vicente, K.J. & Wang, J.H. (1998). An ecological theory of expertise effects in memory recall. Psychological Review, 105, 33 - 57. [22] Yamaguchi, Y. Furukawa, H., & Tanabe, F. (1998). Design of subject training on reactor simulator and feasibility study: Toward an empirical evaluation of interface design concept. Paper presented at the Enlarged Halden Program Meeting, Lillehammer, Norway. Acknowledgments: Sincere thanks to Peter Elzer and Badi Boussoffara for the invitation to participate in the international workshop on "Human Error and System Design & Management." They were very kind and generous hosts. John Flach was supported by grants from the Japan Atomic Energy Research Institute (JAERI) and from the Air Force Office of Scientific Research (AFOSR) during preparation of this manuscript. However, John is solely responsible for the views expressed in this chapter.

Ecological Interface Design (EID) and the Management of Large Numbers of Intelligent Agents

John D. Lee, Thomas Geb, Emily Pollack The University of Iowa, 4135 Seamans Center, Industrial Engineering, Iowa City, Iowa, USA e-mail: jdlee @engineering.uiowa.edu

Abstract: Agent-based automation is an emerging technology that presents new

challenges if humans are to manage it effectively. Current automation has been plagued with mode errors, misuse and disuse. The properties of agent-based automation that make it most useful, those of an adaptive self-organizing system may lead to a poor calibration of operators' trust in the automation and a magnification of automation-related problems. The focus of ecological interface design to graphically display the underlying system constraints offers a promising design strategy to promote effective use of agent-based automation.

1. The Emergence of Multi-agent Automation Technology has dramatically changed the role of the human in complex systems, with the human operator's role having evolved from being primarily physical to primarily cognitive. Frequently, automation has been designed to perform single, specific functions and has limited interactions with other elements of automation. Future automation may not be so simple and may introduce new human factors challenges. For example, technological advancement may enable automation to evolve into agents that may become increasingly powerful, autonomous, and interconnected. Agent-based automation offers increased robustness, flexibility, and adaptability; it will be able to adapt and respond to many types of environments. However, understanding how to support the human supervisory control of these agents remains an unresolved issue. A concrete example of this evolution is planetary exploration. The current approach to remote planetary exploration mirrors the current application of many types of advanced automation. In the case of the Mars missions, this involves a single rover. The design of the rovers has been directed primarily towards building single,

138

Ell) and the Management of Large Numbers of IntelligentAgents

complex device. Designers develop the mechanical structure and software to accomplish specific tasks, such navigating a rock-strewn field. The primary limitation of this approach is that the entire mission has a high chance of failure if the demands of the mission depart from the initial assumptions due to unexpected terrain conditions or a rover malfunction. This lack of robustness in the face of unanticipated variability is an important limitation. The concept of multi-agent automation provides an alternate design paradigm that provides substantial redundancy that may make it possible to respond to environment variability while reducing the chance of system failure. Although this emerging design alternative offers substantial benefits, it is not clear how designers should best support the cognitive demands associated with managing agents. Figure 1 shows some of the complex display and control interactions that arise when a human must manage an agent-based automation. Humans may monitor and control individual agents, shown by the thin lines, or they may need to monitor and control groups of robots, shown by the wide arrows.

Figure 1: Many important challenges exist in supporting an effective and synergistic relationship between humans and agents.

1.1. Using multiple agents for effective system control Agent-based automation is an emergent technology that is likely to play an important role in many areas extending from power generation to space exploration. For example, spacecraft and Mars habitats are likely to have multiple agents controlling power, propulsion, navigation, communication, life support, and maintenance. Each of these agents will interact with and direct the actions of suites of more specialized agents. For example, the power system may have different agents monitoring solar panel alignment, fuel cells, power usage, and power generation. These agents may themselves direct multiple agents. In addition, software agents may control scientific experiments and apparatus on the spacecraft. Hundreds of agents may work simultaneously to ensure the safety and success of the mission.

ElL) and the Management of Large Numbers of IntelligentAgents

139

Agent-based automation may be distributed over a large area, enabling multiple jobs or data collection to be done simultaneously. This distributed control system may also be able to quickly adapt to new assignments and environments. This flexibility also allows an operator to redirect them easily . While agents are adaptive, they may also be only partially predictable because they may combine into a self-organizing system. They may interact with each other and the environment to produce emergent behaviours that may not have been foreseen. Specifically, self-organizing systems generate structure at a global level that grows out of interactions of lower level components. Insect colonies frequently exhibit self-organizing behaviour to achieve complex goals. Critical features of selforganizing systems are positive feedback, negative feedback, randomness, and multiple interactions [4]. The characteristics of self-organizing systems could provide great benefits if the supervisory role of the human controller could be effectively supported. Some have suggested an extreme case where agents autonomously control the system and people have no interaction with the agents. A more promising approach assumes a collaborative relationship between many agents and the human supervisor, whereby the human uses the agents as a tool to extend his or her capabilities. The problem is particularly interesting because the emergent behaviour of the multiple agents may not be directly inferred from the behaviour of the individual agent. This presents a substantial challenge to the design of an interface if it is to support effective management of the agent-based automation. 1.2. Example: Swarms of insect robots An important alternative to the current approach single-rover approach to planetary exploration is to engage many simple rovers in exploration. Robot swarms are large collections of small, relatively simple robots that are controlled using the concepts of agent-based automation. In a robot swarm, each robot is a semi-autonomous agent. The swarm concept is modelled after insects such as bees, ants, and termites that demonstrate robust and adaptive behaviour in a wide range of natural environments. Much inspiration has been drawn from the study of these social insects, and the findings associated with these studies have important implications for the design of swarms of insect robots. The success of insects suggests that robot swarms have a tremendous potential to amplify human capabilities in adverse and dynamically changing environments. In a natural environment, such as the pictured lunar surface, the swarm provides multiple viewpoints, distributed data collection, robust performance in the face of mechanical failures, and the ability to adapt to the complex terrain. These and other important features of swarms may be harnessed as a general tool to improve the safety and effectiveness of space and planetary surface missions; however, techniques must be developed to command and control swarm behaviour effectively. A burgeoning community of biology-inspired roboticists is actively exploring the possibilities of employing swarms of insect robots as an alternative to relying on one or two larger robots to assist astronauts in their missions [6], [11]. The insect

140

EID and the Management of Large Numbers of Intelligent Agents

robot concept is based on the notion that small machines with simple reactive behaviours, possibly adaptive or learning intelligence, can perform important functions more reliably and with lower power and mass requirements than can larger robots [3], [5], [10]. Typically, the simple programs running on the insect robot are designed to elicit desirable emergent behaviours in the insect swarm [2]. For example, a large group of small robots might be programmed to search for concentrations of particular mineral deposits by building upon the foraging algorithms of honeybees or ants. Other useful tasks that could be performed by insect robots are machinery inspection, farming, delivering materials, digging, and building. Organization and control of swarm behaviour stems from fundamentally different sources compared to traditional systems currently being monitored and controlled by people. A specific example of the robust and effective group behaviour that selforganizing swarms generate is the foraging behaviour of ants. The effectiveness of this behaviour could easily be generalized to information foraging of robot swarms as they explore a planet's surface, or to collection of contaminants on a spacecraft. Ant foraging involves a trade-off of speed of trail establishment and search thoroughness, a trail that is more quickly established will sacrifice the thoroughness of the search. Parameters that govern this trade-off include the strength of the positive feedback, and the amount of random variation [27]. Alternate means of control include template following (natural gradients/fields), and differential sensitivity of members to control signal. These control mechanisms offer great potential in extending human capabilities, but only if a thorough empirical and analytic investigation identifies the display requirements, viable control mechanisms, and range of swarm dynamics that can be comprehended and controlled by humans. A specific mechanism that underlies the self-organizing behaviour of swarms is stimergy communication, which allows insects to communicate through a dynamically evolving structure. Stimergy is a powerful alternative to a static set of instructions that specify a sequence of activity. Through stimergy, social insects communicate directly through the products of their work. This sort of communication promotes the swarm to evolve into a self-organizing system that can generate many forms of collective behaviour with individuals producing the same behaviour. Interaction between many simple individuals produces complex behaviour for the group [4]. For example, as a self-organizing system, the swarm could dynamically adjust its information or material foraging behaviour to a dynamic environment to accomplish its goals effectively. The performance of the swarm does not depend on the performance of an individual robot. This characteristic of swarm robots, in contrast to conventional automation, has important implications for the control of individual robots and the control of the overall swarm. Understanding the nature of the control mechanism has important implications for the human role in monitoring and controlling swarms. Seeley [25] argues that honey bees adapt to a changing environment at the level of the' group. A colony of

EID and the Management of Large Numbers of lntelligentAgents

141

honey bees functions as a large, diffuse, amoeboid entity that can extend over great distances and simultaneously tap a vast array of food sources [25]. The importance of the emergent behaviour of the swarm implies that the human must receive feedback from, monitor, and control the swarm and not just the individuals. Information and control at the level of individual agents is not likely to prove successful. New display and control paradigms may be required to discover how a human can effectively exert control over a swarm of individual robots. Organization and behaviour of social insects, suggests that control of agent-based automation may be fundamentally different as compared to traditional systems currently being monitored and controlled by people. The specific demands that these most sophisticated systems may impose on operators have not been explored. Failing to understand the cognitive consequences associated with these increasingly sophisticated supervisory control situations may negate any technical benefit they provide.

2. H u m a n Factors Issues of Multi-agent A u t o m a t i o n 2.1. Supervisory control and the new challenges of multi-agent automation Human supervision of multi-agent automation will become increasingly important. As more robotic and software agents are employed to extend the capabilities of humans, the operator's ability to successfully monitor and respond to unexpected events and agent behaviours will be severely tested. Agent-based automation represents a new type of automation that encompasses the human factors challenges witnessed with current automation and introduces new challenges. To be most effective, supervisory control of multiple agents would allow the operator to understand agent behaviours, predict future activity, and maintain control over the agents. In order to understand agent behaviour, the operator needs to identify how an agent's future state is related to its current state. Since this relationship may vary from agent to agent and over time, this information must be easy to assimilate. To predict the future state of the system the operator must be able to estimate the current state of the system and estimate the collective behaviour of the agents. To control the system, the operator must be able to redirect behaviour of either individual agents or groups of agents. To realize the potential of large numbers of interacting agents, agent-based automation must consider the design of the interaction of the agents, their interactions with a human supervisor, and the design of the human-computer interface. Agent behaviour that is counterintuitive, overly complex, or unpredictable may inhibit effective control. To create behaviour that can be effectively controlled requires an understanding of how system complexity is interpreted by people. New

142


analytic techniques may be necessary in order to identify human-centered design constraints on the behaviour of increasingly complex automation. Figure 2 shows a series of simple diagrams that illustrate some of the important changes in the relationship between the operator and technology. Each diagram represents an alternate supervisory control situation that is likely to have important implications for the control of multi-agent automation.

Figure 2: Possible methods for use in human supervisory control of multi-agent automation. a) The technology fills a relatively simple role of augmenting the humans perception and control. The operator requests a specific behaviour and the technology responds with a deterministic response. In this situation significant human performance issues emerge as the operator is forced to bridge the gaps between functionally isolated technological aids [ 1], [ 17], [ 18]. b) Here we have a more sophisticated sort of automation, which changes modes in response to the human operator, to other elements of automation, and to the environment. This sort of automation can extend operator capabilities in important ways; however, extensive literature on mode error illustrates the potential for inducing errors and degrading system safety and performance [8], [23], [34]. Providing information regarding both the domain task and the behaviours of the intelligent agent emerge as critical needs [ 19]. c) This scenario shows a qualitative change in supervisory control associated with an increasing number and sophistication of the agents. Because there are many interacting intelligent agents, macro-level behaviour may emerge that cannot be easily predicted by the behaviour of individual agents. The clouds show this macrolevel behaviour. In these situations, mode transitions may surprise the operator even if the operator understands how an agent responds to environmental changes. The operator is unlikely to successfully control this configuration of automation. While this added complexity can undermine the operator's understanding and monitoring efficiency, it supports a much more powerful and adaptive system. The operator is now responsible for managing the overall behaviour of a group of semiautonomous entities that interact in a complex and non-deterministic manner. This group may exhibit properties of a self-organizing system that could have very beneficial consequences, but they may also confuse the operator and lead to inefficient monitoring and management.

EID and the Management of Large Numbers of IntelligentAgents

143

2.2. Complex, counterintuitive agent interactions Multi-agent automation confronts humans with challenges not seen in current forms of automation. Mode errors, misuse, and disuse of automation could dramatically increase with poorly designed multi-agent automation. The factors that contribute to mode errors with current automation include indirect mode changes, inadequate feedback, and inconsistent behaviour [23]. The emergent behaviour of multi-agent automation may exacerbate all of these factors. The fundamental challenge is that multiple agents may interact to produce emergent behaviour that is not an intuitive extension of the behaviour of the individual agents. This emergent behaviour may be a very useful characteristic if it is properly designed and managed. For example, multiple agents could adapt their behaviour in novel ways to preserve overall system performance. The difficulty is trying to anticipate the emergent behaviour that multiple interacting agents may exhibit. Emergent behaviour can provide substantial benefits if it can be focussed on adaptive, goal oriented directions; however, emergent behaviour can also lead to emergent errors. These errors can pose a serious challenge to system performance and safety. Some important emergent error types include:

Loose Cannon - This is when an individual agent assumes an inappropriate influence over one or more subsystems. For example, if the agent monitoring the power reserves causes life support to shut down, the power reserves agent has become a loose cannon. Civil W a r - This occurs when two agents or groups of agents compete for the same resource, causing poor performance. If a laser range-finder on a Mars vehicle wants to steer left to avoid an obstacle, the vision system wants to steer right, and the wheels either oscillate or steer towards center, the range-finder and vision system are in civil war. The interaction among agents may occur directly, through software, indirectly, or through the environment. These interactions may result in Agent-

induced Instabilities. Software Causality - This occurs when an agent silently shuts down, or because of adaptation, it no longer influences the behaviour of the system. When the agent responsible for maintaining power reserves is overridden for immediate needs so often that the power distribution agent adapts for this strategy, the reserves agent may lose its priority and quietly allow the batteries to be depleted. Emergent behaviour of multiple agents introduces the potential for Macro-level and micro-level mode confusion. Attention and understanding of behaviour modes must be distributed between the macro (group behaviour) and micro (individual agent behaviour) levels. An improper balance of focus on one level may allow problems to go undetected on the other. The operator must be able to intervene effectively to stop problems before they become serious. This requires that the operator has

144


knowledge of the "normal," expected deviations for both individual and group behaviour. One advantage of agents, as opposed to integrated systems, is that they may be independently designed and tested. However, exhaustive testing, with either integrated systems or agents, is often a serious concern with automation. Testing for numerous failure modes in particular can become prohibitively expensive. When agents are combined, testing becomes exponentially more complex. Agents may interact directly or indirectly through the systems they control, which creates the possibility of a huge variety of errors that are difficult to predict and prevent. Designing interfaces that support effective agent management, particularly in unanticipated situations, may be the most effective means of combating the inevitable failings of multi-agent automation. An operator must be able to understand whether or not agents are acting productively so that he or she can proactively intervene and guide the agents to more productive behaviours.

2.3. Trust in Multi-agent Automation Appropriate reliance is a critical factor in the success of complex, multi-agent automation. Several researchers have suggested that operators' trust in automation may play a major role in guiding this collaborative relationship [12], [15], [16], [21], [26], [32]. Just as the relationships between humans are influenced by trust, so trust may mediate the relationship between the astronaut and the robot swarm. Research has shown that highly trusted automation may be used frequently, whereas operators may choose to control the system manually rather than engage automation they distrust [ 15]. Review of these studies allows the development of a multidimensional definition of trust including four dimensions: foundation of trust, performance, process, and purpose.

Foundation of trust represents the fundamental assumptions of natural and social order that make the other levels of trust possible. Performance rests on the expectation of consistent, stable, and desirable performance or behaviour. Having knowledge of current and historical performance results, the user expects similar ones in the future. This dimension corresponds to a user's perception of reliability or efficiency, having the automation produce results that the user expects. Process corresponds to a user's understanding of how the underlying qualities or characteristics that govern the performance of the automation match the current task demands. Understanding how the automation, algorithms, and control limits operate.


145

Purpose rests

on the user's perception of underlying motives or intention, such as why the automation was developed. With humans this might represent motivations and responsibilities. With machines, purpose reflects the user's perception of the designer's intention in creating the system. These dimensions identify general information with which a person calibrates their trust [15]. Ideally, trust in automation is calibrated at a high level of resolution so that people trust the automation when it achieves their goals and do not trust it when the automation cannot achieve their goals. It is difficult to predict the reaction human managers will have to a self-organizing group of agents. It is possible that certain emergent behaviours will induce poor calibration of trust. The consideration of different dimensions of trust and the role of trust on reliance suggests that the interface must be carefully structured to help the operator understand the control system, performance of the agents, and the factors guiding their performance. An effective interface would help calibrate trust by enhancing helping the understanding of emergent behaviours that otherwise might lead the operator to distrust the agent-based automation.

3. Supporting Effective Management of Multi-agent Automation with Ecological Interface Design (EID) Effective management of agent-based automation requires a clear understanding of its behaviour and a well-calibrated level of trust in its capabilities. Excessive trust may lead to misuse and inadequate trust may lead to inappropriate interventions. It may be possible to maintain a well-calibrated level of trust if the human-computer interface reveals the structure and constraints of the system and the agent-based automation. Ecological Interface Design (EID) provides a theoretical basis for interface design that may encourage highly calibrated levels of trust.

3.1. Componentsof EID The essence of the EID philosophy can be summarized by two objectives. First, EID defines the information content required to control the system under normal and abnormal situations. Second, EID defines the form of the information (e.g., digital readouts, bar charts, or integrated objects into which information is embedded) so that it is consistent with the information processing capabilities of the operator. Thus, EID provides a means of identifying the critical information requirements of a good interface and guides the designer in selecting appropriate visual forms. Although EID has not been applied to the displaying of automationrelated information, it is a promising and tractable extension.

146

Ell) and the Management of Large Numbers of Intelligent Agents SystemCharacleristics

Ecological Interface Design

Abslraetion 1 Hierarchy Interface Content Ecological Interface Interface Form 'Skill,rule, and I knowledge-basndI behavior I

J

OperatorCharacterislies

Figure 3: The process of ecological interface design. Figure 3 shows how EID is used to identify the interface content and form and how they combine into the interface design. An important aspect of EID is the cognitive task analysis approach that identifies the physical and functional properties of a system that must be displayed to the operator. These properties are defined by the abstraction hierarchy, which is a description of system constraints at multiple levels of abstraction and detail. Because the abstraction hierarchy describes the system in terms that are consistent with humans natural parsing or chunking of a domain, it offers a powerful technique to describe the information required for system control. Identifying these properties is important because they specify the information required for routine operations as well as unanticipated situations. This type of analysis is particularly important for managing a self-organizing system, where particular behaviour may be impossible to predict, but functional relationships may be possible to specify and display. As a consequence, EID provides a very useful theoretical framework for examining agent-based automation because it defines important functional relationships. Because EID identifies the information requirements of an interface through an analysis of the system goals and the operating environment it has particular promise in helping people control multiagent systems that dynamically adapt to the environment. Even if relevant system constraints are correctly identified, a challenge remains in displaying the large volume of complex information to the operator and to providing intuitive mechanisms for the operator to monitor and command the agents. The skill-, rule-, and knowledge-based distinctions are fundamental characteristics of human performance and they help specify the form of the interface. Skill-based performance governs processes such as patter matching and movement co-ordination, while rule-based performance governs the application of rules-of-thumb and procedures. Knowledge-based performance governs problem solving and generating response to novel situations. Unforeseen situations require the flexibility of knowledge-based behaviour, but knowledge-based behaviour is more demanding than rule or skill-based behaviour. Therefore, the form of the

EID and the Management of Large Numbers of lntelligentAgents

147

interface should support all three levels of cognitive control, without pushing cognitive control to a higher level than is required [29]. This requirement guides interface designers to represent data graphically, making important functional relationships directly visible in the interface. 3.2. Benefits of EID

Several experimental investigations have shown the practical benefits of EID. These investigations demonstrate that ecological interfaces support superior decision making and fault diagnosis, when compared to traditional approaches to interface design [7], [31], [14]. In addition to better diagnosis, experiments have shown that ecological interfaces lead to a better understanding of the system and more efficient control strategies. For example, an ecological interface for monitoring heart activity led to more effective treatment with fewer drug administrations [9]. EID provides a proven foundation that can be built upon to address the challenges of describing the cognitive tasks and demands associated with supervisory control of agents. Another benefit of EID may be that the appropriate use of software agents improves system safety and reliability by providing highly redundant adaptive systems. However, without proper consideration of the human controller, agent-based automation may generate behaviour that is extremely difficult for the human to understand and manage effectively. Various design alternatives may need to be considered to optimize diagnosis performance for a suite of potential, difficult to control emergent errors. Figure 4 contrasts a traditional interface with one based on the principles of ecological interface design. The dramatic differences show the power of EID to generate novel interface concepts. The ecological interface provides operators with a performance advantage by integrating the otherwise disparate data into a meaningful object display. By overlaying that object on the saturation curve for water the individual data points are given meaning in terms of safety critical system states [2], [30]. Cognitive task analysis and display design components of EID will identify an effective human-centered approach to supervisory control of agentbased automation.

148

Ell) and the Management of Large Numbers of Intelligent Agents

/

err ~

PI-I~II W

l

/

I

I

I

/

o,o

1oi

................. _'?_'?'~_~~ '_'~..................................................................................

+.-~._.

"1I

Figure 4: A standard interface and one based on the principles of ecological interface design.

4. Conclusion Agent-based automation is an emerging technology that epitomizes the increasingly complex automation that faces human supervisors. A review of current problems

EID and the Management of Large Numbers of lntelligent Agents

149

with automation suggests that the factors that contribute to mode errors with current automation include indirect mode changes, inadequate feedback, and inconsistent behaviour. An analysis of likely emergent behaviour of agent-based automation, suggests that it may exacerbate all of these factors. Mode errors, misuse, and disuse of automation could dramatically increase with poorly designed agent-based automation. This chapter takes a problem-driven, human-centered perspective in exploring how best to capitalize on the burgeoning field of agent-based automation. To capitalize on the capabilities of agent-based automation, calibration of trust emerged as a critical consideration. One particularly promising approach to enhance the calibration of trust and the effective management of agents is to adapt the theory of Ecological Interface Design (EID) to support effective control of a selforganizing system of agents. This would help develop effective visualizations for displaying agent behaviours, presenting the current state of the system, and the predicting future states. EID is particularly well-suited to this application because EID focuses on identifying and displaying system constraints. Because the behaviour of agents emerges from their interaction with the system EID will represent agent behaviour in a way that makes the emergent behaviour understandable. Using EID as an interface to manage agent-based automation is an important extension to EID, which has produced a series of elegant interface solutions for complex systems. The EID paradigm is particularly well poised to support control of agent-based automation because it emphasizes holistic understanding and pattern recognition that has been successful in supporting the management of other complex systems. Addressing the details of how to adapt the EID principles to the management of an agent-based, self-organizing system, however, remains an important challenge.

References [ 1] [2] [3] [4] [5] [6] [7]

Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775-779. Beltracchi, E. L. (1987). A direct manipulation interface for heat engines based upon the Rankine cycle. 1EEE Transactions on Systems, Man, and Cybernetics, 17(3), 478487. Beni, G., & Wang, J. (1993). Swarm Intelligence in Cellular Robotic Systems, Robots and Biological Systems: Towards a New Bionics. Berlin: Springer-Verlag. Bonabeau, E., Theraulaz, G., Deneubourg, J. L., Aron, S., & Camazine, S. (1997). Selforganization in social insects. Trends in Ecology & Evo|ution, 12(5), 188-193. Brooks, R. A., & Flynn, A. M. (1993). A Robot Being, Robots and Biological Systems: Towards a New Bionics. Berlin: Springer-Verlag. Brooks, R. A., Maes, P., Mataric, M. J., & More, G. (1990). Lunar base construction robots. Proceedings of the 1990 International Workshop on Intelligent Robots and Systems, 389-392. Christoffersen, K., Hunter, C. N., & Vicente, K. J. (1998). A longitudinal study of the effects of ecological interface design on deep knowledge. International Journal of Human-Computer Studies, 48(6), 729-762.

150

[8]

[9] [10]

[11] [12]

[13]

[14]

[15] [16] [17]

[18]

[19] [201

[21] [22]

[23]

[24] [25]


Degani, A., & Kirlik, A. (1995). Modes in human-automation interaction: Initial observations about a modeling approach. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Vancouver, British Columbia, Canada. Effken, J. A., Kim, N. G., & Shaw, R. E. (1997). Making the constraints visible: Testing the ecological approach to interface design. Ergonomics, 40( 1), 1-27. Fukuda, T., Funato, D., Sekiyama, K., & Ari, F. (1998). Evaluation on flexibility of swarm intelligent system,. Proceedings of the 1998 IEEE International Conference on Robotics and Automation. Johnson, P. J., & Bay, J. S. (1995) Distributed control of simulated autonomous mobile robot collectives in payload transportation. Autonomous Robots, 2(1 ), 43-63. Kantowitz, B. H., Hanowski, R. J., & Kantowitz, S. C. (1997). Driver reliability requirements for traffic advisory information. In Y. I. Noy (Ed.), Ergonomics and Safety of Intelligent Driver Interfaces (pp. 1-22). Mahwah, NJ: Lawrence Erlbaum Associates. Lee, J. D. (1995). Ecological interface design: Applications in the maritime industry. In W. Wittig (Ed.), The influence of the man-machine interface on safety of navigation (pp. 89-95). Bremen, Germany: Verlag TUV Rheinland. Lee, J. D., Kinghorn, R. A., & Sanquist, T. F. (1995). Review of Ecological Interface Design Research: Applications of the design philosophy and results of empirical evaluations. Seattle, WA: Battelle Human Factors Transportation Center. Lee, J. D., & Moray, N. (1992b). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243-1270. Lee, J. D., & Moray, N. (1994). Trust, self-confidence, and operators' adaptation to automation. Int. J. Human-Computer Studies, 40, 153-184. Lee, J. D., & Sanquist, T. F. (1993). A systematic evaluation of technological innovation: A case study of ship navigation. IEEE International Conference on Systems, Man, and Cybernetics, 102-108. Lee, J. D., & Sanquist, T. F. (1996). Maritime automation. In R. Parasuraman & M. Mouloua (Eds.), Automation and Human Performance (pp. 365-384). Mahwah, NJ: Erlbaum. Malin, J. T., Schreckenghost, D. L., & Rhoads, R. W. (1993). Making intelligent systems teanz players: Additional case studies (104786): NASA Johnson Space Center. Min, T. W., & Yin, H. K. (1998). A decentralized approach for cooperative sweeping by multiple mobile robots. Proceedings of the 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Parasuraman, R., & Riley, V. (1997). Humans and Automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230-253. Rasmussen, J., & Vicente, K. J. (1989). Coping with htunan errors through system design: Implications for ecological interface design. International Journal of ManMachine Studies, 31,517-534. Sarter, N. B., & Woods, D. D. (1992). Pilot interaction with cockpit automation I: Operational experiences with the flight management system. International Journal of Aviation Psychology, 2(4), 303-321. Saner, N. B., & Woods, D. D. (1995). How in the world did we ever get in that mode? Mode error and awareness in supervisory control. Human Factors, 37( 1), 5-19. Seeley, T. D. (1997). Honey bee colonies are group-level adaptive units. American Naturalist, 150, $22-$41.

EID and the Management of Large Numbers of lntelligent Agents

151

[26] Sheridan, T. B. (1975). Considerations in modeling the htanan supervisory controller. Paper presented at the Proceedings of the IFAC 6th World Congress, Boston, Massachusetts. [27] Stickland, T. R., Britton, N. F., & Franks, N. R. (1995). Complex Trails and Simple Algorithms in Ant Foraging. Proceedings of the Royal Society of London Series BBiological Sciences, 260(1357), 53-58. [28] Sugihara, K., & Suzuki, I. (1990). Distributed motion coordination of multiple mobile robots. 5th IEEE International Symposium on Intelligent Control, 138-143. [29] Vicente, K. J., & Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEE Transactions on Systems, Man, and Cybernetics, SCM-22(4), 589606. [30] Vicente, K. J., Moray, N., Lee, J. D., Rasmussen, J. D., Jones, B. G., Brock, R., & Djemil, T. (1996). Evaluation of a rankine cycle display for nuclear power plant monitoring and diagnosis. Human Factors, 38(3), 506-521. [31] Vicente, K. J., Christofferson, K., & Pereklita, A. (1995). Supporting operator problem solving through ecological interface design. IEEE Transactions on Systems, Man, and Cybernetics, SMC-25(4), 529-545. [32] Wickens, C. D. (1998, March). Automation in air traffic control: The human performance issues. Third Human Factors in Automation Conference, Norfolk, VA. [33] Woods, D. D. (1991). Nosocomial automation: Technology-induced complexity and human performance. Proceedings of the International Conference on Systems, Man, and Cybernetics, 1279-1282. [34] Woods, D. D. (1991). Nosocomial automation: Technology-induced complexity and human performance. Proceedings of the International Conference on Systems, Man, and Cybernetics, 1279-1282.

Operator Support in Technical Systems

Heiner Bubb Chair of Ergonomics, Technical University Munich, Boltzmannstr. 15, D-85747 Garching e-mall: [email protected]

Abstract: The idea, the complexity, and the acceptance of operator support in technical systems is explained by the example of driving a motor car. The subjective judgement of future driving aids were investigated in a driving simulator experiment. The new systems are the Heading Control (HC) device, which enables the car to follow the road boundary automatically, and the Adaptive Cruise Control (ACC), which measures the distance to the car ahead and keeps the correct safety distance automatically. In the experimental case of the HC system, a feed back was given to the driver by an artificial reverse moment in the steering wheel. Additionally, the human behaviour was investigated by using a conventional control element in connection with an Active Control Element (ACE), a joystick like control device, which feeds back the dynamic system states of speed and yaw angle velocity by changing the position of the stick. In connection with this ACE, the same feed back can be presented to the driver as in the case of the conventional control elements. The results of the experiment show the superiority of the ACE, although the implementations of the new aids HC and ACC are at present not very well accepted.

1. T e c h n i c a l S o l u t i o n When developing any technical system, the usual procedure is to start with a technical approach and to make improvements step by step according to the experience gained by using this new system. Therefore, planning the use or invention of a machine normally starts with the technical conditions and not with the human preferences. As an example, the familiar processes of driving a motor car may be used. Presumably nobody ever asked how the driver would wish to steer the car. Since a car with four wheels can be moved on a curved path when the axes o f the front wheels have an angle to the ones of the rear wheels, the steering wheel was introduced purely by technical reasons to adjust this angle. The longitudinal dynamic of a car was developed in a similar way. Although we have only two feet, we have to operate three different pedals in a car in this case: the pedals for the accelerator, the brake and the clutch. In addition the gear lever has to be operated by hand. All these control elements must be used and coordinated, just to adjust the one longitudinal movement! What is valid for driving a motor car, is as well valid for the manipulation of other machines or systems.

154


As an example, just think of the well known fact that up to 80 % of the population is quite incapable to program a video-recorder correctly. As experience shows, an operator does not know how to operate a technical system. The usual consequence is to recommend adequate training and education. Also the observation of accidents leads normally to better safety education of the operators. However, like the experience in public traffic indicates, education campaigns only have a short-term effect and must often be repeated when external conditions change. According to statistics, the most frequent accident categories in all road categories (motorway, country road and city traffic) are deviations from the road to the right or to the left, parallel or opposite collision with vehicles, and front-end collision. Therefore, in opposite to a first impression, it seems to be difficult to control the transversal and the longitudinal dynamic of motorcars. In order to improve the performance of these controlling tasks, several technical assistance systems were

Figure 1: Principal information flow of the electronic stability control (ESC) system in motor cars. developed as for instance ABS (Antiblocking Braking System) and ESC (Electronic Stability Control) which compensate for mistakes of the driver. As Fig. 1 outlines by the example of ESC, the general idea of this kind of system is that the human operator (here the driver) gives an order to the control elements in the same way as it is done with a conventional non aided system (here steering wheel). The actual position of this control element is used to calculate the nominal behaviour of the system (in this example "dynamic calculator II"). By measuring the influencing physical parameters, the actual behaviour is calculated in a parallel branch of the dynamic computer (here "dynamic calculator I"). In the case of an observed


155

difference between actual and nominal behaviour in a third branch (here "ESC control calculator"), an adequate intervention is calculated. A further technical system, currently on the way to be commercialized, is the Adaptive Cruise Control (ACC). It can be seen as a progressive development of the cruise control system used up to now maintaining a pre-selected speed automatically. The advantage of ACC is the additional radar detection of obstructing objects ahead. The information is used to automatically decrease the speed of the car to keep an adequate distance to the car ahead. Another system still in the phase of research is the Heading Control System (developed within the scope of the PROMETHEUS research project). It is able to define road limitations using a camera. An automatic system keeps the car on the defined trail. In order to keep the driver within the control loop of this system, the actual course given by the driver is compared with the nominal course calculated by the automatic system similar to the one of the ESC-system. A corrective moment is given to the steering wheel in case of deviation. If we summarize all these developments, we can see a general tendency towards automation of the car driving process, which can exclude the driver from the closed loop driver-vehicle system in the end. As a consequence to this technical development, the following questions arise: How is the human operator involved in this new system? What are the tasks left to him now? What are his feelings about and reactions to such a system that imposes inactivity?

2. Ergonomic Analysis and Recommendations The human operator is very adaptive, that means: if he experiences a process to be sufficiently reliable, he does think that the system will ever have a failure or breakdown, and he assumes that the system will work properly at any time. Therefore, one of the most important questions in connection with automation is: What happens if the system is out of order? To answer this question, a system ergonomic analysis [2] of the driving process is to be carried out to comprehend the driver's mental workload on the one hand and to get an idea of the driver's understanding of his role in this process on the other hand. To allow a prediction of the driver's behaviour in the case of breakdown of the system, the system ergonomic approach is used. It starts with a general description of every task and draws ergonomic recommendations for the partial aspects of the task based on experimental experience. It does not create any images about the internal model of the user. The fundamental idea is that the tasks to be performed by the operator may be designed by considering the knowledge about the information transfer between the subsystems man and machine. System ergonomics is concerns three main areas:

156


Task contents - f e e d back - compatibility." Task contents describes how far the system corresponds to the internal models of the operator, this also means: How big is the effort to learn to operate the system? 9

Feed back answers the question to what extend the operator is able to receive information about his position in the actual state of the system.

9

Compatibility describes the effort of information processing between different information channels that connect the operator to the outside world.

For each of these areas, we know about the specific human workload given by deviations from the demanded lay-out and about the ergonomic rules that can achieve an improvement. For each of these areas, we know about the specific human workload given by deviations from the demanded lay-out and about the ergonomic rules that can achieve an improvement. In order to apply system ergonomics to the driving process, it is useful to consider the hierarchical interconnected levels of driving: navigation, guidance and stabilization [3] (see Fig. 2). These correspond to general hierarchical encapsulated tasks: "planning", "programming and action", and "observing and controlling" [4]. nominal: ~lace me

actual:

. . . ~ , ~. _ ~ s # ' /

.

.

.

lace, ' e

N=~ ':..

(l~ma~)

...

ergonomlr solution proposals: 9 navigationcomputer 9 Detection and indication of and indicationof technical realisable objects ~npost and traffic participants 9 indication or keeping of a - safety distance (e.g. ACC), - overtaking distance, - max. transverse acceleration (e.g. Heading

Contro/)

9 indication of velocity 9 help in unstable driving manoeuvres as for instance ABS and ESP

(Active Control Element)

* ACE

Figure 2: The three level of the driving task [3] and ergonomic aids for the human operator. By the navigation task, the exact route between the starting point and the destination is selected. This can be performed with the help of a brain-stored map or with the help of a hardware map. In any case it is a thought process that needs. The


157

result of the navigation task is the input for the guidance task. This task is accomplished immediately during the driving process. It consists of the finding of the safe way on the road through the standing and moving traffic participants. For finding this way, geometrical as well as time aspects (i.e. the correct selection of speed, the correct judgement of the moving behaviour of the other traffic participants) must be considered. This task must be accomplished within a very short time frame. It can therefore be called "dynamic task". The outcome of this task is the input for the lowest level, the stabilization task, which often is seen as the real driving task, which determines the manipulation of the control elements (steering wheel, accelerator, brake-pedal, gear shift and clutch pedal) and the forcing function defined by the guidance task into reality. This task is also a dynamic task. The final result must fit to the corresponding tasks on all levels. If a deviation is found anywhere, normally a corrective action must be performed on the next higher level (e.g. if the car is not able to be driven through a desired curve on the level of stabilization, a new way and a new nominal speed must be created on the level of guidance considering the new conditions; if a wrong and not desired way is chosen on the level of guidance, a new corrective way must be found on the level of navigation). If we look at the technical supports being actually in development, we observe that they are especially related to the internal shell "stabilization task" (e.g. ABS and ESC) and to the external shell "navigation task" (in form of the presently offered navigation systems for motorcars) but not for the "guidance task". However, a scientific project currently in progress [5] shows that deficits on this level are the main reason for accidents.

3. Ergonomic Solution Proposals The combination of the Heading Control Device and the ACC-system could be a first approach to support the driver in the guidance task, because this system allows to receive the available area ahead of the driver (HC) and to project a trajectory of the actual path of the car into this area under consideration of obstructing moving objects (principal abilities of the ACC-system). However, this will happen in the far future only. Presently, neither the HC-system nor the ACC-system offers a sufficient technical reliability. Consequently, a speedy introduction of such a system cannot be expected. Apart from the solution of technical problems, the methods of informing the driver as well as means of displaying the system's feedback are objects of major research efforts. This can be done by using a driving simulator that allows to investigate and adjust the technical reliability of new systems and also allows scientific investigations of the related human behaviour. This would already in an early stage allow to foresee future developments. Presently, we are investigating the effect of two concurring systems:

158


a) Use of conventional control elements: an artificially created reverse moment is added to the steering wheel that informs the driver about the technically determined path. The information effect of this reverse moment is investigated in the form of "narrow guidance" (i.e. every deviation is immediately indicated) and "boundary guidance" (i.e. a reverse moment is applied only, if the driver leaves the safety zone), see Fig. 3.

""::'t"

"

"

=

rl~

~

,~{

reset moment'

reset moment . ..!.'."

AA

left

right

Deviation from nominal course 9 ...... "'= ""~ :'"" ;":": " ] "llll "" ~"

" Ill~fl :'9

j;i

AE

fight iii

left

Deviation from nominal course :... I IlI

Ill'

I

L',.

~'PY "l'"

Figure 3: The two thinkable reset moments in the case of a Heading Car device. In the case of conventional control elements, the information of the longitudinal control intervention is can only be obtained by increasing or reducing the engine power and activating the braking system. In addition, different forms of acoustical or optical indicators are investigated.

b) Use of the so-called Active Control Element (ACE) [6]: the ACE is a specifically-developed control element similar to a joystick that measures the force applied by hand and uses this information for the steering function. The effect of the machine (in case of the motor car it is the actual speed and the actual yaw-speed) is fed back to servo motors of the control elements. By this, the position of the joystick represents always the state of the machine dynamics. The driver holds, so to speak, the machine in his hand (see Fig. 4 and 5).


Conventional control element task ~ ,.t.o~e

,--.~[~--~~

159

9 result~ machine ~--~ =

Active Control Element(ACE)

.

9: l u . . .

i.

I ,~e conlrolelement,

.!

i

Figure 4: The information flow of the conventional control element in relation to the Active Control Element (ACE).

itudinal velocity

angle velocity

Figure 5: The application of the ACE in motor cars. The ACE is an ergonomic aid on the level of the stabilization task. The effect of it is shown in Fig. 6. The figure illustrates that the closed loop driver-vehicle system shows a outstanding resonance peek at about 2,5 rad/s ( = 0,4 Hz, dotted line) in the case of the conventional steering wheel. In practice, this means that the drivervehicle system shows an instability when the driver makes too fast reactions. The cars seem to go into a skid. Only when the hands are kept off the steering wheel, the car would stabilize itself. The non interrupted line shows the behaviour of the closed loop driver-vehicle system in the case of the ACE. The resonance peek now nearly disappears totally [1].

160


ZO[.

-

'

m' ~ 00~ m

"

H~I

t,

~

f steedngwheelwithaspeed

I0

..=_

E

activecontrolelementwitha

-1

'~ = -300'

yaw angle dependent position feed back

02

~ 02

0A

lZrll

~

3

]i. . . illlii . . . . . . . . .i'N.I 0,4

frequency

1

3

[rad/s]

Figure 6: Effect of the active control element on the frequency response of the closed loop of the driver-car-system In the same way as it can be used in connection with conventional control elements, this new aid can be combined with the two different types of artificial moments described above, which feed back the information of the technically determined path. The advantage of the ACE in this application is, that the information of the boundaries in transversal and longitudinal direction is transferred to the human operator always in the same way.

4. Investigation of Acceptance Preliminary investigations in a driving simulator where carried out. 5 1 candidates had to drive a round course. During the driving experiments, they additionally had to react on suddenly appearing unexpected tasks like "getting out of the way of a broken down car", "breakdown of HC-System", "coming out of tunnel", "slow car ahead without overtaking opportunity", and "HC triggers a wrong path". They had to participate in two main experiments: one with conventional control elements (steering wheel, accelerator and brake pedal; the simulator was programmed as a car with automatic gear shift) and one with the ACE. In every case, the behaviour "without assistance system", with "narrow guidance" and with "boundary guidance" were investigated. First results are collected in Tab. 1 on the basis of a questionnaire about the experiments. The main questions asked for 9 9

the necessary concentration, the concentration in the case of system brake down,


9 9 9

161

the judgement of the help that is offered by the new system, the self judgement of the ability for correct reaction, and the individual priority, whether the subjects would buy such a new system. ,, S t e e r i n g

nmry cormer~i~allon in 9t ~ , i ~ e o f

Wh'~.

narrow

guidance

without

assistance

sys-

tembreaktJown

help

,, ,~ ,~ , ,.~.~. ,..

priority

.....':~: ......

without assistance

" "

9: Active Control Elemeiat narrow guidance without assistance, but if assistance then narrow guidance without assistance, but if assistance then narrow guidance

without assistance

without assistance, but if assistance then narrow guidance

without assistance

without assistance, but if assistance then narrow guidance

Table 1: Preferences of the subjects after their experience with the four experimental situations. The table shows, that for both main layouts of the feed back system the "narrow guidance" is preferred. The operator seems to prefer the feeling, that the automatism works. In case of faulty operation or breakdown of the system the unaided hand operation is preferred for both layouts. When a faulty operation or breakdown of the system is taking place, an advantage of "narrow guidance" compared to "narrow guidance" is observed under the condition of ACE . This advantage was never observed with conventional control elements. That means: if we want to benefit from safety from an automatic system, conventional control elements (and indicators) are not appropriate. We have to use new interfaces between man and machine, which specifically take into account ergonomic demands.

References [1] [2] [3]

Bolte, U. (1991): Das aktive Stellteil - ein ergonomisches Bedienkonzept. FortschrittsBerichte VDI, Reihe 17 "Biotechnik", VDI-Verlag, DUsseldorf. Bubb, H. (1988). System Ergonomics as an Approach to Improve ttuman Reliability. Nuclear Engineering and Design 110, S. 233 - 245. Bemotat, R.(1970): Plenary Session: Operation Functions in Vehicle Control, Anthropotechnik in der Fahrzeugf'tihrung, Ergonomics, Vol. 13.

162

[4] [5] [6]


Sheridan, TB. (1976): Toward a General Model of Supervisory Control. In: Sheridan, T.B., Johannsen, G. (Hrsg.), Monitoring Behaviour and Supervisory Control. New York, London. Reichart, G, (1999): Menschliche Zuverli~ssigkeit im Straflenverkehr; Dissertation an der Technischen Technischen Universitat M0nchen. Bubb, H. (1985): Arbeitsplatz Fahrer - eine ergonomische Studie , Zeitschrift: Automobilindustrie 30, S. 265 - 275.

Interfaces for Every Day Things

Kerstin ROse Institute for Production Automation, University Kaiserslautem, P.O. Box 3049, 67653 Kaiserslautern, Germany e-mail: [email protected] Abstract: Interfaces for Every Day Things - Specialist required? What is the meaning of "Everyday Things" for Industrial Design in general, what for the design of Human-Machine-lnterfaces with high complexity? Based on these questions, this paper aims to show, how important the sensitivity for the design of Everyday Things is and to give an understanding for the correlation between design of Everyday Things and that of Human-Machine-Interfaces. Human operations can generally be seen as being aimed at achieving certain tasks. They can be divided into three main aspects, i.e. type, principles or patterns, and context of operation. The context may vary for different situations, but patterns and types stay principally the same. According to Rasmussen [4], one can distinguish three types of operation: knowledge-based, rule-based and skill-based. Knowledge- and rule-based operations are performed on a conscious level, while skill- based operations are those which are often recurring and thus handled in a increasingly automatic manner, like stimulus-response-operation. The underlying principles, or patterns, of operations can be explained with the Theory of Mental Models 1 [5]: Correlations between input and effect of past operations are stored in the user's memory, together with their basic assumptions. If these assumptions are encountered again, the corresponding mental model is retrieved and used (rule-based action).If it is not possible to transfer the existing mental models directly, the operation cycle is modified. If the desired goal is still not reached, the assumptions are verified. They remain unchanged if they are rated as correct; instead, a new model for the procedure is compiled (knowledge-based action). For more complex tasks, the new model can be a combination of existing 'simple' models. The familiarity with the new mental model increases with the frequency of its use, turning it into a rule-based operation.

1 Mental models represent explanations for proceedings in the real world.

164

Interfacesfor Every Day Things

Research has shown [2,3] that humans possess ,,natural standards", so-called compatibility patterns, which are based on a defined anticipation of a system's reaction to a certain operation, e.g. turn to the right = value increase, a to an inherent system change, e.g. value increase (temperature rise) leads to indicator increase (pointer deflection to the right). Compatibility patterns can be seen as a basic form of mental models. Thus, with the collection of mental models a human being is building a data base, storing effect correlations i.e. operations and related reactions. In this way, the human being is able to retrieve a fitting or modifiable mental model for a number of standard situations. These standard situations are, due to their frequent occurrences, usually encountered in the context of Everyday Things, such as household appliances or cars.

Level of vigilance

Level of automalion

Y

Curve of vigilance in a stressfull situation/by lime

t

I1

Curve of automation operation / by cases of use

Figure 1: Levels of Vigilance and Automation. For operating technical devices, the user employs his mental models and makes assumptions about the necessary input (or action) as well as the subsequent reaction of the device. The user interprets continued confirmation of his assumptions as verification of his mental models. It can be assumed that by use of Everyday Things effect correlations are recognised and tested on everyday technical devices (e.g. HiFi). Consequently, mental models for the functionality of technical devices are established with the use of Everyday Things. It can be assumed that basic knowledge for handling technical devices can be acquired through the use of everyday things. The statement is made that for the use

Interfaces for Every Day Things

165

of Everyday Things and the use of human-machine-interfaces the same mental models are taken as a basis for corresponding operations. Mental models and resulting patterns of operation associated with dealings of Everyday Things are more present in the memory of the user due to the high number of applications. This is an aspect which is especially of importance in routine situations and time critical situations. Time critical situations are encountered in the area of production and process equipment, especially in abnormal conditions. An abnormal condition creates an unforeseen situation in the course of the operation and therefore a stress situation for the user. In stress situations the human operation is mostly based on a simple operation level (often skill-based). In this case the user goes back to application-safe and quickly available operation patterns, i.e. simple mental models, such as those acquired from Everyday Things. If one succeeded to convert these simple mental models into the area of material processing and consider the compatibility pattern, then the user would react more adequately in emergency situations. In this way, human error could be avoided by preventive proceedings in development and design of complex technical systems, together with a general simplification of operation by applying familiar mental models. Therefore, it is apparent that more importance has to be given to the design of Everyday Things, so that the user's operation patterns are already established at an early stage by the use of Everyday Things. They will then form a better basis for operation patterns relevant in complex technical demand situations, especially emergency situations.

References [ 1] [2] [3] [4] [5]

Reason J., 1990, Human Error. Cambridge University Press. Norman D.A., 1988, The Psychology of Everyday Things. Basic Books. Murrell K.F.H., 1971, Ergonomie. l.Auflage, Econ Verlag GmbH, Dilsseldorf, S. 289-310. Rasmussen J., 1983, Skills, Rules, Signals, Signs and Symbols and Other Distinctions in Human Performance. IEEE-Transaction, Vol. SMC-13 Anderson J.R., 1996, Kognitive Psychologie. 2.Auflage, Spektrum Akademischer Verlag, Heidelberg.

Operator Process Interfaces - A Retrospective View of the '90s

Klaus Zinser ABB Utility Automation, Mannheim, Germany e-mail: [email protected]

Abstract: Visualisations for complex dynamic processes represent one of the most demanding fields in the area of developing better operator-process interfaces. This paper describes the comprehensive work done by guiding and integrating advanced university research in the application domain of power plant supervisory control. The results are put into perspective of a decade of many activities in this field.

1. Introduction The last decade was characterised by tremendous advances in the Computer Technology, mostly the CPU and graphics hardware performance. This mostly became apparent in the computer games industry - more power and finesse on ever smaller footprints - an optimized packaged solution for the big business. On the SW side, of course, advances were not that dramatic: X-windows and Motif (the first powerful graphical User Interfaces for Unix, and hence process control systems utilizing this operating system) dominated the first half of the decade, Microsoft Windows the second half (Windows NT being the control system operating system of choice then). Another trend, Artificial Intelligence development environments showed as one its most noticeable effects on computer industry the advancements of object-oriented technology.

2. A p p r o a c h a n d Results 2.1 Visualisations for Process Supervisory Control In ABB, most noticeably in its Corporate Research organisation - together with its Power Plant Control and Network Control divisions, throughout the 90's a

168


continuous effort was made to advance operator interfaces and particularly supervisory control visualisations for the large and dynamic masses of process data. These visualisations encompass: 9 Picture Hierarchy, Picture Pyramid: efforts to arrange a large number of individual process displays in only a few levels of abstraction (from overview to loop) that allow for information panning within a level and so-called information zoom (including dynamic information decluttering) between levels. 9 Context Specific Visualisations: adjustment of the contents of either a process display, or the particular part of a whole Picture Pyramid level according and optimized to the status of the process itself, by support of an expert system for process state assessment and graphical picture composition. 9 Overview Visualisations: one of most demanding challenges is still to visualize the large amount of dynamically changing process data in a way both meaningful to the operator and processable by his cognitive capabilities. The work on Mass Data Displays still represents the most successful and unique work here. 9 Fisheye Views: this name derived from optics stands for the simultaneous, manipulation of overview and detail information presentations, interactively by the operator and with immediate, dynamic graphical feedback. 9 3D process visualisation: most common visualisations today are coherent with the 2-dimensional computer screen (so called 3dimensional displays today are really 2Y2 dimensional, as they only contain 3D effects (shading) of plant components). The 3D process visualisations depicted here were developed to make meaningful use of the 3~ dimension (rotation, distortion) - and at the same time incorporate interactive, dynamic properties of Fisheye Views. All this work initially was done independent of each other. Each visualisation was developed to a level that is difficult to represent in a printed article, and deployed in a simulated process control environment. Eventually, lastly, a taxonomy was developed, on how to bring it all together, including the crucial aspect, on how to navigate between visualisation displays which are individually optimal for a given aspect: the visual momentum. 2.2. A Taxonomy for Process Visualisation This taxonomy describes how process displays should be placed in the AbstractionAggregation Hierarchy along the diagonal that optimally supports problem solving behaviour.


169

AbstractiOn

GoaP

Purpose

Function

physical Appe~.mr

Plant

Syslmm/Subsystem

Component

Degree of DeM#

Figure 1: A Taxonomy for Process Visualisation. The following figure depicts this taxonomy as applied to Power Plant process visualisations (we developed and implemented also displays for Network Control domain).

Goab' Purpose

Function

Physical Appearance

:===:=,. ~ .

Plant

System~ubsystem

Figure 2: The Taxonomy as realised for Power Plant Domain.

t,l~.-~lr

Component

!

.

Ib

Deta,I

170

Operator Process Interfaces- A Retrospective View of the '90s

2.3. Results A series of experiments provided insights how the taxonomy applies in praxis. The major issue of course, the navigation between individual displays of the abstractionaggregation hierarchy, depends on the actual chosen displays. For navigation both aspects of spatial and temporal proximity were investigated, and the convincing results are shown in the following figure.

Goal,

Purpose Function

Physical Appearance SubSystems System Component Figure 3: Navigation in Visualisation Taxonomy -Proof of Concept.

3. D i s c u s s i o n So, given all this work on individual new forms of process displays, integrating them, and performing experimental evaluations, and given the interesting results and findings, the one may wonder why it is not all available in products. As a short summary, there are a variety of reasons, ranging from market aspects such as less steam power plants (primary users of the described approaches) and almost full stop of nuclear plants (traditionally driver of technology), to more popular IPP power plant operators (typically combined cycle power plants that have less supervisory control complexity) - all in the overall scenario of cost pressure due to deregulation, and ranging all the way to soft issues such as conservatism and cultural/education background of plant operators. Then, will we ever get it? - and if yes: when? Given the ubiquitous UI technology, (Microsoft windows UIs, Internet/WWW, palm computers, mobile communication), common DCS technology based on Windows NT that helps to easier implement and leverage investment, and given the next generation operators with more computer literacy, chances are good!


171

References [1]

[2] [3] [4]

[5]

Elzer, P. F., Weisang, C.; Zinser, K., 1990, Knowledege-based System Support for Operator Tasks in S&C Environments, IEEE Conference on Systems, Man and Cybernetics. Zinser, K., 1993, Neue Formen und Medien der Prozeflvisualiserung, atp 9/93, S.499504. Zinser, K, 1995, Fisheye Views - lnteraktive, dynamische Visualisierungen, atp 9/95, S.42-50. Beuthel, C., Elzer, P. F., 1997, Advantages of 3D Displays for S&C of Large Technical Systems, Conference on Human Decision Making and Manual Control, Kassel, Germany. Bums, C; VicenteK., 1996, Comparing Functional Information Content of Displays, Proceedings 28 ~ Annual Conference of Human Factors Association of Canada HFCA, pp. 59-64.

Acknowledgements:The work done in ABB was performed on different aspects of the overall scope with the following organisations who the Author wants to credit here: University of Clausthal, Prof.Dr.Elzer, Dr.Boussoffara, Dr.Beuthel University of Toronto, Prof.Dr.Vicente, Dr.Bums, and last not least the ABB units that supported work with funding and enthusiam over the years.

Acceptance of new Technology and Layout in Control Rooms

J.W. de Vries Dept. of Nuclear Technology, N.V. EPZ NPP Borssele, Belgium e-mail: nucvries @ zeelandnet.nl

Abstract: EPZ, the Electricity Generating Company Ltd. for the Southern Netherlands, owns and operates the Borssele Nuclear Power Plant which is located near Flushing on the Schelde estuary. In 1997 an extensive backfitting programme was carried out at this twin-loop 480 MWe PWR in order to raise nuclear safety to a level comparable with that of a modern plant. One of the key issues in order to accomplish this was the definition of a 'protected zone'. This zone includes the reactor containment and the bunkered building, housing the alternative cooling system, the RPS and the back-up control room. Systems in the protected zone are able to withstand external events as earthquake, airplane crash, gas-cloud explosion and flooding. Of course internal events are taken in account too. From the back-up Control Room it is possible to bring the reactor to a safe and stable cold sub-critical state.

Another major part of the work focused on electrical systems, instrumentation and control and Human-Machine Interaction (HMI). During the project a lot of experience was gathered on (operator) acceptance of new technology and layout in control rooms. Two sub-projects were of importance here, namely the renewal of the main control room and the revamp of the process computer.

1. Revamp Main Control Room Originally there were no plans to revamp the whole of the main control room. However, during the engineering stage it became clear that it would not be possible to accommodate the extended HMI demands in the existing main control room (MCR). Gradual upgrading during the past 20 years had used up any available space on the control desk and wall-panels. A complete renewal of the MCR was called for. In order to ensure that all design-aspects would be covered a HMI-

174


Working Group was established with participants from relevant departments in the NPP's organization as well as external ergonomic advisers. Its goal was to have a consultative body where departments as operations, training, process-technology and maintenance can - from their own field of expertise- with respect to the HMI, give input and participate in the design of the systems which are modified or newly built. Modern design pays a lot of attention to those engineering design aspects which allow tolerance for delayed human interaction during the course of automatic actions of safety systems. This approach gives breathing space to the operator after initiation of the protection system and leads to a more level-headed reaction on disturbances. After analyzing the shortcomings of the old MCR by walktrough/talk-trough of problems and interviews of the operators, a complete operations shift was made available to design the lay-out of control-desks and panels for both the existing Main Control Room and a new back-up CR. This was done using mock-ups, observing a set of rigorous ergonomic limiting conditions and the operational concept of the NPP. This process took five people working for three months. The Human Machine Interaction Working Group conducted this process. After review by all end-users (shifts) a high degree of acceptance was achieved and two control rooms were ordered. One for the full-scale replica simulator - also part of the back-fitting program- and one for the real NPP. Experiences from test-runs with the first delivered simulator CR were incorporated in the MCR design for the NPP.

2. Revamp Process Computer Another back-fitting sub-project was the revamp of the process computer. The main reason to do this was the extended number of analogue and binary point ID's required by the back-fitting project. The old process computer MMI, requiring a lot of key-board input for information retrieval, was replaced by a modern graphic user-interface with X-Windows for multi-window applications. The operator workstations have a wide variety of displays with modern intuitive user interface. All process displays are based on extensive human factors research conducted in the 1980's. User interaction has been updated to use modern techniques unavailable at the time of the original studies. The old hard-wired integrated plant status overview panel (IPSO) was replaced by a Dr.Seufert computer animated large screen projection system. The new technology replaced the key-board with a trackball or mouse as pointing device and was with its improved operator interface easily


175

accepted by the control room crew after some training sessions on the simulator. Here again from the start on operational staff was involved in the selection, design and review of the new system displays.

3. Acceptance of new Technology and Layout in Control Rooms Experience gathered during the execution of the project learned that, in order to get good acceptance of new technology and layout in control rooms, one has to start very early in the project working to this goal. Make a timely start with giving project-information to all future users. Involvement in the early project days is the beginning of the acceptance process in the operators heads. Keep in mind that operators in an electrical power station are among the more conservative species in the world. Interview operating staff and other users for early experience, look for the older guys who where there during the plant's first commissioning; they know what was designed in a bad way bur where everyone is accustomed to now; e_g. alarms misplaced vs. panel-seclions, mosaic tiles with control buttons in odd places in the mimic diagram or out of operators reach. Involve future users in the design process of the layout of mimic diagrams for control desks and wall-panels and displays for the computer workstations. Give the operational staff the feeling that the new design is their product. Execute 2 or 3 design reviews by the operational staff during the project in extending order of detail. Talk review comments over with each and every guy and treat both the remarks and the commentators seriously; don't leave grudges behind that way. As an important part of the review process organize usability tests, e.g dry runs of start-up and emergency procedures both on mock-up and simulator. Involvement of users in FAT and SAT works for you in getting commitment and acceptance; people will feel responsible for the end-product that way.

4. Conclusion Involvement of end-users in the process of design, engineering and commissioning of projects with a HMI nature is crucial for the acceptance of new Technology and Layout in Control Rooms.

Advanced User Interface Design for Aircraft Cockpit Devices

JOrg Marrenbach Department of Technical Computer Science, Aachen University of Technology, Ahornstr. 55, D-52074 Aachen, Germany e-mail: marrenbach @techinfo.rwth-aachen.de

Abstract: A Flight Management System (FMS) is a complex computer system that is

used for flight planning in all modem commercial aircrafts. The FMS' humancomputer interface needs substantial improvement to make the best use of the increasing number of features. This article outlines some ideas for a new user interface to replace today's Control and Display Units (CDUs). The alphanumerical flight plan editing is replaced by a graphical user interface. A software prototype of such a CDU has been created, using Statecharts for the definition of this interface. The developed prototype was evaluated analytically as well as in experiments. Overall the pilots envisaged after the experiments that they could use such a system in the future.

I. Introduction A Flight Management System (FMS) is an important part of the automatic flight guidance system of modern aircrafts. The FMS of today has introduced operational advantages and significant cost savings through offering the possibility of an automatic, fuel-efficient flight from take-off to landing and to reduce the pilots workload. However, the FMS with its high level of automation has changed the pilot's role considerably. This has caused dominant problems with respect to human factors. Shortcomings and the most dominant problems of the present FMS are the deterioration of the crew's situational awareness, when out of the guidance loop, a poor ergonomic computer-human interface, and the missing possibility for rapid flight plan changes. Analysis of accidents and training sessions performed in flight simulators show that system behaviour is not always as expected by the crew [1,2,31.

178

Advanced User Interface Designfor Aircraft Cockpit Devices

2. The User Interface of an Advanced FMS In order to make using the FMS easier, especially when changes have to be entered quickly, a demonstrator of an Advanced Flight Management System (AFMS) is developed and evaluated by the Department of Technical Computer Science and other partners in an european research project. The most noticeable change in the user interface is the use of a graphical output device for user inputs, as depicted in figure I. Further, the system-oriented composition of functions is transferred into a much more operational structure. In general, the AFMS provides two ways of access with different functionality: a function-oriented and an object-oriented access mode.

~ IrE] I-;71

lq D IZl IE]DD UIDI-4t E]DB

~k-~--~ ,,~,i,:~;"~"~~ i '"

Figure I: User interface of the Advanced Flight Management System. For the function-oriented mode the operator has to know how the menus and functions are organised. All functions are organised in a so called "menu tree". The user has to select the right "branch" and "sub-branches" with the column and line selection keys to access the function he wants. To inform the pilot, which top level (main menu) he is currently working with, the selected menu selection key will be highlighted until the function tree has left again. The object-oriented mode is quite different and is much more closer to the way a human operator actually thinks, It is created for quick modifications or alterations in flight. This mode offers direct access to the object on that the function has to be executed by moving the cursor of the touchpad to the object. The interaction between the AFMS and the crew is based on the concept of direct manipulation. Due to this, the object-oriented design

180


References [1] [2]

[3]

Domheim M., 1996, Recovered FMC Memory Puts New Spin on Cali Accident. Aviation Week & Space Technology, 36, pp. 58-61. Salter N.B. and D.D. Woods, 1992, Pilot Interaction with Cockpit Automation: Operational Experiences with the Flight Management System. The International Journal of Aviation Psychology, 2, pp. 303-321. Sarter N.B. and D.D. Woods, 1994, Pilot Interaction with Cockpit Automation II: An

Experimental Study of Pilots'Model and Awareness of the Flight Management System. [4]

[5]

The Intemational Journal of Aviation Psychology, 4, pp. 1-28. Marrenbach J., et al, 1997, Konzept zur ergonomischen Gestaltung der Benutzungsoberfliiche eines zukiinftigen Flight Management Systems. Jahrbuch der Deutschen Gesellschaft ftir Luft- und Raumfahrt, I, Bonn, pp. 497-505. Marrenbach J. and M. Gerlach, 1999, Validierung des Benutzerfiihrungskonzepts eines zukiinftigen Flight Management Systems. Jahrbuch der Deutschen Gesellschaft f(ir Luft- und Raumfahrt, III, Bonn.

180


References [1] [2]

[3]



[5]


AMEBICA - An Auto Adaptive Multimedia Environment Based on Intelligent Collaborating Agents

Chris J. Khalil IMPACT Research Group,Loughborough University, Loughborough, Leicestershire, LEI 1 3TU. [email protected]

Abstract: This paper reports on some of the progress made by the AMEBICA

project. AMEBICA (Auto-adaptive Multimedia Environment Based on Intelligent Collaborating Agents) is an ESPRIT funded project that aims to examine the use of multi-agent controlled auto-adaptive interfaces in the domain of process control. The AMEBICA architecture is described along with conceptual and functional considerations

1. What is AMEBICA? A M E B I C A represents an approach to improving the representation, form and timeliness of data in a complex process control interface. In traditional interfaces, a mapping is made at design time between the process parameters and an appropriate rendering at the interface. This mapping is usually the best all purpose mapping under a set of general constraints. It is not, however, the only mapping - others may have been discarded which represented a better mapping under a different set of constraints. In the general case of the system functioning under normal condition the general mapping may be appropriate. However, if the process moves into a disturbed state, one of the other discarded mappings [Figure 1] may be more appropriate for the new conditions. The goal of A M E B I C A is to capture these other mappings within the system and implement a flexible mapping system, whereby at run-time A M E B I C A makes a decision on which mapping to use, based on the current state of process, the environment, its operator team model and its knowledge of human factors.

182

AMEBICA An Auto Adaptive Multimedia Environment -

[~]~

I ~ ~ [ I~ s

,4, ,,'" ', ProcessState -

I ~ t[-----~" Set of Different _~ Representation . "~ /" Classes .....k, ~.... Possible "~.s.~\ Mapping ProcessState

,v ChosenMapping . . . . . . . . . . -I~ DiscardedMapping

\

Setof Different ProcessEvents

Figure 1: The Set of Possible Mappings Between Process State and Representation.

2. Why Adapt? In modern control rooms, the traditional hard-desk approach has been replaced by a soft-desk approach [1]. In this new approach the operator monitors plant conditions on a large screen or over multiple monitors, but is unable to view all the information simultaneously and must switch between different views as appropriate. The choice of displayed information is set at design time, and any flexibility in the system must be left to the user. It is the aim of AMEBICA to semi-automate the selection and representation of bandwidth limited information. AMEBICA will thus ensure that the main presentation parameters of form, location, and modality correspond to the contents and nature of the information. It is hoped that this will lead to higher predictability of important process occurrences, less information search, less screen cluttering, quicker response times and a generally improved operator effectiveness. ocal )omain ;raphical ttudlo :ngine

ment ~perator sam spects

Figure 2: A More Detailed Viewpoint.

AMEBICA - An Auto Adaptive Multimedia Environment

183

One of Shackel's [2] requirements for usability is flexibility, and as interfaces evolve it is proving crucial. The objective of the AMEBICA system is to introduce an element of system flexibility to a highly complex process control interface. In normal circumstances, it is envisaged that very little adaptivity will be required at the interface (or is indeed desirable, the operators are highly trained, expert users and any unjustified adaptation may well hinder their effectiveness). However, when the system moves into a disturbed state, its role shifts from monitoring system to pro-active alarm handler. This shift is frequently accompanied by alarm flood [3], and consequently information overload. In such situations AMEBICA will assign incoming alarm signals with levels of importance, and use this information along with its knowledge of the current environment, the state of the operators and its knowledge of human factors presentation heuristics to select an appropriate mapping. By dynamically selecting the most appropriate mapping the operator is presented with the most salient information, in the most effective way at the most appropriate time.

3. Conceptual Background AMEBICA has several fundamental issues on which it operates. These issues are crucial to AMEBICA operating in the way envisaged. These can be summarised as follows:

3.1. Timeliness For AMEBICA to work effectively it is crucial that there is as small a delay as possible between signals/measurements arriving from the process, and the rendering at the interface. To this end the system has been designed so as not to process and act directly on the streams, but instead to act on the rendering mechanism that displays the streams. This concept is illustrated in Figure 2, whereby stream information arrives from the process and passes on, unhindered, to the appropriate rendering objects at the interface. As the stream flows to the interface its state values are monitored by the Process Model Agent, which has explicit knowledge of the system and the critical values that justify system adaptation. If the Process Model Agent encounters a condition in a stream that warrants adaptation, it will pass a reference to that stream to the multi-agent system. The agent system then uses its reasoning mechanisms to decide which representation and media best suit a condition of that type and of that importance. The result of this reasoning is AMEBICA altering the form or parameters of the rendering as necessary, by manipulating the local domain graphical/auditory engines.

184


3.2. Domain independence AMEBICA aims to be a generic adaptation system that maps events of discrete levels of significance - at the input - to appropriate rendering characteristics at the output. To achieve this no direct process knowledge is embedded within AMEBICA. Rather, AMEBICA has two interfaces - the Process Model Agent and the Rendering Interface - that allow it to interact with its environment. These two interfaces contain all the domain dependant information necessary and translate, as appropriate, AMEBICA generic terms to system dependant calls. We see AMEBICA operating in a similar manner as a Java class file in that such a class file is generic and applicable among many systems. However, to make this possible, the class file requires a Virtual Machine (VM) that is highly platform dependent. The VM operates as a translator, changing generic Java calls to systemdependant calls. In the same manner AMEBICA needs a process VM (The Process Model Agent/Rendering Interface) which translates specific process information to a generic format processable by the AMEBICA general system, and vice versa. Thus if an alarm of relatively high importance occurs from a non-critical subsystem, the Process Model might translate this to an low priority AMEBICA alarm. This is then processed and the resultant output of the AMEBICA system is passed to the Rendering which actualises the rendering at the interface. Thus to move AMEBICA between different systems, one simply selects and attaches an appropriate Process Model Agent and Rendering Interface to AMEBICA.

4. AMEBICA Agents To make the correct adaptation it is necessary to take into consideration information from several different sources, including the current environment (lighting levels, operator position etc), the operator team (we use general characteristic of operators rather than individual operators), the process itself and human factors presentation knowledge. Each of these sources is captured as an agent, which takes part in negotiations with other agents to reach an agreement on the best form of adaptation. A Cognitive Resolver agent, whose job it is to query and broker the negotiations, facilitates this process. AMEBICA does not imbue any special AI capabilities within each agent, and adheres to the weak notion of agency [4]. The total system intelligence will therefore be the result of the collective negotiation and communication capacities of the agents.


185

5. An Example Set of Interactions As an example, consider the actions following a message from the Process Model Agent to a Media Agent. Let us assume that a condition has occurred which requires the information rendered by the Media Agent to be given a much higher priority. (This example is merely one scenario and is highly simplified to illustrate a sample set of interactions.)

Rendering Operator Agent

Humm

Factor Datab~

Proces.q Model Agent

Figure 3: The AMEBICA conceptual architecture. The Process Model detects a condition that may require system adaptation and informs the Media Agent responsible for displaying a condition of that type. The representative Media Agent then queries the AMEBICA system and expects to be returned information on an appropriate rendering and its parameters. To do this it informs the Cognitive Resolver that it has a problem, and that the problem is one of increasing priority for its object (it would probably also describe this as an alarm condition).

186

AMEBICA - A n Auto Adaptive Multimedia Environment

The Rendering Resolution Agent will use its knowledge of context to produce a list of candidate representations. The list is passed to the Media Allocator Agent whose job it is to select the best representation class from the list based on current interface resources usage. It does this by utilising the Presentation Agent which returns information on interface usage. This information is used to determine which of the candidates is most suitable and which can be rendered at the interface. This information is then passed to the Media Agent which implements the rendering and it's parameters. If insufficient interface resources are available, the Media Allocator Agent has the power to alter the configuration of other Media Agents so as to allow the selected Representation to be rendered.

6. Conclusion AMEBICA attempts to deal with the problems of bandwidth-limited interfaces and information overload, by adapting the interface to display the most cogent information at the most appropriate times. It is hoped that by doing so AMEBICA can introduce a key element of flexibility at the interface, and that this flexibility is based on the state of the process, its model of the operator team, the environment and its knowledge of human factors. A multi-agent approach was adopted as it allows all the actors responsible for making decisions about suitable adaptations, to be represented by an autonomous agent. During the remainder of the project many important issues must be addressed such as ways to control and limit adaptation, the effect of positive feedback and ways to capture appropriate heuristics for the governing of agent-agent negotiation. These problems will he addressed along with specifying AMEBICA domain independent behaviour, and characterising domain dependant behaviour within the Process Model Agent and the Rendering Interface.

References [1] [2] [3] [4]

C.R.Dicken (1999) "Soft" Control Desks and Alarm Displays, lEE Computing and Control Engineering Journal, Vol 10, Number I , p l 1-16. Shackel, B. (1990) Human Factors and Usability in Preece, J. and Keller, L. (Eds.) Human-Computer Interaction, Hemel Hempstead, UK: Prentice Hall. M. Bransby and J. Jenkinson (Apr 1998),, lEE Computing and Control Engineering Journal, Vol. 9, Number 2, p61-67. M.Wooldridgeand N.R.Jennings(1995) Intelligent agents: Theory and Practice. The Knowledge Engineering Review, 10(2):1! 5-15.

Safety Culture

Rtldiger Trimpop Departement of Psychology, University of Jena

Abstract: While the influence of technical factors on accidents and disruptions of the work-flow is constantly diminishing, organisational factors become the focus of attention. The basic pattern of values, norms and behaviours underlying the structure and processes as well as decisions is called the culture of an organisation, of which safety climate and safety culture is a subset. Thus, safety culture is determined to a large degree by managerial actions and in turn determines, what behaviour employees show. This text defines the constructs, discusses the role culture and management play in a prevention system and gives examples of how safety culture can both be measured and influenced.

1. Accident Causation Accidents are the result of a multifaceted pattern of behaviours, dysfunctional and inadequate for the given situation at a specific point in time. Thus, the same behaviour can lead to accidents or rewards, e.g. speeding when late for an important meeting. Also, the same behaviour patterns in the same situation, such as two cars running a stop sign at an intersection, can cause a severe accident or nothing at all, if there was a sufficient time-lag between these behaviours. Thus, focussing only on technical features or individual behaviour has only a limited probability of success. Furthermore, as organisational structures, goals, visions and leadership styles are at the core of productivity, quality, health and safety, their combined influence on safety has to be considered as a key factor in both accident causation and prevention. As a consequence, safety culture will be closer examined as to its relation to management, decision-making and behaviour.

2. Safety Culture as a Subset of Organisationai Culture Organisational or corporate culture is a pattern of shared basic assumptions that a group of people learned as it solved its problems of external adaptation and internal integration, that has worked well enough to be considered valid and, therefore, to be

190

Safety Culture

taught to new members as the correct way to perceive, think, and feel in relation to those problems [I]. It manifests itself on three levels: 1. Artefacts: Obvious structures and processes, e.g. safety personnel. 2. Values: Strategies, goals, philosophies, e.g. safety=productivity 3. Basic assumptions: Unconscious, taken-for-granted ideas, opinions, feelings and values e.g., safety works as long as there is no time pressure. While safety cultures, of course, have existed since people engaged in any kind of organised behaviour under uncertainty, the term itself has not been coined or been of major interest until the Tchernobyl nuclear accident in 1986. Before, the concept of safety climate [2] was used, referring to the immediate and situational value system in a working unit as to safety policies and behaviour. Some [3] view climate as a summary of perceptions workers share about their work settings. Climate perceptions summarize an individual's description of his or her organisational experiences rather than his or her affective evaluative reaction to what has been experienced. Some studies examined safety climate empirically. Safety climate was for example measured in 20 industrial organisations in Israel [2], in 10 manufacturing and produce companies in the USA [4], in 9 construction sites in the USA[5] and in 16 construction sites in Finland [6 & 7]. At present, however, the term safety culture has replaced safety climate in the scientific community. A number of different definitions exist that partly lead to different consequences in designing systems and interventions.

2.1. Definitions of Safety Culture The safety culture of an organisation is the product of the individual and group values, attitudes, competencies and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation's health and safety programmes. Organisations with a positive safety culture are characterised by communications founded on mutual trust, by shared perceptions of the importance of safety and by confidence in the efficacy of preventative measures. (IEE; Institution of Electrical Engineers of England [8]). Factors are: * leadership and commitment of the CEO 9 executive safety role of line management 9 involvement of all employees 9 effective communications 9 commonly understood and agreed goals 9 good organisational learning 9 responsiveness to change 9 manifest attention to workplace safety and health 9 a questioning attitude 9 a rigorous and prudent approach by all individuals

Safety Culture

191

ASCOT-Guidelines (IAEA [9]): 9 Awareness of safety culture issues 9 Commitment to good safety performance and to continuous safety performance evaluation 9 Commitment to good safety performance as an end in itself and not merely as a mean to satisfy regulatory requirements 9 Investigation of the fundamental causes of events and near misses 9 The examination of activities with potential safety impacts 9 A co-ordinated programme of regular safety audits 9 Efforts to learn from the safety performance of other organisations ILO-Encyclopaedia of Occupational Health and Safety [10] Safety culture is a concept that includes the values, beliefs and principles that serve as a foundation for the safety management system and also includes the set of practices and behaviours that exemplify and reinforce those principles. These beliefs and practices are meanings produced by organisational members in their search for strategies addressing issues such as occupational hazards, accidents and safety at work. These meanings (beliefs and practices) are not only shared to a certain extent by members of the workplace but also act as a primary source of motivated and co-ordinated activity regarding the question of safety at work. INSAG [11]: Safety culture in the nuclear industry is that assembly of characteristics and attitudes in organisations and individuals which establishes that, as an overriding priority, nuclear power plant safety issues receive the attention warranted by their significance. This is determined by organisational policies, managerial action and the response of individuals within the framework. It can be deduced that culture should be differentiated from both concrete occupational safety structures such as the presence of a safety department, of a safety and health committee etc. and existent occupational safety programmes, such as hazards identification activities such as workplace inspections, accident investigation, job safety analysis, etc. and finally integrated safety management systems. While accidents in the nuclear industry so have the potential to cause a major catastrophe, in office jobs the highest accident likelihood is that of stumbling and falling [12]. Thus, safety systems, procedures and management in the nuclear industry have to develop a culture, that makes the system foolproof and focus largely on organisational and technical aspects to reduce the error-influence of operators. Safety is on the constant focus of attention for everybody. In contrast, in office jobs the focus is more on the person him or herself to behave safety-conscious, while safety is not consciously considered often, as there are no obvious dangers, apart from those of day-to-day living. Here it is important to work with the people, making them more aware of dangers. Thirdly, in quick-moving, high-danger professions with many accidents, such as in logging or the construction business, dangers are unavoidable due to the nature of work and those working there have developed a much higher acceptance of

192

Safety Culture

dangerous situations and accidents as well as injuries. Here the underlying safety culture, values and norms have to be generally influenced on all levels of the company and the general type of industry as such.

3. Industrial Examples for Safety Culture Management Trimpop [ 13] differentiates 5 general types of safety management approaches: 9 Participatory Integrated Safety Management, e.g. Dow Chemical 9 Control-oriented safety management, e.g. Dupont 9 Goal-oriented safety management, e.g. Coal-mining industry 9 Individual Safety Leadership, e.g. Small and medium sized companies 9 Technical procedures oriented approach, e.g. Nuclear power plants These types of cultures, interventions and norms have been studied empirically. For example, Eakin [14] showed that in very small businesses, it is common that managers largely delegate responsibility for safety to workers. In a study of about one thousand first-line supervisors, Simard and Marchand [15] showed that a strong majority of supervisors are involved in occupational safety, though the cultural patterns of their involvement may differ. In some workplaces, the dominant pattern is hierarchical involvement and is generally control-oriented; in other organisations the pattern is "participatory involvement", because supervisors both encourage and allow their employees to participate in accident-prevention activities; and in some organisations, supervisors withdraw and leave safety up to the workers. A fairly consistent result of the studies in industrialised and developing countries, emphasises the importance of senior managers' safety commitment and leadership for safety performance [16 & 17]. Moreover, most studies show that in companies with low accident rates (e.g. Dow Chemical or Dupont), the involvement of top managers in occupational safety is as important as their decisions in the structuring of the safety management system, dealing with planning, resources and safety personnel. Indeed, workforce empowerment and active involvement are also documented as factors of successful occupational safety programmes [18]. At the workplace level, some studies offer evidence that effectively functioning joint health and safety committees significantly contribute to the firm's safety performance [19]. Similarly, at the shop-floor level, work groups that are encouraged by management to develop team safety and self-regulation generally have a satisfactory performance [13] Especially in occupational traffic safety, participatory approaches have proven to be very effective means [20 & 21]. Another powerful means of promoting an integrated safety culture among employees is to conduct perception surveys. Workers generally know of many safety problems, but rarely get involved in safety programs. Such a survey can be done using an interview method combined with anonymous questionnaires [22 & 23]. The

Safety Culture

193

survey follow-up is crucial for building an integrated safety culture. Once the data are available, top management should create changes through work groups. This will ensure commitment and adherence to the initiated changes. Such a perception survey should be continuously repeated and adds to a culture of continuous safety and health improvement. One such programme has been studied scientifically and proven successful. It is the TOP 10-Programme of Dow Chemical [18]. Here workers identified the 10 most dangerous behaviours and conditions, developed plans to reduce the dangers, agreed in group discussions among different shifts and with supervisors on the best procedures, trained these procedures and measured their effectiveness. This was considered an important component of the existing safety culture as virtually every employee was involved hunting for safety improvements, training and comparing results. When comparing effectiveness of safety programs at the safety culture level, one has to come up with a set of measures that describe the given cultural features.

4. Measurement of Safety Culture The following list is a sample of measures that have been proven successful in determining organisational culture and safety culture as well. 9 Document analysis: The internal papers, brochures, guidelines of an organisation are examined as to their referral to health and safety, the amount of space, the underlying connotations, the importance and the portrayed value of safety. 9 Guidelines, Code of Conduct: Here it is examined whether procedures and guidelines for machinery, maintenance and repair etc. emphasise health and safety issues, as well as access to and use of these guidelines are measured. 9 Company property, Safety office: The classic Cleanliness and order of a work-place are observed as well as location of safety offices, interconnections and visibility of safety information as well as of safety officers. 9 Questionnaires: Some questionnaires exist to assess safety climate and culture. Zohar [2] developed seven sets of items that were descriptive of organisational events, practices and procedures and differentiated high- from low-accident factories. Brown and Holmes [4] used Zohar's 40-item questionnaire, and found a three-factor model instead of the Zohar eight-factor model. Dedobbeleer and B61and [24] used nine variables to measure the threefactor model of Brown and Holmes. The variables were chosen to represent safety concerns in the construction industry and were not all identical to those included in Zohar's questionnaire. A two-factor model was found.

194

Safety Culture

9 Interviews: Interviews offer the chance to examine causal features for the given circumstances in more depth, but can hardly be performed anonymously. When interpreting interviews one has to adhere to quite rigorous standards for analysing qualitative data to make results comparable. 9 Systematic observations: Here the focus lies on objective observation of certain behaviours, indicative of safety culture. For example, are safety instructions given, how long, by whom and how. Are safety rules adhered to, are supervisors role-model for safety or safety violations. Comparable standards and objective observations allow for the most effective measures but are also of the highest costs, both timewise as well as financially. 9 Hierarchy of procedures: Is there a clear-cut hierarchy of procedures and systems, such as quality before productivity before safety considerations. 9 Involvement in decision processes: How many people from what hierarchical level of the organisation are involved in the decision making process. The higher the degree of participation the higher the likelihood of both detecting errors and safety hazards as well as adherence to self-developed safety standards. 9 Distribution of responsibility: Similar to the decision process responsibility has to be shared by as many hierarchical levels as possible. While the owner is responsible by law and the worker due to suffering from the injuries, all middle level management have to feel responsible and every one in the organisation has to realise that the safety officers are not responsible for producing safety but for giving advice for all other who produce safety or danger with their orders or behaviour. 9 Organisational Charts, Sociogram: An organisational chart shows who, what department and what communication process is officially designed to deal with safety information and co-operation. Very often, however, the theoretically planned relationships do not coincide with reality. That is were a sociogram has its place, plotting what actual communications and cooperations take place and in what kind of quality. 9 Jokes and Stories: Leaving the observable level of indicators for safety culture, jokes and stories can be interpreted as to their meaning. For example, the farewell speech for a retiring manager read as follows: "After he was not capable to work in the production line anymore, he still served the company for many years as the head safety engineer". Stories, heroes (e.g. production hall acrobats without safety net) and jokes reveal the value of safety within an organisational culture.

Safety Culture

195

9 Metaphors, Symbols, Pictures: Similar to jokes and stories, metaphors and symbols reveal underlying thoughts, fears, values and cultural norms. Is a safety system described with a metaphors such as: "All are equal, but some are more equal, or: team work is when everybody does what I want". One can assume a certain quality of culture. The same holds true for pictures and symbols a working unit uses to describe a situation, such as a sinking ship or a shining star. It is important to ask those who give the description as to their interpretation, because investigators themselves might use their own cultural interpretation while the organisational frame at hand has a different interpretative set.

5. Changing Safety Culture Safety culture can be changed on different levels, i.e.: Legislation, regulation Regulatory, supervisory boards Safety officers Employees Management If one looks at the safety regulations within the nuclear industry, or the standards within some Scandinavian countries and compares these to other industrialised and developing countries, the results of legislation and the enforcement of it becomes obvious. Regulations within a company can also make a remarkable difference in culture and safety records, as the chemical company Dupont shows, where a rigorous, control-oriented safety standard leads to excellent safety records and a climate or culture of total safety observation. Regulatory and supervisory boards must set these standards and control them, while the initiative, knowledge and communicative and motivational skills of safety officers lead to major differences both in safety cultures as well as safety records. That employees themselves can make the difference becomes evident in the company Dow Chemical, where virtually every worker is involved in a voluntary health and safety circle and/or task force group, leading to an equally excellent safety record as reached by Dupont with a control-oriented approach. Finally, without management commitment there is little chance of a positive safety culture to develop.

Safety Culture

196

Trimpop [I 3] identifies 5 leverage points to initiate change: 1. 2. 3. 4. 5.

Individual (motivation, identification, information) Group (responsibility, cohesion, team-leadership) Organisation (guidelines, management, structures) Communication (openness, vertical/horizontal) Co-operation (integration, transformation, success)

On each of these leverage points, one can start culture-changing activities but ultimately one has to work on all five levels to create lasting and effective changes. Without organisational structures, new standards and procedures will fall back into the old habits, without individual motivation, new structures have no effect, without communication new values are not transported and without co-operation they are not lived. Finally, each group sets and develops its own sub-culture. If that differs remarkably from the general safety culture, one has to decide which subset is the one aimed for by the organisation as a whole. Petersen [25] provides the following checklist for a positive safety culture with six criteria that can be met regardless of the style of management of the organisation, whether authoritarian or participative, and with completely different approaches to safety: 1. A system must be in place that ensures regular daily pro-active supervisory (or team) activities. 2. The system must actively ensure that middle-management tasks and activities are carried out in these areas: ensuring subordinate (supervisory or team) regular performance ensuring the quality of that performance engaging in certain well-defined activities to show that safety is so important that even upper managers are doing something about it. 3. Top management must visibly demonstrate and support that safety has a high priority in the organisation. 4. Any worker who chooses to should be able to be actively engaged in meaningful safety-related activities. 5. The safety system must be flexible, allowing choices to be made at all levels. 6. The safety effort must be seen as positive by the workforce.

Safety Culture

197

6. Conclusions Rensis Likert [26] showed that the. better an organisation is in core-aspects, the more likely it will be successful in economic success, and thus in safety. These climate variables are as follows: 9 9 9 9 9 9 9 9 9

increasing the amount of worker confidence and managers' general interest in the understanding of safety problems giving training and help where and as needed offering needed teaching as to how to solve problems providing the available required trust, enabling information sharing between management and their subordinates soliciting the ideas and opinions of the worker providing for approachability of top management recognising workers for doing good jobs rather than for merely answering.

Revenue gr0w..th. 9 Employment growth ,Stock Price growth: Net Income growth

Adaptive corporate Non-adaptive corporate cultures cultures 682 % 166% 282 % 36% 901% 74% 756 % 1%

Weak corporate cultures 3% 0% 5% -10%

Table 1: Difference in business success between organisations with a change-oriented, a stability-oriented and a weak corporate culture (N=500; 1977-1988) Indications that this approach holds true come from a large scale longitudinal study [27]. For more than 10 years 500 companies with a strong a weak and an adaptive culture were examined as to the relation of culture and success (see Table 1). The Criteria of Success they found for the adaptive cultures were: 9 A culture of continuous change 9 Openness for changing environments 9 ,,Care" about members, customers and owners 9 Stability in core qualities Transferred to safety culture, none such study exists so far, but if safety culture is a subset of organisational culture the same core qualities should be pursued by safety oriented organisations.

198

Safety Culture

Peterson [25] summarises the importance of safety culture as follows: It is not which elements are used that determines the safety results; rather it is the culture in which these elements are used that determines success. In a positive safety culture, almost any elements will work; in a negative culture, probably none of the elements will get results.


[4]

[5] [6] [7] [8] [9] [10] [11] [12] [I3] [14]

[151

[16]

[17]

[18]

Schein, EH. 1985. Organisational Culture and Leadership. S.F.: Jossey-Bass. Zohar, D. 1980. Safety climate in industrial organizations: Theoretical and applied implications. Journal of Applied Psychology 65, 96-102. Dedobbeleer, N and Briand, F. 1989. The interrelationship of attributes of the work setting and workers'safety climate perceptions in the construction industry. In Proc. 22rid Annual Conference of the Human Factors Association of Canada. Toronto. Brown, R and Holmes, H. 1986. The use of a factor-analytic procedure for assessing the validity of an employee safety climate model. Accident Analysis and Prevention 18 445-70. Dedobbeleer, N and Brland, F. 1991. A safety climate measure for construction sites. Journal of Safety Research, 22, 97-103. Mattila, M, Hyttinen, M and Rantanen, E. 1994. Effective supervisory behavior and safety at building sites. International Journal of Industrial Ergonomics, 13, 85-93. Mattila, M, Rantanen, E. and Hyttinen, M. 1994. The quality of work environment, supervision and safetyin building construction. Safety Science, 17, 257-268. IAEA, 1993, ASCOT Guidelines of the International Atomic Energy Commission lEE, 1996, International Electrotechnical Engineers. Safety Culture Simard, M. 1998, Safety culture and management. In J. M. Stellman (Ed.), ILO Encyclopedia of Health and Safety, 4th Ed. Geneva: ILO Press. 59.2-4. INSAG: Safety culture. Wien. 1991 Trimpop, RM 1994. The Psychology of Risk Taking Behavior. Amst.: Elsevier. Trimpop, R.M., 1999 Organisationaler Wandet im Arbeits-, Verkehrs-, Gesundheitsund Umweltschutz. (Nord-West Verlag). Eakin, JM. 1992. Leaving it up to the workers: Sociological perspective on the management of health and safety in small workplaces. International Journal of Health and Safety, 22, 689-704. Simard, M& Marchand, A. 1994. The behaviour of first-line supervisors in accident prevention and effectiveness in occupational safety. Safety Science 19 169-184. Zimolong, B and R Trimpop. 1994. Managing human reliability in advanced manufacturing systems. In Design of Work and Development of Personnel in advanced Manufacturing Systems, edited by G Salvendy and W Karwowski. New York: Wiley. Shannon, H, Waiters, V, Lewchuk, W., Richardson, J. Verma, T. Haines, T and Moran, L. 1992. Health and safety approaches in the workplace. Unpub.report. Toronto:McMaster Jacobus, L., Rohn, S. und Trimpop, R. (1998). Evaluation des Ansatzes "'Faktor Mensch in der Arbeitssicherheit" der DOW Chemical A.G.S. 335-340. In H. von

Safety Culture

[19]

[20]

[21]

[22] [23] [24]

[25] [26] [27]

199

Benda & D. Bratge (Hrsg.). Psychologie der Arbeitssicherheit: 9. Workshop. Heidelberg: Asanger. Chew, DCE. 1988. Quelles sont les mesures qui assurent le mieux la s~curit~ du travail? Etude men6e dans trois pays en d6veloppement d'Asie. Rev Int Travail, 127, 129-145. Trimpop, R., Adolph, L. & Rabe, S., 1996. Evaluation betrieblicher Verkehrssicherheitsmaflnahmen im Rahmen eines integrativen Gesundheitsmanagementansatzes. In B. Ludborzs, H. Nold, & B. Ri~ttinger (Eds.), Psychologie der Arbeitssicherheit. 8. Workshop 1995 (401- 414). Heidelberg: Asanger. Gregersen, N. P., Brehmer, B. & Moren, B. (1996). Road safety improvement in large companies: An experimental comparison of different measures. In Swedish Road & Transport Research Institute (Eds.), Accident Analysis and Prevention, 28, 297- 306. Bailey, C. 1993. Improve safety program effectiveness with perception surveys. Professional Safety, 10, 28-32. Petersen, D. 1993. Establishing good "safety culture" helps mitigate workplace dangers. Occupational Health and Safety, 62, 20-24. Dedobbeleer, N, B61and & German. 1990. Is there a relationship between attributes of construction sites and workers' safety practices and climate perceptions? In: Advances in Industrial Ergonomics and Safety, D Biman (Ed.)Taylor& Francis. Petersen, D., 1998 Safety policy, leadership and culture. In J. M. Stellman (Ed.), ILO Encyclopedia of Health and Safety, 4th Ed. Geneva: ILO Press. 59.2-4. Likert, R. 1967. The Human Organization. New York: McGraw Hill. Kotter, J.& Heskett, J. (1992). Corporate Culture and Performance. Free Press.

Study of Errors by Means of Simulation and Training

Asgeir Dr~ivoldsmo OECD Halden Reactor Project, P. O. Box 173, N- 1751 Halden, Norway, Email: [email protected]

Abstract: The ongoing Human Error Analysis Project (HEAP) was initiated within

the OECD Halden Reactor Project in 1994. Its objectives are to develop a better understanding and explicit model of how and why cognitive errors occur, and to provide design guidance to avoid, or compensate for, cognitive errors. During the project period, results have lead to practical insights concerning method development, and investigation of, simulation as a way of investigating how operators diagnose cognitively challenging scenarios.

1. Introduction A main problem, in the study of complex operating situations, is that it is not clear what should actually be measured. In the special case of a program studying "human error," one important question is how to find good measurable indicators describing the origin of human erroneous actions. This question is independent of whether one chooses to use classification [1], [2], deeper qualitative analysis, or more general performance indicators like, i.e., operator performance [3]; plant performance [4]; situation awareness [5], [6] as the operational tools for investigation of the topic. Fortunately, without entering the ambiguous variety of definitions and classifications of human error, one can agree that a basis for this research lies in collection of good and reliable data with a potential for aggregation upon a level where meaningful analyses can take place. One way of acquiring such data is to use simulations. Data are then depending on a realistic simulations and tasks, and a representative population of operators (subjects). It is important to include these requirements to comply with the validity criteria that should guide methodologies in this domain [7], [8]. This article will describe the HEAP application of simulation as a tool for gathering good and valid data with the purpose of studying human erroneous actions.

180


References [1] [2]

[3]



[5]



203

rate of information about the operators' attention [14], [15]. Used with care, verbal protocols are powerful tools for insight into problem solving [16]. Four different types of verbal data have been investigated. These four are operator concurrent verbal protocols, operator interrupted verbal protocols, operator retrospective verbal protocols (auto-confrontation) and SME concurrent verbal protocols [17], [18], [19], [20]. The evolution of new and better technology for EMT (Eye Movement Tracking) has given the opportunity to use this method in HaMMLab experiments without interfering with the operators' problem solving. This is a very powerful source of information about the operators' voluntary visual information gathering, and combined with verbal protocols, EMT can contribute to better accuracy [21], [22], [23]. Promising results from EMT analysis have also been achieved in the development of measures of tracing cognitive activity [24], and [25].

5. Scoring and Analysis of Performance Data One important lesson learned in HEAP is the need for a close co-operation with SME throughout the whole scoring and analysis phase of an experiment. There is no way the complex process like the one found in a nuclear power station can be sufficiently learned and understood of research staff without hands on experience. Scoring and analysis where process knowledge comes into consideration should always be guided by subject matter expertise. A common goal for the data collection, scoring and analysis of data in HEAP has been to make the performance scores comparable across scenarios, and sensitive to different levels of problem complexity. To achieve flexibility in the scoring, and to meet different demands for details, data have been recorded on the highest practical resolution and attached to a common timeline for all data sources. The cost of data gathering implies that one will always have to live with a limited number of subjects [26]. Data scoring connected to the time line allows expanding the use of within subject design, through use of time-windows based on important events or stages in the scenarios. Within-subject designs used across scenarios have also proven useful in the analysis in situations where effects are scenario dependent, although interpretation can be difficult when such explorative techniques are employed [27].

6. Conclusions HaMMLab experiments are better described as realistic simulations applying a set of experimental control mechanisms, than traditional psychological laboratory

204


experiments carried out in a nuclear power plant simulator. The experimental methodology developed in HEAP has aimed at data collection and meaningful analysis within the natural context of simulation, preserving the access to the powerful tools of experimental designs. At the same time as the hypothetical - deductive approach has been the central paradigm in most HEAP experiments, HEAP can be said to be inductive in its underlying research strategy. Technical development allows more and faster data registration and analysis. Tracing both the operator and the process together, through data from e.g., eye-movement tracking, operator activity logs, process logs, audio and video, have opened a new path into the study of operator problem solving. The HEAP methodology has been developed to satisfy the need for objective and continuous data collection in the complex operating situation of a nuclear power plant scenario. Using a controlled experimental approach, in combination with maximised realism, has not been an obstacle for in-depth study of selected operator activities in a representative situation. Future work in the HRP (Halden Reactor Project) will focus on how to simplify and make the methods developed more easily accessible for use in general simulatortraining situations and control room validation testing. The first attempts in this direction are already taking place in the validation study of the human-machine interface for Oskarshamn Reactor unit one. Several tools have been tested in eight crews, running five scenarios at the training centre KSU in Sweden. Results from this study will be presented after a final data collection in year 2000.

References [ 1] [2] 13] [4] [5] [6]

[71 [8]

Reason, J. (1990). A frcvneworkfor classifying errors, in J. Rasmussen, K. Duncan, and J. Leplat (Eds.), New Technology and Human Error. London: John Wiley. Hollnagel, E. (1993). The phenotype of erroneous actions. International Journal of Man-Machine Studies, 39, 1-32. Skraaning, G. (1998). The Operator Performance Assessment System (OPAS) (HWR538). OECD Halden Reactor Project, Halden, Norway. Moracho. M. J. (1998). Plant Performance Assessment System (PPAS) for crew performance evaluations. Lessons learned from an alarm study conducted in HaMMLab (HWR-504). OECD Halden Reactor Project, Halden, Norway. Endsley, M. R. (1995). Toward a Theo~ of Situation Awareness in Dynamic Systems. Human Factors, 37, 32-64. Hogg, D., Follesr K., Strand-Volden, F. et al. (1995). Development of a Situation Awareness Measure to Evaluate Advanced Alarm Systems in Nuclear Power Plant Control Rooms. Ergonomics, 38( 11), 2394-2413. Fracker, M. L. (1991) Measures of Situation Awareness: Review and Future Directions. AD-A262 672, National Technical Information Service, Springfield,

Virginia, 22161. American Institute of Aeronautics and Astronautics, (1992). Guide to human performance measurement. (A1AA Publication No G-035-1992). Washington, D.C.


[9] [10] [11] [12]

113] [14] [15] [16] [17] [18] [19]

[20]

[21]

[221 [23]

[24]

[25]

[26]

205

Follesr K., Drr A., Kaarstad, M., ct al. (1995). Human error - the Third Pilot Study (HWR-430). OECD Halden Reactor Project, Halden, Norway. Weber, M. J. (1999). Goal Conflicts in the Process Control of a Nuclear Power Plant. Diploma thesis submitted to the Department of Psychology, University of Bern, Switzerland. Braarud, P. 0. (1998). Complexity factors and prediction of crew performance (HWR521). OECD Halden Reactor Project, Halden, Norway. Braarud, P.O. Complexi O' rating of abnormal events and operator perfotmance. CNRA/CSNI Specialists' Meeting on Human Performance in Operational Transients, Chattanooga, Tennessee, 1997. Collier, S. G. (1998). Development of a Diagnostic Complexity Questionnaire (HWR536). OECD Halden Reactor Project, Norway. Newell, A. and Simon, H. A. (1972). Human Problem Solving. Englewood Cliffs, N.J.: Prentice-Hall, 1972 Ericsson, K.A. and Simon, H.A., (1980) Verbal Reports as Data. Psychological Review; 87: 215-251. Pretorius, N. and Duncan, K. D. (1986). Verbal Reports in Psychological investigations: a Logical and Psychological Analysis. Psyke & Logos, 7, 259-287. Kaarstad, M., Kirwan, B., Follesr K. et al. (1994). Human Error - the First Pilot Study (HWR-417). OECD Halden Reactor Project, Halden, Norway. Kaarstad, M., Follesr K., Collier, S. et al. (1995). Human Error - the Second Pilot Study (HWR-421). OECD Halden Reactor Project, Halden, Norway. Hollnagel, E., Braarud, P. O, Dr~ivoldsmo, A. et al. (1996).The Fourth Pilot Study: Scoring and Analysis of Raw Data Types (HWR-460). OECD Halden Reactor Project, Halden, Norway. Braarud, P. 0., Drr A. and Hollnagel, E. (1997). Human Error Analysis Project (HEAP) the Fourth Pilot Study: Verbal Data for Analysis of Operator Performance (HWR-495). OECD Halden Reactor Project, Halden Norway. Hauland, G. (1996). Building a Methodology for Studying Cognition in Process Control: a Semantic Analysis of Visual Verbal Behaviour. Post-Graduate Thesis in Psychology, Norwegian University of Technology and Science, Trondheim, Norway. Kaarstad, M., Follesr K., Collier, S. et al. (1995). Human Error - the Second Pilot Study (HWR-421). OECD Halden Reactor Project, Halden, Norway. Hauland, G. & Hallbert, B. (1995). Relations between visual activity and verbalised problem solving: a preliminary study. In: Leena Norros (Ed.), VTI" symposium 158, the 5 European conference in cognitive science approaches to process control, Espoo, Finland, pp. 99-110. Drr A., Skraaning, J., Sverrbo, M., et al. (1998). Continuous Measures of Situation Awareness and Workload (HWR-539). OECD Halden Reactor Project, Halden, Norway. Skraaning, J. and Andresen, G. (1999). The Function of the Alarm System in Advanced Control Rooms: An Analysis of Operator Visual Activity during a Simulated Nuclear Power Plant Disturbance. Paper at the Enlarged Halden Programme Group Meeting at Loen 1999, OECD Halden Reactor Project, Halden, Norway. Drr A., Follesr K., HolmstrCm, C. et al. (1995). The Halden Project Workshop Meeting on Evaluation Methods and Measurements, and Data Analysis in System Test and Evaluation (HWR-431). OECD Halden Reactor Project, Halden, Norway.

206


[27] Miberg, A. B., Hollnagel, E., Skraaning, G. et al. (1999). The impact of automation on operator performance. An explorative study. Preliminary report. Institute of Protection and Nuclear Safety (IPSN), Paris.

Operator Training and Implication for the Practice

Horst-Gtinther St0renburg KRAFTWERKSSCHULE E.V. (KWS), Klinkestrasse 27-31, D - 45136 Essen, Germany e-mail: [email protected]

Abstract: Innovative technologies call for qualified personnel. Permanent retraining

of the technical personnel is necessary. In the past years, reduction of power plant personnel was intensifying. Training is only possible in an intensive, guided manner. Shift supervisors (Kraftwerksmeister) have been trained at KWS (Kraftwerksschule e.V.) since 1957. On simulators employees have the possibility to learn to apply new power plant technologies in a risk-free environment. The ongoing dialogue between maintenance and operation personnel can be improved by sharing experience at the simulator. New MMIs and supervision and control systems change the work of operators (Kraftwerker) and shift supervisors in the power plant.

1. Introduction Innovative technologies call for qualified personnel with know-how corresponding to the actual state-of-the-art of power plant technology. Only by permanent retraining of the technical personnel it is possible to meet these high requirements on a long-term basis. Due to the change in power plant technology and control systems in the years since 1950 also the methods and tools for training of power plant personnel needed a change. The increasing standard of automation went hand in hand with a reduction of personnel in power plants. Training and education were structured and formalised in this period so that in 1957 the KWS was founded by the utilities in West Germany.

208

Operator Training and Implicationfor the Practice

Fig. I: 1957 (50 MW plant)

Fig. 2: 1975 (300 MW plant) Fig. 3: 1997 (2*450 MW plant) Figure I-3: Design of control panels in steps of 20 years since 1957. In 1977 the first two simulators for nuclear power plants were established in the KWS building in Essen. In 1986 a simulator for fossil fired power plants followed because of the good experience achieved with the two first simulators. Training and education of power plant personnel are divided in two major parts, the theoretical/practical education and the simulator training. To reach a high and efficient level of training, leading personnel from the utilities is permanently involved in developing learning objectives, examination standards and teaching.

2. Theoretical and Practical Training 2.1. Basic Training for Operation and Maintenance Personnel Correct operation and maintenance of the plant is the direct prerequisite for the compliance with legal regulations. All interventions in the power plant process require a proper grasp of the entire plant and an understanding of the functions of plant systems. Training is only possible in an intensive, guided manner. Professional training has a theoretical and a practical part. It covers a broad spectrum of subjects and is structured by the collection of material, learning objectives, fields of employment and examination standards. Unit and shift supervisors employed in the operation of a power plant have leading positions with a high responsibility for safe, environmentally compatible and economic operation. Thus, not only a deepened know-how of the process and its automatisation, but also an insight into economic and legal connections and a suitability for the guidance and training of the committed employees are required.

Operator Training and Implicationfi)r the Practice

209

Operator Training (Kraftwerkerausbildung) Operator training needs three years. The first two years are filled with practical training on site. During this period the shift supervisor is teacher and advisor in one person. After these two years, a four months theoretical training provides the theoretical background. The last year of training is necessary to implement the theoretical knowledge into practical work on site. At the end, an operator is in charge of operating the plant in a correct manner. Shift Supervisor Training (Kraftwerksmeisterausbildung) Shift supervisor training has a duration of one year at KWS in Essen. This training consists of the following elements: 9a professional training part 9a field-comprising part with the subjects costs, law and guidance of employees 9a field-specific part with fundamental and technical subjects followed by an examination at IHK (Chamber of Industry and Commerce).

Fig. 4: KWS-classroom Fig. 5 : KWS-thermo Fig. 6: KWS-electrical training dynamical laboratory laboratory Figure 4-6: KWS facilities for theoretical/practical training in Essen, Germany.

2.2. Enhanced Training Know-how and skills decrease, if they are not used regularly, refreshed and adopted to the state-of-the-art. A couple of advanced courses and seminars are developed to support the brushing up of know-how. These courses reflect the needs of power plants.

3. Simulator Training for Power Plant Personnel Well-trained employees capable of recognising quickly and analytically and eliminating problematic operational situations are a prerequisite for the safe operation of modern power plants. Base load power plants for example, which are especially common in lignite coal mining regions do not offer many training opportunities for start-up and shutdown

210

Operator Training and Implication for the Practice

procedures. The analysis of problematic scenarios enhances the understanding of power plant operations.

Fig. 7: Simulator 2 with hard Fig. 8: Briefing panel (desk panel)

Fig. 9: Simulator 1 with graphical user interface (GUI) Figure 7-9: KWS facilities for simulator training in Essen, Germany. Employees can learn to apply new power plant technologies in a risk-free environment. In addition, employees can practice new operating techniques during start-up and shutdown procedures and practice the handling of problematic scenarios. The ongoing dialogue between maintenance and operation personnel can be improved by sharing experience of normal operations and problem situations at the simulator. Over the past decade, many customers of power plant manufacturers received preparatory simulator training for the operation of their new plants. Simulators have been especially popular for retrofitting and upgrading measures. New Supervision and control systems change the work of operators. This makes it necessary to supply simulators with this modern systems to conform to the state of the art in power plant technology.


[4]

Broisch, A., St0renburg, H.-G., Ten Years of Simulator Training for Fossil Fired Power Plants, 1996, VGB-Kraftwerkstechnik 76, D-45136 Essen, Germany Stabler, K., Die Ausbildung des Kraftwerkspersonals, 1994, VGB-Kraftwerkstechnik 74, D-45136 Essen, Germany Schlegel, G., Seibert, D., StOrenburg, H.-G., Die Ausbildung des Betriebspersonals unter besonderer BerOcksichtigung de Umweltschutzes Jahrbuch der Dampferzeugungstechnik; Vulkan-Verlag Essen; 7. Ausgabe 1992 Schlegel, G., St0renburg, H.-G., Die Ausbildung des Betriebspersonals unter besonderer BerOcksichtigungder Simulatorausbildung, Jahrbuch der Dampferzeugungstechnik; Vulkan-Verlag Essen; 6. Ausgabe 1988/8

Function Distribution between Man and Machine: Experiments performed in FANSTIC II

Bemd-Burkhard Borys Systems Engineering and Human-Machine Systems, University of Kassel, 34109 Kassel, Germany. e-mail: borys @imat.maschinenbau.uni-kassel.de

Abstract: In experiments with a Multi-Attribute Task Battery we modified the

number and type of the tasks assigned to a human and took two workload measures and one measure of steadiness of task execution. We found that performance on two tasks in parallel differs from expectations derived from performance on single tasks.

1. Introduction The experiments described below were performed in Brite/EuRAM project FANSTIC II in which we evaluated the impact of changes in future air traffic. The high and increasing degree of aircraft automation combined with the need to keep the pilot in the loop gives room for various different function distributions between pilots and automatic systems. Dependency of performance from function distribution during parallel operation of multiple tasks was the focus of our study. A tool for investigation of human performance on several tasks in parallel is the Multi-Attribute Task Battery (MATB) [ 1].

2. The Experiment In our experiments, we used MATB to vary the number and kind of the tasks assigned to the human. We wanted to find out which combinations of tasks should be allocated to the pilots, leaving the others to the aircraft automation. The following functions usually become automated in aircraft cockpits: (a) attitude control by the autopilot and the vertical and lateral guidance by the flight management system; (b) calculations, like take-off or landing distance; (c) knowledge based operator support, for example for failure diagnostics; and (d) supervisory monitoring, for example by the Ground Proximity Warning System [2]. In accordance to original MATB from [1], we tried to cover this task spectrum with

212

Function Distribution between Mat, and Machine

five independent tasks. A tracking task as well as stepwise selection of new waypoints covered (a) and according to (b), simple arithmetic problems were to be solved. A simulated fuel management during a flight covered (c) and to cover point (d), we implemented a monitoring task. Because interested in workload, we used the calculation task as a secondary task. As the manual control task we implemented compensatory tracking of a second order system. A marker was displaced and to be returned to the centre using a joystick. The management task represented the fuel management of an aircraft with five tanks and eight fuel pumps. Subjects had to hold the tank levels in certain limits by switching the pumps. In the navigation task, subjects had to determine the shortest path to destination and enter the next waypoint. In the supervisory task, subjects had to monitor ten systems states. As the secondary tasks, subjects must perform simple calculations while instructed to work on the secondary task only when time permits, giving the other tasks priority.

3. Implementation and Design Tasks were implemented on three PCs. Their screens were arranged similar to a glass cockpit, a joystick was on the left, special keyboards to the right of the subject, the keys arranged according to the tasks' requirements. For experimental control, data collection, and supervision, we used a fourth PC in an observation room. A stand-alone network connected all four PCs. A first series of experiments involved six pilots. A second series of experiments involved five pilots and five students of mechanical engineering. Most of the 11 pilots held licenses for single-engine planes, three with additional IFR, and two with instructor rating. Three subjects held licenses for commercial and airline transport. We carefully evaluated the question whether it is correct to include students in the experiments. Indeed, students showed different performance on the tasks. Mainly in the manual control task, tracking error (RMS of deviation) was much lower for the students on the cost of higher joystick activity (RMS of deflection - influence of video games?). Also, the measure for scanning irregularity (described below) was significantly lower for pilots. However, we were interested not in absolute values of performance, but in changes of performance and behaviour on variation of function allocation. We used three measures to assess performance: A subjective workload score based on a questionnaire, objective workload ("calculation score") and a score for scanning irregularity, the latter two derived from performance on the secondary task (number of calculations performed and standard deviation of calculation times used). Correctness of the calculation result was of no concern, as only few calculations were incorrect. Indeed, in more than half of the experiments no error was made.

Function Distribution between Man and Machine

213

4. Results All measures showed large inter-individual differences and were Z-transformed resulting in "scores" of 0.0 mean and standard deviation 1.0 for each subject. The secondary task also showed a training effect, which was compensated when calculating the scores. Changes of these scores under different task conditions were evaluated further. The subjective workload score did not show significant differences with the task, when performing one task manually. It did when performing two or three tasks in parallel: Workload was lower when involving the monitoring or navigation task. Nay x

LowWorkload

Mon

Ctr~woTasks SingleTask~ Nl~!-.c~n Mgt ~ _ _ _ _ _ ~ . . ~ - ~

,gh Workload~

Ctr-Mgt

~~ t'

Figure 1: Workload trends in single task and two-task situation Regarding the objective workload measure "calculation score", workload was highest (score was lowest) when performing the management task manually while the other three tasks showed lower workload in the one-task-manual design. However, these experiments showed that the workload of two tasks performed in parallel could not simply be derived from single-task results. The following figure shows calculation scores for the single-task situation (left) and the two-tasks situation (right, on a separate scale) and indicates trends in workload when adding a second manual task by the connecting lines. Combining two of the low-workload tasks (navigation Nay, monitoring Mon, and control Ctr) also results in a low-workload combination. But combining the management task (Mgt) with other tasks showed existing interdependencies between tasks: Although navigation as a single task produced the lowest workload it is not the optimum choice when adding a second task to the management task.

214


An evaluation of the scanning irregularity reveals similar results; those combinations showing good results with respect to workload also show regular scanning patterns. For a transition from the two- to the three-task-situation comparable results could be expected. However, the number of combinations was too high compared to the number of experiments that could be performed to show significant differences.

5. Consequences for Future Systems The result shows that finding a suitable combination of manually performed tasks requires both regarding the workload caused by each task alone and regarding interference between the work on the appropriate tasks. Details can be found in [3, 4]. Although exact numbers and even trends cannot be generalised and similar experiments may be necessary for different task combinations these experiments show the importance of mutual influence between different tasks in a multi-task environment. For future evaluations of workload in experimental multi-taskenvironments, we would propose to classify tasks according to physical and cognitive resources needed and estimate changes in workload based on overlap of resource use. For a specific application like the future cockpits or the air traffic management system [5], we proposed to avoid task combinations that use the same resources and evaluate workload for candidate task combinations. With the new possibilities of function distribution between pilots, controllers, and aircraft and ground automation, careful planning of suitable task combinations is necessary.

References [I] [2] [3]

[4] [5]

Hilbum B.G., 1993, The Effect of Long- versus Short-Cycle Schedule on Performance with an Adaptively Automated System. Catholic University of America. Wickens C.D., 1993, Designing for Situation Awareness and Trust in Automation. IFAC Conference on Integrated Systems Engineering, pp 77-82, Baden-Baden. Tiemann M. and Borys B.-B., 1995, Verringemng der Belastung von Piloten durch ver~inderte Aufgabenteilung zwischen Mensch und Maschine. In H.-P. WiUumeit and H. Kolrep (Eds.), Verli~fllichkeit yon Mensch-Maschine-Systemen, pp 139-153. Berlin: Technische Universit~it. ISBN 3-7983-1650-3. Borys B.-B. and Tiemann M., 1995, Experiments for a new Distribution of Function. Chapter 3.5 in [5]. Borys B.-B (Ed.), 1995, Recommendations for a Human-Centred Air Traffic Management Systems. FANSTIC II Sub-Task 2.3 "ATM Implications" Synthesis Report, UKS-2.3-SR001 (Bo007/95). University of Kassel.

The work described herein has been undertaken by the author as part of the European Union FANSTIC 11 project, within the framework of the IMT Community Research Programme, with a financial contribution by the European Commission. The following companies,


2 I5

research centres, and universities were involved in the project: AEROSPATIALE, DAIMLER-CHRYSLER AEROSPACE AIRBUS, BRITISH AEROSPACE CA, FOKKER, SEXTANT AVIONIQUE, NLR, CENA, SMITH INDUSTRIES, VDO, ALENIA, CAP'I~C, NATIONAL TECHNICAL UNIVERSITY OF ATHENS, SPACE APPLICATIONS SERVICES, UNIVERSITY OF KASSEL, AIRBUS INDUSTRIES TRAINING, DEFENCE RESEARCH AGENCY, BRITISH AEROSPACE SOWERBY RESEARCH CENTRE, THOMSON-CSF, DORNIER, and FIAR.

Discussion Session V

In the final discussion of the workshop it was tried to deal with such aspects that had been touched upon in more than one paper but not dealt with in detail or that had implicitly surfaced as consequences of what had been said. One of them was the role of 'new technologies' like Virtual Reality, Multimedia etc. Sometimes the impression arises as if they are just applied for their own sake or because their usage by 'computer kids' would guarantee their role as carriers of progress. In the discussion it was widely agreed that this should be avoided. However, the potential of these media should not be underestimated. They may be able to give back to the users to a certain extent what has been taken away from them by the current VTU-based interfaces: non-visual cues like vibration, smell, unidentifiable noises. It was even remarked that in some cases users had not been aware of the existence and usefulness of such cues before they were taken away from them. This observation led to a rather fundamental discussion about the necessity and the proper use of such cues as a means of direct feedback. Examples of devices that triggered this discussion were e.g. the 'playstation joystick' for controlling a power station, the data glove, instruments for minimal invasive surgery, or the 'single joystick control' for cars. It turned out that some deeper questions have to be answered before the usefulness of any particular input device can be discussed. So e.g. it is of interest why a person has to control a technical system - as a professional, for fun, or casually. It is also makes a difference whether the person wants to operate the machine or control the effect of that process. In the first case direct feedback may be regarded as necessary, in the latter case rather indirect supervisory control may be the proper choice. In any case, however, operators want to be able to 'see through' the computer. Finally it was agreed that the discussion about the appropriateness of certain interface designs should neither be based upon a technological view ('the newest is the best') nor on the (rather human) tendency to believe in 'shortcuts' (like e.g. 'Multimedia is the solution', 'situation awareness is the true key to operator performance' or 'ecological interfaces are the only way to go'). Proper interface design should rather be based on a thorough analysis of the information requirements, tedious as this may be, should take into account the results of accident analysis in order to avoid the pitfalls of the past and take an asymptotic approach towards the use of emerging technologies.

Summarized by Peter F. Elzer

Human error and system design and management

Human Error: Causes and Control

Human Error

Measurement Error and Research Design

Human Error

Fatal system error

Fatal System Error

Fatal system error

Human Error: Species-Being and Media Machines

Beyond Human Error: Taxonomies and Safety Science

Networks Design and Management

Networks: Design and Management

Human Reliability and Error in Transportation Systems

Paul Preuss - Human Error

Requirements Analysis and System Design

Human Factors in System Design, Development, and Testing

Computer Architecture and System Design

Ten Questions About Human Error: A New View of Human Factors and System Safety

Human Reliability and Error in Medical System (Industrial and Systems Engineering, 2)

ISO 9001: 2000 Quality Management System Design

ISO 9001: 2000 Quality Management System Design

Ten Questions About Human Error: A New View of Human Factors and System Safety (Human Factors in Transportation)

Ten Questions About Human Error: A New View of Human Factors and System Safety (Human Factors in Transportation)

The Human Nervous System. Structure and Function

Trial and Error

Trial And Error

Error Analysis and Interlanguage

Trial and Error

Embedded Linux System Design and Development

System Synthesis: Product and Process Design

Professional Stock Trading: System Design and Automation

Human error and system design and management

Human Error: Causes and Control

Human Error

Measurement Error and Research Design

Human Error

Fatal system error

Fatal System Error

Fatal system error

Human Error: Species-Being and Media Machines

Beyond Human Error: Taxonomies and Safety Science

Networks Design and Management

Networks: Design and Management

Human Reliability and Error in Transportation Systems

Paul Preuss - Human Error

Requirements Analysis and System Design

Human Factors in System Design, Development, and Testing

Computer Architecture and System Design

Ten Questions About Human Error: A New View of Human Factors and System Safety

Human Reliability and Error in Medical System (Industrial and Systems Engineering, 2)

ISO 9001: 2000 Quality Management System Design

ISO 9001: 2000 Quality Management System Design

Ten Questions About Human Error: A New View of Human Factors and System Safety (Human Factors in Transportation)

Ten Questions About Human Error: A New View of Human Factors and System Safety (Human Factors in Transportation)

The Human Nervous System. Structure and Function

Trial and Error

Trial And Error

Error Analysis and Interlanguage

Trial and Error

Embedded Linux System Design and Development

System Synthesis: Product and Process Design

Professional Stock Trading: System Design and Automation

Recommend Documents